[05:27:00] 10Scoring-platform-team, 10drafttopic-modeling: Follow-up cleanup to topic models - https://phabricator.wikimedia.org/T246909 (10MMiller_WMF) [05:28:53] 10Scoring-platform-team, 10drafttopic-modeling: Follow-up cleanup to topic models - https://phabricator.wikimedia.org/T246909 (10MMiller_WMF) @Halfak -- I wanted to ping you about the questions in my previous comment. Also, I added a subtask in which @Etonkovidova details the performance of the "Central Afric... [05:30:07] 10Scoring-platform-team, 10drafttopic-modeling: Follow-up cleanup to topic models - https://phabricator.wikimedia.org/T246909 (10MMiller_WMF) [05:55:33] 10Scoring-platform-team, 10drafttopic-modeling: Clean up History and Society.Society in the topic taxonomy. - https://phabricator.wikimedia.org/T246912 (10MMiller_WMF) @Isaac -- thanks for working on this. Do you need input from me (or anyone else)? [10:56:44] 10Scoring-platform-team, 10Research-Backlog, 10Research ideas, 10Wiki-Loves-Monuments (2017), 10artificial-intelligence: General image classifier for commons - https://phabricator.wikimedia.org/T155538 (10Miriam) a:03Miriam [10:56:55] 10Scoring-platform-team, 10Research-Backlog, 10Research ideas, 10Wiki-Loves-Monuments (2017), 10artificial-intelligence: General image classifier for commons - https://phabricator.wikimedia.org/T155538 (10Miriam) @leila done! [13:18:03] o/ [13:18:18] o/ chtnnh [13:18:22] How's hacking? [13:18:34] heyy halfak [13:18:41] off to a good start [13:18:49] got the green light on the tests last night [13:18:56] i realised the error [13:19:12] i will commit it now for you to have a look [13:19:44] Sounds good. Link me to the PR when you're ready. [13:20:28] https://github.com/wikimedia/articlequality/pull/107/ [13:22:59] halfak: annual planning time! I was asked to gather budget requests for the next fiscal year! If you have any, send them my way please. [13:23:19] and now that I spread the joy I 'll go back to filling budget spreadsheets [13:23:33] XD [13:24:09] akosiaris, we don't have a specific request but we do have some issues that we're blocked on that might involve new hardware. E.g. the uwsgi issue and our memory limitations. The redis SPOF and possibility of setting sentinel or some other strategy. [13:24:39] halfak we need to work on testing with multiple labels though and I might need your help with that [13:25:31] halfak: redis wise we already got it covered in https://phabricator.wikimedia.org/T245591. We will fold that infrastructure into the main redis cluster. [13:25:52] but as far as memory goes, maybe we need more capacity there? [13:25:58] Aha! Gotcha. [13:26:38] I am assuming you 'll want to add more wikis/models/you name it in the next FY ? [13:26:42] akosiaris, yes. Simplest solution is to buy machines with more RAM. Even 10GB more headspace could gain us a year's worth of new development. [13:27:03] Originally we bought these machines with the intention that they would become part of the k8s cluster. [13:27:18] And ORES would migrate to k8s. That's been put on hold. [13:28:15] We're exploring alternative infrastructure for model hosting, but our memory usage is likely to go *up* no matter what option we choose. Though it might be easier to spread horizontally. [13:28:20] that's true. The ores hosts are currently in warranty btw so if we are to add more capacity it's going to be in the form of new machines [13:28:36] Right. Gotcha. [13:29:00] not refreshing the current ones, so that alternative infra for model hosting (as the new machines will not be uniform to the old ones) might be needed anyway [13:29:23] it would also be needed anyway for the move to k8s (which has been put on hold indeed, we need to find time to work on that) [13:30:27] Interesting. Tell me more about what we'd need for a move to k8s. [13:30:36] but anyway, this budgeting exercise is pretty high level. If you can come up with some CPU/memory requirements, that's all I need for now. [13:31:15] Hard to estimate CPU needs as we rarely are getting high utilization. [13:31:31] But I could ballpark some memory requirements. [13:32:06] fine by me [13:32:14] Our hosts currently have 64GB and we use about 44GB with 20GB available. [13:32:37] Assuming we didn't have the uwsgi restart issue, would you like us to keep 20GB available or could we cut that down to 10GB? [13:34:44] 10GB is fine [13:34:48] chtnnh, this is good work. I don't have any concerns with merging this as-is. [13:34:55] akosiaris, OK. [13:35:00] oh wow [13:35:10] thank you so much [13:35:17] chtnnh, how do you feel about working on the nsfw model today. That will help us in this memory-usage exercise. [13:35:26] that would be amazing [13:35:33] and it would help me draft my proposal [13:35:35] All I would need you to do is to try to get the NSFW image classifier to do *something* and we'll check it's memory profile. [13:35:56] akosiaris, when is your deadline? [13:36:34] okay let me try that [13:36:59] chtnnh, before we merge this PR, I'd like to have you squash all of your commits into one. You can do that with rebase. Then I won't take credit for your work by squashing them myself :) [13:37:01] What do you think? [13:37:06] xD [13:37:09] ofcourse [13:37:20] halfak: annual planning is getting delayed, you probably have plenty of time. Probably mid april? [13:37:23] also please check your email, i have sent my third draft proposal for ptwiki [13:37:47] akosiaris, got it. We'll get on it soon though so I'll have a draft to talk through with you well ahead of time. [13:37:53] cool. thanks! [13:38:02] chtnnh, catching up on a lot of stuff. I'll get there :) [13:38:10] sure senpai [13:38:20] Thank you, akosiaris :) And godspeed on your spreadsheets. [13:38:23] thanks! [13:39:36] ill let you know when i have squashed the commit [13:48:05] rebased [13:48:10] and commits squashed [16:07:51] Async! [16:07:57] yay [16:19:23] Y: Worked with chtnnh on some GSOC proposals.  We're finishing up the ptwiki label extraction code and starting in on exploring the NSFW classifier.  I met with Angie from Readers to talk about topic modeling. [16:19:24] T: Working with chtnnh on NSFW classifier.  It should give us some insights into memory/CPU needs for image classification.  I'm hoping to finish up a few design tasks for Jade based on user testing.  Otherwise, I'll be doing some work for MTP OKRs.  [17:05:38] Y: Concluded digging into representing localized Jade dates on the frontend with Aaron advising that we move forward with the UTC date rather than the localized moment.js approach since the latter doesn't rely on MW user preferences. [17:05:38] T: Added click event to proposalWidget endorsementsButton - Expand/Collapse endorsements list whenever clicked to match click events of expand and collapse icons. [17:05:38] QN: [17:05:38] Should the author's name be colored red on the delete endorsement dialog box as seen in the wireframe? [17:05:41] https://phab.wmfusercontent.org/file/data/fkgy4imyhd3bytkk3cz4/PHID-FILE-y74crbvib2p6ti7opcl3/Wireframe_-_Entity_view_-_Plus_buttons_-_Expanded_-_Delete_endorsement.svg.png [17:05:44] Please confirm. Thanks! [17:33:40] halfak [17:33:48] so about the work on the open_nsfw model [17:34:03] kevinbazira, username links should be red when their user-page doesn't exist. [17:34:11] I see that the pre trained model has a low memory footprint [17:34:19] This is a common pattern across the wiki [17:34:25] Same for user_talk pages. [17:34:40] and thanks to the kind @rahiel on github we can run a docker instance of the model on a server [17:35:12] which can then respond to api requests that return float between 0 and 1 indicating the probability of the image being nsfw/sfw [17:35:19] the observations ive made [17:35:36] we will need dedicated servers to run these docker instances to handle the load [17:36:09] on top of which we will need to engineer scripts to achieve concurrent processing of uploaded images across multiple instances [17:36:38] and ensure constant processing of uploaded images [17:36:42] what do you think [17:36:58] I am sorry i have not yet been exposed to the wikimedia backend infra [19:38:51] 10Jade, 10Scoring-platform-team: Clicking "Endorsements (#)" should expand/contract the set of endorsements. - https://phabricator.wikimedia.org/T247452 (10ACraze) Great work @kevinbazira -- I +2'd your patchset and now it's merged. [20:00:23] chtnnh, that was really difficult. Something is messed up with your fork of the articlequality repo. [20:00:39] I want to help clean it up. [20:00:44] hmm [20:00:45] sure [20:00:52] Could you go to the local repo on your machine and checkout the master branch? [20:00:53] what do you need me to do [20:01:25] done [20:01:47] then do `git pull https://github.com/wikimedia/articlequality master` [20:01:57] And then `git push` [20:02:05] ohh [20:03:31] it says everything up-to date [20:03:48] should I commit before pushing? [20:04:08] because there are uncommitted changes in my master now somehow [20:04:38] You shouldn't need to commit. [20:05:17] All looks good as far as I can see. [20:05:22] you sure? [20:05:36] nothing changes on the PR page [20:08:09] I think I need to work on fixing the git on my local [20:10:30] on it [20:10:50] lost some serious github contributions on this :( [20:59:52] 10Scoring-platform-team (Research), 10Structured-Data-Backlog, 10artificial-intelligence: Implement NSFW image classifier using Open NSFW - https://phabricator.wikimedia.org/T214201 (10Chtnnh) a:03Chtnnh After going through the links mentioned in the task description, I have been able to run a Docker insta... [21:47:23] 10Jade, 10Scoring-platform-team: Render edit comments in Jade - https://phabricator.wikimedia.org/T247457 (10Halfak) Here's some design work on what I'd like to see in the comments. {F31692964} [21:51:00] 10Jade, 10Scoring-platform-team: Include IP address in edit comments when rendering a "user" - https://phabricator.wikimedia.org/T248134 (10Halfak) [21:52:36] 10Jade, 10Scoring-platform-team: Render usernames in Jade edit comments. - https://phabricator.wikimedia.org/T248135 (10Halfak) [22:23:29] wikimedia/revscoring#1878 (no_textstat - dbbf556 : halfak): The build was fixed. https://travis-ci.org/wikimedia/revscoring/builds/664595278