[02:02:32] 10Scoring-platform-team (Current), 10Security-Team, 10Wikilabels, 10Security: Wikilabels does not show the exact wikitext on diff: it parses s as HTML - https://phabricator.wikimedia.org/T250313 (10sbassett) p:05Triage→03Medium [09:38:53] 10Jade, 10Scoring-platform-team (Current), 10MW-1.35-notes (1.35.0-wmf.30; 2020-04-28): Can endorse two different labels for the same facet via jadeproposeandendorse. That should be impossible. - https://phabricator.wikimedia.org/T247974 (10kevinbazira) [13:48:23] o/ [15:06:19] 10Scoring-platform-team, 10Wikilabels: Internal links on comment/summary point to Wikilabels instead of the target wiki - https://phabricator.wikimedia.org/T250704 (10He7d3r) [15:20:25] 10Scoring-platform-team, 10Wikilabels: Internal links on comment/summary point to Wikilabels instead of the target wiki - https://phabricator.wikimedia.org/T250704 (10He7d3r) [15:20:38] chtnnh, looks like we need to update https://quarry.wmflabs.org/query/43261 based on feedback. [15:20:48] See Helder's last comment in https://phabricator.wikimedia.org/T246667 [15:21:52] let me have a look halfak [15:34:28] Looks like we're blocked on review for the ORES deployment. [15:34:32] https://phabricator.wikimedia.org/T250536 [15:36:58] why are we blocked halfak [15:37:11] Waiting on review from accraze or kevinbazira [15:37:32] hasnt accraze already reviewed [15:37:37] i saw it on gerrit [15:38:30] https://gerrit.wikimedia.org/r/c/mediawiki/services/ores/deploy/+/589713 [15:39:28] Aha! He just forgot to submit. [15:39:36] OK moving this to beta. [15:39:44] doesnt seem like hes online tho [15:39:55] yay :D [15:40:01] It's OK. So long as he +2s I can submit. [15:40:09] great! [15:52:01] ORES seems to be struggling in beta. Looks like we only have 8GB of RAM there and we're trying to make it work with a reduced worker pool [15:52:22] ouch [15:52:44] but we just added 2 models right? [15:52:59] Right. [15:53:22] how many models in total [15:53:36] Hmm. I just tried restarting our celery workers and it failed. But not because of memory as far as I can tell. [15:53:57] is this somehow related to the memory issue you were working on? [15:54:19] I don't know. [15:57:08] Hmm. Celery seems to start up but it doesn't work any workers. [16:00:31] Yeah. This is super weird. [16:00:36] uwsgi is also not starting workers. [16:00:48] im sorry i am not able to understand all this :( havent dealt with this part of the codebase before [16:02:33] but is it possible that a mistake in our code may have caused this? [16:03:27] 10ORES, 10Scoring-platform-team (Current): ORES beta won't start workers - https://phabricator.wikimedia.org/T250712 (10Halfak) [16:04:14] 10ORES, 10Scoring-platform-team (Current): ORES beta won't start workers - https://phabricator.wikimedia.org/T250712 (10Halfak) I tried running `sudo service ores-celery-workers restart` and `sudo service uwsgi-ores restart`. Everything proceeded as expected except that workers didn't start up. [16:09:52] 10ORES, 10Scoring-platform-team (Current): ORES beta won't start workers - https://phabricator.wikimedia.org/T250712 (10Halfak) Aha! I found this in the syslog: ` Apr 20 16:08:56 deployment-ores01 celery-ores-worker[12127]: Traceback (most recent call last): Apr 20 16:08:56 deployment-ores01 celery-ores-work... [16:10:15] 10ORES, 10Scoring-platform-team (Current): ORES Beta startup errors not being routed to our app logging. - https://phabricator.wikimedia.org/T250712 (10Halfak) [16:10:47] Looks like I need to update the revscoring wheel. [16:12:24] sounds like an important step [16:35:46] did it work after you rebuilt the wheel? [16:41:28] halfak ^ [16:44:22] 10ORES, 10Scoring-platform-team (Current): ORES Beta startup errors not being routed to our app logging. - https://phabricator.wikimedia.org/T250712 (10Halfak) a:03Halfak [16:50:20] 10Scoring-platform-team, 10Wikilabels: Internal links on comment/summary point to Wikilabels instead of the target wiki - https://phabricator.wikimedia.org/T250704 (10Halfak) p:05Triage→03Low [16:52:16] 10Scoring-platform-team, 10revscoring, 10Documentation, 10artificial-intelligence: Scoring example no longer works - https://phabricator.wikimedia.org/T250635 (10Halfak) p:05Triage→03Medium [16:54:44] 10Scoring-platform-team, 10revscoring, 10artificial-intelligence: Template matching regex features only matches prefix - https://phabricator.wikimedia.org/T250522 (10Halfak) p:05Triage→03Lowest [16:57:03] 10Scoring-platform-team, 10Wikilabels: Labeling gadget has non-functional expansion buttons - https://phabricator.wikimedia.org/T250176 (10Halfak) I think getting rid of the button would probably be the best solution here. [16:59:45] 10Scoring-platform-team, 10Wikilabels: Labeling gadget has non-functional expansion buttons - https://phabricator.wikimedia.org/T250176 (10Halfak) p:05Triage→03Low [17:00:12] 10Scoring-platform-team (Current), 10Discovery-Search, 10Elasticsearch, 10revscoring, 10artificial-intelligence: Improve the performance and quality of tokenization in revscoring - https://phabricator.wikimedia.org/T248480 (10Halfak) [17:04:11] 10Jade, 10Scoring-platform-team (Current): Address UI issues identified in user-testing - https://phabricator.wikimedia.org/T247897 (10Halfak) [17:08:01] 10Jade, 10Scoring-platform-team: Apply i18n to rendered edit comments in Jade - https://phabricator.wikimedia.org/T250721 (10Halfak) [17:08:47] 10Jade, 10Scoring-platform-team: Apply i18n to rendered edit comments in Jade - https://phabricator.wikimedia.org/T250721 (10Halfak) p:05Triage→03High [17:11:09] 10Jade, 10Scoring-platform-team: [Spike] What facilities are available to us when rendering edit comments? - https://phabricator.wikimedia.org/T250723 (10Halfak) [17:11:31] 10Jade, 10Scoring-platform-team: [Spike] What facilities are available to us when rendering edit comments? - https://phabricator.wikimedia.org/T250723 (10Halfak) p:05Triage→03High [17:14:24] 10Jade, 10Scoring-platform-team, 10MediaWiki-Page-Diffs: [Spike] How can we render custom diffs for Jade pages? - https://phabricator.wikimedia.org/T250725 (10Halfak) [17:14:44] 10Jade, 10Scoring-platform-team, 10MediaWiki-Page-Diffs: [Spike] How can we render custom diffs for Jade pages? - https://phabricator.wikimedia.org/T250725 (10Halfak) p:05Triage→03Medium [17:30:42] (03PS1) 10Halfak: Updates for revscoring==2.6.9 [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/591114 (https://phabricator.wikimedia.org/T250536) [17:32:48] Hey folks. I'd love a quick review of https://gerrit.wikimedia.org/r/#/c/research/ores/wheels/+/591114 if you have a chance. [17:32:57] It's the wheels we need to move forward with a deployment. [17:35:13] OK I'm running to lunch. [17:57:54] (03CR) 10Accraze: [C: 03+2] Updates for revscoring==2.6.9 [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/591114 (https://phabricator.wikimedia.org/T250536) (owner: 10Halfak) [18:07:09] posting our async update notes -- [18:07:31] kevinbazira - [18:07:33] Last Week: [18:07:35] Focused on addressing UI issues identified in user-testing from https://phabricator.wikimedia.org/T247897 [18:07:37] - Pushed a couple of patchsets [18:07:39] T: [18:07:41] Jade [18:07:43] Looking into rendering usernames in Jade edit comments: https://phabricator.wikimedia.org/T248135 [18:07:45] MW Core [18:07:47] As advised by VolkerE on mw.notify, I have emphasized Base color (Base0) as foreground when setting the new background color and put the comments above each of the respective lines. [18:07:50] halfak - [18:07:52] Last week: Did a lot of work for the tuning session. Otherwise debugged memory usage issues in ORES and put together a capex proposal. Otherwise, I onboarded Habeeb and tried to carry chtnnh's work to production. [18:07:54] Today: Well, I tried a beta deployment of ORES and everything is very weird. See https://phabricator.wikimedia.org/T250712 I'll be digging into this today. If I get past that, I'll get to work on the ORES paper rewrite that I'd been putting off for weeks now. [18:07:56] haksoat - [18:08:06] Last week: [18:08:07] I spent a lot of time looking through the tokenizer modules on revscoring and the deltas package. I also did research on the possibility of using our generated regex match on elasticsearch. [18:08:09] Today: [18:08:11] I was able to come up with a short script that converts the 32-bit hex unicode values from Python into Java's supported 16-bit hex values. Currently, the regex pipe | doesn't seem to work yet, so I'm trying to understand why and fix it on the elasticsearch tokenizer. [18:08:13] and me - [18:08:22] Last week: Mostly continued work on secondary integrations for Jade and lots of code review. [18:08:23] Today: Continuing the ad hoc approach for jade 2ndary schemas and re-enabling hooks, also some more code review as well [18:43:26] hey halfak re: helders comments on the draftquality query [18:43:50] yeah. [18:44:05] i checked out helders query, i think the 718 results include ER6 and ER20 in them [18:44:13] i ran another query to verify [18:44:36] Oh! So we're already good with this query? [18:44:57] https://quarry.wmflabs.org/query/44152 [18:45:08] the thing is, we might be missing some of the results [18:47:34] Right. I think that we need an "OR" condition to be able to catch this stuff too. [18:47:52] yes exactly [18:48:12] also, the query you linked [18:48:24] https://quarry.wmflabs.org/query/43261 [18:48:34] is not the one we linked in the Makefile [18:48:40] Aha. [18:48:59] let me show you the one we linked [18:49:14] (03PS1) 10Halfak: Bumps wheels for revscoring-2.6.9 [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/591129 [18:50:04] (03CR) 10Halfak: [V: 03+2 C: 03+2] "(Self-merging) a simple bump to wheels." [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/591129 (owner: 10Halfak) [18:52:13] https://quarry.wmflabs.org/run/444292/output/0/json-lines? [18:52:18] halfak ^ [18:52:44] That's the query output. [18:52:49] It's not the query itself. [18:53:50] Hmm. I can't find the query we actually used. [18:55:06] i have the query [18:55:24] let me link it up https://quarry.wmflabs.org/query/43197 [18:55:29] its a fork of your query [18:58:33] I updated https://quarry.wmflabs.org/query/43197 [18:59:23] What do you think? [18:59:41] Looks like we'll double our "spam" article count. [18:59:55] we should ideally [19:00:03] the IF structure is killing me tho [19:00:17] no way we can OR this is in SQL? [19:00:18] xD [19:01:38] Our click dependency changed it's dang capitalization. [19:01:54] chtnnh, not a nice way to do that :\ [19:02:51] yeah i guess so [19:02:54] (03PS1) 10Halfak: Removes old Click dependency [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/591136 [19:03:16] accraze, around? Looks like I missed something in that wheels change. https://gerrit.wikimedia.org/r/#/c/research/ores/wheels/+/591136 [19:03:30] The capitalization changed so I didn't notice the old version there. [19:03:41] ah ok, taking a look now halfak [19:03:53] Thanks! [19:06:54] (03CR) 10Accraze: [C: 03+2] Removes old Click dependency [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/591136 (owner: 10Halfak) [19:09:58] Some good news halfak [19:10:28] I've found a way to tokenize on regex [19:10:35] Tested with our regex [19:10:52] Works, though, there are some things that still needs to be worked on [19:11:14] \o/ [19:11:30] That's great news. I wonder what kind of performance you are seeing. [19:14:47] (03PS1) 10Halfak: Bumps wheels (remove double Click dep) [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/591142 [19:15:33] (03CR) 10Halfak: [V: 03+2 C: 03+2] "(self-merging) Bumping wheels to HEAD" [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/591142 (owner: 10Halfak) [19:16:09] I'm not running on the actual elasticsearch in the test environment, because it needs me to create an index, and I don't want to mess things up there [19:16:18] Just using my local installation [19:16:32] https://gist.github.com/HAKSOAT/16ca8ae47830cd51605f7f7c4c272ac1 the results halfak [19:16:56] I intend polishing the regex more and also see how I can extract the named groups as well [19:17:14] I currently don't know how to get that [19:17:17] That looks awesome. Getting types eventually will be important. [19:17:26] I see that there is a type. [19:17:37] I wonder if ES can associate a token regex with a type [19:17:41] Elasticsearch has its own type system [19:17:56] But since I included named groups in the regex [19:18:04] There should be a way to fetch it [19:19:10] For performance, I'm yet to profile it since it's still a bit rough getting it to work as we want, but I think it's quite fast [19:20:52] Hello! [19:21:13] How did this happen!? I missed zipp too! [19:21:22] * halfak looks for duplicates again. [19:21:29] Hello rotuna! [19:21:37] hello rotuna [19:21:40] ls [19:26:01] I found this job opening [https://boards.greenhouse.io/wikimedia/jobs/2117176] and applied, I thought it would be good to talk to the current team to get know more about the team and the work you guys do. [19:26:02] (03PS1) 10Halfak: Removes double zipp [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/591148 [19:26:45] rotuna, yup! That's us. Here's our public face: https://www.mediawiki.org/wiki/Wikimedia_Scoring_Platform_team [19:26:51] Oh! I need to add haksoat [19:27:10] haksoat, do you have a photo of you up on commons? [19:27:29] Yup! I found this chat through that link. :D [19:28:50] I can't really find the code though, I keep finding that seems to be the trello boards. [19:29:03] Ha! Those still exist!? [19:29:17] See the "Projects" section :) [19:30:16] Not yet [19:30:20] rotuna, can you see this? https://github.com/orgs/wikimedia/teams/scoring-platform/repositories [19:30:22] I could add a picture [19:30:48] No, GitHub is trying to go Jedi on me [19:30:55] Jedi? [19:31:19] This is the board we work from: https://phabricator.wikimedia.org/tag/scoring-platform-team-current/ [19:31:25] Here's our backlog: https://phabricator.wikimedia.org/tag/scoring-platform-team [19:32:18] The 404 image is GitHub trying to tell me that isn't what I'm looking for. I've always seen that as Github trying to go Jedi on me [ Bad joke, sorry ] [19:32:27] Oh! Ha. [19:33:44] There's really not a good way to find a list of our repos. That's annoying. [19:34:20] https://github.com/wikimedia/ores & https://github.com/wikimedia/revscoring represent the frameworks we are building on top of. [19:34:30] We have repos for each type of model we support. [19:34:55] https://github.com/wikimedia/editquality & https://github.com/wikimedia/articlequality & https://github.com/wikimedia/draftquality & https://github.com/wikimedia/drafttopic are examples. [19:34:57] Found it [19:35:28] accraze, sorry to bother you again. I don't know how I missed this one: https://gerrit.wikimedia.org/r/#/c/research/ores/wheels/+/591148 [19:35:33] Oh I just found ores [19:35:37] But I did a scan and I think that is the last one. [19:36:37] (03CR) 10Accraze: [C: 03+2] Removes double zipp [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/591148 (owner: 10Halfak) [19:37:24] accraze, I want to talk to you about a good way to run CI on our wheels. These kind of issues should be easy to catch. I'll make an item for the backlog :) [19:37:27] More automation! [19:37:46] cool yeah was just going to say we should automate all of the wheels stuff [19:38:59] 10Scoring-platform-team, 10Continuous-Integration-Infrastructure: CI should check to see if our wheels are good - https://phabricator.wikimedia.org/T250746 (10Halfak) [19:39:13] halfak: For some reason, I can't log into commons.wikimedia.org for an upload [19:39:25] But I can log into my regular wiki account [19:39:53] If you are already logged into your wiki account, you should be automatically logged into commons. [19:40:03] Oh wait. wikitech and commons are separate. [19:40:10] wikipedia and commons use the same account. [19:40:12] yeah [19:40:19] wikitech has a separate registration [19:40:40] Haha [19:40:44] Exactly [19:40:54] Changed passwords now... I'm in [19:41:49] rotuna, ORES is pretty old tech at this point. We're gearing up to revamp it on top of a nice open source framework. [19:42:06] Right now we're looking at KubeFlow, but there are other frameworks we want to look at before we settle. [19:44:19] (03PS1) 10Halfak: Bumps wheels to HEAD (rm double zipp) [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/591151 [19:45:31] (03CR) 10Halfak: [V: 03+2 C: 03+2] "(Self-merging) Bumps wheels to HEAD" [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/591151 (owner: 10Halfak) [19:52:08] ITS ALIVE [19:52:08] https://ores-beta.wmflabs.org/v3/scores/ptwiki/7239 [19:52:47] halfak: https://commons.wikimedia.org/wiki/File:Habeeb_Shopeju_2020.jpg [19:53:17] Nice photo! [19:53:34] :) Thanks [19:54:00] great photo haksoat <3 [19:55:06] haksoat, https://www.mediawiki.org/wiki/Wikimedia_Scoring_Platform_team#Team [19:56:11] Thanks chtnnh [19:57:04] Looks like we are good on beta. I'm going to try to send this to prod in 3 minutes. [19:59:11] yay :D [20:04:32] Interesting that, while I'm doing this deploy, top on ores1001 reports 50GB free while grafan reports 22gb free. [20:05:53] that sounds good [20:05:55] Looks like a lag of ~3 minutes. [20:06:08] ouch [20:20:00] Make your user/common.js page look like this: https://pt.wikipedia.org/wiki/Usu%C3%A1rio(a):EpochFail/common.js [20:20:06] chtnnh, ^ [20:20:49] It works! [20:21:00] user/common.js ? [20:21:10] User:Chtnnh/common.js [20:21:15] Or whatever you have as a username. [20:22:20] i should add importScript("User:EpochFail/ArticleQuality.js") to my page? [20:23:29] yes i see ORES predicted quality [20:23:31] xD [20:32:10] \o/ [20:33:28] I'm trying to look into this: https://phabricator.wikimedia.org/T242013 Could anyone tell me how to access stat1007.eqiad.wmnet? [20:37:31] halfak ^ [20:38:18] rotuna, stat1007 is an internal server that only staff have access to. Do you need a file discussed in the ticket? [20:38:52] It would be nice if you could send it to me, yes. [20:41:15] Which file? [20:41:40] Oh ha. You want isaacj's code, probably. [20:42:02] isaacj, any chance we could get you to host your work on https://phabricator.wikimedia.org/T242013 publicly? [20:43:23] halfak: yep, that's easy. I'll throw it into a repo and ping [20:44:30] Thanks! [20:44:37] rotuna: just in case you sign off, i'll put it here: https://github.com/geohci/miscellaneous-wikimedia [20:50:20] Thanks! [20:50:37] Yeah, I just wanted the code [21:13:00] rotuna: https://github.com/geohci/miscellaneous-wikimedia/blob/master/topic-classification/topic_classification.md [21:32:57] 10Scoring-platform-team (Research), 10revscoring, 10artificial-intelligence: Implement native NN model in revscoring - https://phabricator.wikimedia.org/T242013 (10Isaac) FYI -- I gathered the code for this and put it in this repository: [[https://github.com/geohci/miscellaneous-wikimedia/blob/master/topic-c...