[13:50:29] halfak: here's my best effort to port diyi's edittypes work to the revscoring world - https://github.com/wikimedia/edittypes/pull/2 [13:52:29] halfak: also, i need a suggestion - i have to process revisions from the enwiki dump, like say - "get all revisions where rev_comment contains `pov`". Will i be good with using python-mwxml and processing on a single server or should i go for processing it on a hadoop cluster? [13:52:52] i have not processed entire dumps before, so I'm wondering if you have thoughts on that [13:53:13] codezee, I think mwxml will be great for that. [13:53:36] If you don't need the text, you can download the "stub-meta-history" dump. [13:54:08] Hadoop would be good except you'd still need to run mwxml to convert the dump to "revdocs" before you can process it. [13:54:16] I can show you how that works. [13:55:00] i was looking at - https://github.com/whym/wikihadoop and it looked like it can process the dumps directly [13:55:21] but yeah, i've never done it before so i can't be sure [13:55:46] i think https://dumps.wikimedia.org/other/mediawiki_history/readme.html has the comment field [13:55:49] so you might not have to parse dumps [13:56:28] i believe it has this schema [13:56:33] https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Edits/MediaWiki_history#Schema [13:56:42] (in tsv) [13:56:44] thanks! ottomata that'd work for comments, tho i'd still need the dumps bec i may need to look at diffs too [13:56:52] aye [13:57:31] the schema is a useful piece of info! *bookmarks* :) [13:59:05] codezee, looks like there are a lot of utility functions in this PR for edittypes. Any chance I could convince you to write some tests for them? [14:00:41] codezee, just dropped you an invite to my virtual office hours. Feel free to drop in when you like. [14:01:20] halfak: thanks for the office hours! I can try writing the tests since i'm familiar with the code. Do you want them in this PR or separately in followup? [14:21:44] codezee & chtnnh: hangouts is stuck and won't reload. Trying a reboot. [14:29:05] halfak: o/. Just letting you know I have numbers for guesstimates based on current usage for the next annual budget. Used an OLS method built into prometheus. See https://grafana.wikimedia.org/d/xEDjLvgMz/cluster-resource-predictions?orgId=1&from=now-6M&to=now&var-datasource=codfw%20prometheus%2Fglobal&var-cluster=ores&var-months=12. So I don't need anymore those numbers from you. I don't know about numbers I should add to those ab [14:29:05] out future plans however, so if you have something that's bound to be done (I guess that means not dependent on being approved in the upcoming annual planning cycle), let me know. [15:08:54] akosiaris, hey! [15:08:56] Just got out of a meeting. [15:08:58] So, I do have some estimates of what we'll need to get over our memory limitations in that capex task. [15:09:09] https://phabricator.wikimedia.org/T249917 [15:10:33] If spec'ing servers with twice as much RAM is some version of "cheap" and "easy", let's just settle on that. If it's a hard sell. I wonder about adding *more* servers at the same capacity and experimenting with splitting web and celery workers. [15:11:22] brb [15:27:49] halfak: in a meeting, but I 'll have a look at the task, I missed it [15:28:12] OK sounds good :) [15:40:38] 10ORES, 10Scoring-platform-team (Current), 10artificial-intelligence: Write report about misclassification reports - https://phabricator.wikimedia.org/T251905 (10Chtnnh) https://www.mediawiki.org/wiki/ORES/Issues/Article_quality The table has been updated under the summary section of the misclassification r... [15:44:10] 10ORES, 10Scoring-platform-team (Current), 10Operations: ORES uwsgi consumes a large amount of memory and CPU when shutting down (as part of a restart) - https://phabricator.wikimedia.org/T242705 (10Halfak) Oh yes. We need help from SRE. I'm at my limit here. @akosiaris was working with us on this in the... [15:59:40] halfak: btw, I also have this for you. https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/595167/. I 'll orchestrate it in a sane way to avoid disruption as much as possible ofc, by applying this in small chunks and restarts properly orchestrated to honor dependencies. But TL;DR the gain is a pretty nice 6->15GB increase for the cache, better ownership of the redis nodes and better monitoring, not to mention the fact we consoli [15:59:40] date hardware. [16:01:39] Oh yeah. This sounds fine to me. I don't think we'll need to be too clever in making the switch since celery and our cache don't struggle to much with a fresh start. [16:01:55] Maybe we can target a lower traffic period. [16:50:26] 10Jade, 10Scoring-platform-team (Current): Implement diff *of* Jade entity pages - https://phabricator.wikimedia.org/T247762 (10Halfak) a:03Halfak [16:50:37] 10Scoring-platform-team (Current), 10editquality-modeling, 10artificial-intelligence: Create editquality labeling campaign for Hindi Wikipedia - https://phabricator.wikimedia.org/T252594 (10Halfak) a:03Halfak [16:51:58] 10Scoring-platform-team (Current), 10editquality-modeling, 10artificial-intelligence: Create editquality labeling campaign for Hindi Wikipedia - https://phabricator.wikimedia.org/T252594 (10Halfak) https://github.com/wikimedia/editquality/pull/222 [16:53:45] 10ORES, 10Scoring-platform-team (Current), 10editquality-modeling, 10artificial-intelligence: Visualize the relationship between the probability of reversion and ores scores - https://phabricator.wikimedia.org/T224918 (10Halfak) Finally moving this to done. I have seen the visualizations. Maybe @Groceryh... [16:58:24] 10Jade, 10Scoring-platform-team (Current): Implement Special:Diff integration for Jade - https://phabricator.wikimedia.org/T212387 (10Halfak) a:03kevinbazira [16:59:55] 10Scoring-platform-team, 10artificial-intelligence: Unsophisticated bad actors dataset - https://phabricator.wikimedia.org/T252894 (10Halfak) p:05Triage→03Medium [17:00:47] 10Scoring-platform-team, 10artificial-intelligence: Create wikilabels campaign for unsophisticated bad actors dataset. - https://phabricator.wikimedia.org/T252895 (10Halfak) p:05Triage→03Medium [17:00:59] 10Scoring-platform-team, 10artificial-intelligence: Decide on unsophisticated bad actors labels. - https://phabricator.wikimedia.org/T252896 (10Halfak) p:05Triage→03Medium [17:02:09] 10Scoring-platform-team, 10Research: Write Python util for converting Wikidata claims to features for ML models - https://phabricator.wikimedia.org/T252775 (10Halfak) p:05Triage→03High [17:02:49] 10Scoring-platform-team, 10drafttopic-modeling: Build article topic model for Swedish Wikipedia - https://phabricator.wikimedia.org/T252599 (10Halfak) p:05Triage→03Medium [17:03:29] 10Scoring-platform-team, 10editquality-modeling, 10artificial-intelligence: Complete editquality labeling for Hindi Wikipedia - https://phabricator.wikimedia.org/T252582 (10Halfak) p:05Triage→03Medium [17:04:00] 10Scoring-platform-team, 10editquality-modeling, 10artificial-intelligence: Train and test editquality models for Hindi Wikipedia - https://phabricator.wikimedia.org/T252581 (10Halfak) p:05Triage→03Medium [17:05:37] 10Jade, 10Scoring-platform-team: Mismatch between Jade requirements and some wikitools - https://phabricator.wikimedia.org/T252477 (10Halfak) [17:06:44] 10Jade, 10Scoring-platform-team: 'endorsementcomment' is required on jadeproposeorendorse. Shouldn't be. - https://phabricator.wikimedia.org/T253038 (10Halfak) [17:07:33] 10Jade, 10Scoring-platform-team: 'endorsementcomment' is required on jadeproposeorendorse. Shouldn't be. - https://phabricator.wikimedia.org/T253038 (10Halfak) p:05Triage→03Low Making this low because it's not a blocker. You can set a blank endorsementcomment and it should work. But we'll still want to... [17:09:15] 10Jade, 10Scoring-platform-team: [Discussion] How to advise patrolling tools in submitting to Jade when they don't consider "goodfaith". - https://phabricator.wikimedia.org/T253039 (10Halfak) [17:09:34] 10Jade, 10Scoring-platform-team: [Discussion] How to advise patrolling tools in submitting to Jade when they don't consider "goodfaith". - https://phabricator.wikimedia.org/T253039 (10Halfak) p:05Triage→03Medium [17:10:11] 10Jade, 10Scoring-platform-team: Mismatch between Jade requirements and some wikitools - https://phabricator.wikimedia.org/T252477 (10Halfak) p:05Triage→03Medium [17:10:24] 10Jade, 10Scoring-platform-team: Mismatch between Jade requirements and some wikitools - https://phabricator.wikimedia.org/T252477 (10Halfak) a:05Halfak→03None [17:10:34] 10ORES, 10Scoring-platform-team, 10Documentation, 10Epic: [Epic] Clean up ORES documentation - https://phabricator.wikimedia.org/T148974 (10Halfak) a:05Halfak→03None [17:11:27] 10Scoring-platform-team (Current), 10Wikilabels: Wikilabels: SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY - https://phabricator.wikimedia.org/T252441 (10Halfak) a:03He7d3r [17:11:49] 10Scoring-platform-team (Current), 10Wikilabels: Wikilabels: SERVER HAS FAILED AND IS DISABLED UNTIL TIMED RETRY - https://phabricator.wikimedia.org/T252441 (10Halfak) 05Open→03Resolved Thanks for your work on this, @He7d3r [17:12:33] 10Scoring-platform-team (Current), 10articlequality-modeling, 10artificial-intelligence: Extracted labels might not be accurate when there are multiple reverts - https://phabricator.wikimedia.org/T252152 (10Halfak) a:03He7d3r [17:13:21] 10Scoring-platform-team (Current), 10revscoring, 10artificial-intelligence: Feature request: add weighted sum utility in articlequality to handle revscoring score utility output - https://phabricator.wikimedia.org/T252053 (10Halfak) [17:31:02] 10Scoring-platform-team, 10artificial-intelligence: Unsophisticated bad actors dataset - https://phabricator.wikimedia.org/T252894 (10ST47) So, I've started trying to build the set of known sockpuppet groups based on userpage tagging and block summaries. So far I have 14,700 masters and a total of 174,000 acco... [17:49:27] 10ORES, 10Scoring-platform-team (Current), 10artificial-intelligence: Review model performance for ptwiki 'articlequality' and 'draftquality' - https://phabricator.wikimedia.org/T250809 (10He7d3r) @Halfak: I've made two tests: 1. restricting the dataset to 2014-2020; 2. removing bots (but still using the who... [17:51:33] 10ORES, 10Scoring-platform-team (Current), 10artificial-intelligence: Review model performance for ptwiki 'articlequality' and 'draftquality' - https://phabricator.wikimedia.org/T250809 (10He7d3r) PS: I didn't change the thresholds in the Makefile, so the samples were not as balanced as might be wanted: ` $... [20:32:54] wikimedia/articlequality#377 (he7d3r-extractor-and-reverts - 9bbc3b4 : halfak): The build passed. https://travis-ci.org/wikimedia/articlequality/builds/688553625 [20:33:29] halfak, chtnnh : I'm back :) [20:33:47] nice! back to business then [20:53:36] halfak, would you have any feedback about the script for exposing draftquality predictions? I posted it to https://phabricator.wikimedia.org/T246667#6079484 some time ago, but I don't remember if we talked about it [20:59:16] * halfak clicks [21:00:02] wikimedia/ores#1451 (blubber - f62594d : Andy Craze): The build passed. https://travis-ci.org/wikimedia/ores/builds/688563142 [21:02:47] Helder_, this looks great. [21:03:21] I wonder if making the icon smaller and giving the
  • a background would make it a bit more visually clear. [21:03:36] I'm imagining mimicking Special:Recentchanges [21:03:59] One thing I suspect we'll want to do is set some thresholds on confidence level we expect for flagging a new page. [21:06:01] https://ores.wikimedia.org/v3/scores/ptwiki/?models=draftquality&model_info=statistics.thresholds.OK.%22maximum%20!precision%20@%20!recall%20%3E=%200.9%22 [21:06:31] Looks like we will catch 90% of the "not OK" stuff if we flag anything with <= 0.49 probability of being "OK" [21:11:24] I'm sure the "design" can be improved a lot. I just picked some images I could find without spending too much time searching for one (they needed to be in public domain, because they would be used as background, and as such, without providing the attribution link to commons required by most licenses) [21:12:04] 10ORES, 10Scoring-platform-team (Current), 10artificial-intelligence: Review model performance for ptwiki 'articlequality' and 'draftquality' - https://phabricator.wikimedia.org/T250809 (10Halfak) Wow! Removing bot contributions ended up removing a ton of labels. What do you think explains that? Are there... [21:12:30] Wait. Is that true? That's insane. [21:12:45] One thing I was not sure about is the weighted sum: it does not make sense for draftquality, right? [21:12:51] Right. [21:16:57] There is a note about removing the link to the description pages which serves as the attribution at: https://www.mediawiki.org/wiki/Help:Images#Altering_the_default_link_target [21:17:24] and I remember for there is the option of adding the (i) blue icon over the images for which we remove the attribution link [21:18:00] re [21:18:40] re: using a background for the
  • : that could get confusing given that special:newpages already uses a yellow background for unpatrolled new pages [21:20:30] halfak, the unpatrolled pages appear like this: https://i.postimg.cc/LXSW2jWn/Screenshot-from-2020-05-18-18-19-32.png [21:21:01] Oh that makes sense then. Nevermind. [21:21:15] I think the icons are sufficiently visually striking. [21:22:15] https://commons.wikimedia.org/wiki/Category:OOUI_icons [21:22:41] Here's some guaranteed safe to use icons if you feel like looking at 'em. As it stands, I think this works amazingly well for a test. [21:25:05] btw: the code is mostly a copy of your script, as shown in the diff, but it could probably be refactored so to avoid code duplication [21:25:31] Helder_, I would like that. The ORES querying pattern should be straightforward to split out. [21:57:49] I'm taking off for the day. y'all take care! [22:01:43] 10ORES, 10Scoring-platform-team (Current), 10Operations, 10Release Pipeline (Blubber): Build blubber file for ORES - https://phabricator.wikimedia.org/T210268 (10ACraze) [22:01:59] 10ORES, 10Scoring-platform-team (Current), 10Operations, 10Release Pipeline (Blubber): Build blubber file for ORES - https://phabricator.wikimedia.org/T210268 (10ACraze) a:03ACraze [22:08:19] 10ORES, 10Scoring-platform-team (Current), 10Operations, 10Release Pipeline (Blubber): Build blubber file for ORES - https://phabricator.wikimedia.org/T210268 (10ACraze) Got a WIP PR here: https://github.com/wikimedia/ores/pull/345 Still need to slim down the production image and handle the redis dep for... [22:25:13] 10Scoring-platform-team (Current), 10Wikilabels, 10editquality-modeling, 10artificial-intelligence: Build draft quality model for ptwikipedia - https://phabricator.wikimedia.org/T246667 (10GoEThe) @He7d3r , I tried it briefly. I think the script works fine. The symbols are not immediately intuitive, but I... [22:28:21] 10Scoring-platform-team (Current), 10Wikilabels, 10editquality-modeling, 10artificial-intelligence: Build draft quality model for ptwikipedia - https://phabricator.wikimedia.org/T246667 (10He7d3r) @GoEThe: in case you have any suggestions on better images for this purpose, we can try changing them. @Halfak... [22:33:36] 10ORES, 10Scoring-platform-team (Current): Estimate ORES CapEx for FY21 - https://phabricator.wikimedia.org/T249917 (10ACraze) @Halfak I do like the idea of adding more servers to split the workers, though I'm wondering if that would limit us in terms of growth/hosting more types of models/etc. with the smalle... [22:40:26] 10Scoring-platform-team (Current), 10Wikilabels, 10editquality-modeling, 10artificial-intelligence: Build draft quality model for ptwikipedia - https://phabricator.wikimedia.org/T246667 (10GoEThe) @He7d3r , perhaps https://commons.wikimedia.org/wiki/File:OOjs_UI_icon_alert-warning.svg and https://commons.w...