[06:03:21] legoktm: hey, if you are around, Please check this: https://gerrit.wikimedia.org/r/#/c/297540/ [06:17:35] 06Revision-Scoring-As-A-Service, 10Wikilabels, 10rsaas-editquality: Auto-label French edit quality campaign - https://phabricator.wikimedia.org/T107730#2432457 (10Ladsgroup) First step: https://github.com/wiki-ai/editquality/pull/38 [08:22:22] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: Have a metric for direct requests to ORES not scoring ones - https://phabricator.wikimedia.org/T139446#2432568 (10Ladsgroup) [09:53:58] 06Revision-Scoring-As-A-Service, 10ORES: Planning long-term ORES architecture - https://phabricator.wikimedia.org/T137125#2432667 (10schana) [14:59:30] Hi all, just saying hi, I talked with someone from this project in Wikimania (don't remember the name, sorry), :( [15:04:41] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-edittypes: Complete itwiki edit types campaign - https://phabricator.wikimedia.org/T131261#2433270 (10Halfak) p:05Triage>03Lowest [15:05:35] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-edittypes: Complete enwiki edit types campaign - https://phabricator.wikimedia.org/T131260#2433272 (10Halfak) p:05Triage>03Lowest [15:09:35] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-editquality: Complete arwiki edit quality campaign - https://phabricator.wikimedia.org/T139367#2433279 (10Halfak) p:05Triage>03Lowest [15:10:25] 10Revision-Scoring-As-A-Service-Backlog: Azerbaijani language utilities - https://phabricator.wikimedia.org/T112294#2433280 (10Halfak) p:05High>03Lowest [15:10:37] 10Revision-Scoring-As-A-Service-Backlog, 10revscoring: Chinese language utilities - https://phabricator.wikimedia.org/T109366#2433281 (10Halfak) p:05High>03Lowest [15:11:10] 10Revision-Scoring-As-A-Service-Backlog: Edit quality campaign for Greek Wikipedia - https://phabricator.wikimedia.org/T122724#2433283 (10Halfak) p:05Normal>03Lowest [15:11:33] 10Revision-Scoring-As-A-Service-Backlog: Edit quality campaign for Chinese Wikipedia - https://phabricator.wikimedia.org/T116474#2433284 (10Halfak) p:05Normal>03Lowest [15:11:50] 10Revision-Scoring-As-A-Service-Backlog: Afrikaans language utilities - https://phabricator.wikimedia.org/T106846#2433285 (10Halfak) p:05Normal>03Lowest [15:12:27] 10Revision-Scoring-As-A-Service-Backlog: Armenian language utilities - https://phabricator.wikimedia.org/T106843#2433299 (10Halfak) p:05Normal>03Lowest [15:13:00] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-editquality: Complete azwiki edit quality campaign - https://phabricator.wikimedia.org/T129699#2433300 (10Halfak) p:05Triage>03Lowest [15:13:33] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-editquality: Complete viwiki edit quality campaign - https://phabricator.wikimedia.org/T130273#2433301 (10Halfak) p:05Triage>03Lowest [15:13:50] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-editquality: Complete urwiki edit quality campaign - https://phabricator.wikimedia.org/T130272#2433302 (10Halfak) p:05Triage>03Lowest [15:13:58] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-editquality: Complete hewiki edit quality campaign - https://phabricator.wikimedia.org/T130263#2433303 (10schana) p:05Triage>03Lowest [15:14:09] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-editquality: Complete idwiki edit quality campaign - https://phabricator.wikimedia.org/T130264#2433304 (10schana) p:05Triage>03Lowest [15:14:15] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-editquality: Complete itwiki edit quality campaign - https://phabricator.wikimedia.org/T130265#2433305 (10schana) p:05Triage>03Lowest [15:14:21] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-editquality: Complete ukwiki edit quality campaign - https://phabricator.wikimedia.org/T130271#2433306 (10Halfak) p:05Triage>03Lowest [15:14:23] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-editquality: Complete jawiki edit quality campaign - https://phabricator.wikimedia.org/T130266#2433307 (10schana) p:05Triage>03Lowest [15:14:25] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-editquality: Complete nowiki edit quality campaign - https://phabricator.wikimedia.org/T130268#2433308 (10Halfak) p:05Triage>03Lowest [15:14:35] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-editquality: Complete dewiki edit quality campaign - https://phabricator.wikimedia.org/T129700#2433309 (10schana) p:05Triage>03Lowest [15:14:42] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-editquality: Complete fawiki (v2) edit quality campaign - https://phabricator.wikimedia.org/T129703#2433310 (10ggellerman) p:05Triage>03Lowest [15:14:44] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-editquality: Complete etwiki edit quality campaign - https://phabricator.wikimedia.org/T129702#2433311 (10schana) p:05Triage>03Lowest [15:14:49] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-editquality: Complete frwiki edit quality campaign - https://phabricator.wikimedia.org/T130261#2433312 (10Halfak) p:05Triage>03Lowest [15:15:17] 10Revision-Scoring-As-A-Service-Backlog, 10Bad-Words-Detection-System, 10rsaas-editquality: migrate bad words detection to editquality repo - https://phabricator.wikimedia.org/T131861#2433314 (10Halfak) p:05Triage>03Low [15:16:48] Oscar_, hey! [15:16:54] It was probably Amir1. [15:16:58] He's AFK right now [15:18:47] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: Investigate 20 minute period of bad scores for ptwiki - https://phabricator.wikimedia.org/T131606#2433330 (10Halfak) This was resolved. It was a feature extraction issue. [15:18:55] 10Revision-Scoring-As-A-Service-Backlog, 10ORES: Investigate 20 minute period of bad scores for ptwiki - https://phabricator.wikimedia.org/T131606#2433334 (10Halfak) 05Open>03Resolved [15:19:02] halfak: o/ [15:19:24] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels: Add wiki labels detail to deployment docs on wikitech - https://phabricator.wikimedia.org/T131768#2433335 (10Halfak) p:05Triage>03Normal [15:40:42] https://www.youtube.com/watch?v=eaADQTeZRCY "Professor Wikipedia" [15:40:43] :D [15:54:22] old but gold [15:56:30] yes [15:56:46] it demonstrates why ores is needed [15:56:52] :) [15:57:00] yup [15:57:10] GhassanMas so its gOLD [15:57:12] :D [15:57:26] hahah i can't agree more [15:59:03] halfak, aren't T131669 and T139367 same [15:59:04] T131669: Complete edit quality campaign for Arabic Wikipedia - https://phabricator.wikimedia.org/T131669 [15:59:04] T139367: Complete arwiki edit quality campaign - https://phabricator.wikimedia.org/T139367 [16:05:38] 10Revision-Scoring-As-A-Service-Backlog, 10ORES, 10revscoring, 07Documentation: Add MacOS instructions for installation to README - https://phabricator.wikimedia.org/T139355#2433485 (10schana) [16:11:58] GhassanMas, yeah. it looks like they are [16:12:01] Feel free to merge. [16:12:12] I'll get to it soon if you don't beat me to it :) [16:14:11] 10Revision-Scoring-As-A-Service-Backlog, 10rsaas-editquality: Complete edit quality campaign for Arabic Wikipedia - https://phabricator.wikimedia.org/T131669#2433526 (10Ghassanmas) [16:14:13] 10Revision-Scoring-As-A-Service-Backlog, 10Wikilabels, 10rsaas-editquality: Complete arwiki edit quality campaign - https://phabricator.wikimedia.org/T139367#2433524 (10Ghassanmas) [16:32:45] 10Revision-Scoring-As-A-Service-Backlog, 10Discussion-modeling: [Discuss] Detox integration with ORES - https://phabricator.wikimedia.org/T139007#2433596 (10ellery) [17:25:37] 10Revision-Scoring-As-A-Service-Backlog, 10Discussion-modeling: [Discuss] Detox integration with ORES - https://phabricator.wikimedia.org/T139007#2433830 (10ellery) Our current thinking is to make Detox into its own service that exposes a scoring API. This way ORES can just submit revision ids or diffs to the... [18:47:45] 06Revision-Scoring-As-A-Service, 10ORES, 13Patch-For-Review: Get health check warnings when a web node goes down from the load balancer's perspective - https://phabricator.wikimedia.org/T134782#2434294 (10Halfak) https://ores.wmflabs.org/node/ores-web-03/ and https://ores.wmflabs.org/node/ores-web-05/ now work [18:57:04] 10Revision-Scoring-As-A-Service-Backlog, 10Discussion-modeling: [Discuss] Detox integration with ORES - https://phabricator.wikimedia.org/T139007#2434325 (10Halfak) That seems like it would be overly complicated and would result in a bunch of duplicated infrastructure. Why not just let ORES host the predictio... [19:13:48] PROBLEM - ORES web node labs ores-web-03 on ores.wmflabs.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds [19:15:59] RECOVERY - ORES web node labs ores-web-03 on ores.wmflabs.org is OK: HTTP OK: HTTP/1.1 200 OK - 352 bytes in 0.619 second response time [19:16:08] Damn right [19:16:10] It works! [19:37:24] halfak: nl.wikipedia.org/wiki/Speciaal:Voorkeuren#mw-prefsection-betafeatures [19:37:36] the next one is ruwiki [19:38:07] \o/ Amir1 [19:38:14] hey btw :D [19:38:21] BTW, Oscar_ was in here earlier and looking to say "hi" [19:39:08] nice. I talked to him in wikimania and I'm supposed to send him some notes on how to recruit more labelers [19:39:51] I want to do after deployment of ores in ptwiki (since they are close) [19:40:50] halfak: If you have some time, please check this: https://github.com/wiki-ai/editquality/pull/38 [19:41:02] if it looks okay, I make the new campaign [19:42:19] Amir1, looks good. Do you end up with enough obs. though? [19:42:28] Are there 2500 revisions that need review? [19:42:38] 2500 needing review and 2500 without [19:42:56] my biggest concern is that this dataset is old [19:42:57] Yeah. Just making sure that there were enough in the 20k file! [19:43:06] (2015) [19:43:12] do I need to get a new one? [19:43:24] Hmm... Not critical, but it doesn't seem like a bad idea. [19:43:28] I can't see a drawback [19:44:00] okay [19:44:08] I'm on it [20:14:27] halfak: https://github.com/wiki-ai/editquality/pull/38/files [20:14:30] updated [20:14:40] and tested some edits there, looked okay [20:15:56] Amir1, looks good to me. Ready? [20:16:04] yeah [20:16:13] and then I make the new campaign [20:16:18] and close the old one [20:16:27] {{merged}} [20:16:27] 10[1] 04https://meta.wikimedia.org/wiki/Template:merged [20:17:34] yess [20:17:34] Amir1, make sure you a make a DB backup before editing the DB directly. [20:17:41] thanks halfak [20:17:42] sure [20:17:44] ( I figure you know, but just a reminder ) [20:18:02] my plan was to inject it to staging first and then moving to pord [20:18:04] *prod [20:18:18] but both of them can work too [20:19:46] I do a backup with a line like this: pg_dump -h wikilabels-database --user u_wikilabels u_wikilabels -W | gzip -c > 2016-07-06.sql.gz [20:20:19] Each backup is ~3MB [20:20:23] so no big deal. [20:20:25] I make a lot of 'em [20:24:31] 06Revision-Scoring-As-A-Service, 10ORES, 13Patch-For-Review: Get health check warnings when a web node goes down from the load balancer's perspective - https://phabricator.wikimedia.org/T134782#2434752 (10Dzahn) Yes, and the checks also look good in Icinga. If you want to we can do that test and break someth... [20:26:19] 06Revision-Scoring-As-A-Service, 10ORES, 13Patch-For-Review: Get health check warnings when a web node goes down from the load balancer's perspective - https://phabricator.wikimedia.org/T134782#2434763 (10Halfak) Fun story, we actually had a minor amount of downtime on ores-web-03 during a puppet run today a... [20:29:32] halfak: we should have cron job to do it automatically every day [20:29:36] what do you think? [20:29:47] for time stamp we can use bash [20:30:03] it has been long time since I wrote scripts in bsh [20:30:05] *bash [20:30:54] Agreed. We should have a backup scheme [20:31:00] 06Revision-Scoring-As-A-Service, 10ORES, 13Patch-For-Review: Get health check warnings when a web node goes down from the load balancer's perspective - https://phabricator.wikimedia.org/T134782#2434782 (10Dzahn) Confirmed, i have seen that minor outage too. And we have history here to proof it. https://ici... [21:01:53] how does ORES detect vandalism when changing facts ? [21:56:01] 06Revision-Scoring-As-A-Service, 10Wikilabels, 10rsaas-editquality: Auto-label French edit quality campaign - https://phabricator.wikimedia.org/T107730#2435170 (10Ladsgroup) https://meta.wikimedia.org/wiki/Research_talk:Revision_scoring_as_a_service/Work_log/2016-07-06#New_frwiki_labeling_campaign [21:56:41] SPAM'S COMING [21:56:53] 06Revision-Scoring-As-A-Service, 10MediaWiki-extensions-ORES, 13Patch-For-Review: Deploy ORES review tool in Dutch Wikipedia - https://phabricator.wikimedia.org/T139432#2435174 (10Ladsgroup) 05Open>03Resolved [21:57:00] 06Revision-Scoring-As-A-Service, 10Mobile-Content-Service, 10ORES, 06Operations, and 2 others: Investigate increased memory pressure on scb1001/2 - https://phabricator.wikimedia.org/T139177#2435178 (10Ladsgroup) 05Open>03Resolved [21:57:06] 06Revision-Scoring-As-A-Service, 10Wikilabels: Update article topic label data set - https://phabricator.wikimedia.org/T139073#2435180 (10Ladsgroup) 05Open>03Resolved [21:57:10] 06Revision-Scoring-As-A-Service, 10rsaas-editquality: Train `reverted` model for cswiki - https://phabricator.wikimedia.org/T138885#2435181 (10Ladsgroup) 05Open>03Resolved [21:57:13] 06Revision-Scoring-As-A-Service, 10revscoring, 10rsaas-editquality: Train/test reverted model for enwiktionary - https://phabricator.wikimedia.org/T138630#2435182 (10Ladsgroup) 05Open>03Resolved [21:57:16] 06Revision-Scoring-As-A-Service, 10MediaWiki-extensions-ORES, 10ORES, 07Epic: [Epic] Complete outstanding tasks for ORES extension deployments - https://phabricator.wikimedia.org/T138251#2435184 (10Ladsgroup) [21:57:18] 06Revision-Scoring-As-A-Service, 10ORES, 07Documentation: Fix circular linking in ORES docs - https://phabricator.wikimedia.org/T138089#2435183 (10Ladsgroup) 05Open>03Resolved [21:57:20] 06Revision-Scoring-As-A-Service, 10ORES: [discuss] What to do with vagrant? - https://phabricator.wikimedia.org/T135623#2435185 (10Ladsgroup) 05Open>03Resolved [21:57:22] 10Revision-Scoring-As-A-Service-Backlog, 10rsaas-editquality: Deploy edit quality models for plwiki - https://phabricator.wikimedia.org/T130292#2435187 (10Ladsgroup) [21:57:24] 06Revision-Scoring-As-A-Service, 10Wikilabels, 10rsaas-editquality: Complete plwiki edit quality campaign - https://phabricator.wikimedia.org/T130269#2435186 (10Ladsgroup) 05Open>03Resolved [21:58:33] halfak: btw. Is it okay to request for deleting the revscoring project in labs? There is no instance in it and we plan not to use it [22:19:11] Amir1, +1 [22:19:55] halfak: kk. frwiki is done btw. Tested everything and also made a back up too. [22:20:01] everything is just fine [22:20:47] fun story. I've been pushing fawiki people to label the new campaign to increase data accuracy. It wasn't successful [22:21:36] but today some people came and told me it's not working for them. After lots of discussion I realized they are using gadgets that are broken. I went and fixed them one by one (four cases!) [22:21:52] then we have some people labeling today [22:22:14] https://labels.wmflabs.org/campaigns/fawiki/?campaigns=stats [22:22:22] about 200 labels just made today [22:24:15] \o/ [22:24:18] Awesome news Amir1 [22:24:19] :) [22:24:30] We should report on that in the next weekly update [22:24:35] 06Revision-Scoring-As-A-Service, 06Labs: Delete 'revscoring' project in labs - https://phabricator.wikimedia.org/T139537#2435324 (10Ladsgroup) [22:25:05] I definitely need to set up the web app to show progress in campaigns [22:25:11] OH MY GOD. THE PICKLE BUG WAS A BUILT-IN PYTHON ERROR AHHH [22:25:25] Apparently IndexError is not pickle-able [22:25:30] * halfak punches self in the face [22:25:47] Amir1, I would love to see that. [22:26:08] Would be great for highlighting the master labelers too. [22:27:14] * halfak is seriously considering switching to use dill instead of pickle [22:27:21] It's a hack, but a lot of people get it working [22:27:29] lol @ dill pickle celery [22:27:51] :)))) [22:28:21] OMG IT WORTKS [22:28:36] * halfak finally feels a bit of sanity returning. [22:38:04] 06Revision-Scoring-As-A-Service, 10MediaWiki-extensions-ORES: Deploy ORES review tool in Russian Wikipedia - https://phabricator.wikimedia.org/T139541#2435412 (10Ladsgroup) [22:38:18] Amir1: you wanted me to review something I think? [22:38:32] legoktm: yeah, https://gerrit.wikimedia.org/r/#/c/297540/ [22:38:36] It would be great [22:38:49] hey by the way, FYI, it's deployed in nlwiki too [22:41:12] (03CR) 10Legoktm: [C: 032] Remove oresc_is_predicted = 1 in db queries [extensions/ORES] - 10https://gerrit.wikimedia.org/r/297540 (owner: 10Ladsgroup) [22:41:15] \o/ [22:41:25] legoktm: thanks :) [22:41:46] https://wikiapiary.com/wiki/Extension:ORES [22:42:13] (03Merged) 10jenkins-bot: Remove oresc_is_predicted = 1 in db queries [extensions/ORES] - 10https://gerrit.wikimedia.org/r/297540 (owner: 10Ladsgroup) [22:49:49] 06Revision-Scoring-As-A-Service, 10ORES, 07Epic: ORES refactor: Scoring structure - https://phabricator.wikimedia.org/T139408#2435477 (10Danny_B) [22:52:37] legoktm: nice, a cool tool [22:58:51] 06Revision-Scoring-As-A-Service, 06Labs: Delete 'revscoring' project in labs - https://phabricator.wikimedia.org/T139537#2435514 (10Andrew) 05Open>03Resolved thanks! [23:11:46] 06Revision-Scoring-As-A-Service, 10ORES, 13Patch-For-Review: Get health check warnings when a web node goes down from the load balancer's perspective - https://phabricator.wikimedia.org/T134782#2435583 (10Dzahn) @schana do you agree it's resolved? [23:13:21] OKAY [23:13:22] https://github.com/wiki-ai/ores/pull/144 [23:13:27] Is now ready for review. [23:13:51] BUT, if you're confused by all of the changes, I plan to go back and split up the changes into cleaner commits. [23:14:00] Hopefully that will be done tomorrow AM. [23:14:22] (AM UTC-6) [23:14:33] And with that, I'm out of here. [23:14:37] have a good one! [23:14:39] o/