[07:12:36] (03PS1) 10Awight: Update English models with new badwords [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/439839 (https://phabricator.wikimedia.org/T196468) [09:03:59] 10Scoring-platform-team (Current), 10ORES, 10Release-Engineering-Team, 10Scap: ORES deployments blocked by mysterious tin.eqiad.wmnet error - https://phabricator.wikimedia.org/T196663#4274748 (10awight) 05Open>03Resolved a:03awight It works now, thank you! Also, we're getting our LFS file loud and c... [09:04:01] 10Scoring-platform-team (Current), 10ORES, 10WMF-Design, 10WikimediaUI Style Guide, and 3 others: Give a new look to the home page - https://phabricator.wikimedia.org/T196580#4274752 (10awight) [10:20:00] 10Scoring-platform-team (Current), 10ORES, 10WMF-Design, 10WikimediaUI Style Guide, and 3 others: Give a new look to the home page - https://phabricator.wikimedia.org/T196580#4274943 (10awight) 05Open>03Resolved Cache finally expired, this is confirmed deployed. [10:24:55] 10Scoring-platform-team, 10JADE, 10Operations, 10User-Joe: Scalability concerns creating a page per revision - https://phabricator.wikimedia.org/T196547#4274967 (10awight) Wikidata wouldn't survive a year of this upper-bound unscalability. It has received 200M edits in the past 12 months, so we would have... [11:06:45] 10Scoring-platform-team, 10JADE, 10Operations, 10User-Joe: Scalability concerns creating a page per revision - https://phabricator.wikimedia.org/T196547#4275076 (10awight) Some negatives to the per-page approach: * Slightly incompatible with ORES, which is per-revision. For example, fetching an ORES+JADE... [11:13:35] 10Scoring-platform-team (Current), 10ORES, 10Wikimedia-Hackathon-2018: Rewrite ORES "reference" UI using React - https://phabricator.wikimedia.org/T195274#4275082 (10awight) @Jdlrobson Ping, I'd be curious to get your feedback about http://github.com/wiki-ai/ores-reference-ui [12:48:24] (03PS1) 10Awight: Provide an index from revision to JADE page. [extensions/JADE] - 10https://gerrit.wikimedia.org/r/439895 [12:48:26] (03PS1) 10Awight: [WIP] Look at page-judgment schema [extensions/JADE] - 10https://gerrit.wikimedia.org/r/439896 [12:49:31] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Look at page-judgment schema [extensions/JADE] - 10https://gerrit.wikimedia.org/r/439896 (owner: 10Awight) [12:50:04] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Look at page-judgment schema [extensions/JADE] - 10https://gerrit.wikimedia.org/r/439896 (owner: 10Awight) [13:15:32] 10Scoring-platform-team, 10JADE, 10Operations, 10User-Joe: Scalability concerns creating a page per revision - https://phabricator.wikimedia.org/T196547#4275468 (10awight) In the per-page schema proposed above, the page-revision index would grow at the scary rate, up to one index entry per revision added t... [13:41:58] (03PS2) 10Awight: [WIP] Look at page-judgment schema [extensions/JADE] - 10https://gerrit.wikimedia.org/r/439896 [13:43:11] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Look at page-judgment schema [extensions/JADE] - 10https://gerrit.wikimedia.org/r/439896 (owner: 10Awight) [13:47:04] halfak: Any interest in doing our routine deployment today, at 17:00 UTC? [13:47:13] It's not a big rush, just the new badword [13:47:14] Sure! [13:47:16] cool [13:47:22] Did you rebuild the model? [13:48:00] I think it'll be good to have someone else try the git-lfs stuff. There shouldn't be any sleight-of-hand involved, but it would be good to know that I'm not relying on arcane dot-files having been created by earlier runs, that kind of crap. [13:48:17] halfak: yes, here are the commits: [13:49:37] awight, roger that [13:50:49] 10Scoring-platform-team (Current), 10Analytics, 10EventBus, 10ORES, and 3 others: Numeric keys in ORES models causing downstream Hive ingestion to fail - https://phabricator.wikimedia.org/T195979#4275547 (10Ottomata) Thanks @awight. I just tried to re-enable, but there are more (possibly MANY more) proble... [13:51:03] 10Scoring-platform-team (Current), 10Analytics, 10EventBus, 10ORES, and 3 others: Invalid field names in ORES models causing downstream Hive ingestion to fail - https://phabricator.wikimedia.org/T195979#4275548 (10Ottomata) [13:52:01] 10Scoring-platform-team (Current), 10editquality-modeling, 10revscoring, 10Patch-For-Review, 10artificial-intelligence: Catch a specific new badword in English - https://phabricator.wikimedia.org/T196468#4275549 (10awight) https://github.com/wiki-ai/articlequality/pull/69 https://github.com/wiki-ai/editq... [13:53:42] 10Scoring-platform-team (Current), 10Analytics, 10EventBus, 10ORES, and 3 others: Invalid field names in ORES models causing downstream Hive ingestion to fail - https://phabricator.wikimedia.org/T195979#4275583 (10awight) @Ottomata Thanks for the investigation and explanations! This should be fun ;-) [13:53:53] halfak: https://phabricator.wikimedia.org/T196468#4275549 [13:55:23] awight, we don't use badwords in the wp10 models. [13:55:49] aha, thanks I should have caught that [13:55:57] And certainly not in drafttopic... [13:57:30] (03PS2) 10Awight: Update English models with new badwords [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/439839 (https://phabricator.wikimedia.org/T196468) [13:57:49] halfak: abandoned the articlequality changes. [13:58:01] {{merged}} the other [13:58:02] 10[3] 04https://meta.wikimedia.org/wiki/Template:merged [13:58:11] draftquality! [13:58:35] https://github.com/wiki-ai/draftquality/blob/master/draftquality/feature_lists/enwiki.py#L123 [13:58:41] /o\ [13:58:45] cool, training [13:59:33] 10Scoring-platform-team (Current), 10Analytics, 10EventBus, 10ORES, and 2 others: Modify revision-score schema so that model probabilities won't conflict - https://phabricator.wikimedia.org/T197000#4275600 (10Ottomata) [13:59:50] \o/ [14:00:06] We need to pull in the word2vec stuff soon :) [14:00:14] For draftquality and editquality [14:00:23] awight, want to delay 1:1? [14:00:55] eh sorry brt [14:09:48] 10Scoring-platform-team (Current), 10Analytics, 10EventBus, 10ORES, and 2 others: Modify revision-score schema so that model probabilities won't conflict - https://phabricator.wikimedia.org/T197000#4275641 (10Pchelolo) > To avoid this, can we change the schema so that scores is an object keyed by model nam... [14:38:43] 10Scoring-platform-team (Current), 10Analytics, 10EventBus, 10ORES, and 3 others: Modify revision-score schema so that model probabilities won't conflict - https://phabricator.wikimedia.org/T197000#4275722 (10Ottomata) @Ladsgroup are there any compatibility constraints between model versions? [14:56:05] halfak: I think I just delete and regenerate the datasets/en*with_cache* for this, right? [14:59:59] 10Scoring-platform-team (Current), 10Packaging, 10Epic: [Epic] Support word2vec for production ORES models - https://phabricator.wikimedia.org/T187217#4275786 (10awight) [15:00:02] 10Scoring-platform-team (Current), 10ORES, 10Patch-For-Review: Update ORES wheels for new revscoring requirements - https://phabricator.wikimedia.org/T188447#4275784 (10awight) 05Open>03Resolved [15:00:10] awight, +1 [15:00:51] 10Scoring-platform-team (Current), 10ORES, 10drafttopic-modeling, 10Patch-For-Review, 10artificial-intelligence: Deploy drafttopic model to production ORES - https://phabricator.wikimedia.org/T176336#4275799 (10awight) [15:01:06] 10Scoring-platform-team (Current), 10Packaging, 10Epic: [Epic] Support word2vec for production ORES models - https://phabricator.wikimedia.org/T187217#3968196 (10awight) 05Open>03Resolved a:03awight [15:04:46] excellent [15:10:04] (03PS3) 10Awight: [WIP] Look at page-judgment schema [extensions/JADE] - 10https://gerrit.wikimedia.org/r/439896 [15:11:45] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Look at page-judgment schema [extensions/JADE] - 10https://gerrit.wikimedia.org/r/439896 (owner: 10Awight) [15:15:58] 10Scoring-platform-team (Current), 10Wikilabels, 10articlequality-modeling, 10Patch-For-Review, 10artificial-intelligence: Train/test article quality model for euwiki - https://phabricator.wikimedia.org/T171119#4275886 (10Halfak) This model is now live! see https://ores.wikimedia.org/v3/scores/euwiki/62... [15:19:57] 10Scoring-platform-team, 10Collaboration-Team-Triage (Collab-Team-This-Quarter): Enable bswiki edit quality features - https://phabricator.wikimedia.org/T197010#4275895 (10awight) [15:20:07] 10Scoring-platform-team, 10Collaboration-Team-Triage (Collab-Team-This-Quarter): Enable bswiki edit quality features - https://phabricator.wikimedia.org/T197010#4275905 (10awight) [15:20:10] 10Scoring-platform-team (Current), 10editquality-modeling, 10Patch-For-Review, 10User-Ladsgroup, 10artificial-intelligence: Train damaging/goodfaith models for Bosnian Wikipedia - https://phabricator.wikimedia.org/T194876#4275906 (10awight) [15:20:46] 10Scoring-platform-team, 10Collaboration-Team-Triage (Collab-Team-This-Quarter): Enable srwiki edit quality features - https://phabricator.wikimedia.org/T197012#4275920 (10awight) [15:30:25] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Look at page-judgment schema [extensions/JADE] - 10https://gerrit.wikimedia.org/r/439896 (owner: 10Awight) [15:32:57] halfak: any idea how long draftquality normally takes to cv_train? [15:33:11] I needa get out of here but it's still crunching [15:33:13] Good Q. I can't remember. [15:33:20] The old model will have info on that, I think [15:33:50] I couldn't revscoring model_info with a newer revscoring :-/ -- where do you keep your venv? [15:34:06] Oh hmm. Let me check it. [15:34:16] ~/venv, it seems [15:34:44] that worked. [15:34:47] Hmm... it looks like nothing shows up in model_info [15:34:51] I'm using revscoring 2.2.5 [15:35:17] I don't see stats about how long it took to train [TODO] [15:35:31] right [15:35:32] It would be an interesting metric to include, fur shure [15:35:36] +1 [15:36:48] 10Scoring-platform-team, 10revscoring, 10artificial-intelligence: Include training performance metrics in model_info - https://phabricator.wikimedia.org/T197013#4275974 (10awight) [15:37:28] well, I'll try to log in << the deployment window and commit the draftquality model, otherwise we can just wait a day. [15:37:48] or deploy the editquality model, I s'pose [15:44:03] o/ [15:44:42] that works. [15:44:48] no huge rush on this one [15:45:13] * halfak monitors stat1006 [15:46:19] :) if you ping when the CPUs idle, I can jump on opportunistically [15:46:32] * awight goes out of focus [15:51:52] 10Scoring-platform-team, 10JADE, 10Operations, 10User-Joe: Scalability concerns creating a page per revision - https://phabricator.wikimedia.org/T196547#4276013 (10Halfak) I don't think we should be designing for the worst-case scenario here. There are many situations where content creation patterns are c... [16:00:19] Two processes left. It's winding down :) [16:00:31] ewhit_, I posted on your talk page. [16:00:41] Looks like a reasonable proposal for future recruiting messages. [16:00:50] I saw that, thanks! I responded there. I will add that to my future messages [16:01:08] I also added that the dataset this produces will be public when it's finished [17:00:59] halfak: trained! [17:01:04] I'm trying to export from stat6 now [17:01:41] kk. I'm waiting on the pull [17:02:34] how do I... [17:02:36] argh [17:02:48] I don't really want to pull and push over cell network but here goes [17:02:54] stupid firewalls [17:02:56] No push from stat1006? [17:03:05] You can set up a remotee password [17:03:23] to GitHub [17:04:10] Yeah. But I can't find it now :( [17:04:19] Also, we'll need to force diffusion to cycle. [17:04:24] So I'll prepare for that. [17:05:24] Can't find docs on my weird ass password hack [17:05:35] 10Scoring-platform-team (Current), 10Wikilabels, 10articlequality-modeling, 10Patch-For-Review, 10artificial-intelligence: Train/test article quality model for euwiki - https://phabricator.wikimedia.org/T171119#4276445 (10Theklan) :) Great! So, now that we have this system... which would be the possibi... [17:06:27] https://help.github.com/articles/creating-a-personal-access-token-for-the-command-line/ [17:06:29] Aha! [17:06:58] https://github.com/settings/tokens [17:06:58] ty, bookmarking [17:13:02] (03PS3) 10Awight: Update English models with new badwords [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/439839 (https://phabricator.wikimedia.org/T196468) [17:13:21] halfak: I think everything is uploaded now. I'm gonna put the computer under the table for a minute, pls ping here if you need anything [17:13:32] kk [17:13:34] looking for PRs [17:13:58] got it [17:14:39] Durn kids have been stealing all my poffertjes [17:15:14] 10Scoring-platform-team (Current), 10Wikilabels, 10articlequality-modeling, 10Patch-For-Review, 10artificial-intelligence: Train/test article quality model for euwiki - https://phabricator.wikimedia.org/T171119#4276483 (10Theklan) Also, is it possible to integrate this results into the Outreach Dashboard... [17:15:44] Curse you diffusion! [17:16:19] hargh [17:16:28] yeah this is 11th hour [17:17:14] * halfak forces update [17:17:22] I think we should go to beta today and try again tomorrow :) [17:18:04] +1 [17:19:12] force update won't work :) [17:19:14] * halfak waits a good wait [17:19:17] it's parsing over 8 million refs [17:20:24] what [17:23:28] awight with a gerrit update it moved content from the db into git commits under refs/changes/*/*/meta [17:23:35] which is quite alot [17:23:48] we have patched phab to ignore those refs (any more thats cloned) [17:23:54] but existing ones will be parsed [17:24:03] hargh. [17:24:07] Done! [17:24:10] Well updated.. [17:24:56] Arg! awight we forgot to update the revscoring wheel! [17:25:12] no I got that [17:25:23] check the parent commits in ores-deploy [17:27:04] (03PS1) 10Halfak: Bumps revscoring to 2.2.5 [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/439983 [17:27:24] Oh? It doesn't look like it made it into wheels though :| [17:27:27] https://gerrit.wikimedia.org/r/#/c/research/ores/wheels/+/439983 [17:27:50] Did I miss something? Looked like I just replaced 2.2.4 with 2.2.5 in master of our wheels repo [17:28:15] awight, ^ [17:28:35] 439983 master Bumps revscoring to 2.2.5 [17:28:35] 439785 master New wheel for revscoring 2.2.5; update libraries [17:29:24] Where are you copying that from? [17:29:30] https://phabricator.wikimedia.org/source/ores-deploy/ has none of that [17:29:31] git review -l [17:29:38] this in the wheels repo [17:29:54] here's the ores-deploy patch [17:29:54] 439790 master Bump wheels, including revscoring 2.2.5; fix articlequality typo [17:30:03] it's a parent of the one we're working on [17:30:16] Yes. Can you link? [17:30:22] "parent"? [17:30:33] it's all linked in the task [17:30:42] * halfak looks for a task [17:30:42] https://phabricator.wikimedia.org/T196468#4275549 [17:32:06] awight, I don't see a change to the wheels repo though. :| [17:32:33] This one bumps the submodule pointer, https://gerrit.wikimedia.org/r/#/c/mediawiki/services/ores/deploy/+/439790/ [17:33:05] This is the wheels commit, https://gerrit.wikimedia.org/r/#/c/research/ores/wheels/+/439785/ [17:33:10] ok I gtg if we're not deploying [17:33:32] Ahh. yeah. I didn't see that one linked. [17:33:50] hehe probably not [17:33:57] (03CR) 10Halfak: [C: 032] New wheel for revscoring 2.2.5; update libraries [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/439785 (owner: 10Awight) [17:34:03] yeah it's hard to line up a deployment then hand it off, sorry [17:34:04] (03CR) 10Halfak: [V: 032 C: 032] New wheel for revscoring 2.2.5; update libraries [research/ores/wheels] - 10https://gerrit.wikimedia.org/r/439785 (owner: 10Awight) [17:35:00] (03CR) 10Halfak: [V: 032 C: 032] Bump wheels, including revscoring 2.2.5; fix articlequality typo [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/439790 (owner: 10Awight) [17:35:30] (03CR) 10Halfak: [V: 032 C: 032] Update English models with new badwords [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/439839 (https://phabricator.wikimedia.org/T196468) (owner: 10Awight) [17:35:39] You'll need https://gerrit.wikimedia.org/r/#/c/mediawiki/services/ores/deploy/+/439789/ [17:35:48] just cos I haven't rebased [17:35:53] (03CR) 10Halfak: [V: 032 C: 032] Fix Makefile for wheel edge cases (031 comment) [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/439789 (owner: 10Awight) [17:36:56] (03CR) 10Awight: Fix Makefile for wheel edge cases (031 comment) [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/439789 (owner: 10Awight) [17:38:49] OK all looks good. Going to beta [17:42:20] :) will log in again in 2-3hr [17:49:14] it works. [17:49:17] AFK for a bit [18:31:18] 10Scoring-platform-team (Current), 10Analytics, 10Analytics-Kanban, 10EventBus, and 4 others: Modify revision-score schema so that model probabilities won't conflict - https://phabricator.wikimedia.org/T197000#4276662 (10Ottomata) [18:31:42] 10Scoring-platform-team (Current), 10Analytics, 10Analytics-Kanban, 10EventBus, and 4 others: Modify revision-score schema so that model probabilities won't conflict - https://phabricator.wikimedia.org/T197000#4275600 (10Ottomata) a:05Ladsgroup>03Ottomata [18:57:23] o/ [18:57:25] * halfak pants [18:57:47] got my bike into the shop (new wheels + servicing old wheels) and got lunch [19:03:23] 10Scoring-platform-team (Current), 10editquality-modeling, 10revscoring, 10Patch-For-Review, 10artificial-intelligence: Catch a specific new badword in English - https://phabricator.wikimedia.org/T196468#4276803 (10Halfak) Live in beta. Minor improvements to some models :) [19:44:55] whew. [19:48:53] I bet that gravel was murder on the wheels [20:05:09] 10Scoring-platform-team (Current), 10Bad-Words-Detection-System, 10revscoring, 10User-Ladsgroup, 10artificial-intelligence: Add language support for Serbian - https://phabricator.wikimedia.org/T174687#4277003 (10Acamicamacaraca) Good for now... Models are added. [20:08:28] 10Scoring-platform-team, 10Collaboration-Team-Triage (Collab-Team-This-Quarter): Enable srwiki edit quality features - https://phabricator.wikimedia.org/T197012#4277017 (10awight) [20:08:31] 10Scoring-platform-team (Current), 10editquality-modeling, 10Patch-For-Review, 10User-Zoranzoki21, 10artificial-intelligence: Train / test reverted model for srwiki - https://phabricator.wikimedia.org/T194745#4277018 (10awight) [20:08:44] 10Scoring-platform-team (Current), 10Bad-Words-Detection-System, 10revscoring, 10User-Ladsgroup, 10artificial-intelligence: Add language support for Serbian - https://phabricator.wikimedia.org/T174687#4277019 (10awight) 05Open>03Resolved [20:08:46] 10Scoring-platform-team (Current), 10editquality-modeling, 10Patch-For-Review, 10User-Zoranzoki21, 10artificial-intelligence: Train / test reverted model for srwiki - https://phabricator.wikimedia.org/T194745#4207231 (10awight) [20:18:01] 10Scoring-platform-team (Current), 10Bad-Words-Detection-System, 10revscoring, 10User-Ladsgroup, 10artificial-intelligence: Add language support for Serbian - https://phabricator.wikimedia.org/T174687#4277027 (10Acamicamacaraca) @awight Do we have to wait a new MediaWiki version until ORES appear, or? [20:45:13] 10Scoring-platform-team, 10Collaboration-Team-Triage (Collab-Team-This-Quarter): Enable srwiki edit quality features - https://phabricator.wikimedia.org/T197012#4275920 (10Halfak) The model is now deployed. @Acamicamacaraca has been asking (at T174687) when we can get the filters enabled. [20:46:04] 10Scoring-platform-team (Current), 10Bad-Words-Detection-System, 10revscoring, 10User-Ladsgroup, 10artificial-intelligence: Add language support for Serbian - https://phabricator.wikimedia.org/T174687#3570233 (10Halfak) I just pinged the #global-collaboration team in T192012 about this. They'll need to... [20:46:26] 10Scoring-platform-team, 10Collaboration-Team-Triage (Collab-Team-This-Quarter): Enable srwiki edit quality filters in RecentChanges - https://phabricator.wikimedia.org/T197012#4277135 (10Halfak) [20:47:48] 10Scoring-platform-team, 10Collaboration-Team-Triage (Collab-Team-This-Quarter): Enable bswiki edit quality filters in RecentChanges - https://phabricator.wikimedia.org/T197040#4277148 (10Halfak) [20:48:24] 10Scoring-platform-team, 10Collaboration-Team-Triage (Collab-Team-This-Quarter): Enable bswiki edit quality filters in RecentChanges - https://phabricator.wikimedia.org/T197040#4277139 (10Halfak) @jmatazzoni & @Catrope, we've got this model deployed and it is ready to enable in RecentChanges [20:52:09] 10Scoring-platform-team (Current), 10Wikilabels, 10articlequality-modeling, 10Patch-For-Review, 10artificial-intelligence: Train/test article quality model for euwiki - https://phabricator.wikimedia.org/T171119#4277161 (10Halfak) @Ragesoss ^ [21:01:40] 10Scoring-platform-team (Current), 10Wikilabels, 10articlequality-modeling, 10Patch-For-Review, 10artificial-intelligence: Train/test article quality model for euwiki - https://phabricator.wikimedia.org/T171119#4277181 (10Ragesoss) Thanks for the ping. @Theklan, I've filed an issue for it: https://github... [21:34:58] OK I think that's good for today. I'll be hacking on vision statements tomorrow! [21:35:03] plan the plan to plan [21:35:05] ! [21:35:07] o/