[00:00:53] logs are clean, job failure ratio is clean, yay [00:01:05] moving forward to delete lots of code :D [00:23:32] (03PS1) 10Ladsgroup: Clean up old config style [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421196 (https://phabricator.wikimedia.org/T189948) [00:26:03] (03CR) 10jerkins-bot: [V: 04-1] Clean up old config style [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421196 (https://phabricator.wikimedia.org/T189948) (owner: 10Ladsgroup) [00:26:39] (03PS2) 10Ladsgroup: Clean up old config style [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421196 (https://phabricator.wikimedia.org/T189948) [00:29:10] (03CR) 10jerkins-bot: [V: 04-1] Clean up old config style [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421196 (https://phabricator.wikimedia.org/T189948) (owner: 10Ladsgroup) [00:29:28] (03CR) 10Ladsgroup: "recheck" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421196 (https://phabricator.wikimedia.org/T189948) (owner: 10Ladsgroup) [00:46:36] (03PS1) 10Ladsgroup: Move tests covering PreferencesHookHandler to a dedicated file [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421200 [00:49:29] (03CR) 10jerkins-bot: [V: 04-1] Move tests covering PreferencesHookHandler to a dedicated file [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421200 (owner: 10Ladsgroup) [01:02:23] (03PS2) 10Ladsgroup: Move tests covering PreferencesHookHandler to a dedicated file [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421200 [01:08:50] (03CR) 10jerkins-bot: [V: 04-1] Move tests covering PreferencesHookHandler to a dedicated file [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421200 (owner: 10Ladsgroup) [01:09:42] (03PS3) 10Ladsgroup: Move tests covering PreferencesHookHandler to a dedicated file [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421200 [01:13:21] (03CR) 10Krinkle: Move tests covering PreferencesHookHandler to a dedicated file (031 comment) [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421200 (owner: 10Ladsgroup) [01:17:53] (03PS4) 10Ladsgroup: Move tests covering PreferencesHookHandler to a dedicated file [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421200 [01:17:59] (03CR) 10Ladsgroup: "Oh, Thanks :)" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421200 (owner: 10Ladsgroup) [01:20:03] 10Scoring-platform-team (Current), 10editquality-modeling, 10Epic, 10User-Ladsgroup, 10artificial-intelligence: [Epic] Implement code generation for model makefile maintenance - https://phabricator.wikimedia.org/T168455#4071512 (10Ladsgroup) At this state, I would consider it done, anything after this, i... [01:28:46] (03CR) 10Krinkle: [C: 032] Move tests covering PreferencesHookHandler to a dedicated file [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421200 (owner: 10Ladsgroup) [01:32:13] (03Merged) 10jenkins-bot: Move tests covering PreferencesHookHandler to a dedicated file [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421200 (owner: 10Ladsgroup) [01:34:22] (03CR) 10jenkins-bot: Move tests covering PreferencesHookHandler to a dedicated file [extensions/ORES] - 10https://gerrit.wikimedia.org/r/421200 (owner: 10Ladsgroup) [14:23:11] 10Scoring-platform-team, 10ORES, 10Documentation: Document ORES from a high level - https://phabricator.wikimedia.org/T190411#4072795 (10Halfak) [14:24:55] 10Scoring-platform-team, 10ORES, 10Documentation: ELI5 ORES docs - https://phabricator.wikimedia.org/T190411#4072806 (10Halfak) [14:27:46] 10Scoring-platform-team, 10ORES, 10Documentation: ELI5 ORES docs - https://phabricator.wikimedia.org/T190411#4072812 (10Halfak) [14:28:23] 10Scoring-platform-team, 10ORES, 10Documentation: ELI5 ORES docs - https://phabricator.wikimedia.org/T190411#4072795 (10Halfak) [14:29:15] o/ Amir1 [14:29:25] o/ [14:29:32] Congrats on the deploy yesterday. I got distracted with chores [14:30:20] yay, Now I made a patch that removes around 100 lines from the extension! [14:31:28] bw. I wanted to talk to you about T189954 [14:31:29] T189954: Build Scoring platform community monitor - https://phabricator.wikimedia.org/T189954 [14:31:43] I need to grab lunch but after that let's sit and talk [14:32:38] OK sounds good [14:35:10] 10Scoring-platform-team (Current), 10User-Ladsgroup: Build Scoring platform community monitor - https://phabricator.wikimedia.org/T189954#4058771 (10Halfak) Agreed. This is related to {T140364}. We should have a UI that is rendered based on queries to ORES and queries to Wikilabels. We can worry about rende... [14:44:46] awight is commenting on git stuff but he's not online [14:45:14] * halfak does an awight summoning spell [14:49:11] Bad news, my laptop is not charging this morning. [14:49:18] Seeing if a reboot helps... [14:55:28] Oh noo [14:55:56] Good thing you're traveling back to the land of fast-shipping-from-SF soon. [15:03:05] Yeah this is not gonna be fun [15:03:27] I'm keeping scope small today, gonna write a quick test for codegen [15:05:04] OK sounds good. [15:05:18] awight, wasn't sure if I should merge what you have or leave it for you to do that test. [15:05:45] Up to you, I'm fine with the change since it hasn't emitted smoke [15:06:01] BTW, I was still thinking that having a codegen.load_template("Makefile.j2", template_dir="templates/") call would be good. [15:06:10] Woos [15:06:23] That's still not right because it would use the filename [15:06:26] crud—I was planning to deploy the venv change today. [15:06:34] Probably not a good idea with declining battery. [15:06:41] So reboot didn't work? [15:06:44] nope [15:06:52] I have a ticket out to techsupport, but it's early there. [15:06:56] Do you think you could get some service today? [15:07:02] Like, some local shop? [15:07:28] The problem is... this thing has two USB-C ports for power in. One mysteriously burned out last December, and now the other is behaving the same way. [15:07:46] Yeah good call, there are some highly skilled shops in town. [15:08:30] I guess I have to plan to not deploy. [15:09:30] I think so. [15:09:42] Frustrating, I can imagine [15:10:04] Honestly, it's fantastic timing [15:10:33] I'll only miss one scheduled work day, and was planning to fly straight to a pile of laptops either way. [15:13:31] lol [15:13:46] yeah. Glad this didn't happen right in the middle of your time in Peru [15:20:01] wiki-ai/editquality#233 (isolate_codegen - 921d508 : Adam Wight): The build was broken. https://travis-ci.org/wiki-ai/editquality/builds/356928334 [15:28:04] wiki-ai/editquality#235 (isolate_codegen - 6b002be : Adam Wight): The build was fixed. https://travis-ci.org/wiki-ai/editquality/builds/356931812 [15:29:48] halfak: Be aware that the ores deployment branch has a breaking change merged to it. [15:30:03] uhoh. What's the status of that? [15:30:18] The venv stuff got merged a bit earlier than I'd expected, which was kind of okay when I was going to deploy today, but not okay over the weekend. [15:30:27] Let me revert, on second thought. [15:30:58] (03PS1) 10Awight: Revert "Build venv into deployed source dir" [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/421316 [15:31:06] (03CR) 10Awight: [V: 032 C: 032] Revert "Build venv into deployed source dir" [services/ores/deploy] - 10https://gerrit.wikimedia.org/r/421316 (owner: 10Awight) [15:31:16] et voila. [15:34:39] wiki-ai/editquality#238 (isolate_codegen - bc08124 : Adam Wight): The build was broken. https://travis-ci.org/wiki-ai/editquality/builds/356935099 [15:39:31] wiki-ai/editquality#239 (isolate_codegen - dc55e38 : Adam Wight): The build was fixed. https://travis-ci.org/wiki-ai/editquality/builds/356938871 [15:50:24] wiki-ai/editquality#241 (isolate_codegen - 6d7fdc8 : Adam Wight): The build was broken. https://travis-ci.org/wiki-ai/editquality/builds/356943331 [15:55:08] 10Scoring-platform-team (Current): Investigate runtime of tune with high number of estimators - https://phabricator.wikimedia.org/T190288#4073112 (10Sumit) [15:56:11] noisy [15:56:20] I think that's it for tests. [15:56:33] halfak: training with 200 estimators took 180 min in user space and 143min in real time with 2 folds, so thats our upper limit [15:56:49] cool! That's do-able. [15:57:04] What's the fitness look like? [15:57:04] Diminishing returns? [15:57:17] I've recorded in the task - pr_auc (micro=0.815, macro=0.783) [15:57:54] because of the large output from each run I had data only till 150 estimators in the tmux session [15:58:10] but it looked like the pr_auc for 150,175,200 estimators was pretty close [16:01:31] cool. [16:04:13] If it's close, we know that 150 estimators is probably OK [16:04:25] How long did the 150 estimator training take? [16:04:28] codezee, ^ [16:04:42] * halfak is in a meeting and reviewing your work on the side. [16:07:56] yeah 150 seems about right [16:08:15] but i've started another run, saving all output in a file so later i can extract the useful parts from it [16:08:47] 10Scoring-platform-team (Current), 10Operations, 10Release-Engineering-Team (Watching / External), 10Wikimedia-Incident: Cache ORES virtualenv within versioned source - https://phabricator.wikimedia.org/T181071#4073147 (10awight) This has been reverted in https://gerrit.wikimedia.org/r/#/c/421316/, since I... [16:19:06] I have to turn off in order to have power for a final backup [16:19:10] wheee! [16:19:15] off I go [17:11:51] LUNCH! [18:51:19] Well that took a bit longer than expected. [18:51:28] Looks like I'm staying a bit later today ^_^ [19:22:47] o/ [19:25:15] o/ awight [19:25:18] success? [19:26:00] nope [19:26:10] I'm on a backup laptop, win10 [19:26:58] The apple shop needed a firmware password that I don't have, and regardless would take 3 business days to fix anything [19:28:42] Gotcha. Glad you have a sort-of backup :) [19:28:51] https://www.mediawiki.org/wiki/JADE/Intro_blog/Short_story [19:28:54] ^ BTW [19:29:01] I think that's starting to feel like a blog post. [19:29:50] I don't know why I called it "Short story" but there it is :) [19:33:49] 10Scoring-platform-team (Current), 10articlequality-modeling, 10draftquality-modeling, 10editquality-modeling, 10artificial-intelligence: Re-train models with revscoring 2.2.0 - https://phabricator.wikimedia.org/T188775#4073889 (10Halfak) I think I'm going to wait until we finish T186453, T163006, and T1... [19:50:30] It reads well, nice start! [19:51:52] awight there's a firmewhere password [19:51:53] ? [20:00:27] "windows subsystem for linux" ftw! [20:03:10] awight yeh :) [20:03:17] you can run debian and ubuntu at the same time [20:04:11] * awight shudders at the thought [20:06:15] awight it should be safe [20:06:22] awight but remeber no rm -rf / [20:07:05] um, yes but I already have one computer on cinder blocks. No need to do donuts in the parking lot with the other one. [20:07:17] ^ lol [20:07:54] awight, let me know if you want to go back to outline. It helps to write things out, but I think it's always easier to iterate on an outline. [20:08:25] lol [20:10:08] halfak: I haven't put much thought into it yet, but I think we're okay producing multiple outputs. Your blog post is great, I'll try to review for anything itmissing and will [20:10:20] *anything it's missing, and yours should become the blog post [20:10:34] I can keep working on the presentation for tool authors, for example... [20:12:16] OK sounds good. Let me know your notes when you get to it :) [21:19:04] halfak: I read through the huwiki PR and Phabricator tasks, and still don't understand what makes it unique. [21:21:34] fyi, I'm inviting tgr to the GitHub wiki-ai org. [21:24:26] awight, the "needs_review" bit is missing from wikilabels [21:24:30] +1 [21:28:42] halfak: oh cool, that does make it a perfect candidate for the new merge_labels [21:29:05] Right :) [21:29:17] It's the reason I started working on intersect_merge_observations. [21:44:34] Nettrom, I'm working with hall1467 on an analysis that uses your perfect alignment hypothesis. [21:44:52] I was wondering if you considered developing a scalar distance metric [21:45:04] E.g. comparing the quantile of quality to the quantile of view-rate directly. [21:45:24] So you could say if an article was a half-class off. Or maybe even a 10th of a class off. [21:45:51] I supposed we didn't have the weighted sum measure at that time, but I wonder if there was another reason. [21:47:13] the WP analysis approaches it from an ordinal category perspective, which leans into a confusion matrix and subsequent analysis of distance based on that [21:47:32] it has several drawbacks, e.g. that the categories are of vastly different size [21:48:19] but as you mention, we didn’t have the weighed sum at the time, but we did discuss approaches similar to it [21:48:37] the problem was to find a solid theoretical foundation for using the weighed sum, and I could never find [21:48:41] *find it [21:49:30] once you move into contiunous space, quantiles become useful [21:49:53] alignment also becomes a linear regression, so you can look at residuals [21:50:07] which is what we did for OSM, except the regression is spatial rather than OLS [21:50:31] 10Scoring-platform-team (Current), 10ORES, 10articlequality-modeling, 10draftquality-modeling, and 2 others: Discuss surfacing ORES for AFC/NPP - https://phabricator.wikimedia.org/T190464#4074346 (10Halfak) [21:51:09] actually, one of the challenges with being in continuous space is to define how far “far enough” is, i.e. when is an article misaligned [21:51:56] we had to define that for OSM, so that work (which needs to be worked on and published) does an analysis of the residuals to enable us to make a judgement [21:57:20] Nettrom, squared distance? [21:57:24] Ohh [21:57:26] With hall? [21:57:30] Or someone else [21:57:41] the OSM work, you mean? [22:00:07] Where you compare residuals. [22:03:07] it’s complicated :) we calculate the percentage difference of the model’s estimate and the true value, find that these are log-normal and use standard deviations, and then find that two standard deviations is “large enough" [22:03:22] IIRC, I’ll have to go look at my thesis to confirm [22:06:22] Nettrom, I'm confused. Did you compare residuals between perfect alignment and actual quality in "the OSM work"? [22:07:40] halfak: yeah, we did [22:08:42] we just convert them to percentages because of the large scale differences in that dataset, in WP I suspect you won’t have that because the weighed sum is [0,5] [22:16:41] halfak: fyi, my temporary environment is sort-of okay. I can run pytest on our repos and stuff... So there's a good chance I'll work Monday :) [22:21:54] Nettrom, working in wikidata with the itemquality model. [22:21:59] Is this OSM paper published? [22:22:13] awight, good to know. :) [22:22:19] halfak: no, it’s in my queue of work to do… it’s in my thesis, though [22:22:24] If not, I was gonna make you be the primary reviewer for the day :P [22:22:46] OK. What should I have hall1467 look at specifically? Is there a chapter number? [22:22:55] gimme a sec [22:24:08] note to self: make your PhD thesis easily retrievable [22:25:27] chapter 5.5 [22:25:42] lol [22:27:23] \o/ [22:27:55] it’s also fun to check the download statistics of your thesis [22:28:00] I get about 30 a month :D [22:30:21] woah [22:35:16] Spiders from Mars? [22:42:05] o/ hall1467 [22:44:17] halfak: hey [22:46:46] 10Scoring-platform-team, 10articlequality-modeling, 10draftquality-modeling, 10drafttopic-modeling, 10artificial-intelligence: Read ORES predictions for article quality, draft quality, and draft topic into MediaWiki - https://phabricator.wikimedia.org/T190471#4074496 (10Halfak) [22:47:10] 10Scoring-platform-team (Current), 10ORES, 10articlequality-modeling, 10draftquality-modeling, and 2 others: Discuss surfacing ORES for AFC/NPP - https://phabricator.wikimedia.org/T190464#4074508 (10Halfak) [22:47:12] 10Scoring-platform-team, 10articlequality-modeling, 10draftquality-modeling, 10drafttopic-modeling, 10artificial-intelligence: Read ORES predictions for article quality, draft quality, and draft topic into MediaWiki - https://phabricator.wikimedia.org/T190471#4074507 (10Halfak) [22:47:34] 10Scoring-platform-team, 10articlequality-modeling, 10draftquality-modeling, 10drafttopic-modeling, 10artificial-intelligence: Read ORES predictions for article quality, draft quality, and draft topic into MediaWiki - https://phabricator.wikimedia.org/T190471#4074496 (10Halfak) [22:47:36] 10Scoring-platform-team (Current), 10MediaWiki-extensions-ORES, 10MW-1.31-release-notes (WMF-deploy-2018-03-20 (1.31.0-wmf.26)), 10Patch-For-Review, 10User-Ladsgroup: Store wp10 predictions in the MediaWiki database. - https://phabricator.wikimedia.org/T175757#4074511 (10Halfak) [22:47:45] 10Scoring-platform-team, 10articlequality-modeling, 10draftquality-modeling, 10drafttopic-modeling, 10artificial-intelligence: Store ORES predictions for article quality, draft quality, and draft topic in MediaWiki - https://phabricator.wikimedia.org/T190471#4074496 (10Halfak) [22:49:04] awight, do we have a phab task for the ORES-JADE connector? [22:59:59] https://phabricator.wikimedia.org/T190472 [23:07:48] I'm heading out for the day. [23:08:08] I'm AFK tomorrow for bike race prep :)