[00:49:30] (03PS1) 10Reedy: s/ApiQueryContributions/ApiQueryUserContributions/ [extensions/ORES] - 10https://gerrit.wikimedia.org/r/435071 [00:52:57] (03CR) 10jerkins-bot: [V: 04-1] s/ApiQueryContributions/ApiQueryUserContributions/ [extensions/ORES] - 10https://gerrit.wikimedia.org/r/435071 (owner: 10Reedy) [00:54:36] (03CR) 10Reedy: "recheck" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/435071 (owner: 10Reedy) [01:10:23] (03CR) 10Krinkle: [C: 031] s/ApiQueryContributions/ApiQueryUserContributions/ [extensions/ORES] - 10https://gerrit.wikimedia.org/r/435071 (owner: 10Reedy) [13:49:32] o/ [13:49:38] I'm back [13:49:41] Hey awight [13:50:05] halfak: o/ How's it going? [13:50:27] It's going OK. Fun story, I realized last night that I broke a rib in my crash. [13:50:43] Heh. All the other pain calmed down enough that I can feel it. [13:50:51] that's insane [13:51:04] So the adamantium is corroding [13:51:10] lol [13:51:29] * awight fumbles for the green sauce [13:51:37] My doc told me that he thinks I have a very high threshold for pain. I was skeptical of that at first but maybe it seems like I have a high threshold because the system was just overloaded! [13:52:16] You could make a heap of cash if you feel like surrogate pregnancy [13:52:16] Anyway, my shoulder is doing pretty good. I got authorization to ditch the sling on Wednesday :) [13:52:21] lol [13:52:30] Painful and lucrative! [13:53:16] Well, I'm not blocked on this but if you feel like taking a spin, https://adamwight.github.io/ores-reference-ui/dist/ [13:53:43] I want to apply to wikilabels, but think I should try to get feedback from JS heads first. [14:06:29] I like the UI. I expecially like experimenting with the visualization. [14:07:13] I'm really bothered by the two checkboxes for info, which don't work together [14:07:20] Can you have the probability rendered in the plot? [14:07:29] which plot? [14:07:34] Right. Hmm. You can make a second request. [14:07:37] I did render score probabilities, which looks silly [14:07:43] Oh the prediction probability plot [14:08:12] ah, the numbers? yeah chart.js turns out to be too simple for us, I had to abuse it severely to get the confusion matrices [14:08:31] those are a stacked bar chart, and the internals are groaning [14:09:06] We chould probably switch to a library that exposes d3 customizability, so we can get fancy [14:09:22] Let's split the reference UI and model-info, I think. [14:09:39] It is kind of confusing to have them show up like this. [14:10:03] +1 for a more flexible plotting library. [14:10:18] +1 to splitting, but I was thinking of tabs rather than a separate UI [14:10:18] Oh say, what do you think about having a bit of config for how to represent the scores. [14:10:25] interesting [14:10:30] like, include thresholds? [14:10:30] Tabs sounds interesting to me. [14:10:56] awight, for config, maybe "damaging": true = green and "damaging": false = red. [14:10:59] Or something like that [14:11:00] It feels like thresholds should be demonstrated right next to scores, to make the point that you shouldn't use one without the other [14:11:04] aha yeah let's do that. [14:11:09] Maybe with wp10, include a weighted_sum score. [14:11:11] Also, wp10 ordering is annoying as shit [14:11:14] ooh nice [14:11:20] Models know the class ordering :) [14:11:24] It should be in model-info [14:11:28] oh? excellent [14:12:20] see "params" --> "labels" :) [14:12:54] I have a lot of directions to go in, but should probably limit scope to smaller iterations... [14:14:17] more thoughts, https://phabricator.wikimedia.org/T195274#4223336 [14:17:53] 10Scoring-platform-team (Current), 10ORES, 10Wikimedia-Hackathon-2018: Rewrite ORES "reference" UI using React - https://phabricator.wikimedia.org/T195274#4231591 (10awight) More TODOs, * wp10 categories are already ordered in "labels" response, use that to order confusion matrix and prediction bar graph. *... [14:24:47] awight, what do you think about focusing on getting draft topic in prod today? [14:25:11] I can spend some time iterating on design, tasks, etc. for reference-ui and model-info so you can hit the ground running next time. [14:25:26] I feel like those two things are firmly inside of my PM head-space. [14:26:14] Friday? [14:27:02] Cool, a bit of reference UI attention would be fantastic. One problem I'm thinking about however, is that it's already to heavyweight to be reusable. [14:27:29] As small and reusable as the code is, the compiled, minimized thing is like 700kB of ECMAScript, which is not to be envied. [14:27:36] it's like half the size of VisualEditor. [14:30:57] awight, fair point. Let's focus on minimizing functionality per-page and see if we can get the size down a bit. [14:31:09] It's the libraries... [14:31:14] In the end, I don't think it's crazy big especially since it'll be cached after the first load. [14:31:15] Right. [14:31:19] Maybe we're not eliminating dead code, though. [14:32:18] Just thinking that maybe we can get away without some functionality (i.e. libraries) in each UI. [14:32:19] I think this is the issue, https://github.com/parcel-bundler/parcel/issues/392 [14:32:24] probably not. [14:32:38] err that was a response to !-2 comment [14:32:40] not to myself. [14:33:01] But I think the problem might be our packager, it's the "fast and easy" one, not a real production solution. [14:33:16] Ahh... Either way, it's OK if we're not super performant. [14:33:19] seems to be lacking dead code elimination because it's overly parellelized [14:33:26] It's just important that most people can use what we have. [14:33:31] Are you minifying? [14:33:37] That can make a big difference. [14:33:37] +1 for now, I just want to be sure that our code can be reused [14:33:43] yah it's minified. horrifying, eh? [14:33:58] ls -1 node_modules/ | wc -l [14:33:59] Ahh. That's a different goal. Or are you thinking about just internal reuse? [14:34:00] 665 [14:34:30] well the React rewrite is very nice for our maintenance purposes, so not a loss, but I was thinking external reuse in this case. [14:38:24] Reading through the parcel issue, it might even support the optimization we need soon, there are patchsets and active discussion. [14:38:52] We should have a larger discussion about external re-use. E.g. is there something we can tie into for that? [14:39:07] Like, are there other, similar projects that package up some JS for reuse? [14:39:14] We should learn from their success/failures. [14:39:40] Specifically looking for projects that are primarily a service and secondarily, some resources for using that service. [14:41:25] Good point, and if anyone wants to reuse our threshold graph for instance, they're going to be a unicorn and will probably have their own choice of arcane platform anyway. [14:42:54] Right. Maybe we can have a service endpoint that renders a graph. Hmm. [14:43:13] That might prove useful for tools that just want something to work as easily as possible. [14:43:30] interesting [14:46:45] Either way, I'll get to work on spec'ing some of this out and reviewing what you have. [14:46:56] What are the next steps for draft-topic in prod? [14:46:59] ty [14:47:10] It's really just attempting to deploy it [14:47:28] Oh I suppose we don't have a window today. [14:47:31] I want to do a limited deployment to one prod machine, to make sure that "git lfs install" is happening [14:47:34] exactly. [14:47:43] right. I wonder if that would be acceptable. [14:47:54] ah one more rather important thing. [14:48:09] ores1001 was unhealthy after my last trial deployment [14:48:20] I should investigate the exact timing [14:48:40] I can't explain, cos it continued to behave badly even after rollback [14:51:38] Ahh yeah. seems like we can investigate that first. Would you add some tasks for that? [14:51:56] I'm thinking: investigate ores1001 weirdness --> trial deployment --> full deployment [14:52:13] yup that seems like the plan [14:53:48] OK cool. I'm going to churn through email and reviews for a little while and then I'll get to tasks. [14:53:58] and design for model_info/reference_ui [14:54:30] I might include a quick user study with some known users of the UI so that we can check with them that we didn't break anything important. [14:55:52] E.g. Andrew Li uses it in his classes :) [14:56:03] 10Scoring-platform-team (Current), 10ORES, 10drafttopic-modeling: Investigate memory issues surrounding trial drafttopic deployment - https://phabricator.wikimedia.org/T195593#4231698 (10awight) p:05Triage>03Normal [14:56:10] oh nice! [14:56:23] brb [14:56:31] yeah another fun detail is how to integrate this ReactJS work into ORES. [14:56:57] Should we continue to embed, maybe as a submodule, or provide it as a separate service? It can be hosted statically, obviously. [15:17:08] awight, my sense is that we should have a very limited reference UI (like this) as part of the ORES service. [15:17:28] And that we should separate anything *interesting* as a separate system. [15:18:17] Why would we ship a UI at all, though? The only reason I can imagine is so that rogue third-party installs include a UI? [15:18:31] but then they would probably want the interesting bits... [15:18:46] e.g. to self-diagnose model health and stuff [15:20:52] awight, my thought is that ORES should be self-explanatory in a minimal way. So, having a UI to make simple requests and to report model info fits into that. [15:21:30] Re. diagnosing non-trivial model issues, I think that's going to be complicated enough for an external UI. [15:21:44] trivial == obvious from fitness stats. [15:23:34] Looks like srwiki is another case where the goodfaith model doesn't have good fitness. [15:23:37] That seems so unlikely. [15:23:43] I'm confused by how this happens. [15:24:09] good-faith/bad-faith should be an easier problem than damaging/not-damaging. [15:24:36] Maybe POV-pushing is more common. [15:27:14] It's not clear to me why we would make people go somewhere else to get visualizations, if we're offering the data in the API, we should either include interpretation aids for all the data or none. [15:27:27] Luckily, it'll be really easy to split if we have to, cos everything is packaged as a component. [15:27:49] awight, it's a lot to maintain. Part of the reason for making a service is to not make a monolith. [15:28:23] +1 to the sentiment that the default UI should be minimal though, so it's really clear that we * score * revisions for a * wiki [15:28:33] +1 [15:28:46] I think model_info being a bit more complicated makes more sense though. [15:29:07] reference UI should do one thing and only one thing. It would be cool if model-info did a couple of things. [15:29:13] What are your thoughts about not including any UI? [15:29:36] I don't like it. We've not been burnt by having the UI we have now, so I don't see why we'd drop it. [15:30:00] just for the reasons above, that we're shipping an API and not a UI [15:30:19] Right, so, let's drop swagger then too :P [15:30:29] And the ORES main page :) [15:30:32] it's quite weird to me that we would host a UI at the same base URI, I can't think of any other examples [15:30:35] hmm [15:30:37] maybe :) [15:30:38] I think there's a few bits of UI that pair nicely. [15:30:43] seiously [15:30:56] * awight squints [15:31:08] I almost got every vowel there [15:31:13] saeiously [15:31:18] lol [15:31:22] Home page, API docs, reference UI, model information. [15:32:17] ah--and this is annoying of me, but what if we package everything non-API as a static directory... [15:32:30] so you're free to double-click on "docs/index.html" but it's not part of the service [15:33:14] * awight meanwhile, rolls in the mud grappling with webpack and babel [15:33:44] awight, I'm confused because it seems that this already happens to a large extent from wsgi/static and wsgi/templates [15:34:10] I think swagger is served live? [15:34:16] not a big deal, it just feels icky [15:35:35] Right. Swagger is almost all static. There's a bit of config that is dynamically set. [15:37:28] o/ MtDu [15:37:47] Hey-o! [15:39:33] awight and I were just discussing what bits of UI belong within ORES as opposed to as external systems. [15:39:37] Backing up a step, the React stuff is gross to build, having a nodejs install is a much different skillset than having a working python+pip+virtualenv. However, I'm including the compiled .js and .css so if we pull in the repo as a submodule, it can be served statically at no cost, regardless of how much flair we staple on. [15:39:57] It's a bit like automake + gcc, everyone can ./configure, but we don't want to ask people to autoconf [15:41:16] awight, we could package up JS in the repo. I wonder if licensing would allow that. [15:41:35] It's been a bit of a pain for wikilabels/reference-ui but not crazy bad. [15:41:41] I hadn't thought of that, wow [15:44:33] We have a few external modules that we import via pip for managing big JS blobs. [15:51:38] Oh. Looks like mtdu had to run. [15:58:10] 10Scoring-platform-team (Current), 10MediaWiki-extensions-ORES, 10Patch-For-Review, 10User-Ladsgroup: Make wp10 rows be a squeezed to a weighted sum in ores_classification - https://phabricator.wikimedia.org/T194297#4231919 (10Halfak) I like this, but I think we should use the term "aggregate" rather than... [15:59:13] blargh, I just got the "reference ui" compiling with the more production-y packager, and it's actually bigger than before. [16:01:52] on that note, I saw this on a window in Wales, maybe it's where semi-English "argh" comes from, https://en.wiktionary.org/wiki/argyfwng [16:02:20] "The British are coming" -- "ARGH..." [16:02:43] lol [16:02:56] awight, where are you hacking from today? [16:04:15] Last day in Barcelona, I'm at a fresh-air place near a park [16:04:57] Gotcha. So it's getting kind of late there. [16:05:26] 10Scoring-platform-team (Current), 10MediaWiki-extensions-ORES, 10Patch-For-Review, 10User-Ladsgroup: Make wp10 rows be a squeezed to a weighted sum in ores_classification - https://phabricator.wikimedia.org/T194297#4231945 (10Halfak) It seems that this metaphor works for damaging and goodfaith too. There... [16:05:39] I'll be at it for another hour [16:05:44] (03CR) 10Halfak: [C: 04-1] "I left some notes in https://phabricator.wikimedia.org/T194297" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/434689 (https://phabricator.wikimedia.org/T194297) (owner: 10Ladsgroup) [16:06:01] Why, you think we should sneak the drafttopic model into production? ;-) [16:06:26] Just thinking about getting some progress made on that front -- whether it is possible or impossible. [16:06:47] Also, I'll be meeting the other WMFMN people at a coffee shop soon. Will wait until you're done to move. :D [16:07:00] oh no worries, but thank you [16:07:25] Sure, I probably have enough time to make progress on diagnosing the ores1001 failure [16:07:36] Cool :) [16:07:42] * awight washes webpack's blood off the keyboard [16:07:51] Just finishing up reviewing some stuff that Am.ir1 was working on [16:08:16] Oh! I have a thing that could use a quick review. https://github.com/wiki-ai/articlequality/pull/67 [16:08:18] awight, ^ [16:09:10] will do [16:11:13] random thought--maybe model training should have a mandatory parameter for writing model statistics to a text file... [16:11:40] awight, I like this idea. [16:11:45] It makes sense. [16:11:50] And we'd like to have it. [16:14:15] The euwiki PR is merged. [16:14:24] * awight glares at github-irc bridge [16:14:39] right. WTF [16:14:42] Thanks man [16:14:49] 10Scoring-platform-team, 10ORES: Add a mandatory parameter to output model statistics after training - https://phabricator.wikimedia.org/T195603#4231957 (10awight) [16:15:53] 10Scoring-platform-team (Current), 10Wikilabels, 10articlequality-modeling, 10artificial-intelligence: Train/test article quality model for euwiki - https://phabricator.wikimedia.org/T171119#4231968 (10Halfak) [16:20:02] 10Scoring-platform-team, 10articlequality-modeling, 10artificial-intelligence: Extend feature extractors for euwiki wp10 model - https://phabricator.wikimedia.org/T195604#4231976 (10Halfak) [16:21:59] 10Scoring-platform-team (Current), 10Wikilabels, 10articlequality-modeling, 10artificial-intelligence: Train/test article quality model for euwiki - https://phabricator.wikimedia.org/T171119#3454661 (10Halfak) [16:22:01] 10Scoring-platform-team, 10articlequality-modeling, 10artificial-intelligence: Extend feature extractors for euwiki wp10 model - https://phabricator.wikimedia.org/T195604#4231996 (10Halfak) [16:22:14] 10Scoring-platform-team (Current), 10articlequality-modeling, 10artificial-intelligence: Complete labeling campaign for euwiki - https://phabricator.wikimedia.org/T195106#4231999 (10Halfak) [16:22:16] 10Scoring-platform-team (Current), 10Wikilabels, 10articlequality-modeling, 10artificial-intelligence: Train/test article quality model for euwiki - https://phabricator.wikimedia.org/T171119#4231998 (10Halfak) [16:22:34] EEk! Just saw that we are behind on bswiki. [16:22:42] I'm going to assign that to Amir1 :) [16:23:10] 10Scoring-platform-team (Current), 10editquality-modeling, 10artificial-intelligence: Train damaging/goodfaith models for Bosnian Wikipedia - https://phabricator.wikimedia.org/T194876#4232000 (10Halfak) @Ladsgroup, looks like you might have time for this. What do you think? [16:23:53] 10Scoring-platform-team (Current), 10ORES, 10drafttopic-modeling: Investigate memory issues surrounding trial drafttopic deployment - https://phabricator.wikimedia.org/T195593#4232002 (10awight) [16:27:28] 10Scoring-platform-team (Current), 10ORES: ores1001 is in some kind of death spiral - https://phabricator.wikimedia.org/T194329#4232014 (10awight) [16:27:30] 10Scoring-platform-team (Current), 10ORES, 10drafttopic-modeling: Investigate memory issues surrounding trial drafttopic deployment - https://phabricator.wikimedia.org/T195593#4232016 (10awight) [16:33:13] 10Scoring-platform-team, 10articlequality-modeling, 10Easy, 10artificial-intelligence: Extend feature extractors for euwiki wp10 model - https://phabricator.wikimedia.org/T195604#4232051 (10Halfak) [16:33:25] 10Scoring-platform-team, 10articlequality-modeling, 10artificial-intelligence: Extend feature extractors for euwiki wp10 model - https://phabricator.wikimedia.org/T195604#4231976 (10Halfak) [16:35:09] 10Scoring-platform-team, 10ORES: Add a parameter to cv_train to output model information - https://phabricator.wikimedia.org/T195603#4232074 (10Halfak) [16:35:27] 10Scoring-platform-team, 10ORES, 10revscoring, 10artificial-intelligence: Add a parameter to cv_train to output model information - https://phabricator.wikimedia.org/T195603#4232077 (10Halfak) [16:35:45] That should be really easy because we already dump model_info to stderr. [16:35:50] We just need to add a parameter :) [16:36:30] awight if you update to git-review 1.6 that will fix your issue [16:36:40] also im supprised that 1.4 is working for you [16:36:49] awight the fix is https://github.com/openstack-infra/git-review/commit/f918bf76d22ea2ea92aaa5856da83fc267705249 [16:36:56] also 1.6 contains a gerrit 2.14 fix too [16:39:17] 10Scoring-platform-team (Current), 10ORES, 10drafttopic-modeling: Investigate memory issues surrounding trial drafttopic deployment - https://phabricator.wikimedia.org/T195593#4232081 (10awight) [16:40:31] paladox: Not sure which issue this is, but I'm using git-review 1.26 at the moment [16:40:40] awight https://github.com/openstack-infra/git-review/commit/f918bf76d22ea2ea92aaa5856da83fc267705249 [16:40:51] https://phabricator.wikimedia.org/T163242#4232067 [16:42:35] awight, what should I call the parent task for your reference UI work? [16:42:56] I'm guessing "try out react" doesn't quite capture it [16:43:27] Hehe well, I've already blown the scope of T195274 [16:43:27] T195274: Rewrite ORES "reference" UI using React - https://phabricator.wikimedia.org/T195274 [16:43:51] "do crazy stuff on the plane" [16:43:54] Aha! [16:44:25] FWIW, I was mostly doing that as a trial for rewriting wikilabels [16:44:38] to make sure the tech was reasonable and so on [16:44:55] so maybe the epic is, "Give some TLC to our UIs" [16:45:07] I'd only see rewriting wikilabels in react as a good idea if it'll get us mobile support easier. [16:45:21] I like that. I'll start from there and do what I can. [16:47:17] I think the original thought was that jdlrobson was interested in "modernizing" wikilabels, but now I can't quite remember if it was him or someone else... [16:47:20] fail [16:47:52] & I wanted to see if that was a terrible idea or not [16:48:45] With Wikilabels, I think we have big gains to be made by (1) making it easier for people to manage their own campaigns and (2) letting people label things from a mobile UI. [16:49:00] General modernizing is OK, but probably not worth our time. [16:49:33] Conclusion is that it's a decent environment, and easy to make changes after the rewrite. [16:49:42] +1 that modernizing for its own sake is silly [16:50:04] and IMO modernizing just means, following the latest trends which will be reversed in a year. [16:50:26] But in this case, it would make it really easy to play with things like 1- vs 2-column diffs [16:50:46] We can swap out components, styles, and in a sane way. [16:51:28] caveat is that I haven't looked at the existing wikilabels code yet. If Amir1 says it's clean, we can probably eat off of it. [16:51:30] Right. That could be a big win. I wonder if jrobson would be interested in contributing some 20% time towards that. [16:51:53] ha! I worked hard to make it nice-ish, but I'm no UI dev. [16:51:58] It's consistent at least :D [16:52:09] I'm sort of thinking his 1.5hr training me on React and the build environment is all we need to ask :) [16:52:18] maybe some CR down the road... [16:52:37] he's a great teacher, btw [16:52:39] Still, it would be a nice 20% project. [16:52:46] They do 20% in some audience teams. [16:52:55] hahaha. throwback thursdays. [16:53:07] like, teams with enough staffing to get their core work done. [16:53:09] Maybe it's 10% now that I think about it. [16:53:11] lol [16:53:12] sounds nice [16:53:45] Unrelated: https://i.imgur.com/AyuLsf3.gifv [16:54:09] fantastic [16:54:20] didyu know that spiders are hydraulic? [16:54:27] jumping spiders, at least... [16:54:56] Yes! It's crazy. They pump blood around to move their legs. Makes them extra creepy. [16:55:44] "A skeleton's body is filled with a fluid that is like blood (though some what different) called hemolymph" [16:55:48] Oh. Not blood [16:55:51] http://infinitespider.com/spider-legs-work/ [16:56:12] Creepy, or incredibly cool. Depends on the ambient light level, I guess. [16:56:24] OK i gotta stand on the sidewalk and nibble a pastry. [16:56:24] lol [16:56:34] This is why research labs are well lit. [16:56:37] Less creepy [16:56:44] lawl, fact [16:57:20] 10Scoring-platform-team, 10articlequality-modeling, 10artificial-intelligence: Extend feature extractors for euwiki wp10 model - https://phabricator.wikimedia.org/T195604#4232154 (10Theklan) [16:57:21] omg "searching through the rubble" is always the justification for making small, killer robots [16:57:23] 10Scoring-platform-team, 10ORES, 10UI-Standardization: TLC for ORES UIs - https://phabricator.wikimedia.org/T195607#4232155 (10Halfak) [16:57:29] <3 [16:57:36] kk bye [16:57:40] 10Scoring-platform-team, 10articlequality-modeling, 10artificial-intelligence: Extend feature extractors for euwiki wp10 model - https://phabricator.wikimedia.org/T195604#4232158 (10Theklan) [16:58:34] heading out awight? [16:59:57] 10Scoring-platform-team, 10articlequality-modeling, 10artificial-intelligence: Extend feature extractors for euwiki wp10 model - https://phabricator.wikimedia.org/T195604#4232160 (10Halfak) Are automatic infoboxes more or less desirable than regular infoboxes? Does it make a difference? [17:02:01] I'm heading to lunch now then. [17:02:08] Should be back online in about 1 hour [17:05:03] 10Scoring-platform-team, 10articlequality-modeling, 10artificial-intelligence: Extend feature extractors for euwiki wp10 model - https://phabricator.wikimedia.org/T195604#4232184 (10Theklan) Community is trying to use more extensively automatic infoboxes, as the final result is usually better than filling th... [17:16:56] 10Scoring-platform-team, 10articlequality-modeling, 10artificial-intelligence: Extend feature extractors for euwiki wp10 model - https://phabricator.wikimedia.org/T195604#4232207 (10Halfak) Makes sense. We should keep those as separate features then so that the model can learn people's preferences :)