[00:02:10] iono, I ran across other reddish flags such as https://www.mediawiki.org/wiki/Template:Cite_web [07:01:00] 10Scoring-platform-team, 10Wikilabels, 10editquality-modeling, 10User-Ladsgroup, 10artificial-intelligence: Edit quality campaign for hr.wiki - https://phabricator.wikimedia.org/T172047#3568984 (10Ladsgroup) The UI i18n is deployed now. [08:39:40] 10Scoring-platform-team-Backlog, 10ORES, 10Operations, 10Patch-For-Review, 10User-Ladsgroup: Review and fix file handle management in worker and celery processes - https://phabricator.wikimedia.org/T174402#3569163 (10awight) We'll need a manual puppet run on ores1001. I'm seeing some weirdness, just pok... [09:45:06] 10Scoring-platform-team, 10ORES, 10Wikilabels, 10revscoring, 10artificial-intelligence: [Discuss] Moving from nosetests to pytest - https://phabricator.wikimedia.org/T174660#3569337 (10Ladsgroup) [13:04:03] 10Scoring-platform-team, 10DBA, 10Operations, 10cloud-services-team: Labsdb* servers need to be rebooted - https://phabricator.wikimedia.org/T168584#3569772 (10jcrespo) I do not think we should postpone the reboots too much, my proposal would be to: 0) document access to the new hosts (bare essentials) 1)... [13:17:36] o/ Amir1 [13:17:53] halfak: goat morning [13:18:09] Sorry about not getting to the todo list but I see there's two good ones on there still :) [13:19:01] a person who forgets to assign me tasks, you are a good boss [13:19:24] halfak: I started it https://etherpad.wikimedia.org/p/SPT_status_update_August [13:19:32] but I haaaaaate it [13:19:51] It's hard work! [13:19:58] I even started making nosetests for wikilabels instead of my real job :D [13:19:58] also lol [13:21:32] What about the fawiki article quality modeling stuff? [13:22:49] https://www.mediawiki.org/wiki/Topic:Tx9hwftq823yfi7t [13:22:55] We have a huge bug in wikilabels right now [13:22:59] This is a showstopper [13:23:11] We should take the labeling interface down before anyone submits another label. [13:25:26] halfak: I will fix it right now [13:25:40] We might want to delete any labels saved since the last deploiy [13:25:47] Was this only live since the last deploy? [13:27:57] halfak: I yeah [13:28:04] It won't be needed I guess [13:28:15] they are practically nothing [13:28:20] I'll look into how many labels were saved. [13:28:30] If you can get a fix ready [13:29:35] on it [13:30:31] 21 rows [13:31:42] All labeled using damaging_and_goodfaith [13:31:48] So we'll want to clear them all. [13:33:35] I'm going to be ready to do that once we get the deployment made. [13:37:46] Woops! There's actually 45 rows [13:37:55] I was looking at a deployment that happened earlier today [13:38:04] 0700 UTC [13:38:56] halfak: https://github.com/wiki-ai/wikilabels/pull/199 please confirm locally if it works [13:39:07] or on staging, doesn't really matter [13:39:20] Weird. This doesn't make sense to me at all. [13:39:30] should be the other way around. [13:39:40] * halfak looks again [13:40:10] O/ [13:41:42] halfak: to me too but I'm guessing action=compare is very very stupid [13:42:13] we should flip the arguments there [13:42:25] https://github.com/wiki-ai/wikilabels/blob/master/wikilabels/wsgi/static/js/wikiLabels/api.js#L84 [13:42:41] fromrev: diffToRev [13:42:44] torev: revId [13:44:27] halfak: done [13:47:49] halfak: ^ I'm around to deploy [13:48:14] Merged. Please update the deploy repo directly and go to staging. [13:48:21] I'll start drafting an email today. [13:48:29] *for today [13:49:15] halfak: did we announce the phame post yet to ai-l? If not i can write up a draft and send it to you for review [13:49:33] Zppix, dealing with an event [13:49:52] Ack let me know when you can halfak [13:51:30] https://etherpad.wikimedia.org/p/wikilables_reverse_diff [13:53:15] halfak: deployed [13:53:18] in prod [13:56:51] confirmed [13:57:42] deletion of labels confirmed. [13:57:46] Continuing work on email [13:57:49] thanks [14:02:32] https://phabricator.wikimedia.org/phame/post/view/69/wikilabels_incident_reversed_diffs/ [14:02:43] Amir1, would you post that to mailing lists? [14:03:03] You wrote it, I can post it though if you want me to [14:03:06] And with that I think we can close the door on this and go back to regularly scheduled stuff. [14:03:19] Sure! I'm thinking a short statement + link to the post [14:03:33] It would be great to convince more people to watch that blog. :) [14:03:37] halfak: etherpad looks good. [14:03:38] I see [14:03:42] okay [14:06:27] halfak: sent to ai-l and wikitech-l [14:07:01] Amir1: recieved, looks good [14:08:26] Good show! Stuff happens. That was a nice fix and good reporting. [14:09:16] halfak: I made this today: https://phabricator.wikimedia.org/T174660 [14:09:24] let's back to normal work [14:09:32] * Amir1 calms halfak down [14:09:47] Interesting. [14:09:48] :p [14:10:41] the thing is it doesn't have its own asserting system (like eq_) [14:10:53] in php we have phpunit library which does wonders [14:13:57] halfak: https://phabricator.wikimedia.org/T174576 [14:14:24] we can make our own eq_ or maybe we could continue to use nose.tools? [14:14:37] nosetests built on Python's UnitTest system [14:14:43] So maybe pytest builds on nosetests? [14:15:04] Pytest with nosetools sounds good to me personally [14:15:19] I'd miss the @raises decorator too [14:15:23] yeah, I've been thinking about it but then what's the whole point of having pytest (what's its use) [14:15:49] we can just move on with nosetests [14:16:04] Amir1: travis-ci can use it to interact with unit tests i believe [14:16:04] btw https://phabricator.wikimedia.org/T174576 :D [14:16:22] Zppix: it can with both of them [14:17:47] Oh [14:17:57] Amir1: +1 that task [14:20:04] Halfak ping me when your ready to talk about annoucing the phame post from 2 days ago (if you havent already) [14:23:44] Zppix, I think we should add some of awight's figures to the post before moving forward. [14:23:54] Ok [14:23:54] Actually, maybe that should be a followup post. [14:24:06] Let's talk to him either way. [14:24:13] Amir1: ^ [14:24:19] Oops [14:24:34] A tab to quick [14:24:36] :P [14:25:34] Ill make sure to say something to adam then if i remember beforehand [14:47:21] 10Scoring-platform-team-Backlog, 10articlequality-modeling, 10artificial-intelligence: Article quality campaign for Persian Wikipedia - https://phabricator.wikimedia.org/T174684#3570117 (10Ladsgroup) [14:47:25] halfak: https://phabricator.wikimedia.org/T174684 [14:48:14] 10Scoring-platform-team-Backlog, 10articlequality-modeling, 10artificial-intelligence: Article quality campaign for Persian Wikipedia - https://phabricator.wikimedia.org/T174684#3570130 (10Halfak) [14:48:46] 10Scoring-platform-team-Backlog, 10articlequality-modeling, 10artificial-intelligence: Article quality campaign for Persian Wikipedia - https://phabricator.wikimedia.org/T174684#3570117 (10Halfak) How do InfoBoxes work? Are they used like on English Wikipedia? Are there "citation needed" templates? How do... [14:50:02] 10Scoring-platform-team-Backlog, 10articlequality-modeling, 10artificial-intelligence: Article quality campaign for Persian Wikipedia - https://phabricator.wikimedia.org/T174684#3570135 (10Halfak) Re labeling, we can do a stratified sampling strategy where we select N articles above a certain length and N ar... [14:50:21] 10Scoring-platform-team-Backlog, 10articlequality-modeling, 10artificial-intelligence: Article quality campaign for Persian Wikipedia - https://phabricator.wikimedia.org/T174684#3570141 (10Ladsgroup) we have both "infobox foo" and "جعبه اطلاعات .+?" so they are the same, yeah there are citation needed templa... [14:54:09] halfak: I'm waiting for things to do :D [14:54:30] oh we have this reverted and damaging model [14:54:40] I need ores-misc-01 [14:55:19] Amir1, you should try to get the best sample of labels you can from fawiki. [14:55:47] You can see how we do it for the other wikis in the wikiclass project [14:55:56] okay [14:56:00] will work on it [14:56:10] but we definitely need users to label some stuff for us [14:56:44] +1 I think you're right. We might even want to have people re-label some of the observations for enwiki. [14:57:12] I just realized my post assumed LTR for the wikilabels isse [14:57:15] * halfak feels shame [14:57:38] Awight got a minute i and halfak were wondering... if you wanted to add figures to the post from 2 days ago? [14:58:07] o/ [14:58:23] Time to rebuild the stats with new model info? [14:58:30] Yes [14:58:32] 10Scoring-platform-team, 10DBA, 10Operations, 10cloud-services-team: Labsdb* servers need to be rebooted - https://phabricator.wikimedia.org/T168584#3570152 (10bd808) @jcrespo's plan sounds like a good one. Working on the announce of the new cluster was already on my todo list for today, so I'll add foresh... [15:01:28] halfak: ^ how did you work around the Github 120MB limit? [15:01:38] no idea [15:01:51] Halfak is a hacker confirmed xD [15:01:52] awight, are you going to join Keegan and I right now? [15:08:55] 10Scoring-platform-team: Create list of ORES collaborators (focus on language asset helpers) - https://phabricator.wikimedia.org/T174685#3570165 (10Halfak) [15:09:16] 10Scoring-platform-team: Create list of ORES collaborators (focus on language asset helpers) - https://phabricator.wikimedia.org/T174685#3570178 (10Halfak) [15:09:28] 10Scoring-platform-team: Create list of ORES collaborators (focus on language asset helpers) - https://phabricator.wikimedia.org/T174685#3570165 (10Halfak) See also @awight's work on https://www.mediawiki.org/wiki/ORES/Stakeholders [15:10:41] 10Scoring-platform-team: Create list of ORES collaborators (focus on language asset helpers) - https://phabricator.wikimedia.org/T174685#3570184 (10Halfak) See also https://it.wikipedia.org/wiki/Progetto:Patrolling/ORES and https://www.wikidata.org/wiki/Wikidata:ORES/Report_mistakes [15:30:15] 10Scoring-platform-team-Backlog, 10Bad-Words-Detection-System, 10revscoring, 10artificial-intelligence: Add language support for Serbian - https://phabricator.wikimedia.org/T174687#3570233 (10Srdjan_m) [15:57:20] wiki-ai/revscoring#1206 (tqdm - 5aa6e4b : Amir Sarabadani): The build passed. https://travis-ci.org/wiki-ai/revscoring/builds/270476712 [15:57:31] halfak: around for a quick question [15:57:39] ? [15:58:14] Amir1, yes. What's up? [15:58:35] for reverted model, how I can determine --pop-rate "true=0.049775581219426095" \ and false? [15:58:44] just the proportion of true cases? [15:59:14] So it depends on how we did our sampling. But in most cases, yes. Just the proportion of trues. [15:59:24] E.g. in enwiktionary, we did stratified sampling. [16:01:11] okay, thanks [17:03:46] * halfak orders a big set of new ORES stickers. [17:04:01] Niiiice [17:04:39] They were going really fast at wikimania ^_^ [17:04:48] Everyone seemed to find the IPA amusing [17:05:17] https://www.mediawiki.org/wiki/JADE/Schema [17:08:53] halfak: to make your happiness drop I must inform you revscoring tune doesn't work the way it's mentioned in editquality make file [17:09:08] https://www.irccloud.com/pastebin/wGyQTdVB/ [17:09:09] Yeah. I noticed that. I'm working on it. [17:09:13] Mke me a task? :D [17:10:20] 10Scoring-platform-team, 10editquality-modeling, 10revscoring, 10artificial-intelligence: editquality make file rules doesn't work for revscoring tune - https://phabricator.wikimedia.org/T174704#3570707 (10Ladsgroup) [17:10:22] https://phabricator.wikimedia.org/T174704 [17:10:34] Done, let me know when you're done, my hrwiki work is blocked on it [17:10:40] kk [17:10:47] awight, https://www.mediawiki.org/wiki/JADE/Schema [17:11:03] I made a link to https://www.mediawiki.org/wiki/JADE/Implementation in the main page. [17:11:11] You notice I took some liberties with the schema. [17:11:36] delete branched after merging :((((( [17:12:12] I just deleted more than five (at least) [17:13:24] Looks like my 125MB model turns into a 42MB model with bz2 compressions >:( [17:13:35] wiki-ai/revscoring#1208 (tune_fix - 783e4bd : halfak): The build passed. https://travis-ci.org/wiki-ai/revscoring/builds/270506569 [17:13:35] So maybe we need to support that now. [17:15:23] * halfak runs test on tune utility fix [17:15:33] Amir1, if this works, ETA will be 15 minutes. [17:15:48] okay [17:15:56] In the mean time, i move to hotel [17:20:41] 10Scoring-platform-team-Backlog, 10Bad-Words-Detection-System, 10revscoring, 10User-Ladsgroup, 10artificial-intelligence: Add language support for Bosnian - https://phabricator.wikimedia.org/T173087#3570765 (10Srdjan_m) [[ https://meta.wikimedia.org/w/index.php?diff=17177571 | Sorted ]]. [17:20:42] 10[5] 04https://meta.wikimedia.org/wiki/https://meta.wikimedia.org/w/index.php%3Fdiff%3D17177571 [17:44:03] arg. so there's a new problem. THis will take longer than expected. [17:48:14] halfak: oh I was gonna take that pickle crunching work! [17:55:03] pickle crunching work? you mean lunch??? ahahaha [18:27:13] awight, hi, i finished the admin interface for groups in polygerrit last week :) [19:50:02] halfak: quick question - whats the preferred way to manage graceful error handling in a method that makes a lot of web requests and iterates over the results? [19:50:23] something like logging and skipping over the invalid results... and continuing over [19:56:44] halfak: I’m back for a minute [19:57:22] halfak: So are you mid-bz2 model streaming or can I take over? [19:58:11] halfak: actually, let’s cheat this out for blogging purposes. Can you run the api locally, using the Makefile in treshold_diagrams? [19:58:37] just check in the resulting .json (I’ve been bz2’ing that manually) [19:59:00] * awight checks whether this model was built on a machine I can access [20:04:03] Hey! Just got done with a meeting. [20:04:44] codezee, generally, I'd try to catch the exception at the level you are iterating. Hard to say something more clearly without an example. [20:04:54] exception + log a warning is nice. [20:05:07] halfak: i suppose wikiclass utilities would have good examples right? [20:05:30] awight, I've given up on the bz2 stuff while I work on the tune utility. [20:05:50] BTW, I was thinking we should have a separate blog post about threshold graphs. [20:05:53] perfect. [20:05:59] ahaha no problem [20:06:00] Seems like that's a big topic. [20:06:18] although just these two might spice up yesterday’s epistle [20:06:18] codezee, right! [20:06:40] you feel like pulling the stats locally as a workaround? [20:07:23] it’s a “make” job, then copy the json output somewhere for me or check it in [20:11:55] ah—either way, please tell me where this 120MB model file lives so I don’t have to retrain. [20:13:21] Oh I can do that [20:15:56] ty, then at least the graphs are not blocked and work can be independent [20:18:48] I guess a bz2 wrapper is really easy in python, but it would be nice to spend time on figuring out why the size jumped, git-lfs, etc... [20:19:34] I bet it's thresholds [20:19:41] That would be interesting [20:19:51] really? [20:19:51] A related thing that has been bothering me: [20:19:58] I think we're just storing a bunch more information [20:20:10] email inbound. [20:20:11] interesting as in, we should be able to find an efficient data type for the data [20:20:21] I'm not sure that's the exact right format, but it'll be close. [20:21:22] related botherment is: I’m pretty sure we load the model file in each thread, not for each request but at least when forking new workers. There might be some efficiency to gain there. [20:21:51] What kind of efficiency [20:22:22] Looks like ~32MB of that 125 MB can be attributed to thresholds [20:22:39] halfak: I thought it was damaging that looked wrong to you? [20:22:40] harumm. cool that you could tell [20:23:00] Oh. Damaging looked OK. It was draftquality that I had a better model for :) [20:24:35] efficiency like, having the workers hashed by (wiki, model) so the model can stay in memory and somehow be shared across forked processes. [20:24:54] Can’t make an educated guess yet though, since I don’t know that code and what’s already been done. [20:25:15] cool thx, I’ll redraw the draftquality pic [20:25:34] I think they are shared in memory [20:25:43] Given how our real memory usage is going... [20:26:07] awight, when you draw it, do !precision, !recall, and match_rate [20:26:13] Since the target class is !OK :) [20:34:37] halfak: oops! https://github.com/adamwight/thresholds_diagrams/blob/master/draftquality-OK.svg [20:34:41] ok going back in [20:35:20] halfak: do you want me to write up that email draft for you now? [20:35:46] Zppix, sure. Why don't you start up an etherpad with it. [20:35:51] Yep [20:36:02] Is wmfs etherpad public? [20:36:16] halfak: looks like !precision is wrong [20:36:33] Yes they are Zppix [20:36:43] awight, wut [20:36:46] Ok [20:37:13] uh sorry [20:37:27] first row is empty string, but the rest of the fail is probably caused by a typo in my .ipynb [20:37:43] Could be a null. nulls happen there [20:38:00] no items selected, precision is undefined. [20:38:25] At the lowest threshold, everything is selected and nothing is !selected. [20:38:33] https://etherpad.wikimedia.org/p/VX0lpWtAMb halfak [20:39:06] Zppix, why not something descriptive? [20:39:25] It errored out when i tried [20:39:47] https://etherpad.wikimedia.org/p/thresholds_announcement_email [20:39:54] Ok ty [20:40:22] gtg for an hour... [20:45:46] Halfak looking good so far? [20:51:02] Sure. Not bad. You might crib the intro from the blog [20:52:30] I am working into that i just wanted to write own content just to make it feel personalised and not just a copy and paste feel :) [20:53:58] 10Scoring-platform-team, 10Edit-Review-Improvements-RC-Page, 10MediaWiki-extensions-ORES, 10Collaboration-Team-Triage (Collab-Team-Q1-Jul-Sep-2017): Very long search times on RC Page for "Very likely good faith" + "Likely have problems" (on en.wiki only?) - https://phabricator.wikimedia.org/T164796#3571485 (... [20:56:48] halfak: waiting for you to approve before i send. [20:57:35] Woops. Looks like the first line is wrong [20:57:45] We did not introduce the breaking change yet. [20:57:50] The change will not happen for a month. [20:58:04] Ah i meant to fix that [20:59:03] Fixed [21:00:33] * halfak goes in for the edit. [21:00:34] halfak: wtf is "feature.enwiki.revision.who_remplates": 0, [21:00:43] {{who}} [21:00:45] 10[6] 04https://meta.wikimedia.org/wiki/Template:who [21:00:51] {{:en:who}} [21:00:53] 10[7] 04https://meta.wikimedia.org/wiki/Template::en:who [21:01:08] Damn it asimov bot [21:01:10] why is it called who_remplates? [21:01:16] typing is hard [21:01:18] lol [21:01:55] r looks like t to my old stupid eyeballs [21:02:39] that's the only difference in output on ores-misc for the wp10 queries, so it looks like the thresholds stuff won't break my code. [21:02:51] :-) [21:03:00] Yeah I think it won't be an issue for you. [21:03:04] You're not setting thresholds. [21:03:13] Not with confidence scores anyway [21:04:12] Halfak is the draft ready for me to send? [21:04:27] * halfak looks again [21:04:34] or yeah I was gonna go in for an edit. [21:04:50] Lol ok [21:10:59] ragesoss: thanks for the patch :) [21:11:20] definitely putting "machine learning' on my resume now. [21:11:27] Lol [21:11:40] ragesoss, +🐐 [21:12:15] dammit, quassel doesn't render emoji [21:12:26] Its a goat [21:12:32] yeah, I pasted it. [21:13:03] Let me know when your done halfak and ill send it off [21:16:13] Confirmed. [21:16:15] Zppix, ^ [21:16:32] halfak: I'm back [21:16:37] Ok [21:16:39] Sending [21:16:43] aaan I'm out, [21:16:53] goodnight Amir1 [21:17:02] Sorry for the big delay on the tuning utility [21:17:06] Took a long time to finish my test :/ [21:17:14] I will come back soon [21:19:04] Zppix, you deleted the link to the blog post. [21:19:20] Ill send it in a reply [21:19:31] was the whole point :P [21:21:01] :P [21:21:23] Lets train ores to send announcements next [22:02:08] 10Scoring-platform-team, 10Edit-Review-Improvements-RC-Page, 10MediaWiki-extensions-ORES, 10Collaboration-Team-Triage (Collab-Team-Q1-Jul-Sep-2017): Very long search times on RC Page for "Very likely good faith" + "Likely have problems" (on en.wiki only?) - https://phabricator.wikimedia.org/T164796#3571614 (... [22:54:23] 10Scoring-platform-team, 10Edit-Review-Improvements-RC-Page, 10ORES, 10Collaboration-Team-Triage (Collab-Team-Q3-Jan-Mar-2017), and 2 others: Enable parts of ORES extension by default and manage impacts on the RC Page and the RC page Preferences tab - https://phabricator.wikimedia.org/T159763#3571827 (10jma... [22:54:26] 10Scoring-platform-team, 10Edit-Review-Improvements-RC-Page, 10ORES, 10Collaboration-Team-Triage (Collab-Team-Q4-Apr-Jun-2017), and 2 others: Conform ORES sensitivity levels to the new ERI standards - https://phabricator.wikimedia.org/T160575#3571826 (10jmatazzoni) 05Open>03Resolved [23:53:06] 10Scoring-platform-team-Backlog, 10Wikilabels, 10editquality-modeling, 10User-Tgr, 10artificial-intelligence: Complete edit quality campaign for Hungarian Wikipedia - https://phabricator.wikimedia.org/T167968#3572018 (10Tgr)