[00:25:25] 10Scoring-platform-team, 10Edit-Review-Improvements-RC-Page, 10MediaWiki-extensions-ORES, 10Collaboration-Team-Triage (Collab-Team-This-Quarter), and 2 others: [1.31.0-wmf.2 ]&hidenondamaging=0 is ignored when "Show only likely problem edits" preference is ... - https://phabricator.wikimedia.org/T177884#3743298 [00:25:30] 10Scoring-platform-team, 10Edit-Review-Improvements-RC-Page, 10MediaWiki-extensions-ORES, 10Collaboration-Team-Triage (Collab-Team-This-Quarter), and 2 others: [1.31.0-wmf.2 ]&hidenondamaging=0 is ignored when "Show only likely problem edits" preference is ... - https://phabricator.wikimedia.org/T177884#3743299 [10:24:23] o/ [10:24:35] Just about to head out to the university to deliver my keynote on ORES for the Internation Wiki Science Conference. :) [10:24:55] halfak: good luck! [10:25:06] Thanks ^_^ [12:19:09] (03PS1) 10Ladsgroup: Store stats of accessing ores service for getting thresholds [extensions/ORES] - 10https://gerrit.wikimedia.org/r/389960 (https://phabricator.wikimedia.org/T179862) [12:54:44] 10Scoring-platform-team (Current), 10ORES: Deploy ORES early Nov 2017 - https://phabricator.wikimedia.org/T179837#3744192 (10elukey) I am currently seeing the following on scb1002: ``` Nov 08 12:00:29 scb1002 systemd[1]: Unit celery-ores-worker.service entered failed state. Nov 08 12:30:19 scb1002 systemd[1]:... [13:26:48] 10Scoring-platform-team (Current), 10MediaWiki-extensions-ORES, 10draftquality-modeling, 10User-Ladsgroup, 10artificial-intelligence: Collect all data for draftquality model in enwiki - https://phabricator.wikimedia.org/T179861#3744260 (10Ladsgroup) This an amazing bug, It fails to hold more than two row... [13:27:33] 10Scoring-platform-team (Current), 10MediaWiki-extensions-ORES, 10draftquality-modeling, 10User-Ladsgroup, 10artificial-intelligence: Drop oresc_rev_predicted_model index - https://phabricator.wikimedia.org/T180026#3744261 (10Ladsgroup) [13:30:18] o/ [13:30:20] Just finished my keynote. They started 40 minutes late. :S [13:30:22] Apparently, that's how things work in Rio. [14:59:05] 10Scoring-platform-team, 10Cleanup, 10Gerrit, 10Release-Engineering-Team, 10Repository-Admins: Remove deprecated research/ores/deploy repo - https://phabricator.wikimedia.org/T179610#3744600 (10MarcoAurelio) [15:05:50] 10Scoring-platform-team, 10ORES: [Spec] Celery worker monitoring - https://phabricator.wikimedia.org/T139948#2447518 (10awight) I got interested in this question while stress testing on the new cluster. During most of my tests, some of the machines wouldn't process any jobs, and rarely were the workloads bala... [15:13:16] 10Scoring-platform-team, 10Scap: Deployment fails with something gitty - https://phabricator.wikimedia.org/T179987#3744687 (10awight) [15:20:19] 10Scoring-platform-team, 10Cleanup, 10Gerrit, 10Release-Engineering-Team, 10Repository-Admins: Remove deprecated research/ores/deploy repo - https://phabricator.wikimedia.org/T179610#3744743 (10demon) 05Open>03Resolved a:03demon Done. I didn't see anything in Github or Phabricator for it either. [15:28:25] Amir1: These seem to be unused, want to confirm? https://gerrit.wikimedia.org/r/#/admin/projects/mediawiki/services/ores [15:28:26] https://gerrit.wikimedia.org/r/#/admin/projects/research/ores [15:32:34] 10Scoring-platform-team, 10Cleanup, 10Gerrit, 10Repository-Admins, 10Release-Engineering-Team (Kanban): Remove deprecated research/ores/deploy repo - https://phabricator.wikimedia.org/T179610#3744804 (10awight) 05Resolved>03Open I found two more unused repos: https://gerrit.wikimedia.org/r/#/admin/pr... [15:35:03] awight: I don't remember but I guess there is a plan to move some of our stuff to that place but not so sure [15:35:12] Everything is blurry about that [15:36:19] Okay lemme change the task to “investigate”, then [15:36:33] (03PS1) 10Ladsgroup: Drop oresc_rev_predicted_model index on ores_classification [extensions/ORES] - 10https://gerrit.wikimedia.org/r/389999 (https://phabricator.wikimedia.org/T180026) [15:36:55] awight: I have two patches in ores extension for your review :D [15:37:31] Amir1: neat—okay thanks [15:38:09] 10Scoring-platform-team, 10Cleanup, 10Gerrit, 10Repository-Admins, 10Release-Engineering-Team (Kanban): Investigate and clean up deprecated ORES repos - https://phabricator.wikimedia.org/T179610#3744831 (10awight) [15:38:59] 10Scoring-platform-team, 10Cleanup, 10Gerrit, 10Repository-Admins, 10Release-Engineering-Team (Kanban): Investigate and clean up deprecated ORES repos - https://phabricator.wikimedia.org/T179610#3730659 (10awight) @Halfak @Ladsgroup Let's chat about the last two repos, they have weird things in them and... [15:41:15] (03CR) 10Awight: [C: 032] Drop oresc_rev_predicted_model index on ores_classification [extensions/ORES] - 10https://gerrit.wikimedia.org/r/389999 (https://phabricator.wikimedia.org/T180026) (owner: 10Ladsgroup) [15:42:26] Thanks! [15:42:46] (03Merged) 10jenkins-bot: Drop oresc_rev_predicted_model index on ores_classification [extensions/ORES] - 10https://gerrit.wikimedia.org/r/389999 (https://phabricator.wikimedia.org/T180026) (owner: 10Ladsgroup) [15:48:01] (03CR) 10Awight: "Great idea!" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/389960 (https://phabricator.wikimedia.org/T179862) (owner: 10Ladsgroup) [15:48:31] Amir1: What happens if no Statsd service exists? [15:48:51] mediawiki core should handle that part and I guess nothing [15:49:03] it's a core service, I'd just depend on it [15:49:17] Even for local or third-party installs? [15:50:15] Hmm I guess there should be no 3rd-party installs yet. So just us debugging locally. [15:52:35] Looks like getService will throw an exception. [15:55:28] 10Scoring-platform-team (Current), 10MediaWiki-extensions-ORES, 10Blocked-on-schema-change, 10User-Ladsgroup: Review and deploy schema change on dropping oresc_rev_predicted_model index - https://phabricator.wikimedia.org/T180045#3744862 (10Ladsgroup) [15:55:41] 10Scoring-platform-team (Current), 10DBA, 10MediaWiki-extensions-ORES, 10Blocked-on-schema-change, 10User-Ladsgroup: Review and deploy schema change on dropping oresc_rev_predicted_model index - https://phabricator.wikimedia.org/T180045#3744862 (10Ladsgroup) a:05Ladsgroup>03None [16:14:45] (03PS1) 10Umherirrender: build: Always exclude vendor [extensions/ORES] - 10https://gerrit.wikimedia.org/r/390015 [16:37:59] (03CR) 10Jforrester: [C: 032] build: Always exclude vendor [extensions/ORES] - 10https://gerrit.wikimedia.org/r/390015 (owner: 10Umherirrender) [16:44:13] (03Merged) 10jenkins-bot: build: Always exclude vendor [extensions/ORES] - 10https://gerrit.wikimedia.org/r/390015 (owner: 10Umherirrender) [17:02:03] halfak|Mobile, awight anyone in the call? [17:02:12] o/ [17:02:22] halfak|Mobile: oops, jumping on [17:02:29] Just logged in quick from my phone to say I can't join the call from my shitty mobile situation [17:02:56] aha [17:04:14] 10Scoring-platform-team (Current), 10DBA, 10MediaWiki-extensions-ORES, 10Blocked-on-schema-change, 10User-Ladsgroup: Review and deploy schema change on dropping oresc_rev_predicted_model index - https://phabricator.wikimedia.org/T180045#3745087 (10Marostegui) [17:04:18] * halfak|Mobile updates the etherpad [17:05:59] awight: no staff backlog? [17:06:07] No one is approving me in [17:06:17] me too :P [17:06:43] i guess there's no one there [17:06:49] sorry, I jumped in and it was empty [17:06:51] approoving [17:18:53] awight: did you invite Amir1 to do a brown bag? [17:19:38] * halfak|Mobile reads agenda as it grows [17:32:29] awight: what should we do with https://gerrit.wikimedia.org/r/#/c/389960/ now? I amend it for removing TTL_DAY? [17:32:56] Amir1: I think TTL_DAY is the best part :) [17:32:59] merging now... [17:33:23] \o/ [17:33:24] halfak|Mobile: I haven’t—wanted to do an minor edit iteration on the TAB [17:33:33] (03CR) 10Awight: [C: 032] Store stats of accessing ores service for getting thresholds [extensions/ORES] - 10https://gerrit.wikimedia.org/r/389960 (https://phabricator.wikimedia.org/T179862) (owner: 10Ladsgroup) [17:33:37] Thanks <3 [17:34:07] now I'm back on Icelandic language support, also would love to do a deployment of wikilables after that [17:34:23] awight: Is there anything I can help in debugging the prod deploy of ores? [17:34:44] I think the prod deploy is okay at the moment—I’ll fix that one pytest thing I noticed. [17:36:54] * Amir1 break things and awight fixes them, he goes to his room and thinks about his bad behavior :D [17:37:25] hehe just trying to get you back for fixing my own breakage [17:39:05] there a shared culture of breaking things [17:41:49] How's the scap issue? [17:42:04] Oh wait... we determined it wasn't a scap issue. [17:42:11] The version issue. [17:45:45] halfak|Mobile: It seems there is a continuous chain of issues, something gets fixed and something else gets broken, every issue looks new to me [17:46:07] so weird. How did this happen? [17:46:15] I'd love to help but I don't know what the status is. [17:50:19] halfak|Mobile: There are two issues blocking us from deploying the frwiki 500 thing. * https://phabricator.wikimedia.org/T179966 prevents us from deploying to beta, * some references to pytest in production scripts need to be resolved [17:50:58] The references to revscoring 1 classes should be fixed and merged, but we can’t see if that worked w/o fixing the above issues first. [17:51:12] OK. I could look into pytest stuff if no one else is. [17:51:37] Where are the fixes to revscoring living right now? [17:51:41] Is 2.0.10 out? [17:51:53] Oh... maybe that isn't needed [17:51:57] (03Merged) 10jenkins-bot: Store stats of accessing ores service for getting thresholds [extensions/ORES] - 10https://gerrit.wikimedia.org/r/389960 (https://phabricator.wikimedia.org/T179862) (owner: 10Ladsgroup) [17:52:45] halfak|Mobile: just for an update, the labeling service took 1h, 18m to label 93,000 observations [17:53:06] codezee: that's not bad :) [17:53:07] halfak|Mobile: 2.0.9 is ready to deploy, that’s the one with the frwiki/500 fixes. I don’t think we’ll need a new release of that lib. [17:53:16] We'll not need to manually extract labels that often. [17:53:19] awight: I thought pytest issue is fixed [17:53:31] but i ran into an issue that the page_ids are those of talk pages and i was not storing page-titles, so extraction in revscoring is an issue [17:53:31] if not, show me the bug and I make a patch [17:53:34] Amir1: Not that I know of. [17:53:46] * halfak|Mobile will leave it to Amir1 [17:53:48] The bug is just that we reference pytest.mark in a few non-test files [17:54:26] Do we need those marks? [17:56:34] awight: if it's picking it up, yes [17:56:51] why it's a bug? [17:57:23] Only because pytest isn’t installed on the production machines. [17:57:39] awight: we can add it as a wheel [17:57:44] it's super easy [17:57:51] That’s a valid option [17:58:52] I’m not convinced the marks are necessary, but if you say so... [17:59:45] e.g. revscoring/utilities/cv_train.py:@mark.skip('Not test') [18:00:19] Usually I try to do any test-related changes externally to the code itself [18:00:45] There’s no way to mark that file as non-test using some other mechanism? [18:01:17] I don’t see why “cv_train” would be autodetected as a test, anyway. [18:02:02] https://docs.pytest.org/en/latest/example/pythoncollection.html#ignore-paths-during-test-collection [18:04:04] Amir1: IMO the extension changes are more urgent, since we can’t deploy the service today [18:09:15] awight: I highly doubt some of the tests are being discovered at all [18:12:29] This obviates the need for “mark”, on my machine at least: https://github.com/wiki-ai/revscoring/pull/371 [18:13:22] Amir1: ^ [18:13:53] awight: thanks, that's exactly what I meant, I wait for travis and then I merge it [18:13:59] wiki-ai/revscoring#1292 (ignore_testlike - c9ee4fb : Adam Roses Wight): The build passed. https://travis-ci.org/wiki-ai/revscoring/builds/299233919 [18:14:05] ah I see, cool. [18:14:41] Amir1: Looks like your extension changes are good! https://en.wikipedia.beta.wmflabs.org/wiki/Special:RecentChanges?hidebots=1&hidecategorization=1&hideWikibase=1&damaging__likelygood_color=c2&damaging__maybebad_color=c3&damaging__likelybad_color=c4&urlversion=2 [18:15:14] That's my boy [18:15:42] btw whats the issue under discussion regarding production and tests? [18:16:22] codezee: It was just a minor thing, there were some “from pytest import mark” in code that gets scanned when running on production. [18:16:28] see the PR above [18:16:41] now let's rush it to prod :D [18:16:46] lolol [18:16:51] Those days are over for me. [18:17:06] awight: you don't push to prod anymore? [18:17:08] I’m a beta man to the core : [18:17:10] hehe [18:17:26] :D [18:17:31] codezee: I burned us all more than a few times lately, I’m trying to reform my act [18:18:09] Mostly it didn’t actually hit production, but I was playing with the new cluster and had to use the production deployment server to do so. It still stings. [18:18:39] nvm, its good sometimes to move fast and break things for learning [18:20:29] :D That’s an optimistic reading—I do have a tremendous if false sense of confidence in my wheels and scapping skills now! [18:24:49] Amir1: Want me to backport the extension patches for SWAT? [18:45:10] awight: As you wish [18:47:42] 10Scoring-platform-team (Current), 10Bad-Words-Detection-System, 10revscoring, 10User-Ladsgroup, 10artificial-intelligence: Add language support for Icelandic - https://phabricator.wikimedia.org/T178524#3745561 (10Ladsgroup) a:03Ladsgroup [18:50:51] 10Scoring-platform-team (Current), 10Bad-Words-Detection-System, 10revscoring, 10User-Ladsgroup, 10artificial-intelligence: Add language support for Icelandic - https://phabricator.wikimedia.org/T178524#3745576 (10Ladsgroup) https://github.com/wiki-ai/revscoring/pull/372 [18:55:03] I'm deploying wikilables and then calling it a day [19:03:31] Amir1: Do you have a DBA to drop the index for us? [19:07:51] (03PS1) 10Awight: Merge master into wmf/1.31.0-wmf.6 [extensions/ORES] - 10https://gerrit.wikimedia.org/r/390067 [19:08:26] awight: Yeah, don't worry about that part [19:09:56] awight: are you around for a fast PR? https://github.com/wiki-ai/wikilabels-wmflabs-deploy/pull/43 [19:10:07] (03PS1) 10Awight: Merge master into wmf/1.31.0-wmf.7 [extensions/ORES] - 10https://gerrit.wikimedia.org/r/390069 [19:10:45] awight: https://gerrit.wikimedia.org/r/390069 and https://gerrit.wikimedia.org/r/390067 [19:10:48] no no [19:10:52] let me make patches [19:11:01] this is wrong [19:11:23] it won't pass SWAT [19:11:28] ok [19:12:03] I was concerned about the merge —no-ff, but as far as I could remember, the alternative was a pile of random patches. [19:12:31] The only one that looks important is “Store stats of accessing ores service for getting thresholds” [19:12:54] Amir1: Do you need it out today or shall we wait for the train? [19:12:59] (03PS1) 10Ladsgroup: Store stats of accessing ores service for getting thresholds [extensions/ORES] (wmf/1.31.0-wmf.7) - 10https://gerrit.wikimedia.org/r/390070 (https://phabricator.wikimedia.org/T179862) [19:13:14] today would be better [19:13:36] (03PS1) 10Ladsgroup: Store stats of accessing ores service for getting thresholds [extensions/ORES] (wmf/1.31.0-wmf.6) - 10https://gerrit.wikimedia.org/r/390071 (https://phabricator.wikimedia.org/T179862) [19:13:48] awight: These two patches can go to SWAT ^ [19:14:14] ty [19:16:16] halfak|Mobile: awight http://labels-staging.wmflabs.org/stats/enwiki/4 [19:16:21] That's so fancy [19:16:39] Look at that Wiki labeler fella [19:16:40] (03CR) 10Niharika29: [C: 032] "SWAT." [extensions/ORES] (wmf/1.31.0-wmf.7) - 10https://gerrit.wikimedia.org/r/390070 (https://phabricator.wikimedia.org/T179862) (owner: 10Ladsgroup) [19:16:42] ;) [19:17:21] Oh no! Now everyone can see us stuffing the ballot box. [19:18:00] (03CR) 10Niharika29: [C: 032] Store stats of accessing ores service for getting thresholds [extensions/ORES] (wmf/1.31.0-wmf.6) - 10https://gerrit.wikimedia.org/r/390071 (https://phabricator.wikimedia.org/T179862) (owner: 10Ladsgroup) [19:18:37] (03Abandoned) 10Awight: Merge master into wmf/1.31.0-wmf.6 [extensions/ORES] - 10https://gerrit.wikimedia.org/r/390067 (owner: 10Awight) [19:18:45] (03Abandoned) 10Awight: Merge master into wmf/1.31.0-wmf.7 [extensions/ORES] - 10https://gerrit.wikimedia.org/r/390069 (owner: 10Awight) [19:19:08] (03Merged) 10jenkins-bot: Store stats of accessing ores service for getting thresholds [extensions/ORES] (wmf/1.31.0-wmf.7) - 10https://gerrit.wikimedia.org/r/390070 (https://phabricator.wikimedia.org/T179862) (owner: 10Ladsgroup) [19:20:37] (03Merged) 10jenkins-bot: Store stats of accessing ores service for getting thresholds [extensions/ORES] (wmf/1.31.0-wmf.6) - 10https://gerrit.wikimedia.org/r/390071 (https://phabricator.wikimedia.org/T179862) (owner: 10Ladsgroup) [19:20:42] 10Scoring-platform-team (Current), 10ORES, 10Wikimedia-log-errors: Notice: Undefined property: stdClass::$ores_damaging_threshold in /srv/mediawiki/php-1.31.0-wmf.6/extensions/ORES/includes/Hooks.php on line 602 - https://phabricator.wikimedia.org/T179830#3745641 (10awight) [19:20:56] (03PS1) 10Catrope: Don't use $row->ores_damaging_threshold in Contributions where it isn't set [extensions/ORES] - 10https://gerrit.wikimedia.org/r/390072 (https://phabricator.wikimedia.org/T179830) [19:21:19] (03CR) 10Catrope: "Untested" [extensions/ORES] - 10https://gerrit.wikimedia.org/r/390072 (https://phabricator.wikimedia.org/T179830) (owner: 10Catrope) [19:23:32] (03CR) 10jerkins-bot: [V: 04-1] Don't use $row->ores_damaging_threshold in Contributions where it isn't set [extensions/ORES] - 10https://gerrit.wikimedia.org/r/390072 (https://phabricator.wikimedia.org/T179830) (owner: 10Catrope) [19:41:42] awight: halfak|Mobile https://grafana.wikimedia.org/dashboard/db/ores-extension?orgId=1 [19:41:48] Check out bottom of the page [19:41:59] It probably needs some work I think [19:43:20] Amir1: Thanks, that’ll be really useful! [19:44:13] 10Scoring-platform-team (Current), 10MediaWiki-extensions-ORES, 10Graphite, 10MW-1.31-release-notes (WMF-deploy-2017-11-14 (1.31.0-wmf.8)), and 2 others: Keep statistics about ores service hits for storing thresholds - https://phabricator.wikimedia.org/T179862#3745710 (10Ladsgroup) Bottom of the page: http... [19:45:04] I'm calling it a day [19:45:10] definitely was lots of work [19:46:13] o/ I’ll follow up on RoanKattouw’s patch [19:47:59] Arg, Trying to load up the grafana [19:48:09] Stupid wifi [19:51:14] Oh! I see it. Looks good. Will want to change the colors so falures are red :) [19:51:27] Have a good night, Amir1 :) [19:55:25] (03PS2) 10Catrope: Don't use $row->ores_damaging_threshold in Contributions where it isn't set [extensions/ORES] - 10https://gerrit.wikimedia.org/r/390072 (https://phabricator.wikimedia.org/T179830) [19:59:23] (03CR) 10jerkins-bot: [V: 04-1] Don't use $row->ores_damaging_threshold in Contributions where it isn't set [extensions/ORES] - 10https://gerrit.wikimedia.org/r/390072 (https://phabricator.wikimedia.org/T179830) (owner: 10Catrope) [20:03:42] RoanKattouw: I was thinking of making your change in all of those hooks… [20:04:27] Yeah me too [20:04:33] :D [20:04:40] I just amended my patch to fix a Jenkins failure but now it's failing a unit test [20:04:45] So I give up and am going to get lunch [20:37:14] * paladox likes the new sky q box http://www.sky.com/shop/tv/sky-q/?refererIs=skyid&ExistingCustomer=true&returnToAuthFilter=true /me is upgrading soon to it :) [20:38:58] awight have you herd of sky q? :) [20:40:21] Is Sky owned by Murdoch yet? [20:42:25] But no I haven’t heard of the box yet. User-customizable with the aid of a can opener, I hope? [20:46:35] halfak|Mobile: Remind me why we have requirements.txt in ores-prod-deploy which overlap with the submodules? e.g. ores and revscoring [20:47:53] awight heh, nope not brought by them yet [20:48:04] sky are threatning to shut down sky news [20:48:11] to get them to get the deal through [20:48:31] awight it allows you to watch tv on your mobile / tablet :) [20:49:44] hahaha thank god. Would it save us from email and senseless browsing... [20:51:04] lol [20:51:23] awight well the new box is a much much better upgrade then sky+ hd [20:51:35] you an now move multiroom boxes around. [20:51:47] and you can watch the tv on your mobile device (live tv). [20:51:52] but not outside the house though. [20:51:56] What you on about paladox [20:52:04] Zppix skyq :) [20:52:12] ? [20:52:19] Zppix http://www.sky.com/shop/tv/sky-q/?refererIs=skyid&ExistingCustomer=true&returnToAuthFilter=true [20:52:23] halfak|Mobile: hows seattle [20:52:45] grey's anatomy :) [20:53:22] You know that didnt even enter my mind paladox [20:53:28] lol [20:54:28] Paladox im shocked your not doing gci [20:54:35] What's gci? [20:54:46] Google code in [20:55:11] oh [22:19:03] awight /me has spent the last two days trying to get gerrit's ci onto newer os as ubuntu 15.04 is unsupported. lol i wasted time on updating to 16.04, now we are going to go with debian stretch :). [22:19:40] Whew…. I hate that game. [22:21:24] paladox: you mean 14.04? [22:21:31] Sagan nope [22:21:33] 15.04 [22:21:39] a very old release [22:21:40] heh [22:21:48] i am suppised it lasted this long [22:22:04] though, nodejs we had to use trusty dist [22:22:47] though it's all broken now as google discontinued support for 15.04 with chrome. So we need to migrate.