[01:02:09] 10Scoring-platform-team, 10editquality-modeling, 10artificial-intelligence: Split editquality repo to two repos, one with full history, one shallow - https://phabricator.wikimedia.org/T170967#3449254 (10awight) Thinking about whether these repos should share a common ancestor, I realized I don't understand t... [06:30:02] PROBLEM - https://grafana.wikimedia.org/dashboard/db/ores grafana alert on einsteinium is CRITICAL: CRITICAL: https://grafana.wikimedia.org/dashboard/db/ores is alerting: 5xx rate (Change prop) alert. [06:42:03] RECOVERY - https://grafana.wikimedia.org/dashboard/db/ores grafana alert on einsteinium is OK: OK: https://grafana.wikimedia.org/dashboard/db/ores is not alerting. [07:07:45] 10Scoring-platform-team, 10editquality-modeling, 10User-Ladsgroup, 10artificial-intelligence: Split editquality repo to two repos, one with full history, one shallow - https://phabricator.wikimedia.org/T170967#3451406 (10Ladsgroup) a:05AnotherLadsgroup>03Ladsgroup [07:07:52] 10Scoring-platform-team, 10editquality-modeling, 10User-Ladsgroup, 10artificial-intelligence: Add new data for damaging models of Persian Wikipedia - https://phabricator.wikimedia.org/T170960#3451408 (10Ladsgroup) a:05AnotherLadsgroup>03Ladsgroup [07:08:02] 10Scoring-platform-team, 10Wikilabels, 10User-Ladsgroup: Wikilabels uses deprecated api calls - https://phabricator.wikimedia.org/T170758#3451410 (10Ladsgroup) a:05AnotherLadsgroup>03Ladsgroup [08:26:22] PROBLEM - https://grafana.wikimedia.org/dashboard/db/ores grafana alert on einsteinium is CRITICAL: CRITICAL: https://grafana.wikimedia.org/dashboard/db/ores is alerting: 5xx rate (Change prop) alert. [08:31:12] RECOVERY - https://grafana.wikimedia.org/dashboard/db/ores grafana alert on einsteinium is OK: OK: https://grafana.wikimedia.org/dashboard/db/ores is not alerting. [10:15:03] 10Scoring-platform-team, 10Wikidata, 10editquality-modeling, 10User-Ladsgroup, and 2 others: Add entropy-related and uppercase-related measures to comments - https://phabricator.wikimedia.org/T170835#3451735 (10Ladsgroup) This is the new one with proper data: ``` ScikitLearnClassifier - type: GradientBoos... [11:11:24] 10Scoring-platform-team, 10Wikilabels, 10User-Ladsgroup: Apply mediawiki core styling convention on javascript files of wikilabels - https://phabricator.wikimedia.org/T169578#3402366 (10Ladsgroup) The last part: https://github.com/wiki-ai/wikilabels/pull/195 [11:20:33] 10Scoring-platform-team, 10ORES: Add regression test for bad regex in ores - https://phabricator.wikimedia.org/T171039#3451900 (10Ladsgroup) [11:23:22] 10Scoring-platform-team, 10editquality-modeling, 10User-Ladsgroup, 10artificial-intelligence: Split editquality repo to two repos, one with full history, one shallow - https://phabricator.wikimedia.org/T170967#3451929 (10Ladsgroup) Yeah, I think every time we should clean up the history in the new one. It... [11:24:19] 10Scoring-platform-team, 10ORES: Add regression test for bad regex in ores - https://phabricator.wikimedia.org/T171039#3451930 (10Ladsgroup) 05Open>03Invalid [11:24:41] 10Scoring-platform-team, 10ORES: Add regression test for bad regex in ores - https://phabricator.wikimedia.org/T171039#3451900 (10Ladsgroup) Duplicate of {T170205} [13:22:01] halfak: o/ [13:22:06] I fixed the bug for wikidata [13:22:08] o/ Amir1 [13:22:25] tldr we were feeding data in the wrong order into merge_utility [13:22:56] Oh! [13:22:59] Great! [13:51:07] o/ mavrikant_ [13:51:22] FYI we just deployed the fixed article quality model for trwiki. [14:19:21] Amir1, re. the valid argument deletion, I really want to leave it there, but if there's some strong norm in JS land, I'll concede. [14:24:21] halfak: I think so [14:24:51] as a person who works with php and javascript every day [14:31:50] o/ [14:31:56] 10Scoring-platform-team, 10editquality-modeling, 10Performance, 10artificial-intelligence: [Discuss] Should we remove all reverted models when there is a damaging one? - https://phabricator.wikimedia.org/T171059#3452588 (10Ladsgroup) [14:48:18] Amir1, gotcha. OK then, I guess. [14:54:26] halfak: rebuilt all wikidata models [14:54:30] thanks for noticing it [14:54:42] Cool :) [14:54:48] * halfak looks again [14:55:18] BTW, Amir1, did you build these models on ores-misc-01 [14:55:20] ? [14:55:30] yup [14:55:43] should I do it in another place? [14:56:02] Nope. Just making sure :) [14:56:12] Soon I'll just inspect the model to know! Mwahahaha [14:57:08] {{done}} [14:57:14] You rule, halfak! [14:57:21] I know AsimovBot ;) [14:58:22] I know that too! [15:00:45] :D [15:01:24] Next step, bad word detection (only english for now) in edit summary [15:01:53] \o/ [15:02:45] leaving for dinner [15:02:47] be back soon [15:02:51] o/ [15:02:54] o/ [16:17:52] * halfak heads to lunch [17:10:17] halfak: there? [17:10:26] Hey! Yeah. Sorry to miss you the other day [17:14:13] codezee, ^ [17:14:38] HaeB: I tried the sentiment feature on enwiki damaging, and it gave 92.3% on gradient boosting as compared to 91.3% previously, whats more interesting is GaussianNB jumped up from 65.8% to 77.8% [17:14:43] halfak: ^ [17:14:47] HaeB: sorry :P [17:15:10] codezee, that looks pretty good :) [17:15:13] halfak: does it look like something worth creating a PR? [17:15:19] I think so, yes. [17:15:30] ok [17:16:56] halfak: also, why the need for disabling the cache as mentioned by you in https://github.com/wiki-ai/ores/issues/100 ? [17:18:20] codezee, cache would contain scores without features injected. Also cache does not include the extracted features -- just the score. [17:19:32] oh, okay [17:21:44] codezee, but actually, note that we found a way to cache scores after feature injection -- so if you request a revision with the exact same cache, we'll return a cached score. [17:22:50] e.g. If you request https://ores.wikimedia.org/v3/scores/enwiki/781154471/damaging?feature.english.badwords.revision.diff.match_delta_decrease=1 [17:22:59] It will come from the cache because I just requested it. [17:23:06] I show that ORES responds in ~50ms [17:25:53] yes i see that...so I think emission of raw features should be doable, given the trim() functionality is already in place [17:40:57] codezee, does this have something to do with the sentiment work you have been doing? [17:43:12] halfak: oh, I should've mentioned the break in topic, it is an altogether different task I was looking at to familiarize myself with ORES webservice [17:43:24] Oh cool :) [17:43:47] BTW: https://github.com/wiki-ai/ores/blob/master/ores/scoring_context.py#L44 [17:43:55] awight checkout my commit for translations in gerrit https://gerrit-review.googlesource.com/#/c/114092/ heh [17:44:22] halfak: sorry to bother you, have you seen the document I shared with you? [17:44:46] Amir1, Only just now [17:45:15] halfak: The most important thing I want is to know when we will deploy it, exact date would be fantastic [17:45:43] Amir1, we could set up for a new deploy tomorrow if you wanted to do the PR [17:45:51] I can do the patchset for prod if you do a pr for wmflabs [17:46:02] halfak: I think we should do it in late July [17:46:19] That'd be OK [17:46:27] So next week? [17:46:29] for several reasons: 1- we should have a phab card and it's better to have subtasks and stuff [17:46:47] 2- I want to get bad word detection out of the door I need some days for that [17:47:03] 3- I don't have prod access yet, I hopefully will get it soon [17:47:06] :DDD Yesss. Let's get that together. [17:47:20] Understood on 3. I'll fill in the gaps for you re. ORES stuff. [17:47:49] It's better to be able to read logstash when things go south :D [17:47:55] awight: regarding the puppet recommendation-api thing, that's something I've been working with services on to get up and running [17:47:58] more pair of eyes [17:48:08] I think they would be the correct team to ask about the puppet issue [17:48:22] Amir1, agreed. Will be great to have you back at 100% [17:52:40] halfak: I have this stupid problem with google calendar which stops "Loading" mode forever [17:52:56] otherwise i would make an event for the deploy next week [17:53:03] do you want to do it? [17:53:03] https://productforums.google.com/forum/#!topic/calendar/Yo-mFogBjog [17:53:07] halfak am i needed to merge anything or anything? [17:53:38] halfak: FYI, I have my phab and gerrit back, meaning I can merge things in gerrit [17:54:03] Amir1, what day do you want to do it? [17:54:33] Tuesday looks good to me [17:54:37] When is the window? [17:54:40] Yeah sure [17:54:47] * halfak would look it up but he is in a meeting :) [17:54:56] I hope you don't mind if I use you to look things up :D [17:54:58] 17:00–18:00 UTC [17:55:02] Zppix, nothing ATM [17:55:05] ]https://wikitech.wikimedia.org/wiki/Deployments#Week_of_July_24th [17:55:08] k [17:55:18] let me know if i need to pull out the merge wand [17:55:37] lol kk [17:55:43] halfak: It's okay but you should pay me extra [17:55:48] :P [17:55:50] lol :D I'd love to [17:58:27] hey... wheres my pay xD jk [17:59:29] *sigh* if only we had a base income, it'd be easier to volunteer. [18:01:06] paladox: wow, that patch looks good! [18:01:12] yeh :) [18:01:32] english to start with. Tested with the french language locally only to see if it actually works. It does [18:02:14] bonjour [18:04:29] lol [18:04:53] I'm impressed, I tried and failed to make a tiny change to the Gerrit UI maybe two years ago. I found that my feature had to be repeated in three disparate parts of the code base, each with its own structure and apparently unrelated to the others. My final defeat came when I tried to build using "buck", with no luck. [18:05:49] lol [18:05:57] awight they switched from buck to bazel now [18:06:00] much more easier [18:06:13] schana: Nice, thanks for the note, and for pinging from the task. [18:07:23] awight you can now make contributions to the ui as they now have a ci that builds [18:07:25] everything [18:07:42] http://gerrit-ci.gerritforge.com [18:10:07] :D hey, that makes it look like much less of a trap to punish us for doing open source [18:10:59] lol \o/ [18:13:33] tried to build using "buck", with no luck , nice rhyme :D [18:14:17] halfak: awight|brb https://phabricator.wikimedia.org/T171059 [18:19:45] 10Scoring-platform-team-Backlog, 10ORES: [Spike] [bike shed] Name the Meta ORES service - https://phabricator.wikimedia.org/T153146#3453519 (10Halfak) [18:20:34] 10Scoring-platform-team-Backlog, 10ORES: [Spike] [bike shed] Name the Meta ORES service - https://phabricator.wikimedia.org/T153146#2871167 (10Halfak) I just added BiasBuster (@Tnegrin) and Assay (@bd808) to the list of ideas. [18:20:40] codezee: You don't want to know what other words I considered :p [18:21:56] 10Scoring-platform-team-Backlog, 10ORES: [Spike] [bike shed] Name the Meta ORES service - https://phabricator.wikimedia.org/T153146#3453529 (10Halfak) [18:22:22] Amir1, what do you think? [18:22:38] 10Scoring-platform-team, 10Wikilabels: The great test making of Wikilabels - https://phabricator.wikimedia.org/T171080#3453530 (10Ladsgroup) [18:22:54] Seems like a good idea. We could move forward with it. I think that we should have a good-long notice period and a campaign for notifying tool devs who we know are using ORES [18:22:59] halfak: I'm in favor of removing it but let's make an announcement and say "Does anyone use it?" [18:23:19] Also we should have something https://www.wikidata.org/wiki/Wikidata:Stable_Interface_Policy [18:23:42] 10Scoring-platform-team, 10editquality-modeling, 10Performance, 10artificial-intelligence: [Discuss] Should we remove all reverted models when there is a damaging one? - https://phabricator.wikimedia.org/T171059#3452588 (10Halfak) Seems like a good idea. We could move forward with it. I think that we sho... [18:24:13] Amir1, that's a good idea. We can at least say something about v1, v2, v3, ... of the API [18:24:17] And schemas of "score" [18:24:31] The revscoring 2.0 models will have a schema available. [18:24:38] *JSON schema [18:25:20] yeah [18:25:44] fajne: https://github.com/adamwight/ores-diagrams fwiw [18:28:41] 10Scoring-platform-team, 10Wikilabels: nosetests for wikilabels - https://phabricator.wikimedia.org/T171082#3453570 (10Ladsgroup) [18:29:02] 10Scoring-platform-team, 10Wikilabels: qunit tests for wikilabels - https://phabricator.wikimedia.org/T171083#3453583 (10Ladsgroup) [18:30:33] 10Scoring-platform-team, 10Wikilabels: linting tests for wikilabels - https://phabricator.wikimedia.org/T171084#3453597 (10Ladsgroup) [18:31:28] 10Scoring-platform-team-Backlog, 10Wikilabels: nosetests for wikilabels - https://phabricator.wikimedia.org/T171082#3453624 (10Ladsgroup) [18:31:45] 10Scoring-platform-team-Backlog, 10Wikilabels: linting tests for wikilabels - https://phabricator.wikimedia.org/T171084#3453597 (10Ladsgroup) [18:32:00] 10Scoring-platform-team-Backlog, 10Wikilabels: qunit tests for wikilabels - https://phabricator.wikimedia.org/T171083#3453626 (10Ladsgroup) [18:33:45] https://blog.wikimedia.org/2017/07/19/scoring-platform-team/ [18:33:49] We're live! [18:33:52] ALIVE [18:36:40] 10Scoring-platform-team, 10MediaWiki-extensions-ORES: Make list of features and locations of ORES Review Tool for handoff - https://phabricator.wikimedia.org/T167911#3453641 (10Ladsgroup) [18:40:16] yeah! [18:41:48] * Amir1 dances [18:41:52] Looks really good all blogged up! [18:43:31] > developing new types of AI predictions, and pushing the state of the art with regards to ethical practice of AI development [18:44:01] it looks like a goal for the whole AI community in general :P [18:44:21] armalcolite, I don't think that many people are really pushing on ethical practice. [18:44:27] If they are, they aren't publishing about it. [18:44:42] They are more -- critiquing without trying new stuff. [18:45:29] halfak: ah that is something I have been reading about nowadays in lots of reviews [18:46:42] the focus should eventually shift [18:46:43] brb [18:53:17] fajne: https://github.com/wiki-ai/draftquality/pull/3 https://phabricator.wikimedia.org/T167305#3421459 [19:11:51] halfak: hi! how do i open .model ? [19:12:18] I think we're looking for the canonical way to run utility model_info [19:12:19] (to mess it up of course :) [19:17:05] fajne: revscoring model_info models/ruwiki.goodfaith.gradient_boosting.model [19:17:27] however, I don't have a great answer about how to manage the python dependencies to make that work correctly. [19:50:26] \o/ [19:50:32] Looks like y'all got it worked out [19:50:46] awight, if you do "pip install revscoring" [19:51:02] and then installed the relevant library (in this case "editquality") [19:51:06] then it should work as intended. [19:51:48] https://phabricator.wikimedia.org/T171080 [19:52:39] halfak: awight: I'm about to send this out. Feel free to edit mercilessly https://etherpad.wikimedia.org/p/yHMkIGU2iq [19:52:46] Amir1, love the name [19:52:59] tell me when you're done [19:53:29] halfak: well, I stole it from https://phabricator.wikimedia.org/T166010 [19:56:25] 10Scoring-platform-team, 10editquality-modeling, 10Performance, 10RfC, 10artificial-intelligence: [RfC] Should we remove all reverted models when there is a damaging one? - https://phabricator.wikimedia.org/T171059#3454006 (10Ladsgroup) [19:57:00] naughty naughty amir [19:58:51] :D [20:03:41] halfak: kk. I did pip install -r requirements.txt in editquality, then python setup.py develop to link the dev repo into the lib path. [20:04:43] whens the next deploy from scoring team? [20:07:40] Zppix, next Tuesday [20:07:51] 1700 UTC [20:08:00] ok ty [20:08:15] Amir1, awight: I'm done editing https://etherpad.wikimedia.org/p/yHMkIGU2iq [20:08:42] whats this for halk? [20:08:45] halfak* [20:09:05] If you read through it, you'll see :) [20:09:38] Basically, all models take up memory. The 'damaging' model is better than the 'reverted' model wherever we have it available. So, let's remove the 'reverted' models and free up some memory. [20:11:26] halfak: I added a sentence to point (1) [20:15:26] Thanks! [20:16:38] I'm sending this out [20:16:49] awight can you login into http://gerrit-icinga.wmflabs.org/authentication/login ? [20:16:58] there's ldap issues so i carn't seem to login in [20:17:23] sure, lemme see... [20:17:31] thanks :) [20:17:52] Sent [20:18:08] paladox: I can log in. Did you want me to do anything? [20:18:21] nope, just wanted to see if it worked for you :) [20:18:23] e.g. set your password? [20:18:24] ok [20:18:32] thanks [20:18:37] for doing that :) [20:18:49] it's the least I can do! [20:19:02] :) [20:21:51] i belive puppet will start throwing errors soon [20:21:58] i just got this [20:21:59] PROBLEM - puppet on jenkins-slave-01.git.eqiad.wmflabs is CRITICAL: CRITICAL: Puppet has 2 failures. Last run 1 minute ago with 2 failures. Failed resources (up to 3 shown): File[/mnt/home/jenkins-deploy],File[/mnt/jenkins-workspace] [20:24:13] this may be the best comment i've seen ever: [03:22]<+wikibugs> (CR) Reedy: "jerkins is a dick though :(" [mediawiki-config] [20:25:32] Yeah... so is travis [20:26:38] halfak now that i can agree on [20:29:25] halfak: one of fawiki users was asking for edit type model/etc. for fawiki, i don't know how that will be useful but he wants it [20:29:35] he wants the campaign to be open [20:29:40] just fyi [20:30:11] OK. We should have a task. If he can recruit people to do the labeling, then we can build the model! [20:33:05] PROBLEM - puppet on ores-redis-02 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): File[/var/lib/prometheus/node.d] [20:33:18] here we go [20:33:22] awight ^^ lol [20:33:32] (was not me) [20:33:37] uhh [20:33:47] happily, it will take an hour(?) before bothering us again [20:33:56] halfak awight ldap is having problems [20:33:57] uhh good question [20:33:59] see operations [20:34:04] gotcha [20:34:13] * halfak ignores [20:34:24] PROBLEM - puppet on ores-worker-05 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): File[/var/lib/prometheus/node.d] [20:34:57] i can ack them all if you want halfak [20:35:08] I will go to sleep, see you tomorrow [20:35:09] o/ [20:35:13] o/ [20:35:16] have a good night! [20:35:34] Zppix, it's OK. Just so long as they don't flood the channel; [20:35:46] i dont think they will [20:35:49] paladox? [20:35:59] Zppix they were fixed [20:36:04] to only do it every 40 mins [20:36:12] so not much noise as long as they fix the issue soon [20:36:13] ok [20:36:23] i think 40 mins is a reasonable length [20:36:33] yep [20:38:49] +1 [20:46:09] Zppix halfak awight the CA expired for ldap [20:46:26] they said it was good til nov 2017 [20:46:33] wups [20:47:04] lol [20:47:13] they are building a new one now [20:47:22] how do you know this? [20:51:28] 10Scoring-platform-team, 10editquality-modeling, 10Performance, 10RfC, 10artificial-intelligence: [RfC] Should we remove all reverted models when there is a damaging one? - https://phabricator.wikimedia.org/T171059#3452588 (10Catrope) Sounds good to me! Of course I would say that, though, because our sof... [20:51:30] PROBLEM - puppet on ores-worker-07 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): File[/var/lib/prometheus/node.d] [20:54:51] PROBLEM - puppet on ores-lb-02 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 41 seconds ago with 1 failures. Failed resources (up to 3 shown): File[/var/lib/prometheus/node.d] [20:56:21] PROBLEM - puppet on ores-web-05 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): File[/var/lib/prometheus/node.d] [21:05:33] PROBLEM - puppet on ores-redis-01 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): File[/var/lib/prometheus/node.d] [21:05:43] PROBLEM - puppet on ores-worker-10 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 52 seconds ago with 1 failures. Failed resources (up to 3 shown): File[/var/lib/prometheus/node.d] [21:08:31] PROBLEM - puppet on ores-worker-09 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): File[/var/lib/prometheus/node.d] [21:13:10] PROBLEM - puppet on ores-redis-02 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 10 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/var/lib/prometheus/node.d] [21:13:21] PROBLEM - puppet on ores-worker-06 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): File[/var/lib/prometheus/node.d] [21:14:25] PROBLEM - puppet on ores-worker-05 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 10 minutes ago with 1 failures. Failed resources (up to 3 shown): File[/var/lib/prometheus/node.d] [21:14:49] PROBLEM - puppet on ores-worker-08 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 42 seconds ago with 1 failures. Failed resources (up to 3 shown): File[/var/lib/prometheus/node.d] [21:18:45] PROBLEM - puppet on ores-web-03 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 1 minute ago with 1 failures. Failed resources (up to 3 shown): File[/var/lib/prometheus/node.d] [21:31:30] PROBLEM - puppet on ores-worker-07 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [21:33:43] RECOVERY - puppet on ores-worker-05 is OK: OK: Puppet is currently enabled, last run 30 seconds ago with 0 failures [21:34:52] RECOVERY - puppet on ores-redis-01 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures [21:34:55] PROBLEM - puppet on ores-lb-02 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [21:36:03] OMG spam [21:36:12] RECOVERY - puppet on ores-worker-10 is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures [21:36:18] halfak it should start recoverying soon [21:36:24] ldap has a new cert [21:36:25] PROBLEM - puppet on ores-web-05 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [21:36:31] so it has to be installed every where [21:37:07] gotcha [21:38:08] RECOVERY - puppet on ores-worker-09 is OK: OK: Puppet is currently enabled, last run 51 seconds ago with 0 failures [21:42:42] RECOVERY - puppet on ores-worker-06 is OK: OK: Puppet is currently enabled, last run 42 seconds ago with 0 failures [21:44:07] RECOVERY - puppet on ores-worker-08 is OK: OK: Puppet is currently enabled, last run 8 seconds ago with 0 failures [21:48:04] RECOVERY - puppet on ores-web-03 is OK: OK: Puppet is currently enabled, last run 48 seconds ago with 0 failures [21:50:17] RECOVERY - puppet on ores-worker-07 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [21:53:15] PROBLEM - puppet on ores-redis-02 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [21:54:10] RECOVERY - puppet on ores-lb-02 is OK: OK: Puppet is currently enabled, last run 23 seconds ago with 0 failures [21:54:42] RECOVERY - puppet on ores-web-05 is OK: OK: Puppet is currently enabled, last run 2 seconds ago with 0 failures [21:55:44] hehe this spam is a helpful cover for my lack of output today. [21:59:10] lol [22:01:34] * awight pushes another VPS under the bus [22:02:25] RECOVERY - puppet on ores-redis-02 is OK: OK: Puppet is currently enabled, last run 28 seconds ago with 0 failures [22:03:31] lol [22:07:59] awight we live to serve :P [22:08:40] did you do that paladox? [22:12:12] * halfak eyes up awight's outputs [22:12:17] ;) [22:12:58] * Zppix slips halfak a $5 on behalf of awight [22:13:31] Zppix yes [22:14:38] * halfak looks the other way [22:14:40] ;) [22:16:26] awight you owe me 5 dollars [22:19:47] lol [22:19:57] I'm off for the evening. Have a good one, folks! [22:20:12] smh halfak your never off :P [22:23:25] 10Scoring-platform-team-Backlog, 10Wikilabels, 10articlequality-modeling, 10artificial-intelligence: Build article quality model for euwiki - https://phabricator.wikimedia.org/T171119#3454661 (10Theklan) [23:05:45] RoanKattouw: https://github.com/mediawiki-utilities/python-mwreverts/blob/master/mwreverts/api.py [23:50:32] halfak: the log report you asked https://meta.wikimedia.org/wiki/Research_talk:Automated_classification_of_edit_quality/Work_log/2017-07-19 [23:51:55] Wow, nice write-up!