[03:58:47] (03CR) 10Tim Starling: [C: 03+2] Update for removal of $wgActorTableSchemaMigrationStage [extensions/ORES] - 10https://gerrit.wikimedia.org/r/525602 (https://phabricator.wikimedia.org/T188327) (owner: 10Anomie) [04:11:31] (03Merged) 10jenkins-bot: Update for removal of $wgActorTableSchemaMigrationStage [extensions/ORES] - 10https://gerrit.wikimedia.org/r/525602 (https://phabricator.wikimedia.org/T188327) (owner: 10Anomie) [04:13:54] (03CR) 10jenkins-bot: Update for removal of $wgActorTableSchemaMigrationStage [extensions/ORES] - 10https://gerrit.wikimedia.org/r/525602 (https://phabricator.wikimedia.org/T188327) (owner: 10Anomie) [06:33:47] PROBLEM - check disk on ORES-web02.Experimental is CRITICAL: connect to address 172.16.6.234 port 5666: Connection refusedconnect to host ores-web-02.ores.eqiad.wmflabs port 5666: Connection refused [06:33:52] PROBLEM - check users on ORES-web02.Experimental is CRITICAL: connect to address 172.16.6.234 port 5666: Connection refusedconnect to host ores-web-02.ores.eqiad.wmflabs port 5666: Connection refused [06:35:11] PROBLEM - check load on ORES-web02.Experimental is CRITICAL: connect to address 172.16.6.234 port 5666: Connection refusedconnect to host ores-web-02.ores.eqiad.wmflabs port 5666: Connection refused [06:36:12] PROBLEM - puppet on ORES-web02.Experimental is CRITICAL: connect to address 172.16.6.234 port 5666: Connection refusedconnect to host ores-web-02.ores.eqiad.wmflabs port 5666: Connection refused [06:38:48] 10ORES, 10Scoring-platform-team, 10SRE-tools, 10Patch-For-Review: Migrate ORES pool counters to Buster - https://phabricator.wikimedia.org/T227640 (10akosiaris) p:05Triage→03Normal [07:15:46] RECOVERY - puppet on ORES-web02.Experimental is OK: OK: Puppet is currently enabled, last run 9 seconds ago with 0 failures [07:15:47] RECOVERY - check disk on ORES-web02.Experimental is OK: DISK OK [07:15:52] RECOVERY - check users on ORES-web02.Experimental is OK: USERS OK - 1 users currently logged in [07:17:11] RECOVERY - check load on ORES-web02.Experimental is OK: OK - load average: 0.05, 0.08, 0.12 [13:35:41] o/ [14:42:51] 10Scoring-platform-team (Current), 10revscoring, 10artificial-intelligence: On en.wikipedia, ref tags inserted by the shortened footnote template, {{sfn}}, are not counted in ORES features - https://phabricator.wikimedia.org/T227153 (10Halfak) There was an issue with pytest version compatibility. I've pinne... [14:42:51] 10[1] 10https://meta.wikimedia.org/wiki/Template:sfn [14:50:24] 10Scoring-platform-team (Current), 10revscoring, 10artificial-intelligence: On en.wikipedia, ref tags inserted by the shortened footnote template, {{sfn}}, are not counted in ORES features - https://phabricator.wikimedia.org/T227153 (10Halfak) OK looks like we're passing tests now. Also, I've made a PR for... [14:50:26] 10[2] 10https://meta.wikimedia.org/wiki/Template:sfn [14:52:09] wikimedia/revscoring#1664 (bump_more-itertools - e539ee4 : halfak): The build passed. https://travis-ci.org/wikimedia/revscoring/builds/565016206 [15:08:17] 10Jade, 10Scoring-platform-team (Current): Write API contract for Jade - https://phabricator.wikimedia.org/T217904 (10Halfak) I just made some notes on the patch and updated the etherpad with the new "created" field per our discussion. [16:36:07] 10Scoring-platform-team (Current), 10editquality-modeling, 10artificial-intelligence: Build revert model for glwiki - https://phabricator.wikimedia.org/T228331 (10Halfak) 05Open→03Resolved [16:41:58] 10Scoring-platform-team, 10articlequality-modeling, 10draftquality-modeling, 10drafttopic-modeling, and 2 others: Implement versioning in the model repos - https://phabricator.wikimedia.org/T229248 (10Halfak) [16:42:45] 10Scoring-platform-team, 10articlequality-modeling, 10draftquality-modeling, 10drafttopic-modeling, and 2 others: Implement versioning in the model repos - https://phabricator.wikimedia.org/T229248 (10Halfak) [16:42:47] 10Scoring-platform-team (Current), 10articlequality-modeling, 10draftquality-modeling, 10drafttopic-modeling, and 3 others: Decide on criteria for releases/versioning of model repos - https://phabricator.wikimedia.org/T228215 (10Halfak) [16:49:03] 10MediaWiki-extensions-ORES, 10Scoring-platform-team (Current), 10Patch-For-Review, 10User-Jdlrobson, 10User-Ladsgroup: Remove SpecialContributions::getForm::filters hook call - https://phabricator.wikimedia.org/T219238 (10Halfak) @Jdlrobson, it looks like we're still blocked on https://gerrit.wikimedia.... [16:49:59] 10Scoring-platform-team (Current), 10revscoring, 10artificial-intelligence: Solve version mismatch between revscoring and pytest (more-itertools) - https://phabricator.wikimedia.org/T229125 (10Halfak) https://github.com/wikimedia/revscoring/pull/444 should address this -- at least for now. [16:50:12] 10Scoring-platform-team (Current), 10revscoring, 10artificial-intelligence: Solve version mismatch between revscoring and pytest (more-itertools) - https://phabricator.wikimedia.org/T229125 (10Halfak) a:03Halfak [17:18:55] 10ORES, 10Scoring-platform-team, 10SRE-tools: Migrate ORES pool counters to Buster - https://phabricator.wikimedia.org/T227640 (10Dzahn) hosts alerting in Icinga .. needs decom script? https://icinga.wikimedia.org/cgi-bin/icinga/status.cgi?host=all&style=hostdetail&hoststatustypes=4&hostprops=2097162 [18:09:14] 10MediaWiki-extensions-ORES, 10Scoring-platform-team (Current), 10Patch-For-Review, 10User-Jdlrobson, 10User-Ladsgroup: Remove SpecialContributions::getForm::filters hook call - https://phabricator.wikimedia.org/T219238 (10Volker_E) @Halfak Yes, it will proceed. The patch itself is blocked on another pat... [18:22:15] 10MediaWiki-extensions-ORES, 10Scoring-platform-team (Current), 10Patch-For-Review, 10User-Jdlrobson, 10User-Ladsgroup: Remove SpecialContributions::getForm::filters hook call - https://phabricator.wikimedia.org/T219238 (10Halfak) Thank you, @Volker_E! No hurry. I just wanted to make sure I understood... [18:37:57] brb [19:34:31] wikimedia/revscoring#1666 (ref-count-fix - c14da93 : Andy Craze): The build passed. https://travis-ci.org/wikimedia/revscoring/builds/565141801 [20:23:33] wikimedia/revscoring#1667 (ref-count-fix - 92d8df9 : Andy Craze): The build was broken. https://travis-ci.org/wikimedia/revscoring/builds/565162873 [20:28:22] wikimedia/revscoring#1669 (ref-count-fix - 095330b : Andy Craze): The build was fixed. https://travis-ci.org/wikimedia/revscoring/builds/565166149 [20:31:06] 10Scoring-platform-team (Current), 10revscoring, 10artificial-intelligence: Wikibase references is a count of ref claims, should be reference statements - https://phabricator.wikimedia.org/T229029 (10ACraze) @Halfak here's a patch that fixes the sources feature and also adds a new feature specifically for re... [21:48:18] woops. [21:48:34] I accidentally cut a new version of revscoring right before that new version fix was merged. [21:48:40] Hmm. How to fix this... [21:49:09] accraze, I think I want to force-push master and then just cut a new version. So it'll mess up the history a bit. [21:50:05] haha I was wondering if that would happen :) [21:50:19] I think thats fine [21:52:45] OK master is gonna be weird for a sec :| [21:53:09] I should have been paying better attention [21:53:36] YES [21:53:37] it worked [21:53:43] OK making as new version :D [21:54:42] OK should be good. We'll have 2.5.1 in pypi shortly. [21:55:02] We'll need to retrain all of our models to use it because it updates a critical dependency: sklearn. [21:55:14] We'd need to retrain enwiki and wikidatawiki articlequality models anyway. [21:55:33] I'll start a batch retraining and then call it a day. [21:55:38] Sorry icebergs/glaciers. [21:56:29] * halfak waits the requisite 5 minutes for ssh connections from the university [21:56:51] I'd file a ticket with the local tech support but I bet I'd ways hours and hours on that. [22:00:17] ah yes ... university tech support :) [22:01:35] halfak do you have a script or config that does batch retraining? [22:02:34] in editquality, "make models" [22:02:38] accraze, ^ [22:02:49] And then wait forever [22:02:52] ;) [22:03:05] extra points for doing "nice make models" [22:03:24] Oh also, you have to delete all of the old models and the old cache files [22:04:11] pypi upload is just hanging. :/ [22:04:44] I think that might use ssh too? [22:05:50] Ohh. Makes sense! [22:05:56] Eventually worked :D [22:08:37] OK off to the races. I'm hitting the road. [22:08:42] Have a good evening! [22:13:16] have a good one halfak!