[06:44:36] PROBLEM - check disk on ORES-web01.Experimental is CRITICAL: connect to address 172.16.3.131 port 5666: Connection refusedconnect to host ores-web-01.ores.eqiad.wmflabs port 5666: Connection refused [06:44:38] PROBLEM - check users on ORES-web01.Experimental is CRITICAL: connect to address 172.16.3.131 port 5666: Connection refusedconnect to host ores-web-01.ores.eqiad.wmflabs port 5666: Connection refused [06:45:16] PROBLEM - check load on ORES-web01.Experimental is CRITICAL: connect to address 172.16.3.131 port 5666: Connection refusedconnect to host ores-web-01.ores.eqiad.wmflabs port 5666: Connection refused [06:46:31] PROBLEM - puppet on ORES-web01.Experimental is CRITICAL: connect to address 172.16.3.131 port 5666: Connection refusedconnect to host ores-web-01.ores.eqiad.wmflabs port 5666: Connection refused [07:23:16] RECOVERY - check load on ORES-web01.Experimental is OK: OK - load average: 0.38, 0.16, 0.38 [07:24:37] RECOVERY - check disk on ORES-web01.Experimental is OK: DISK OK [07:24:38] RECOVERY - check users on ORES-web01.Experimental is OK: USERS OK - 1 users currently logged in [07:25:53] RECOVERY - puppet on ORES-web01.Experimental is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [09:26:43] 10Scoring-platform-team, 10Community comms and outreach, 10CommRel-Specialists-Support (Jul-Sep-2019): Outreach campaign to raise awareness of Scoring Platform - https://phabricator.wikimedia.org/T217232 (10Elitre) I commented on the linked task - while there may be points of contact etc., I wouldn't want th... [14:29:53] 10ORES, 10Scoring-platform-team (Current), 10Wikimedia-Logstash, 10observability, and 2 others: Send celery logs and events to logstash - https://phabricator.wikimedia.org/T169586 (10fgiunchedi) [14:34:50] o/ [15:23:26] PROBLEM - ORES web node labs ores-web-02 on ores.wmflabs.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/ORES [15:23:42] PROBLEM - ORES web node labs ores-web-01 on ores.wmflabs.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/ORES [15:25:30] RECOVERY - ORES web node labs ores-web-02 on ores.wmflabs.org is OK: HTTP OK: HTTP/1.1 200 OK - 1009 bytes in 0.043 second response time https://wikitech.wikimedia.org/wiki/ORES [15:25:48] RECOVERY - ORES web node labs ores-web-01 on ores.wmflabs.org is OK: HTTP OK: HTTP/1.1 200 OK - 979 bytes in 0.088 second response time https://wikitech.wikimedia.org/wiki/ORES [16:36:22] 10Scoring-platform-team (Current), 10articlequality-modeling, 10draftquality-modeling, 10drafttopic-modeling, and 2 others: Implement versioning in the model repos - https://phabricator.wikimedia.org/T229248 (10ACraze) a:03ACraze [16:37:07] 10Scoring-platform-team (Current), 10editquality-modeling, 10artificial-intelligence: Generate dataset for MSFT explainability experiements - https://phabricator.wikimedia.org/T230737 (10Halfak) [16:39:23] 10Jade, 10Scoring-platform-team (Current), 10Documentation: Implement a MW API module for interacting with Jade entities - https://phabricator.wikimedia.org/T199834 (10ACraze) a:03ACraze [16:46:12] 10ORES, 10Scoring-platform-team (Current): Investigate Timeout for ORES - https://phabricator.wikimedia.org/T230381 (10Halfak) [16:47:16] 10ORES, 10Scoring-platform-team (Current): Investigate Timeout for ORES - https://phabricator.wikimedia.org/T230381 (10Halfak) a:03Halfak [16:50:45] 10ORES, 10Scoring-platform-team (Current), 10editquality-modeling, 10artificial-intelligence: Qualitative data collection for ores bias analysis - https://phabricator.wikimedia.org/T225441 (10Halfak) @Groceryheist, you mentioned putting this on the backburner for a while. Could we move this to the backlog? [16:51:45] 10Jade, 10Scoring-platform-team (Current), 10Operations, 10TechCom, and 4 others: Deploy Jade extension MVP to production - https://phabricator.wikimedia.org/T183381 (10Halfak) [17:56:49] 10Scoring-platform-team (Current): Automate docs build for model repos - https://phabricator.wikimedia.org/T230517 (10ACraze) As mentioned in standup, two repos needed docs added. PRs are up for both of them: Draft Topic: https://github.com/wikimedia/drafttopic/pull/37 Draft Quality: https://github.com/wikime... [18:29:14] Was back but doing an interview :) [19:32:57] lunchin [19:39:16] brb changing locations [20:46:58] The Wifi is a bit iffy in a few places. [21:14:27] Working on session orientation. [21:14:33] Trying to be the right amount of clever... [21:47:12] Changing locations again. Will be back on quicker this time because I'm headed home and I know what the internet is like there. [22:14:38] ahh yes mountain town wifi [22:14:52] im pretty sure the instability is a feature, not a bug :) [22:18:39] haha! Might be. Although I was starting to cook while walking between different potential Wifi sources. [22:19:15] I think I made some good progress today on the session-orientation proposal. I'll need a couple more hours of work before I'm ready to present and discuss though. [22:19:23] DRY gets complicated sometimes! [23:10:26] ^ agreed [23:10:50] ok my family just arrived, headin out to the beach