[06:33:15] PROBLEM - check load on ORES-web01.Experimental is CRITICAL: connect to address 172.16.3.131 port 5666: Connection refusedconnect to host ores-web-01.ores.eqiad.wmflabs port 5666: Connection refused [06:34:36] PROBLEM - check disk on ORES-web01.Experimental is CRITICAL: connect to address 172.16.3.131 port 5666: Connection refusedconnect to host ores-web-01.ores.eqiad.wmflabs port 5666: Connection refused [06:34:38] PROBLEM - check users on ORES-web01.Experimental is CRITICAL: connect to address 172.16.3.131 port 5666: Connection refusedconnect to host ores-web-01.ores.eqiad.wmflabs port 5666: Connection refused [06:35:40] PROBLEM - puppet on ORES-web01.Experimental is CRITICAL: connect to address 172.16.3.131 port 5666: Connection refusedconnect to host ores-web-01.ores.eqiad.wmflabs port 5666: Connection refused [06:52:37] RECOVERY - check disk on ORES-web01.Experimental is OK: DISK OK [06:52:38] RECOVERY - check users on ORES-web01.Experimental is OK: USERS OK - 1 users currently logged in [06:53:16] RECOVERY - check load on ORES-web01.Experimental is OK: OK - load average: 0.07, 0.30, 0.73 [06:55:22] RECOVERY - puppet on ORES-web01.Experimental is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [08:22:44] PROBLEM - ORES web node labs ores-web-01 on ores.wmflabs.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/ORES [08:22:46] PROBLEM - ORES web node labs ores-web-02 on ores.wmflabs.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/ORES [08:23:10] PROBLEM - ORES worker labs on ores.wmflabs.org is CRITICAL: CRITICAL - Socket timeout after 10 seconds https://wikitech.wikimedia.org/wiki/ORES [08:25:46] RECOVERY - ORES web node labs ores-web-01 on ores.wmflabs.org is OK: HTTP OK: HTTP/1.1 200 OK - 1009 bytes in 0.049 second response time https://wikitech.wikimedia.org/wiki/ORES [08:25:48] RECOVERY - ORES web node labs ores-web-02 on ores.wmflabs.org is OK: HTTP OK: HTTP/1.1 200 OK - 979 bytes in 0.056 second response time https://wikitech.wikimedia.org/wiki/ORES [08:27:52] RECOVERY - ORES worker labs on ores.wmflabs.org is OK: HTTP OK: HTTP/1.1 200 OK - 979 bytes in 4.873 second response time https://wikitech.wikimedia.org/wiki/ORES [15:00:16] 10Jade, 10Scoring-platform-team (Current), 10User-Testing: Design Special:Diff integration for Jade - https://phabricator.wikimedia.org/T210558 (10Halfak) [16:05:34] Async Standup time -- [16:05:34] Y: changelogs, updated some of the release automation PRs, wrote down interview questions [16:05:34] T: legal training, cleanup revscoring changelog, maybe test release automation with editquality v0.5.0 [16:09:02] Last week: Worked on Jade product: Epic task and design assets. I supported the ORES deploy. I did some hiring work: : Interviews (EM and Assoc SWE), Review applications. I rewrote our Opportunity Fund pitch. I reviews the general algorithms FATML paper [16:09:02] I also reviewer paper for ToSC. I also did some random engineering work: mwoauth error page issue. [16:09:03] Y: Nothing. I took the WMF holiday. [16:09:03] T: Working on some papers. (1) The ORES systems paper. I'm hoping to resubmit it to the FATML conference. I need to improve the flow a bit and add some explicit statements to it. (2) The ORES values paper (lead by a grad student @ UMN) is headed to CHI. (3) It looks like the general algorithms paper is going to fizzle and that's OK with me. Otherwise, I'm going to work on reviewing accraze's automation and changeset PRs. [16:09:36] ToSC = Transactions of Social Computing [16:09:52] FATML = Fairness and transparency in Machine Learning [16:10:14] CHI = Transactions on Human Computer Interaction [16:14:16] cool! let me know if you want an extra pair of eyes to look over the next draft of ORES systems paper, sounds interesting. [16:16:37] Hmm. That might be welcome. I'll let you know :) [16:17:35] 10ORES, 10Scoring-platform-team (Current): Timeout for ORES - https://phabricator.wikimedia.org/T230381 (10Halfak) How much more often? What kind of access pattern are you engaging in when you see these timeouts? [16:25:29] Added two notes to https://github.com/wikimedia/draftquality/pull/30 [16:25:31] accraze, ^ [16:27:50] updated! [16:28:11] 10ORES, 10Scoring-platform-team (Current): Timeout for ORES - https://phabricator.wikimedia.org/T230381 (10Halfak) For some reason, https://grafana.wikimedia.org/d/vAN_bQemz/ores-advanced-metrics?refresh=1m&panelId=11&fullscreen&orgId=1&from=now-7d&to=now-1m doesn't seem to load for me right now. But when I... [16:37:51] I just merged the articlequality release automation, but I think that maybe we should do a smoke test of the editquality automation first. [16:37:57] So I'm working on that now. [16:41:25] awesome! [16:45:12] 10Scoring-platform-team (Current): Develop automated release strategy from travis CI - https://phabricator.wikimedia.org/T229850 (10Halfak) Test release for editquality 0.5.0: https://github.com/wikimedia/editquality/pull/213 [16:45:28] accraze, ^ [16:45:49] I updated the changelog and added the CHANGELOG.md to the changelog ^_^ [16:46:30] nice! merging [16:46:47] wikimedia/editquality#660 (test_automated_release - bad82d9 : halfak): The build passed. https://travis-ci.org/wikimedia/editquality/builds/571413177 [16:47:12] It really bothers me that "CODE OF CONDUCT.md" uses spaces and not underscores :( [16:47:41] haha yeah me too, is that required? [16:48:06] Yeah. Required by WMF policy. [16:48:29] but no underscores? [16:48:34] * halfak watches the build: https://travis-ci.org/wikimedia/editquality/builds/571414217?utm_source=github_status&utm_medium=notification [16:50:16] Uploading editquality-0.5.0-py2.py3-none-any.whl [16:50:39] BOOM [16:50:40] https://pypi.org/project/editquality/ [16:50:43] it worked [16:51:11] and there is an email notification to scoring-internal [16:51:18] via libraries.io [16:52:37] Woo! [16:52:41] Nice! [16:55:52] and the recent docs build has the changelogs added now too: https://editquality.readthedocs.io/en/latest/changelog.html [16:55:56] Cool! [17:01:48] I just merged all of the release automation PRs. [17:02:12] We should do a bunch of releases, but I'm starving and need lunch so I'll look into that when I get back if you don't beat me to it. [17:02:30] ok sounds good [17:03:06] i'm gonna go for a quick run before it gets too hot, back in a bit [18:41:24] wikimedia/revscoring#1684 (dump_cache - e308946 : Aaron Halfaker): The build passed. https://travis-ci.org/wikimedia/revscoring/builds/571460043 [18:43:41] wikimedia/revscoring#1685 (dump_cache - 715b3ab : halfak): The build passed. https://travis-ci.org/wikimedia/revscoring/builds/571461088 [19:43:20] accraze, do you think we could kick off our 1:1 a half hour early today? [19:43:33] sure, no problem! [20:22:48] Cool. I'll move the event. [20:22:55] * halfak was lost in paper editing for a bit.