[15:11:42] 10Jade, 10Scoring-platform-team (Current), 10DBA, 10Operations, and 2 others: Introduce a new namespace for collaborative judgements about wiki entities - https://phabricator.wikimedia.org/T200297 (10kchapman) TechCom is placing this on last call ending July 6th 05:00 UTC/07:00 CEST/July 5 22:00 PDT [17:13:43] PROBLEM - check load on ORES-web02.Experimental is CRITICAL: connect to address 172.16.6.234 port 5666: Connection refusedconnect to host ores-web-02.ores.eqiad.wmflabs port 5666: Connection refused [17:14:30] PROBLEM - check users on ORES-web02.Experimental is CRITICAL: connect to address 172.16.6.234 port 5666: Connection refusedconnect to host ores-web-02.ores.eqiad.wmflabs port 5666: Connection refused [17:14:38] Hmmm [17:14:48] PROBLEM - check disk on ORES-web02.Experimental is CRITICAL: connect to address 172.16.6.234 port 5666: Connection refusedconnect to host ores-web-02.ores.eqiad.wmflabs port 5666: Connection refused [17:15:44] RECOVERY - check load on ORES-web02.Experimental is OK: OK - load average: 0.97, 1.55, 0.81 [17:16:30] RECOVERY - check users on ORES-web02.Experimental is OK: USERS OK - 1 users currently logged in [17:16:48] RECOVERY - check disk on ORES-web02.Experimental is OK: DISK OK [18:17:57] halfak: idk what those alerts for icinga2-wm were about earlier [20:08:13] No idea earlier. I assumed it was something cloud-maintenance related. [20:28:22] 10Jade, 10Gerrit, 10Repository-Admins: Can't update extensions.git because gerrit won't serve the commit at which the Jade extension is pointed? - https://phabricator.wikimedia.org/T224316 (10Jdforrester-WMF) [20:47:18] 10Scoring-platform-team, 10Data-Services, 10Wikilabels, 10Patch-For-Review, 10cloud-services-team (Kanban): clouddb1002 low on space -- move wikilabelsdb - https://phabricator.wikimedia.org/T224062 (10Bstorm) Ok, the database is moved to a new location. I have some docs to update, things to clean up on... [20:54:51] halfak: i may talk to cloud at some point and suggest better communication about maintaince [20:55:02] Yeah. I had the same thought. [20:55:11] Cause alerts with no known reason can cause unneeded worry halfak [20:56:01] halfak: i know they are under staffed (last i heard anyway) so i wont expect it every time but any effort to notify people will be appreciated [20:56:43] Unneeded worry and complacency [20:57:51] halfak: i may talk to them tomorrow via email or irc if not i will monday cause im busy tonight [20:58:03] halfak: feel free to talk to them yourself if you want aswell [20:58:25] Yeah. I think we need a solid proposal. Complaining is something they hear plenty of. [20:58:35] Maybe a post in -cloud when any maintenance is happening? [20:59:25] halfak: i mean im not complaining by any means i understand they are not a full team atm, so i simply ask that they make an effort and if they arent able to at the time then so be it :) [20:59:34] But yes a simple post on irc could work [20:59:49] Or even a short email to wikitech-l [21:01:55] halfak: cloud’s lead is bryan right? [21:02:05] yup. bd808. [21:03:09] Ok [21:03:33] Ill figure out a propsoal and ill write it up on etherpad or something and send it to you when its done for your thoughts [21:03:51] 10Scoring-platform-team, 10Data-Services, 10cloud-services-team (Kanban): Set up clouddb-wikilabels-02 as postgresql replica - https://phabricator.wikimedia.org/T224318 (10Bstorm) [21:20:14] cool Zppix thaanks [21:20:18] I'm running away for the evening. [21:20:20] Have a good one! [23:06:42] 10Scoring-platform-team, 10Data-Services, 10cloud-services-team (Kanban): Set up clouddb-wikilabels-02 as postgresql replica - https://phabricator.wikimedia.org/T224318 (10Bstorm) 05Open→03Resolved [23:06:45] 10Scoring-platform-team, 10Data-Services, 10Wikilabels, 10Patch-For-Review, 10cloud-services-team (Kanban): clouddb1002 low on space -- move wikilabelsdb - https://phabricator.wikimedia.org/T224062 (10Bstorm) [23:38:46] 10Scoring-platform-team, 10Data-Services, 10Wikilabels, 10Patch-For-Review, 10cloud-services-team (Kanban): clouddb1002 low on space -- move wikilabelsdb - https://phabricator.wikimedia.org/T224062 (10Bstorm) 05Open→03Resolved clouddb1002 is at 44% after cleanup and moving wikilabels. [23:41:39] 10Scoring-platform-team, 10Cloud-VPS, 10Wikilabels, 10Epic, and 2 others: Replace labsdb100[4567] with instances on cloudvirt1019 and cloudvirt1020 - https://phabricator.wikimedia.org/T193264 (10Bstorm)