[02:47:57] 10DBA, 10Jade, 10Operations, 10TechCom-RFC, and 3 others: Introduce a new namespace for collaborative judgements about wiki entities - https://phabricator.wikimedia.org/T200297 (10Harej) The secondary schema that was requested was addressed in T202596. I think at one point the schema was submitted to @Maro... [05:16:01] 10DBA, 10Operations, 10ops-eqiad, 10Goal, and 2 others: rack/setup/install db11[26-38].eqiad.wmnet - https://phabricator.wikimedia.org/T211613 (10Marostegui) I have increased the priority cause s4 master is having memory errors again and needs to be replaced as soon as we can [08:05:17] 10DBA, 10Operations, 10Patch-For-Review: correctable memory errors db1068 (commons primary master database) - https://phabricator.wikimedia.org/T213664 (10Marostegui) Thanks for letting us know! This master will be replaced once the hosts at {T211613} are racked and installed. [08:07:17] 10DBA, 10Operations: Predictive failures on disk S.M.A.R.T. status - https://phabricator.wikimedia.org/T208323 (10Marostegui) db2047 has another disk failed: ` logicaldrive 1 (3.3 TB, RAID 1+0, OK) physicaldrive 1I:1:1 (port 1I:box 1:bay 1, SAS, 600 GB, Predictive Failure) physicaldrive 1I:1... [14:40:20] 10DBA, 10Cloud-Services: Prepare and check storage layer for initiativeswiki - https://phabricator.wikimedia.org/T221201 (10Krinkle) [14:54:31] 10DBA, 10MediaWiki-Database, 10Operations, 10Wikimedia-production-error: Special:Log on commons -- entire web request took longer than 60 seconds and timed out - https://phabricator.wikimedia.org/T221458 (10CDanis) [14:58:25] 10DBA, 10MediaWiki-Database, 10Operations, 10Wikimedia-production-error: Special:Log on commons -- entire web request took longer than 60 seconds and timed out - https://phabricator.wikimedia.org/T221458 (10Framawiki) Is {T221380} related? [15:55:45] 10DBA, 10Cloud-Services: Prepare and check storage layer for initiativeswiki - https://phabricator.wikimedia.org/T221201 (10Marostegui) p:05Triage→03Normal [16:19:29] 10DBA, 10MediaWiki-Database, 10MediaWiki-Logging, 10Operations, 10Wikimedia-production-error: Special:Log on commons -- entire web request took longer than 60 seconds and timed out - https://phabricator.wikimedia.org/T221458 (10Aklapper) [16:24:52] 10DBA: questions about standalone wmf-mariadb103 - https://phabricator.wikimedia.org/T221463 (10Andrew) [16:30:23] 10DBA: questions about standalone wmf-mariadb103 - https://phabricator.wikimedia.org/T221463 (10Andrew) Huh, the socket is /tmp/systemd-private-838b9005a03d4e24b1649b6ecbae7bb3-mariadb.service-rKXi7L/tmp/mysql.sock Unexpected! [16:30:34] 10DBA: questions about standalone wmf-mariadb103 - https://phabricator.wikimedia.org/T221463 (10Krenair) For the record I ran into this at T219087 and basically my way around it was `find /tmp -name mysql.sock`, which returned something like `/tmp/systemd-private-d6c71da3465641b3aa68e8390a2cc75c-mariadb.service... [16:37:54] 10DBA: questions about standalone wmf-mariadb103 - https://phabricator.wikimedia.org/T221463 (10Andrew) [22:34:54] 10DBA, 10Operations, 10ops-codfw: Degraded RAID on db2047 - https://phabricator.wikimedia.org/T221481 (10colewhite) p:05Triage→03High