[00:00:59] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/c155b89f88896b2da444ff3ea03b8f52eb5cf4d3 [00:00:59] 02statichelp/03main 07WikiTideBot 03c155b89 Bot: Auto-update Tech namespace pages 2026-10-19 00:00:56 [00:09:15] RECOVERY - mon181 Puppet on mon181 is OK: OK: Puppet is currently enabled, last run 11 seconds ago with 0 failures [00:13:21] [02puppet] 07Universal-Omega pushed 1 new commit to 03export 13https://github.com/miraheze/puppet/commit/7ce907b6a1209a48ce210fcf24530a376a1de7bb [00:14:13] 02puppet/03export 07CosmicAlpha 037ce907b Update [00:14:15] [02puppet] 07Universal-Omega pushed 1 new commit to 03export 13https://github.com/miraheze/puppet/commit/273d1de171cd57747f5e1d7226ef119a8de1b5a5 [00:14:15] 02puppet/03export 07CosmicAlpha 03273d1de Fix [00:14:34] [02puppet] 07Universal-Omega pushed 1 new commit to 03export 13https://github.com/miraheze/puppet/commit/6cdb57ac855c7f1b620ed972b83f585c57e214dc [00:14:34] 02puppet/03export 07CosmicAlpha 036cdb57a Fix [00:25:25] [02puppet] 07Universal-Omega retitled PR #5082: "monitoring: use export" ➜ "monitoring: collect icinga objects with icinga2::query_objects" 13https://github.com/miraheze/puppet/pull/5082 [00:26:04] [02puppet] 07Universal-Omega merged pull request #5082: monitoring: collect icinga objects with icinga2::query_objects (03main...03export) 13https://github.com/miraheze/puppet/pull/5082 [00:26:06] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/035d25ae111f109534a564950e4c6640bbd0217d [00:26:08] [02puppet] 07Universal-Omega 04deleted 03export at 036cdb57a 13https://api.github.com/repos/miraheze/puppet/commit/6cdb57a [00:26:10] 02puppet/03main 07CosmicAlpha 03035d25a monitoring: collect icinga objects with icinga2::query_objects… [00:27:17] RECOVERY - mon181 Puppet on mon181 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [00:27:25] RECOVERY - mon181 IRCEcho on mon181 is OK: PROCS OK: 1 process with args '/usr/local/bin/ircecho' [00:58:55] RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 83903MiB (18.5% inode=100%); [01:00:27] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/c4d19e30eaa88a46e3cdef3f2b284329fefff397 [01:00:27] 02ssl/03main 07WikiTideBot 03c4d19e3 Bot: Auto-update domain lists [01:10:26] miraheze/MirahezeMagic - ccxtwf the build passed. [01:20:52] PROBLEM - cp161 Puppet on cp161 is WARNING: WARNING: Puppet last ran 1 hour ago [01:20:56] PROBLEM - changeprop202 Puppet on changeprop202 is WARNING: WARNING: Puppet last ran 1 hour ago [01:21:08] PROBLEM - db193 Puppet on db193 is WARNING: WARNING: Puppet last ran 1 hour ago [01:21:08] PROBLEM - cloud21 Puppet on cloud21 is WARNING: WARNING: Puppet last ran 1 hour ago [01:21:08] PROBLEM - cloud18 Puppet on cloud18 is WARNING: WARNING: Puppet last ran 1 hour ago [01:21:26] PROBLEM - cp201 Puppet on cp201 is WARNING: WARNING: Puppet last ran 1 hour ago [01:21:29] PROBLEM - db161 Puppet on db161 is WARNING: WARNING: Puppet last ran 1 hour ago [01:21:34] PROBLEM - bast181 Puppet on bast181 is WARNING: WARNING: Puppet last ran 1 hour ago [01:21:43] PROBLEM - cloud15 Puppet on cloud15 is WARNING: WARNING: Puppet last ran 1 hour ago [01:21:43] PROBLEM - cloud20 Puppet on cloud20 is WARNING: WARNING: Puppet last ran 1 hour ago [01:21:44] PROBLEM - db171 Puppet on db171 is WARNING: WARNING: Puppet last ran 1 hour ago [01:21:50] PROBLEM - db151 Puppet on db151 is WARNING: WARNING: Puppet last ran 1 hour ago [01:21:50] PROBLEM - bots171 Puppet on bots171 is WARNING: WARNING: Puppet last ran 1 hour ago [01:21:51] PROBLEM - cloud17 Puppet on cloud17 is WARNING: WARNING: Puppet last ran 1 hour ago [01:21:53] PROBLEM - db192 Puppet on db192 is WARNING: WARNING: Puppet last ran 1 hour ago [01:21:53] PROBLEM - cp171 Puppet on cp171 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:03] PROBLEM - db201 Puppet on db201 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:07] PROBLEM - cloud19 Puppet on cloud19 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:16] PROBLEM - bast161 Puppet on bast161 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:19] PROBLEM - cloud16 Puppet on cloud16 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:25] PROBLEM - cp191 Puppet on cp191 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:26] PROBLEM - db172 Puppet on db172 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:36] PROBLEM - db181 Puppet on db181 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:36] PROBLEM - mem201 Puppet on mem201 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:38] PROBLEM - mem151 Puppet on mem151 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:38] PROBLEM - eventgate182 Puppet on eventgate182 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:41] PROBLEM - mw161 Puppet on mw161 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:42] PROBLEM - db182 Puppet on db182 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:44] PROBLEM - mw172 Puppet on mw172 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:44] PROBLEM - mw163 Puppet on mw163 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:51] PROBLEM - mw181 Puppet on mw181 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:56] PROBLEM - matomo211 Puppet on matomo211 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:57] PROBLEM - graylog161 Puppet on graylog161 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:58] PROBLEM - mw151 Puppet on mw151 is WARNING: WARNING: Puppet last ran 1 hour ago [01:22:58] PROBLEM - mw183 Puppet on mw183 is WARNING: WARNING: Puppet last ran 1 hour ago [01:23:12] PROBLEM - mw182 Puppet on mw182 is WARNING: WARNING: Puppet last ran 1 hour ago [01:23:13] PROBLEM - mem161 Puppet on mem161 is WARNING: WARNING: Puppet last ran 1 hour ago [01:23:22] PROBLEM - matomo151 Puppet on matomo151 is WARNING: WARNING: Puppet last ran 1 hour ago [01:23:27] PROBLEM - kafka182 Puppet on kafka182 is WARNING: WARNING: Puppet last ran 1 hour ago [01:23:43] PROBLEM - mw171 Puppet on mw171 is WARNING: WARNING: Puppet last ran 1 hour ago [01:23:44] PROBLEM - mw152 Puppet on mw152 is WARNING: WARNING: Puppet last ran 1 hour ago [01:23:51] PROBLEM - mw192 Puppet on mw192 is WARNING: WARNING: Puppet last ran 1 hour ago [01:23:56] PROBLEM - mw191 Puppet on mw191 is WARNING: WARNING: Puppet last ran 1 hour ago [01:23:57] PROBLEM - llm191 Puppet on llm191 is WARNING: WARNING: Puppet last ran 1 hour ago [01:24:01] PROBLEM - mem191 Puppet on mem191 is WARNING: WARNING: Puppet last ran 1 hour ago [01:24:04] PROBLEM - ldap171 Puppet on ldap171 is WARNING: WARNING: Puppet last ran 1 hour ago [01:24:04] PROBLEM - mw162 Puppet on mw162 is WARNING: WARNING: Puppet last ran 1 hour ago [01:24:08] PROBLEM - mattermost2 Puppet on mattermost2 is WARNING: WARNING: Puppet last ran 1 hour ago [01:24:09] PROBLEM - mw153 Puppet on mw153 is WARNING: WARNING: Puppet last ran 1 hour ago [01:24:22] PROBLEM - os191 Puppet on os191 is WARNING: WARNING: Puppet last ran 1 hour ago [01:24:24] PROBLEM - prometheus151 Puppet on prometheus151 is WARNING: WARNING: Puppet last ran 1 hour ago [01:24:25] PROBLEM - speedscope211 Puppet on speedscope211 is WARNING: WARNING: Puppet last ran 1 hour ago [01:24:40] PROBLEM - mw201 Puppet on mw201 is WARNING: WARNING: Puppet last ran 1 hour ago [01:24:43] PROBLEM - mwtask161 Puppet on mwtask161 is WARNING: WARNING: Puppet last ran 1 hour ago [01:24:43] PROBLEM - mw193 Puppet on mw193 is WARNING: WARNING: Puppet last ran 1 hour ago [01:24:45] PROBLEM - mw173 Puppet on mw173 is WARNING: WARNING: Puppet last ran 1 hour ago [01:24:57] PROBLEM - os201 Puppet on os201 is WARNING: WARNING: Puppet last ran 1 hour ago [01:24:59] PROBLEM - phorge171 Puppet on phorge171 is WARNING: WARNING: Puppet last ran 1 hour ago [01:25:04] PROBLEM - swiftobject201 Puppet on swiftobject201 is WARNING: WARNING: Puppet last ran 1 hour ago [01:25:06] PROBLEM - rdb191 Puppet on rdb191 is WARNING: WARNING: Puppet last ran 1 hour ago [01:25:09] PROBLEM - swiftac171 Puppet on swiftac171 is WARNING: WARNING: Puppet last ran 1 hour ago [01:25:10] PROBLEM - ns2 Puppet on ns2 is WARNING: WARNING: Puppet last ran 1 hour ago [01:25:20] PROBLEM - ns1 Puppet on ns1 is WARNING: WARNING: Puppet last ran 1 hour ago [01:25:23] PROBLEM - os161 Puppet on os161 is WARNING: WARNING: Puppet last ran 1 hour ago [01:25:26] PROBLEM - os151 Puppet on os151 is WARNING: WARNING: Puppet last ran 1 hour ago [01:25:28] PROBLEM - mw203 Puppet on mw203 is WARNING: WARNING: Puppet last ran 1 hour ago [01:25:31] PROBLEM - reports171 Puppet on reports171 is WARNING: WARNING: Puppet last ran 1 hour ago [01:25:35] PROBLEM - swiftobject151 Puppet on swiftobject151 is WARNING: WARNING: Puppet last ran 1 hour ago [01:25:38] PROBLEM - mw202 Puppet on mw202 is WARNING: WARNING: Puppet last ran 1 hour ago [01:25:40] PROBLEM - os162 Puppet on os162 is WARNING: WARNING: Puppet last ran 1 hour ago [01:25:46] PROBLEM - mwtask151 Puppet on mwtask151 is WARNING: WARNING: Puppet last ran 1 hour ago [01:26:00] PROBLEM - swiftobject191 Puppet on swiftobject191 is WARNING: WARNING: Puppet last ran 1 hour ago [01:26:01] PROBLEM - swiftobject181 Puppet on swiftobject181 is WARNING: WARNING: Puppet last ran 1 hour ago [01:26:03] PROBLEM - mwtask171 Puppet on mwtask171 is WARNING: WARNING: Puppet last ran 1 hour ago [01:26:04] PROBLEM - swiftobject161 Puppet on swiftobject161 is WARNING: WARNING: Puppet last ran 1 hour ago [01:26:05] PROBLEM - mwtask181 Puppet on mwtask181 is WARNING: WARNING: Puppet last ran 1 hour ago [01:26:09] PROBLEM - swiftproxy171 Puppet on swiftproxy171 is WARNING: WARNING: Puppet last ran 1 hour ago [01:26:13] PROBLEM - os202 Puppet on os202 is WARNING: WARNING: Puppet last ran 1 hour ago [01:26:29] PROBLEM - swiftobject211 Puppet on swiftobject211 is WARNING: WARNING: Puppet last ran 1 hour ago [01:26:38] PROBLEM - puppet181 Puppet on puppet181 is WARNING: WARNING: Puppet last ran 1 hour ago [01:26:41] PROBLEM - swiftproxy161 Puppet on swiftproxy161 is WARNING: WARNING: Puppet last ran 1 hour ago [01:26:43] PROBLEM - swiftobject171 Puppet on swiftobject171 is WARNING: WARNING: Puppet last ran 1 hour ago [01:27:26] PROBLEM - test151 Puppet on test151 is WARNING: WARNING: Puppet last ran 1 hour ago [01:29:13] PROBLEM - mon181 Puppet on mon181 is WARNING: WARNING: Puppet last ran 1 hour ago [04:18:26] [Grafana] FIRING: The mediawiki JobQueue backlog is increasing by more than 100 jobs a minute over an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1792379870000&orgId=1&to=1792383506679 [06:06:46] miraheze/MirahezeMagic - RhinosF1 the build has errored. [06:12:20] [02MirahezeMagic] 07Universal-Omega approved pull request #675 13https://github.com/miraheze/MirahezeMagic/pull/675#pullrequestreview-5567563245 [06:12:49] [02MirahezeMagic] 07Universal-Omega merged 07ccxtwf's pull request #675: Maintenance Script: Clear real names from user table (07miraheze:03main...07ccxtwf:03maint/clear-real-names) 13https://github.com/miraheze/MirahezeMagic/pull/675 [06:12:51] [02MirahezeMagic] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/MirahezeMagic/commit/7c7ed72dc4e0ef1a0661b7860c5fcb55e64672bb [06:12:52] 02MirahezeMagic/03main 07ccxtwf 037c7ed72 Maintenance Script: Clear real names from user table (#675)… [06:16:29] miraheze/MirahezeMagic - Universal-Omega the build has errored. [06:28:26] [Grafana] RESOLVED: MediaWiki Jobs are rapidly increasing https://grafana.wikitide.net/d/GtxbP1Xnk?from=1792387550000&orgId=1&to=1792391150000 [06:34:44] !log [universalomega@test151] sudo -u www-data php /srv/mediawiki/1.47/maintenance/run.php MirahezeMagic:UpgradeWiki --wiki=exttestwikibeta --json=/srv/mediawiki/1.47.json --change-version (START) [06:34:45] !log [universalomega@test151] sudo -u www-data php /srv/mediawiki/1.47/maintenance/run.php MirahezeMagic:UpgradeWiki --wiki=exttestwikibeta --json=/srv/mediawiki/1.47.json --change-version (END - exit=0; time=1s) [06:34:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019063444-3128d45f [06:34:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019063448-f61dcba3 [06:42:19] PROBLEM - test151 MediaWiki Rendering on test151 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 1052 bytes in 0.170 second response time [06:45:48] [02mw-config] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/2439fc44037434272e6684cd44b21a9641d03846 [06:45:48] 02mw-config/03main 07CosmicAlpha 032439fc4 index.php: support 1.47 [06:46:19] RECOVERY - test151 MediaWiki Rendering on test151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.411 second response time [06:46:36] !log [universalomega@test151] Starting deploy of {'pull': 'config', 'config': True} to test151 [06:46:37] !log [universalomega@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 1s [06:46:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019064636-ab9b245d [06:46:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019064639-b90d12cf [06:46:53] miraheze/mw-config - Universal-Omega the build passed. [06:50:57] [02mw-config] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/08e377b699e5ac3ee93e0bf584c7c95f3dfc283f [06:50:57] 02mw-config/03main 07CosmicAlpha 0308e377b index.php: drop unnecessary parentheses… [06:51:12] !log [universalomega@test151] Starting deploy of {'pull': 'config', 'config': True} to test151 [06:51:13] !log [universalomega@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 0s [06:51:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019065112-ab9b245d [06:51:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019065114-dc45bc63 [06:52:06] miraheze/mw-config - Universal-Omega the build passed. [07:24:23] [Grafana] FIRING: The estimated time for the MediaWiki JobQueue to clear is excessively high (8 hours) for an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1792391030000&orgId=1&to=1792394663352 [07:34:23] [Grafana] RESOLVED: MediaWiki JobQueue is stalled https://grafana.wikitide.net/d/GtxbP1Xnk?from=1792391570000&orgId=1&to=1792395170000 [08:30:24] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/4ffd558e0e280d9d03b7ea36e4300c28c8aa3983 [08:30:24] 02ssl/03main 07WikiTideBot 034ffd558 Bot: Auto-update domain lists [09:10:37] RECOVERY - mon181 Puppet on mon181 is OK: OK: Puppet is currently enabled, last run 21 seconds ago with 0 failures [09:15:19] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/0ef7ec7c8e2a738fbc4e6f4245f83696165872a9 [09:15:19] 02puppet/03main 07CosmicAlpha 030ef7ec7 base: fix puppet-run timer [09:17:23] RECOVERY - cp161 Puppet on cp161 is OK: OK: Puppet is currently enabled, last run 0 seconds ago with 0 failures [09:17:28] RECOVERY - cp201 Puppet on cp201 is OK: OK: Puppet is currently enabled, last run 5 seconds ago with 0 failures [09:17:29] RECOVERY - db161 Puppet on db161 is OK: OK: Puppet is currently enabled, last run 11 seconds ago with 0 failures [09:17:33] RECOVERY - bast181 Puppet on bast181 is OK: OK: Puppet is currently enabled, last run 11 seconds ago with 0 failures [09:17:35] RECOVERY - db171 Puppet on db171 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures [09:17:39] RECOVERY - cloud20 Puppet on cloud20 is OK: OK: Puppet is currently enabled, last run 21 seconds ago with 0 failures [09:17:41] RECOVERY - bast161 Puppet on bast161 is OK: OK: Puppet is currently enabled, last run 20 seconds ago with 0 failures [09:17:42] RECOVERY - bots171 Puppet on bots171 is OK: OK: Puppet is currently enabled, last run 15 seconds ago with 0 failures [09:17:43] RECOVERY - cp171 Puppet on cp171 is OK: OK: Puppet is currently enabled, last run 21 seconds ago with 0 failures [09:17:44] RECOVERY - cloud15 Puppet on cloud15 is OK: OK: Puppet is currently enabled, last run 28 seconds ago with 0 failures [09:17:51] RECOVERY - cloud17 Puppet on cloud17 is OK: OK: Puppet is currently enabled, last run 33 seconds ago with 0 failures [09:17:58] RECOVERY - db182 Puppet on db182 is OK: OK: Puppet is currently enabled, last run 36 seconds ago with 0 failures [09:18:07] RECOVERY - cloud19 Puppet on cloud19 is OK: OK: Puppet is currently enabled, last run 49 seconds ago with 0 failures [09:18:11] RECOVERY - cp191 Puppet on cp191 is OK: OK: Puppet is currently enabled, last run 48 seconds ago with 0 failures [09:18:12] RECOVERY - cloud21 Puppet on cloud21 is OK: OK: Puppet is currently enabled, last run 57 seconds ago with 0 failures [09:18:12] RECOVERY - db181 Puppet on db181 is OK: OK: Puppet is currently enabled, last run 49 seconds ago with 0 failures [09:18:22] RECOVERY - db172 Puppet on db172 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:18:26] RECOVERY - db193 Puppet on db193 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:18:33] RECOVERY - db192 Puppet on db192 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:18:35] RECOVERY - db201 Puppet on db201 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:18:38] RECOVERY - changeprop202 Puppet on changeprop202 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:18:42] RECOVERY - cloud18 Puppet on cloud18 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:18:47] RECOVERY - cloud16 Puppet on cloud16 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:18:54] RECOVERY - mem161 Puppet on mem161 is OK: OK: Puppet is currently enabled, last run 1 second ago with 0 failures [09:18:57] RECOVERY - matomo211 Puppet on matomo211 is OK: OK: Puppet is currently enabled, last run 1 second ago with 0 failures [09:18:58] RECOVERY - mem201 Puppet on mem201 is OK: OK: Puppet is currently enabled, last run 4 seconds ago with 0 failures [09:19:03] RECOVERY - eventgate182 Puppet on eventgate182 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:19:18] RECOVERY - db151 Puppet on db151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:19:21] RECOVERY - mw152 Puppet on mw152 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [09:19:31] RECOVERY - kafka182 Puppet on kafka182 is OK: OK: Puppet is currently enabled, last run 36 seconds ago with 0 failures [09:19:33] RECOVERY - mw192 Puppet on mw192 is OK: OK: Puppet is currently enabled, last run 14 seconds ago with 0 failures [09:19:44] RECOVERY - mw153 Puppet on mw153 is OK: OK: Puppet is currently enabled, last run 28 seconds ago with 0 failures [09:19:44] RECOVERY - mw171 Puppet on mw171 is OK: OK: Puppet is currently enabled, last run 27 seconds ago with 0 failures [09:19:44] RECOVERY - llm191 Puppet on llm191 is OK: OK: Puppet is currently enabled, last run 47 seconds ago with 0 failures [09:19:46] RECOVERY - ldap171 Puppet on ldap171 is OK: OK: Puppet is currently enabled, last run 51 seconds ago with 0 failures [09:19:51] RECOVERY - mattermost2 Puppet on mattermost2 is OK: OK: Puppet is currently enabled, last run 56 seconds ago with 0 failures [09:19:57] RECOVERY - mw173 Puppet on mw173 is OK: OK: Puppet is currently enabled, last run 37 seconds ago with 0 failures [09:20:02] RECOVERY - mw191 Puppet on mw191 is OK: OK: Puppet is currently enabled, last run 44 seconds ago with 0 failures [09:20:07] RECOVERY - mw151 Puppet on mw151 is OK: OK: Puppet is currently enabled, last run 52 seconds ago with 0 failures [09:20:11] RECOVERY - graylog161 Puppet on graylog161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:20:12] RECOVERY - mw161 Puppet on mw161 is OK: OK: Puppet is currently enabled, last run 54 seconds ago with 0 failures [09:20:15] RECOVERY - mw182 Puppet on mw182 is OK: OK: Puppet is currently enabled, last run 54 seconds ago with 0 failures [09:20:22] RECOVERY - mem151 Puppet on mem151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:20:26] RECOVERY - mem191 Puppet on mem191 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:20:40] RECOVERY - mw183 Puppet on mw183 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:20:41] RECOVERY - matomo151 Puppet on matomo151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:20:42] RECOVERY - mw172 Puppet on mw172 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:20:46] RECOVERY - mw163 Puppet on mw163 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:20:52] RECOVERY - mw181 Puppet on mw181 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:21:05] RECOVERY - os151 Puppet on os151 is OK: OK: Puppet is currently enabled, last run 21 seconds ago with 0 failures [09:21:06] RECOVERY - mw162 Puppet on mw162 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:21:15] RECOVERY - os202 Puppet on os202 is OK: OK: Puppet is currently enabled, last run 11 seconds ago with 0 failures [09:21:19] RECOVERY - mw203 Puppet on mw203 is OK: OK: Puppet is currently enabled, last run 4 seconds ago with 0 failures [09:21:20] RECOVERY - os201 Puppet on os201 is OK: OK: Puppet is currently enabled, last run 19 seconds ago with 0 failures [09:21:21] RECOVERY - phorge171 Puppet on phorge171 is OK: OK: Puppet is currently enabled, last run 21 seconds ago with 0 failures [09:21:24] RECOVERY - os161 Puppet on os161 is OK: OK: Puppet is currently enabled, last run 23 seconds ago with 0 failures [09:21:26] RECOVERY - swiftobject151 Puppet on swiftobject151 is OK: OK: Puppet is currently enabled, last run 25 seconds ago with 0 failures [09:21:26] RECOVERY - puppet181 Puppet on puppet181 is OK: OK: Puppet is currently enabled, last run 2 seconds ago with 0 failures [09:21:35] RECOVERY - speedscope211 Puppet on speedscope211 is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures [09:21:55] RECOVERY - ns1 Puppet on ns1 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:22:11] RECOVERY - swiftproxy171 Puppet on swiftproxy171 is OK: OK: Puppet is currently enabled, last run 29 seconds ago with 0 failures [09:22:12] RECOVERY - swiftac171 Puppet on swiftac171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:22:13] RECOVERY - swiftproxy161 Puppet on swiftproxy161 is OK: OK: Puppet is currently enabled, last run 33 seconds ago with 0 failures [09:22:15] RECOVERY - mwtask171 Puppet on mwtask171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:22:15] RECOVERY - ns2 Puppet on ns2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:22:20] RECOVERY - swiftobject201 Puppet on swiftobject201 is OK: OK: Puppet is currently enabled, last run 38 seconds ago with 0 failures [09:22:20] RECOVERY - mwtask181 Puppet on mwtask181 is OK: OK: Puppet is currently enabled, last run 32 seconds ago with 0 failures [09:22:20] RECOVERY - swiftobject211 Puppet on swiftobject211 is OK: OK: Puppet is currently enabled, last run 45 seconds ago with 0 failures [09:22:21] RECOVERY - mw193 Puppet on mw193 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:22:22] RECOVERY - swiftobject191 Puppet on swiftobject191 is OK: OK: Puppet is currently enabled, last run 49 seconds ago with 0 failures [09:22:27] RECOVERY - os191 Puppet on os191 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:22:28] RECOVERY - rdb191 Puppet on rdb191 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:22:35] RECOVERY - swiftobject171 Puppet on swiftobject171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:22:38] RECOVERY - reports171 Puppet on reports171 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:22:41] RECOVERY - os162 Puppet on os162 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:22:43] RECOVERY - mwtask161 Puppet on mwtask161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:22:45] RECOVERY - mw201 Puppet on mw201 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:22:46] RECOVERY - prometheus151 Puppet on prometheus151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:22:57] RECOVERY - mw202 Puppet on mw202 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:22:58] RECOVERY - swiftobject161 Puppet on swiftobject161 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:23:03] RECOVERY - swiftobject181 Puppet on swiftobject181 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:23:03] RECOVERY - mwtask151 Puppet on mwtask151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [09:23:35] PROBLEM - test151 Puppet on test151 is CRITICAL: CHECK_NRPE STATE CRITICAL: Socket timeout after 60 seconds. [09:25:04] PROBLEM - test151 SSH on test151 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:25:12] PROBLEM - test151 Puppet on test151 is WARNING: WARNING: Puppet last ran 9 hours ago [09:25:53] RECOVERY - test151 Puppet on test151 is OK: OK: Puppet is currently enabled, last run 6 seconds ago with 0 failures [09:26:19] PROBLEM - test151 Current Load on test151 is CRITICAL: LOAD CRITICAL - total load average: 27.10, 26.99, 11.85 [09:27:01] RECOVERY - test151 SSH on test151 is OK: SSH OK - OpenSSH_10.0p2 (protocol 2.0) [09:28:19] RECOVERY - test151 Current Load on test151 is OK: LOAD OK - total load average: 3.87, 18.13, 10.43 [10:55:03] !log [jenny@mwtask171] curl -X DELETE https://opensearch-mw.wikitide.net/kamikoiwiki_content https://opensearch-mw.wikitide.net/kamikoiwiki_general [10:55:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019105504-02dbe751 [10:55:18] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php CirrusSearch:UpdateSearchIndexConfig --wiki=kamikoiwiki --startOver (END - exit=0) (T16345) [10:55:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019105518-37834b00 [10:56:14] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php CirrusSearch:ForceSearchIndex --wiki=kamikoiwiki (END - exit=0) (T16345) [10:56:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019105614-083b6e4d [12:00:26] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/c646dbe6620449c6d19d91161ce1292a7301761c [12:00:26] 02ssl/03main 07WikiTideBot 03c646dbe Bot: Auto-update domain lists [12:02:41] [02CreateWiki] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/CreateWiki/commit/5dfd45d1a39aaf5d4acb9a29ed33a128ee1caf82 [12:02:41] 02CreateWiki/03main 07translatewiki.net 035dfd45d Localisation updates from https://translatewiki.net. [12:02:43] [02MirahezeMagic] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/MirahezeMagic/commit/22e48ae03e027f18c216df48a86b13f92fee82dd [12:02:45] [02WikiDiscover] 07translatewiki pushed 1 new commit to 03main 13https://github.com/miraheze/WikiDiscover/commit/b5635a6b61b9d6779ebac35dde4fa0d14f758c12 [12:02:47] 02MirahezeMagic/03main 07translatewiki.net 0322e48ae Localisation updates from https://translatewiki.net. [12:02:48] 02WikiDiscover/03main 07translatewiki.net 03b5635a6 Localisation updates from https://translatewiki.net. [12:06:01] miraheze/MirahezeMagic - translatewiki the build has errored. [12:11:02] miraheze/CreateWiki - translatewiki the build passed. [13:36:20] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?from=1792413350000&orgId=1&to=1792416980035 [13:41:20] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?from=1792413350000&orgId=1&to=1792417280036[Grafana] FIRING: The mediawiki JobQueue backlog is increasing by more than 100 jobs a minute over an extended time period https://grafana.wikitide.net/d/GtxbP1Xnk?from=1792413530000&orgId=1&to=1792417280036 [13:48:19] PROBLEM - mw183 Current Load on mw183 is CRITICAL: LOAD CRITICAL - total load average: 24.70, 20.45, 17.43 [13:48:37] [02puppet] 07paladox created 03icinga2-module-upgrade (+1 new commit) 13https://github.com/miraheze/puppet/commit/5db61efa5128 [13:48:37] 02puppet/03icinga2-module-upgrade 07paladox 035db61ef Update icinga module to 7.0.0 [13:48:48] [02puppet] 07paladox opened pull request #5083: Update icinga module to 7.0.0 (03main...03icinga2-module-upgrade) 13https://github.com/miraheze/puppet/pull/5083 [13:49:05] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 22.30, 19.48, 16.85 [13:49:16] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.95, 18.79, 16.03 [13:49:57] [02puppet] 07paladox merged pull request #5083: Update icinga module to 7.0.0 (03main...03icinga2-module-upgrade) 13https://github.com/miraheze/puppet/pull/5083 [13:49:59] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/ea1fd462ac5ad20f024bb82333fc651647990010 [13:50:01] [02puppet] 07paladox 04deleted 03icinga2-module-upgrade at 035db61ef 13https://api.github.com/repos/miraheze/puppet/commit/5db61ef [13:50:02] 02puppet/03main 07paladox 03ea1fd46 Update icinga module to 7.0.0 (#5083) [13:52:42] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 22.06, 20.18, 17.19 [13:54:13] [02puppet] 07paladox created 03paladox-patch-3 (+1 new commit) 13https://github.com/miraheze/puppet/commit/b48f2f126b95 [13:54:13] 02puppet/03paladox-patch-3 07paladox 03b48f2f1 Update hosts.pp [13:54:14] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 23.61, 23.10, 19.67 [13:54:18] PROBLEM - mw201 Current Load on mw201 is WARNING: LOAD WARNING - total load average: 21.51, 19.81, 16.54 [13:54:18] [02puppet] 07paladox opened pull request #5084: Update hosts.pp (03main...03paladox-patch-3) 13https://github.com/miraheze/puppet/pull/5084 [13:54:45] [02puppet] 07paladox pushed 1 new commit to 03paladox-patch-3 13https://github.com/miraheze/puppet/commit/43b2cdc2c3775319c0c8d39619342e7573fca480 [13:54:46] 02puppet/03paladox-patch-3 07paladox 0343b2cdc Add ensure parameter to icinga2 service definition… [13:55:33] [02puppet] 07paladox closed pull request #5084: Update hosts.pp (03main...03paladox-patch-3) 13https://github.com/miraheze/puppet/pull/5084 [13:55:36] [02puppet] 07paladox 04deleted 03paladox-patch-3 at 0343b2cdc 13https://api.github.com/repos/miraheze/puppet/commit/43b2cdc [13:55:50] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 20.17, 20.29, 17.67 [13:56:13] RECOVERY - mw201 Current Load on mw201 is OK: LOAD OK - total load average: 19.46, 19.61, 16.85 [13:58:05] PROBLEM - mw183 Current Load on mw183 is CRITICAL: LOAD CRITICAL - total load average: 24.50, 23.70, 20.66 [14:00:00] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 19.45, 22.33, 20.53 [14:02:33] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 15.74, 19.24, 18.45 [14:03:51] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 16.80, 19.55, 19.84 [14:04:36] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 19.03, 20.38, 19.64 [14:06:20] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?from=1792413350000&orgId=1&to=1792418780039[Grafana] RESOLVED: MediaWiki Jobs are rapidly increasing https://grafana.wikitide.net/d/GtxbP1Xnk?from=1792415090000&orgId=1&to=1792418690000 [14:09:33] !log [jenny@mwtask171] curl -X DELETE https://opensearch-mw.wikitide.net/aunsroomwiki_content https://opensearch-mw.wikitide.net/aunsroomwiki_general [14:09:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019140934-5dc6b6b9 [14:09:48] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php CirrusSearch:UpdateSearchIndexConfig --wiki=aunsroomwiki --startOver (END - exit=0) [14:09:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019140948-a831054d [14:11:20] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?from=1792415390000&orgId=1&to=1792418990000[Grafana] RESOLVED: MediaWiki Jobs are rapidly increasing https://grafana.wikitide.net/d/GtxbP1Xnk?from=1792415090000&orgId=1&to=1792418690000 [14:13:52] [02mw-config] 07dependabot[bot] created 03dependabot/composer/phpunit/phpunit-13.4.3 (+1 new commit) 13https://github.com/miraheze/mw-config/commit/7ebd115c4f9b [14:13:52] 02mw-config/03dependabot/composer/phpunit/phpunit-13.4.3 07dependabot[bot] 037ebd115 Update phpunit/phpunit requirement from 13.4.2 to 13.4.3… [14:13:55] [02mw-config] 07dependabot[bot] opened pull request #6602: Update phpunit/phpunit requirement from 13.4.2 to 13.4.3 (03main...03dependabot/composer/phpunit/phpunit-13.4.3) 13https://github.com/miraheze/mw-config/pull/6602 [14:13:56] [02mw-config] 07dependabot[bot] added the label 'dependencies' to pull request #6602 (Update phpunit/phpunit requirement from 13.4.2 to 13.4.3) 13https://github.com/miraheze/mw-config/pull/6602 [14:13:57] [02mw-config] 07dependabot[bot] added the label 'php' to pull request #6602 (Update phpunit/phpunit requirement from 13.4.2 to 13.4.3) 13https://github.com/miraheze/mw-config/pull/6602 [14:14:56] miraheze/mw-config - dependabot[bot] the build passed. [14:15:25] !log [jenny@mwtask181] Starting import for fliplinestudioswiki (XML: None; Images: images) (START) [14:15:26] !log [jenny@mwtask181] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importImages --wiki=fliplinestudioswiki --sleep=1 '--comment=Importing images from https://fliplinestudios.fandom.com ([[phorge:T16254|T16254]])' -- images (START) [14:15:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019141525-d6142282 [14:15:29] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019141527-b0218417 [14:34:11] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php CirrusSearch:ForceSearchIndex --wiki=aunsroomwiki (END - exit=0) [14:34:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019143411-cc6f4d56 [14:42:16] !log [jenny@mwtask171] Starting import for harrypotterzhwiki (XML: t.xml; Images: None) (START) [14:42:17] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importDump --wiki=harrypotterzhwiki --no-updates --username-prefix=wikia:harrypotter:zh -- t.xml (START) [14:42:19] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019144216-c0d32455 [14:42:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019144219-99ef04c4 [14:57:04] PROBLEM - swift-lb.wikitide.net - LetsEncrypt on sslhost is WARNING: WARNING - Certificate 'wikitide.net' expires in 14 day(s) (Tue 03 Nov 2026 02:40:17 PM GMT +0000). [15:00:28] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/49d844ea188fc68c220d368909c4995086e62b4e [15:00:28] 02ssl/03main 07WikiTideBot 0349d844e Bot: Auto-update domain lists [15:10:31] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importDump --wiki=harrypotterzhwiki --no-updates --username-prefix=wikia:harrypotter:zh -- t.xml (END - exit=0) [15:10:32] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php rebuildall --wiki=harrypotterzhwiki (START) [15:10:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019151031-4b697474 [15:10:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019151034-9682671f [16:05:50] [02dns] 07MacFan4000 pushed 1 new commit to 03main 13https://github.com/miraheze/dns/commit/d3701cc8ae397530419a233ba28d695a37e9abd7 [16:05:50] 02dns/03main 07MacFan4000 03d3701cc add ACME challenges for wikitide.net [16:07:11] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/a3fb5db1a217d076414b5972b2bca7ec2f57b7aa [16:07:11] 02ssl/03main 07WikiTideBot 03a3fb5db Bot: Update SSL cert for wikitide.net [16:27:22] RECOVERY - graylog161 graylog tls port 12210 ssl cert check on graylog161 is OK: OK - Certificate 'wikitide.net' will expire on Sun Jan 17 15:08:30 2027 +0000.TCP OK - 0.085 second response time on localhost port 12210 [16:30:34] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/a08c4e836aace7c70f8a6c4237103a349be369dd [16:30:34] 02ssl/03main 07WikiTideBot 03a08c4e8 Bot: Auto-update domain lists [16:34:46] !log installed nethogs on mon181 and puppet181 [16:34:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019163446-f368a8a1 [16:50:17] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php rebuildall --wiki=harrypotterzhwiki (END - exit=0) [16:50:18] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php initEditCount --wiki=harrypotterzhwiki (START) [16:50:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019165017-4993cd1a [16:50:20] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php initEditCount --wiki=harrypotterzhwiki (END - exit=0) [16:50:21] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php initSiteStats --wiki=harrypotterzhwiki --update (START) [16:50:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019165020-eaa1fec9 [16:50:22] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php initSiteStats --wiki=harrypotterzhwiki --update (END - exit=0) [16:50:23] !log [jenny@mwtask171] Finished import for harrypotterzhwiki (XML: t.xml; Images: None) (END - exit=0) [16:50:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019165022-18a675e1 [16:50:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019165024-9a61e968 [16:50:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019165026-26fd81d4 [16:50:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019165028-e5fa89c6 [17:04:35] !log [jenny@mwtask171] Starting import for harrypotterzhwiki (XML: None; Images: images/) (START) [17:04:36] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importImages --wiki=harrypotterzhwiki --sleep=1 '--comment=Importing images from https://harrypotter.fandom.com/zh ([[phorge:T16314|T16314]])' -- images/ (START) [17:04:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170435-9aa18efd [17:04:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170437-d8606a56 [17:04:40] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importImages --wiki=harrypotterzhwiki --sleep=1 '--comment=Importing images from https://harrypotter.fandom.com/zh ([[phorge:T16314|T16314]])' -- images/ (END - exit=-2) [17:04:41] !log [jenny@mwtask171] Finished import for harrypotterzhwiki (XML: None; Images: images/) (END - exit=-2) [17:04:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170440-1bf9afd7 [17:04:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170442-0fe3bf85 [17:05:09] !log [jenny@mwtask171] Starting import for harrypotterzhwiki (XML: None; Images: images) (START) [17:05:10] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importImages --wiki=harrypotterzhwiki --sleep=1 '--comment=Importing images from https://harrypotter.fandom.com/zh ([[phorge:T16314|T16314]])' -- images (START) [17:05:11] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170509-3b5e1efb [17:05:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170511-633f046a [17:05:23] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importImages --wiki=harrypotterzhwiki --sleep=1 '--comment=Importing images from https://harrypotter.fandom.com/zh ([[phorge:T16314|T16314]])' -- images (END - exit=-2) [17:05:24] !log [jenny@mwtask171] Finished import for harrypotterzhwiki (XML: None; Images: images) (END - exit=-2) [17:05:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170523-0483dff1 [17:05:28] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170525-997153f6 [17:06:13] !log [jenny@mwtask171] Starting import for harrypotterzhwiki (XML: None; Images: images) (START) [17:06:14] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importImages --wiki=harrypotterzhwiki --sleep=1 '--comment=Importing images from https://harrypotter.fandom.com/zh ([[phorge:T16314|T16314]])' -- images (START) [17:06:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170613-3b5e1efb [17:06:17] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importImages --wiki=harrypotterzhwiki --sleep=1 '--comment=Importing images from https://harrypotter.fandom.com/zh ([[phorge:T16314|T16314]])' -- images (END - exit=-2) [17:06:18] !log [jenny@mwtask171] Finished import for harrypotterzhwiki (XML: None; Images: images) (END - exit=-2) [17:06:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170615-633f046a [17:06:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170618-0483dff1 [17:06:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170620-997153f6 [17:08:10] !log [jenny@mwtask171] Starting import for harrypotterzhwiki (XML: None; Images: images) (START) [17:08:10] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importImages --wiki=harrypotterzhwiki --sleep=1 '--comment=Importing images from https://harrypotter.fandom.com/zh ([[phorge:T16314|T16314]])' -- images (START) [17:08:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170810-3b5e1efb [17:08:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170812-633f046a [17:09:44] !log [jenny@mwtask171] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importImages --wiki=harrypotterzhwiki --sleep=1 '--comment=Importing images from https://harrypotter.fandom.com/zh ([[phorge:T16314|T16314]])' -- images (END - exit=-2) [17:09:45] !log [jenny@mwtask171] Finished import for harrypotterzhwiki (XML: None; Images: images) (END - exit=-2) [17:09:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170944-0483dff1 [17:09:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019170946-997153f6 [17:10:21] [1/3] every image fails to import, with a message like ``` [17:10:21] [2/3] Importing 国际魔法合作司.jpg...failed. (The file images/国际魔法合作司.jpg does not exist.) [17:10:22] [3/3] ``` [17:30:39] !log bounce nginx on swiftproxy and mattermost [17:30:41] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019173039-c525d7b2 [17:33:05] !log bounce slapd on ldap* [17:33:07] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019173305-8fd7e32f [17:56:27] RECOVERY - swift-lb.wikitide.net - LetsEncrypt on sslhost is OK: OK - Certificate 'wikitide.net' will expire on Sun 17 Jan 2027 03:08:30 PM GMT +0000. [19:53:01] [02puppet] 07Universal-Omega created 03facts (+1 new commit) 13https://github.com/miraheze/puppet/commit/6b5a3b2e7a07 [19:53:01] 02puppet/03facts 07CosmicAlpha 036b5a3b2 Fully replace direct usages of legacy facts [19:53:07] [02puppet] 07Universal-Omega opened pull request #5085: Fully replace direct usages of legacy facts (03main...03facts) 13https://github.com/miraheze/puppet/pull/5085 [19:53:58] [02puppet] 07Universal-Omega pushed 1 new commit to 03facts 13https://github.com/miraheze/puppet/commit/cddc00a2937dfe84340adab348a3a695e4d4a888 [19:53:59] 02puppet/03facts 07CosmicAlpha 03cddc00a Update [19:54:28] [02puppet] 07Universal-Omega pushed 1 new commit to 03facts 13https://github.com/miraheze/puppet/commit/c965b258908b98c8faccdccb94a75a7b26b21517 [19:54:28] 02puppet/03facts 07CosmicAlpha 03c965b25 Update [19:55:17] [02puppet] 07Universal-Omega pushed 1 new commit to 03facts 13https://github.com/miraheze/puppet/commit/91ef8d26fc4b4b2695da855213bf48fc49aab98a [19:55:17] 02puppet/03facts 07CosmicAlpha 0391ef8d2 Update [19:56:05] [02puppet] 07Universal-Omega pushed 1 new commit to 03facts 13https://github.com/miraheze/puppet/commit/50f5a34354db1ac2a66df9091a04774c1146e3cc [19:56:05] 02puppet/03facts 07CosmicAlpha 0350f5a34 Update [19:57:12] [02puppet] 07Universal-Omega pushed 1 new commit to 03facts 13https://github.com/miraheze/puppet/commit/21e57b8798ed4dbccd2eb3c4c251d1cbbd258bf9 [19:57:12] 02puppet/03facts 07CosmicAlpha 0321e57b8 Update [20:40:46] !log fixed networking issues on db193 and matomo211 by changing /24 to /8 in /etc/network/interfaces [20:40:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019204046-0cc2a0c2 [20:47:09] PROBLEM - matomo211 conntrack_table_size on matomo211 is CRITICAL: connect to address 10.0.21.173 port 5666: No route to hostconnect to host 10.0.21.173 port 5666: No route to host [20:47:19] PROBLEM - matomo211 SSH on matomo211 is CRITICAL: connect to address 10.0.21.173 and port 22: No route to host [20:47:45] PROBLEM - Host matomo211 is DOWN: CRITICAL - Host Unreachable (10.0.21.173) [20:51:39] RECOVERY - Host matomo211 is UP: PING OK - Packet loss = 0%, RTA = 0.21 ms [20:51:59] PROBLEM - matomo211 Check unit status of matomo-queuedtracking-2 on matomo211 is CRITICAL: CRITICAL: Status of the systemd unit matomo-queuedtracking-2 [20:52:08] PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 45153MiB (10.0% inode=100%); [20:52:24] PROBLEM - matomo211 Puppet on matomo211 is CRITICAL: connect to address 10.0.21.173 port 5666: Connection refusedconnect to host 10.0.21.173 port 5666: Connection refused [20:52:44] PROBLEM - matomo211 HTTPS on matomo211 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 7 - Failed to connect to matomo211.fsslc.wtnet port 443 after 1007 ms: Could not connect to server [20:53:09] RECOVERY - matomo211 conntrack_table_size on matomo211 is OK: OK: nf_conntrack is 1 % full [20:53:14] RECOVERY - matomo211 SSH on matomo211 is OK: SSH OK - OpenSSH_10.0p2 (protocol 2.0) [20:53:54] RECOVERY - matomo211 Check unit status of matomo-queuedtracking-2 on matomo211 is OK: OK: Status of the systemd unit matomo-queuedtracking-2 [20:54:43] RECOVERY - matomo211 HTTPS on matomo211 is OK: HTTP OK: HTTP/2 200 - 553 bytes in 0.426 second response time [20:56:18] RECOVERY - matomo211 Puppet on matomo211 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [21:31:27] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/fa5614b24f808ad5a276cfcbe7ac0582439ab4d1 [21:31:27] 02puppet/03main 07CosmicAlpha 03fa5614b Rename puppet_major_version and remove default [21:33:20] [02mw-config] 07Universal-Omega merged 07dependabot[bot]'s pull request #6602: Update phpunit/phpunit requirement from 13.4.2 to 13.4.3 (03main...03dependabot/composer/phpunit/phpunit-13.4.3) 13https://github.com/miraheze/mw-config/pull/6602 [21:33:20] [02mw-config] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/e654d95acbfee57b6532dfa4d3a4bfdd5f2ba63a [21:33:21] 02mw-config/03main 07dependabot[bot] 03e654d95 Update phpunit/phpunit requirement from 13.4.2 to 13.4.3 (#6602)… [21:33:23] [02mw-config] 07Universal-Omega 04deleted 03dependabot/composer/phpunit/phpunit-13.4.3 at 037ebd115 13https://api.github.com/repos/miraheze/mw-config/commit/7ebd115 [21:34:21] miraheze/mw-config - Universal-Omega the build passed. [22:06:14] PROBLEM - cp191 Disk Space on cp191 is WARNING: DISK WARNING - free space: / 44510MiB (9.8% inode=100%); [22:08:14] RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 90232MiB (19.9% inode=100%); [22:17:05] !log [jenny@mwtask171] fixconts --wiki hwrwiki [22:17:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019221705-2cb75459 [22:18:10] !log [jenny@mwtask171] Wiki reset jiggery-pokery for hwrwiki, also with sudo MediaWiki\MediaWikiServices::getInstance()->getRevisionStore()->getSlotRowsForBatch( [ 1 ] ); [22:18:12] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019221810-8a578349 [22:22:46] !log [jenny@mwtask181] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importImages --wiki=fliplinestudioswiki --sleep=1 '--comment=Importing images from https://fliplinestudios.fandom.com ([[phorge:T16254|T16254]])' -- images (END - exit=-2) [22:22:47] !log [jenny@mwtask181] Finished import for fliplinestudioswiki (XML: None; Images: images) (END - exit=-2) [22:22:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019222246-b53b1dff [22:22:50] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019222248-7a13bf29 [22:25:11] !log [jenny@mwtask181] Starting import for fliplinestudioswiki (XML: None; Images: images) (START) [22:25:12] !log [jenny@mwtask181] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importImages --wiki=fliplinestudioswiki --sleep=1 '--comment=Importing images from https://fliplinestudios.fandom.com ([[phorge:T16254|T16254]])' -- images (START) [22:25:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019222511-d6142282 [22:25:16] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019222514-b0218417 [22:28:59] PROBLEM - cp171 Disk Space on cp171 is WARNING: DISK WARNING - free space: / 44550MiB (9.8% inode=100%); [22:30:43] !log [universalomega@puppet181] Upgraded packages on cloud21: cifs-utils [22:30:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019223043-3b60bcdc [22:30:48] RECOVERY - cloud21 APT on cloud21 is OK: APT OK: 178 packages available for upgrade (0 critical updates). [22:31:01] !log [universalomega@puppet181] Upgraded packages on cloud16: cifs-utils [22:31:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019223101-9d87b921 [22:31:18] !log [universalomega@puppet181] Upgraded packages on cloud15: cifs-utils [22:31:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019223118-df4606db [22:31:37] !log [universalomega@puppet181] Upgraded packages on cloud20: cifs-utils [22:31:39] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019223137-6e863dd3 [22:31:56] !log [universalomega@puppet181] Upgraded packages on cloud19: cifs-utils [22:31:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019223156-3d7ba5fb [22:32:02] RECOVERY - cloud15 APT on cloud15 is OK: APT OK: 186 packages available for upgrade (0 critical updates). [22:32:11] RECOVERY - cloud19 APT on cloud19 is OK: APT OK: 186 packages available for upgrade (0 critical updates). [22:32:15] !log [universalomega@puppet181] Upgraded packages on cloud17: cifs-utils [22:32:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019223215-c6af7b91 [22:32:34] !log [universalomega@puppet181] Upgraded packages on cloud18: cifs-utils [22:32:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019223234-6a1ab55b [22:32:54] RECOVERY - cloud16 APT on cloud16 is OK: APT OK: 180 packages available for upgrade (0 critical updates). [22:33:18] RECOVERY - cloud20 APT on cloud20 is OK: APT OK: 186 packages available for upgrade (0 critical updates). [22:33:21] !log [universalomega@puppet181] Upgraded packages on changeprop202: haproxy [22:33:23] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019223321-983dfca5 [22:33:46] RECOVERY - changeprop202 APT on changeprop202 is OK: APT OK: 115 packages available for upgrade (0 critical updates). [22:33:54] !log [universalomega@puppet181] Upgraded packages on cp191: haproxy [22:33:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019223354-3b080d13 [22:33:59] RECOVERY - cloud17 APT on cloud17 is OK: APT OK: 180 packages available for upgrade (0 critical updates). [22:34:01] RECOVERY - cloud18 APT on cloud18 is OK: APT OK: 180 packages available for upgrade (0 critical updates). [22:34:16] RECOVERY - cp191 APT on cp191 is OK: APT OK: 108 packages available for upgrade (0 critical updates). [22:34:25] !log [universalomega@puppet181] Upgraded packages on cp201: haproxy [22:34:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019223425-7ff68fec [22:34:37] RECOVERY - cp201 APT on cp201 is OK: APT OK: 108 packages available for upgrade (0 critical updates). [22:34:47] !log [universalomega@puppet181] Upgraded packages on cp171: haproxy [22:34:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019223447-909e232a [22:34:50] RECOVERY - cp171 APT on cp171 is OK: APT OK: 108 packages available for upgrade (0 critical updates). [22:36:59] RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 89583MiB (19.7% inode=100%); [22:37:45] RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 47054MiB (10.4% inode=100%); [22:54:59] PROBLEM - mw153 Current Load on mw153 is CRITICAL: LOAD CRITICAL - total load average: 38.67, 26.52, 15.82 [22:55:23] PROBLEM - mw163 Current Load on mw163 is CRITICAL: LOAD CRITICAL - total load average: 37.73, 27.82, 16.96 [22:57:19] PROBLEM - mw163 Current Load on mw163 is WARNING: LOAD WARNING - total load average: 16.76, 23.78, 16.79 [22:58:59] PROBLEM - mw153 Current Load on mw153 is WARNING: LOAD WARNING - total load average: 9.39, 20.51, 16.22 [22:59:05] [02puppet] 07Universal-Omega created 03os3 (+1 new commit) 13https://github.com/miraheze/puppet/commit/24bb8ffee7d0 [22:59:05] 02puppet/03os3 07CosmicAlpha 0324bb8ff Support OpenSearch 3 [22:59:11] [02puppet] 07Universal-Omega opened pull request #5086: Support OpenSearch 3 (03main...03os3) 13https://github.com/miraheze/puppet/pull/5086 [22:59:15] RECOVERY - mw163 Current Load on mw163 is OK: LOAD OK - total load average: 8.70, 18.61, 15.72 [23:00:45] [02puppet] 07Universal-Omega pushed 1 new commit to 03os3 13https://github.com/miraheze/puppet/commit/1ea0484e9bc4761d938e43ee5034d4004360ce53 [23:00:46] 02puppet/03os3 07CosmicAlpha 031ea0484 Update [23:00:59] RECOVERY - mw153 Current Load on mw153 is OK: LOAD OK - total load average: 8.71, 16.46, 15.25 [23:01:21] [02puppet] 07Universal-Omega pushed 1 new commit to 03os3 13https://github.com/miraheze/puppet/commit/c75eae1cd305bd5fe7a35be04ca5085136c13941 [23:01:21] 02puppet/03os3 07CosmicAlpha 03c75eae1 Update [23:02:02] [02puppet] 07Universal-Omega pushed 1 new commit to 03os3 13https://github.com/miraheze/puppet/commit/7a46038c884bdcf40757cfef7736aa06a35c52ce [23:02:02] 02puppet/03os3 07CosmicAlpha 037a46038 Update [23:03:25] [02puppet] 07Universal-Omega pushed 1 new commit to 03os3 13https://github.com/miraheze/puppet/commit/d502758e7e5e823f6a6beec29b306ba8d6061c3a [23:03:25] 02puppet/03os3 07CosmicAlpha 03d502758 Update [23:11:42] !log [jenny@mwtask181] sudo -u www-data php /srv/mediawiki/1.46/maintenance/run.php importImages --wiki=fliplinestudioswiki --sleep=1 '--comment=Importing images from https://fliplinestudios.fandom.com ([[phorge:T16254|T16254]])' -- images (END - exit=-2) [23:11:43] !log [jenny@mwtask181] Finished import for fliplinestudioswiki (XML: None; Images: images) (END - exit=-2) [23:11:44] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019231142-b53b1dff [23:11:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log#sal-20261019231144-7a13bf29 [23:12:58] [02puppet] 07Universal-Omega merged pull request #5086: Support OpenSearch 3 (03main...03os3) 13https://github.com/miraheze/puppet/pull/5086 [23:12:58] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/c2200312155b35191f2560819c60830f1e6bcde4 [23:12:59] 02puppet/03main 07CosmicAlpha 03c220031 Support OpenSearch 3 [23:13:01] [02puppet] 07Universal-Omega 04deleted 03os3 at 03d502758 13https://api.github.com/repos/miraheze/puppet/commit/d502758 [23:16:46] [02puppet] 07Universal-Omega created 03os-role (+1 new commit) 13https://github.com/miraheze/puppet/commit/5444a5b35d04 [23:16:46] 02puppet/03os-role 07CosmicAlpha 035444a5b opensearch: use role-based hiera properly [23:16:51] [02puppet] 07Universal-Omega opened pull request #5087: opensearch: use role-based hiera properly (03main...03os-role) 13https://github.com/miraheze/puppet/pull/5087 [23:17:15] PROBLEM - os162 Puppet on os162 is CRITICAL: CRITICAL: Failed to apply catalog, zero resources tracked by Puppet. It might be a dependency cycle. [23:17:56] [02puppet] 07Universal-Omega pushed 1 new commit to 03os-role 13https://github.com/miraheze/puppet/commit/4568ef0d67048ce9dda593b1b6f89e923a08504f [23:17:57] 02puppet/03os-role 07CosmicAlpha 034568ef0 Update [23:18:21] [02puppet] 07Universal-Omega merged pull request #5087: opensearch: use role-based hiera properly (03main...03os-role) 13https://github.com/miraheze/puppet/pull/5087 [23:18:21] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/40056601bd6b421964817e1d84b8c67a56985a20 [23:18:21] 02puppet/03main 07CosmicAlpha 034005660 opensearch: use role-based hiera properly [23:18:23] [02puppet] 07Universal-Omega 04deleted 03os-role at 034568ef0 13https://api.github.com/repos/miraheze/puppet/commit/4568ef0 [23:19:15] RECOVERY - os162 Puppet on os162 is OK: OK: Puppet is currently enabled, last run 0 seconds ago with 0 failures [23:21:41] [02mw-config] 07lihaohong6 left a file comment in pull request #6601 033d7bc52: `mmdb` becomes `mndb`. Typo? 13https://github.com/miraheze/mw-config/pull/6601#discussion_r4319873191 [23:25:30] [02puppet] 07Universal-Omega created 03os161-os3 (+1 new commit) 13https://github.com/miraheze/puppet/commit/b097adfe5b2c [23:25:30] 02puppet/03os161-os3 07CosmicAlpha 03b097adf os161: upgrade to opensearch 3 [23:25:37] [02puppet] 07Universal-Omega opened pull request #5088: os161: upgrade to opensearch 3 (03main...03os161-os3) 13https://github.com/miraheze/puppet/pull/5088 [23:28:10] [02puppet] 07Universal-Omega merged pull request #5088: os161: upgrade to opensearch 3 (03main...03os161-os3) 13https://github.com/miraheze/puppet/pull/5088 [23:28:10] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/11e4e055af4052085faa8d0206fdb63b5b3654ce [23:28:10] 02puppet/03main 07CosmicAlpha 0311e4e05 os161: upgrade to opensearch 3 [23:28:12] [02puppet] 07Universal-Omega 04deleted 03os161-os3 at 03b097adf 13https://api.github.com/repos/miraheze/puppet/commit/b097adf [23:32:08] PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 45242MiB (10.0% inode=100%); [23:36:00] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/5e99d0de19681bbf5589b38abcb4d3fa43a8ecc9 [23:36:00] 02puppet/03main 07CosmicAlpha 035e99d0d Update opensearch.key [23:50:24] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/a785a4769de297f6dd07bb06a208fbebfec6eed5 [23:50:24] 02puppet/03main 07CosmicAlpha 03a785a47 Temp fix for OpenSearch 3 upgrade [23:57:44] [02puppet] 07Universal-Omega pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/8c9acfa8116d6754fbe2bdc7893ad2c20e914827 [23:57:44] 02puppet/03main 07CosmicAlpha 038c9acfa Another opensearch 3 fix