[00:00:00] !log [www-data@mwtask181] Began automatic backing up [00:00:06] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:01:19] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/f94fba055d4182128a89292db90b58e9e9ccb5f8 [00:01:19] 02statichelp/03main 07WikiTideBot 03f94fba0 Bot: Auto-update Tech namespace pages 2025-11-01 00:01:16 [00:02:43] [02mw-config] 07SomeMWDev created 03T14445 (+1 new commit) 13https://github.com/miraheze/mw-config/commit/817d1f468791 [00:02:43] 02mw-config/03T14445 07SomeRandomDeveloper 03817d1f4 T14445: Add $wgMirageTheme to ManageWiki [00:03:14] !log [somerandomdeveloper@test151] starting deploy of {'config': True} to test151 [00:03:15] !log [somerandomdeveloper@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [00:03:18] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:03:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:08:55] [02mw-config] 07SomeMWDev opened pull request #6158: T14445: Add $wgMirageTheme to ManageWiki (03main...03T14445) 13https://github.com/miraheze/mw-config/pull/6158 [00:09:46] miraheze/mw-config - SomeMWDev the build passed. [00:11:56] [02mw-config] 07SomeMWDev 04force-pushed 03T14445 from 03817d1f4 to 0376e4b66 13https://api.github.com/repos/miraheze/mw-config/commits/T14445 [00:11:56] 02mw-config/03T14445 07SomeRandomDeveloper 0376e4b66 T14445: Add $wgMirageTheme to ManageWiki [00:12:10] !log [somerandomdeveloper@test151] starting deploy of {'config': True} to test151 [00:12:11] !log [somerandomdeveloper@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [00:12:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:12:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:12:22] RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 69628MiB (15% inode=99%); [00:12:32] RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 69169MiB (15% inode=99%); [00:12:53] miraheze/mw-config - SomeMWDev the build passed. [00:13:41] [02mw-config] 07SomeMWDev pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/9663c6be9d80e530646319afd205c527afc91edd [00:13:42] 02mw-config/03main 07SomeRandomDeveloper 039663c6b T14445: Add $wgMirageTheme to ManageWiki (#6158)… [00:13:44] [02mw-config] 07SomeMWDev merged pull request #6158: T14445: Add $wgMirageTheme to ManageWiki (03main...03T14445) 13https://github.com/miraheze/mw-config/pull/6158 [00:13:46] [02mw-config] 07SomeMWDev 04deleted 03T14445 at 0376e4b66 13https://api.github.com/repos/miraheze/mw-config/commit/76e4b66 [00:13:55] !log [somerandomdeveloper@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [00:13:56] !log [somerandomdeveloper@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 0s [00:13:59] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:14:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:14:03] !log [somerandomdeveloper@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [00:14:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:14:12] RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 69987MiB (15% inode=99%); [00:14:26] !log [somerandomdeveloper@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 22s [00:14:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:14:31] miraheze/mw-config - SomeMWDev the build passed. [14:50:32] PROBLEM - cp191 Disk Space on cp191 is WARNING: DISK WARNING - free space: / 49863MiB (10% inode=99%); [15:04:12] PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 49775MiB (10% inode=99%); [15:12:22] PROBLEM - cp171 Disk Space on cp171 is WARNING: DISK WARNING - free space: / 49848MiB (10% inode=99%); [15:27:44] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 5.71, 10.70, 8.80 [15:31:44] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 4.10, 8.01, 8.23 [16:16:39] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.98, 19.87, 15.35 [16:16:50] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 20.82, 18.59, 14.54 [16:17:43] PROBLEM - mw191 Current Load on mw191 is WARNING: LOAD WARNING - total load average: 23.53, 20.20, 15.42 [16:19:51] PROBLEM - mw193 Current Load on mw193 is WARNING: LOAD WARNING - total load average: 21.20, 19.19, 15.05 [16:22:51] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 23.12, 20.86, 16.61 [16:22:55] PROBLEM - mw201 Current Load on mw201 is WARNING: LOAD WARNING - total load average: 23.00, 20.62, 16.23 [16:23:11] PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 23.03, 19.95, 15.51 [16:23:31] PROBLEM - mw153 Current Load on mw153 is WARNING: LOAD WARNING - total load average: 21.57, 19.30, 14.50 [16:23:44] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 20.87, 19.12, 15.42 [16:24:43] PROBLEM - mw202 Current Load on mw202 is WARNING: LOAD WARNING - total load average: 21.79, 20.34, 16.79 [16:25:31] RECOVERY - mw153 Current Load on mw153 is OK: LOAD OK - total load average: 19.16, 19.19, 15.05 [16:26:52] RECOVERY - mw201 Current Load on mw201 is OK: LOAD OK - total load average: 17.71, 19.64, 16.89 [16:27:44] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 20.25, 20.12, 16.70 [16:28:03] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 23.05, 21.69, 17.81 [16:29:11] RECOVERY - mw172 Current Load on mw172 is OK: LOAD OK - total load average: 16.58, 20.09, 17.16 [16:30:43] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 17.91, 19.35, 17.48 [16:31:51] RECOVERY - mw193 Current Load on mw193 is OK: LOAD OK - total load average: 14.87, 18.63, 17.75 [16:32:03] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 14.67, 19.20, 17.82 [16:32:48] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 14.29, 19.42, 18.39 [16:33:42] RECOVERY - mw191 Current Load on mw191 is OK: LOAD OK - total load average: 16.14, 19.63, 18.91 [16:34:50] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 16.81, 19.24, 18.70 [16:38:20] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?from=1762011470000&orgId=1&to=1762015100049 [16:38:39] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.68, 19.19, 19.47 [16:48:20] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?from=1762012070000&orgId=1&to=1762015670000 [20:55:27] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/ce1f52308a3136a200b5215431506b03bc7af452 [20:55:27] 02ssl/03main 07WikiTideBot 03ce1f523 Bot: Auto-update domain lists [23:00:07] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 8.15, 12.43, 9.13 [23:02:03] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 4.25, 9.80, 8.57