[00:02:21] [02statichelp] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/statichelp/commit/87308a24562df2efe32cba5121e08507bffa4435 [00:02:21] 02statichelp/03main 07WikiTideBot 0387308a2 Bot: Auto-update Tech namespace pages 2025-11-14 00:02:18 [00:05:20] [02ssl] 07WikiTideBot pushed 1 new commit to 03main 13https://github.com/miraheze/ssl/commit/b3a92846fb497681cab8ae66620b615c58eae5f6 [00:05:20] 02ssl/03main 07WikiTideBot 03b3a9284 Bot: Auto-update domain lists [00:07:37] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 14.95, 11.43, 9.95 [00:11:35] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 10.85, 11.53, 10.40 [00:14:51] RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 73488MiB (16% inode=99%); [00:15:07] RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 73375MiB (16% inode=99%); [00:15:11] RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 70406MiB (15% inode=99%); [00:15:32] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 6.32, 8.91, 9.60 [00:18:15] PROBLEM - puppet181 Current Load on puppet181 is CRITICAL: LOAD CRITICAL - total load average: 17.16, 8.61, 3.83 [00:20:15] RECOVERY - puppet181 Current Load on puppet181 is OK: LOAD OK - total load average: 5.00, 7.62, 4.11 [00:20:50] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763074520000&orgId=1&to=1763079410000 [00:29:08] [02puppet] 07paladox created 03paladox-patch-6 (+1 new commit) 13https://github.com/miraheze/puppet/commit/91c8f79ccaa2 [00:29:08] 02puppet/03paladox-patch-6 07paladox 0391c8f79 mariadb: set aria_pagecache_buffer_size to 1g… [00:29:12] [02puppet] 07paladox opened pull request #4601: mariadb: set aria_pagecache_buffer_size to 1g (03main...03paladox-patch-6) 13https://github.com/miraheze/puppet/pull/4601 [00:29:17] [02puppet] 07coderabbitai[bot] commented on pull request #4601:

[…] 13https://github.com/miraheze/puppet/pull/4601#issuecomment-3530298085 [00:34:49] !log db161: MariaDB [parsercache]> SET GLOBAL max_heap_table_size = 67108864; [00:35:08] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:35:53] [02puppet] 07paladox merged pull request #4601: mariadb: set aria_pagecache_buffer_size to 1g (03main...03paladox-patch-6) 13https://github.com/miraheze/puppet/pull/4601 [00:35:53] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/82af5d4bf9f092b76ba68137a60531ea59994ce3 [00:35:53] 02puppet/03main 07paladox 0382af5d4 mariadb: set aria_pagecache_buffer_size to 1g (#4601)… [00:35:55] [02puppet] 07paladox 04deleted 03paladox-patch-6 at 0391c8f79 13https://api.github.com/repos/miraheze/puppet/commit/91c8f79 [00:35:56] PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 17.48, 11.01, 9.41 [00:37:12] [02puppet] 07paladox created 03paladox-patch-6 (+1 new commit) 13https://github.com/miraheze/puppet/commit/ccc2f89be872 [00:37:13] 02puppet/03paladox-patch-6 07paladox 03ccc2f89 Fix variable names for tmp_table_size settings [00:37:16] [02puppet] 07paladox opened pull request #4602: Fix variable names for tmp_table_size settings (03main...03paladox-patch-6) 13https://github.com/miraheze/puppet/pull/4602 [00:37:20] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 17.55, 11.92, 9.49 [00:37:21] [02puppet] 07coderabbitai[bot] commented on pull request #4602:

[…] 13https://github.com/miraheze/puppet/pull/4602#issuecomment-3530323389 [00:38:26] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/04753f531fc0dca0fcea3d888aaf0984660677a5 [00:38:26] 02puppet/03main 07paladox 0304753f5 Fix variable names for tmp_table_size settings (#4602) [00:38:26] [02puppet] 07paladox merged pull request #4602: Fix variable names for tmp_table_size settings (03main...03paladox-patch-6) 13https://github.com/miraheze/puppet/pull/4602 [00:38:27] [02puppet] 07paladox 04deleted 03paladox-patch-6 at 03ccc2f89 13https://api.github.com/repos/miraheze/puppet/commit/ccc2f89 [00:39:01] !log ignore, was already apply, me searching for it and it was under a different name (used - as opposed to _) [00:39:19] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 11.11, 11.56, 9.67 [00:39:20] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:39:52] !log emergency restart to pick up config change on db161 [00:39:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:41:18] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 4.32, 8.98, 8.96 [00:41:37] PROBLEM - mwtask181 Check unit status of mediawiki_job_purge-parsercache on mwtask181 is CRITICAL: CRITICAL: Status of the systemd unit mediawiki_job_purge-parsercache [00:42:43] !log [paladox@mwtask181] starting deploy of {'config': True, 'force': True} to all [00:43:27] PROBLEM - mw171 HTTPS on mw171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [00:43:29] PROBLEM - mw171 MediaWiki Rendering on mw171 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [00:43:43] PROBLEM - mw202 MediaWiki Rendering on mw202 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:43:46] PROBLEM - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is CRITICAL: CRITICAL - NGINX Error Rate is 74% [00:43:49] PROBLEM - mw193 HTTPS on mw193 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [00:43:49] PROBLEM - mw202 HTTPS on mw202 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10000 milliseconds with 0 bytes received [00:43:49] PROBLEM - mw162 HTTPS on mw162 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10002 milliseconds with 0 bytes received [00:43:50] PROBLEM - mw193 MediaWiki Rendering on mw193 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:43:52] PROBLEM - cp171 HTTPS on cp171 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [00:43:52] PROBLEM - cp201 Varnish Backends on cp201 is CRITICAL: 16 backends are down. mw152 mw161 mw162 mw172 mw181 mw182 mw153 mw163 mw173 mw183 mw191 mw192 mw193 mw201 mw202 mw203 [00:43:56] PROBLEM - mw151 MediaWiki Rendering on mw151 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [00:44:02] PROBLEM - mw173 HTTPS on mw173 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [00:44:02] PROBLEM - mw172 HTTPS on mw172 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [00:44:04] PROBLEM - mw201 HTTPS on mw201 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [00:44:04] PROBLEM - mw201 MediaWiki Rendering on mw201 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:44:06] PROBLEM - cp191 HTTPS on cp191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [00:44:06] PROBLEM - mw163 MediaWiki Rendering on mw163 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:44:07] PROBLEM - cp191 Varnish Backends on cp191 is CRITICAL: 16 backends are down. mw151 mw161 mw162 mw172 mw181 mw182 mw153 mw163 mw173 mw183 mw191 mw192 mw193 mw201 mw202 mw203 [00:44:13] PROBLEM - mw191 MediaWiki Rendering on mw191 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:44:16] PROBLEM - mw191 HTTPS on mw191 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [00:44:19] PROBLEM - mw163 HTTPS on mw163 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [00:44:19] PROBLEM - mw183 HTTPS on mw183 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10004 milliseconds with 0 bytes received [00:44:21] PROBLEM - mw181 HTTPS on mw181 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: cURL returned 28 - Operation timed out after 10003 milliseconds with 0 bytes received [00:44:23] PROBLEM - mw172 MediaWiki Rendering on mw172 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:44:26] PROBLEM - mw183 MediaWiki Rendering on mw183 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:44:30] PROBLEM - mw203 MediaWiki Rendering on mw203 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:44:37] PROBLEM - mw182 MediaWiki Rendering on mw182 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:44:45] PROBLEM - cp201 HTTPS on cp201 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [00:44:49] PROBLEM - mw161 MediaWiki Rendering on mw161 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.010 second response time [00:45:00] PROBLEM - cp171 Varnish Backends on cp171 is CRITICAL: 16 backends are down. mw151 mw152 mw162 mw172 mw181 mw182 mw153 mw163 mw173 mw183 mw191 mw192 mw193 mw201 mw202 mw203 [00:45:01] PROBLEM - mw192 MediaWiki Rendering on mw192 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [00:45:06] PROBLEM - mw182 HTTPS on mw182 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [00:45:06] PROBLEM - mw181 MediaWiki Rendering on mw181 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 9.894 second response time [00:45:17] !log [paladox@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 153s [00:45:18] PROBLEM - mw192 HTTPS on mw192 is CRITICAL: HTTP CRITICAL - Invalid HTTP response received from host on port 443: HTTP/2 502 [00:45:20] PROBLEM - mw173 MediaWiki Rendering on mw173 is CRITICAL: HTTP CRITICAL: HTTP/1.1 502 Bad Gateway - 8191 bytes in 0.011 second response time [00:45:58] !log [paladox@mwtask181] starting deploy of {'config': True, 'force': True} to all [00:46:17] !log [paladox@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 19s [00:46:24] RECOVERY - mw203 MediaWiki Rendering on mw203 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.372 second response time [00:46:27] RECOVERY - mw183 MediaWiki Rendering on mw183 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 1.627 second response time [00:46:37] RECOVERY - mw182 MediaWiki Rendering on mw182 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.376 second response time [00:46:45] RECOVERY - mw161 MediaWiki Rendering on mw161 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.356 second response time [00:46:59] RECOVERY - mw192 MediaWiki Rendering on mw192 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 3.572 second response time [00:47:00] RECOVERY - mw181 MediaWiki Rendering on mw181 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.327 second response time [00:47:08] RECOVERY - mw182 HTTPS on mw182 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 2.633 second response time [00:47:15] RECOVERY - mw192 HTTPS on mw192 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 1.653 second response time [00:47:18] RECOVERY - mw173 MediaWiki Rendering on mw173 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.598 second response time [00:47:22] RECOVERY - mw171 HTTPS on mw171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 2.499 second response time [00:47:24] !log [paladox@mwtask181] starting deploy of {'config': True, 'force': True} to all [00:47:31] RECOVERY - mw171 MediaWiki Rendering on mw171 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 2.341 second response time [00:47:35] RECOVERY - mw202 MediaWiki Rendering on mw202 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 2.680 second response time [00:47:36] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:47:38] RECOVERY - mw193 HTTPS on mw193 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.066 second response time [00:47:40] RECOVERY - mw202 HTTPS on mw202 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 1.880 second response time [00:47:40] RECOVERY - mw162 HTTPS on mw162 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 1.384 second response time [00:47:46] RECOVERY - cp161 HTTP 4xx/5xx ERROR Rate on cp161 is OK: OK - NGINX Error Rate is 38% [00:47:47] RECOVERY - mw193 MediaWiki Rendering on mw193 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.237 second response time [00:47:49] RECOVERY - mw151 MediaWiki Rendering on mw151 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.232 second response time [00:47:51] RECOVERY - cp171 HTTPS on cp171 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4273 bytes in 0.060 second response time [00:47:52] RECOVERY - mw172 HTTPS on mw172 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.076 second response time [00:47:52] RECOVERY - cp201 Varnish Backends on cp201 is OK: All 31 backends are healthy [00:47:55] RECOVERY - mw163 MediaWiki Rendering on mw163 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.356 second response time [00:47:57] RECOVERY - mw201 MediaWiki Rendering on mw201 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.263 second response time [00:48:00] !log [paladox@mwtask181] finished deploy of {'config': True, 'force': True} to all - SUCCESS in 36s [00:48:01] RECOVERY - mw173 HTTPS on mw173 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.063 second response time [00:48:03] RECOVERY - mw201 HTTPS on mw201 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.264 second response time [00:48:03] RECOVERY - cp191 HTTPS on cp191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4273 bytes in 0.064 second response time [00:48:04] RECOVERY - cp191 Varnish Backends on cp191 is OK: All 31 backends are healthy [00:48:05] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [00:48:10] PROBLEM - puppet181 Check unit status of listdomains_github_push on puppet181 is CRITICAL: CRITICAL: Status of the systemd unit listdomains_github_push [00:48:11] RECOVERY - mw181 HTTPS on mw181 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.072 second response time [00:48:12] RECOVERY - mw191 MediaWiki Rendering on mw191 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.228 second response time [00:48:15] RECOVERY - mw191 HTTPS on mw191 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.068 second response time [00:48:15] RECOVERY - mw172 MediaWiki Rendering on mw172 is OK: HTTP OK: HTTP/1.1 200 OK - 8191 bytes in 0.801 second response time [00:48:17] RECOVERY - mw163 HTTPS on mw163 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.051 second response time [00:48:18] RECOVERY - mw183 HTTPS on mw183 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4224 bytes in 0.090 second response time [00:48:42] RECOVERY - cp201 HTTPS on cp201 is OK: HTTP OK: HTTP/2 410 - Status line output matched "HTTP/2 410" - 4273 bytes in 0.056 second response time [00:48:57] RECOVERY - cp171 Varnish Backends on cp171 is OK: All 31 backends are healthy [00:54:14] PROBLEM - matomo151 Check unit status of matomo-archiver-2 on matomo151 is CRITICAL: CRITICAL: Status of the systemd unit matomo-archiver-2 [00:55:09] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 15.20, 10.36, 8.28 [00:55:25] PROBLEM - matomo151 Check unit status of matomo-archiver-1 on matomo151 is CRITICAL: CRITICAL: Status of the systemd unit matomo-archiver-1 [00:55:47] PROBLEM - matomo151 Check unit status of matomo-archiver-3 on matomo151 is CRITICAL: CRITICAL: Status of the systemd unit matomo-archiver-3 [00:56:10] RECOVERY - puppet181 Check unit status of listdomains_github_push on puppet181 is OK: OK: Status of the systemd unit listdomains_github_push [00:58:10] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 22.37, 18.24, 15.12 [01:00:10] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 18.16, 18.23, 15.52 [01:03:05] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 8.82, 11.95, 10.20 [01:03:40] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php /srv/mediawiki/1.44/maintenance/rebuildall.php --wiki=wixosswiki (END - exit=0) [01:03:42] !log [macfan@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php /srv/mediawiki/1.44/maintenance/initSiteStats.php --wiki=wixosswiki --update (END - exit=0) [01:03:49] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:03:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [01:09:29] [02RequestCustomDomain] 07dependabot[bot] created 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.32.1 (+1 new commit) 13https://github.com/miraheze/RequestCustomDomain/commit/330cfd4d16f5 [01:09:29] 02RequestCustomDomain/03dependabot/npm_and_yarn/eslint-config-wikimedia-0.32.1 07dependabot[bot] 03330cfd4 Bump eslint-config-wikimedia from 0.31.0 to 0.32.1… [01:09:30] [02RequestCustomDomain] 07dependabot[bot] added the label 'dependencies' to pull request #98 (Bump eslint-config-wikimedia from 0.31.0 to 0.32.1) 13https://github.com/miraheze/RequestCustomDomain/pull/98 [01:09:30] [02RequestCustomDomain] 07dependabot[bot] added the label 'javascript' to pull request #98 (Bump eslint-config-wikimedia from 0.31.0 to 0.32.1) 13https://github.com/miraheze/RequestCustomDomain/pull/98 [01:09:32] [02RequestCustomDomain] 07dependabot[bot] opened pull request #98: Bump eslint-config-wikimedia from 0.31.0 to 0.32.1 (03main...03dependabot/npm_and_yarn/eslint-config-wikimedia-0.32.1) 13https://github.com/miraheze/RequestCustomDomain/pull/98 [01:09:34] [02RequestCustomDomain] 07dependabot[bot] added the label 'dependencies' to pull request #98 (Bump eslint-config-wikimedia from 0.31.0 to 0.32.1) 13https://github.com/miraheze/RequestCustomDomain/pull/98 [01:09:36] [02RequestCustomDomain] 07dependabot[bot] added the label 'javascript' to pull request #98 (Bump eslint-config-wikimedia from 0.31.0 to 0.32.1) 13https://github.com/miraheze/RequestCustomDomain/pull/98 [01:09:38] [02RequestCustomDomain] 07dependabot[bot] commented on pull request #97: Superseded by #98. 13https://github.com/miraheze/RequestCustomDomain/pull/97#issuecomment-3530382571 [01:09:40] [02RequestCustomDomain] 07dependabot[bot] closed pull request #97: Bump eslint-config-wikimedia from 0.31.0 to 0.32.0 (03main...03dependabot/npm_and_yarn/eslint-config-wikimedia-0.32.0) 13https://github.com/miraheze/RequestCustomDomain/pull/97 [01:09:42] [02RequestCustomDomain] 07dependabot[bot] 04deleted 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.32.0 at 03efae885 13https://api.github.com/repos/miraheze/RequestCustomDomain/commit/efae885 [01:09:44] [02RequestCustomDomain] 07coderabbitai[bot] commented on pull request #98: --- […] 13https://github.com/miraheze/RequestCustomDomain/pull/98#issuecomment-3530382753 [01:12:59] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 9.48, 9.76, 9.83 [01:20:54] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 9.64, 10.75, 10.17 [01:22:53] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 7.68, 9.49, 9.77 [01:28:10] PROBLEM - mw192 Current Load on mw192 is CRITICAL: LOAD CRITICAL - total load average: 24.59, 20.50, 18.36 [01:29:21] miraheze/RequestCustomDomain - dependabot[bot] the build passed. [01:30:10] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 18.06, 19.37, 18.21 [01:35:01] PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 7.65, 7.94, 11.75 [01:35:23] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 22.51, 20.18, 18.82 [01:37:21] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 18.53, 19.57, 18.77 [01:44:10] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 21.06, 19.76, 18.89 [01:44:20] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763081030000&orgId=1&to=1763084660036 [01:46:39] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 11.79, 11.25, 9.79 [01:47:01] RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 7.60, 8.22, 9.95 [01:48:10] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 19.67, 19.67, 19.06 [01:48:38] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 12.04, 11.65, 10.12 [01:50:37] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 10.19, 11.54, 10.30 [01:56:34] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 8.63, 9.83, 9.95 [02:02:29] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 8.50, 10.83, 10.46 [02:04:28] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 15.73, 14.43, 11.93 [02:08:26] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 6.43, 10.51, 10.92 [02:12:24] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 7.67, 8.36, 9.89 [02:24:20] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763083400000&orgId=1&to=1763087000000 [02:50:10] PROBLEM - mw202 Current Load on mw202 is CRITICAL: LOAD CRITICAL - total load average: 24.03, 21.14, 17.13 [02:52:10] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 16.36, 19.18, 16.90 [03:30:04] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 12.13, 9.81, 8.80 [03:34:01] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 9.74, 10.35, 9.31 [03:36:00] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 8.27, 9.51, 9.12 [03:49:52] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 14.31, 12.00, 10.14 [03:51:51] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 11.75, 11.58, 10.19 [03:54:20] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763088830000&orgId=1&to=1763092460038 [03:57:48] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 6.94, 9.48, 9.79 [04:02:15] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.57, 18.69, 16.28 [04:04:15] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.24, 17.62, 16.16 [04:19:20] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763090150000&orgId=1&to=1763093750000 [04:24:15] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.30, 18.20, 17.08 [04:24:32] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 13.90, 10.59, 9.32 [04:26:15] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 16.03, 17.52, 16.98 [04:26:30] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 7.20, 9.09, 8.92 [05:48:20] PROBLEM - mw193 Current Load on mw193 is WARNING: LOAD WARNING - total load average: 22.70, 17.96, 15.36 [05:50:20] RECOVERY - mw193 Current Load on mw193 is OK: LOAD OK - total load average: 16.05, 16.86, 15.26 [06:28:10] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 22.62, 20.19, 17.46 [06:32:10] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 19.57, 20.08, 18.09 [06:47:20] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763099210000&orgId=1&to=1763102840035 [06:59:40] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 22.60, 19.99, 18.61 [07:01:39] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 17.48, 18.85, 18.36 [07:10:30] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 22.95, 20.27, 19.11 [07:11:16] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.14, 20.18, 18.52 [07:11:49] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 22.45, 20.29, 17.95 [07:13:49] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 19.99, 19.73, 18.01 [07:14:27] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 19.36, 20.32, 19.44 [07:15:10] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 16.43, 19.38, 18.69 [07:27:20] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763099210000&orgId=1&to=1763105090000 [07:36:42] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 22.89, 19.72, 18.07 [07:37:14] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 21.29, 18.51, 17.06 [07:37:49] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 21.93, 18.43, 16.53 [07:38:08] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 22.14, 19.68, 18.60 [07:38:36] PROBLEM - mw202 Current Load on mw202 is WARNING: LOAD WARNING - total load average: 23.23, 19.50, 17.29 [07:39:11] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 19.88, 18.75, 17.31 [07:39:49] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 14.32, 17.12, 16.31 [07:40:34] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 18.22, 19.04, 17.40 [07:40:38] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 19.64, 20.35, 18.76 [07:48:01] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 18.46, 19.70, 19.34 [07:49:16] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.19, 19.74, 18.53 [07:51:13] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 16.98, 18.78, 18.33 [07:55:17] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 23.24, 21.02, 19.60 [07:55:50] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763103320000&orgId=1&to=1763106950040 [07:59:13] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 16.92, 19.34, 19.33 [08:01:25] RECOVERY - matomo151 Check unit status of matomo-archiver-1 on matomo151 is OK: OK: Status of the systemd unit matomo-archiver-1 [08:02:14] RECOVERY - matomo151 Check unit status of matomo-archiver-2 on matomo151 is OK: OK: Status of the systemd unit matomo-archiver-2 [08:03:47] RECOVERY - matomo151 Check unit status of matomo-archiver-3 on matomo151 is OK: OK: Status of the systemd unit matomo-archiver-3 [08:06:10] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 22.07, 20.69, 19.93 [08:10:10] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 16.90, 19.06, 19.51 [08:10:55] PROBLEM - mw202 Current Load on mw202 is WARNING: LOAD WARNING - total load average: 22.40, 20.48, 18.96 [08:12:53] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 18.86, 19.90, 18.94 [08:13:33] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.21, 19.99, 19.12 [08:13:39] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 22.81, 21.44, 20.23 [08:15:31] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 19.22, 19.95, 19.23 [08:18:54] PROBLEM - mw193 Current Load on mw193 is WARNING: LOAD WARNING - total load average: 21.16, 19.49, 18.36 [08:20:50] RECOVERY - mw193 Current Load on mw193 is OK: LOAD OK - total load average: 17.91, 18.82, 18.25 [08:21:33] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 16.24, 19.09, 19.81 [08:35:03] PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 11.50, 10.45, 9.17 [08:36:23] PROBLEM - mw202 Current Load on mw202 is WARNING: LOAD WARNING - total load average: 22.48, 19.51, 18.46 [08:37:49] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 21.86, 19.72, 18.18 [08:39:01] RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 8.57, 10.02, 9.35 [08:39:49] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 18.82, 19.29, 18.22 [08:42:16] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 17.02, 19.12, 18.75 [08:46:38] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 21.89, 19.75, 18.71 [08:48:35] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 17.71, 19.19, 18.66 [09:04:56] PROBLEM - mw201 Current Load on mw201 is WARNING: LOAD WARNING - total load average: 20.64, 18.28, 16.68 [09:06:56] RECOVERY - mw201 Current Load on mw201 is OK: LOAD OK - total load average: 17.07, 17.73, 16.67 [09:09:10] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 22.68, 20.38, 19.01 [09:11:08] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 18.05, 19.57, 18.89 [09:11:52] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 23.73, 21.03, 19.49 [09:13:51] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 17.70, 19.62, 19.16 [09:15:50] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763103320000&orgId=1&to=1763111750000 [09:32:40] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 22.01, 18.03, 16.45 [09:32:56] PROBLEM - mw201 Current Load on mw201 is WARNING: LOAD WARNING - total load average: 22.38, 17.84, 15.24 [09:33:01] PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 21.15, 17.44, 15.07 [09:33:34] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 22.97, 19.10, 17.07 [09:33:49] PROBLEM - mw182 Current Load on mw182 is WARNING: LOAD WARNING - total load average: 23.46, 18.84, 15.98 [09:34:15] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 23.31, 19.34, 16.51 [09:34:20] PROBLEM - mw193 Current Load on mw193 is WARNING: LOAD WARNING - total load average: 22.63, 18.83, 15.77 [09:34:29] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 22.15, 18.97, 16.37 [09:35:01] RECOVERY - mw172 Current Load on mw172 is OK: LOAD OK - total load average: 18.63, 17.86, 15.53 [09:36:20] RECOVERY - mw193 Current Load on mw193 is OK: LOAD OK - total load average: 20.13, 19.00, 16.19 [09:36:56] RECOVERY - mw201 Current Load on mw201 is OK: LOAD OK - total load average: 18.16, 18.39, 16.14 [09:37:49] RECOVERY - mw182 Current Load on mw182 is OK: LOAD OK - total load average: 16.70, 19.11, 16.87 [09:38:15] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 17.79, 19.55, 17.33 [09:38:24] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 19.19, 20.09, 17.51 [09:40:31] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 19.18, 19.31, 17.81 [09:45:52] PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 10.47, 10.27, 8.99 [09:47:47] RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 9.29, 9.92, 9.00 [09:53:20] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 15.47, 19.20, 19.57 [09:53:20] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763110370000&orgId=1&to=1763114000038 [09:59:34] PROBLEM - mw202 Current Load on mw202 is WARNING: LOAD WARNING - total load average: 22.46, 20.24, 18.57 [10:01:31] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 18.17, 19.66, 18.58 [10:01:52] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 21.04, 19.43, 18.52 [10:05:17] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 20.73, 20.44, 19.78 [10:07:52] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 16.31, 19.03, 18.78 [10:17:15] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.17, 20.37, 19.59 [10:17:25] PROBLEM - mw191 Current Load on mw191 is WARNING: LOAD WARNING - total load average: 21.12, 19.02, 17.30 [10:17:52] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 21.05, 20.11, 19.20 [10:19:12] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 18.80, 19.81, 19.48 [10:19:24] RECOVERY - mw191 Current Load on mw191 is OK: LOAD OK - total load average: 17.70, 18.32, 17.25 [10:19:52] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 17.22, 19.28, 19.02 [10:23:17] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 12.59, 17.43, 19.46 [10:23:20] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763112170000&orgId=1&to=1763115770000 [10:44:33] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 21.21, 18.10, 16.04 [10:47:01] PROBLEM - mw172 Current Load on mw172 is WARNING: LOAD WARNING - total load average: 22.05, 18.26, 15.22 [10:47:01] PROBLEM - db161 Current Load on db161 is CRITICAL: LOAD CRITICAL - total load average: 13.46, 11.71, 8.55 [10:47:43] PROBLEM - mw153 Current Load on mw153 is WARNING: LOAD WARNING - total load average: 21.70, 17.39, 13.97 [10:47:52] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 22.30, 19.30, 16.79 [10:47:57] PROBLEM - mw173 Current Load on mw173 is WARNING: LOAD WARNING - total load average: 21.74, 19.01, 15.29 [10:48:13] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 22.29, 19.30, 16.50 [10:48:20] PROBLEM - mw193 Current Load on mw193 is WARNING: LOAD WARNING - total load average: 22.39, 19.10, 15.94 [10:48:23] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 15.30, 9.91, 6.98 [10:48:32] PROBLEM - mw202 Current Load on mw202 is WARNING: LOAD WARNING - total load average: 21.95, 18.79, 15.93 [10:49:01] PROBLEM - db161 Current Load on db161 is WARNING: LOAD WARNING - total load average: 9.05, 10.78, 8.61 [10:49:06] PROBLEM - mw171 Current Load on mw171 is WARNING: LOAD WARNING - total load average: 20.94, 18.22, 14.93 [10:49:17] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 22.40, 20.23, 17.65 [10:49:43] RECOVERY - mw153 Current Load on mw153 is OK: LOAD OK - total load average: 18.06, 17.10, 14.26 [10:49:56] RECOVERY - mw173 Current Load on mw173 is OK: LOAD OK - total load average: 17.02, 17.91, 15.32 [10:50:20] RECOVERY - mw193 Current Load on mw193 is OK: LOAD OK - total load average: 19.79, 19.30, 16.41 [10:50:23] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 8.92, 9.40, 7.16 [10:51:01] RECOVERY - db161 Current Load on db161 is OK: LOAD OK - total load average: 8.01, 10.03, 8.61 [10:52:21] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 19.83, 20.30, 18.07 [10:53:06] RECOVERY - mw171 Current Load on mw171 is OK: LOAD OK - total load average: 14.79, 17.54, 15.51 [10:53:15] PROBLEM - mw191 Current Load on mw191 is WARNING: LOAD WARNING - total load average: 23.06, 20.27, 16.89 [10:53:52] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 16.31, 19.59, 17.95 [10:54:10] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 19.16, 19.46, 17.53 [10:55:14] RECOVERY - mw191 Current Load on mw191 is OK: LOAD OK - total load average: 17.89, 19.62, 17.09 [10:55:17] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 18.10, 19.81, 18.46 [10:56:23] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 17.04, 18.36, 17.09 [10:57:01] RECOVERY - mw172 Current Load on mw172 is OK: LOAD OK - total load average: 15.84, 19.19, 17.62 [10:59:20] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763114330000&orgId=1&to=1763117960034 [11:28:58] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 22.52, 20.73, 18.91 [11:34:52] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 16.91, 19.68, 19.16 [11:43:48] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 21.26, 19.86, 18.94 [11:44:20] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763114330000&orgId=1&to=1763120420000 [11:45:07] PROBLEM - mw181 Current Load on mw181 is CRITICAL: LOAD CRITICAL - total load average: 24.04, 21.54, 19.40 [11:45:46] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 17.25, 18.53, 18.54 [11:47:04] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 18.26, 20.40, 19.26 [11:57:11] PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 49794MiB (10% inode=99%); [11:57:22] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 23.19, 20.92, 19.66 [11:57:36] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 21.11, 20.75, 19.61 [12:00:10] PROBLEM - mw202 Current Load on mw202 is WARNING: LOAD WARNING - total load average: 21.69, 19.09, 17.49 [12:01:17] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 19.11, 20.17, 19.69 [12:01:33] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 18.79, 20.25, 19.70 [12:02:10] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 19.81, 19.59, 17.89 [12:02:39] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.21, 19.99, 18.99 [12:04:36] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 18.56, 19.20, 18.81 [12:11:20] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763118650000&orgId=1&to=1763122280042 [12:14:21] [02mw-config] 07SomeMWDev created 03wgRCLinkLimits (+1 new commit) 13https://github.com/miraheze/mw-config/commit/a3c26fa3efeb [12:14:21] 02mw-config/03wgRCLinkLimits 07SomeRandomDeveloper 03a3c26fa Remove high values for wgRCLinkLimits… [12:14:36] [02mw-config] 07SomeMWDev opened pull request #6178: Remove high values for wgRCLinkLimits (03main...03wgRCLinkLimits) 13https://github.com/miraheze/mw-config/pull/6178 [12:15:31] miraheze/mw-config - SomeMWDev the build passed. [12:16:03] [02mw-config] 07SomeMWDev pushed 1 new commit to 03main 13https://github.com/miraheze/mw-config/commit/ad8c995438b5e9f52f0d32087ae9870885644f6e [12:16:04] 02mw-config/03main 07SomeRandomDeveloper 03ad8c995 Remove high values for wgRCLinkLimits (#6178)… [12:16:06] [02mw-config] 07SomeMWDev merged pull request #6178: Remove high values for wgRCLinkLimits (03main...03wgRCLinkLimits) 13https://github.com/miraheze/mw-config/pull/6178 [12:16:08] [02mw-config] 07SomeMWDev 04deleted 03wgRCLinkLimits at 03a3c26fa 13https://api.github.com/repos/miraheze/mw-config/commit/a3c26fa [12:16:18] !log [somerandomdeveloper@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [12:16:19] !log [somerandomdeveloper@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 0s [12:16:20] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763118950000&orgId=1&to=1763122550000 [12:16:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:16:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:16:56] miraheze/mw-config - SomeMWDev the build passed. [12:16:59] !log [somerandomdeveloper@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [12:17:03] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:17:21] !log [somerandomdeveloper@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 22s [12:17:22] !log [somerandomdeveloper@test151] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.php /srv/mediawiki/1.45/maintenance/run.php ManageWiki:PopulateWikiSettings --setting=wgRCLinkLimits --remove (START) [12:17:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:17:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:17:52] !log [somerandomdeveloper@test151] sudo -u www-data /usr/local/bin/foreachwikiindblist /srv/mediawiki/cache/databases.php /srv/mediawiki/1.45/maintenance/run.php ManageWiki:PopulateWikiSettings --setting=wgRCLinkLimits --remove (END - exit=0) [12:17:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [12:21:18] [02puppet] 07paladox pushed 1 new commit to 03main 13https://github.com/miraheze/puppet/commit/1f7161bb9b2599237cc8dee6d0c749909df3b6cf [12:21:18] 02puppet/03main 07SomeRandomDeveloper 031f7161b Add PopulateWikiSettings to longscripts (#4567)… [12:21:18] [02puppet] 07paladox merged 07SomeMWDev's pull request #4567: Add PopulateWikiSettings to longscripts (07miraheze:03main...07SomeMWDev:03patch-3) 13https://github.com/miraheze/puppet/pull/4567 [12:22:31] miraheze/puppet - paladox the build passed. [12:30:10] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 23.00, 17.63, 15.17 [12:30:10] PROBLEM - mw202 Current Load on mw202 is WARNING: LOAD WARNING - total load average: 20.78, 16.81, 14.64 [12:30:15] PROBLEM - mw181 Current Load on mw181 is WARNING: LOAD WARNING - total load average: 22.59, 17.61, 14.82 [12:30:32] PROBLEM - mw183 Current Load on mw183 is WARNING: LOAD WARNING - total load average: 23.37, 18.62, 15.66 [12:31:17] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 23.36, 19.10, 16.36 [12:32:10] RECOVERY - mw202 Current Load on mw202 is OK: LOAD OK - total load average: 17.40, 16.87, 14.93 [12:32:15] RECOVERY - mw181 Current Load on mw181 is OK: LOAD OK - total load average: 19.23, 18.08, 15.34 [12:32:30] RECOVERY - mw183 Current Load on mw183 is OK: LOAD OK - total load average: 15.18, 17.35, 15.57 [12:33:17] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 20.19, 19.74, 16.95 [12:34:10] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 19.20, 19.59, 16.62 [12:43:17] PROBLEM - mw203 Current Load on mw203 is WARNING: LOAD WARNING - total load average: 22.45, 20.43, 18.27 [12:44:10] PROBLEM - mw192 Current Load on mw192 is WARNING: LOAD WARNING - total load average: 21.50, 20.31, 18.21 [12:45:17] RECOVERY - mw203 Current Load on mw203 is OK: LOAD OK - total load average: 20.34, 20.12, 18.41 [12:48:10] RECOVERY - mw192 Current Load on mw192 is OK: LOAD OK - total load average: 15.80, 19.24, 18.37 [12:55:31] [02CreateWiki] 07SomeMWDev created 03dblist-metrics (+1 new commit) 13https://github.com/miraheze/CreateWiki/commit/4695c4da19b3 [12:55:33] 02CreateWiki/03dblist-metrics 07SomeRandomDeveloper 034695c4d Track metrics related to dblist generation [12:55:42] [02CreateWiki] 07SomeMWDev opened pull request #790: Track metrics related to dblist generation (03main...03dblist-metrics) 13https://github.com/miraheze/CreateWiki/pull/790 [12:55:50] [Grafana] FIRING: Some MediaWiki Appservers are running out of PHP-FPM workers. https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763121320000&orgId=1&to=1763124950042 [12:56:21] [02CreateWiki] 07coderabbitai[bot] commented on pull request #790:

[…] 13https://github.com/miraheze/CreateWiki/pull/790#issuecomment-3532607251 [12:59:05] [02mw-config] 07SomeMWDev created 03cw-dblist-metrics (+1 new commit) 13https://github.com/miraheze/mw-config/commit/bfd7e56929da [12:59:05] 02mw-config/03cw-dblist-metrics 07SomeRandomDeveloper 03bfd7e56 Enable CreateWiki dblist metrics [12:59:25] [02mw-config] 07SomeMWDev opened pull request #6179: Enable CreateWiki dblist metrics (03main...03cw-dblist-metrics) 13https://github.com/miraheze/mw-config/pull/6179 [13:00:15] miraheze/CreateWiki - SomeMWDev the build has errored. [13:00:25] miraheze/mw-config - SomeMWDev the build passed. [13:04:30] [02ImportDump] 07dependabot[bot] created 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.32.1 (+1 new commit) 13https://github.com/miraheze/ImportDump/commit/90854843ea82 [13:04:30] 02ImportDump/03dependabot/npm_and_yarn/eslint-config-wikimedia-0.32.1 07dependabot[bot] 039085484 Bump eslint-config-wikimedia from 0.32.0 to 0.32.1… [13:04:32] [02ImportDump] 07dependabot[bot] added the label 'dependencies' to pull request #158 (Bump eslint-config-wikimedia from 0.32.0 to 0.32.1) 13https://github.com/miraheze/ImportDump/pull/158 [13:04:34] [02ImportDump] 07dependabot[bot] added the label 'javascript' to pull request #158 (Bump eslint-config-wikimedia from 0.32.0 to 0.32.1) 13https://github.com/miraheze/ImportDump/pull/158 [13:04:36] [02ImportDump] 07dependabot[bot] opened pull request #158: Bump eslint-config-wikimedia from 0.32.0 to 0.32.1 (03main...03dependabot/npm_and_yarn/eslint-config-wikimedia-0.32.1) 13https://github.com/miraheze/ImportDump/pull/158 [13:04:38] [02ImportDump] 07dependabot[bot] added the label 'dependencies' to pull request #158 (Bump eslint-config-wikimedia from 0.32.0 to 0.32.1) 13https://github.com/miraheze/ImportDump/pull/158 [13:04:40] [02ImportDump] 07dependabot[bot] added the label 'javascript' to pull request #158 (Bump eslint-config-wikimedia from 0.32.0 to 0.32.1) 13https://github.com/miraheze/ImportDump/pull/158 [13:04:48] [02ImportDump] 07coderabbitai[bot] commented on pull request #158: --- […] 13https://github.com/miraheze/ImportDump/pull/158#issuecomment-3532634538 [13:06:14] [02CreateWiki] 07SomeMWDev pushed 1 new commit to 03dblist-metrics 13https://github.com/miraheze/CreateWiki/commit/52823a27b8e33c204153633991672e3cef778e05 [13:06:14] 02CreateWiki/03dblist-metrics 07SomeRandomDeveloper 0352823a2 fix phan errors [13:12:01] miraheze/CreateWiki - SomeMWDev the build has errored. [13:20:50] [Grafana] RESOLVED: PHP-FPM Worker Usage High https://grafana.wikitide.net/d/GtxbP1Xnk?from=1763122760000&orgId=1&to=1763126360000 [13:22:25] miraheze/ImportDump - dependabot[bot] the build passed. [13:27:13] [02WikiTideDebug] 07dependabot[bot] created 03dependabot/npm_and_yarn/eslint-config-wikimedia-0.32.1 (+1 new commit) 13https://github.com/miraheze/WikiTideDebug/commit/7e0fb127785b [13:27:15] 02WikiTideDebug/03dependabot/npm_and_yarn/eslint-config-wikimedia-0.32.1 07dependabot[bot] 037e0fb12 Bump eslint-config-wikimedia from 0.32.0 to 0.32.1… [13:27:17] [02WikiTideDebug] 07dependabot[bot] added the label 'dependencies' to pull request #26 (Bump eslint-config-wikimedia from 0.32.0 to 0.32.1) 13https://github.com/miraheze/WikiTideDebug/pull/26 [13:27:19] [02WikiTideDebug] 07dependabot[bot] added the label 'javascript' to pull request #26 (Bump eslint-config-wikimedia from 0.32.0 to 0.32.1) 13https://github.com/miraheze/WikiTideDebug/pull/26 [13:27:21] [02WikiTideDebug] 07dependabot[bot] opened pull request #26: Bump eslint-config-wikimedia from 0.32.0 to 0.32.1 (03main...03dependabot/npm_and_yarn/eslint-config-wikimedia-0.32.1) 13https://github.com/miraheze/WikiTideDebug/pull/26 [13:27:23] [02WikiTideDebug] 07dependabot[bot] added the label 'dependencies' to pull request #26 (Bump eslint-config-wikimedia from 0.32.0 to 0.32.1) 13https://github.com/miraheze/WikiTideDebug/pull/26 [13:27:25] [02WikiTideDebug] 07dependabot[bot] added the label 'javascript' to pull request #26 (Bump eslint-config-wikimedia from 0.32.0 to 0.32.1) 13https://github.com/miraheze/WikiTideDebug/pull/26 [13:27:28] [02WikiTideDebug] 07coderabbitai[bot] commented on pull request #26: --- […] 13https://github.com/miraheze/WikiTideDebug/pull/26#issuecomment-3532786430 [13:27:42] [02CreateWiki] 07SomeMWDev pushed 1 new commit to 03dblist-metrics 13https://github.com/miraheze/CreateWiki/commit/5732924aab7727d3004202110fdc92ceb42ce8cb [13:27:43] 02CreateWiki/03dblist-metrics 07SomeRandomDeveloper 035732924 fix phan again [13:33:02] [02CreateWiki] 07codecov[bot] commented on pull request #790: :white_check_mark: All modified and coverable lines are covered by tests. […] 13https://github.com/miraheze/CreateWiki/pull/790#issuecomment-3532837859 [13:38:43] !log [somerandomdeveloper@mwtask181] sudo -u www-data php /srv/mediawiki/1.44/maintenance/run.php ManageWiki:PopulateWikiSettings --wiki=metawiki --setting=wgRCLinkLimits --remove --all-wikis --execute (END - exit=0) [13:38:48] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:39:20] !log [somerandomdeveloper@test151] starting deploy of {'config': True, 'folders': '1.45/extensions/CreateWiki'} to test151 [13:39:21] !log [somerandomdeveloper@test151] finished deploy of {'config': True, 'folders': '1.45/extensions/CreateWiki'} to test151 - SUCCESS in 0s [13:39:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:39:33] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [13:48:23]