[00:07:10] RECOVERY - cp171 Disk Space on cp171 is OK: DISK OK - free space: / 57037MiB (12% inode=99%); [00:07:22] RECOVERY - cp191 Disk Space on cp191 is OK: DISK OK - free space: / 57562MiB (12% inode=99%); [00:07:24] RECOVERY - cp201 Disk Space on cp201 is OK: DISK OK - free space: / 56831MiB (12% inode=99%); [01:44:18] PROBLEM - graylog161 Disk Space on graylog161 is CRITICAL: DISK CRITICAL - free space: / 0MiB (0% inode=93%); [01:50:18] RECOVERY - graylog161 Disk Space on graylog161 is OK: DISK OK - free space: / 3759MiB (20% inode=93%); [02:28:00] !log [blankeclair@mwtask181] starting deploy of {'versions': '1.43', 'upgrade_extensions': 'RemoveRedlinks'} to all [02:28:04] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:28:24] !log [blankeclair@mwtask181] finished deploy of {'versions': '1.43', 'upgrade_extensions': 'RemoveRedlinks'} to all - SUCCESS in 23s [02:28:27] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:34:06] !log [blankeclair@mwtask181] curl -X DELETE https://opensearch-mw.wikitide.net/inextandkaylaandrachelversewiki_content https://opensearch-mw.wikitide.net/inextandkaylaandrachelversewiki_general [02:34:09] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [02:34:19] !log [blankeclair@mwtask181] curl -X DELETE https://opensearch-mw.wikitide.net/inextandkaylaandrachelversewiki_content https://opensearch-mw.wikitide.net/inextandkaylaandrachelversewiki_general -vk [02:34:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:02:05] PROBLEM - matomo151 Check unit status of matomo-archiver-1 on matomo151 is CRITICAL: CRITICAL: Status of the systemd unit matomo-archiver-1 [03:18:49] !log restart nginx on os151 and os162 [03:18:52] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:24:31] miraheze/mw-config - Universal-Omega the build passed. [03:26:32] miraheze/mw-config - Universal-Omega the build passed. [03:27:38] miraheze/mw-config - Universal-Omega the build passed. [03:29:34] miraheze/mw-config - Universal-Omega the build passed. [03:30:58] miraheze/mw-config - Universal-Omega the build passed. [03:32:06] miraheze/mw-config - Universal-Omega the build passed. [03:34:14] !log [universalomega@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [03:34:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:34:35] miraheze/mw-config - Universal-Omega the build passed. [03:34:40] !log [universalomega@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 25s [03:34:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:35:30] !log [universalomega@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php ManageWiki:ToggleExtension --wiki=metawiki --name=contactpage --disable --all-wikis --execute (END - exit=0) [03:35:34] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:39:10] !log [universalomega@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php ManageWiki:PopulateWikiSettings --wiki=metawiki --setting=wmgMirahezeContactPageFooter --remove --all-wikis --execute (END - exit=0) [03:39:13] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:41:44] !log [universalomega@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php ManageWiki:PopulateWikiSettings --wiki=metawiki --setting=wmgContactPageRecipientUser --remove --all-wikis --execute (END - exit=0) [03:41:47] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [03:47:07] wait what [07:07:39] PROBLEM - db151 Backups SQL on db151 is CRITICAL: FILE_AGE CRITICAL: /var/log/db-backups/db-backups/db-backups.log is 1468810 seconds old and 2010053 bytes [08:00:05] RECOVERY - matomo151 Check unit status of matomo-archiver-1 on matomo151 is OK: OK: Status of the systemd unit matomo-archiver-1 [09:03:24] PROBLEM - cp201 Disk Space on cp201 is WARNING: DISK WARNING - free space: / 49833MiB (10% inode=99%); [09:19:10] PROBLEM - cp171 Disk Space on cp171 is WARNING: DISK WARNING - free space: / 49881MiB (10% inode=99%); [09:37:22] PROBLEM - cp191 Disk Space on cp191 is WARNING: DISK WARNING - free space: / 49936MiB (10% inode=99%); [15:50:16] [02mw-config] 07pskyechology opened pull request #6037: T14052: Add multi-level interwiki prefix for utgwiki (07miraheze:03main...07pskyechology:03T14052) 13https://github.com/miraheze/mw-config/pull/6037 [15:51:01] miraheze/mw-config - pskyechology the build passed. [15:56:51] !log [skye@test151] starting deploy of {'config': True} to test151 [15:56:52] !log [skye@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [15:56:54] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [15:56:57] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [16:07:05] PROBLEM - test151 Puppet on test151 is CRITICAL: CRITICAL: Puppet has 1 failures. Last run 3 minutes ago with 1 failures. Failed resources (up to 3 shown): Exec[git_checkout_MediaWiki config] [16:34:43] [02mw-config] 07pskyechology commented on pull request #6037: Tested at https://skye.mirabeta.org/wiki/Special:Interwiki and https://skye.mirabeta.org/wiki/Utginterwiki […] 13https://github.com/miraheze/mw-config/pull/6037#issuecomment-3124528191 [17:16:31] hoes mad that I made a local change ^ [19:57:18] !log [somerandomdeveloper@test151] starting deploy of {'versions': '1.44', 'upgrade_extensions': 'UnusedRedirects'} to test151 [19:57:19] !log [somerandomdeveloper@test151] finished deploy of {'versions': '1.44', 'upgrade_extensions': 'UnusedRedirects'} to test151 - SUCCESS in 1s [19:57:22] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:57:24] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:57:34] !log [somerandomdeveloper@mwtask181] starting deploy of {'versions': '1.44', 'upgrade_extensions': 'UnusedRedirects'} to all [19:57:37] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [19:57:59] !log [somerandomdeveloper@mwtask181] finished deploy of {'versions': '1.44', 'upgrade_extensions': 'UnusedRedirects'} to all - SUCCESS in 24s [19:58:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:38:37] PROBLEM - db171 Current Load on db171 is WARNING: LOAD WARNING - total load average: 10.64, 8.30, 4.96 [21:40:37] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 3.26, 6.21, 4.59 [21:52:26] miraheze/mw-config - pskyechology the build passed. [21:55:10] !log [skye@test151] starting deploy of {'config': True} to test151 [21:55:11] !log [skye@test151] finished deploy of {'config': True} to test151 - SUCCESS in 0s [21:55:14] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [21:55:17] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:02:18] !log [somerandomdeveloper@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [22:02:19] !log [somerandomdeveloper@test151] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [22:02:21] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:02:25] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:02:26] !log [somerandomdeveloper@mwtask181] starting deploy of {'pull': 'config', 'config': True} to all [22:02:30] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:02:53] !log [somerandomdeveloper@mwtask181] finished deploy of {'pull': 'config', 'config': True} to all - SUCCESS in 26s [22:02:54] @pskyechology ^ you can remove the config patch from beta [22:02:54] miraheze/mw-config - SomeMWDev the build passed. [22:02:56] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:04:04] `sudo -u www-data git restore *` my beloved [22:04:40] !log [skye@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [22:04:41] !log [skye@test151] DEPLOY ABORTED: Non-Zero Exit Code in prep, see output. [22:04:43] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:04:45] gg [22:04:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:04:48] i forgor [22:05:39] !log [skye@test151] starting deploy of {'pull': 'config', 'config': True} to test151 [22:05:40] !log [skye@test151] finished deploy of {'pull': 'config', 'config': True} to test151 - SUCCESS in 0s [22:05:42] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:05:46] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [22:05:49] i unforgor [22:05:53] or `sudo -u www-data git fetch && sudo -u www-data git reset --hard origin/main` [22:06:12] assuming you're on the main branch [22:07:43] true [22:07:53] always trust the random german dev to give me better options [22:08:35] for my next trick I'm going to call something obviously broken working as intended once again [22:12:04] (it's the confidence that matters, trust) [22:12:35] ((actually the most dangerous idiot is a confident one but shhhh)) [22:12:37] [1/2] I have severe dementia so half of the time I do something I have to check a discord channel on a server only I'm on where I post the commands whenever I learn something new lol [22:12:37] [2/2] https://cdn.discordapp.com/attachments/808001911868489748/1399152522752692326/image.png?ex=6887f5d4&is=6886a454&hm=10e6666ea29122dddb801de74e687bccf155dcc4f913af8276fa0cc7be56c448& [22:12:42] I should really document those on meta or somewhere else [22:13:09] im ngl everytime I need to deploy a pr I look up "pr" on phorge for your task about adding --pr to mwdeploy [22:13:16] lol [22:13:18] on meta pls [22:13:48] im gonna make phorge wiki just to mess with you [22:13:51] when I was in the tech team we kind of had a problem with docs, even if I'm not with you guys anymore I welcome improvements to docs [22:14:26] actually a page with misc. commands & tips and tricks would be nice [22:14:48] yeah [22:14:59] it's quite hard to find the correct article to add this stuff to [22:15:37] my favourite content on meta are the user essays which are so good but good luck finding them [22:16:03] it's hard because it doesn't exist... [22:16:12] but you can be the one to create the article [22:16:27] [[Category:User essays]] [22:16:27]

[22:17:33] *if you're just navigating the site [22:17:54] true [22:17:56] you'll find them if you search for them, but they could use more links to them [22:19:33] PROBLEM - matomo151 Disk Space on matomo151 is WARNING: DISK WARNING - free space: / 1962MiB (10% inode=90%); [22:34:37] PROBLEM - db171 Current Load on db171 is CRITICAL: LOAD CRITICAL - total load average: 10.65, 13.10, 7.50 [22:35:05] RECOVERY - test151 Puppet on test151 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [22:36:37] RECOVERY - db171 Current Load on db171 is OK: LOAD OK - total load average: 2.87, 9.30, 6.78 [22:39:23] Puppet is enabled?!? [22:39:37] is this a fnaf reference!!!!! [22:57:59] added [[Tech:Mwdeploy#Deploying_an_mw-config_PR_on_beta]] and created [[Tech:Mediawiki-repos]], the rest of the stuff is either outdated, untested or too specific [22:58:00]

[1/2] [22:58:00]

[2/2] [22:58:35] woohoo!!! [23:03:33] RECOVERY - matomo151 Disk Space on matomo151 is OK: DISK OK - free space: / 7064MiB (39% inode=90%); [23:52:28] !log curl -X DELETE https://opensearch-mw.wikitide.net/wmwiki_{content,general} [23:52:31] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:52:58] !log [somerandomdeveloper@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php CirrusSearch:UpdateSearchIndexConfig --wiki=wmwiki --startOver (END - exit=0) [23:53:02] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log [23:53:22] !log [somerandomdeveloper@mwtask181] sudo -u www-data php /srv/mediawiki/1.43/maintenance/run.php CirrusSearch:ForceSearchIndex --wiki=wmwiki (END - exit=0) [23:53:26] Logged the message at https://meta.miraheze.org/wiki/Tech:Server_admin_log