[00:24:18] [02puppet] 07Universal-Omega closed pull request 03#34: Cleanup phorge - 13https://github.com/WikiTideOrg/puppet/pull/34 [00:24:20] [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+1/-2/±4] 13https://github.com/WikiTideOrg/puppet/compare/71e356e0f205...befacceccb13 [00:24:22] [02WikiTideOrg/puppet] 07Universal-Omega 03befacce - Cleanup phorge [00:24:24] [02WikiTideOrg/puppet] 07Universal-Omega deleted branch 03cleanup-phorge [00:24:26] [02puppet] 07Universal-Omega deleted branch 03cleanup-phorge - 13https://github.com/WikiTideOrg/puppet [00:32:42] [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/befacceccb13...705a0fa11611 [00:32:44] [02WikiTideOrg/puppet] 07Universal-Omega 03705a0fa - Add full path and remove absented service [00:47:09] PROBLEM - nexttide.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query nexttide.org. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [01:16:36] PROBLEM - nexttide.org - reverse DNS on sslhost is CRITICAL: rDNS CRITICAL - nexttide.org All nameservers failed to answer the query. [01:46:00] RECOVERY - nexttide.org - reverse DNS on sslhost is OK: SSL OK - nexttide.org reverse DNS resolves to cp1.wikitide.net - NS RECORDS OK [02:16:20] RECOVERY - newcascadia.net - reverse DNS on sslhost is OK: SSL OK - newcascadia.net reverse DNS resolves to cp1.wikitide.net - NS RECORDS OK [02:30:59] [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/705a0fa11611...8b278d580424 [02:31:00] [02WikiTideOrg/puppet] 07Universal-Omega 038b278d5 - Fix [02:47:08] PROBLEM - newcascadia.net - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out. [03:15:45] RECOVERY - newcascadia.net - reverse DNS on sslhost is OK: SSL OK - newcascadia.net reverse DNS resolves to cp1.wikitide.net - NS RECORDS OK [03:29:38] UO, I know you still there in the backend, can you populate the site links for me? [03:33:45] PROBLEM - valorantjp.wiki - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query valorantjp.wiki. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [03:39:14] [02WikiTideOrg/dns] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/dns/compare/a4f2ff4300f3...c0cac56cf25d [03:39:16] [02WikiTideOrg/dns] 07Universal-Omega 03c0cac56 - Use public IP for puppet1 [03:39:42] [02WikiTideOrg/dns] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/dns/compare/c0cac56cf25d...5d7ace99ebe9 [03:39:44] [02WikiTideOrg/dns] 07Universal-Omega 035d7ace9 - IPv6 [03:42:30] @songngu.xyz can you link me to what you want ran again? [03:43:45] updateSiteLinks or something [03:44:18] populateSiteLinks [03:44:28] [1/2] [[User:Pisces/Pisces's_Bizarre_Adventure#rebuildWikibaseIdCounters.sql]] for `lhmndatawikitide` [03:44:28] [2/2] And add site links at `lhmnwikitide` to `lhmndatawikitide` [03:44:29]

[03:47:32] Alright I will do in in about 30 minutes [03:49:09] oh and may us enable syntaxhighlight at meta? [03:49:45] I though that it was enabled there while adding the SQL script [03:52:03] Sure [03:53:43] I will in 30 minutes get to both [03:54:39] [1/4] Oh and few things to notes while running: [03:54:39] [2/4] - Target db: `lhmndatawikitide` [03:54:40] [3/4] - Site group: `lhmngroup` [03:54:40] [4/4] - Site should be informed already at `Wikibase.php` [03:54:55] good luck [03:57:55] RECOVERY - cp2 Puppet on cp2 is OK: OK: Puppet is currently enabled, last run 20 seconds ago with 0 failures [04:02:14] RECOVERY - cp4 Puppet on cp4 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:03:34] RECOVERY - cp5 Puppet on cp5 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:04:45] RECOVERY - cp6 Puppet on cp6 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:05:23] RECOVERY - cp3 Puppet on cp3 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:17:57] PROBLEM - db1 Backups SQL on db1 is WARNING: FILE_AGE WARNING: /var/log/backup-logs/sql-backup.log is 864191 seconds old and 1977 bytes [04:23:46] [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/8b278d580424...c92010a72a05 [04:23:47] [02WikiTideOrg/puppet] 07Universal-Omega 03c92010a - Remove cadvisor [04:24:15] [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/c92010a72a05...3d4d870af3ec [04:24:16] [02WikiTideOrg/puppet] 07Universal-Omega 033d4d870 - Remove prometheus::exporter::nutcracker [04:25:32] RECOVERY - mw1 Puppet on mw1 is OK: OK: Puppet is currently enabled, last run 35 seconds ago with 0 failures [04:28:05] RECOVERY - mw2 Puppet on mw2 is OK: OK: Puppet is currently enabled, last run 1 minute ago with 0 failures [04:35:25] [1/13] @songngu.xyz [04:35:26] [2/13] ```sql [04:35:26] [3/13] MariaDB [lhmndatawikitide]> SELECT FROM /_*/wb_id_counters; [04:35:26] [4/13] +-------------------+----------+ [04:35:27] [5/13] | id_type | id_value | [04:35:27] [6/13] +-------------------+----------+ [04:35:27] [7/13] | wikibase-item | 14 | [04:35:27] [8/13] | wikibase-property | 21 | [04:35:28] [9/13] +-------------------+----------+ [04:35:28] [10/13] 2 rows in set (0.000 sec) [04:35:28] [11/13] MariaDB [lhmndatawikitide]> REPLACE INTO /_/wb_id_counters VALUE((SELECT COALESCE(MAX(CAST(SUBSTRING(`page_title`, 2) AS UNSIGNED)), 0) FROM `page` WHERE `page_namespace` = 860), 'wikibase-item'); [04:35:29] [12/13] ERROR 1366 (22007): Incorrect integer value: 'wikibase-item' for column `lhmndatawikitide`.`wb_id_counters`.`id_value` at row 157 [04:35:29] [13/13] ``` [04:42:59] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03replacements [+0/-0/±1] 13https://github.com/WikiTideOrg/MediaWikiDebugJS/commit/a16307bde801 [04:43:00] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega 03a16307b - Add more replacements [04:43:02] [02MediaWikiDebugJS] 07Universal-Omega created branch 03replacements - 13https://github.com/WikiTideOrg/MediaWikiDebugJS [04:43:04] [02MediaWikiDebugJS] 07Universal-Omega opened pull request 03#5: Add more replacements - 13https://github.com/WikiTideOrg/MediaWikiDebugJS/pull/5 [04:44:29] [02MediaWikiDebugJS] 07Universal-Omega synchronize pull request 03#5: Add more replacements - 13https://github.com/WikiTideOrg/MediaWikiDebugJS/pull/5 [04:44:32] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03replacements [+0/-0/±1] 13https://github.com/WikiTideOrg/MediaWikiDebugJS/compare/a16307bde801...3f0260d53ac3 [04:44:34] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega 033f0260d - Fix [04:48:02] Weird, it's suppose to work [04:48:22] It was written in 2020 by someone in Pro.wiki [04:49:42] RECOVERY - jobrunner1 Puppet on jobrunner1 is OK: OK: Puppet is currently enabled, last run 49 seconds ago with 0 failures [04:52:45] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03replacements [+0/-0/±1] 13https://github.com/WikiTideOrg/MediaWikiDebugJS/compare/3f0260d53ac3...9bf2a5393705 [04:52:48] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega 039bf2a53 - Fix [04:52:51] [02MediaWikiDebugJS] 07Universal-Omega synchronize pull request 03#5: Add more replacements - 13https://github.com/WikiTideOrg/MediaWikiDebugJS/pull/5 [04:53:40] [02MediaWikiDebugJS] 07Universal-Omega closed pull request 03#5: Add more replacements - 13https://github.com/WikiTideOrg/MediaWikiDebugJS/pull/5 [04:53:43] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/MediaWikiDebugJS/compare/74d469a5b048...107ad465d7a8 [04:53:44] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega 03107ad46 - Add more replacements [04:53:46] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega deleted branch 03replacements [04:53:49] [02MediaWikiDebugJS] 07Universal-Omega deleted branch 03replacements - 13https://github.com/WikiTideOrg/MediaWikiDebugJS [04:53:50] hmm... [05:09:29] [02WikiTideOrg/mw-config] 07Universal-Omega pushed 031 commit to 03main [+0/-0/±1] 13https://github.com/WikiTideOrg/mw-config/compare/6a1725046b41...051dfbef8298 [05:09:30] [02WikiTideOrg/mw-config] 07Universal-Omega 03051dfbe - Fix Url [05:10:30] WikiTideOrg/mw-config - Universal-Omega the build passed. [05:12:09] [02WikiTideOrg/puppet] 07Universal-Omega pushed 0312 commits to 03mcrouter [+2/-4/±31] 13https://github.com/WikiTideOrg/puppet/compare/4ac0baec57c2...d9de5291714b [05:12:12] [02WikiTideOrg/puppet] 07Universal-Omega 03d9de529 - Merge branch 'master' into mcrouter [05:12:15] [02puppet] 07Universal-Omega synchronize pull request 03#32: mediawiki: use mcrouter instead of nutcracker - 13https://github.com/WikiTideOrg/puppet/pull/32 [05:16:13] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/MediaWikiDebugJS/compare/107ad465d7a8...501ad798dc68 [05:16:16] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega 03501ad79 - replace cache-den [05:16:55] [1/2] UO, you might wants to look at this article from WBConsulting: [05:16:55] [2/2] https://www.wikibase.consulting/transferring-wikibase-data-between-wikis/ [05:17:20] This actually mentions my cases and how should the import be handled [05:18:17] !log [@jobrunner1] starting deploy of {'config': True, 'versions': ['']} to [mw1, mw2, jobrunner1] [05:18:18] !log [@jobrunner1] finished deploy of {'config': True, 'versions': ['']} to [mw1, mw2, jobrunner1] - SUCCESS in 1s [05:18:18] Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log [05:18:20] Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log [05:18:50] @songngu.xyz I actually just turned my PC off for the night but I might jump back on in a few hours if needed or I'll handle tomorrow... [05:19:06] Oh [05:19:17] And it's the site links handled yet? [05:19:27] I ran a script. [05:19:35] I'm not sure if it worked [05:20:00] I might jump back on in 2-3 hrs if I am still awake. [05:20:04] I didn't see that in the log- [05:20:14] I didn't log it [05:20:17] oh [05:20:37] It wasn't working with our mwscript that auto logs so I ran manually and forgot to log [05:21:17] ah [05:21:35] I didn't believe that the script works just yet [05:21:55] Alright I'll look into it more in a few hours or tomorrow for sure [05:31:06] RECOVERY - valorantjp.wiki - reverse DNS on sslhost is OK: SSL OK - valorantjp.wiki reverse DNS resolves to cp1.wikitide.net - NS RECORDS OK [05:31:39] !log [@test1] starting deploy of {'config': True, 'versions': ['']} to [test1] [05:31:40] !log [@test1] finished deploy of {'config': True, 'versions': ['']} to [test1] - SUCCESS in 0s [05:31:41] Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log [05:31:43] Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log [05:50:11] PROBLEM - newcascadia.net - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.401 seconds: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out. [07:18:39] RECOVERY - newcascadia.net - reverse DNS on sslhost is OK: SSL OK - newcascadia.net reverse DNS resolves to cp1.wikitide.net - NS RECORDS OK [08:06:49] [02WikiTideOrg/dns] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/dns/compare/5d7ace99ebe9...7600bae80c4e [08:06:52] [02WikiTideOrg/dns] 07AgentIsai 037600bae - Add cloud4 and gh challenge [08:08:11] [02WikiTideOrg/dns] 07AgentIsai pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/dns/compare/7600bae80c4e...11b5ebccba7c [08:08:13] [02WikiTideOrg/dns] 07AgentIsai 0311b5ebc - Add GitHub challenge [09:50:49] PROBLEM - newcascadia.net - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out. [11:48:37] RECOVERY - newcascadia.net - reverse DNS on sslhost is OK: SSL OK - newcascadia.net reverse DNS resolves to cp1.wikitide.net - NS RECORDS OK [12:52:05] PROBLEM - newcascadia.net - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out. [14:48:48] RECOVERY - newcascadia.net - reverse DNS on sslhost is OK: SSL OK - newcascadia.net reverse DNS resolves to cp1.wikitide.net - NS RECORDS OK [15:22:07] PROBLEM - valorantjp.wiki - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query valorantjp.wiki. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [16:20:25] RECOVERY - valorantjp.wiki - reverse DNS on sslhost is OK: SSL OK - valorantjp.wiki reverse DNS resolves to cp1.wikitide.net - NS RECORDS OK [16:28:49] almost midnight for me rn lol [16:29:23] yet don't know what to do other than cleaning my messes in my wikibase or going to sleep- [16:31:19] PROBLEM - nexttide.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query nexttide.org. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [16:51:22] PROBLEM - newcascadia.net - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.404 seconds: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out. [17:22:56] let us prepare to site links those wiki again [17:23:28] finally populated the labels for all of the items that I can fix- [17:29:36] RECOVERY - nexttide.org - reverse DNS on sslhost is OK: SSL OK - nexttide.org reverse DNS resolves to cp1.wikitide.net - NS RECORDS OK [17:40:56] ill look later today [17:41:47] Going to sleep rn as it is almost 1 A.M [17:42:38] hope that the sciprt ran fine, or instead just bundle the SQL script to a file then ran it with sql.php- [18:06:12] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03fallback-api [+0/-0/±1] 13https://github.com/WikiTideOrg/MediaWikiDebugJS/commit/b2c862e723a9 [18:06:13] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega 03b2c862e - Add fallback and error handling to API request Signed-off-by: CosmicAlpha [18:06:16] [02MediaWikiDebugJS] 07Universal-Omega created branch 03fallback-api - 13https://github.com/WikiTideOrg/MediaWikiDebugJS [18:06:18] [02MediaWikiDebugJS] 07Universal-Omega opened pull request 03#6: Add fallback and error handling to API request - 13https://github.com/WikiTideOrg/MediaWikiDebugJS/pull/6 [18:10:55] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03fallback-api [+0/-0/±1] 13https://github.com/WikiTideOrg/MediaWikiDebugJS/compare/b2c862e723a9...8db6d67087fa [18:10:58] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega 038db6d67 - Fix Signed-off-by: CosmicAlpha [18:11:01] [02MediaWikiDebugJS] 07Universal-Omega synchronize pull request 03#6: Add fallback and error handling to API request - 13https://github.com/WikiTideOrg/MediaWikiDebugJS/pull/6 [18:20:22] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03fallback-api [+0/-0/±1] 13https://github.com/WikiTideOrg/MediaWikiDebugJS/compare/8db6d67087fa...6bf6f980776f [18:20:25] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega 036bf6f98 - Fix Signed-off-by: CosmicAlpha [18:20:26] [02MediaWikiDebugJS] 07Universal-Omega synchronize pull request 03#6: Add fallback and error handling to API request - 13https://github.com/WikiTideOrg/MediaWikiDebugJS/pull/6 [18:23:53] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03fallback-api [+0/-0/±1] 13https://github.com/WikiTideOrg/MediaWikiDebugJS/compare/6bf6f980776f...2fa71db24e68 [18:23:56] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega 032fa71db - Just log Signed-off-by: CosmicAlpha [18:23:58] [02MediaWikiDebugJS] 07Universal-Omega synchronize pull request 03#6: Add fallback and error handling to API request - 13https://github.com/WikiTideOrg/MediaWikiDebugJS/pull/6 [18:28:06] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03fallback-api [+0/-0/±1] 13https://github.com/WikiTideOrg/MediaWikiDebugJS/compare/2fa71db24e68...987a0c8f56f9 [18:28:08] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega 03987a0c8 - Fix Signed-off-by: CosmicAlpha [18:28:11] [02MediaWikiDebugJS] 07Universal-Omega synchronize pull request 03#6: Add fallback and error handling to API request - 13https://github.com/WikiTideOrg/MediaWikiDebugJS/pull/6 [18:36:26] [02MediaWikiDebugJS] 07Universal-Omega closed pull request 03#6: Add fallback and error handling to API request - 13https://github.com/WikiTideOrg/MediaWikiDebugJS/pull/6 [18:36:28] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/MediaWikiDebugJS/compare/501ad798dc68...6b0dc4a9182d [18:36:29] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega 036b0dc4a - Add fallback and error handling to API request Signed-off-by: CosmicAlpha [18:36:32] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega deleted branch 03fallback-api [18:36:34] [02MediaWikiDebugJS] 07Universal-Omega deleted branch 03fallback-api - 13https://github.com/WikiTideOrg/MediaWikiDebugJS [19:01:09] [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/3d4d870af3ec...3d0e6b5d221b [19:01:10] [02WikiTideOrg/puppet] 07Universal-Omega 033d0e6b5 - matomo: remove DB SSL Signed-off-by: CosmicAlpha [19:02:37] !log remove REQUIRE SSL from matomo user on db1 [19:02:40] Logged the message at https://meta.wikitide.org/wiki/Tech:Server_admin_log [19:20:53] PROBLEM - valorantjp.wiki - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query valorantjp.wiki. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [19:31:17] [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/3d0e6b5d221b...0ce86197c3ef [19:31:19] [02WikiTideOrg/puppet] 07Universal-Omega 030ce8619 - matomo: upgrade to 4.16.0 Signed-off-by: CosmicAlpha [19:41:37] PROBLEM - matomo1 APT on matomo1 is CRITICAL: APT CRITICAL: 20 packages available for upgrade (3 critical updates). [19:50:18] PROBLEM - test1 JobChron Service on test1 is CRITICAL: PROCS CRITICAL: 0 processes with args 'redisJobChronService' [20:01:42] RECOVERY - test1 JobChron Service on test1 is OK: PROCS OK: 1 process with args 'redisJobChronService' [20:25:42] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03editthis [+0/-0/±1] 13https://github.com/WikiTideOrg/MediaWikiDebugJS/commit/89d4ce933041 [20:25:45] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega 0389d4ce9 - Add support for editthis.info Signed-off-by: CosmicAlpha [20:25:48] [02MediaWikiDebugJS] 07Universal-Omega created branch 03editthis - 13https://github.com/WikiTideOrg/MediaWikiDebugJS [20:25:54] [02MediaWikiDebugJS] 07Universal-Omega opened pull request 03#7: Add support for editthis.info - 13https://github.com/WikiTideOrg/MediaWikiDebugJS/pull/7 [20:34:09] PROBLEM - nexttide.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query nexttide.org. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [20:50:06] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03editthis [+0/-0/±1] 13https://github.com/WikiTideOrg/MediaWikiDebugJS/compare/89d4ce933041...b979eb075881 [20:50:08] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega 03b979eb0 - Fix Signed-off-by: CosmicAlpha [20:50:09] [02MediaWikiDebugJS] 07Universal-Omega synchronize pull request 03#7: Add support for editthis.info - 13https://github.com/WikiTideOrg/MediaWikiDebugJS/pull/7 [20:54:14] [02MediaWikiDebugJS] 07Universal-Omega synchronize pull request 03#7: Add support for editthis.info - 13https://github.com/WikiTideOrg/MediaWikiDebugJS/pull/7 [20:54:15] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03editthis [+0/-0/±1] 13https://github.com/WikiTideOrg/MediaWikiDebugJS/compare/b979eb075881...5b447a945cce [20:54:17] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega 035b447a9 - Exit if non-mediawiki Signed-off-by: CosmicAlpha [21:00:47] [02MediaWikiDebugJS] 07Universal-Omega closed pull request 03#7: Add support for editthis.info - 13https://github.com/WikiTideOrg/MediaWikiDebugJS/pull/7 [21:00:48] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/MediaWikiDebugJS/compare/6b0dc4a9182d...26abc4b55585 [21:00:49] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega 0326abc4b - Add support for editthis.info Signed-off-by: CosmicAlpha [21:00:50] [02WikiTideOrg/MediaWikiDebugJS] 07Universal-Omega deleted branch 03editthis [21:00:51] [02MediaWikiDebugJS] 07Universal-Omega deleted branch 03editthis - 13https://github.com/WikiTideOrg/MediaWikiDebugJS [21:03:27] RECOVERY - nexttide.org - reverse DNS on sslhost is OK: SSL OK - nexttide.org reverse DNS resolves to cp1.wikitide.net - NS RECORDS OK [21:16:14] RECOVERY - newcascadia.net - reverse DNS on sslhost is OK: SSL OK - newcascadia.net reverse DNS resolves to cp1.wikitide.net - NS RECORDS OK [21:31:11] PROBLEM - mw2 MediaWiki Rendering on mw2 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [21:33:05] PROBLEM - mw2 MediaWiki Rendering on mw2 is WARNING: HTTP WARNING: HTTP/1.1 404 Not Found - 8191 bytes in 0.174 second response time [21:40:26] [02WikiTideOrg/puppet] 07Universal-Omega created branch 03grafana-sqlite 13https://github.com/WikiTideOrg/puppet/commit/0ce86197c3ef6b8c8f0efe45a3b56b49dd120142 [21:40:28] [02puppet] 07Universal-Omega created branch 03grafana-sqlite - 13https://github.com/WikiTideOrg/puppet [21:42:20] [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03grafana-sqlite [+0/-0/±5] 13https://github.com/WikiTideOrg/puppet/compare/0ce86197c3ef...55b08fff0b9b [21:42:22] [02WikiTideOrg/puppet] 07Universal-Omega 0355b08ff - grafana: use sqlite [21:42:32] [02puppet] 07Universal-Omega opened pull request 03#35: grafana: use sqlite - 13https://github.com/WikiTideOrg/puppet/pull/35 [21:45:27] RECOVERY - valorantjp.wiki - reverse DNS on sslhost is OK: SSL OK - valorantjp.wiki reverse DNS resolves to cp1.wikitide.net - NS RECORDS OK [22:36:50] PROBLEM - nexttide.org - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query nexttide.org. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [22:45:08] PROBLEM - valorantjp.wiki - reverse DNS on sslhost is WARNING: NoNameservers: All nameservers failed to answer the query valorantjp.wiki. IN NS: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered SERVFAIL [22:48:58] PROBLEM - newcascadia.net - reverse DNS on sslhost is WARNING: LifetimeTimeout: The resolution lifetime expired after 5.402 seconds: Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out.; Server 1.1.1.1 UDP port 53 answered The DNS operation timed out. [23:13:54] RECOVERY - valorantjp.wiki - reverse DNS on sslhost is OK: SSL OK - valorantjp.wiki reverse DNS resolves to cp1.wikitide.net - NS RECORDS OK [23:17:00] [02puppet] 07Universal-Omega closed pull request 03#35: grafana: use sqlite - 13https://github.com/WikiTideOrg/puppet/pull/35 [23:17:02] [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±5] 13https://github.com/WikiTideOrg/puppet/compare/0ce86197c3ef...cddd616f203d [23:17:05] [02WikiTideOrg/puppet] 07Universal-Omega 03cddd616 - grafana: use sqlite [23:17:07] [02WikiTideOrg/puppet] 07Universal-Omega deleted branch 03grafana-sqlite [23:17:09] [02puppet] 07Universal-Omega deleted branch 03grafana-sqlite - 13https://github.com/WikiTideOrg/puppet [23:17:20] [02puppet] 07Universal-Omega closed pull request 03#27: recursor: add IPv6 support - 13https://github.com/WikiTideOrg/puppet/pull/27 [23:17:21] [02WikiTideOrg/puppet] 07Universal-Omega pushed 031 commit to 03master [+0/-0/±1] 13https://github.com/WikiTideOrg/puppet/compare/cddd616f203d...0d9b93c9410b [23:17:22] [02WikiTideOrg/puppet] 07Universal-Omega 030d9b93c - recursor: add IPv6 support [23:17:23] [02WikiTideOrg/puppet] 07Universal-Omega deleted branch 03recursor-ipv6 [23:17:24] [02puppet] 07Universal-Omega deleted branch 03recursor-ipv6 - 13https://github.com/WikiTideOrg/puppet [23:20:11] PROBLEM - mon1 Backups Grafana on mon1 is CRITICAL: FILE_AGE CRITICAL: File not found - /var/log/grafana-backup.log [23:47:19] RECOVERY - newcascadia.net - reverse DNS on sslhost is OK: SSL OK - newcascadia.net reverse DNS resolves to cp1.wikitide.net - NS RECORDS OK