[13:53:02] <jem>	 Hi, is there any possibility to rescue a Toolforge bastion file from a 2-4 days old backup?
[14:38:45] <dhinus>	 andrewbogott: ^ in theory we have VM backups for 7 days don't we?
[14:42:01] <andrewbogott>	 dhinus: it would be a cinder backup. jem is making a ticket, I will probably assign it to you :)
[14:42:34] <andrewbogott>	 we would/will have to restore the whole nfs volume
[14:45:41] <jem>	 Ticket opened: T381582
[14:45:41] <stashbot>	 T381582: Rescue file from Toolforge backup - https://phabricator.wikimedia.org/T381582
[14:45:52] <jem>	 Thanks, andrewbogott and dhinus
[14:46:51] <jem>	 I'm marking it High priority as andrewbogott pointed me that backups are refreshed after a week
[14:50:58] <dhinus>	 sounds good, I'll have a look
[14:52:12] <andrewbogott>	 thanks dhinus 
[15:35:53] <TheresNoTime>	 hihi, trying to get my bot (`TNTBot`) to log in to wikitech, but getting the error "Auto-creation of a local account failed: You are not allowed to execute the action you have requested." when trying to do an initial manual login. 
[15:37:20] <TheresNoTime>	 is this a case of using `Special:CreateLocalAccount` to force a creation, or would that Break Things(tm)?
[15:38:24] <AntiComposite>	 that's a new one
[15:38:59] <AntiComposite>	 are you on an IP that would be globally blocked?
[15:39:04] <AntiComposite>	 https://meta.wikimedia.org/wiki/Special:CentralAuth/TNTBot
[15:39:37] <TheresNoTime>	 nae, logging in as normal from home
[15:39:46] <AntiComposite>	 bot has GIPBE so that should allow local autocreation over blocks now anyway...
[15:40:20] <taavi>	 https://gerrit.wikimedia.org/g/operations/mediawiki-config/+/1a1c224eaefdd48d5585e08a1694357b7d4d1fca/wmf-config/core-Permissions.php#1652
[15:40:33] <taavi>	 TheresNoTime: i think you should be safe to Special:CreateLocalAccount
[15:40:41] <taavi>	 athough i wonder why it's disabled in the first place
[15:40:58] <TheresNoTime>	 judging by https://phabricator.wikimedia.org/T376267#10380054 down, seems like forcibly creating is probably okay
[15:41:51] * TheresNoTime will wait another few minutes to be told not to & then will try
[15:43:19] <stw>	 https://gerrit.wikimedia.org/r/plugins/gitiles/operations/mediawiki-config/+/7837cd275a27a8f50c182b523d74ca6cb1bfef21%5E%21/wmf-config/core-Permissions.php
[15:43:25] <AntiComposite>	 https://phabricator.wikimedia.org/T161859#10196071
[15:43:43] <AntiComposite>	 https://phabricator.wikimedia.org/T377074 is the task to re-open account creation, no response on that task or the parent
[15:49:30] <TheresNoTime>	 !log forcibly created [[User:TNTBot]] via Special:CreateLocalAccount
[15:49:32] <stashbot>	 TheresNoTime: Unknown project "forcibly"
[15:49:36] <TheresNoTime>	 :)
[17:18:06] <wm-bb>	 <Nintendofan885> looks like FlickreviewR 2 needs a restart again
[18:47:34] <wm-bb>	 <lucaswerkmeister> have the maintainers been notified?
[18:52:03] <AntiComposite>	 nope
[18:54:13] <AntiComposite>	 https://commons.wikimedia.org/wiki/User_talk:Zhuyifei1999#c-AntiCompositeNumber-20241205185300-FlickreviewR_2_has_stopped_again
[19:00:40] <andrewbogott>	 I can certainly 'webservice restart' something but there is no tool named flickreviewr so I assume this requires deeper understanding
[19:03:07] <wm-bb>	 <lucaswerkmeister> https://sal.toolforge.org/log/Cl0Ef5MBFFSCpsJzSJqD would be the command to run
[19:03:22] <wm-bb>	 <lucaswerkmeister> but I think I’d like to give the maintainers some more time to fix it
[19:03:44] <andrewbogott>	 ok
[19:04:10] <andrewbogott>	 I'm getting intermittent pages from the toolforge test tool, are people noticing flakiness other than flickreviewr?
[19:07:24] <AntiComposite>	 my bots have all been fine
[19:07:37] <andrewbogott>	 that's good!
[19:09:39] <wm-bb>	 <lucaswerkmeister> quickcategories background runner also looks fine to me
[19:14:37] <andrewbogott>	 great, then I'm going to close my laptop and head for the airport. I assume I'll get paged again at the worst possible time.
[20:09:28] <wm-bb>	 <lucaswerkmeister> I replied on-wiki – it looks like tools getting stuck waiting for a response from text-lb.eqiad is a recurring problem
[20:09:51] <wm-bb>	 <lucaswerkmeister> sounds like hell to debug though :/
[20:10:35] <wm-bb>	 <lucaswerkmeister> /me wonders if T374830 could be at all related to this
[20:11:14] <wm-bb>	 <lucaswerkmeister> (based on no other evidence than “both are sporadic networking issues inside cloud VPS”)