[01:36:01] !log tools adding additional worker node in response to repeated capacity alerts [01:36:03] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [03:09:19] Quick question: Since T183490, it's no longer possible to access page content via Quarry, right? I don't know what to do with "es:DB://cluster1/692465?flags=utf-8,gzip" [03:16:00] the WMCS replicas have never supported page content. If you need page content, you'll need to use dumps or the API [03:16:28] !log tools acking and silencing CPU capacity alerts to handle on Monday, they should not page [03:16:32] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools/SAL [06:23:26] Is there some issue with the Toolforge jobs framework recently? Over the past few days, many of my jobs that usually finish within seconds have been taking anywhere from 1 to 5 hours. I heard another user mention the same, but I could not find anything on Phabricator, so I am asking here in case there is something I am not aware of [06:25:58] dcaro: Can you please check this? [07:38:23] DreamRimmer: can you open a task with the tool and the list of jobs you are seeing slowness with? [07:58:01] sure [09:13:33] I manually restarted three of the kubernetes workers that had some pods stuck on NFS (being worked on T404584), and saw one of your tools there I think, so that might help. Overall the cluster is also having some capacity issues that will get sorted early next week (T404726) [09:13:34] T404584: Address tools NFS getting stuck with processes in D state - https://phabricator.wikimedia.org/T404584 [09:13:34] T404726: [tools,infra,k8s] scale up the cluster, specifically CPU - https://phabricator.wikimedia.org/T404726 [13:49:02] !log peterbowman@tools-bastion-14 tools.pbbot Fix NKJP servlet and legacy SGE-era links, bump MySQL/J connector [13:49:05] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.pbbot/SAL [19:58:27] !log jeanfred@tools-bastion-15 tools.heritage Reloaded SQL table configuration for 0fe6c07 (T346681) [19:58:32] Logged the message at https://wikitech.wikimedia.org/wiki/Nova_Resource:Tools.heritage/SAL