[06:57:01] 3Wikimedia Labs / 3tools: tools-db is down; need to flush hosts - 10https://bugzilla.wikimedia.org/69828#c2 (10bgwhite) For the past few weeks, web services have continually died. Big Brother does restart it. However, on which webgrid node things restart on can be random. Long story short, a good part of... [07:05:18] 3Wikimedia Labs / 3tools: Web services continually restarting - 10https://bugzilla.wikimedia.org/69934 (10bgwhite) 3UNCO p:3Unprio s:3normal a:3Marc A. Pelletier Since August 17, web services have continually gone up and down. Big Brother does restart web services. The few times I was logged into l... [07:10:50] Hi [07:10:56] any one here [07:19:58] no [07:37:58] Hi. Anybody able to help with my failing database connection (from java/tomcat to the osm postgresql-database on tool-labs)? [08:03:47] 3Wikimedia Labs / 3tools: Include pagecounts dumps in datasets - 10https://bugzilla.wikimedia.org/48894#c25 (10jeremyb) (In reply to Tim Landscheidt from comment #24) > New server tracked in RT #7578. I can't see that. I assume it's in procurement. There's several more relevant tickets e.g. RT 7948, RT 809... [09:38:45] 3Wikimedia Labs / 3tools: Web services continually restarting - 10https://bugzilla.wikimedia.org/69934#c1 (10metatron) 5UNCO>3NEW p:5Unprio>3High I can confirm this. Multiple restarts for no clear reason. In both emaples a running service had been terminated + restarted automatically. Bigbrother mai... [09:42:31] 3Wikimedia Labs / 3tools: Web services continually restarting - 10https://bugzilla.wikimedia.org/69934#c2 (10metatron) *examples [11:13:00] 3Wikimedia Labs / 3deployment-prep (beta): bits.beta.wmflabs.org down with 503 - 10https://bugzilla.wikimedia.org/69921#c3 (10physikerwelt) 5PATC>3NEW The link seems to be still broken (503). [15:04:38] whym: your query was stuck for 13h, I just got it unstuck. investigating the underlying cause [15:14:46] YuviPanda: not sure which one it was, but thanks. [15:15:40] YuviPanda: Thanks also for fixing the non-latin character problem. [15:16:50] whym: :) yw! thanks for reporting [17:07:52] !log tools.gerrit-reviewer-bot Deployed d1515d24eac39b578529390c10e243a3053e76ac unix-style matching for project names [17:07:54] Logged the message, Master [17:42:00] 3Wikimedia Labs / 3tools: I can't access ptwiki_p database - 10https://bugzilla.wikimedia.org/69701#c1 (10Alex Monk) Is this still broken? It's working for me. [17:44:03] !log tools qmod -cq task@tools-exec-07 (job #2796555, "11 : before job") [17:44:06] Logged the message, Master [18:21:15] 3Wikimedia Labs / 3tools: dewiki / wikidatawiki replication (s5) has stopped on labsdb1001/labsdb1002 - 10https://bugzilla.wikimedia.org/69776#c1 (10Alex Monk) Is this still broken? I can see that new revisions are going in to the revision table when I `sql dewiki` (or directly to the mentioned hosts with `m... [19:41:30] 3Wikimedia Labs / 3tools: I can't access ptwiki_p database - 10https://bugzilla.wikimedia.org/69701#c2 (10Rodrigo Sampaio Primo) Yes, it is still broken for me. [20:02:05] did someone restart wm-bot recently?? [20:07:12] yes [20:07:15] it was broken for a long time [20:07:22] THO restarted it a few days ago [23:44:42] scfc_de: legoktm do you know if user accounts (not tool accounts) can submit jobs? [23:44:51] I guess they can't... [23:45:19] Coren: ^ [23:45:59] YuviPanda: You can submit a task (non-continuous) job only, but it's not a supported thing. [23:46:10] hmm, right [23:46:43] Coren: jsub doen't print a success message, and qstat is always empty... [23:47:25] Like I said, "not supported". I didn't disable it, but it's not something I test on changes. [23:47:32] right [23:47:35] I guess it's broken then [23:47:36] oh well [23:47:54] Might just be jsub. Have you checked the .err file? [23:48:04] (Or use -stderr with jsub) [23:56:58] Coren: ah, yes. not executable