[00:10:55] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 1.35 ms [00:16:20] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.38 deployment blockers - https://phabricator.wikimedia.org/T254175 (10brennen) Will do a `scap sync` at local start-of-workday for T256151 - see T256151#6250427 for de... [00:20:51] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [01:45:52] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 0.43 ms [01:50:50] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [01:55:53] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 0.72 ms [02:05:52] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [03:30:55] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 1.04 ms [03:40:51] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [03:58:08] 10Project-Admins: Create project for "dark mode" support - https://phabricator.wikimedia.org/T256175 (10Aklapper) 05Invalid→03Declined [04:22:54] 10Gerrit, 10Wikibugs, 10Patch-For-Review: Prepare wikibugs gerrit bot for gerrit 2.14 / 2.15 - https://phabricator.wikimedia.org/T175929 (10Legoktm) I did a bit of testing against gerrit-test, and most everything looks like it works... I tested creating a new change, abandoning, rebasing. The only scenario I... [06:01:41] 10Project-Admins, 10Math, 10PM: Organize the open tasks for Project Math - https://phabricator.wikimedia.org/T239359 (10Aklapper) 05Open→03Resolved a:03Aklapper [x] Created new parent project tag #Math-Chemistry-Support: https://phabricator.wikimedia.org/project/view/4847/ [x] Created new parent projec... [07:03:35] 10Continuous-Integration-Config, 10Wikispeech-Jobrunner, 10Wikispeech-Text-to-Speech, 10Wikispeech-WMSE: Coverage report disappeared from doc.wikimedia.org/cover-extensions/Wikispeech/ - https://phabricator.wikimedia.org/T256197 (10Lokal_Profil) [07:19:54] 10Continuous-Integration-Config, 10Wikispeech-Jobrunner, 10Wikispeech-Text-to-Speech, 10Wikispeech-WMSE: Coverage report disappeared from doc.wikimedia.org/cover-extensions/Wikispeech/ - https://phabricator.wikimedia.org/T256197 (10Sebastian_Berlin-WMSE) It usually works for me. If I keep "hard" refreshing... [07:32:01] 10Diffusion: Can't push to diffusion - https://phabricator.wikimedia.org/T243848 (10Magnus) tools.quickstatements@tools-sgebastion-07:~$ git remote remove origin tools.quickstatements@tools-sgebastion-07:~$ git remote add origin ssh://phabricator.wikimedia.org/diffusion/2010/tool-quickstatements.git/ tools.quick... [07:34:33] 10Diffusion: Can't push to diffusion - https://phabricator.wikimedia.org/T243848 (10Magnus) Tried again with `ssh://vcs@git-ssh.wikimedia.org/source/tool-quickstatements.git`, no joy: ` Permission denied (publickey,keyboard-interactive). fatal: Could not read from remote repository. Please make sure you have th... [07:38:30] 10Diffusion: Can't push to diffusion - https://phabricator.wikimedia.org/T243848 (10RhinosF1) > tools.quickstatements@tools-sgebastion-07:~$ git push --set-upstream origin master ssh: connect to host phabricator.wikimedia.org port 22: Network is unreachable fatal: Could not read from remote repository. What's y... [07:59:12] 10Continuous-Integration-Config, 10Wikispeech-Jobrunner, 10Wikispeech-Text-to-Speech, 10Wikispeech-WMSE: Coverage report disappeared from doc.wikimedia.org/cover-extensions/Wikispeech/ - https://phabricator.wikimedia.org/T256197 (10hashar) The code serving that page is in `integration/docroot.git` in `shar... [08:03:42] 10Diffusion: Can't push to diffusion - https://phabricator.wikimedia.org/T243848 (10Magnus) 05Declined→03Resolved I have "forked" it into github. Using diffusion was a mistake in the first place. Now at https://github.com/magnusmanske/quickstatements [08:28:06] 10VPS-project-codesearch: Switch codesearch to codesearch.wmcloud.org - https://phabricator.wikimedia.org/T256206 (10Legoktm) [08:31:54] 10Diffusion: Can't push to diffusion - https://phabricator.wikimedia.org/T243848 (10Aklapper) @Magnus: Does that mean that https://phabricator.wikimedia.org/source/tool-quickstatements/browse/master/ will get out of sync and should be archived? Asking also as I lost track what's hosted on https://bitbucket.org/m... [08:37:40] (03CR) 10Jforrester: "> Patch Set 1:" [integration/config] - 10https://gerrit.wikimedia.org/r/606797 (owner: 10ProcrastinatingReader) [08:37:53] (03PS2) 10Jforrester: Add proc to CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/606797 (owner: 10ProcrastinatingReader) [08:38:20] (03PS3) 10Jforrester: Add proc to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/606797 (owner: 10ProcrastinatingReader) [08:38:29] (03CR) 10Jforrester: [C: 03+2] Add proc to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/606797 (owner: 10ProcrastinatingReader) [08:39:36] (03Merged) 10jenkins-bot: Add proc to CI allowlist [integration/config] - 10https://gerrit.wikimedia.org/r/606797 (owner: 10ProcrastinatingReader) [08:40:25] !log Zuul: Add proc to CI allowlist [08:40:26] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:40:49] (03CR) 10Jforrester: [C: 03+2] Replace fabfile.py with a shell script [integration/config] - 10https://gerrit.wikimedia.org/r/605434 (https://phabricator.wikimedia.org/T236689) (owner: 10Krinkle) [08:42:02] (03Merged) 10jenkins-bot: Replace fabfile.py with a shell script [integration/config] - 10https://gerrit.wikimedia.org/r/605434 (https://phabricator.wikimedia.org/T236689) (owner: 10Krinkle) [08:45:59] 10Diffusion: Can't push to diffusion - https://phabricator.wikimedia.org/T243848 (10Magnus) >>! In T243848#6251382, @Aklapper wrote: > @Magnus: Does that mean that https://phabricator.wikimedia.org/source/tool-quickstatements/browse/master/ will get out of sync and should be archived? Yes. > Asking also as I lo... [08:52:12] 10VPS-project-codesearch, 10cloud-services-team (Kanban): Switch codesearch to codesearch.wmcloud.org - https://phabricator.wikimedia.org/T256206 (10aborrero) [08:52:47] 10VPS-project-codesearch, 10cloud-services-team (Kanban): Switch codesearch to codesearch.wmcloud.org - https://phabricator.wikimedia.org/T256206 (10aborrero) p:05Triage→03Medium For the WMCS team meeting: clarify our workflow for this kind of requests. [09:09:16] 10LibUp: Puppetize setup on upgrader-06.wmflabs - https://phabricator.wikimedia.org/T173478 (10Legoktm) a:03Legoktm Going to take a stab at this again. @dzahn: I'll be basing this off the codesearch puppetization we did. I added you to the `library-upgrader` project as well in case you can help out again :) [09:30:41] 10Gerrit: Gerrit upload-pack send ALL references causing massive network I/O on common operations - https://phabricator.wikimedia.org/T103990 (10hashar) On the Gerrit 3.2 server gerrit-test.wikimedia.org, protocol v2 is definitely faster: ` $ git remote -v origin https://gerrit-test.wikimedia.org/r/operations/pu... [09:40:12] Project beta-scap-eqiad build #305889: 04FAILURE in 5 min 33 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/305889/ [09:42:24] 10LibUp, 10Patch-For-Review: Puppetize setup on upgrader-06.wmflabs - https://phabricator.wikimedia.org/T173478 (10Legoktm) Once ^ is merged, I created an `upgrader-07` instance to test with. After that patch, the next step is to migrate over the systemd units. [09:45:51] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 3.30 ms [09:48:11] Yippee, build fixed! [09:48:12] Project beta-scap-eqiad build #305890: 09FIXED in 3 min 39 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/305890/ [09:50:50] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [10:01:27] 10Gerrit: Make a PolyGerrit plugin to welcome new users - https://phabricator.wikimedia.org/T201246 (10Aklapper) [10:01:32] 10Gerrit, 10Patch-For-Review: Gerrit 3.x upgrade - https://phabricator.wikimedia.org/T254158 (10Aklapper) [10:05:53] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 1.45 ms [10:10:54] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [10:44:33] qchris: eventually gerrit-test died somehow :-] [10:45:19] I guess some operations are going on. Not a big deal, I was just trying to debug something. That can wait [10:50:53] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 0.46 ms [10:55:49] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [11:15:57] 10Project-Admins, 10WPCleaner: Create a project/tag for WPCleaner in Phabricator - https://phabricator.wikimedia.org/T256026 (10NicoV) [11:30:59] 10Project-Admins, 10Structured-Data-Backlog: Clarify confusion between Multimedia and Structured Data team and update docs accordingly - https://phabricator.wikimedia.org/T240281 (10dr0ptp4kt) Thanks for posting those @Krinkle. As part of our Code Health objective in Product Engineering in this forthcoming fis... [11:55:32] (03PS1) 10Jforrester: jjjb: [language-screenshots-VisualEditor] Migrate to node 10 [integration/config] - 10https://gerrit.wikimedia.org/r/607478 [12:20:06] Project beta-update-databases-eqiad build #43076: 04FAILURE in 5.2 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43076/ [12:25:32] 10Gerrit: Gerrit upload-pack send ALL references causing massive network I/O on common operations - https://phabricator.wikimedia.org/T103990 (10hashar) I guess that is working as intended, it fetch every single objects when there is no refspec set. A remote does have a default set to `+refs/heads/*:refs/remote... [12:41:08] (03CR) 10Hashar: [C: 04-1] "That seems like a useless abstraction of a list. I do not see the point in this, even after reading the other change https://gerrit.wikime" [integration/quibble] - 10https://gerrit.wikimedia.org/r/588083 (owner: 10Awight) [12:43:42] (03CR) 10Hashar: [C: 04-1] "docopt seems fine for some small utilities, but I would rather stick to argparse. There are some tricks that do not seem to be possibly ex" [integration/quibble] - 10https://gerrit.wikimedia.org/r/546167 (owner: 10Awight) [12:57:02] (03CR) 10Awight: "> That seems like a useless abstraction of a list." [integration/quibble] - 10https://gerrit.wikimedia.org/r/588083 (owner: 10Awight) [12:58:09] (03Abandoned) 10Awight: Replace argparse with docopt [integration/quibble] - 10https://gerrit.wikimedia.org/r/546167 (owner: 10Awight) [13:01:49] awight: yeah sorry looks like I thought I had commented on those quibble patches but actually never did :-\ [13:02:15] hashar: Today in the morning, I got notice that Ops need to reboot the host around the time you pinged me. Still catching up on messages, but according to them Gerrit was up again at 12:51 CEST the latest. [13:02:23] Looks like it's still up. [13:02:37] qchris: yeah no worries. I was just in the middle of debugging something with git protocol v2 ;] [13:02:44] I just went for lunch instead \o/ [13:02:55] Lunch is better than git v2 anyways :-D [13:03:01] if one does git -c protocol.version=2 fetch https://somerepo.git [13:03:06] that still fetches everything [13:03:18] but if one passes a refspec to fetch, that is super fast anyway [13:03:20] Argh! [13:03:21] so not a big deal ;-] [13:03:38] I guess it is expected to fallback to the protocol v1 and just require every single reference in that very very specific case [13:03:38] Tbh, I don't really know much about git protocol v2. [13:03:53] turns out I don't think we use that specific pattern so we are all fine [13:04:01] I just know that one has to switch it on explicitly in Gerrit (which is what we did) [13:04:19] the devil eventually is that I get to get a recent git on all containers and in case v2 is not the default add a gitcnfig file to force it on [13:04:28] yeah [13:04:36] none of that is a blocker for the gerrit upgrade anyway [13:04:41] I am just preparing ;-] [13:04:47] Ok. Cool :-D [13:05:08] I had verified that git protocol v2 was usable from our gerrit, but that's about it. [13:05:15] yeah that is good [13:12:56] 10Diffusion: Set Git configuration on all Phab repositories - https://phabricator.wikimedia.org/T107156 (10hashar) 05Open→03Declined That seems like an obsolete task. [13:20:01] Project beta-update-databases-eqiad build #43077: 04STILL FAILING in 1 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43077/ [13:20:29] (03CR) 10Hashar: [C: 03+2] Run Quibble fullrun job after tox [integration/config] - 10https://gerrit.wikimedia.org/r/606492 (https://phabricator.wikimedia.org/T235118) (owner: 10Hashar) [13:21:45] (03Merged) 10jenkins-bot: Run Quibble fullrun job after tox [integration/config] - 10https://gerrit.wikimedia.org/r/606492 (https://phabricator.wikimedia.org/T235118) (owner: 10Hashar) [13:31:44] (03Abandoned) 10Hashar: dockerfile for mediawiki-vendor version checker script [integration/config] - 10https://gerrit.wikimedia.org/r/560413 (https://phabricator.wikimedia.org/T179663) (owner: 10Addshore) [13:36:24] (03PS1) 10Hashar: tox: skip sdist and use develop to install [integration/quibble] - 10https://gerrit.wikimedia.org/r/607506 [13:49:07] 10Continuous-Integration-Infrastructure, 10Patch-For-Review, 10User-Zoranzoki21: Change Zuul status page icons to support unofficial dark mode (transparent) - https://phabricator.wikimedia.org/T256132 (10MrGusted) Oh, Il make a fix for that in Dark Reader, with a simple invert it should become white, as I do... [13:50:51] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 0.75 ms [13:51:12] 10Continuous-Integration-Infrastructure, 10Patch-For-Review, 10User-Zoranzoki21: Change Zuul status page icons to support unofficial dark mode (transparent) - https://phabricator.wikimedia.org/T256132 (10MrGusted) https://github.com/darkreader/darkreader/pull/2929 [13:57:26] (03PS1) 10Hashar: (WIP WIP WIP) test python 3.5, 3.6, 3.7 [integration/quibble] - 10https://gerrit.wikimedia.org/r/607512 [14:00:49] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [14:03:34] ^^ that is "normal" [14:03:41] I deleted it, shinken is out of date bah [14:04:14] It should only warn once, though. [14:04:19] And it's been warning for a while. [14:10:19] (03CR) 10Hashar: "CI lacks Python 3.8 support :-\" [integration/utils] - 10https://gerrit.wikimedia.org/r/606846 (owner: 10Legoktm) [14:20:02] Project beta-update-databases-eqiad build #43078: 04STILL FAILING in 1.1 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43078/ [14:31:59] (03CR) 10Hashar: "Invoking Volans cause he knows about tox magics :] In short the aim is to run tests on CI with 3.5, 3.6, 3.7 but locally just use whateve" [integration/quibble] - 10https://gerrit.wikimedia.org/r/607512 (owner: 10Hashar) [14:35:52] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 1.45 ms [14:37:23] 10Gerrit: Make a PolyGerrit plugin to welcome new users - https://phabricator.wikimedia.org/T201246 (10Apap04) > Im wondering how do we want this to look? Do we want to put this under the top bar? Google has a little bar above the navigation bar telling the users to activate "critical notifications" on [[ https:... [14:40:53] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [15:04:18] (03Abandoned) 10Daimona Eaytoy: Add CirrusSearch as phan dependency for Wikibase [integration/config] - 10https://gerrit.wikimedia.org/r/541227 (owner: 10Daimona Eaytoy) [15:21:15] 10VPS-project-codesearch, 10cloud-services-team (Kanban): Switch codesearch to codesearch.wmcloud.org - https://phabricator.wikimedia.org/T256206 (10Andrew) [15:22:00] PROBLEM - Parsoid on deployment-parsoid11 is CRITICAL: connect to address 172.16.1.115 and port 8000: Connection refused [15:22:11] Yippee, build fixed! [15:22:11] Project beta-update-databases-eqiad build #43079: 09FIXED in 2 min 10 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/43079/ [15:47:05] 10Release-Engineering-Team, 10Product-Infrastructure-Team-Backlog, 10Push-Notification-Service, 10serviceops: Migrate push-notifications service to latest node LTS version - https://phabricator.wikimedia.org/T256131 (10LGoto) p:05Triage→03Medium [16:16:32] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Quibble, 10Patch-For-Review: Create an integration test running Quibble with mediawiki/core - https://phabricator.wikimedia.org/T235118 (10hashar) 05Open→03Resolved [16:18:25] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Quibble, 10Patch-For-Review: Create an integration test running Quibble with mediawiki/core - https://phabricator.wikimedia.org/T235118 (10Jdforrester-WMF) 🎉 [16:23:58] PROBLEM - App Server Main HTTP Response on deployment-mediawiki-07 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:25:52] PROBLEM - English Wikipedia Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:26:04] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10Release Pipeline: Create WMF Cassandra image - https://phabricator.wikimedia.org/T256281 (10jeena) [16:30:41] RECOVERY - English Wikipedia Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 97695 bytes in 1.250 second response time [16:32:32] !log Cherry-picking new version of change 603550 into beta puppet for testing, and pointing beta XHGui at new instance. [16:32:33] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:36:15] PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:37:53] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.38 deployment blockers - https://phabricator.wikimedia.org/T254175 (10brennen) [16:41:04] RECOVERY - English Wikipedia Mobile Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 60972 bytes in 2.202 second response time [16:43:48] RECOVERY - App Server Main HTTP Response on deployment-mediawiki-07 is OK: HTTP OK: HTTP/1.1 200 OK - 97385 bytes in 1.104 second response time [16:56:03] 10MediaWiki-Releasing, 10Security: Release MediaWiki 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248534 (10Reedy) [16:56:14] 10MediaWiki-Releasing, 10Security: Tracking bug for MediaWiki 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248535 (10Reedy) [16:56:30] PROBLEM - Host deployment-changeprop is DOWN: CRITICAL - Host Unreachable (172.16.5.21) [16:56:30] PROBLEM - Host deployment-cpjobqueue is DOWN: CRITICAL - Host Unreachable (172.16.4.124) [16:57:18] 10MediaWiki-Releasing, 10Security: Release MediaWiki 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248534 (10Reedy) [16:57:26] 10MediaWiki-Releasing, 10Security: Write and send release announcements for MediaWiki 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248537 (10Reedy) [16:59:41] 10MediaWiki-Releasing, 10Security: Obtain CVEs for 1.31.8/1.33.4/1.34.2 security releases - https://phabricator.wikimedia.org/T248541 (10Reedy) [16:59:48] 10MediaWiki-Releasing, 10Security: Tag 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248540 (10Reedy) [16:59:55] 10MediaWiki-Releasing, 10Security: Tag 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248540 (10Reedy) a:03Reedy [17:00:04] 10MediaWiki-Releasing, 10Security: Update HISTORY in master after 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248539 (10Reedy) [17:00:10] 10MediaWiki-Releasing, 10Security: Update HISTORY in master after 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248539 (10Reedy) a:03Reedy [17:00:17] 10MediaWiki-Releasing, 10Security: Update onwiki release notes for 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248538 (10Reedy) [17:00:47] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10VisualEditor, 10Editing-team (Tracking): Firefox CI tests keep failing in VE - https://phabricator.wikimedia.org/T240955 (10Jdforrester-WMF) a:05Jdforrester-WMF→03None Back-burner-ing. [17:00:53] 10MediaWiki-Releasing, 10Security: Tracking bug for MediaWiki 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248535 (10Reedy) [17:01:24] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10MediaWiki-General, 10MW-1.33-release: Formally EOL REL1_33 - https://phabricator.wikimedia.org/T256087 (10Reedy) https://lists.wikimedia.org/pipermail/mediawik... [17:02:38] 10Release-Engineering-Team, 10Security-Team: Write a script to create security release task tree - https://phabricator.wikimedia.org/T256285 (10Reedy) [17:02:56] 10MediaWiki-Releasing, 10Release-Engineering-Team, 10Security-Team: Write a script to create security release task tree - https://phabricator.wikimedia.org/T256285 (10Reedy) [17:05:27] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10MediaWiki-General, 10MW-1.33-release: Formally EOL REL1_33 - https://phabricator.wikimedia.org/T256087 (10Jdforrester-WMF) a:03Jdforrester-WMF I'll do the ne... [17:07:09] 10Phabricator, 10Patch-For-Review: Weekly phabricator-reports mail: List project tag/color combinations which violate the "Types of Projects" definition - https://phabricator.wikimedia.org/T249806 (10Aklapper) 05Open→03Resolved Merged. [17:17:52] 10MediaWiki-Releasing, 10Security: Update onwiki release notes for 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248538 (10Reedy) 05Open→03Resolved a:03Reedy [17:17:54] 10MediaWiki-Releasing, 10Security: Release MediaWiki 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248534 (10Reedy) [17:18:07] 10MediaWiki-Releasing, 10Security: Tag 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248540 (10Reedy) 05Open→03Resolved [17:18:09] 10MediaWiki-Releasing, 10Security: Release MediaWiki 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248534 (10Reedy) [17:18:17] 10MediaWiki-Releasing, 10Security: Tracking bug for MediaWiki 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248535 (10Reedy) 05Open→03Resolved a:03Reedy [17:18:19] 10MediaWiki-Releasing, 10Security: Release MediaWiki 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248534 (10Reedy) [17:20:43] 10MediaWiki-Releasing, 10Security: Release MediaWiki 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248534 (10Reedy) [17:20:46] 10MediaWiki-Releasing, 10Patch-For-Review, 10Security: Update HISTORY in master after 1.31.8/1.33.4/1.34.2 - https://phabricator.wikimedia.org/T248539 (10Reedy) 05Open→03Resolved [17:23:12] 10Release-Engineering-Team-TODO, 10Quality-and-Test-Engineering-Team (QTE), 10User-Vidhi-Mody, 10User-zeljkofilipin, 10good first task: Create MediaWiki-Docker documentation for all repositories with Selenium tests - https://phabricator.wikimedia.org/T256239 (10zeljkofilipin) [17:25:13] 10MediaWiki-Releasing, 10Release-Engineering-Team, 10Security-Team: Write a script to create security release task tree - https://phabricator.wikimedia.org/T256285 (10Reedy) Seems to be https://github.com/wikimedia/mediawiki-tools-release/blob/76df0dfc51a197621476d666fa732cc9c2d1e785/scap/plugins/task-series... [17:25:50] 10Release-Engineering-Team-TODO, 10Quality-and-Test-Engineering-Team (QTE), 10User-Vidhi-Mody, 10User-zeljkofilipin, 10good first task: Create MediaWiki-Docker documentation for all repositories with Selenium tests - https://phabricator.wikimedia.org/T256239 (10zeljkofilipin) [17:26:12] 10Phabricator, 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Developer Productivity: Enhance phabricator project progress reporting - https://phabricator.wikimedia.org/T255703 (10mmodell) [17:26:26] 10Release-Engineering-Team-TODO, 10Quality-and-Test-Engineering-Team (QTE), 10User-Vidhi-Mody, 10User-zeljkofilipin, 10good first task: Create MediaWiki-Docker documentation for all repositories with Selenium tests - https://phabricator.wikimedia.org/T256239 (10zeljkofilipin) [17:29:31] 10Release-Engineering-Team-TODO, 10Quality-and-Test-Engineering-Team (QTE), 10User-Vidhi-Mody, 10User-zeljkofilipin, 10good first task: Create MediaWiki-Docker documentation for all repositories with Selenium tests - https://phabricator.wikimedia.org/T256239 (10zeljkofilipin) [17:29:57] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10MediaWiki-General, 10MW-1.33-release: Formally EOL REL1_33 - https://phabricator.wikimedia.org/T256087 (10Jdforrester-WMF) p:05Triage→03High [17:31:19] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10Scap, 10Operations, and 3 others: scap's logstash_checker.py is blissfully unaware of any logstash indexing latency - https://phabricator.wikimedia.org/T255197 (10thcipriani) [17:39:25] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Scap, 10Developer Productivity, and 2 others: Create a development & testing environment for scap + train deployment... - https://phabricator.wikimedia.org/T255695 [17:45:51] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 0.99 ms [17:55:01] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Release Pipeline: Create WMF Cassandra image - https://phabricator.wikimedia.org/T256281 (10thcipriani) [17:55:51] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [17:57:22] a lot shinken alerts again [18:03:07] 10Deployments, 10Gerrit, 10Phabricator, 10Release-Engineering-Team (Development services), and 2 others: Indicate deployed status (wmf branch) for Gerrit patches in Phabricator - https://phabricator.wikimedia.org/T88136 (10mmodell) [18:03:50] 10Phabricator Antivandalism Extension, 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Patch-For-Review, 10User-brennen: PHAVA: Create a CLI and Test suite for phabricator antivandalism - https://phabricator.wikimedia.org/T249741 (10mmodell) [18:04:14] (03PS1) 10Mstyles: jjb: remove archiva profile [integration/config] - 10https://gerrit.wikimedia.org/r/607580 [18:04:39] 10Release-Engineering-Team-TODO (Release-Engineering-Team-TODO (2020-07-01 to 2020-09-30 (Q1))), 10Phatality: Phatality can't report bugs from very long messages - https://phabricator.wikimedia.org/T255671 (10mmodell) [18:05:38] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Scap: Rename `scap swat` to `scap backport` - https://phabricator.wikimedia.org/T254787 (10mmodell) I think the scap swat plugin could be simply deleted. It's unused and incomplete. [18:09:05] 10Deployments, 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)): Script dropping of old staging branches for train - https://phabricator.wikimedia.org/T247076 (10mmodell) Maybe duplicate of {T250095} [18:36:55] 10Beta-Cluster-Infrastructure: Email spam from Betawiki - https://phabricator.wikimedia.org/T256297 (10XanonymusX) [18:38:14] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library: Email spam from Betawiki - https://phabricator.wikimedia.org/T256297 (10Majavah) [18:46:19] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Release, 10Train Deployments: 1.35.0-wmf.39 deployment blockers - https://phabricator.wikimedia.org/T254176 (10Nikerabbit) [18:57:36] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library: Email spam from Betawiki - https://phabricator.wikimedia.org/T256297 (10Samwalton9) This seems like a bug relating to T132084 - @Catrope authored the code for that. [19:02:21] 10Continuous-Integration-Infrastructure, 10Upstream: [Development tooling] grunt stylelint gives false positives inside comments, attributed to the wrong line number (tracking upstream reports) - https://phabricator.wikimedia.org/T245957 (10Demian) a:05Demian→03None Not working on this atm. [19:11:44] 10Continuous-Integration-Config: stylelint reports invalid line numbers - https://phabricator.wikimedia.org/T243116 (10Demian) [19:18:09] 10Continuous-Integration-Config: stylelint reports invalid line numbers - https://phabricator.wikimedia.org/T243116 (10Demian) >>! In T243116#6102635, @hashar wrote: > A bug got filled later on with a bit more details and references to #upstream issues: T245957 That looks very similar? For the record: https://... [19:22:31] 10Continuous-Integration-Infrastructure, 10Upstream: [Development tooling] grunt stylelint gives false positives inside comments, attributed to the wrong line number (tracking upstream reports) - https://phabricator.wikimedia.org/T245957 (10Demian) [19:23:29] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.38 deployment blockers - https://phabricator.wikimedia.org/T254175 (10brennen) [19:23:54] 10Continuous-Integration-Config, 10Upstream: stylelint reports invalid line numbers - https://phabricator.wikimedia.org/T243116 (10Demian) [19:25:06] 10Continuous-Integration-Infrastructure, 10Upstream: [Development tooling] grunt stylelint gives false positives inside comments, attributed to the wrong line number (tracking upstream reports) - https://phabricator.wikimedia.org/T245957 (10Demian) [20:05:19] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library: Email spam from Betawiki - https://phabricator.wikimedia.org/T256297 (10XanonymusX) Ah, I forgot to mention: I have never seen an echo notification about this. [20:05:56] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library: Email spam from Betawiki - https://phabricator.wikimedia.org/T256297 (10Samwalton9) Could you paste the email text, please? [20:15:50] 10phan, 10MediaWiki-Vagrant, 10Operations: It should be possible to install php-ast using apt-get on MediaWiki-Vagrant - https://phabricator.wikimedia.org/T234240 (10Lokal_Profil) >>! In T234240#6230719, @Lokal_Profil wrote: > @Mainframe98 Many thanks ! Now as a shell script in case that might come in handy... [20:26:10] 10Continuous-Integration-Config, 10Wikispeech-Jobrunner, 10Wikispeech-Text-to-Speech, 10Wikispeech-WMSE: Coverage report disappeared from doc.wikimedia.org/cover-extensions/Wikispeech/ - https://phabricator.wikimedia.org/T256197 (10Lokal_Profil) 05Open→03Declined Looks like it was just cached at an unf... [20:26:19] 10Continuous-Integration-Config, 10Wikispeech-Jobrunner, 10Wikispeech-Text-to-Speech, 10Wikispeech-WMSE: Coverage report disappeared from doc.wikimedia.org/cover-extensions/Wikispeech/ - https://phabricator.wikimedia.org/T256197 (10Lokal_Profil) 05Declined→03Invalid [20:35:51] RECOVERY - Host integration-cumin-01 is UP: PING OK - Packet loss = 0%, RTA = 1.38 ms [20:40:51] PROBLEM - Host integration-cumin-01 is DOWN: CRITICAL - Host Unreachable (172.16.1.185) [20:52:03] 10Release-Engineering-Team-TODO (2020-04 to 2020-06 (Q4)), 10Patch-For-Review, 10Release, 10Train Deployments, 10User-brennen: 1.35.0-wmf.38 deployment blockers - https://phabricator.wikimedia.org/T254175 (10brennen) [20:52:15] 10Project-Admins: Create User-Demian project - https://phabricator.wikimedia.org/T256320 (10Demian) [21:20:24] 10Beta-Cluster-Infrastructure, 10The-Wikipedia-Library: Email spam from Betawiki - https://phabricator.wikimedia.org/T256297 (10XanonymusX) Since I posted this, I got three more emails, one in German (as usual) and two in English; probably because I visited (no edit!) some other projects from the Beta cluster.... [21:31:57] 10Release-Engineering-Team-TODO, 10User-brennen: Onboard Ahmon - https://phabricator.wikimedia.org/T256074 (10brennen) [22:34:09] 10Project-Admins: Create User-Demian project - https://phabricator.wikimedia.org/T256320 (10MGChecker) a:03MGChecker [22:36:34] 10Project-Admins: Create User-Demian project - https://phabricator.wikimedia.org/T256320 (10MGChecker) 05Open→03Resolved Requested public project #User-Demian has been created: https://phabricator.wikimedia.org/project/view/4852/ Please encourage interested people to visit the project and to join the projec... [22:36:37] 10Phabricator (Upstream), 10Upstream: Per-user projects for personal work in progress tracking - https://phabricator.wikimedia.org/T555 (10MGChecker) [22:46:28] 10Project-Admins: Create User-Demian project - https://phabricator.wikimedia.org/T256320 (10Demian) Thank you very much for the swift action @MGChecker! [22:49:17] 10Project-Admins: Create User-Demian project - https://phabricator.wikimedia.org/T256320 (10MGChecker) [22:54:05] PROBLEM - jenkins_service_running on releases1002 is CRITICAL: PROCS CRITICAL: 2 processes with regex args .*/bin/java .*-jar /usr/share/jenkins/jenkins.war https://wikitech.wikimedia.org/wiki/Jenkins [22:54:28] ^ me, not in prod, just got added [22:55:20] ACKNOWLEDGEMENT - jenkins_service_running on releases1002 is CRITICAL: PROCS CRITICAL: 2 processes with regex args .*/bin/java .*-jar /usr/share/jenkins/jenkins.war daniel_zahn new install https://wikitech.wikimedia.org/wiki/Jenkins [23:41:46] 10Continuous-Integration-Infrastructure, 10Operations, 10serviceops, 10Patch-For-Review: replace backends for releases.wikimedia.org with buster VMs - https://phabricator.wikimedia.org/T247652 (10Dzahn) @hashar For some reason on releases1002/2002 (new VMs on buster), after applying the releases role, one... [23:42:47] 10MediaWiki-Releasing, 10Release-Engineering-Team, 10Security-Team: Write a script to create security release task tree - https://phabricator.wikimedia.org/T256285 (10Reedy) [23:42:55] RECOVERY - jenkins_service_running on releases1002 is OK: PROCS OK: 1 process with regex args .*/bin/java .*-jar /usr/share/jenkins/jenkins.war https://wikitech.wikimedia.org/wiki/Jenkins [23:44:22] 10MediaWiki-Releasing, 10Release-Engineering-Team, 10Security-Team: Write a script to create security release task tree - https://phabricator.wikimedia.org/T256285 (10Reedy) [23:48:11] https://integration.wikimedia.org/ci/view/Beta/job/beta-scap-eqiad/ seems to be having trouble [23:54:57] (03PS1) 10Reedy: Add script for creating MW Security Release Phab tasks [tools/release] - 10https://gerrit.wikimedia.org/r/607643 (https://phabricator.wikimedia.org/T256285)