[07:19:14] (03CR) 10Legoktm: [C: 03+1] "I don't remember why I didn't do this at the time, but I think it's probably safe. The chance of arbitrary code execution from Mozilla ser" [integration/config] - 10https://gerrit.wikimedia.org/r/539724 (owner: 10MarcoAurelio) [07:20:49] 10Project-Admins, 10User-DannyS712: Create WikiJournals tag - https://phabricator.wikimedia.org/T234178 (10DannyS712) [07:23:58] 10Project-Admins, 10User-DannyS712: Create WikiJournals tag - https://phabricator.wikimedia.org/T234178 (10Aklapper) Hi, can you please provide a project name, project description (with links to more information) and who should be a member of this project? Also see https://www.mediawiki.org/wiki/Phabricator/Cr... [07:26:17] 10Project-Admins, 10User-DannyS712: Create WikiJournals tag - https://phabricator.wikimedia.org/T234178 (10DannyS712) Project name: WikiJournals (like #wikisource #wiktionary) Who should be a member: @DannyS712 (only partially associated with the project, but interested and willing to help), @Thomas_Shafee (or... [07:40:09] (03CR) 10Awight: [C: 03+2] Apply Chronometer when executing the build plan [integration/quibble] - 10https://gerrit.wikimedia.org/r/539539 (owner: 10Hashar) [07:40:55] (03Merged) 10jenkins-bot: Apply Chronometer when executing the build plan [integration/quibble] - 10https://gerrit.wikimedia.org/r/539539 (owner: 10Hashar) [07:43:57] 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Documentation, 10User-MModell: Document subtypes / task types in Phabricator - https://phabricator.wikimedia.org/T224417 (10Aklapper) 05Open→03Resolved Thanks @mmodell, this is helpful! https://www.me... [07:44:25] 10Project-Admins, 10User-DannyS712: Create WikiJournals tag - https://phabricator.wikimedia.org/T234178 (10Thomas_Shafee) Other interested members on phabricator may include: @Bryandamon , @jeblad , @Daniel_Mietchen, @Faendalimas. If logical, may also be worth pining those involved in wikicite, scholia, & orc... [08:00:04] Project mwcore-phpunit-coverage-master build #205: 04FAILURE in 5 hr 0 min: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/205/ [08:39:02] RECOVERY - Puppet staleness on saucelabs-02 is OK: OK: Less than 1.00% above the threshold [3600.0] [09:36:13] PROBLEM - English Wikipedia Main page on beta-cluster is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2829 bytes in 3.056 second response time [09:36:13] PROBLEM - Apertium APY on deployment-sca01 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:36:29] Project beta-scap-eqiad build #269194: 04FAILURE in 2 min 2 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/269194/ [09:36:33] PROBLEM - App Server Main HTTP Response on deployment-mediawiki-09 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2259 bytes in 6.081 second response time [09:37:50] PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2829 bytes in 3.057 second response time [09:38:13] PROBLEM - App Server Main HTTP Response on deployment-mediawiki-07 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2259 bytes in 3.100 second response time [09:38:20] PROBLEM - App Server Main HTTP Response on deployment-mediawiki-parsoid10 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 Internal Server Error - 2413 bytes in 3.083 second response time [09:46:24] Project beta-scap-eqiad build #269195: 04STILL FAILING in 1 min 58 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/269195/ [09:56:22] Project beta-scap-eqiad build #269196: 04STILL FAILING in 1 min 56 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/269196/ [10:06:36] Project beta-scap-eqiad build #269197: 04STILL FAILING in 2 min 10 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/269197/ [10:16:25] Project beta-scap-eqiad build #269198: 04STILL FAILING in 2 min 0 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/269198/ [10:21:02] RECOVERY - Apertium APY on deployment-sca01 is OK: HTTP OK: HTTP/1.1 200 OK - 5996 bytes in 0.003 second response time [10:26:15] RECOVERY - English Wikipedia Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 48631 bytes in 4.024 second response time [10:26:22] Yippee, build fixed! [10:26:23] Project beta-scap-eqiad build #269199: 09FIXED in 1 min 57 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/269199/ [10:26:26] RECOVERY - App Server Main HTTP Response on deployment-mediawiki-09 is OK: HTTP OK: HTTP/1.1 200 OK - 48092 bytes in 0.498 second response time [10:27:48] RECOVERY - English Wikipedia Mobile Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 37155 bytes in 0.840 second response time [10:28:11] RECOVERY - App Server Main HTTP Response on deployment-mediawiki-07 is OK: HTTP OK: HTTP/1.1 200 OK - 48014 bytes in 0.697 second response time [10:28:19] RECOVERY - App Server Main HTTP Response on deployment-mediawiki-parsoid10 is OK: HTTP OK: HTTP/1.1 200 OK - 48086 bytes in 1.575 second response time [10:32:52] (03PS2) 10Hashar: Dedicated Selenium job for gated repos [integration/config] - 10https://gerrit.wikimedia.org/r/539630 (https://phabricator.wikimedia.org/T232759) [10:32:56] (03CR) 10jerkins-bot: [V: 04-1] Dedicated Selenium job for gated repos [integration/config] - 10https://gerrit.wikimedia.org/r/539630 (https://phabricator.wikimedia.org/T232759) (owner: 10Hashar) [10:34:33] (03CR) 10Hashar: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/539630 (https://phabricator.wikimedia.org/T232759) (owner: 10Hashar) [10:35:27] (03CR) 10jerkins-bot: [V: 04-1] Dedicated Selenium job for gated repos [integration/config] - 10https://gerrit.wikimedia.org/r/539630 (https://phabricator.wikimedia.org/T232759) (owner: 10Hashar) [10:36:18] hashar: https://integration.wikimedia.org/ci/job/wmf-quibble-core-vendor-mysql-php72-docker/6831/console are you seeing this on other jobs? Could not resolve host... [10:36:25] Maybe I should write in -operations [10:37:26] (03CR) 10Hashar: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/539630 (https://phabricator.wikimedia.org/T232759) (owner: 10Hashar) [10:38:17] (03CR) 10jerkins-bot: [V: 04-1] Dedicated Selenium job for gated repos [integration/config] - 10https://gerrit.wikimedia.org/r/539630 (https://phabricator.wikimedia.org/T232759) (owner: 10Hashar) [10:42:27] kostajh: yeah it is mysteriously broken due to some dns issue [10:46:33] hashar: are you able to reproduce the issue on the commandline on any particular worker? [10:46:35] (I'm not) [10:47:15] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10cloud-services-team: CI docker container fails to resolve DNS name: fatal: Unable to look up contint1001.wikimedia.org (port 94... - https://phabricator.wikimedia.org/T234197 [10:47:24] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10cloud-services-team: CI docker container fails to resolve DNS name: fatal: Unable to look up contint1001.wikimedia.org (port 94... - https://phabricator.wikimedia.org/T234197 [10:47:42] andrewbogott: yes from inside a container [10:47:42] https://phabricator.wikimedia.org/T234197 [10:48:44] hm, if it works from the worker but not from in the container that /probably/ isn't related to wmcs infra? But I'll poke a bit [10:50:01] andrewbogott: given it worked until whatever magic happened on the infra, that is 100% related ;D [10:50:10] but the docker containers my use a slightly different path [10:50:13] ;D [10:50:40] andrewbogott: any hint as to what has changed earlier? [10:50:49] just firewall changes [10:51:04] is there a way for me to get a shell in that docker container? [10:51:23] what I pasted on T234197 [10:51:24] T234197: CI docker container fails to resolve DNS name: fatal: Unable to look up contint1001.wikimedia.org (port 9418) (Temporary failure in name resolution) - https://phabricator.wikimedia.org/T234197 [10:51:29] OH [10:51:29] what you pasted doesn't work [10:51:33] wrongly pasted bah [10:51:52] sudo docker run --rm -it --entrypoint=bash docker-registry.wikimedia.org/releng/ci-src-setup-simple:0.2.1 [10:51:55] sorry :-\\ [10:52:01] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10cloud-services-team: CI docker container fails to resolve DNS name: fatal: Unable to look up contint1001.wikimedia.org (port 94... - https://phabricator.wikimedia.org/T234197 [10:52:21] though it works from the instance for some reason bah [10:52:34] ok, got it now [10:52:56] of course, dig not present in the container :/ [10:53:05] ah yeah hmm [10:53:11] nor ping [10:53:18] should run it as root I guess [10:53:31] sudo docker run --rm -it --entrypoint=bash --user=root docker-registry.wikimedia.org/releng/ci-src-setup-simple:0.2.1 [10:53:34] then apt-get update [10:54:51] yeah well it can not resolve anything anyway [10:55:11] is that command already stopping/restarting the container entirely? [10:55:27] it starts a fresh one yes [10:56:22] ok [10:56:29] I'd be interested in what happens if the host is rebooted [10:56:42] I would guess the ferm rule changed some iptables [10:56:49] and somehow the docker process still has the old rules [10:56:51] but I don't know enough about docker to understand why it would be failing when clearly the dns server is working and the host is working [10:56:52] if that is all possible [10:57:20] that's why I was wondering about w/not we're restarting the container [10:57:32] but is ferm really running /inside/ the container? [10:57:52] just a note - docker can run multiple networks [10:58:09] maybe those containers are running in some different network that has no access to DNS/internet [10:58:57] * hashar restarts docker [10:59:20] andrewbogott: so yeah restarting docker solved it :-\ [10:59:29] that's easy! [10:59:39] !log Restarted Docker on integration-agent-docker-1001 T234197 [10:59:41] what a mess :-\ [10:59:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:59:42] T234197: CI docker container fails to resolve DNS name: fatal: Unable to look up contint1001.wikimedia.org (port 9418) (Temporary failure in name resolution) - https://phabricator.wikimedia.org/T234197 [10:59:59] * andrewbogott -> back to bed [11:00:04] andrewbogott: thx ;) [11:04:07] hashar, still broken :( [11:04:11] https://integration.wikimedia.org/ci/job/operations-mw-config-hhvm-composer-lint-docker/615/console [11:08:03] !log Restarting Docker on CI agents to clear out some docker/iptables oddity # T234197 [11:08:08] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:08:08] T234197: CI docker container fails to resolve DNS name: fatal: Unable to look up contint1001.wikimedia.org (port 9418) (Temporary failure in name resolution) - https://phabricator.wikimedia.org/T234197 [11:08:23] Project mediawiki-core-doxygen-docker build #10245: 04FAILURE in 4 min 21 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10245/ [11:14:36] (03CR) 10Daniel Kinzler: Force value of $wgSecretKey. (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/537118 (https://phabricator.wikimedia.org/T233092) (owner: 10Daniel Kinzler) [11:15:11] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10cloud-services-team: CI docker container fails to resolve DNS name: fatal: Unable to look up contint1001.wikimedia.org (port 94... - https://phabricator.wikimedia.org/T234197 [11:28:19] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909): cumin broken on integration project - https://phabricator.wikimedia.org/T234203 (10hashar) [11:35:05] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909): cumin broken on integration project - https://phabricator.wikimedia.org/T234203 (10hashar) [11:37:39] RECOVERY - Puppet staleness on integration-agent-puppet-docker-1001 is OK: OK: Less than 1.00% above the threshold [3600.0] [11:55:43] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909): [Cloud VPS alert] Puppet failure on (saucelabs-02|integration-agent-puppet-docker-1001).integration.eqiad.wmflabs - https://phabricator.wikimedia.org/T234206 (10zeljkofilipin) [11:55:55] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909): [Cloud VPS alert] Puppet failure on (saucelabs-02|integration-agent-puppet-docker-1001).integration.eqiad.wmflabs - https://phabricator.wikimedia.org/T234206 (10zeljkofilipin) p:05Triage→03High [12:10:43] Yippee, build fixed! [12:10:43] Project mediawiki-core-doxygen-docker build #10246: 09FIXED in 6 min 41 sec: https://integration.wikimedia.org/ci/job/mediawiki-core-doxygen-docker/10246/ [12:45:51] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909): cumin broken on integration project - https://phabricator.wikimedia.org/T234203 (10hashar) 05Open→03Resolved On https://horizon.wikimedia.org/project/puppet/ : ` lang=yam... [12:49:49] !log Fixed cumin on integration project # T234203 [12:49:54] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:49:55] T234203: cumin broken on integration project - https://phabricator.wikimedia.org/T234203 [12:51:12] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO (201909), 10local-charts, 10User-zeljkofilipin: Error: error installing: the server could not find the requested resource - https://phabricator.wikimedia.org/T233960 (10zeljkofilipin) Thanks, that worked! 😁 But something is still broke... [13:01:00] (03CR) 10Hashar: "recheck T234197" [integration/config] - 10https://gerrit.wikimedia.org/r/539630 (https://phabricator.wikimedia.org/T232759) (owner: 10Hashar) [13:03:48] (03CR) 10Hashar: "Looking at https://gerrit.wikimedia.org/r/#/q/owner:%22SLF+bot%22 , I would advise against syncing the list of domains from Mozilla. That " [integration/config] - 10https://gerrit.wikimedia.org/r/539724 (owner: 10MarcoAurelio) [13:10:08] RECOVERY - Puppet staleness on integration-agent-docker-1006 is OK: OK: Less than 1.00% above the threshold [3600.0] [13:10:24] !log integration-castor03 : sudo mkdir -p /usr/local/lib/python2.7/dist-packages [13:10:26] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:36:18] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909): [Cloud VPS alert] Puppet failure on (saucelabs-02|integration-agent-puppet-docker-1001).integration.eqiad.wmflabs - https://phabricator.wikimedia.org/T234206 (10hashar) 05Op... [14:05:08] 10Phabricator: Herald rule: KaiOS -> Inuka - https://phabricator.wikimedia.org/T234217 (10SBisson) [14:07:52] 10Release-Engineering-Team (Code Health), 10Release-Engineering-Team-TODO (201908), 10Code-Stewardship-Reviews, 10Graphoid, and 3 others: graphoid: Code stewardship request - https://phabricator.wikimedia.org/T211881 (10Milimetric) @dr0ptp4kt and how will this work? The graphs will start out as some place... [14:26:38] (03CR) 10Krinkle: Dedicated Selenium job for gated repos (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/539630 (https://phabricator.wikimedia.org/T232759) (owner: 10Hashar) [14:39:40] 10Release-Engineering-Team, 10Wikimedia-Logstash: Single row view in Logstash hidden by Phatality - https://phabricator.wikimedia.org/T234223 (10Krinkle) [14:47:21] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201909), 10User-zeljkofilipin: Update existing Selenium documentation - https://phabricator.wikimedia.org/T232598 (10zeljkofilipin) [14:59:56] Did someone delete the shinken repo? [15:00:23] * paladox sees com.google.gerrit.server.permissions.PermissionBackendException: project 'operations/debs/shinken' is unavailable [15:19:37] (03CR) 10Jforrester: [C: 03+2] Whitelist TheSandDoctor [integration/config] - 10https://gerrit.wikimedia.org/r/539720 (owner: 10Urbanecm) [15:21:40] (03Merged) 10jenkins-bot: Whitelist TheSandDoctor [integration/config] - 10https://gerrit.wikimedia.org/r/539720 (owner: 10Urbanecm) [15:23:35] !log Zuul: Added TheSandDoctor to the CI whitelist. [15:23:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:30:03] (03PS1) 10WMDE-leszek: Also publish JS docs for Wikibase [integration/config] - 10https://gerrit.wikimedia.org/r/539887 [15:35:05] (03CR) 10Jforrester: "LGTM." [integration/config] - 10https://gerrit.wikimedia.org/r/539887 (owner: 10WMDE-leszek) [15:36:44] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO, 10Developer Productivity, 10Release Pipeline, and 2 others: Define a .pipeline/blubber.yaml for mediawiki/core - https://phabricator.wikimedia.org/T218360 (10Jdforrester-WMF) https://integration.wikimedia.org/ci/job/trigger-mediawiki-... [15:51:00] 10Release-Engineering-Team, 10Wikimedia-Logstash: Single row view in Logstash hidden by Phatality - https://phabricator.wikimedia.org/T234223 (10fgiunchedi) cc @mmodell [15:52:42] 10Release-Engineering-Team, 10Wikimedia-Logstash: Single row view in Logstash hidden by Phatality - https://phabricator.wikimedia.org/T234223 (10mmodell) a:03mmodell [15:53:06] 10Release-Engineering-Team, 10Wikimedia-Logstash: Single row view in Logstash hidden by Phatality - https://phabricator.wikimedia.org/T234223 (10mmodell) Hopefully this is an easy fix. [15:54:04] (03CR) 10WMDE-leszek: "this change was initially created as a prep for publishing some more JS docs of Wikibase (e.g. T227285, I25392a74225ff3fd3c49766574f61804f" [integration/config] - 10https://gerrit.wikimedia.org/r/539887 (owner: 10WMDE-leszek) [15:55:10] (03CR) 10Jforrester: "> Patch Set 1:" [integration/config] - 10https://gerrit.wikimedia.org/r/539887 (owner: 10WMDE-leszek) [15:55:15] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10JavaScript: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10Mholloway) [15:55:18] (03Abandoned) 10Jforrester: Also publish JS docs for Wikibase [integration/config] - 10https://gerrit.wikimedia.org/r/539887 (owner: 10WMDE-leszek) [15:56:57] (03CR) 10WMDE-leszek: "thanks, i missed that other template!" [integration/config] - 10https://gerrit.wikimedia.org/r/539887 (owner: 10WMDE-leszek) [15:58:39] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO, 10Developer Productivity, 10Release Pipeline, and 2 others: Define a .pipeline/blubber.yaml for mediawiki/core - https://phabricator.wikimedia.org/T218360 (10brennen) 05Open→03Resolved > Shall we declare this Resolved? There's so... [15:58:43] 10Gerrit: Gerrit workflow: "Merge review" AND "Open push" for BlueSpice? - https://phabricator.wikimedia.org/T234224 (10Mainframe98) [16:00:14] 10Phabricator: Reset 2FA for Phabricator account `Apap04` - https://phabricator.wikimedia.org/T234233 (10jrbs) [16:02:45] 10Gerrit: Gerrit workflow: "Merge review" AND "Open push" for BlueSpice? - https://phabricator.wikimedia.org/T234224 (10Paladox) Does it give any clearer error? I would think it wants you to add "Forge Author Identity" to the project's ACL. [16:03:00] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10JavaScript: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10Mholloway) [16:10:57] (03PS1) 1020after4: Move phatality'a tab to the end in the list of doc view tabs. [releng/phatality] - 10https://gerrit.wikimedia.org/r/539898 (https://phabricator.wikimedia.org/T234223) [16:13:17] 10Release-Engineering-Team, 10Wikimedia-Logstash, 10Patch-For-Review: Single row view in Logstash hidden by Phatality - https://phabricator.wikimedia.org/T234223 (10mmodell) p:05Triage→03High [16:15:09] 10Release-Engineering-Team (Local Dev), 10Release-Engineering-Team-TODO (201909), 10Developer Productivity, 10Release Pipeline, and 2 others: Define a .pipeline/blubber.yaml for mediawiki/core - https://phabricator.wikimedia.org/T218360 (10Jdforrester-WMF) [16:20:00] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10Operations, 10Release Pipeline, and 4 others: Revisit the logging work done on Q1 2017-2018 for the standard pod setup - https://phabricator.wikimedia.org/T207200 (10akosiaris) [16:25:31] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (201909), 10phan: ci-src-setup job (used by mediawiki-core-php72-phan-docker) is still running on PHP 7.0.33 - https://phabricator.wikimedia.org/T234062 (10Jdforrester-WMF) [16:27:10] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (201909), 10phan: ci-src-setup job (used by mediawiki-core-php72-phan-docker) is still running on PHP 7.0.33 - https://phabricator.wikimedia.org/T234062 (10Jdforrester-WMF) Eurgh, it's also still running on jessie(!). [16:27:23] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10Operations, 10Release Pipeline, and 4 others: Revisit the logging work done on Q1 2017-2018 for the standard pod setup - https://phabricator.wikimedia.org/T207200 (10akosiaris) Logs are now making it to logstash so I am gonna boldly... [16:35:07] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201909), 10User-zeljkofilipin: Update existing Selenium documentation - https://phabricator.wikimedia.org/T232598 (10zeljkofilipin) [16:35:44] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201909), 10User-zeljkofilipin: Update existing Selenium documentation - https://phabricator.wikimedia.org/T232598 (10zeljkofilipin) [16:41:08] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (201909), 10phan: ci-src-setup job (used by mediawiki-core-php72-phan-docker) is still running on PHP 7.0.33 - https://phabricator.wikimedia.org/T234062 (10Jdforrester-WMF) Hmm, `docker-ci-src-setup-mw` is the only user? Maybe we could make it... [16:42:58] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10Operations, 10Release Pipeline, and 3 others: Migrate production services to kubernetes using the pipeline - https://phabricator.wikimedia.org/T198901 (10akosiaris) [16:43:01] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10Operations, 10Release Pipeline, and 4 others: Revisit the logging work done on Q1 2017-2018 for the standard pod setup - https://phabricator.wikimedia.org/T207200 (10akosiaris) 05Open→03Resolved [16:53:58] 10Release-Engineering-Team, 10Wikimedia-Logstash, 10Patch-For-Review: Single row view in Logstash hidden by Phatality - https://phabricator.wikimedia.org/T234223 (10Krinkle) (sorry, pinged wrong account) This seems to affect the main view as well. There, things are not hidden, but it causes the `(+)` plus a... [16:54:05] (03CR) 10Krinkle: [C: 03+1] Move phatality'a tab to the end in the list of doc view tabs. [releng/phatality] - 10https://gerrit.wikimedia.org/r/539898 (https://phabricator.wikimedia.org/T234223) (owner: 1020after4) [16:55:22] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201909), 10User-zeljkofilipin: Update existing Selenium documentation - https://phabricator.wikimedia.org/T232598 (10zeljkofilipin) [16:55:36] 10Release-Engineering-Team, 10Wikimedia-Logstash, 10Patch-For-Review: Single row view in Logstash hidden by Phatality - https://phabricator.wikimedia.org/T234223 (10mmodell) @krinkle: The phatality view doesn't include the details, you have to click to the table view to see the + / - buttons. But my patch (a... [16:55:48] (03CR) 1020after4: [V: 03+2 C: 03+2] Move phatality'a tab to the end in the list of doc view tabs. [releng/phatality] - 10https://gerrit.wikimedia.org/r/539898 (https://phabricator.wikimedia.org/T234223) (owner: 1020after4) [16:56:02] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201909), 10MediaWiki-Core-Testing, 10User-zeljkofilipin: Selenium tests should be easier to run - https://phabricator.wikimedia.org/T182691 (10zeljkofilipin) [16:56:04] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201909), 10User-zeljkofilipin: Update existing Selenium documentation - https://phabricator.wikimedia.org/T232598 (10zeljkofilipin) [16:56:10] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201909), 10User-zeljkofilipin: Update existing Selenium documentation - https://phabricator.wikimedia.org/T232598 (10zeljkofilipin) [16:56:12] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201909), 10MediaWiki-Core-Testing, 10Documentation, and 2 others: Blog posts about new Selenium framework features - https://phabricator.wikimedia.org/T191982 (10zeljkofilipin) [16:56:20] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201909), 10User-zeljkofilipin: Update existing Selenium documentation - https://phabricator.wikimedia.org/T232598 (10zeljkofilipin) [16:56:22] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201909), 10MediaWiki-Core-Testing, 10User-zeljkofilipin: Update files required by Selenium in core, extensions and skins - https://phabricator.wikimedia.org/T210726 (10zeljkofilipin) [16:56:29] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201909), 10MediaWiki-Core-Testing, 10Browser-Tests, 10User-zeljkofilipin: Usage instructions in tests/selenium/README.md are confusing - https://phabricator.wikimedia.org/T214708 (10zeljkofilipin) [16:56:31] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201909), 10User-zeljkofilipin: Update existing Selenium documentation - https://phabricator.wikimedia.org/T232598 (10zeljkofilipin) [16:57:07] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201909), 10MediaWiki-Core-Testing, 10User-zeljkofilipin: Update Selenium/Node.js/Target Custom MediaWiki - https://phabricator.wikimedia.org/T210518 (10zeljkofilipin) 05Open→03Resolved a:03zeljkofilipin [16:57:09] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201909), 10User-zeljkofilipin: Update existing Selenium documentation - https://phabricator.wikimedia.org/T232598 (10zeljkofilipin) [16:57:16] 10Release-Engineering-Team, 10Wikimedia-Logstash, 10Patch-For-Review: Single row view in Logstash hidden by Phatality - https://phabricator.wikimedia.org/T234223 (10mmodell) Ok not I just need to deploy it. [17:14:23] 10Phabricator, 10Release-Engineering-Team-TODO (201909), 10Wikimedia-Logstash, 10observability, 10User-MModell: Deploy phatality into kibana - https://phabricator.wikimedia.org/T230752 (10mmodell) So the latest changes to the sudo rules apparently don't work... [17:30:43] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201909), 10User-zeljkofilipin: Update existing Selenium documentation - https://phabricator.wikimedia.org/T232598 (10zeljkofilipin) [17:37:40] 10Release-Engineering-Team, 10Wikimedia-Logstash: Single row view in Logstash hidden by Phatality - https://phabricator.wikimedia.org/T234223 (10mmodell) 05Open→03Resolved fixed. [18:48:35] (03CR) 10Hashar: Dedicated Selenium job for gated repos (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/539630 (https://phabricator.wikimedia.org/T232759) (owner: 10Hashar) [18:49:23] (03CR) 10Jforrester: [C: 03+1] Dedicated Selenium job for gated repos [integration/config] - 10https://gerrit.wikimedia.org/r/539630 (https://phabricator.wikimedia.org/T232759) (owner: 10Hashar) [18:50:00] so yeah one step at a time [18:50:05] +1 [18:52:42] hashar: BTW, I guesstimate that we'll be able to drop HHVM from CI next week. [18:52:50] awesome [18:52:51] Which will be nice. [18:53:25] !log Create jobs wmf-quibble-selenium-hhvm-docker wmf-quibble-selenium-php72-docker | https://gerrit.wikimedia.org/r/#/c/integration/config/+/539630/ [18:53:27] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:54:38] (03CR) 10Hashar: [C: 03+2] "I have deployed:" [integration/config] - 10https://gerrit.wikimedia.org/r/539630 (https://phabricator.wikimedia.org/T232759) (owner: 10Hashar) [18:55:42] and I have to dig again into Quibble changes + release it again bah ;D [18:57:25] (03Merged) 10jenkins-bot: Dedicated Selenium job for gated repos [integration/config] - 10https://gerrit.wikimedia.org/r/539630 (https://phabricator.wikimedia.org/T232759) (owner: 10Hashar) [19:00:43] (03PS1) 10Umherirrender: [WikimediaMaintenance] Add phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/539938 (https://phabricator.wikimedia.org/T224770) [19:01:42] !log Reloading Zuul for https://gerrit.wikimedia.org/r/539630 [19:01:44] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:01:58] (03PS6) 10Umherirrender: Create new sniff for doc comments [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/528280 [19:02:48] Yippee, build fixed! [19:02:48] Project mwcore-phpunit-coverage-master build #206: 09FIXED in 4 hr 2 min: https://integration.wikimedia.org/ci/job/mwcore-phpunit-coverage-master/206/ [19:03:39] !log Update jobs wmf-quibble-core-vendor-mysql-hhvm-docker wmf-quibble-core-vendor-mysql-php72-docker wmf-quibble-vendor-mysql-hhvm-docker wmf-quibble-vendor-mysql-php72-docker | to stop running selenium tests | https://gerrit.wikimedia.org/r/#/c/integration/config/+/539630/ [19:03:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [19:05:47] 10Phabricator: Reset 2FA for Phabricator account `Apap04` - https://phabricator.wikimedia.org/T234233 (10Aklapper) Hmm, https://www.mediawiki.org/wiki/Phabricator/Help/Two-factor_Authentication_Resets ... offers a few options for verification but not sure if one of them works for you? [19:06:03] 00:03:45.968 [0-3] PASSED in chrome - /tests/selenium/specs/specialrecentchanges.js [19:06:03] 00:03:45.968 [0-4] RUNNING in chrome - /tests/selenium/specs/specialwatchlist.js [19:06:04] hmm [19:06:16] the wdio selenium tests seem to be running slightly differently now [19:06:18] and concurrently [19:10:04] and there is a bug which is that the mwcore-codehealth-master-non-voting jobs keep some executors busy :-\ [19:12:12] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909): Move mwcore-codehealth-master-non-voting to a dedicated Jenkins label / agent - https://phabricator.wikimedia.org/T234259 (10hashar) [19:14:12] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909): Move mwcore-codehealth-master-non-voting to a dedicated Jenkins label / agent - https://phabricator.wikimedia.org/T234259 (10hashar) The non concurrent jobs such as `mwcore-codehealt... [19:19:05] (03PS1) 10Hashar: Move mwext-codehealth jobs to a dedicated label [integration/config] - 10https://gerrit.wikimedia.org/r/539944 (https://phabricator.wikimedia.org/T234259) [19:22:44] (03CR) 10Hashar: [C: 03+2] "INFO:jenkins_jobs.builder:Reconfiguring jenkins job mwcore-codehealth-master-non-voting" [integration/config] - 10https://gerrit.wikimedia.org/r/539944 (https://phabricator.wikimedia.org/T234259) (owner: 10Hashar) [19:25:38] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10Patch-For-Review: Move mwcore-codehealth-master-non-voting to a dedicated Jenkins label / agent - https://phabricator.wikimedia.org/T234259 (10hashar) INFO:jenkins_jobs.builder:R... [19:25:53] (03Merged) 10jenkins-bot: Move mwext-codehealth jobs to a dedicated label [integration/config] - 10https://gerrit.wikimedia.org/r/539944 (https://phabricator.wikimedia.org/T234259) (owner: 10Hashar) [19:32:24] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909): Move CI selenium/qunit tests of mediawiki repository to a standalone job - https://phabricator.wikimedia.org/T232759 (10hashar) For the record, using cold caches: | wmf-quibble-sele... [19:32:52] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909): Move CI selenium/qunit tests of mediawiki repository to a standalone job - https://phabricator.wikimedia.org/T232759 (10hashar) p:05Triage→03High [19:56:20] 10Phabricator: Reset 2FA for Phabricator account `Apap04` - https://phabricator.wikimedia.org/T234233 (10Apap04) oh crap, I forgot to mention that I created the task via email so I have no access to the web interface and I'm not logged in. [20:07:40] 10Phabricator: Improve Phabricator error messages when uploading files - https://phabricator.wikimedia.org/T234261 (10Der_Keks) [20:07:47] 10Phabricator: Improve Phabricator error messages when uploading files - https://phabricator.wikimedia.org/T234261 (10Der_Keks) p:05Triage→03Normal [20:08:23] !sal [20:08:24] https://tools.wmflabs.org/sal/releng [20:10:12] 10Release-Engineering-Team, 10Release-Engineering-Team-TODO: Reassess idea of CI point person role - https://phabricator.wikimedia.org/T129295 (10greg) Just noting: There have been thoughts around a clinic duty type practice for RelEng, maybe EngProd, maybe even in conjunction with other teams. [20:43:27] (03CR) 10Jforrester: [C: 03+2] [WikimediaMaintenance] Add phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/539938 (https://phabricator.wikimedia.org/T224770) (owner: 10Umherirrender) [20:43:57] (03PS2) 10Jforrester: [WikibaseQualityConstraints] Run phan job [integration/config] - 10https://gerrit.wikimedia.org/r/539588 (https://phabricator.wikimedia.org/T224773) (owner: 10Umherirrender) [20:44:05] (03CR) 10Jforrester: [C: 03+2] [WikibaseQualityConstraints] Run phan job [integration/config] - 10https://gerrit.wikimedia.org/r/539588 (https://phabricator.wikimedia.org/T224773) (owner: 10Umherirrender) [20:45:22] (03Merged) 10jenkins-bot: [WikimediaMaintenance] Add phan dependency [integration/config] - 10https://gerrit.wikimedia.org/r/539938 (https://phabricator.wikimedia.org/T224770) (owner: 10Umherirrender) [20:45:53] (03Merged) 10jenkins-bot: [WikibaseQualityConstraints] Run phan job [integration/config] - 10https://gerrit.wikimedia.org/r/539588 (https://phabricator.wikimedia.org/T224773) (owner: 10Umherirrender) [20:47:08] !log Zuul: Add phan job for WikibaseQualityConstraints T224773 and some phan dependencies for WikimediaMaintenance T224770. [20:47:12] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:47:13] T224770: Add phan to WikimediaMaintenance extension - https://phabricator.wikimedia.org/T224770 [20:47:13] T224773: Add phan to WikibaseQualityConstraints extension - https://phabricator.wikimedia.org/T224773 [21:42:53] 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO, 10Wikimedia-General-or-Unknown, 10Epic, and 2 others: Enable mediawiki/mediawiki-phan-config on all Wikimedia-deployed repositories - https://phabricator.wikimedia.org/T224783 (10Jdforrester-WMF) [22:04:43] 10Phabricator: Improve Phabricator error messages when uploading files - https://phabricator.wikimedia.org/T234261 (10Aklapper) p:05Normal→03Triage (Resetting priority as per https://www.mediawiki.org/wiki/Phabricator/Project_management#Setting_task_priorities ) [22:04:51] 10Phabricator: Improve Phabricator error messages when uploading files - https://phabricator.wikimedia.org/T234261 (10Aklapper) [22:04:53] 10Phabricator (Upstream), 10Upstream: Unclear error message when uploading a larger attachment: "Exception: No configured storage engine can store this file." - https://phabricator.wikimedia.org/T155130 (10Aklapper) [22:11:57] 10Phabricator: Reset 2FA for Phabricator account `Apap04` - https://phabricator.wikimedia.org/T234233 (10Aklapper) Do you maybe have SSH access to Toolforge etc and could upload a file, to verify this request? (Not sure if this makes sense, that's why I asked which way you think you could verify that you are you.) [22:30:24] 10Phabricator: Reset 2FA for Phabricator account `Apap04` - https://phabricator.wikimedia.org/T234233 (10Apap04) How do I do that? I don't mess around with Toolforge or any other developer tools like that. [22:39:54] 10Phabricator: Reset 2FA for Phabricator account `Apap04` - https://phabricator.wikimedia.org/T234233 (10Apap04) I do have my SSH key setup, I just wanna know how to upload a file. [22:42:04] (03PS1) 10Jforrester: [DNM,WIP] dockerfiles: [ci-src-setup] Move to php72/stretch [integration/config] - 10https://gerrit.wikimedia.org/r/539984 (https://phabricator.wikimedia.org/T234062) [22:42:07] hashar: ^^ I'm lost as to how we were pulling in zuul before. [22:49:27] James_F: there is on zuul package for Stretch [22:49:30] gerr [22:49:31] aze [22:49:32] english [22:49:41] there is NO zuul package on Stretch. It is only on Jessie [22:54:23] Oh. [22:54:27] Well, that's going to go badly. [22:54:42] Can we just scrap ci-src-setup and make MediaWiki use ci-src-setup-simple like everything else? [22:55:24] Or at least splice python and whatever onto a fork of ci-src-setup-simple? [22:57:32] (03CR) 10Hashar: "Probably we should just drop that ci-src-setup container. It really ends up doing zuul-cloner and then composer update." (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/539984 (https://phabricator.wikimedia.org/T234062) (owner: 10Jforrester) [22:57:47] James_F: yeah probably [22:57:55] depends what releng/ci-src-setup: is used for [22:58:04] but that is probably one of the very first container we have created [22:58:28] It's only used by docker-ci-src-setup-mw. [22:58:53] Which is only used by mediawiki-core-php72-phan-docker and mediawiki-core-php72-phan-seccheck-docker [22:59:02] ohhh [22:59:12] We probably don't need all the extra crap? [22:59:54] maybe not? [23:00:03] We run setup-mw.sh which is special. [23:00:08] (zuul-cloner) [23:00:12] But that's it. [23:00:22] apparently I tried to migrate that macro to use quibble https://gerrit.wikimedia.org/r/#/c/integration/config/+/503362/6/jjb/macro-docker.yaml [23:00:23] oh no [23:00:29] another macro for mwext [23:00:30] But we actually have a docker-zuul-cloner jjb? [23:01:04] So maybe we should switch to just doing ci-src-setup-simple and then docker-zuul-cloner? [23:01:05] maybe yes [23:01:12] +1 yeah [23:01:34] if that is at all ending up doing the same as the ci-src-setup setup-mw.sh script [23:01:57] it is probably a leftover that predates quibble [23:02:00] pre date [23:02:03] .. [23:02:26] `zuul-cloner --color --verbose --map /srv/deployment/integration/slave-scripts/etc/zuul-clonemap.yaml --workspace /src --cache-dir /srv/git https://gerrit.wikimedia.org/r/p mediawiki/core` [23:02:27] * hashar promotes James Chief Technicaldebt Officer [23:02:34] Argh. [23:02:38] No no no. :-) [23:02:41] you get perks!! [23:02:44] donuts ;] [23:02:51] Uh-huh. [23:02:57] Endless misery. But doughnuts. [23:03:01] Lol [23:03:06] and the pleasure to listen to a dedicated praise WITH FRENCH ACCENT! [23:03:13] Mais oui. [23:03:20] ;]]] [23:03:34] so in short yeah that ci-src-setup container can probably be phased out entirely :-] [23:04:22] I would like to dig in the phan jobs as well and opitmizes / clean them out [23:04:32] but well hmm I don't know from where to start ;] [23:07:04] (03PS1) 10Jforrester: jjb: Replace docker-ci-src-setup-mw with docker-zuul-cloner followed by docker-ci-src-setup-simple [integration/config] - 10https://gerrit.wikimedia.org/r/539987 (https://phabricator.wikimedia.org/T234062) [23:07:10] ^ Not tested. [23:08:05] (03CR) 10jerkins-bot: [V: 04-1] jjb: Replace docker-ci-src-setup-mw with docker-zuul-cloner followed by docker-ci-src-setup-simple [integration/config] - 10https://gerrit.wikimedia.org/r/539987 (https://phabricator.wikimedia.org/T234062) (owner: 10Jforrester) [23:09:18] (03PS2) 10Jforrester: jjb: Replace docker-ci-src-setup-mw with docker-zuul-cloner followed by docker-ci-src-setup-simple [integration/config] - 10https://gerrit.wikimedia.org/r/539987 (https://phabricator.wikimedia.org/T234062) [23:09:20] (03PS1) 10Jforrester: dockerfiles: Drop ci-src-setup, unused [integration/config] - 10https://gerrit.wikimedia.org/r/539989 [23:10:56] (03PS3) 10Jforrester: jjb: Replace docker-ci-src-setup-mw with docker-zuul-cloner followed by docker-ci-src-setup-simple [integration/config] - 10https://gerrit.wikimedia.org/r/539987 (https://phabricator.wikimedia.org/T234062) [23:10:58] (03PS2) 10Jforrester: dockerfiles: Drop ci-src-setup, unused [integration/config] - 10https://gerrit.wikimedia.org/r/539989 [23:15:56] (03CR) 10Jforrester: "(Not tested yet.)" [integration/config] - 10https://gerrit.wikimedia.org/r/539987 (https://phabricator.wikimedia.org/T234062) (owner: 10Jforrester) [23:19:05] Eurgh. Tomorrow.