[00:20:09] Project beta-update-databases-eqiad build #32010: 04STILL FAILING in 8.8 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32010/ [00:57:57] 10Phabricator (Upstream), 10Upstream: Add note about triggering notification emails for doing actions like on chromium project managment software - https://phabricator.wikimedia.org/T216330 (10Aklapper) 05Open→03Declined I am not in favor of adding more and more and more and more of "Do not do XYZ" hints e... [01:00:50] Project beta-scap-eqiad build #238160: 04FAILURE in 8.4 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/238160/ [01:12:13] ^^ `[scap] 01:00:44 Started rsync common; sudo: a password is required` [01:12:49] !log beta-scap-eqiad (cron) failing with "sudo: a password is required" [01:12:50] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [01:14:25] Yippee, build fixed! [01:14:26] Project beta-scap-eqiad build #238161: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/238161/ [01:19:46] 10Release-Engineering-Team: Release Engineering Data Collection and Retention (aka Data³) - https://phabricator.wikimedia.org/T216085 (10Aklapper) [adding #release-engineering-team tag; feel free to correct] [01:20:05] Project beta-update-databases-eqiad build #32011: 04STILL FAILING in 4.3 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32011/ [01:24:46] 10Deployments, 10Release-Engineering-Team (Backlog), 10wikitech.wikimedia.org, 10User-MModell: Create an easier way to add/remove/modify patches for SWAT - https://phabricator.wikimedia.org/T171940 (10Aklapper) [01:31:08] Krinkle: that sounded like keyholder not being armed after the wmcs hardware failures, but: https://phabricator.wikimedia.org/P8100 [02:20:09] Project beta-update-databases-eqiad build #32012: 04STILL FAILING in 8.8 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32012/ [03:20:04] Project beta-update-databases-eqiad build #32013: 04STILL FAILING in 4.2 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32013/ [04:05:43] 10Deployments, 10Release-Engineering-Team (Backlog), 10wikitech.wikimedia.org, 10User-MModell: Create an easier way to manage SWAT deploys on (Deployments calendar) - https://phabricator.wikimedia.org/T171940 (10Krinkle) [04:05:58] 10Deployments, 10Release-Engineering-Team (Backlog), 10wikitech.wikimedia.org, 10User-MModell: Create an easier way to manage SWAT deploys on (Deployments calendar) - https://phabricator.wikimedia.org/T171940 (10Krinkle) Proof of concept using TemplateData: Project beta-update-databases-eqiad build #32014: 04STILL FAILING in 9.4 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32014/ [05:20:05] Project beta-update-databases-eqiad build #32015: 04STILL FAILING in 4.2 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32015/ [05:21:53] 10Project-Admins: Create project for GraphQL extension - https://phabricator.wikimedia.org/T216310 (10Aklapper) Uhm. Has it been considered to not give different and separate codebases the very same names instead, to avoid ambiguity and confusion? [05:49:55] 10Release-Engineering-Team (Kanban), 10Developer Productivity: Gather and Analyze Information Around Developer Tooling Woes - https://phabricator.wikimedia.org/T212454 (10Aklapper) [05:49:59] 10Release-Engineering-Team, 10Developer Productivity, 10Epic: Improve Developer Tooling - https://phabricator.wikimedia.org/T212449 (10Aklapper) [06:13:31] 10Phabricator, 10Release-Engineering-Team (Backlog): Some specific Maniphest search queries are shown for all users - https://phabricator.wikimedia.org/T214579 (10Aklapper) [06:20:04] Project beta-update-databases-eqiad build #32016: 04STILL FAILING in 4.2 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32016/ [06:24:32] (03PS2) 10Legoktm: Rename tox-certcentral to tox-acme-chief [integration/config] - 10https://gerrit.wikimedia.org/r/489725 (https://phabricator.wikimedia.org/T207389) (owner: 10Vgutierrez) [06:24:34] (03CR) 10Legoktm: [C: 03+2] Rename tox-certcentral to tox-acme-chief [integration/config] - 10https://gerrit.wikimedia.org/r/489725 (https://phabricator.wikimedia.org/T207389) (owner: 10Vgutierrez) [06:26:21] (03Merged) 10jenkins-bot: Rename tox-certcentral to tox-acme-chief [integration/config] - 10https://gerrit.wikimedia.org/r/489725 (https://phabricator.wikimedia.org/T207389) (owner: 10Vgutierrez) [06:28:27] !log building new tox-acme-chief docker image https://gerrit.wikimedia.org/r/489725 [06:28:28] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [06:29:02] [contint1001.wikimedia.org] out: => Building image docker-registry.discovery.wmnet/releng/tox-acme-chief:0.3.3 [06:29:02] [contint1001.wikimedia.org] out: ERROR: image docker-registry.discovery.wmnet/releng/tox-acme-chief failed to build, see logs for details [06:29:27] but not diff between tox-acme-chief/Dockerfile.template tox-certcentral/Dockerfile.template [06:59:35] [contint1001.wikimedia.org] out: docker.errors.BuildError: The command '/bin/sh -c echo 'Acquire::http::Proxy "http://webproxy.eqiad.wmnet:8080";' > /etc/apt/apt.conf.d/80_proxy && apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install --yes golang-1.10-go --no-install-recommends && rm -f /etc/apt/apt.conf.d/80_proxy && apt-get clean && rm -rf /var/lib/apt/lists/*' returned a non-zero code: 100 [07:01:31] uh [07:01:34] it's in stretch backports [07:06:40] (03PS1) 10Legoktm: Use golang-1.11-go in tox-acme-chief image [integration/config] - 10https://gerrit.wikimedia.org/r/491028 [07:07:32] (03CR) 10Legoktm: [C: 03+2] Use golang-1.11-go in tox-acme-chief image [integration/config] - 10https://gerrit.wikimedia.org/r/491028 (owner: 10Legoktm) [07:09:24] (03Merged) 10jenkins-bot: Use golang-1.11-go in tox-acme-chief image [integration/config] - 10https://gerrit.wikimedia.org/r/491028 (owner: 10Legoktm) [07:10:17] !log Building image docker-registry.discovery.wmnet/releng/tox-acme-chief:0.3.4 [07:10:18] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:10:43] \o/ success [07:17:49] (03PS1) 10Legoktm: Rename jenkins job to acme-chief-tox-docker [integration/config] - 10https://gerrit.wikimedia.org/r/491029 [07:18:06] (03PS2) 10Legoktm: Rename jenkins job to acme-chief-tox-docker [integration/config] - 10https://gerrit.wikimedia.org/r/491029 (https://phabricator.wikimedia.org/T207389) [07:18:50] (03CR) 10Legoktm: [C: 03+2] Rename jenkins job to acme-chief-tox-docker [integration/config] - 10https://gerrit.wikimedia.org/r/491029 (https://phabricator.wikimedia.org/T207389) (owner: 10Legoktm) [07:20:15] Project beta-update-databases-eqiad build #32017: 04STILL FAILING in 15 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32017/ [07:21:35] (03Merged) 10jenkins-bot: Rename jenkins job to acme-chief-tox-docker [integration/config] - 10https://gerrit.wikimedia.org/r/491029 (https://phabricator.wikimedia.org/T207389) (owner: 10Legoktm) [07:21:59] !log deploying https://gerrit.wikimedia.org/r/491029 [07:22:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:23:10] (03PS1) 10Legoktm: Remove obsolete tox-certcentral docker image [integration/config] - 10https://gerrit.wikimedia.org/r/491030 (https://phabricator.wikimedia.org/T207389) [07:23:25] (03CR) 10Legoktm: [C: 03+2] Remove obsolete tox-certcentral docker image [integration/config] - 10https://gerrit.wikimedia.org/r/491030 (https://phabricator.wikimedia.org/T207389) (owner: 10Legoktm) [07:25:19] (03Merged) 10jenkins-bot: Remove obsolete tox-certcentral docker image [integration/config] - 10https://gerrit.wikimedia.org/r/491030 (https://phabricator.wikimedia.org/T207389) (owner: 10Legoktm) [07:46:22] 10Phabricator: MinervaNeue tag links to nonexisting page - https://phabricator.wikimedia.org/T212183 (10Aklapper) [07:46:26] 10Phabricator (Upstream), 10Release-Engineering-Team (Kanban), 10Upstream, 10User-MModell, 10User-Ryasmeen: 404 on workboard for an existing project (due to custom filter applied which did not exist in database) - https://phabricator.wikimedia.org/T199207 (10Aklapper) [08:20:04] Project beta-update-databases-eqiad build #32018: 04STILL FAILING in 3.8 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32018/ [08:27:18] 10Project-Admins, 10Security-Team: Create maybe-public tag - https://phabricator.wikimedia.org/T215981 (10Aklapper) I'm afraid of overengineering here and the tag name sounds very broad and misleading without reading its tag description... [09:20:05] Project beta-update-databases-eqiad build #32019: 04STILL FAILING in 4.3 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32019/ [09:36:27] 10Project-Admins, 10Security-Team: Create maybe-public tag - https://phabricator.wikimedia.org/T215981 (10Bawolff) Personally, I think all tasks not explicitly marked private, should be "maybe-public", and it is primarily the responsibility of the bug closer to make security bugs public (yeah I know, lots of b... [10:00:15] 10Phabricator, 10I18n: Please add Brazilian Portuguese (pt-br) language to Phabricator - https://phabricator.wikimedia.org/T215697 (10Aklapper) Removing #Language-Team as this has nothing to do with the Language Team (see description) and it is up to each team what they plan to work on [10:20:08] Project beta-update-databases-eqiad build #32020: 04STILL FAILING in 8.2 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32020/ [11:20:05] Project beta-update-databases-eqiad build #32021: 04STILL FAILING in 4.2 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32021/ [12:16:16] 10Continuous-Integration-Config: False positive PhanUndeclaredVariable in try-catch - https://phabricator.wikimedia.org/T207835 (10Bawolff) rm tag phan-taint-check tag - That tag is only for the XSS detector part of phan [12:20:05] Project beta-update-databases-eqiad build #32022: 04STILL FAILING in 4.3 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32022/ [13:00:46] Project beta-scap-eqiad build #238225: 04FAILURE in 6 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/238225/ [13:14:17] Yippee, build fixed! [13:14:18] Project beta-scap-eqiad build #238226: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/238226/ [13:20:05] Project beta-update-databases-eqiad build #32023: 04STILL FAILING in 4.3 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32023/ [14:14:12] 10Project-Admins: Create project for GraphQL extension - https://phabricator.wikimedia.org/T216310 (10dbarratt) >>! In T216310#4959828, @Aklapper wrote: > Uhm. Has it been considered to not give different and separate codebases the very same names instead, to avoid ambiguity and confusion? I thought about it, b... [14:20:04] Project beta-update-databases-eqiad build #32024: 04STILL FAILING in 4.2 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32024/ [15:12:25] Project mediawiki-core-code-coverage-docker build #4074: 04FAILURE in 12 min: https://integration.wikimedia.org/ci/job/mediawiki-core-code-coverage-docker/4074/ [15:20:04] Project beta-update-databases-eqiad build #32025: 04STILL FAILING in 4.2 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32025/ [16:20:13] Project beta-update-databases-eqiad build #32026: 04STILL FAILING in 12 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32026/ [17:20:05] Project beta-update-databases-eqiad build #32027: 04STILL FAILING in 4.2 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32027/ [18:20:05] Project beta-update-databases-eqiad build #32028: 04STILL FAILING in 4.4 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32028/ [18:32:19] phan/phan-seccheck is failing on lots of patches lately [18:33:15] ba (baowolff) would know more [18:34:18] I'll see if I can speak to him [18:35:51] lots of SlowTimer as well, connectivity issues? [18:36:04] cf. https://integration.wikimedia.org/ci/job/wmf-quibble-vendor-mysql-hhvm-docker/16989/console [18:43:18] greg-g: CI is somewhat broken, if you know any releng people that is on-duty today I'd kindly ask him to take a look [18:51:10] hauskatze: there is still on-going maintenance with the WMCS VPS project (afaik) so there may not be anything we can do right now, especially on a sunday. [18:54:26] all of the failures I'm seeing right now (phan/seccheck) are of this type: [18:54:28] every Wednesday at 16:00 UTC (we always keep the meeting at 17:00 MEZ) [18:54:31] grr, mispaste [18:54:40] 18:21:08 [Composer\Downloader\TransportException] [18:54:42] 18:21:08 Content-Length mismatch, received 215489 bytes out of the expected 330567 [18:54:59] that was from https://integration.wikimedia.org/ci/job/release-quibble-vendor-mysql-php70-docker/364/console, which isn't phan related [18:55:02] it's composer [18:57:44] castor-save is having issues, which was one of the instances that was curropted by the wmcs hardware outage [18:58:44] 10Continuous-Integration-Config: CI unstable/mass erroring (2019-02-17) - https://phabricator.wikimedia.org/T216371 (10MarcoAurelio) p:05Triage→03Unbreak! [18:59:28] greg-g: I understand Sunday is a problem indeed. I filed a task. Feel free to decrease priority, etc. [19:00:03] also SlowTimer [19:00:13] which makes the tests expire, leaving patches untested [19:01:08] 10Continuous-Integration-Config: CI unstable/mass erroring (2019-02-17) - https://phabricator.wikimedia.org/T216371 (10greg) p:05Unbreak!→03High ` 18:51:10 +greg-g | hauskatze: there is still on-going maintenance with the WMCS VPS project (afaik) so there may not be anything we can do right no... [19:02:09] 10Continuous-Integration-Config: CI unstable/mass erroring (2019-02-17) - https://phabricator.wikimedia.org/T216371 (10greg) Setting to high to set expectations that this may not be resolved before normal business hours. There are a multitude of confounding circumstances involved (WMCS hardware outage and the re... [19:20:09] Project beta-update-databases-eqiad build #32029: 04STILL FAILING in 8.7 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32029/ [19:41:10] 10Phabricator, 10Release-Engineering-Team (Kanban), 10User-greg: Reset 2FA for @Fan - https://phabricator.wikimedia.org/T215861 (10greg) 05Open→03Resolved {{done}} [19:41:43] 10Release-Engineering-Team: Release Engineering Data Collection and Retention (aka Data³) - https://phabricator.wikimedia.org/T216085 (10greg) p:05Triage→03Normal [19:43:31] 10Beta-Cluster-Infrastructure: beta-scap-eqiad repeat failures - https://phabricator.wikimedia.org/T216321 (10greg) 05Open→03Resolved [19:44:52] * paladox is updating the zuul-status desgn [19:44:53] https://phabricator.wikimedia.org/F28235523 [19:45:04] looks nicer and cleaner (matches https://i.imgur.com/l0y7OVw.png) [20:10:19] Krinkle btw this is what project dashboard looks like under polygerrit (obvously not the same data) but gives you a idea https://gerrit-review.googlesource.com/admin/repos/gerrit,dashboards [20:11:33] paladox: Yeah, looks straight forward. Even in 2.15 polygerrit already has dashboards, e.g. the home page when you're logged in is basically a dashboard. Three sections with names and some query results. [20:11:43] Yeh [20:20:05] Project beta-update-databases-eqiad build #32030: 04STILL FAILING in 4.3 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32030/ [21:20:05] Project beta-update-databases-eqiad build #32031: 04STILL FAILING in 4.3 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32031/ [22:06:27] there we go https://phabricator.wikimedia.org/F28236908 with icons [22:19:24] https://gerrit-review.googlesource.com/c/plugins/zuul-status/+/214812 [22:20:05] Project beta-update-databases-eqiad build #32032: 04STILL FAILING in 4.4 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32032/ [23:20:05] Project beta-update-databases-eqiad build #32033: 04STILL FAILING in 4.4 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32033/