[00:02:37] 10Release-Engineering-Team: Sniff error for FunctionCommentSniff - https://phabricator.wikimedia.org/T188053#3994849 (10Legoktm) [00:02:40] 10MediaWiki-Codesniffer, 10Patch-For-Review: FunctionCommentSniff fails on substr_compare - https://phabricator.wikimedia.org/T187575#3994852 (10Legoktm) [00:31:16] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [00:59:40] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<33.33%) [01:31:36] 10Beta-Cluster-Infrastructure, 10Commons, 10MediaWiki-File-management, 10Multimedia, 10Tracking: Thumbnail generation should happen via the same setup in the beta cluster and in production (tracking) - https://phabricator.wikimedia.org/T84950#3995049 (10Krinkle) [01:33:53] (03CR) 10Legoktm: "TBH I don't think we've really ever made a distinction between errors and warnings since CI/"composer test" will fail on either." (031 comment) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/413218 (owner: 10Thiemo Kreuz (WMDE)) [01:33:56] (03PS2) 10Legoktm: Grade errors about duplicate spaces in PHPDoc tags down to warnings [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/413218 (owner: 10Thiemo Kreuz (WMDE)) [01:34:00] (03CR) 10Legoktm: [C: 032] Grade errors about duplicate spaces in PHPDoc tags down to warnings [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/413218 (owner: 10Thiemo Kreuz (WMDE)) [01:35:09] (03Merged) 10jenkins-bot: Grade errors about duplicate spaces in PHPDoc tags down to warnings [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/413218 (owner: 10Thiemo Kreuz (WMDE)) [01:39:01] (03PS2) 10Legoktm: Fix SpaceBeforeControlStructureBraceSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/413161 (owner: 10Thiemo Kreuz (WMDE)) [01:43:14] (03CR) 10Legoktm: [C: 032] "Thanks :)" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/413161 (owner: 10Thiemo Kreuz (WMDE)) [01:44:15] (03Merged) 10jenkins-bot: Fix SpaceBeforeControlStructureBraceSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/413161 (owner: 10Thiemo Kreuz (WMDE)) [01:45:40] (03CR) 10jenkins-bot: Grade errors about duplicate spaces in PHPDoc tags down to warnings [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/413218 (owner: 10Thiemo Kreuz (WMDE)) [01:47:02] (03CR) 10jenkins-bot: Fix SpaceBeforeControlStructureBraceSniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/413161 (owner: 10Thiemo Kreuz (WMDE)) [01:49:28] (03PS2) 10Legoktm: Streamline SpaceBeforeControlStructureBraceSniff implementation [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/413225 (owner: 10Thiemo Kreuz (WMDE)) [01:50:00] (03CR) 10Legoktm: [C: 032] "I don't know why git made me do a manual rebase" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/413225 (owner: 10Thiemo Kreuz (WMDE)) [01:50:38] (03Merged) 10jenkins-bot: Streamline SpaceBeforeControlStructureBraceSniff implementation [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/413225 (owner: 10Thiemo Kreuz (WMDE)) [01:52:13] (03CR) 10jenkins-bot: Streamline SpaceBeforeControlStructureBraceSniff implementation [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/413225 (owner: 10Thiemo Kreuz (WMDE)) [03:34:47] (03PS1) 10Legoktm: Close pipes to avoid too many files being open [integration/audit-resources] - 10https://gerrit.wikimedia.org/r/413663 [03:34:49] (03PS1) 10Legoktm: Downgrade slimerjs to support Firefox 52 ESR [integration/audit-resources] - 10https://gerrit.wikimedia.org/r/413664 [03:34:59] (03CR) 10Legoktm: [V: 032 C: 032] Close pipes to avoid too many files being open [integration/audit-resources] - 10https://gerrit.wikimedia.org/r/413663 (owner: 10Legoktm) [03:35:07] (03CR) 10Legoktm: [V: 032 C: 032] Downgrade slimerjs to support Firefox 52 ESR [integration/audit-resources] - 10https://gerrit.wikimedia.org/r/413664 (owner: 10Legoktm) [03:36:11] can the CI docker image build host talk to github? [04:17:53] 10Continuous-Integration-Infrastructure, 10MediaWiki-Core-Tests, 10Operations, 10HHVM: HHVM 3.18.5+dfsg-1+wmf3 changes parse_url causing unit tests to fail - https://phabricator.wikimedia.org/T185024#3903647 (10greg) >>! In T185024#3931478, @MoritzMuehlenhoff wrote: > A revised fix has been released (along... [04:50:31] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [05:20:33] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [10.0] [06:54:38] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [08:07:50] (03PS1) 10Legoktm: Run properly with Firefox 52 ESR in a docker container [integration/audit-resources] - 10https://gerrit.wikimedia.org/r/413680 [08:08:06] (03PS1) 10Legoktm: Add tox.ini to run flake8 [integration/audit-resources] - 10https://gerrit.wikimedia.org/r/413681 [08:08:57] (03PS1) 10Legoktm: Improve README [integration/audit-resources] - 10https://gerrit.wikimedia.org/r/413682 [08:10:53] (03CR) 10Legoktm: [V: 032 C: 032] Run properly with Firefox 52 ESR in a docker container [integration/audit-resources] - 10https://gerrit.wikimedia.org/r/413680 (owner: 10Legoktm) [08:11:24] (03CR) 10Legoktm: [V: 032 C: 032] Add tox.ini to run flake8 [integration/audit-resources] - 10https://gerrit.wikimedia.org/r/413681 (owner: 10Legoktm) [08:11:26] (03CR) 10Legoktm: [V: 032 C: 032] Improve README [integration/audit-resources] - 10https://gerrit.wikimedia.org/r/413682 (owner: 10Legoktm) [08:18:12] (03PS1) 10Hashar: Archive apps/android/java-mwapi [integration/config] - 10https://gerrit.wikimedia.org/r/413683 (https://phabricator.wikimedia.org/T187995) [08:19:02] !log gerrit: marked apps/android/java-mwapi.git read-only | T187995 [08:19:09] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:19:09] T187995: java-mwapi fails to run gradle 3.4 with java7: java.security.InvalidKeyException: EC parameters error - https://phabricator.wikimedia.org/T187995 [08:19:57] (03CR) 10Hashar: [C: 032] Archive apps/android/java-mwapi [integration/config] - 10https://gerrit.wikimedia.org/r/413683 (https://phabricator.wikimedia.org/T187995) (owner: 10Hashar) [08:22:06] (03Merged) 10jenkins-bot: Archive apps/android/java-mwapi [integration/config] - 10https://gerrit.wikimedia.org/r/413683 (https://phabricator.wikimedia.org/T187995) (owner: 10Hashar) [08:24:07] 10Continuous-Integration-Infrastructure (shipyard), 10Android-app-Bugs, 10Wikipedia-Android-App-Backlog, 10Patch-For-Review: java-mwapi fails to run gradle 3.4 with java7: java.security.InvalidKeyException: EC parameters error - https://phabricator.wikimedia.org/T187995#3995488 (10hashar) 05Open>03decli... [08:24:32] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban): Migrate leftover Nodepool jobs to Docker - https://phabricator.wikimedia.org/T187797#3995490 (10hashar) [08:25:43] (03PS1) 10Legoktm: Add docker image to run audit-resources [integration/config] - 10https://gerrit.wikimedia.org/r/413684 (https://phabricator.wikimedia.org/T71519) [08:25:55] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban): Migrate leftover Nodepool jobs to Docker - https://phabricator.wikimedia.org/T187797#3986123 (10hashar) [09:02:53] (03PS1) 10Hashar: Migrate mediawiki-core-jsduck-publish to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413686 [09:03:24] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban): Migrate leftover Nodepool jobs to Docker - https://phabricator.wikimedia.org/T187797#3995554 (10hashar) [09:04:29] (03CR) 10Hashar: [C: 032] Migrate mediawiki-core-jsduck-publish to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413686 (owner: 10Hashar) [09:06:32] (03Merged) 10jenkins-bot: Migrate mediawiki-core-jsduck-publish to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413686 (owner: 10Hashar) [09:29:27] Project beta-scap-eqiad build #196850: 04FAILURE in 15 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196850/ [09:30:41] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Zuul: Zuul no longer starts gate-and-submit when parent is in gate-and-submit pipeline - https://phabricator.wikimedia.org/T186949#3995589 (10Lucas_Werkmeister_WMDE) 05Resolved>03Open For example, [this chain](https://gerrit.... [09:30:50] PROBLEM - Free space - all mounts on deployment-mediawiki04 is CRITICAL: CRITICAL: deployment-prep.deployment-mediawiki04.diskspace.root.byte_percentfree (<55.56%) [09:35:50] Project beta-scap-eqiad build #196851: 04STILL FAILING in 5 min 46 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196851/ [09:40:11] Project beta-scap-eqiad build #196852: 04STILL FAILING in 3 min 42 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196852/ [09:45:01] Project mwext-phpunit-coverage-publish build #1341: 04FAILURE in 30 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/1341/ [09:46:35] Yippee, build fixed! [09:46:36] Project mwext-phpunit-coverage-publish build #1342: 09FIXED in 1 min 34 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/1342/ [09:47:14] Project beta-scap-eqiad build #196853: 04STILL FAILING in 3 min 37 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196853/ [09:55:31] (03CR) 10Thiemo Kreuz (WMDE): Grade errors about duplicate spaces in PHPDoc tags down to warnings (031 comment) [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/413218 (owner: 10Thiemo Kreuz (WMDE)) [09:57:15] Project beta-scap-eqiad build #196854: 04STILL FAILING in 3 min 37 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196854/ [10:03:41] (03PS1) 10Hashar: Introduce docker-zuul-cloner [integration/config] - 10https://gerrit.wikimedia.org/r/413692 [10:04:13] (03PS1) 10Hashar: Migrate mwext-MobileFrontend-npm-run-lint-modules to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413693 [10:04:44] (03CR) 10Hashar: [C: 04-1] "Fails:" [integration/config] - 10https://gerrit.wikimedia.org/r/413693 (owner: 10Hashar) [10:07:16] Project beta-scap-eqiad build #196855: 04STILL FAILING in 3 min 39 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196855/ [10:10:53] Project beta-scap-eqiad build #196856: 04STILL FAILING in 3 min 34 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196856/ [10:11:48] PROBLEM - Free space - all mounts on deployment-eventlog02 is CRITICAL: CRITICAL: deployment-prep.deployment-eventlog02.diskspace.root.byte_percentfree (<100.00%) [10:13:49] (03PS2) 10Hashar: Migrate mwext-MobileFrontend-npm-run-lint-modules to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413693 [10:13:51] (03PS1) 10Hashar: docker: npm-php [integration/config] - 10https://gerrit.wikimedia.org/r/413694 [10:14:00] (03CR) 10Hashar: [C: 032] "Will be used in https://gerrit.wikimedia.org/r/#/c/413693/" [integration/config] - 10https://gerrit.wikimedia.org/r/413692 (owner: 10Hashar) [10:16:11] (03Merged) 10jenkins-bot: Introduce docker-zuul-cloner [integration/config] - 10https://gerrit.wikimedia.org/r/413692 (owner: 10Hashar) [10:17:27] Project beta-scap-eqiad build #196857: 04STILL FAILING in 3 min 46 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196857/ [10:18:37] (03CR) 10Hashar: [C: 032] docker: npm-php [integration/config] - 10https://gerrit.wikimedia.org/r/413694 (owner: 10Hashar) [10:19:57] (03Merged) 10jenkins-bot: docker: npm-php [integration/config] - 10https://gerrit.wikimedia.org/r/413694 (owner: 10Hashar) [10:24:31] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban): releases1001 has full / partition - https://phabricator.wikimedia.org/T188080#3995690 (10hashar) [10:27:22] Project beta-scap-eqiad build #196858: 04STILL FAILING in 3 min 40 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196858/ [10:30:50] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban): releases1001 has full / partition - https://phabricator.wikimedia.org/T188080#3995722 (10hashar) I have deleted the mediawiki snapshots directories from the jobs. They are kept behind on each runs. Surely they should be deleted once... [10:37:22] Project beta-scap-eqiad build #196859: 04STILL FAILING in 3 min 42 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196859/ [10:37:38] (03PS3) 10Hashar: Migrate mwext-MobileFrontend-npm-run-lint-modules to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413693 [10:37:52] (03PS4) 10Hashar: Migrate mwext-MobileFrontend-npm-run-lint-modules to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413693 [10:39:32] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban): Migrate leftover Nodepool jobs to Docker - https://phabricator.wikimedia.org/T187797#3995754 (10hashar) [10:39:44] (03CR) 10Hashar: [C: 032] Migrate mwext-MobileFrontend-npm-run-lint-modules to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413693 (owner: 10Hashar) [10:40:13] 10MediaWiki-Releasing: Add Git tags REL1_XX for historical branches - https://phabricator.wikimedia.org/T188084#3995757 (10Seb35) [10:41:48] (03Merged) 10jenkins-bot: Migrate mwext-MobileFrontend-npm-run-lint-modules to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413693 (owner: 10Hashar) [10:47:28] Project beta-scap-eqiad build #196860: 04STILL FAILING in 3 min 48 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196860/ [10:54:07] PROBLEM - Host deployment-videoscaler01 is DOWN: CRITICAL - Host Unreachable (10.68.19.130) [10:54:53] PROBLEM - Host deployment-tmh01 is DOWN: CRITICAL - Host Unreachable (10.68.16.211) [10:57:21] Project beta-scap-eqiad build #196861: 04STILL FAILING in 3 min 43 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196861/ [11:00:17] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban): Migrate leftover Nodepool jobs to Docker - https://phabricator.wikimedia.org/T187797#3995843 (10hashar) [11:05:42] Project beta-scap-eqiad build #196862: 04STILL FAILING in 3 min 42 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196862/ [11:10:06] Project beta-scap-eqiad build #196863: 04STILL FAILING in 3 min 44 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196863/ [11:17:26] Project beta-scap-eqiad build #196864: 04STILL FAILING in 3 min 46 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196864/ [11:27:29] Project beta-scap-eqiad build #196865: 04STILL FAILING in 3 min 48 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196865/ [11:31:15] Project beta-scap-eqiad build #196866: 04STILL FAILING in 3 min 43 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196866/ [11:37:38] Project beta-scap-eqiad build #196867: 04STILL FAILING in 3 min 53 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196867/ [11:41:29] Project beta-scap-eqiad build #196868: 04STILL FAILING in 3 min 48 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196868/ [11:44:21] (03PS1) 10Hashar: Migrate mwext-VisualEditor-publish to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413705 [11:46:18] (03CR) 10Hashar: [C: 032] Migrate mwext-VisualEditor-publish to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413705 (owner: 10Hashar) [11:46:48] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban): Migrate leftover Nodepool jobs to Docker - https://phabricator.wikimedia.org/T187797#3995974 (10hashar) [11:47:27] Project beta-scap-eqiad build #196869: 04STILL FAILING in 3 min 44 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196869/ [11:48:14] !log deployment-mediawiki04 is out of disk space on / causing beta-scap-eqiad to fail [11:48:20] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:48:20] (03Merged) 10jenkins-bot: Migrate mwext-VisualEditor-publish to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413705 (owner: 10Hashar) [11:50:49] !log deployment-mediawiki04 : rm /var/cache/hhvm/*.sq3 and restarting hhvm [11:50:55] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:50:59] !log deployment-mediawiki04 : apt-get clean && apt-get autoremove --purge [11:51:04] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:54:53] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban): Migrate leftover Nodepool jobs to Docker - https://phabricator.wikimedia.org/T187797#3995983 (10hashar) [11:58:49] Yippee, build fixed! [11:58:50] Project beta-scap-eqiad build #196870: 09FIXED in 5 min 11 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/196870/ [12:00:36] hashar: did you got my PM? [12:05:49] RECOVERY - Free space - all mounts on deployment-mediawiki04 is OK: OK: All targets OK [12:18:02] Hauskatze: the Gerrit "grunt-merge-i18n" group https://gerrit.wikimedia.org/r/#/admin/groups/1442,members ? [12:18:08] realyl I have no idea what this group is for [12:18:17] I created it [12:18:24] you can see it in the audit log [12:18:32] removed myself in error [12:18:35] ;D [12:18:37] you are back in [12:18:41] ok [12:18:43] thanks [12:18:46] !!!! [12:25:24] ? [12:30:54] PROBLEM - Puppet errors on deployment-ores01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [12:36:42] hashar: is depends-on broken? [12:38:07] addshore: why would it be? [12:38:25] https://gerrit.wikimedia.org/r/#/c/413203/ merged but the depend-on is not merged [12:38:48] it got force merged [12:38:56] and Gerrit doesn't enforce Depends-On :D [12:39:17] so one did a CR+2 at 15:39, that hasn't triggered anything (due to the depends-on) [12:39:22] then it got force merged [12:40:42] addshore: working as intended? :] [12:40:46] ahh i didnt see it was force merged [12:40:52] thanks! [12:40:56] addshore: though really Gerrit could be taught about Depends-On [13:19:42] (03PS1) 10Hashar: Migrate Parsoid doc publishing job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413720 [13:20:57] (03CR) 10jerkins-bot: [V: 04-1] Migrate Parsoid doc publishing job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413720 (owner: 10Hashar) [13:21:41] (03PS2) 10Hashar: Migrate Parsoid doc publishing job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413720 [13:23:35] (03CR) 10Hashar: [C: 032] Migrate Parsoid doc publishing job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413720 (owner: 10Hashar) [13:24:06] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban): Migrate leftover Nodepool jobs to Docker - https://phabricator.wikimedia.org/T187797#3996166 (10hashar) [13:24:59] (03Merged) 10jenkins-bot: Migrate Parsoid doc publishing job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413720 (owner: 10Hashar) [13:49:12] Hi, I've got this patch that was +2 yesterday but it's "Ready to submit" instead of "merged" (https://gerrit.wikimedia.org/r/#/c/413381/). Any idea what's going on? [14:10:19] (03PS1) 10Hashar: Migrate Parsoid source repo jobs to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413727 [14:15:25] (03PS9) 10Zoranzoki21: Add few extensions in zuul/layout.yaml to Jenkins can run builds [integration/config] - 10https://gerrit.wikimedia.org/r/406524 (https://phabricator.wikimedia.org/T183674) [14:15:45] (03CR) 10Zoranzoki21: "@Hashar Please, if you can, edit this patch and deploy. Thanks!" [integration/config] - 10https://gerrit.wikimedia.org/r/406524 (https://phabricator.wikimedia.org/T183674) (owner: 10Zoranzoki21) [14:26:07] stephanebisson: jenkins may not be configured to hit the merge button [14:26:18] it looks like "gate" ran but no "get and submit" [14:30:06] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban): Migrate leftover Nodepool jobs to Docker - https://phabricator.wikimedia.org/T187797#3996293 (10hashar) [14:31:48] (03CR) 10Hashar: [C: 032] Migrate Parsoid source repo jobs to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413727 (owner: 10Hashar) [14:32:19] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban): Migrate leftover Nodepool jobs to Docker - https://phabricator.wikimedia.org/T187797#3996295 (10hashar) [14:33:12] (03Merged) 10jenkins-bot: Migrate Parsoid source repo jobs to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413727 (owner: 10Hashar) [14:33:53] (03CR) 10Hashar: [C: 04-1] "I did a review of patchset 8 at https://gerrit.wikimedia.org/r/#/c/406524/8/zuul/layout.yaml but patchset 9 is just a rebase." [integration/config] - 10https://gerrit.wikimedia.org/r/406524 (https://phabricator.wikimedia.org/T183674) (owner: 10Zoranzoki21) [14:34:49] (03CR) 10Zoranzoki21: "> I did a review of patchset 8 at https://gerrit.wikimedia.org/r/#/c/406524/8/zuul/layout.yaml" [integration/config] - 10https://gerrit.wikimedia.org/r/406524 (https://phabricator.wikimedia.org/T183674) (owner: 10Zoranzoki21) [14:39:37] (03PS10) 10Zoranzoki21: Add few extensions in zuul/layout.yaml to Jenkins can run builds [integration/config] - 10https://gerrit.wikimedia.org/r/406524 (https://phabricator.wikimedia.org/T183674) [14:39:57] (03PS11) 10Zoranzoki21: Add few extensions in zuul/layout.yaml to Jenkins can run builds [integration/config] - 10https://gerrit.wikimedia.org/r/406524 (https://phabricator.wikimedia.org/T183674) [14:40:46] (03CR) 10Zoranzoki21: "OMG. I will fix this" [integration/config] - 10https://gerrit.wikimedia.org/r/406524 (https://phabricator.wikimedia.org/T183674) (owner: 10Zoranzoki21) [14:41:23] (03CR) 10jerkins-bot: [V: 04-1] Add few extensions in zuul/layout.yaml to Jenkins can run builds [integration/config] - 10https://gerrit.wikimedia.org/r/406524 (https://phabricator.wikimedia.org/T183674) (owner: 10Zoranzoki21) [14:42:37] stephanebisson: wrt that patch, maybe its because jenkins isn't allowed to submit or because zuul had problems yesterday. Mind if I retry? [14:48:39] yep, jenkins is not submitting in that repo stephanebisson - I can fix that if you're okay [14:52:31] (03PS12) 10Zoranzoki21: Add few extensions in zuul/layout.yaml to Jenkins can run builds [integration/config] - 10https://gerrit.wikimedia.org/r/406524 (https://phabricator.wikimedia.org/T183674) [14:53:16] PROBLEM - Host deployment-puppetdb01 is DOWN: CRITICAL - Host Unreachable (10.68.23.76) [14:54:35] (03PS13) 10Zoranzoki21: Add few extensions in zuul/layout.yaml to Jenkins can run builds [integration/config] - 10https://gerrit.wikimedia.org/r/406524 (https://phabricator.wikimedia.org/T183674) [14:57:47] 10Continuous-Integration-Infrastructure, 10MediaWiki-Core-Tests, 10Operations, 10HHVM: HHVM 3.18.5+dfsg-1+wmf3 changes parse_url causing unit tests to fail - https://phabricator.wikimedia.org/T185024#3996328 (10MoritzMuehlenhoff) @greg Travis uses the debs packaged by Facebook and new releases have been ma... [15:01:23] (03PS14) 10Zoranzoki21: Add few extensions in zuul/layout.yaml to Jenkins can run builds and remove mediawiki/extensions/Collection/OfflineContentGenerator/node_modules [integration/config] - 10https://gerrit.wikimedia.org/r/406524 (https://phabricator.wikimedia.org/T183674) [15:13:25] Hauskatze, addshore: could there be a reason why it was not configured to merge? I'm learning my way in the maps ecosystem so let's hold on that change for now. [15:13:41] stephanebisson: probably an oversight [15:13:50] https://gerrit.wikimedia.org/r/#/c/413736/ [15:13:51] gehel: so you know why jenkins doesn't merge on +2 in maps/kartotherian/deploy? [15:14:14] the patch above, if it's okay, fixes jenkins-bot so it can submit on the repo [15:15:11] (03PS1) 10Hashar: phpunit-coverage-publish job on Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413737 [15:15:11] stephanebisson: I don't think there was any reason... I would blame ignorance and laziness [15:15:35] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban): Migrate leftover Nodepool jobs to Docker - https://phabricator.wikimedia.org/T187797#3996385 (10hashar) [15:15:38] Ok then, let's do this [15:15:42] ok [15:15:44] stephanebisson: auto-submit would actually be nice! [15:16:02] But I personally can't +2 in that repo, that's also something that should be fixed eventually [15:16:03] https://gerrit.wikimedia.org/r/#/c/413736/ <-- if you +1 I'll appove [15:16:05] (03CR) 10jerkins-bot: [V: 04-1] phpunit-coverage-publish job on Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413737 (owner: 10Hashar) [15:18:43] stephanebisson: done! [15:18:54] thanks [15:18:57] see now jenkins can merge there [15:20:19] (03PS2) 10Hashar: phpunit-coverage-publish job on Docker [integration/config] - 10https://gerrit.wikimedia.org/r/413737 [15:22:57] (03CR) 10Hashar: "It is too late on a Friday to give it a try :]" (034 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/413737 (owner: 10Hashar) [15:54:34] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Proton, 10Readers-Web-Backlog, and 2 others: Set up Jenkins for chromium-render repository - https://phabricator.wikimedia.org/T179552#3996491 (10hashar) That is my token of appreciation whenever a task get fixed via team work :-} I h... [15:58:00] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban): zuul-merger /var/log/git-daemon/syslog.log is not log rotated - https://phabricator.wikimedia.org/T188107#3996494 (10hashar) [15:58:41] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban): zuul-merger /var/log/git-daemon/syslog.log is not log rotated - https://phabricator.wikimedia.org/T188107#3996504 (10hashar) p:05Triage>03Low Low priority, it is only 230MBytes [16:01:07] 10Continuous-Integration-Config: fatal: remote error: access denied or repository not exported: /mediawiki/extensions/ReadingLists - https://phabricator.wikimedia.org/T187897#3989482 (10hashar) The git-daemon gave: Feb 21 15:16:43 contint1001 git-daemon[29526]: '/srv/zuul/git/mediawiki/extensions/ReadingLists'... [16:05:42] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban): fatal: remote error: access denied or repository not exported: /mediawiki/extensions/ReadingLists - https://phabricator.wikimedia.org/T187897#3996521 (10hashar) I guess we can try running the git-daemon with `--verbose` to get more i... [16:30:33] paladox: I came up with a solution to the external_plugin_deps.bzl issue with core plugins last night: https://gerrit-review.googlesource.com/c/plugins/gitiles/+/161891 [16:30:49] (the: I have to manually copy them up one level & merge multiple ones by hand issue) [16:33:17] PROBLEM - Puppet errors on deployment-logstash2 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [16:44:45] no_justification ah yep, i saw :). [16:45:23] So building our release would just be add the submodules, update plugins.bzl and external_plugin_deps.bzl and we're set! [16:45:32] (1 line, 1 line, 2 lines) [16:45:33] no_justification i will merge it [16:45:36] Muchhhhhh cleaner [16:45:36] :D [16:45:38] as there's 2 +2 [16:46:59] no_justification i've been having mobile network problems today. A simple restart of the phone fixed it lol. [16:48:00] no_justification done, merged. [16:48:01] :) [16:48:11] no_justification also did you see my its-base fix too? [16:48:25] we may need to cherry pick it for now. Until upstream approve it. [16:50:50] Not yet [16:51:19] no_justification ok. [16:51:52] no_justification all i did was use a java function to do it, then use soy, to change some things. When you have a chance that would be great. [16:52:14] !log created deployment-eventlogging05 to test eventlogging on Debian in deployment-prep [16:52:20] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:52:45] the idea for --^ is to replace in say a couple of weeks deployment-eventlogging03 (same size) that still runs on ubuntu [16:53:09] please let me know if the VM allocates too much space, in that case I'll destroy it [16:53:44] 10Continuous-Integration-Infrastructure, 10Operations-Software-Development: cumin 3.0.1-1 is broken on labs master - https://phabricator.wikimedia.org/T188112#3996695 (10hashar) [16:55:02] elukey: that will probably be fine :] [16:55:47] super thanks :) [16:56:04] elukey: there is a quota set and I think WMCS has way more capacity nowadays [16:59:14] I am off for the week-end! Have fun [17:02:24] PROBLEM - Puppet errors on deployment-eventlogging05 is CRITICAL: CRITICAL: 75.00% of data above the critical threshold [0.0] [17:10:45] GWTUI can be deprecated now as the only blocking feature in polygerrit was implemented [17:25:29] paladox: Ok, I've looked at your change to its-base. It looks ok....on the surface. But I'd love Luca's input as well [17:25:38] Yep [17:25:56] thanks [17:32:45] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10User-zeljkofilipin, 10Wikimedia-Incident: Create selenium-core-jessie daily Jenkins job - https://phabricator.wikimedia.org/T185011#3903268 (10zeljkofilipin) p:05Normal>03High [17:43:31] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10User-zeljkofilipin, 10Wikimedia-Incident: Create selenium-core-jessie daily Jenkins job - https://phabricator.wikimedia.org/T185011#3996931 (10zeljkofilipin) @daniel says there are at least a couple extensions that would log you in if you are c... [17:43:41] no_justification oh i see you doing those changes in 2.15 instead of 2.14 :) [17:43:54] Yeah, figured that was the best branch to target here [17:44:07] We *could* backport to stable-2.14, but I don't think it's super urgent [17:44:12] yep [18:27:26] I noticed that git-sync-upstream was broken on deployment-puppetmaster02 [18:28:00] there was a WIP commit that was causing some issues (it was related to analytics things so I've removed it, I think it has already been merged in main puppet) [18:28:30] now it keeps failing due to local changes not committed, I think due to the kafkatee module? [18:29:24] shall I just issue a git submodule update --init ? [18:31:06] paladox: Did it for all stable-2.15 extensions https://gerrit-review.googlesource.com/q/topic:external_plugin_deps :) [18:31:21] elukey: F'ing submodules [18:31:28] I doubt that'll fix it, but sure. [18:31:48] ahahha [18:32:09] there was an attempt to get it removed earlier on this week, might be it [18:32:20] I'll fix it [18:32:26] elukey: Removing submodules is *hard* [18:32:34] I just went through this in mediawiki-config [18:32:43] PROBLEM - Puppet errors on deployment-cache-upload04 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [18:33:29] elukey: Fixed. [18:33:52] no_justification: ah I was doing the submodules update :) [18:33:56] 2018-02-23T18:33:17Z INFO sync-upstream: Up-to-date: /var/lib/git/labs/private [18:33:59] all good [18:34:09] I did `git reset --hard` on the repo, then ran `git pull -r` [18:34:14] And it says everything's dandy [18:34:35] PROBLEM - App Server Main HTTP Response on deployment-mediawiki07 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Service Unavailable - string 'Wikipedia' not found on 'http://en.wikipedia.beta.wmflabs.org:80/wiki/Main_Page?debug=true' - 1342 bytes in 0.003 second response time [18:40:08] in this one puppet is failing since it can't find the package mcrouter [18:41:03] anyhow, wanted to update my cherry pick for https://gerrit.wikimedia.org/r/#/c/413362/ but I'll wait for Monday [18:43:03] (managed to do it, oook) [18:48:14] PROBLEM - App Server Main HTTP Response on deployment-mediawiki06 is CRITICAL: HTTP CRITICAL: HTTP/1.1 503 Service Unavailable - string 'Wikipedia' not found on 'http://en.wikipedia.beta.wmflabs.org:80/wiki/Main_Page?debug=true' - 1343 bytes in 0.013 second response time [18:49:42] PROBLEM - Free space - all mounts on integration-slave-jessie-1003 is CRITICAL: CRITICAL: integration.integration-slave-jessie-1003.diskspace._srv.byte_percentfree (<33.33%) [18:53:19] RECOVERY - App Server Main HTTP Response on deployment-mediawiki06 is OK: HTTP OK: HTTP/1.1 200 OK - 47202 bytes in 6.956 second response time [18:54:29] PROBLEM - Puppet errors on deployment-secureredirexperiment is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [19:02:15] no_justification :) [19:03:20] no_justification you got a comment on https://gerrit-review.googlesource.com/c/plugins/high-availability/+/162051 [19:03:21] heh [19:07:25] RECOVERY - Puppet errors on deployment-eventlogging05 is OK: OK: Less than 1.00% above the threshold [0.0] [19:07:49] paladox: Yeah I broke that one [19:07:52] Copy+paste fail [19:07:53] Amended [19:07:56] Reception123 heh [19:07:59] oh [19:08:01] wrong ping [19:08:08] no_justification yep [19:12:53] I wish they had a plugins meta-repo. Would make my life easier :p [19:16:24] no_justification what would the meta-repo do? [19:17:21] submodule subscriptions [19:17:26] I presume he means like we have for mw-extensions [19:17:26] So I could clone /all/ plugins easily :) [19:17:35] * no_justification wrote a 10 line python script instead [19:18:49] no_justification ah i see [19:18:50] hmm [19:20:02] no_justification maybe we could create a plugin that does it [19:20:11] as jgit supports submodules [19:23:42] Meh, not worth the effort :) [19:25:11] ok [19:26:26] Or just 17 lines of Python: https://phabricator.wikimedia.org/P6737 [19:26:27] :) [19:27:24] heh [19:49:00] no_justification https://gerrit.wikimedia.org/r/#/c/413785/ :) [19:56:13] no_justification https://gerrit-review.googlesource.com/admin/repos/plugins/server-log-viewer [19:56:18] display the logs through the ui [19:56:45] uh [19:56:52] the plugin dosent have a master branch lol [19:57:37] Heh just saw rhat [19:57:47] There's a bunch of abandoned/empty plugins [19:58:03] yeh [19:58:13] no_justification i think that someone requested that repo be created [19:58:22] but they didnt get around to implementing it [19:58:45] Bunch of em ya [19:59:55] yep [20:48:45] no_justification heh more users are trying out notedb now :). So hopefully they will catch all the bugs and someone will fix them :) [20:58:53] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban): releases1001 has full / partition - https://phabricator.wikimedia.org/T188080#3997544 (10demon) I'm redoing the tarball generation process anyway, might as well just disable the jobs too. [21:00:53] 10Continuous-Integration-Infrastructure, 10Operations-Software-Development: cumin 3.0.1-1 is broken on labs master - https://phabricator.wikimedia.org/T188112#3997552 (10Volans) This will be fixed by https://gerrit.wikimedia.org/r/c/412894/ , that is pending the full release of Cumin 3.0.1 in prod that is wait... [21:08:23] PROBLEM - Puppet errors on deployment-mx02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:08:57] (03CR) 10Legoktm: "\o/" (033 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/413737 (owner: 10Hashar) [21:39:42] 10MediaWiki-Codesniffer: Add sniff to detect class_exists( 'string' ) - https://phabricator.wikimedia.org/T188144#3997673 (10Reedy) [21:51:07] legoktm: thanks for the review on the phpunit-coverage-publish job :] I will take care of that on monday and trigger all repos that uses it :D [21:51:15] legoktm: and yeah +1 on php7 :] [21:51:26] :)) [21:52:22] legoktm: I will migrate more of them to Docker containers then work on migrating the mediawiki phpunit jobs. I found a way to spawn mysql :] [21:53:04] anyway sleep time! :] [21:54:13] sweet [21:54:28] hasharAway: I'm going to move us a step backwards, and start running php70 jobs on nodepool [21:54:30] oops [21:54:42] hopefully he didn't see that ;) [22:19:37] PROBLEM - Free space - all mounts on integration-slave-jessie-1002 is CRITICAL: CRITICAL: integration.integration-slave-jessie-1002.diskspace._mnt.byte_percentfree (No valid datapoints found)integration.integration-slave-jessie-1002.diskspace._srv.byte_percentfree (<40.00%) [22:43:16] 10MediaWiki-Codesniffer: Add sniff to detect class_exists( 'string' ) - https://phabricator.wikimedia.org/T188144#3997961 (10Legoktm) [22:49:40] PROBLEM - Free space - all mounts on integration-slave-jessie-1002 is CRITICAL: CRITICAL: integration.integration-slave-jessie-1002.diskspace._mnt.byte_percentfree (No valid datapoints found)integration.integration-slave-jessie-1002.diskspace._srv.byte_percentfree (<10.00%) [22:54:59] 10Phabricator (Upstream), 10Upstream: "Move on workboard" option to let you move to more than one workboard column - https://phabricator.wikimedia.org/T188155#3998018 (10MarcoAurelio) [22:56:33] RECOVERY - Mediawiki Error Rate on graphite-labs is OK: OK: Less than 1.00% above the threshold [1.0] [23:19:42] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-CentralAuth: Add indices for local_user_id and global_user_id in Beta - https://phabricator.wikimedia.org/T148239#3998131 (10TBolliger) [23:22:20] 10Release-Engineering-Team (Kanban), 10Phlogiston: Phlogiston reports don't have new data since mid-February - https://phabricator.wikimedia.org/T188149#3998135 (10JAufrecht) a:05JAufrecht>03mmodell A preliminary inspection suggests that the dumps are still coming, still fresh, but missing all edges (aka w... [23:22:41] 10Phabricator, 10Phlogiston: Phlogiston reports don't have new data since mid-February - https://phabricator.wikimedia.org/T188149#3998138 (10JAufrecht) [23:23:04] 10Phabricator, 10Release-Engineering-Team (Kanban), 10Phlogiston: Phlogiston reports don't have new data since mid-February - https://phabricator.wikimedia.org/T188149#3997846 (10JAufrecht) [23:23:49] 10Phabricator, 10Release-Engineering-Team (Kanban), 10Phlogiston: Phlogiston reports don't have new data since mid-February - https://phabricator.wikimedia.org/T188149#3998146 (10mmodell) There was a recent schema change related to edges. [23:24:57] Project beta-update-databases-eqiad build #23710: 04FAILURE in 4 min 57 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/23710/ [23:27:06] 10Phabricator, 10Release-Engineering-Team (Kanban), 10Phlogiston: Phlogiston reports don't have new data since mid-February - https://phabricator.wikimedia.org/T188149#3998154 (10JAufrecht) I also checked in new code on Feb 15, which is a superficial clue. I will continue investigating. Do you think the sc... [23:46:24] you can link to comment messages in pg heh [23:49:36] 10Phabricator, 10Zero, 10MW-1.31-release-notes (WMF-deploy-2018-02-06 (1.31.0-wmf.20)): Phab file uploads are blocked for inactive Zero IP ranges - https://phabricator.wikimedia.org/T173537#3998242 (10Mholloway) 05Open>03Resolved a:03Mholloway