[00:20:11] maintenance-disconnect-full-disks build 51685 integration-slave-docker-1037: OFFLINE due to disk space [00:45:13] maintenance-disconnect-full-disks build 51690 integration-slave-docker-1037: OFFLINE due to disk space [01:10:12] maintenance-disconnect-full-disks build 51695 integration-slave-docker-1037: OFFLINE due to disk space [01:20:34] Project beta-update-databases-eqiad build #32163: 04FAILURE in 34 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32163/ [01:35:11] maintenance-disconnect-full-disks build 51700 integration-slave-docker-1037: OFFLINE due to disk space [02:00:12] maintenance-disconnect-full-disks build 51705 integration-slave-docker-1037: OFFLINE due to disk space [02:21:20] Yippee, build fixed! [02:21:21] Project beta-update-databases-eqiad build #32164: 09FIXED in 1 min 20 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32164/ [02:25:11] maintenance-disconnect-full-disks build 51710 integration-slave-docker-1037: OFFLINE due to disk space [02:50:12] maintenance-disconnect-full-disks build 51715 integration-slave-docker-1037: OFFLINE due to disk space [03:15:19] maintenance-disconnect-full-disks build 51720 integration-slave-docker-1037: OFFLINE due to disk space [03:40:12] maintenance-disconnect-full-disks build 51725 integration-slave-docker-1037: OFFLINE due to disk space [04:05:12] maintenance-disconnect-full-disks build 51730 integration-slave-docker-1037: OFFLINE due to disk space [04:30:12] maintenance-disconnect-full-disks build 51735 integration-slave-docker-1037: OFFLINE due to disk space [04:55:11] maintenance-disconnect-full-disks build 51740 integration-slave-docker-1037: OFFLINE due to disk space [05:20:14] maintenance-disconnect-full-disks build 51745 integration-slave-docker-1037: OFFLINE due to disk space [05:45:12] maintenance-disconnect-full-disks build 51750 integration-slave-docker-1037: OFFLINE due to disk space [06:03:32] (03CR) 1020after4: [C: 03+1] "I don't use `git review` but I suppose it's fine if you do ;)" [tools/scap] - 10https://gerrit.wikimedia.org/r/494068 (https://phabricator.wikimedia.org/T191373) (owner: 10Hashar) [06:05:48] Project beta-scap-eqiad build #240065: 04FAILURE in 1.3 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240065/ [06:10:11] maintenance-disconnect-full-disks build 51755 integration-slave-docker-1037: OFFLINE due to disk space [06:24:43] Yippee, build fixed! [06:24:43] Project beta-scap-eqiad build #240066: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240066/ [06:35:12] maintenance-disconnect-full-disks build 51760 integration-slave-docker-1037: OFFLINE due to disk space [07:00:11] maintenance-disconnect-full-disks build 51765 integration-slave-docker-1037: OFFLINE due to disk space [07:25:12] maintenance-disconnect-full-disks build 51770 integration-slave-docker-1037: OFFLINE due to disk space [07:36:15] (03PS1) 10Legoktm: Revert "Revert "Add dependencies for CheckUser and CentralAuth"" [integration/config] - 10https://gerrit.wikimedia.org/r/494167 [07:36:23] (03CR) 10Legoktm: [C: 03+2] Revert "Revert "Add dependencies for CheckUser and CentralAuth"" [integration/config] - 10https://gerrit.wikimedia.org/r/494167 (owner: 10Legoktm) [07:37:52] (03Merged) 10jenkins-bot: Revert "Revert "Add dependencies for CheckUser and CentralAuth"" [integration/config] - 10https://gerrit.wikimedia.org/r/494167 (owner: 10Legoktm) [07:38:12] !log deploying https://gerrit.wikimedia.org/r/494167 [07:38:13] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:50:12] maintenance-disconnect-full-disks build 51775 integration-slave-docker-1037: OFFLINE due to disk space [08:15:12] maintenance-disconnect-full-disks build 51780 integration-slave-docker-1037: OFFLINE due to disk space [08:40:12] maintenance-disconnect-full-disks build 51785 integration-slave-docker-1037: OFFLINE due to disk space [09:05:14] maintenance-disconnect-full-disks build 51790 integration-slave-docker-1037: OFFLINE due to disk space [09:28:12] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap: scap in job beta-scap-eqiad has random failures (eval.php cannot find group 50062 or returns None) - https://phabricator.wikimedia.org/T217539 (10hashar) [09:29:50] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap: scap in job beta-scap-eqiad has random failures (eval.php cannot find group 50062 or returns None) - https://phabricator.wikimedia.org/T217539 (10hashar) [09:30:12] maintenance-disconnect-full-disks build 51795 integration-slave-docker-1037: OFFLINE due to disk space [09:44:51] 10Scap, 10Operations, 10Patch-For-Review, 10User-ArielGlenn, 10User-Joe: Make scap and opcache work consistently together - https://phabricator.wikimedia.org/T211964 (10Joe) 05Open→03Resolved [09:55:13] maintenance-disconnect-full-disks build 51800 integration-slave-docker-1037: OFFLINE due to disk space [09:59:52] !log cleaned docker images on integration-slave-docker-1037 [09:59:53] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:22:08] 10Continuous-Integration-Infrastructure, 10Cleanup, 10Repository-Admins, 10Wikibase-Quality, and 4 others: Archive the WikibaseQualityExternalValidation extension - https://phabricator.wikimedia.org/T204490 (10Lucas_Werkmeister_WMDE) Great, thanks everyone! [10:29:47] 10Release-Engineering-Team (Kanban), 10Release Pipeline: Experiment with continuous deployment using Blubberoid - https://phabricator.wikimedia.org/T214158 (10LarsWirzenius) My understanding is that this is missing the token for Jenkins to be able to push a deployment to K8s. [10:45:55] 10Gerrit, 10Phabricator, 10Release-Engineering-Team (Backlog): Move scap-vagrant to Gerrit - https://phabricator.wikimedia.org/T216483 (10Dzahn) added Tyler per git history of the mentioned repo [10:46:06] (03PS2) 10Hashar: Support to clone repositories in parallel [integration/quibble] - 10https://gerrit.wikimedia.org/r/493764 (https://phabricator.wikimedia.org/T211701) [10:49:44] 10Continuous-Integration-Config, 10MediaWiki-Core-Testing: Update selenium-daily-beta-* jobs to node10/npm6 - https://phabricator.wikimedia.org/T217545 (10hashar) [10:50:45] 10Continuous-Integration-Config, 10MediaWiki-Core-Testing: Update selenium-daily-beta-* jobs to node10/npm6 - https://phabricator.wikimedia.org/T217545 (10hashar) [10:50:59] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10MediaWiki-extensions-Newsletter, 10MediaWiki-extensions-ORES, and 5 others: Jenkins jobs for npm-test fail on project with deps on node-gyp which requires python2.7 - https://phabricator.wikimedia.org/T213944 (10hashar) Filled... [10:56:02] holy hell [10:59:42] so https://integration.wikimedia.org/ci/job/selenium-daily-beta-ORES/174/console fails due to lack of a full python [10:59:48] it uses v [10:59:49] docker-registry.wikimedia.org/releng/npm-browser-test:0.3.1 [10:59:56] https://integration.wikimedia.org/ci/view/Selenium/job/selenium-daily-beta-MediaWiki/196/console does not have the issue [11:00:04] but it uses docker-registry.wikimedia.org/releng/npm-browser-test:0.4.0 [11:00:13] though in the containers changelog I can't see the python being installed bha [11:00:54] and indeed it has python2.7-minimal bah [11:01:40] or that is due to different versions of fibers [11:02:37] OH [11:02:53] found it :D [11:03:22] the ORES job only uses mediawiki/extensions/ORES [11:03:34] which comes with a different version of fibers than the one used in mediawiki/core [11:03:35] magic [11:04:33] zeljkof: oh also, we would have to look at upgtading wdio from 4.x to 5.x :/ [11:05:12] hashar: yeah :/ https://phabricator.wikimedia.org/T213268 [11:06:02] anyway [11:06:09] the jobs gotta be switched to node10/npm6 anyway [11:06:23] and the container ships a full python2.7 so that would fix it [11:07:00] 10Beta-Cluster-Infrastructure, 10Performance-Team: Move XHGui from tungsten to webperf-002 - https://phabricator.wikimedia.org/T180761 (10Dzahn) >>! In T180761#4871650, @Krinkle wrote: > Duplicate declaration: Class[Httpd] is already declared in file /etc/puppet/modules/role/manifests/webperf/profiling_tools.p... [11:20:54] (03PS1) 10Hashar: Default selenium-daily-beta-{project} to node10/npm6 [integration/config] - 10https://gerrit.wikimedia.org/r/494199 (https://phabricator.wikimedia.org/T213944) [11:21:04] zeljkof: ^^^ should be a noop in jjb ;) [11:21:18] that changes the default container used by selenium-daily-* jobs ;) [11:22:27] hashar: looking... [11:24:30] (03CR) 10Zfilipin: [C: 03+2] Default selenium-daily-beta-{project} to node10/npm6 [integration/config] - 10https://gerrit.wikimedia.org/r/494199 (https://phabricator.wikimedia.org/T213944) (owner: 10Hashar) [11:24:37] ;) [11:24:48] hashar: +2-ed it, jenkins says no change anyway :) [11:25:12] yeah that is the point ;) [11:25:13] and [11:25:18] I miss chromedriver ;( [11:26:37] releng/npm-browser-test has an entrypoint that spawns Xvfb and Chromedriver [11:26:58] but releng/node10-test-browser does not :( [11:27:10] fun times [11:27:44] (03Merged) 10jenkins-bot: Default selenium-daily-beta-{project} to node10/npm6 [integration/config] - 10https://gerrit.wikimedia.org/r/494199 (https://phabricator.wikimedia.org/T213944) (owner: 10Hashar) [11:29:36] and [11:29:48] if I rebuild the container that would get chromium 0.72 which is broken :/ [11:29:52] f* thx [11:38:52] will look at it post lunch [11:38:55] *scream* [11:40:43] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Consider and evaluate possible new CI tooling - https://phabricator.wikimedia.org/T217325 (10LarsWirzenius) a:03LarsWirzenius [11:45:47] Project beta-scap-eqiad build #240095: 04FAILURE in 1.3 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240095/ [12:04:31] Yippee, build fixed! [12:04:32] Project beta-scap-eqiad build #240096: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240096/ [12:05:52] Project beta-scap-eqiad build #240097: 04FAILURE in 1.6 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240097/ [12:09:30] (03PS3) 10Thiemo Kreuz (WMDE): Streamline PHPDoc comment parsing in UnusedUseStatement sniff [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/487033 [12:14:04] (03CR) 10Thiemo Kreuz (WMDE): "I introduced a new test case for the situation when a "use" clause is no necessary (an "unused use") because the class only appears as a f" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/487033 (owner: 10Thiemo Kreuz (WMDE)) [12:24:01] 10Continuous-Integration-Config, 10MediaWiki-Core-Testing, 10Patch-For-Review, 10User-zeljkofilipin: Update selenium-daily-beta-* jobs to node10/npm6 - https://phabricator.wikimedia.org/T217545 (10zeljkofilipin) [12:24:36] Yippee, build fixed! [12:24:37] Project beta-scap-eqiad build #240098: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240098/ [12:25:08] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Consider and evaluate possible new CI tooling - https://phabricator.wikimedia.org/T217325 (10zeljkofilipin) [12:55:08] 10Beta-Cluster-Infrastructure, 10Performance-Team: Move XHGui from tungsten to webperf-002 - https://phabricator.wikimedia.org/T180761 (10Dzahn) [13:00:55] (03PS4) 10Hashar: Create selenium-daily-beta-ORES Jenkins job [integration/config] - 10https://gerrit.wikimedia.org/r/460517 (https://phabricator.wikimedia.org/T188742) (owner: 10Zfilipin) [13:01:53] (03CR) 10Hashar: [C: 04-1] "Rebased to use docker-registry.wikimedia.org/releng/node10-test-browser:0.3.3 but that container ENTRYPOINT is wrong (it does not spawn X" [integration/config] - 10https://gerrit.wikimedia.org/r/460517 (https://phabricator.wikimedia.org/T188742) (owner: 10Zfilipin) [13:49:19] !log GitHub: deleted wikimedia/mediawiki-extensions-UploadLocal | T213011 [13:49:21] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [13:49:22] T213011: Archive the UploadLocal extension - https://phabricator.wikimedia.org/T213011 [14:52:41] Hello, have you discussed the Pywikibot issues on the meeting last week? [14:54:11] hm, I think I asked this before: is there an official way to test a config change on beta and then wait for swat to deploy it? [14:54:39] i think there will be a merge -> deploy gap, in which mediawiki-config HEAD would be out of sync with what is in production [14:54:44] but i don't know how to test in beta without merging [14:56:12] Depends on the config.. [14:56:34] If it's a -labs file, you can just merge and deploy from the prod deployment host no problem [14:58:19] its CommonSettings.php [14:58:29] it should be no-op in both places [14:58:53] https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/494231/1/wmf-config/CommonSettings.php [15:00:58] I'm going to merge and test in beta, and if all goes well perhaps I'll just sync file now instead of waiting for swat [15:14:06] !log cherry picking https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/492390/ to deployment-puppetmaster03 [15:14:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:16:15] hm, beta cluster is not readonly, right? [15:16:19] can someone update the channel topic? [15:18:07] did someone fix the db? [15:18:58] ya last week [15:19:13] https://phabricator.wikimedia.org/T216067#4981998 [15:21:06] my change looks good in beta, syncing in prod [15:21:17] 10Scap: Newline in scap sync-file message: no notification, no SAL entry - https://phabricator.wikimedia.org/T217558 (10Gilles) [15:31:01] Reedy what about this one? https://gerrit.wikimedia.org/r/#/c/operations/mediawiki-config/+/490668/ [15:31:08] it also shoudl only enable the new log channel in beta [15:31:14] but touches more files [15:31:43] should I just wait for SWAT for that one? [15:35:37] (03PS1) 10Hashar: docker: pin chromium to version 71 [integration/config] - 10https://gerrit.wikimedia.org/r/494243 (https://phabricator.wikimedia.org/T216702) [15:38:28] Or anyone willing to help Pywikibot with Unbreak now! issue T186208? The amount of pull requests to our GitHub repository are increasing and we can not hide them, because it is the only way how to run CI tests for Pywikibot currently. Is there anyone whou knows how to set Pywikibot repo for Wikimedia CI tests? [15:38:28] T186208: Magul's quick tests doesn't run anymore - https://phabricator.wikimedia.org/T186208 [15:40:31] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Pywikibot, 10Pywikibot-tests: Magul's quick tests doesn't run anymore - https://phabricator.wikimedia.org/T186208 (10Dvorapa) [16:01:11] Project beta-scap-eqiad build #240117: 04FAILURE in 3.3 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240117/ [16:04:58] (03PS1) 10Gilles: Add CI for new performance/asoranking repo [integration/config] - 10https://gerrit.wikimedia.org/r/494259 (https://phabricator.wikimedia.org/T211750) [16:10:55] (03PS2) 10Hashar: docker: pin chromium to version 71 [integration/config] - 10https://gerrit.wikimedia.org/r/494243 (https://phabricator.wikimedia.org/T216702) [16:11:21] (03PS1) 10Thcipriani: Service Pipeline: correct imagename [integration/config] - 10https://gerrit.wikimedia.org/r/494261 [16:14:41] Yippee, build fixed! [16:14:42] Project beta-scap-eqiad build #240118: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240118/ [16:16:09] Project beta-scap-eqiad build #240119: 04FAILURE in 1.3 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240119/ [16:23:23] (03PS2) 10Gilles: Add CI for new performance/asoranking repo [integration/config] - 10https://gerrit.wikimedia.org/r/494259 (https://phabricator.wikimedia.org/T209857) [16:27:08] (03PS3) 10Hashar: docker: pin chromium to version 71 [integration/config] - 10https://gerrit.wikimedia.org/r/494243 (https://phabricator.wikimedia.org/T216702) [16:28:04] PROBLEM - Host integration-publishing02 is DOWN: CRITICAL - Host Unreachable (172.16.4.5) [16:28:26] PROBLEM - Host integration-slave-docker-1046 is DOWN: CRITICAL - Host Unreachable (172.16.1.115) [16:30:41] (03CR) 10Hashar: [C: 03+2] "@Krinkle Chromium 72 as shipped by Debian is broken ( T216702 ). It crashed when using --headless and still crashes when using --remote-de" [integration/config] - 10https://gerrit.wikimedia.org/r/494243 (https://phabricator.wikimedia.org/T216702) (owner: 10Hashar) [16:32:05] (03CR) 10Krinkle: "I don't recall an issue with chrome headless. Context?" [integration/config] - 10https://gerrit.wikimedia.org/r/494243 (https://phabricator.wikimedia.org/T216702) (owner: 10Hashar) [16:32:25] Yippee, build fixed! [16:32:25] Project beta-scap-eqiad build #240120: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240120/ [16:32:39] (03Merged) 10jenkins-bot: docker: pin chromium to version 71 [integration/config] - 10https://gerrit.wikimedia.org/r/494243 (https://phabricator.wikimedia.org/T216702) (owner: 10Hashar) [16:33:56] 10Continuous-Integration-Config, 10Parsoid-PHP: Decide on which unit testing framework (PHPSpec or PHPUnit) to use for Parsoid in the long term - https://phabricator.wikimedia.org/T217458 (10ssastry) Alright, I'll make this decision since it seems we have been going round and round a bit. In terms of purely... [16:41:08] (03CR) 10Hashar: [C: 03+2] "I am referring to quibble-fresnel/Dockerfile.template comment https://gerrit.wikimedia.org/r/plugins/gitiles/integration/config/+/bc197e84" [integration/config] - 10https://gerrit.wikimedia.org/r/494243 (https://phabricator.wikimedia.org/T216702) (owner: 10Hashar) [16:43:00] greg-g: Dvorapa is bringing up that the link to the channel logs needs updated, when someone gets a chance can that be fixed. [16:44:08] 10Continuous-Integration-Config, 10MediaWiki-Core-Testing, 10Patch-For-Review, 10User-zeljkofilipin: Update selenium-daily-beta-* jobs to node10/npm6 - https://phabricator.wikimedia.org/T217545 (10hashar) The container `releng/node10-test-browser` does not come with the proper entry point (the one who had... [16:47:19] PROBLEM - Disk space on contint1001 is CRITICAL: DISK CRITICAL - free space: / 2655 MB (5% inode=63%) [16:47:26] Zppix: Where? Topic of this channel, you mean? [16:47:36] andre__: yes [16:48:06] This server could not prove that it is ur1.ca; its security certificate is from *.blendedperspectives.com. This may be caused by a misconfiguration or an attacker intercepting your connection. [16:48:10] That's not fun :) [16:48:35] Reedy: it also goes to a 404 [16:49:06] yeah, ur1.ca is probably dead [16:49:21] RECOVERY - Disk space on contint1001 is OK: DISK OK [16:50:05] what's the log url? :) [16:50:15] let me look [16:50:44] found it [16:50:47] http://wm-bot.wmflabs.org/browser/index.php?display=%23wikimedia-releng [16:50:50] oh :P [17:08:25] (03CR) 10Zfilipin: [C: 03+2] Add Abijeet Patro to the CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/493374 (owner: 10Nikerabbit) [17:09:58] (03Merged) 10jenkins-bot: Add Abijeet Patro to the CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/493374 (owner: 10Nikerabbit) [17:22:50] (03CR) 10Krinkle: "That uses Chromium 71, not 72. The reason it was broken is because I created a simpler fork of the quibble image without Firefox-ESR and d" [integration/config] - 10https://gerrit.wikimedia.org/r/494243 (https://phabricator.wikimedia.org/T216702) (owner: 10Hashar) [17:30:49] Project beta-scap-eqiad build #240126: 04FAILURE in 1.8 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240126/ [17:34:42] 10Continuous-Integration-Config, 10Jade, 10Patch-For-Review, 10Scoring-platform-team (Current): Rename JADE->Jade in continuous integration - https://phabricator.wikimedia.org/T212181 (10Ladsgroup) There's nothing to review here. It's blocked on T212181 [17:36:09] !log Build Docker containers for chromium=v71 pin ( https://gerrit.wikimedia.org/r/#/c/integration/config/+/494243/ ) [17:36:10] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:41:21] 10Gerrit, 10Proton, 10Reading-Infrastructure-Team-Backlog, 10Core Platform Team Kanban (Done with CPT), and 2 others: Rename chromium-render gerrit project to Proton - https://phabricator.wikimedia.org/T211375 (10CCicalese_WMF) [17:44:28] Yippee, build fixed! [17:44:28] Project beta-scap-eqiad build #240127: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240127/ [17:45:15] (03PS1) 10Hashar: wikidata-query-rdf: upgrade Chromium from v70 to v71 [integration/config] - 10https://gerrit.wikimedia.org/r/494274 (https://phabricator.wikimedia.org/T216702) [17:45:49] Project beta-scap-eqiad build #240128: 04FAILURE in 1.6 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240128/ [17:47:24] (03CR) 10Hashar: "Guillaume, Stas, that is a routine maintenance upgrade. wikidata-query-rdf does run gui tests which use npm/chromium. So I though that Ch" [integration/config] - 10https://gerrit.wikimedia.org/r/494274 (https://phabricator.wikimedia.org/T216702) (owner: 10Hashar) [17:48:41] (03CR) 10Smalyshev: [C: 03+1] wikidata-query-rdf: upgrade Chromium from v70 to v71 [integration/config] - 10https://gerrit.wikimedia.org/r/494274 (https://phabricator.wikimedia.org/T216702) (owner: 10Hashar) [17:51:26] Dvorapa: sorry I have failed to reply earlier. I will be the one in charge of helping on this task T186208 [17:51:27] T186208: Magul's quick tests doesn't run anymore - https://phabricator.wikimedia.org/T186208 [17:56:18] PROBLEM - Puppet errors on integration-slave-docker-1021 is CRITICAL: CRITICAL: 10.00% of data above the critical threshold [3.0] [18:00:24] (03CR) 10Thcipriani: [C: 03+2] "this is deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/494261 (owner: 10Thcipriani) [18:00:37] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Pywikibot, 10Pywikibot-tests: Magul's quick tests doesn't run anymore - https://phabricator.wikimedia.org/T186208 (10hashar) a:03hashar Hello, this got raised to #release-engineering-team meeting last Monday, thought I was busy with... [18:01:11] (03CR) 10Gehel: [C: 03+1] "LGTM" [integration/config] - 10https://gerrit.wikimedia.org/r/494274 (https://phabricator.wikimedia.org/T216702) (owner: 10Hashar) [18:02:26] (03Merged) 10jenkins-bot: Service Pipeline: correct imagename [integration/config] - 10https://gerrit.wikimedia.org/r/494261 (owner: 10Thcipriani) [18:04:43] Yippee, build fixed! [18:04:43] Project beta-scap-eqiad build #240129: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240129/ [18:06:55] twentyafterfour: thcipriani: also I forgot. beta-scap-eqiad above is randomly failing. There are two causes https://phabricator.wikimedia.org/T217539 [18:07:37] "Call to mwscript eval.php returned: None" which is most probably a glitch in how scap handles returns codes/exception [18:07:45] "Call to mwscript eval.php stderr: groups: cannot find name for group ID 50062" [18:07:57] that one, maybe I should have filled a different task for it [18:08:05] actually doing so [18:08:18] I think we should just skip calling that entirely when running on beta [18:08:37] thcipriani and I discussed it last week but then promptly forgot I think [18:09:26] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap: scap in job beta-scap-eqiad has random failures: Call to mwscript eval.php returned: None - https://phabricator.wikimedia.org/T217587 (10hashar) [18:09:56] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap: scap in job beta-scap-eqiad has random failures: eval.php cannot find group 50062 - https://phabricator.wikimedia.org/T217539 (10hashar) [18:10:14] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap: scap in job beta-scap-eqiad has random failures: Call to mwscript eval.php returned: None - https://phabricator.wikimedia.org/T217587 (10hashar) [18:10:28] I have split the tasks [18:12:58] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap: scap in job beta-scap-eqiad has random failures: Call to mwscript eval.php returned: None - https://phabricator.wikimedia.org/T217587 (10hashar) I am not quite sure what is happening here. It comes from `AbstractSync._check_fatals` w... [18:13:51] and I have brain dumped the few bits I looked at this morning [18:13:54] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap: scap in job beta-scap-eqiad has random failures: eval.php cannot find group 50062 - https://phabricator.wikimedia.org/T217539 (10hashar) I thought about a LDAP failure, I could not find any evidence in `deployment-deploy01` `/var/log... [18:15:14] dinner & [18:19:03] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap: scap in job beta-scap-eqiad has random failures: Call to mwscript eval.php returned: None - https://phabricator.wikimedia.org/T217587 (10mmodell) >>! In T217587#4999010, @hashar wrote: > I am not quite sure what is happening here. It... [18:20:43] Project beta-update-databases-eqiad build #32180: 04FAILURE in 43 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32180/ [18:22:19] groups: cannot find name for group ID 50120 hehe [18:23:45] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap: scap in job beta-scap-eqiad has random failures: eval.php cannot find group 50062 - https://phabricator.wikimedia.org/T217539 (10hashar) Ditto on https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32180/console `cou... [18:30:27] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap, 10cloud-services-team: scap in job beta-scap-eqiad has random failures: eval.php cannot find group 50062 - https://phabricator.wikimedia.org/T217539 (10hashar) Seems both LDAP server had troubles reporting metrics to Grafana. They... [18:32:12] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap, 10cloud-services-team: scap in job beta-scap-eqiad has random failures: eval.php cannot find group 50062 - https://phabricator.wikimedia.org/T217539 (10thcipriani) >>! In T217539#4999025, @hashar wrote: > I thought about a LDAP fai... [18:33:58] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap, 10cloud-services-team: scap in job beta-scap-eqiad has random failures: eval.php cannot find group 50062 - https://phabricator.wikimedia.org/T217539 (10bd808) Seems likely to be the same root cause as {T217280}. Should we merge the... [18:50:17] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Find more CI candidates, divide list based on very hard requirements - https://phabricator.wikimedia.org/T217593 (10LarsWirzenius) [18:51:49] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Implement "blubber building" CI task for GitLab-CI - https://phabricator.wikimedia.org/T217594 (10LarsWirzenius) [18:53:19] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Investigate Concourse-CI - https://phabricator.wikimedia.org/T217595 (10LarsWirzenius) [18:53:58] hasharDinner: ok, if any help needed, don't worry to ask anything! :) [19:11:59] Hello! [19:12:19] scap is failing when I want to deploy wdqs on deploy1001 [19:12:29] I get this: [19:12:35] https://www.irccloud.com/pastebin/Ultnkx82/ [19:13:15] I need some help with figuring out what is wrong [19:13:54] also codfw_dsh_targets seems to be present in scap.cfg [19:15:05] hrm [19:16:42] thcipriani: any idea on what could be causing this? [19:17:13] it looks like there's a space getting added somewhere.... [19:17:32] there was a new scap release recently, I'm digging [19:18:03] Ok [19:18:20] looking at the errors again, yes [19:19:58] onimisionipe: ugh, I think I know what caused this, your config is too readable. i.e. in scap.cfg "canary, codfw, codfw_internal, eqiad, eqiad_internal" [19:20:11] onimisionipe: could you try getting rid of those spaces? [19:20:29] i.e., "canary,codfw,codfw_internal,eqiad,eqiad_internal" [19:21:14] thcipriani: Ok [19:21:34] I thought too readable was great! :) [19:21:57] onimisionipe: in this case, you're allowed to blame scap! [19:22:02] ^ [19:22:17] unclear if this is a recent regression, or if this is something we've not allowed. [19:22:18] onimisionipe: ping me for review / merge [19:22:39] thcipriani: this config has not changed on our side for quite a long while [19:22:53] gehel: Ok [19:25:56] Yippee, build fixed! [19:25:56] Project beta-update-databases-eqiad build #32181: 09FIXED in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32181/ [19:26:59] gehel: https://gerrit.wikimedia.org/r/c/wikidata/query/deploy/+/494294 [19:28:34] onimisionipe: merged, try again [19:28:49] Ok [19:29:01] * thcipriani watches via scap deploy-log [19:30:16] yep [19:30:20] its seems to be ok now [19:31:32] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap, 10cloud-services-team: scap in job beta-scap-eqiad has random failures: eval.php cannot find group 50062 - https://phabricator.wikimedia.org/T217539 (10hashar) [19:31:56] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Scap, 10cloud-services-team: scap in job beta-scap-eqiad has random failures: eval.php cannot find group 50062 - https://phabricator.wikimedia.org/T217539 (10hashar) >>! In T217539#4999113, @bd808 wrote: > Seems likely to be the same roo... [19:32:45] ugh, I see what happened. [19:35:12] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Pywikibot, 10Pywikibot-tests: Magul's quick tests doesn't run anymore - https://phabricator.wikimedia.org/T186208 (10Dvorapa) >>! In T186208#4998959, @hashar wrote: > Hello, this got raised to #release-engineering-team meeting last Mon... [19:35:24] 10Release-Engineering-Team (Kanban), 10Scap, 10serviceops: Scap: server_groups regression - https://phabricator.wikimedia.org/T217597 (10thcipriani) [19:41:33] (03PS1) 10Thcipriani: Targets: strip spaces from service_groups [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) [19:42:50] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [19:51:19] RECOVERY - Puppet errors on integration-slave-docker-1021 is OK: OK: Less than 1.00% above the threshold [2.0] [20:00:16] (03PS1) 10Hashar: Pin selenium-daily-beta-* jobs to Chromium v71 [integration/config] - 10https://gerrit.wikimedia.org/r/494299 (https://phabricator.wikimedia.org/T216702) [20:00:48] (03CR) 10Hashar: [C: 03+2] Pin selenium-daily-beta-* jobs to Chromium v71 [integration/config] - 10https://gerrit.wikimedia.org/r/494299 (https://phabricator.wikimedia.org/T216702) (owner: 10Hashar) [20:03:40] (03Merged) 10jenkins-bot: Pin selenium-daily-beta-* jobs to Chromium v71 [integration/config] - 10https://gerrit.wikimedia.org/r/494299 (https://phabricator.wikimedia.org/T216702) (owner: 10Hashar) [20:05:12] (03PS2) 1020after4: Targets: strip spaces from service_groups [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [20:05:29] (03CR) 1020after4: [C: 03+2] "Nice, thanks for adding a test! :)" [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [20:06:36] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [20:06:39] (03CR) 10jerkins-bot: [V: 04-1] Targets: strip spaces from service_groups [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [20:08:10] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Pywikibot, 10Pywikibot-tests: Magul's quick tests doesn't run anymore - https://phabricator.wikimedia.org/T186208 (10Dvorapa) Currently running: | When | What | Where | status | | on every patchset and on +2 | nose, nose32, style and d... [20:08:14] 10Scap, 10WorkType-NewFunctionality: create a scap3 command to bootstrap a new deployment repo - https://phabricator.wikimedia.org/T118760 (10mmodell) 05Open→03Declined I don't really think this is useful after all. [20:08:16] 10Release-Engineering-Team (Backlog), 10Scap, 10Documentation: Define a stable API for scap plugins - https://phabricator.wikimedia.org/T151470 (10mmodell) [20:10:30] 10Release-Engineering-Team (Backlog), 10MediaWiki-Release-Tools: Make make-wmf-branch able to branch extensions with replaced substring of the version of mediawiki being branched - https://phabricator.wikimedia.org/T51392 (10mmodell) 05Open→03Invalid This doesn't apply anymore since we have removed all sup... [20:10:32] 10Deployments, 10RelEng-Archive-FY201718-Q2, 10Release-Engineering-Team (Long-Lived-Branches), 10releng-201617-q1, 10Epic: Merge to deployed branches instead of cutting a new deployment branch every week. - https://phabricator.wikimedia.org/T89945 (10mmodell) [20:11:36] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [20:11:38] (03CR) 10jerkins-bot: [V: 04-1] Targets: strip spaces from service_groups [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [20:26:49] (03CR) 10Reedy: "Based on I22a402a1deee50dff54576f5fded211411d7d249 and hence https://integration.wikimedia.org/ci/job/quibble-vendor-mysql-hhvm-docker/378" [integration/config] - 10https://gerrit.wikimedia.org/r/494167 (owner: 10Legoktm) [20:27:49] (03CR) 10Reedy: "Oh, no. The echo patch needs merging into .19" [integration/config] - 10https://gerrit.wikimedia.org/r/494167 (owner: 10Legoktm) [20:34:09] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.33.0-wmf.19 deployment blockers - https://phabricator.wikimedia.org/T206673 (10Anomie) [20:45:14] 10Release-Engineering-Team, 10Growth-Team, 10MediaWiki-extensions-UserMerge, 10Notifications, 10Patch-For-Review: Various tests failing with Echo/UserMerge issues - https://phabricator.wikimedia.org/T217487 (10Reedy) [20:49:06] 10Deployments, 10MediaWiki-Debug-Logger, 10Performance-Team, 10User-Tgr: Capture PHP warnings with stacktraces in MediaWiki and save to logstash - https://phabricator.wikimedia.org/T45086 (10Krinkle) 05Open→03Resolved [20:49:14] 10Deployments, 10MediaWiki-Debug-Logger, 10Performance-Team, 10User-Tgr: Capture PHP warnings with stacktraces in MediaWiki and save to logstash - https://phabricator.wikimedia.org/T45086 (10Krinkle) [20:49:23] 10Deployments, 10MediaWiki-Debug-Logger, 10Performance-Team, 10Performance-Team-notice, 10User-Tgr: Capture PHP warnings with stacktraces in MediaWiki and save to logstash - https://phabricator.wikimedia.org/T45086 (10Krinkle) [21:05:30] 10Deployments, 10MediaWiki-Debug-Logger, 10Performance-Team, 10Performance-Team-notice, 10User-Tgr: Capture PHP warnings with stacktraces in MediaWiki and save to logstash - https://phabricator.wikimedia.org/T45086 (10Tgr) Thanks for pushing this through the finish line @Krinkle! [21:05:31] (03CR) 10Hashar: "Zeljko raised the need to report the proper url to Gerrit. We went with T217008 which is similar to this change. Might as well implement i" (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/490950 (https://phabricator.wikimedia.org/T215175) (owner: 10Thcipriani) [21:20:35] Project beta-update-databases-eqiad build #32183: 04FAILURE in 34 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32183/ [21:21:01] Project beta-scap-eqiad build #240147: 04FAILURE in 1.1 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240147/ [21:25:59] 10Release-Engineering-Team, 10Scap, 10Parsoid: Could not find config setting ` default_dsh_targets` - https://phabricator.wikimedia.org/T217604 (10Arlolra) T217597 looks related [21:32:04] Krinkle: I have just replied to your mail. sorry missed it earlier due to dinner etc [21:32:20] though I am going to bed in a minute or so [21:32:26] but poke me again tomorrow :) [21:33:07] 04 03 2019 21:32:26.466:ERROR [phantomjs.launcher]: libEGL warning: DRI2: failed to open swrast (search paths /usr/lib/x86_64-linux-gnu/dri:${ORIGIN}/dri:/usr/lib/dri) [21:33:07] 04 03 2019 21:32:26.467:ERROR [phantomjs.launcher]: libEGL warning: DRI2: failed to open swrast (search paths /usr/lib/x86_64-linux-gnu/dri:${ORIGIN}/dri:/usr/lib/dri) [21:33:08] 04 03 2019 21:32:26.537:ERROR [phantomjs.launcher]: process 164: D-Bus library appears to be incorrectly set up; failed to read machine uuid: Failed to open "/etc/machine-id": No such file or directory [21:33:11] See the manual page for dbus-uuidgen to correct this issue. [21:33:13] I hate computers [21:36:57] 10Release-Engineering-Team, 10Scap, 10Parsoid: Could not find config setting ` default_dsh_targets` - https://phabricator.wikimedia.org/T217604 (10Reedy) [21:37:01] Yippee, build fixed! [21:37:02] Project beta-scap-eqiad build #240148: 09FIXED in 12 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240148/ [21:37:03] 10Release-Engineering-Team (Kanban), 10Scap, 10serviceops, 10Patch-For-Review: Scap: server_groups regression - https://phabricator.wikimedia.org/T217597 (10Reedy) [21:37:17] 10Release-Engineering-Team (Kanban), 10Scap, 10serviceops, 10Patch-For-Review: Scap: server_groups regression - https://phabricator.wikimedia.org/T217597 (10Reedy) p:05Triage→03High [21:43:29] 10Release-Engineering-Team (Kanban), 10Scap, 10serviceops, 10Patch-For-Review: Scap: server_groups regression - https://phabricator.wikimedia.org/T217597 (10thcipriani) [21:50:19] (03PS1) 10Hashar: docker: sync node10-test-browser with npm-browser-test [integration/config] - 10https://gerrit.wikimedia.org/r/494364 (https://phabricator.wikimedia.org/T217545) [21:52:07] (03CR) 10Hashar: "That is to upgrade the selenium daily jobs (wdio based) to use node10/npm6 T217545 I don't think the container is used for anything besid" [integration/config] - 10https://gerrit.wikimedia.org/r/494364 (https://phabricator.wikimedia.org/T217545) (owner: 10Hashar) [22:05:12] 10Continuous-Integration-Config, 10Parsoid-PHP: Decide on which unit testing framework (PHPSpec or PHPUnit) to use for Parsoid in the long term - https://phabricator.wikimedia.org/T217458 (10hashar) >>! In T217458#4995885, @Tgr wrote: > Behat is, as you say, a Gherkin-based framework, so it's more of a replace... [22:19:02] Q [22:19:08] I have an EventBus extension change I would like to test in beta [22:19:16] it is a no-op in prod, the code isn 't configured to use there yet [22:19:25] for use* [22:19:47] 1. if i merge a patch in the wmf/1.33.0-wmf.19 branch in the extension [22:19:53] will the code automatically be updated in beta? [22:20:04] wmf branches are unused in beta [22:20:04] nop [22:20:09] 2. I don't need to deploy this change to production, do I need to SWAT it? [22:20:09] beta uses the master branches [22:20:11] master only [22:20:12] oh. great. [22:20:21] how can I tell when it is synced? [22:20:24] stick it behind a feature flag and conditionally enable it on beta [22:20:25] and if you merge it on master [22:20:28] i noticed that core chaanges comment on the patach [22:20:32] it will be included in the next wmf version [22:20:42] (03PS3) 10Thcipriani: Targets: strip spaces from service_groups [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) [22:20:50] if you dont want the code from master to ship on wmf, that requires a feature flag ;) [22:20:55] well [22:20:56] that way you can go to beta immediately and get to pick and choose when you want to use it in prod [22:21:00] ya it has feature flag already [22:21:02] what Krenair said ;] [22:21:14] its alreayd out [22:21:19] Yippee, build fixed! [22:21:19] Project beta-update-databases-eqiad build #32184: 09FIXED in 1 min 18 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32184/ [22:21:28] sounds like you just need to enable the feature flag in beta then ottomata [22:21:36] its on in beta [22:21:40] ok [22:21:42] i'm just waiating for my merge to master to be deployed there [22:21:47] so you're all set? [22:22:13] yes, just wondering how long I need to wait, or where I can look to find out when it will be auto deployed in beta [22:22:14] https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/EventBus/+/494305/ [22:22:17] is what i'm waiting for [22:22:24] ah [22:22:28] jenkins post-merge should do it IIRC [22:22:57] or did it become a cron-like thing [22:22:57] hm [22:23:04] will it comment on the patch like in mw-core? [22:23:15] hashar would know [22:23:34] I know there's some thing somewhere for jenkins to SSH into deployment-deploy hosts [22:23:42] and do stuff [22:23:59] so I'm pretty sure jenkins is responsible for doing this somehow, not sure if it was on a per-commit basis. been a while [22:25:26] OH, i've been checking on deployment-mediawiki-07 looking for my code change [22:25:27] its there now. [22:25:35] no comment on patch, so I guess that answers that question :) [22:25:58] AND ITS WORKING [22:25:59] THANK YOUUUUU [22:26:03] well [22:26:08] it might do other stuff and get there eventually [22:26:11] or it might never do it [22:26:16] aye [22:26:22] I did nothing [22:27:05] well i'm just happy it works [22:27:07] :) [22:27:27] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [22:30:39] woah [22:30:46] post merge console log doesn't look good ottomata [22:30:52] uh oh, no? [22:31:06] 22:27:37 Syncing... [22:31:06] 22:27:37 rsync: failed to set times on "/cache/.": Operation not permitted (1) [22:31:06] 22:27:42 rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1668) [generator=3.1.2] [22:31:07] 22:27:42 [22:31:09] 22:27:42 Done [22:31:13] hm [22:31:14] ? [22:31:24] wonder what impact that has [22:31:48] probably nothing that's going to be a problem for you [22:31:51] aye [22:32:05] someone probably wants to know about it though so hopefully they'll read the backlog [22:32:13] (03PS4) 10Thcipriani: Targets: strip spaces from service_groups [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) [22:33:43] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [22:35:36] (03CR) 1020after4: [C: 03+2] Targets: strip spaces from service_groups [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [22:36:45] (03CR) 10Hashar: "there are stest failures https://integration.wikimedia.org/ci/blue/organizations/jenkins/service-pipeline-test/detail/service-pipeline-tes" [tools/scap] - 10https://gerrit.wikimedia.org/r/494068 (https://phabricator.wikimedia.org/T191373) (owner: 10Hashar) [22:37:07] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [22:37:09] (03CR) 10jerkins-bot: [V: 04-1] Targets: strip spaces from service_groups [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [22:39:48] 10Deployments, 10MediaWiki-Debug-Logger, 10Performance-Team, 10Performance-Team-notice, 10User-Tgr: Capture PHP warnings with stacktraces in MediaWiki and save to logstash - https://phabricator.wikimedia.org/T45086 (10hashar) Nice! That concludes a multiple years project :-] [22:50:26] 10Continuous-Integration-Config, 10Parsoid-PHP: Decide on which unit testing framework (PHPSpec or PHPUnit) to use for Parsoid in the long term - https://phabricator.wikimedia.org/T217458 (10Tgr) [22:50:50] 10Continuous-Integration-Config, 10Parsoid-PHP: Decide on which unit testing framework (PHPSpec or PHPUnit) to use for Parsoid in the long term - https://phabricator.wikimedia.org/T217458 (10Tgr) 05Open→03Resolved a:03Tgr [22:53:35] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Pywikibot, 10Pywikibot-tests: Magul's quick tests doesn't run anymore - https://phabricator.wikimedia.org/T186208 (10hashar) @Dvorapa thank you so much for your replies. I guess I have all the context to jump in and analysis a proper s... [23:00:46] 10Continuous-Integration-Config, 10Parsoid-PHP: Decide on which unit testing framework (PHPSpec or PHPUnit) to use for Parsoid in the long term - https://phabricator.wikimedia.org/T217458 (10hashar) @tgr summarized with: > Per T217458#4998386, PHPUnit will be used for all new tests, and phpspec for converting... [23:05:35] (03CR) 10Reedy: Revert "Get rid of .gitreview" (031 comment) [tools/scap] - 10https://gerrit.wikimedia.org/r/494068 (https://phabricator.wikimedia.org/T191373) (owner: 10Hashar) [23:07:24] (03CR) 10Thcipriani: [C: 03+2] Targets: strip spaces from service_groups [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [23:08:32] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [23:08:34] (03Merged) 10jenkins-bot: Targets: strip spaces from service_groups [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [23:09:14] (03CR) 10jenkins-bot: Targets: strip spaces from service_groups [tools/scap] - 10https://gerrit.wikimedia.org/r/494296 (https://phabricator.wikimedia.org/T217597) (owner: 10Thcipriani) [23:10:34] (03PS2) 10Thcipriani: Revert "Get rid of .gitreview" [tools/scap] - 10https://gerrit.wikimedia.org/r/494068 (https://phabricator.wikimedia.org/T191373) (owner: 10Hashar) [23:11:41] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [tools/scap] - 10https://gerrit.wikimedia.org/r/494068 (https://phabricator.wikimedia.org/T191373) (owner: 10Hashar) [23:20:37] 10Release-Engineering-Team (Kanban), 10Scap, 10serviceops: Deploy scap 3.9.2-1 - https://phabricator.wikimedia.org/T217611 (10thcipriani) [23:55:52] Project beta-scap-eqiad build #240161: 04FAILURE in 1.1 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240161/