[00:01:20] 10Continuous-Integration-Infrastructure (Slipway), 10Release-Engineering-Team (Kanban), 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: Migrate wikimedia-fundraising-civicrm to a Docker container - https://phabricator.wikimedia.org/T210287 (10awight) Woohoo, nice work! I can verify that as a person w... [00:04:07] RECOVERY - English Wikipedia Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 47986 bytes in 1.131 second response time [00:04:44] (03PS3) 10Jforrester: layout: [mediawiki/services/mobileapps] Drop npm-docker [integration/config] - 10https://gerrit.wikimedia.org/r/516517 (owner: 10Mholloway) [00:04:46] (03PS2) 10Jforrester: [DNM] layout: [mediawiki/services/mobileapps/deploy] Archive [integration/config] - 10https://gerrit.wikimedia.org/r/516651 [00:04:48] (03PS1) 10Jforrester: jjb: Drop mobileapps-deploy-npm-node-6-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/516721 [00:05:10] RECOVERY - App Server Main HTTP Response on deployment-mediawiki-07 is OK: HTTP OK: HTTP/1.1 200 OK - 47322 bytes in 0.984 second response time [00:05:40] RECOVERY - App Server Main HTTP Response on deployment-mediawiki-09 is OK: HTTP OK: HTTP/1.1 200 OK - 47372 bytes in 0.855 second response time [00:06:19] RECOVERY - English Wikipedia Mobile Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 36472 bytes in 1.010 second response time [00:07:47] (03PS1) 10Jforrester: jjb: Drop unused service images [integration/config] - 10https://gerrit.wikimedia.org/r/516722 [00:08:20] (03CR) 10Jforrester: [C: 03+2] jjb: Drop unused service images [integration/config] - 10https://gerrit.wikimedia.org/r/516722 (owner: 10Jforrester) [00:09:44] 10Continuous-Integration-Config, 10TechCom: On CI, stop testing MediaWiki with php7.0 - https://phabricator.wikimedia.org/T225628 (10Krinkle) > run less jobs on CI. Having tests for all of hhvm, php 7.0, php 7.1 and php.7.2 use a bit of resources. I agree this is an issue, but from what I can see this isn't g... [00:10:34] (03Merged) 10jenkins-bot: jjb: Drop unused service images [integration/config] - 10https://gerrit.wikimedia.org/r/516722 (owner: 10Jforrester) [00:13:51] !log Dropped jjb jobs for (change-propagation|citoid|cxserver|graphoid)?(-deploy)-npm-node-6-docker [00:13:52] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:23:36] (03PS1) 10Brennen Bearnes: pipeline: Add node for pipeline's trigger jobs [integration/config] - 10https://gerrit.wikimedia.org/r/516723 (https://phabricator.wikimedia.org/T224069) [00:25:00] 10Continuous-Integration-Config, 10MediaWiki-Core-Testing, 10MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), 10User-Ladsgroup: Improve caching in CI tests - https://phabricator.wikimedia.org/T225496 (10Jdforrester-WMF) This seems to have had a ~20% speed improvement – picking two patches merged into MW-core eit... [00:29:03] 10Gerrit, 10Release-Engineering-Team, 10VPS-project-libraryupgrader: Re-enable use of Gerrit HTTP token to push patchsets - https://phabricator.wikimedia.org/T218750 (10Paladox) [00:35:51] (03CR) 10Brennen Bearnes: "This change is ready for review." [integration/config] - 10https://gerrit.wikimedia.org/r/516723 (https://phabricator.wikimedia.org/T224069) (owner: 10Brennen Bearnes) [00:47:00] (03PS1) 10Paladox: Merge remote-tracking branch 'upstream/v2.15.14' into wmf/stable-2.15 [software/gerrit] (wmf/stable-2.15) - 10https://gerrit.wikimedia.org/r/516727 [02:33:01] PROBLEM - Free space - all mounts on integration-trigger-01 is CRITICAL: CRITICAL: integration.integration-trigger-01.diskspace._srv.byte_percentfree (<10.00%) [02:33:50] (03PS1) 10Kosta Harlan: (wip) Add option for using Apache as server [integration/quibble] - 10https://gerrit.wikimedia.org/r/516729 (https://phabricator.wikimedia.org/T225218) [02:34:30] (03CR) 10jerkins-bot: [V: 04-1] (wip) Add option for using Apache as server [integration/quibble] - 10https://gerrit.wikimedia.org/r/516729 (https://phabricator.wikimedia.org/T225218) (owner: 10Kosta Harlan) [02:48:00] RECOVERY - Free space - all mounts on integration-trigger-01 is OK: OK: All targets OK [03:04:04] PROBLEM - Free space - all mounts on integration-trigger-01 is CRITICAL: CRITICAL: integration.integration-trigger-01.diskspace._srv.byte_percentfree (<20.00%) [03:13:59] RECOVERY - Free space - all mounts on integration-trigger-01 is OK: OK: All targets OK [03:35:02] PROBLEM - Free space - all mounts on integration-trigger-01 is CRITICAL: CRITICAL: integration.integration-trigger-01.diskspace._srv.byte_percentfree (<10.00%) [04:39:16] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<44.44%) [05:06:03] PROBLEM - Free space - all mounts on integration-trigger-01 is CRITICAL: CRITICAL: integration.integration-trigger-01.diskspace._srv.byte_percentfree (<20.00%) [05:16:02] RECOVERY - Free space - all mounts on integration-trigger-01 is OK: OK: All targets OK [06:55:55] (03PS1) 10Awight: Remove some extra characters [integration/config] - 10https://gerrit.wikimedia.org/r/516739 [06:59:18] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [07:19:09] 10Release-Engineering-Team-TODO, 10Operations, 10Release Pipeline, 10Core Platform Team Backlog (Watching / External), and 2 others: Migrate production services to kubernetes using the pipeline - https://phabricator.wikimedia.org/T198901 (10mobrovac) [07:21:52] 10MediaWiki-Releasing, 10Release-Engineering-Team, 10Operations, 10Parsoid: signatures were invalid: EXPKEYSIG 90E9F83F22250DD7 MediaWiki releases repository - https://phabricator.wikimedia.org/T225601 (10ArielGlenn) p:05Triage→03High [07:45:30] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO, 10Operations, 10SRE-Access-Requests: Request: add awight to contint-docker - https://phabricator.wikimedia.org/T223262 (10awight) 05Open→03Declined I'm having second thoughts about this request, because I'm no longer see that I'l... [08:50:17] 10MediaWiki-Releasing, 10Release-Engineering-Team, 10Operations, 10Parsoid: signatures were invalid: EXPKEYSIG 90E9F83F22250DD7 MediaWiki releases repository - https://phabricator.wikimedia.org/T225601 (10fgiunchedi) a:03fgiunchedi I'll be looking into renewing this key [09:24:50] 10Continuous-Integration-Infrastructure, 10Developer Productivity, 10Browser-Tests, 10Patch-For-Review, 10User-zeljkofilipin: Consider moving browser based tests (Selenium and QUnit) to a non-voting pipeline - https://phabricator.wikimedia.org/T225248 (10zeljkofilipin) >>! In T225248#5250699, @Krinkle wr... [09:31:59] PROBLEM - Free space - all mounts on integration-trigger-01 is CRITICAL: CRITICAL: integration.integration-trigger-01.diskspace._srv.byte_percentfree (<10.00%) [09:42:02] RECOVERY - Free space - all mounts on integration-trigger-01 is OK: OK: All targets OK [09:42:57] 10MediaWiki-Releasing, 10Release-Engineering-Team, 10Operations, 10Parsoid, 10Patch-For-Review: signatures were invalid: EXPKEYSIG 90E9F83F22250DD7 MediaWiki releases repository - https://phabricator.wikimedia.org/T225601 (10fgiunchedi) Ok this should be done now, the new... [09:43:37] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10MediaWiki-Configuration, 10Patch-For-Review, 10User-zeljkofilipin: Fatal error Cannot redeclare wmfLabsSettings() on Beta cluster wikis - https://phabricator.wikimedia.org/T224899 (10zeljkofilipin) >>! In T224899#5253379, @zeljkofilipin wrote... [09:46:41] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10MediaWiki-Configuration, 10Patch-For-Review, 10User-zeljkofilipin: Fatal error Cannot redeclare wmfLabsSettings() on Beta cluster wikis - https://phabricator.wikimedia.org/T224899 (10zeljkofilipin) So, the problem is that either `InitialiseSe... [09:48:02] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10MediaWiki-Configuration, 10Patch-For-Review, 10User-zeljkofilipin: Fatal error Cannot redeclare wmfLabsSettings() on Beta cluster wikis - https://phabricator.wikimedia.org/T224899 (10hashar) From https://logstash-beta.wmflabs.org/app/kibana... [09:48:38] 10MediaWiki-Releasing, 10Release-Engineering-Team, 10Operations, 10Parsoid, 10Patch-For-Review: signatures were invalid: EXPKEYSIG 90E9F83F22250DD7 MediaWiki releases repository - https://phabricator.wikimedia.org/T225601 (10fgiunchedi) Instructions at https://wikitech.wi... [09:54:49] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10MediaWiki-Configuration, 10Patch-For-Review, 10User-zeljkofilipin: Fatal error Cannot redeclare wmfLabsSettings() on Beta cluster wikis - https://phabricator.wikimedia.org/T224899 (10hashar) Tldr: someone would need to find a stacktrace or ad... [10:03:00] zeljkof: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/516754 ;D [10:17:01] hashar: /me thumbs up [11:10:06] 10Continuous-Integration-Config, 10MediaWiki-Core-Testing, 10MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), 10Patch-For-Review, 10User-Ladsgroup: Improve caching in CI tests - https://phabricator.wikimedia.org/T225496 (10Ladsgroup) >>! In T225496#5255602, @Jdforrester-WMF wrote: > This seems to have had a ~2... [11:21:44] 10Continuous-Integration-Config: Issue with inter-extension-dependencies in CI tests - https://phabricator.wikimedia.org/T224285 (10Osnard) Thanks for your explanations @hashar . I think we now have figured out how to do it. We set the version of ExtJSBase to be 1.31 in `master`. By this the CLI installer can ad... [11:21:50] 10Continuous-Integration-Config: Issue with inter-extension-dependencies in CI tests - https://phabricator.wikimedia.org/T224285 (10Osnard) 05Open→03Resolved [11:27:33] Bisecting: 1790 revisions left to test after this (roughly 11 steps) [11:27:36] * hashar whistles [11:28:01] trying to track a potential regression for HHVM tests duration [11:44:58] 10MediaWiki-Releasing, 10Release-Engineering-Team, 10Operations, 10Parsoid: signatures were invalid: EXPKEYSIG 90E9F83F22250DD7 MediaWiki releases repository - https://phabricator.wikimedia.org/T225601 (10Tkshamburg) Hi @fgiunchedi , thanks for creating the new key (now 10... [12:03:02] PROBLEM - Free space - all mounts on integration-trigger-01 is CRITICAL: CRITICAL: integration.integration-trigger-01.diskspace._srv.byte_percentfree (<10.00%) [12:18:03] RECOVERY - Free space - all mounts on integration-trigger-01 is OK: OK: All targets OK [12:19:29] omg [12:22:55] 10MediaWiki-Releasing, 10Release-Engineering-Team, 10Operations, 10Parsoid: signatures were invalid: EXPKEYSIG 90E9F83F22250DD7 MediaWiki releases repository - https://phabricator.wikimedia.org/T225601 (10fgiunchedi) >>! In T225601#5256399, @Tkshamburg wrote: > Hi @fgiunche... [12:32:36] 10MediaWiki-Releasing, 10Release-Engineering-Team, 10Operations, 10Parsoid: signatures were invalid: EXPKEYSIG 90E9F83F22250DD7 MediaWiki releases repository - https://phabricator.wikimedia.org/T225601 (10Tkshamburg) Everything is fine now, "apt update" shows no errors now.... [12:56:27] (03PS1) 10Hashar: Migrate wikimedia-fundraising-civicrm to Docker job [integration/config] - 10https://gerrit.wikimedia.org/r/516766 (https://phabricator.wikimedia.org/T210287) [12:56:42] (03CR) 10Hashar: [C: 03+2] Migrate wikimedia-fundraising-civicrm to Docker job [integration/config] - 10https://gerrit.wikimedia.org/r/516766 (https://phabricator.wikimedia.org/T210287) (owner: 10Hashar) [12:58:23] (03CR) 10Awight: Migrate wikimedia-fundraising-civicrm to Docker job (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/516766 (https://phabricator.wikimedia.org/T210287) (owner: 10Hashar) [12:58:33] (03CR) 10jerkins-bot: [V: 04-1] Migrate wikimedia-fundraising-civicrm to Docker job [integration/config] - 10https://gerrit.wikimedia.org/r/516766 (https://phabricator.wikimedia.org/T210287) (owner: 10Hashar) [12:59:27] (03PS2) 10Hashar: Migrate wikimedia-fundraising-civicrm to Docker job [integration/config] - 10https://gerrit.wikimedia.org/r/516766 (https://phabricator.wikimedia.org/T210287) [12:59:53] (03CR) 10Hashar: Migrate wikimedia-fundraising-civicrm to Docker job (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/516766 (https://phabricator.wikimedia.org/T210287) (owner: 10Hashar) [13:00:26] (03CR) 10Awight: "Major debt removal!" [integration/config] - 10https://gerrit.wikimedia.org/r/516766 (https://phabricator.wikimedia.org/T210287) (owner: 10Hashar) [13:00:34] (03CR) 10Awight: [C: 03+1] Migrate wikimedia-fundraising-civicrm to Docker job [integration/config] - 10https://gerrit.wikimedia.org/r/516766 (https://phabricator.wikimedia.org/T210287) (owner: 10Hashar) [13:01:59] awight: I will look at those mariadb warning messages about user mysql/nobody/root [13:02:04] but I think it is just spam [13:02:32] I think it goes ahead and runs as root, which is fine to do. Maybe we remove the "--user nobody"? [13:02:43] na there is no root privileges :) [13:03:33] (03CR) 10Hashar: [C: 03+2] Migrate wikimedia-fundraising-civicrm to Docker job [integration/config] - 10https://gerrit.wikimedia.org/r/516766 (https://phabricator.wikimedia.org/T210287) (owner: 10Hashar) [13:03:57] Ah interesting--okay so it runs as user "mysql" or whataver. [13:04:10] actually no, it runs as user "nobody" :] [13:05:43] (03Merged) 10jenkins-bot: Migrate wikimedia-fundraising-civicrm to Docker job [integration/config] - 10https://gerrit.wikimedia.org/r/516766 (https://phabricator.wikimedia.org/T210287) (owner: 10Hashar) [13:10:53] that logline seemed to state otherwise, but pretty sure it doesn't matter. > Ignoring user change to 'nobody' because the user was set to 'mysql' earlier on the command line [13:18:15] 10MediaWiki-Releasing, 10Release-Engineering-Team, 10Operations, 10Parsoid: signatures were invalid: EXPKEYSIG 90E9F83F22250DD7 MediaWiki releases repository - https://phabricator.wikimedia.org/T225601 (10fgiunchedi) 05Open→03Resolved No problem @Tkshamburg ! Thanks for... [13:18:30] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [13:20:49] 10Release-Engineering-Team-TODO: Database busted for CiviCRM tests? - https://phabricator.wikimedia.org/T205950 (10hashar) [13:20:52] 10Continuous-Integration-Infrastructure (Slipway), 10Release-Engineering-Team (Kanban), 10Wikimedia-Fundraising-CiviCRM, 10Patch-For-Review: Migrate wikimedia-fundraising-civicrm to a Docker container - https://phabricator.wikimedia.org/T210287 (10hashar) 05Open→03Resolved >>! In T210287#5255555, @awig... [13:21:54] 10Release-Engineering-Team-TODO: Database busted for CiviCRM tests? - https://phabricator.wikimedia.org/T205950 (10hashar) 05Open→03Resolved a:03hashar I have no idea what happened nor did I ever plan to investigate. The job is now using a Docker container that initialize and spawn MySQL on start ( T21028... [13:22:29] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Patch-For-Review: Drop php56 testing support - https://phabricator.wikimedia.org/T224906 (10hashar) [13:22:31] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Fundraising-Backlog, 10Patch-For-Review: Run wikimedia/fundraising/crm CI jobs on PHP7x, not PHP5x - https://phabricator.wikimedia.org/T223348 (10hashar) 05Open→03Resolved Switched to a Docker container which comes with php 7.0 ! [13:30:14] hashar: I started experimenting with quibble starting apache, and I have Apache serving up /workspace/src on port 9142 in Quibble. But when I test a CI build with selenium, the `npm install` step fails on building fibers :(. Looking at quibble-stretch/Dockerfile.template, it seems like there are a lot of extras there that are not included in integration/quibble/Dockerfile [13:30:40] not sure of a time efficient way to test my branch of integration/quibble with a CI build [13:37:58] 10Release-Engineering-Team, 10Developer Productivity, 10local-charts: Create an interface for the local-charts ecosystem - https://phabricator.wikimedia.org/T224939 (10kostajh) My two cents, I wanted to suggest consideration of PHP and JavaScript as languages to be used for creation of the CLI. In particular... [13:49:52] 10Release-Engineering-Team-TODO, 10Discovery, 10Discovery-Analysis, 10Operations, and 3 others: Setup a mirror for R language dependencies (CRAN) - https://phabricator.wikimedia.org/T170995 (10hashar) 05Open→03Declined maybe one day if we look again at R [13:49:56] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO, 10Discovery, 10Product-Analytics, and 2 others: Add lint/CI to all wikimedia/discovery analytics repositories - https://phabricator.wikimedia.org/T153856 (10hashar) [13:54:32] (03PS1) 10Hashar: docker: rm no more used releng/npm-test-graphoid image [integration/config] - 10https://gerrit.wikimedia.org/r/516779 [13:55:05] (03CR) 10Hashar: jjb: Drop unused service images (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/516722 (owner: 10Jforrester) [13:55:19] (03CR) 10Hashar: [C: 03+2] docker: rm no more used releng/npm-test-graphoid image [integration/config] - 10https://gerrit.wikimedia.org/r/516779 (owner: 10Hashar) [13:57:20] (03Merged) 10jenkins-bot: docker: rm no more used releng/npm-test-graphoid image [integration/config] - 10https://gerrit.wikimedia.org/r/516779 (owner: 10Hashar) [14:03:18] (03PS3) 10Kosta Harlan: Codehealth: Add Translate extension to pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/516500 (https://phabricator.wikimedia.org/T225581) [14:05:53] (03CR) 10Hashar: [C: 04-1] "The double slashes in docker run -v is required by Docker under Windows. That got introduced by Addshore iirc :]" (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/516739 (owner: 10Awight) [14:07:03] (03CR) 10Hashar: [C: 03+1] layout: [mediawiki/services/recommendation-api] Drop npm-docker [integration/config] - 10https://gerrit.wikimedia.org/r/516635 (owner: 10Mholloway) [14:07:11] (03CR) 10Hashar: [C: 03+1] layout: [mediawiki/services/recommendation-api/deploy] Archive [integration/config] - 10https://gerrit.wikimedia.org/r/516718 (owner: 10Jforrester) [14:31:35] (03CR) 10Awight: "Okay, it's harmless either way." (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/516739 (owner: 10Awight) [15:28:20] 10MediaWiki-Releasing, 10Operations, 10Parsoid: signatures were invalid: EXPKEYSIG 90E9F83F22250DD7 MediaWiki releases repository - https://phabricator.wikimedia.org/T225601 (10greg) [15:28:41] 10Continuous-Integration-Config, 10MediaWiki-Core-Testing, 10Epic: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10Krinkle) [15:28:47] 10Continuous-Integration-Config, 10MediaWiki-Core-Testing, 10Epic, 10Performance-Team (Radar): Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10Krinkle) [15:29:35] 10Continuous-Integration-Config, 10MediaWiki-Core-Testing, 10Epic, 10Performance-Team (Radar): Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10Krinkle) [15:29:53] 10Continuous-Integration-Config, 10MediaWiki-Core-Testing, 10Epic, 10Performance-Team (Radar): Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10Krinkle) [15:33:37] !log Disable "Publish/Notify" for quibble.git in diffusion, T143162, T196347 [15:33:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:33:41] T196347: Quibble may need to rebuild localization cache before running tests - https://phabricator.wikimedia.org/T196347 [15:33:41] T143162: Reduce task notification noise/frequency of changes to associated open patchsets - https://phabricator.wikimedia.org/T143162 [15:50:25] (03CR) 10Thcipriani: [C: 03+2] "INFO:jenkins_jobs.builder:Reconfiguring jenkins job trigger-blubber-pipeline-publish" [integration/config] - 10https://gerrit.wikimedia.org/r/516723 (https://phabricator.wikimedia.org/T224069) (owner: 10Brennen Bearnes) [15:53:31] (03Merged) 10jenkins-bot: pipeline: Add node for pipeline's trigger jobs [integration/config] - 10https://gerrit.wikimedia.org/r/516723 (https://phabricator.wikimedia.org/T224069) (owner: 10Brennen Bearnes) [15:56:12] 10Gerrit, 10Security: Cannot assign user name "XXX" to account ####; name already in use. - https://phabricator.wikimedia.org/T216605 (10thcipriani) Thanks for the enthusiasm :) Could I get someone who is up-to-date on our LDAP setup (either @akosiaris or @bd808, I'm guessing) to review my assertions above? I... [15:56:31] (03CR) 10Jforrester: "Nice." [integration/config] - 10https://gerrit.wikimedia.org/r/516779 (owner: 10Hashar) [15:58:58] (03CR) 10Thcipriani: "recheck" [blubber] - 10https://gerrit.wikimedia.org/r/516671 (https://phabricator.wikimedia.org/T215319) (owner: 10Thcipriani) [16:01:37] !log Dropped docker-registry.discovery.wmnet/releng/npm6:latest from contint1001, unused [16:01:38] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:02:18] (03PS1) 10Jforrester: jjb: Drop mathoid?(-deploy)-npm-node-6-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/516795 [16:02:20] (03PS1) 10Jforrester: dockerfiles: Drop npm-test-mathoid, unused [integration/config] - 10https://gerrit.wikimedia.org/r/516796 [16:04:45] (03CR) 10Jforrester: [C: 03+2] jjb: Drop mathoid?(-deploy)-npm-node-6-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/516795 (owner: 10Jforrester) [16:04:49] (03CR) 10Jforrester: [C: 03+2] dockerfiles: Drop npm-test-mathoid, unused [integration/config] - 10https://gerrit.wikimedia.org/r/516796 (owner: 10Jforrester) [16:04:59] PROBLEM - Free space - all mounts on integration-trigger-01 is CRITICAL: CRITICAL: integration.integration-trigger-01.diskspace._srv.byte_percentfree (<10.00%) [16:06:05] !log Dropped mathoid?(-deploy)-npm-node-6-docker from jjb [16:06:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:06:08] 10Release-Engineering-Team (Kanban), 10Release Pipeline, 10Patch-For-Review: Add/reserve a Jenkins node for the pipeline's trigger jobs - https://phabricator.wikimedia.org/T224069 (10brennen) 05Open→03Resolved [16:07:01] (03Merged) 10jenkins-bot: jjb: Drop mathoid?(-deploy)-npm-node-6-docker, unused [integration/config] - 10https://gerrit.wikimedia.org/r/516795 (owner: 10Jforrester) [16:07:03] (03Merged) 10jenkins-bot: dockerfiles: Drop npm-test-mathoid, unused [integration/config] - 10https://gerrit.wikimedia.org/r/516796 (owner: 10Jforrester) [16:07:18] !log Dropped docker-registry.discovery.wmnet/releng/npm-test-mathoid:latest, unused [16:07:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:11:16] (03PS1) 10Jforrester: jjb: Make sure everything is using the latest npm-browser-test image [integration/config] - 10https://gerrit.wikimedia.org/r/516797 [16:14:58] RECOVERY - Free space - all mounts on integration-trigger-01 is OK: OK: All targets OK [16:19:07] (03PS1) 10Jforrester: jjb: Make sure everything is using the latest npm-test image [integration/config] - 10https://gerrit.wikimedia.org/r/516798 [16:27:12] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MediaWiki-Core-Testing, 10Code-Health, and 2 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10Jdforrester-WMF) [16:29:37] 10Continuous-Integration-Config, 10MediaWiki-Core-Testing, 10MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), 10User-Ladsgroup: Improve caching in CI tests - https://phabricator.wikimedia.org/T225496 (10Jdforrester-WMF) How much further improvement are we expecting? Without before/after benchmarks it's hard to k... [16:31:01] PROBLEM - Free space - all mounts on integration-trigger-01 is CRITICAL: CRITICAL: integration.integration-trigger-01.diskspace._srv.byte_percentfree (<10.00%) [16:31:01] hrm, i seem to have created a logspam situation with integration-trigger-01 disk space. [16:31:26] you may want to create a medium or large instance brennen [16:31:56] yeah. [16:32:09] also doin't forget to apply the "::profile::labs::lvm::srv" [16:32:14] (through horizion) [16:45:21] RECOVERY - Mediawiki Error Rate on graphite-labs is OK: OK: Less than 1.00% above the threshold [1.0] [16:45:49] 10Release-Engineering-Team (Kanban), 10Release Pipeline, 10Patch-For-Review: Add/reserve a Jenkins node for the pipeline's trigger jobs - https://phabricator.wikimedia.org/T224069 (10hashar) 05Resolved→03Open The instance is a small one and all most of the disk is used by the `/` partition. When the ext... [16:52:25] (03PS4) 10Kosta Harlan: Publish coverage reports from extension-codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/514016 [16:53:02] 10Release-Engineering-Team (Kanban), 10Release Pipeline, 10Patch-For-Review: Add/reserve a Jenkins node for the pipeline's trigger jobs - https://phabricator.wikimedia.org/T224069 (10brennen) Just so I'm clear, in this instance is it sufficient to change workspace to `/mnt/home/jenkins-deploy` in the Jenkins... [16:53:09] (03PS5) 10Kosta Harlan: Publish coverage reports from extension-codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/514016 [16:54:17] (03PS6) 10Kosta Harlan: Publish coverage reports from extension-codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/514016 (https://phabricator.wikimedia.org/T224990) [16:55:41] 10Continuous-Integration-Infrastructure (phase-out-jessie), 10Release-Engineering-Team: Drop php55 testing support - https://phabricator.wikimedia.org/T224907 (10hashar) Thank you very much! [16:57:17] 10Release-Engineering-Team (Kanban), 10Release Pipeline, 10Patch-For-Review: Add/reserve a Jenkins node for the pipeline's trigger jobs - https://phabricator.wikimedia.org/T224069 (10hashar) >>! In T224069#5257019, @brennen wrote: > Just so I'm clear, in this instance is it sufficient to change workspace to... [17:02:04] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Cleanup CI puppet manifests - https://phabricator.wikimedia.org/T225735 (10hashar) [17:02:47] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team, 10Technical-Debt: Cleanup CI puppet manifests - https://phabricator.wikimedia.org/T225735 (10hashar) I guess I will do it since I am responsible for almost of the mess that happened there. [17:05:28] !log changing jenkins remote root for integration-trigger-01 to /mnt/home/jenkins-deploy per T224069 [17:05:30] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:05:30] T224069: Add/reserve a Jenkins node for the pipeline's trigger jobs - https://phabricator.wikimedia.org/T224069 [17:11:22] (03PS7) 10Kosta Harlan: Publish coverage reports from extension-codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/514016 (https://phabricator.wikimedia.org/T224990) [17:18:14] (03CR) 10Kosta Harlan: "This patch successfully serves /workspace/src at localhost:9142, but I can't get Selenium working because building fiber fails on npm inst" [integration/quibble] - 10https://gerrit.wikimedia.org/r/516729 (https://phabricator.wikimedia.org/T225218) (owner: 10Kosta Harlan) [17:49:14] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MediaWiki-Core-Testing, 10Code-Health, and 2 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10Mainframe98) >>! In T225730#5256852, @Krinkle wrote: > * Look at the PHPUnit "Test Report... [17:50:58] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MediaWiki-Core-Testing, 10Code-Health, and 2 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10kostajh) > splitting the unit and integration test runs into different jenkins votes, del... [17:57:40] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10MediaWiki-Core-Testing, 10Code-Health, and 2 others: Reduce runtime of MW shared gate Jenkins jobs to 5 min - https://phabricator.wikimedia.org/T225730 (10Krinkle) >>! In T225730#5257180, @Jdforrester-WMF wrote: > [..] doesn't have any specific... [18:01:51] Tech Conf open office hours now at https://meet.google.com/exz-zxfy-nuj?authuser=0 [18:04:01] 10Release-Engineering-Team, 10Developer Productivity, 10local-charts: Create an interface for the local-charts ecosystem - https://phabricator.wikimedia.org/T224939 (10mmodell) >>! In T224939#5256605, @kostajh wrote: > My two cents, I wanted to suggest consideration of PHP and JavaScript as languages to be u... [18:05:34] 10Release-Engineering-Team (Kanban), 10Release Pipeline: Add/reserve a Jenkins node for the pipeline's trigger jobs - https://phabricator.wikimedia.org/T224069 (10brennen) Per conversation elsewhere, will pair with @thcipriani on the puppet role on Friday. [18:11:43] (03CR) 10Thcipriani: [C: 03+2] "Job updates deployed, merging to deploy zuul changes." [integration/config] - 10https://gerrit.wikimedia.org/r/514016 (https://phabricator.wikimedia.org/T224990) (owner: 10Kosta Harlan) [18:13:01] RECOVERY - Free space - all mounts on integration-trigger-01 is OK: OK: All targets OK [18:14:01] (03Merged) 10jenkins-bot: Publish coverage reports from extension-codehealth [integration/config] - 10https://gerrit.wikimedia.org/r/514016 (https://phabricator.wikimedia.org/T224990) (owner: 10Kosta Harlan) [18:17:08] !log reloading zuul to deploy https://gerrit.wikimedia.org/r/514016 [18:17:09] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:17:24] ^ kostajh FYI [18:17:32] thcipriani: thanks again! [18:24:22] 10Release-Engineering-Team, 10Developer Productivity, 10local-charts: Create an interface for the local-charts ecosystem - https://phabricator.wikimedia.org/T224939 (10brennen) I'm biased in favor of some non-JS solution partly because I'm not personally great at JS, but mostly because it feels like there's... [18:28:47] 10Release-Engineering-Team, 10Developer Productivity, 10local-charts: Create an interface for the local-charts ecosystem - https://phabricator.wikimedia.org/T224939 (10Jdforrester-WMF) I'd say the order of expertise is probably PHP/JS > Python > Go. But really it doesn't matter as much – we're not expecting... [18:31:30] 10Release-Engineering-Team, 10Developer Productivity, 10local-charts: Create an interface for the local-charts ecosystem - https://phabricator.wikimedia.org/T224939 (10thcipriani) >>! In T224939#5257358, @brennen wrote: > ... > 2. Python (present or installable on most systems?) Noteworthy apple announcemen... [19:38:31] legoktm, hashar: Thoughts on creating a "heavy" variant of quibble-stretch-php70 (well, -php72) with all the extra bloat needed for coverage/etc. and only use that for those jobs, rather than the same image for quibble runs and coverage runs? [19:38:32] hmm, i wonder if this https://groups.google.com/forum/#!topic/repo-discuss/vVI7w1zm15U would be interesting. [19:39:11] James_F: yes, I think moving those tools into an image on top of the standard quibble one is a good idea [19:39:26] Cool. [19:39:33] Want to do that, or should I muddle through? :-D [19:40:42] My main priorities right now are making sure we're stable on the phan stuff, and then figuring out the next steps for moving CI dependencies to extension.json. Also unbreaking libup because I accidentally broke it a few days ago [19:40:51] Kk. [19:40:52] (03PS1) 10Jforrester: [WIP] layout: Skip PHP70 and PHP71 jobs for MW master; simplify regexes [integration/config] - 10https://gerrit.wikimedia.org/r/516838 [19:40:53] So...I probably won't have time for a few weeks, but I should be able to find time to review [19:41:10] Does "stable on phan stuff" include moving that to php72, or should I do that? [19:42:29] (03CR) 10jerkins-bot: [V: 04-1] [WIP] layout: Skip PHP70 and PHP71 jobs for MW master; simplify regexes [integration/config] - 10https://gerrit.wikimedia.org/r/516838 (owner: 10Jforrester) [19:46:43] James_F: php72, Wikibase(Lexeme)? fixes, and ensuring the dependency tree is reasonable [19:47:47] James_F: +1 like kunal said [19:52:12] (03CR) 10Hashar: Core npm composer test as command (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/513515 (https://phabricator.wikimedia.org/T223752) (owner: 10Awight) [19:56:59] (03PS1) 10Jforrester: dockerfiles: Provide a quibble-coverage image [integration/config] - 10https://gerrit.wikimedia.org/r/516840 [19:57:01] (03PS1) 10Jforrester: dockerfiles: [quibble-stretch-php70] Drop coverage tools, use the quibble-coverage image [integration/config] - 10https://gerrit.wikimedia.org/r/516841 [19:57:02] There are some patches, probably broken. [19:57:03] (03PS1) 10Jforrester: jjb: Switch coverage jobs from quibble-stretch-php70 to quibble-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/516842 [19:57:53] hashar: I'm assuming that the "new release of quibble" and the "new docker images for how quibble is deployed" should be done apart so it's obvious what broke what. [20:00:19] James_F: yes. I do the bump in dockerfiles first [20:00:22] 10MediaWiki-Releasing, 10Operations, 10Parsoid: debian signing keyid E84AFDD2 has expired - https://phabricator.wikimedia.org/T141400 (10Kghbln) [20:00:25] eventually download the resulting container test it [20:00:29] then bump some jobs [20:00:33] in a different change [20:00:52] (03CR) 10Hashar: [C: 03+2] "Lets go!" (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/513515 (https://phabricator.wikimedia.org/T223752) (owner: 10Awight) [20:01:03] No, I meant, the work to move to php72 and the top to split thing quibble steps up should be different. [20:01:16] (And all of this blocks dropping PHP71/70 from MW core.) [20:02:05] also I think we have the php7.2 from sury [20:02:11] We do [20:02:15] should probably get it from SRE instead :] [20:02:21] Does that image exist yet? [20:03:26] in short, whenever we have a dockerfiles/* file referring to sury.org, we should drop the gpg key and apt config [20:03:40] and replace it with the apt.wikimedia.org wikimedia-stretch components/php72 (something like that) [20:04:11] Yeah. Do we have an SRE-blessed version of PHP71? [20:04:19] hmm [20:04:25] nop [20:04:34] but *cough* we could just drop 7.1 from CI [20:04:50] Entirely? [20:05:16] the other night I was trying to figure out which linux distributions ship which php versions [20:05:26] among redhat/ubntu/linux mint etc [20:05:31] but could not find a comprehsenive list [20:05:33] With https://gerrit.wikimedia.org/r/#/c/516838/ we'll only be testing PHP71 on REL1_32 and REL1_33. [20:06:20] (03CR) 10Jforrester: "(Not ready until we've migrated phan and coverage jobs to php72.)" [integration/config] - 10https://gerrit.wikimedia.org/r/516838 (owner: 10Jforrester) [20:07:25] (03Merged) 10jenkins-bot: Core npm composer test as command [integration/quibble] - 10https://gerrit.wikimedia.org/r/513515 (https://phabricator.wikimedia.org/T223752) (owner: 10Awight) [20:08:18] (03CR) 10jenkins-bot: Core npm composer test as command [integration/quibble] - 10https://gerrit.wikimedia.org/r/513515 (https://phabricator.wikimedia.org/T223752) (owner: 10Awight) [20:08:35] hashar: Does that quibble stack have a phpunit unit tests vs. phpunit integration tests step split? [20:08:40] Or is that for later? [20:08:52] there is no such thing in mediawiki/core yet [20:09:02] the only split I have done is based on '@group Database' [20:09:04] I was going to merge it now. :-) [20:10:22] * apergos votes for the drop [20:12:08] `vendor/bin/phpunit -d memory_limit=512M -c tests/phpunit/unit-tests.xml` => `Time: 5.2 seconds, Memory: 168.00MB` (with 5303 tests run). [20:12:35] ~1 ms per unit test feels a little slow, but it's /so/ much faster than our current runs. [20:17:26] you are not going to hear any complaints from over here [20:17:33] James_F: yeah there are a few patches floating around to use the raw phpunit [20:17:33] also this won't need any quibble setup [20:17:43] just an environment with php and you're good to go [20:17:47] and get rid of our crazy phpunit.php + MediaWikiTestCase [20:18:43] mszabo-wikia: yes that is the ida [20:18:45] idea [20:18:52] Quibble is just a transient helper [20:19:07] (which is going to last more than a few months unfortunately, but still the aim is to eventually drop it [20:19:43] yeah, I guess quibble will still be needed for the foreseeable future to run the integration/system tests corpus, but at least now the "unit" tests can be run in a separate job without them [20:20:17] as an intermediate step, we could make the integration/system tests use raw phpunit instead of phpunit.php to at least make them compatible with the 3rd party tooling like IDEs' test execution / coverage import [20:20:47] (03CR) 10Hashar: [C: 04-1] "This drop a few things related to environment variable and tests that were covering edge cases. So I gotta review/test that more carefull" [integration/quibble] - 10https://gerrit.wikimedia.org/r/513518 (https://phabricator.wikimedia.org/T223752) (owner: 10Awight) [20:21:43] (03CR) 10Hashar: [C: 03+1] Inline remaining cmd.py steps [integration/quibble] - 10https://gerrit.wikimedia.org/r/515182 (owner: 10Awight) [20:23:00] mszabo-wikia: yes in the grand future of things that sounds like a good plan! :] [20:23:23] there are a lot of things from our phpunit.php and MediaWikiTestCase that would need to be dropped or some how ported [20:23:26] but that is doable [20:23:40] iirc kostajh already has a patch for that [20:25:51] (03CR) 10Hashar: [C: 03+1] "It is a rebuild based on releng/npm:0.3.1" [integration/config] - 10https://gerrit.wikimedia.org/r/516798 (owner: 10Jforrester) [20:29:20] (03PS8) 10Hashar: Create selenium-daily-beta-ORES Jenkins job [integration/config] - 10https://gerrit.wikimedia.org/r/460517 (https://phabricator.wikimedia.org/T188742) (owner: 10Zfilipin) [20:29:59] (03PS2) 10Jforrester: dockerfiles: [quibble-stretch-php70] Drop coverage tools, use the quibble-coverage image [integration/config] - 10https://gerrit.wikimedia.org/r/516841 [20:30:01] (03PS2) 10Jforrester: jjb: Switch coverage jobs from quibble-stretch-php70 to quibble-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/516842 [20:31:27] (03CR) 10Hashar: [C: 03+2] "The job already got deployed but since this change never got merged, the job was really obsolete." [integration/config] - 10https://gerrit.wikimedia.org/r/460517 (https://phabricator.wikimedia.org/T188742) (owner: 10Zfilipin) [20:33:48] (03Merged) 10jenkins-bot: Create selenium-daily-beta-ORES Jenkins job [integration/config] - 10https://gerrit.wikimedia.org/r/460517 (https://phabricator.wikimedia.org/T188742) (owner: 10Zfilipin) [20:33:53] (03CR) 10Awight: "Agreed, thanks for the careful review! I'll restore the missing test." (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/513518 (https://phabricator.wikimedia.org/T223752) (owner: 10Awight) [20:34:23] Can we have jobs gate other jobs? E.g. run "ci-quick" on a patch, if it passes, then run "ci-full"? [20:35:03] (03PS4) 10Hashar: Drop php-compile-hhvm-jessie-docker [integration/config] - 10https://gerrit.wikimedia.org/r/514070 (https://phabricator.wikimedia.org/T224908) (owner: 10Jforrester) [20:35:05] James_F: yes [20:35:16] but it is a bit messy [20:35:37] (03PS2) 10Jforrester: dockerfiles: Provide a quibble-coverage image [integration/config] - 10https://gerrit.wikimedia.org/r/516840 [20:35:55] Could we move to having lints and unit tests in the fast job for all repos, or would it make it too complex to manage CI/ [20:36:01] (03CR) 10Hashar: [C: 03+2] "Cherry picked to tip of master and amended to also remove the container." [integration/config] - 10https://gerrit.wikimedia.org/r/514070 (https://phabricator.wikimedia.org/T224908) (owner: 10Jforrester) [20:36:02] If "no", don't worry about it. :-) [20:36:06] yes [20:36:10] so [20:36:17] one way to do that is [20:36:21] on CR+2, run the light/lint tests [20:36:43] then Zuul would vote +1 on a NEW Gerrit label such as "lint ready" [20:36:55] which then trigger a second pipeline which run the other tests [20:36:56] but really [20:36:58] * James_F nods. [20:37:01] that is hmm .. complicated [20:37:14] another way is [20:37:20] Or we could split the labels? L+2 and V+2. zuul triggers the "real" V runs on an L+2? [20:37:21] zuul has support for a hierarchy of jobs [20:37:23] so you could have: [20:37:27] gate-and-submit: [20:37:33] - linting-job: [20:37:43] - integration-postgre [20:37:45] - integration-mysql [20:37:51] Right. [20:37:53] - integration-php [20:37:54] etc [20:38:01] with that hierarchy [20:38:14] zuul would first run the 'linting-job', and only if it passes the other jobs would get triggered [20:38:17] (03Merged) 10jenkins-bot: Drop php-compile-hhvm-jessie-docker [integration/config] - 10https://gerrit.wikimedia.org/r/514070 (https://phabricator.wikimedia.org/T224908) (owner: 10Jforrester) [20:38:30] and .. you can have two parents ( eg npm test + composer test) [20:38:31] but [20:38:41] (03CR) 10Jforrester: "I had forgotten that I'd made this patch. :-)" [integration/config] - 10https://gerrit.wikimedia.org/r/514070 (https://phabricator.wikimedia.org/T224908) (owner: 10Jforrester) [20:38:47] we might be able to forge some kind of job that would at least run "composer test + npm test" [20:38:51] (03PS3) 10Jforrester: dockerfiles: Provide a quibble-coverage image [integration/config] - 10https://gerrit.wikimedia.org/r/516840 [20:38:53] as the parent for the rest of the jobs [20:38:57] (03CR) 10Jforrester: [C: 03+2] dockerfiles: Provide a quibble-coverage image [integration/config] - 10https://gerrit.wikimedia.org/r/516840 (owner: 10Jforrester) [20:39:24] (03PS3) 10Jforrester: jjb: Switch coverage jobs from quibble-stretch-php70 to quibble-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/516842 [20:40:01] Should I make a task? [20:40:17] !log deleting docker-registry.wikimedia.org/releng/hhvm-jessie-compile from contint1001 # T224908 [20:40:20] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:40:20] T224908: Drop jessie testing support - https://phabricator.wikimedia.org/T224908 [20:40:29] (03Merged) 10jenkins-bot: dockerfiles: Provide a quibble-coverage image [integration/config] - 10https://gerrit.wikimedia.org/r/516840 (owner: 10Jforrester) [20:40:38] "Should I make a task?" [20:40:47] this question only has one valid answer: yes of course! [20:40:48] :D [20:40:50] :-D [20:41:03] but [20:41:09] there is a bit too much moving grounds [20:41:22] I would like to simplify the zuul/layout.yaml a bit [20:41:28] to avoid most of those crazy skip-if statement [20:41:45] Yes. [20:41:55] and have a gate pipeline for each branch [20:42:02] [16:05:16] the other night I was trying to figure out which linux distributions ship which php versions <-- use repology [20:42:13] (REL1_32 / master / wmf.[13579] / wmf.[024680] [20:42:14] (03CR) 10Awight: "> This patch successfully serves /workspace/src at localhost:9142," [integration/quibble] - 10https://gerrit.wikimedia.org/r/516729 (https://phabricator.wikimedia.org/T225218) (owner: 10Kosta Harlan) [20:42:33] I'd also love us to use :latest in jjb (with the expectation that bad images are rapidly reverted if they've broken). [20:42:45] Simplicity and fewer manual bumps. [20:42:48] legoktm: I think https://distrowatch.com/ used to have such a list. Repology is PERFECT! [20:43:44] https://repology.org/project/php/versions :] [20:43:57] yep :D [20:44:02] also https://repology.org/project/mediawiki/versions is the one I watch :) [20:44:55] 10Continuous-Integration-Config, 10Code-Health: Run "fast" tests (lint, units) in a distinct job from "heavy" tests (integration, browser), and trigger the latter only when the former pass - https://phabricator.wikimedia.org/T225754 (10Jdforrester-WMF) [20:46:32] hashar: Did you drop the hhvm-jessie-compile image yet? [20:46:38] James_F: yes [20:46:51] Hmm, fab deploy_docker is saying it's going to deploy that… [20:47:47] yeah that is the deletion [20:47:52] lines should be prefixed with - ? [20:47:54] hmm [20:48:12] They are. [20:48:37] (I'm not going to push quibble-coverage out if it might break things. ;-)) [20:49:52] eek [20:50:20] I don't think we need that quibble-coverage image [20:50:20] I was ruminating about the light->heavy CI jobs question lately, too (https://phabricator.wikimedia.org/T211704#5246951), what I'd like to understand is there's any benefit to managing this complexity using integration-config, or if we should have quibble doing the fun stuff, like conditionally cloning dependencies after the "light" phase. [20:50:31] hashar: You don't? [20:50:43] You +1'ed earlier. :-P [20:50:55] * hashar bans himself [20:51:08] I guess I was not paying attention haha [20:51:15] :-) [20:51:40] most probably we could just ship them within quibble-stretch [20:51:56] The idea is to slim down quibble-stretch-php70 and make it simpler and quicker to bump -coverage to php72 and later php73 etc. [20:52:07] yeah hm [20:52:08] then [20:52:15] Didn't we just move them /out/ of -stretch? [20:52:35] I guess I am too tired to think properly at this hour ;] [20:52:41] * James_F grins. [20:52:51] so anyway [20:52:51] I can revert, no problem. [20:52:54] just build it [20:53:03] I looked at the diff, that is fine [20:53:18] it is just that my change to remove the old jessie compile container got merged as well [20:53:27] OK. !log Updating docker-pkg files to create quibble-coverage image [20:53:39] then if all goes well, you have a /releng/quibble-coverage:x.y.z in docker registry [20:53:42] TIL php -d zend_extension=xdebug.so OMG I have been living in the shadows [20:53:42] the message has to start with !log :) [20:53:56] which is not being used until a Jenkins job is configured to use that. So it is fine ;] [20:54:04] Bah, IRCCloud swallowed my first [20:54:08] OK. [20:54:09] !log Updating docker-pkg files to create quibble-coverage image [20:54:10] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:54:38] !log ¡Learning to log one step at a time! [20:54:39] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:54:44] I am off [20:54:49] Bye! [20:54:51] wife just came back from work ;] [20:54:54] have a good night! [20:56:39] (03PS3) 10Jforrester: dockerfiles: [quibble-stretch-php70] Drop coverage tools, use the quibble-coverage image [integration/config] - 10https://gerrit.wikimedia.org/r/516841 [20:59:43] mszabo-wikia: I started working on a patch to have a single phpunit.xml.dist, so vendor/bin/phpunit would run both unit & integration tests, at https://gerrit.wikimedia.org/r/c/mediawiki/core/+/514381 [20:59:54] when your patch is merged I'll come back to it :) [21:04:01] PROBLEM - Free space - all mounts on integration-trigger-01 is CRITICAL: CRITICAL: integration.integration-trigger-01.diskspace._srv.byte_percentfree (<10.00%) [21:04:18] @kos [21:04:26] riot UI is a disaster [21:04:47] (03CR) 10Kosta Harlan: "> Patch Set 1:" [integration/quibble] - 10https://gerrit.wikimedia.org/r/516729 (https://phabricator.wikimedia.org/T225218) (owner: 10Kosta Harlan) [21:04:48] kostajh: cool, so the unit tests job would probably filter based on testsuite and run only unit tests right? [21:06:40] mszabo-wikia: well there are a few things to sort out. I'd like to organize into "unit" and "integration" test suites, with their corresponding directories, with the understanding that "integration" probably needs to be further split up. [21:07:05] sounds good [21:07:21] mszabo-wikia: and then vendor/bin/phpunit should either run no suites or all of them (probably all of them) [21:08:12] we have some existing test suites that are already defined but it might be possible to replace existing usages in CI by just passing their respective directories as an argument to vendor/bin/phpunit [21:08:29] oh I see. so separate configurations with a single test suite each [21:10:09] I'd prefer to have a single phpunit.xml.dist file and a single bootstrap.php file that handles common configuration, then the test suite class handles any further configuration [21:10:56] !log restarted jenkins agent on integration-trigger-01 to pick up new root configuration [21:10:57] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:12:07] yeah, but how would we define the integration/unit suites? in the XML file or someplace else? [21:12:25] (03CR) 10Awight: browser tests as command (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/513518 (https://phabricator.wikimedia.org/T223752) (owner: 10Awight) [21:14:02] RECOVERY - Free space - all mounts on integration-trigger-01 is OK: OK: All targets OK [21:14:13] SMalyshev: try asking here or in [21:22:26] 10Continuous-Integration-Infrastructure, 10Code-Health: Create structure test for MediaWikiUnitTestCase usage - https://phabricator.wikimedia.org/T225758 (10kostajh) [21:23:45] 10Continuous-Integration-Infrastructure, 10Code-Health: Create structure test for MediaWikiUnitTestCase usage - https://phabricator.wikimedia.org/T225758 (10kostajh) [21:29:54] (03CR) 10Jforrester: [C: 03+2] jjb: Switch coverage jobs from quibble-stretch-php70 to quibble-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/516842 (owner: 10Jforrester) [21:31:51] 10Release-Engineering-Team, 10MediaWiki-extensions-CodeReview, 10Security: Parameters to Special:Code allows to prefill the textarea for a comment reply with unsafe user input - https://phabricator.wikimedia.org/T225321 (10sbassett) p:05Triage→03Normal [21:32:48] (03Merged) 10jenkins-bot: jjb: Switch coverage jobs from quibble-stretch-php70 to quibble-coverage [integration/config] - 10https://gerrit.wikimedia.org/r/516842 (owner: 10Jforrester) [21:42:27] !log Update JJB job mediawiki-core-code-coverage-docker to point to quibble-coverage [21:42:29] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:43:50] !log Updated JJB job mediawiki-phpunit-coverage-patch-docker to point to quibble-coverage [21:43:51] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:48:41] (03PS1) 10Jforrester: Revert "jjb: Switch coverage jobs from quibble-stretch-php70 to quibble-coverage" [integration/config] - 10https://gerrit.wikimedia.org/r/516961 [21:48:46] (03CR) 10Jforrester: [C: 03+2] Revert "jjb: Switch coverage jobs from quibble-stretch-php70 to quibble-coverage" [integration/config] - 10https://gerrit.wikimedia.org/r/516961 (owner: 10Jforrester) [21:49:42] !log Reverted JJB jobs mediawiki-core-code-coverage-docker and mediawiki-phpunit-coverage-patch-docker back to quibble-stretch-php70 [21:49:43] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:50:58] (03Merged) 10jenkins-bot: Revert "jjb: Switch coverage jobs from quibble-stretch-php70 to quibble-coverage" [integration/config] - 10https://gerrit.wikimedia.org/r/516961 (owner: 10Jforrester) [21:56:35] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban): Run phan on PHP 7.2, not PHP 7.0 - https://phabricator.wikimedia.org/T225456 (10Jdforrester-WMF) [21:56:37] 10Continuous-Integration-Config, 10phan: Prepare phan-docker on CI for phan2.0.0 - https://phabricator.wikimedia.org/T223847 (10Jdforrester-WMF) 05Open→03Resolved [21:56:52] 10Continuous-Integration-Config, 10MediaWiki-Core-Testing, 10MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), 10User-Ladsgroup: Improve caching in CI tests - https://phabricator.wikimedia.org/T225496 (10Krinkle) >>! In T225496#5252365, @hashar wrote: > I am not quite sure how MediaWiki selects the caches its is... [21:59:32] 10Release-Engineering-Team, 10Developer Productivity, 10local-charts: Create an interface for the local-charts ecosystem - https://phabricator.wikimedia.org/T224939 (10mmodell) @thcipriani: at least on a mac with [[ https://brew.sh/ | homebrew ]], it's as simple as `brew install python` [22:00:13] (03PS2) 10Jforrester: layout: Switch extension/skin-phan jobs from php70-phan-docker to php72-phan-docker [integration/config] - 10https://gerrit.wikimedia.org/r/515293 (https://phabricator.wikimedia.org/T225456) [22:00:26] (03PS2) 10Jforrester: layout: Switch parsoid from php70-phan-docker to php72-phan-docker [integration/config] - 10https://gerrit.wikimedia.org/r/515294 (https://phabricator.wikimedia.org/T225456) [22:00:40] (03PS2) 10Jforrester: layout: Switch MediaWiki from php70-phan-docker to php72-phan-docker [integration/config] - 10https://gerrit.wikimedia.org/r/515295 (https://phabricator.wikimedia.org/T225456) [22:02:42] (03CR) 10jerkins-bot: [V: 04-1] layout: Switch extension/skin-phan jobs from php70-phan-docker to php72-phan-docker [integration/config] - 10https://gerrit.wikimedia.org/r/515293 (https://phabricator.wikimedia.org/T225456) (owner: 10Jforrester) [22:02:53] (03CR) 10jerkins-bot: [V: 04-1] layout: Switch parsoid from php70-phan-docker to php72-phan-docker [integration/config] - 10https://gerrit.wikimedia.org/r/515294 (https://phabricator.wikimedia.org/T225456) (owner: 10Jforrester) [22:02:58] (03CR) 10jerkins-bot: [V: 04-1] layout: Switch MediaWiki from php70-phan-docker to php72-phan-docker [integration/config] - 10https://gerrit.wikimedia.org/r/515295 (https://phabricator.wikimedia.org/T225456) (owner: 10Jforrester) [22:03:58] 10Continuous-Integration-Config, 10MediaWiki-Core-Testing, 10MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), 10User-Ladsgroup: Improve caching in CI tests - https://phabricator.wikimedia.org/T225496 (10Krinkle) >>! In T225496#5256985, @Jdforrester-WMF wrote: > How much further improvement are we expecting? With... [22:05:16] 10Continuous-Integration-Config, 10MediaWiki-Core-Testing, 10MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), 10User-Ladsgroup: Improve caching in CI tests - https://phabricator.wikimedia.org/T225496 (10Jdforrester-WMF) 05Open→03Resolved WFM. Thanks! [22:06:10] James_F: might be a chmod issue, depending on how you copied those .sh files. That bit me in the past as well. [22:06:33] at least some of those .sh files somewhere need 755 instead of 644, but don't know if it that's the case here [22:06:37] Krinkle: Ah, interesting. [22:06:39] * James_F checks. [22:11:46] 10Release-Engineering-Team, 10MediaWiki-extensions-CodeReview, 10MW-1.34-notes (1.34.0-wmf.10; 2019-06-18), 10Security: Parameters to Special:Code allows to prefill the textarea for a comment reply with unsafe user input - https://phabricator.wikimedia.org/T225321 (10Reedy) 05Open→03Resolved a:03Umher... [22:11:51] (03PS1) 10Jforrester: dockerfiles: [quibble-coverage] Correct chmod from 0644 to 0755 for executable files [integration/config] - 10https://gerrit.wikimedia.org/r/516964 [22:11:57] Krinkle: Indeed, got it in one. :-( [22:14:55] (03PS2) 10Jforrester: dockerfiles: [quibble-coverage] Correct chmod from 0644 to 0755 for executable files [integration/config] - 10https://gerrit.wikimedia.org/r/516964 [22:15:03] (03CR) 10Jforrester: [C: 03+2] dockerfiles: [quibble-coverage] Correct chmod from 0644 to 0755 for executable files [integration/config] - 10https://gerrit.wikimedia.org/r/516964 (owner: 10Jforrester) [22:16:41] (03Merged) 10jenkins-bot: dockerfiles: [quibble-coverage] Correct chmod from 0644 to 0755 for executable files [integration/config] - 10https://gerrit.wikimedia.org/r/516964 (owner: 10Jforrester) [22:17:28] !log Deploying quibble-coverage fix with chmod 0755 files. [22:17:29] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:18:10] Oh, eurgh. [22:18:16] [contint1001.wikimedia.org] out: 2019-06-13 22:17:48,764 [docker-pkg-build] INFO - ln: failed to create symbolic link '/usr/local/bin/phpunit-patch-coverage': File exists [22:18:29] Because I've not removed that from the parent image yet. [22:18:31] * James_F sighs. [22:19:00] Simple enough solution to that. [22:19:33] (03PS4) 10Jforrester: dockerfiles: [quibble-stretch-php70] Drop coverage tools, use the quibble-coverage image [integration/config] - 10https://gerrit.wikimedia.org/r/516841 [22:24:12] James_F: You can do most of the image building locally btw, but I thought you knew that - is docker-pkg still troubling on your machine? [22:24:27] It works locally, I just didn't notice the error. [22:24:30] I haven't tried in a while either, maybe it stopped working again on macOS/homebrew. [22:24:48] docker-pkg is working great locally, but it's an older version. [22:27:12] (03CR) 10Jforrester: [C: 03+2] dockerfiles: [quibble-stretch-php70] Drop coverage tools, use the quibble-coverage image [integration/config] - 10https://gerrit.wikimedia.org/r/516841 (owner: 10Jforrester) [22:28:47] James_F: right, same here. Once a python tool works on Mac, don't touch it until and unless you have to! [22:28:53] :-) [22:29:15] (03Merged) 10jenkins-bot: dockerfiles: [quibble-stretch-php70] Drop coverage tools, use the quibble-coverage image [integration/config] - 10https://gerrit.wikimedia.org/r/516841 (owner: 10Jforrester) [22:29:20] i had the impression the current version didn't work on macs, but haven't tried personally. [22:29:29] Certainly I ran into issues. [22:29:33] Not sure if that was me or it. [22:30:09] i think longma mentioned running into some trouble. [22:30:11] !log Deploying quibble-stretch-php70 update. [22:30:12] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [22:31:44] I'm a Python noob with 5 years of very causual noob experience around Python CLI tools. And between homebrew/system python, python 2/3, sudo /no-sudo, local -e or not, virtualenv / no venv, real virtualenv, PYTHONPATH ; It's always a game of trying a bunch of things until it works. Embarrassing really. I wouldn't tolerate that for anything else I use, but just held off on properly trying to understand it. [22:32:24] (also sometimes the easy_install/pip is still exposed/linked against the "wrong" python and it gets all confused) [22:33:12] Everything about Python is broken. The language. The parallel versions. The bundlers. The ecosystem. [22:33:42] It's better than ruby, which is apparently meant to make us think it's good. [22:35:19] and i still think significant whitespace was a bad idea, gosh darnit! [22:36:42] I'm in this position where I can deal with traditional C/cpp stuff and install it without issue, and on the other end of the spectrum a grip on doing npm the "right" way. But everything in-between (php cli tools caus yeah composer--global is a thing, or indeed python, or gems) is pretty much a mess for me. Some of it is just badly designed/documented in terms of user experience but works well once you understand it (like Gems, that seems [22:36:43] pretty solid once you figure out the PATHs once) [22:37:05] i only very recently got my head around any of python version / virtualenvs / packaging / setuptools by doing many hundreds of hours of software bureaucracy on a project made out of dozens of little libraries. [22:37:27] i don't resent the language itself the way i once did, but i would agree that the ecosystem has a pretty staggering amount of brokenness in it. [22:39:01] He, whitespace sensitive language, yeah, it's a love/hate thing. I love that for beginners it removes the whole curly brace situation and enforces very early on that the code should "look" like the way it behaves. In other languages (PHP, JS) I see beginners almost always do little to no indentation and initially that's fine because they are so slow at reading that indentation doesn't help, but then it stagnates at some point. [22:39:19] On the other hand, when you're more experienced, it's quite nice to be able to prototype quickly and not have a compile error just for missing a space. [22:41:14] yeah. it has some reasonable affordances, but it also seems to cause a lot of pain with moving code around, and i'm really not sold on the idea that it improves legibility all that greatly once you get past a few screen's worth of code. [22:42:08] it's fine, whatever, but it has certainly caused me more irritation in practice than delimited blocks ever have. [22:45:28] yeah, we discovered the newer version of docker-pkg doesn't work on a mac [22:45:51] Is there a task for that to be fixed? [22:45:56] Or is it an "oh well"? [22:46:11] I guess I could run docker-pkg inside a Debian docker image… [22:47:15] I don't think so about the task...I'm not sure about the "oh well" though [22:48:01] kostajh might know about if there is a task? [22:48:25] I've created T225764. [22:48:26] T225764: Make docker-pkg work on Macs again - https://phabricator.wikimedia.org/T225764 [22:56:15] (03PS1) 10Jforrester: layout: Briefly make mediawiki-quibble-vendor-postgres-php72-docker experimental whilst we fix [integration/config] - 10https://gerrit.wikimedia.org/r/516970 [22:57:22] err [22:57:26] James_F: what's wrong with postgres? [22:58:01] legoktm: Some idiot (hey) force-merged a patch which breaks postgres because it uses "exit;" rather than "die;" in an error condition which makes CI continue… [22:58:41] And so it's blocking all merges in core, and I'd /really/ rather not revert. [22:58:51] which patch? [22:59:10] legoktm: https://gerrit.wikimedia.org/r/513106 [22:59:14] if its broken its broken [22:59:30] Yes. [22:59:41] If it's still there in half an hour I'd revert. [22:59:51] But I don't want to hold up other people. [22:59:53] just revert it now? [23:00:23] Changing the fundamental nature of unit tests is not something that rebases easily. [23:01:04] ans [23:01:10] this really should not have been one giant patch [23:01:10] I will try one more fix in MediaWikiTestCase for that [23:01:25] legoktm: Yeah, a series of patches would have landed a lot better. [23:02:00] Of course, there will be more patches ahead. This was meant to be the easy ones. [23:02:25] ok, there are other problems in this patch [23:03:22] legoktm: Every patch has problems. [23:03:58] (03Abandoned) 10Jforrester: layout: Briefly make mediawiki-quibble-vendor-postgres-php72-docker experimental whilst we fix [integration/config] - 10https://gerrit.wikimedia.org/r/516970 (owner: 10Jforrester) [23:05:29] I'm about to hop on a train, I'll post some comments shortly [23:06:01] Thanks. [23:22:29] 10Project-Admins, 10Phlogiston: Rename #Category to #Category-Phlogiston - https://phabricator.wikimedia.org/T224450 (10Aklapper) @JAufrecht: Any comment? [23:22:56] (03CR) 10Jforrester: [C: 03+2] jjb: Make sure everything is using the latest npm-browser-test image [integration/config] - 10https://gerrit.wikimedia.org/r/516797 (owner: 10Jforrester) [23:23:00] (03CR) 10Jforrester: [C: 03+2] jjb: Make sure everything is using the latest npm-test image [integration/config] - 10https://gerrit.wikimedia.org/r/516798 (owner: 10Jforrester) [23:25:53] (03Merged) 10jenkins-bot: jjb: Make sure everything is using the latest npm-browser-test image [integration/config] - 10https://gerrit.wikimedia.org/r/516797 (owner: 10Jforrester) [23:25:55] (03Merged) 10jenkins-bot: jjb: Make sure everything is using the latest npm-test image [integration/config] - 10https://gerrit.wikimedia.org/r/516798 (owner: 10Jforrester) [23:26:47] 10Release-Engineering-Team, 10Scap, 10serviceops, 10PHP 7.2 support, 10User-jijiki: Enhance MediaWiki deployments for support of php7.x - https://phabricator.wikimedia.org/T224857 (10thcipriani) >>! In T224857#5253300, @Joe wrote: >>>! In T224857#5247276, @thcipriani wrote: >> Checking my understanding o... [23:41:49] 10Release-Engineering-Team, 10Scap, 10Operations: Enable scap to roll back broken changes to MediaWiki - https://phabricator.wikimedia.org/T225207 (10greg) [23:49:21] (03PS1) 10Jforrester: jjb: Switch coverage jobs from quibble-stretch-php70 to quibble-coverage, try II [integration/config] - 10https://gerrit.wikimedia.org/r/516976 [23:50:14] (03PS2) 10Jforrester: jjb: Switch coverage jobs from quibble-stretch-php70 to quibble-coverage, try II [integration/config] - 10https://gerrit.wikimedia.org/r/516976