[00:00:12] maintenance-disconnect-full-disks build 51395 integration-slave-docker-1037 (/var/lib/docker: 95%): OFFLINE due to disk space [00:25:13] maintenance-disconnect-full-disks build 51400 integration-slave-docker-1037: OFFLINE due to disk space [00:50:12] maintenance-disconnect-full-disks build 51405 integration-slave-docker-1037: OFFLINE due to disk space [01:15:48] maintenance-disconnect-full-disks build 51410 integration-slave-docker-1037: OFFLINE due to disk space [01:40:43] maintenance-disconnect-full-disks build 51415 integration-slave-docker-1037: OFFLINE due to disk space [02:05:53] maintenance-disconnect-full-disks build 51420 integration-slave-docker-1037: OFFLINE due to disk space [02:20:25] !log integration-slave-docker-1021 (ci1.medium) has jobs failing on it due to ENOMEM. Horizon shows in log: integration-slave-docker-1021 login: [4961938.696837] Out of memory: Kill process 21770 (chromium) score 841 or sacrifice child; [4961938.699176] Killed process 21770 (chromium) total-vm:3171496kB, anon-rss:1379288kB, file-rss:0kB, shmem-rss:1636kB [02:20:42] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [02:26:13] !log tried rebooting or shutting down integration-slave-docker-1021, no response on horizon. Did pause/resume instead, which did work, after which shutdown/start worked. Jenkins agent has been relaunched and seems online again. [02:26:14] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [02:32:39] maintenance-disconnect-full-disks build 51425 integration-slave-docker-1037: OFFLINE due to disk space [04:47:16] maintenance-disconnect-full-disks build 51450 integration-slave-docker-1037: OFFLINE due to disk space [05:10:17] maintenance-disconnect-full-disks build 51455 integration-slave-docker-1037: OFFLINE due to disk space [05:20:36] Project beta-update-databases-eqiad build #32143: 04FAILURE in 36 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32143/ [05:20:53] Project beta-scap-eqiad build #239934: 04FAILURE in 1 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/239934/ [05:34:29] Yippee, build fixed! [05:34:29] Project beta-scap-eqiad build #239935: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/239935/ [05:35:18] maintenance-disconnect-full-disks build 51460 integration-slave-docker-1037: OFFLINE due to disk space [05:51:25] 10Continuous-Integration-Config, 10Parsoid-PHP: Decide on which unit testing framework (PHPSpec or PHPUnit) to use for Parsoid in the long term - https://phabricator.wikimedia.org/T217458 (10Legoktm) I'd like to see Parsoid use PHPUnit in the long-term for standardization of testing frameworks and CI infra. Ho... [06:00:18] maintenance-disconnect-full-disks build 51465 integration-slave-docker-1037: OFFLINE due to disk space [06:21:04] Yippee, build fixed! [06:21:04] Project beta-update-databases-eqiad build #32144: 09FIXED in 1 min 4 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/32144/ [06:25:21] maintenance-disconnect-full-disks build 51470 integration-slave-docker-1037: OFFLINE due to disk space [06:50:17] maintenance-disconnect-full-disks build 51475 integration-slave-docker-1037: OFFLINE due to disk space [06:55:47] Project beta-scap-eqiad build #239943: 04FAILURE in 1.8 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/239943/ [07:14:32] Yippee, build fixed! [07:14:33] Project beta-scap-eqiad build #239944: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/239944/ [07:15:17] maintenance-disconnect-full-disks build 51480 integration-slave-docker-1037: OFFLINE due to disk space [07:15:53] Project beta-scap-eqiad build #239945: 04FAILURE in 1.4 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/239945/ [07:34:33] Yippee, build fixed! [07:34:33] Project beta-scap-eqiad build #239946: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/239946/ [07:35:55] Project beta-scap-eqiad build #239947: 04FAILURE in 1.7 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/239947/ [07:40:18] maintenance-disconnect-full-disks build 51485 integration-slave-docker-1037: OFFLINE due to disk space [07:54:37] Yippee, build fixed! [07:54:38] Project beta-scap-eqiad build #239948: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/239948/ [08:05:18] maintenance-disconnect-full-disks build 51490 integration-slave-docker-1037: OFFLINE due to disk space [08:30:17] maintenance-disconnect-full-disks build 51495 integration-slave-docker-1037: OFFLINE due to disk space [08:55:18] maintenance-disconnect-full-disks build 51500 integration-slave-docker-1037: OFFLINE due to disk space [09:20:16] maintenance-disconnect-full-disks build 51505 integration-slave-docker-1037: OFFLINE due to disk space [09:45:17] maintenance-disconnect-full-disks build 51510 integration-slave-docker-1037: OFFLINE due to disk space [10:10:13] maintenance-disconnect-full-disks build 51515 integration-slave-docker-1037: OFFLINE due to disk space [10:35:13] maintenance-disconnect-full-disks build 51520 integration-slave-docker-1037: OFFLINE due to disk space [10:54:46] 10Release-Engineering-Team, 10Growth-Team, 10MediaWiki-extensions-OAuth, 10MediaWiki-extensions-UserMerge, and 2 others: OAuth tests failing with Echo/UserMerge issues - https://phabricator.wikimedia.org/T217487 (10Daimona) While I know the CI revert is temporary, I'd like to point out that CheckUser is cu... [11:00:18] maintenance-disconnect-full-disks build 51525 integration-slave-docker-1037: OFFLINE due to disk space [11:25:14] maintenance-disconnect-full-disks build 51530 integration-slave-docker-1037: OFFLINE due to disk space [11:50:12] maintenance-disconnect-full-disks build 51535 integration-slave-docker-1037: OFFLINE due to disk space [12:15:15] maintenance-disconnect-full-disks build 51540 integration-slave-docker-1037: OFFLINE due to disk space [12:40:15] maintenance-disconnect-full-disks build 51545 integration-slave-docker-1037: OFFLINE due to disk space [13:05:12] maintenance-disconnect-full-disks build 51550 integration-slave-docker-1037: OFFLINE due to disk space [13:30:18] maintenance-disconnect-full-disks build 51555 integration-slave-docker-1037: OFFLINE due to disk space [13:46:51] 10Release-Engineering-Team, 10Growth-Team, 10MediaWiki-extensions-OAuth, 10MediaWiki-extensions-UserMerge, and 2 others: OAuth tests failing with Echo/UserMerge issues - https://phabricator.wikimedia.org/T217487 (10Umherirrender) In case of `table_name.*` the `table_name` could be the table alias and not t... [13:55:16] maintenance-disconnect-full-disks build 51560 integration-slave-docker-1037: OFFLINE due to disk space [14:20:17] maintenance-disconnect-full-disks build 51565 integration-slave-docker-1037: OFFLINE due to disk space [14:40:54] 10Release-Engineering-Team, 10Growth-Team, 10MediaWiki-extensions-OAuth, 10MediaWiki-extensions-UserMerge, and 2 others: OAuth tests failing with Echo/UserMerge issues - https://phabricator.wikimedia.org/T217487 (10Daimona) @Umherirrender Your patch for echo is definitely good, and fine for now. I'm trying... [14:45:20] maintenance-disconnect-full-disks build 51570 integration-slave-docker-1037: OFFLINE due to disk space [15:10:12] maintenance-disconnect-full-disks build 51575 integration-slave-docker-1037: OFFLINE due to disk space [15:35:15] maintenance-disconnect-full-disks build 51580 integration-slave-docker-1037: OFFLINE due to disk space [15:58:52] 10Deployments, 10MediaWiki-Debug-Logger, 10Performance-Team, 10User-Tgr: Capture PHP warnings with stacktraces in MediaWiki and save to logstash - https://phabricator.wikimedia.org/T45086 (10Krinkle) a:03Krinkle [16:00:13] maintenance-disconnect-full-disks build 51585 integration-slave-docker-1037: OFFLINE due to disk space [16:20:29] Project beta-scap-eqiad build #239993: 04FAILURE in 2.3 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/239993/ [16:25:13] maintenance-disconnect-full-disks build 51590 integration-slave-docker-1037: OFFLINE due to disk space [16:34:16] Yippee, build fixed! [16:34:17] Project beta-scap-eqiad build #239994: 09FIXED in 9 min 58 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/239994/ [16:50:12] maintenance-disconnect-full-disks build 51595 integration-slave-docker-1037: OFFLINE due to disk space [17:15:12] maintenance-disconnect-full-disks build 51600 integration-slave-docker-1037: OFFLINE due to disk space [17:20:50] Project beta-scap-eqiad build #239999: 04FAILURE in 1.2 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/239999/ [17:34:22] Yippee, build fixed! [17:34:22] Project beta-scap-eqiad build #240000: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/240000/ [17:40:12] maintenance-disconnect-full-disks build 51605 integration-slave-docker-1037: OFFLINE due to disk space [18:05:14] maintenance-disconnect-full-disks build 51610 integration-slave-docker-1037: OFFLINE due to disk space [18:30:14] maintenance-disconnect-full-disks build 51615 integration-slave-docker-1037: OFFLINE due to disk space [18:55:12] maintenance-disconnect-full-disks build 51620 integration-slave-docker-1037: OFFLINE due to disk space [19:20:18] maintenance-disconnect-full-disks build 51625 integration-slave-docker-1037: OFFLINE due to disk space [19:20:51] 10Beta-Cluster-Infrastructure, 10Performance-Team: Move XHGui from tungsten to webperf-002 - https://phabricator.wikimedia.org/T180761 (10Krinkle) a:05Krinkle→03None [19:45:13] maintenance-disconnect-full-disks build 51630 integration-slave-docker-1037: OFFLINE due to disk space [20:10:12] maintenance-disconnect-full-disks build 51635 integration-slave-docker-1037: OFFLINE due to disk space [20:34:34] (03PS1) 10Hashar: Revert "Get rid of .gitreview" [tools/scap] - 10https://gerrit.wikimedia.org/r/494068 (https://phabricator.wikimedia.org/T191373) [20:35:12] maintenance-disconnect-full-disks build 51640 integration-slave-docker-1037: OFFLINE due to disk space [20:36:03] (03CR) 10PipelineBot: "pipeline-dashboard: service-pipeline-test" [tools/scap] - 10https://gerrit.wikimedia.org/r/494068 (https://phabricator.wikimedia.org/T191373) (owner: 10Hashar) [20:36:05] (03CR) 10jerkins-bot: [V: 04-1] Revert "Get rid of .gitreview" [tools/scap] - 10https://gerrit.wikimedia.org/r/494068 (https://phabricator.wikimedia.org/T191373) (owner: 10Hashar) [21:00:12] maintenance-disconnect-full-disks build 51645 integration-slave-docker-1037: OFFLINE due to disk space [21:25:12] maintenance-disconnect-full-disks build 51650 integration-slave-docker-1037: OFFLINE due to disk space [21:50:12] maintenance-disconnect-full-disks build 51655 integration-slave-docker-1037: OFFLINE due to disk space [22:15:14] maintenance-disconnect-full-disks build 51660 integration-slave-docker-1037: OFFLINE due to disk space [22:40:11] maintenance-disconnect-full-disks build 51665 integration-slave-docker-1037: OFFLINE due to disk space [23:05:12] maintenance-disconnect-full-disks build 51670 integration-slave-docker-1037: OFFLINE due to disk space [23:30:11] maintenance-disconnect-full-disks build 51675 integration-slave-docker-1037: OFFLINE due to disk space [23:55:11] maintenance-disconnect-full-disks build 51680 integration-slave-docker-1037: OFFLINE due to disk space