[00:01:12] 10Beta-Cluster-Infrastructure: Problems using modified swift puppet classes on deployment-ms-be0[56] - https://phabricator.wikimedia.org/T220895 (10Krenair) [00:05:50] 10Beta-Cluster-Infrastructure: Problems using modified swift puppet classes on deployment-ms-be0[56] - https://phabricator.wikimedia.org/T220895 (10Krenair) As far as I can tell the puppet patch is fine, I can't explain udev not making that symlink but maybe I just don't know enough about udev. [00:07:31] 10Beta-Cluster-Infrastructure: Udev (?) problems using modified swift puppet classes on deployment-ms-be0[56] - https://phabricator.wikimedia.org/T220895 (10Krenair) [00:10:16] maintenance-disconnect-full-disks build 63490 integration-slave-docker-1037: OFFLINE due to disk space [00:10:17] maintenance-disconnect-full-disks build 63490 integration-slave-docker-1050: OFFLINE due to disk space [00:22:59] 10Phabricator, 10Project-Admins, 10User-greg: Can't manage columns on #fundraising-tech-ops board - https://phabricator.wikimedia.org/T220448 (10Aklapper) 05Open→03Resolved a:03greg assuming this is fixed, if not please reopen [00:24:32] 10Phabricator (Upstream), 10Upstream: Problem with box which shows when user want to create new subtask - https://phabricator.wikimedia.org/T220897 (10Zoranzoki21) [00:34:52] 10Phabricator (Upstream), 10Upstream: Problem with box which shows when user want to create new subtask - https://phabricator.wikimedia.org/T220897 (10Aklapper) Cannot reproduce in Firefox 66. Is this Opera? Which version? Please always include such information in bug reports. Any output in Opera's developer t... [00:35:12] maintenance-disconnect-full-disks build 63495 integration-slave-docker-1037: OFFLINE due to disk space [00:35:13] maintenance-disconnect-full-disks build 63495 integration-slave-docker-1050: OFFLINE due to disk space [00:35:26] 10Phabricator (Upstream), 10Upstream: "Choose subtype" box does not disappear after choosing an option when creating new subtask - https://phabricator.wikimedia.org/T220897 (10Aklapper) [01:00:15] maintenance-disconnect-full-disks build 63500 integration-slave-docker-1037: OFFLINE due to disk space [01:00:15] maintenance-disconnect-full-disks build 63500 integration-slave-docker-1050: OFFLINE due to disk space [01:25:14] maintenance-disconnect-full-disks build 63505 integration-slave-docker-1037: OFFLINE due to disk space [01:25:14] maintenance-disconnect-full-disks build 63505 integration-slave-docker-1050: OFFLINE due to disk space [01:31:32] 10Beta-Cluster-Infrastructure: Udev (?) problems using modified swift puppet classes on deployment-ms-be0[56] - https://phabricator.wikimedia.org/T220895 (10Krenair) Interestingly `sudo udevadm test /devices/virtual/block/dm-0` on 03 says `LINK 'swift/lv-a1' /etc/udev/rules.d/90-swift-lvm.rules:1` which is a see... [01:34:14] 10Beta-Cluster-Infrastructure: Udev (?) problems using modified swift puppet classes on deployment-ms-be0[56] - https://phabricator.wikimedia.org/T220895 (10Krenair) I bet we need a priority > 56 for our rule using DM_LV_NAME/DM_VG_NAME: `krenair@deployment-ms-be03:~$ grep DM_LV_NAME /lib/udev/rules.d/* /lib/ude... [01:38:06] 10Beta-Cluster-Infrastructure: Udev (?) problems using modified swift puppet classes on deployment-ms-be0[56] - https://phabricator.wikimedia.org/T220895 (10Krenair) a:03Krenair yeah [01:39:12] 10Beta-Cluster-Infrastructure: Udev (?) problems using modified swift puppet classes on deployment-ms-be0[56] - https://phabricator.wikimedia.org/T220895 (10Krenair) 04 also has another dupe of this with priority 99 [01:42:23] 10Phabricator (Upstream), 10Upstream: "Choose subtype" box does not disappear after choosing an option when creating new subtask - https://phabricator.wikimedia.org/T220897 (10Zoranzoki21) >>! In T220897#5109412, @Aklapper wrote: > Cannot reproduce in Firefox 66. Is this Opera? Which version? Please always inc... [01:50:13] maintenance-disconnect-full-disks build 63510 integration-slave-docker-1037: OFFLINE due to disk space [01:50:13] maintenance-disconnect-full-disks build 63510 integration-slave-docker-1050: OFFLINE due to disk space [02:05:20] 10Beta-Cluster-Infrastructure, 10Patch-For-Review: Udev (?) problems using modified swift puppet classes on deployment-ms-be0[56] - https://phabricator.wikimedia.org/T220895 (10Krenair) I just found https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/371642/5/modules/profile/manifests/swift/storage/labs.pp... [02:15:12] maintenance-disconnect-full-disks build 63515 integration-slave-docker-1037: OFFLINE due to disk space [02:15:13] maintenance-disconnect-full-disks build 63515 integration-slave-docker-1050: OFFLINE due to disk space [02:23:36] 10Beta-Cluster-Infrastructure, 10Patch-For-Review: Udev (?) problems using modified swift puppet classes on deployment-ms-be0[56] - https://phabricator.wikimedia.org/T220895 (10Krenair) Anyway with all of that out of the way, udev works, puppet works, and after a reboot there is `/dev/mapper/vd-lv--a1 113G... [02:40:12] maintenance-disconnect-full-disks build 63520 integration-slave-docker-1037: OFFLINE due to disk space [02:40:13] maintenance-disconnect-full-disks build 63520 integration-slave-docker-1050: OFFLINE due to disk space [03:05:12] maintenance-disconnect-full-disks build 63525 integration-slave-docker-1037: OFFLINE due to disk space [03:05:12] maintenance-disconnect-full-disks build 63525 integration-slave-docker-1050: OFFLINE due to disk space [03:30:12] maintenance-disconnect-full-disks build 63530 integration-slave-docker-1037: OFFLINE due to disk space [03:30:12] maintenance-disconnect-full-disks build 63530 integration-slave-docker-1050: OFFLINE due to disk space [03:55:12] maintenance-disconnect-full-disks build 63535 integration-slave-docker-1037: OFFLINE due to disk space [03:55:12] maintenance-disconnect-full-disks build 63535 integration-slave-docker-1050: OFFLINE due to disk space [04:20:12] maintenance-disconnect-full-disks build 63540 integration-slave-docker-1037: OFFLINE due to disk space [04:20:13] maintenance-disconnect-full-disks build 63540 integration-slave-docker-1050: OFFLINE due to disk space [04:45:12] maintenance-disconnect-full-disks build 63545 integration-slave-docker-1037: OFFLINE due to disk space [04:45:12] maintenance-disconnect-full-disks build 63545 integration-slave-docker-1050: OFFLINE due to disk space [05:10:12] maintenance-disconnect-full-disks build 63550 integration-slave-docker-1037: OFFLINE due to disk space [05:10:12] maintenance-disconnect-full-disks build 63550 integration-slave-docker-1050: OFFLINE due to disk space [05:35:12] maintenance-disconnect-full-disks build 63555 integration-slave-docker-1037: OFFLINE due to disk space [05:35:13] maintenance-disconnect-full-disks build 63555 integration-slave-docker-1050: OFFLINE due to disk space [06:00:12] maintenance-disconnect-full-disks build 63560 integration-slave-docker-1037: OFFLINE due to disk space [06:00:12] maintenance-disconnect-full-disks build 63560 integration-slave-docker-1050: OFFLINE due to disk space [06:25:15] maintenance-disconnect-full-disks build 63565 integration-slave-docker-1037: OFFLINE due to disk space [06:25:15] maintenance-disconnect-full-disks build 63565 integration-slave-docker-1050: OFFLINE due to disk space [06:50:12] maintenance-disconnect-full-disks build 63570 integration-slave-docker-1037: OFFLINE due to disk space [06:50:13] maintenance-disconnect-full-disks build 63570 integration-slave-docker-1050: OFFLINE due to disk space [07:15:13] maintenance-disconnect-full-disks build 63575 integration-slave-docker-1037: OFFLINE due to disk space [07:15:13] maintenance-disconnect-full-disks build 63575 integration-slave-docker-1050: OFFLINE due to disk space [07:40:12] maintenance-disconnect-full-disks build 63580 integration-slave-docker-1037: OFFLINE due to disk space [07:40:12] maintenance-disconnect-full-disks build 63580 integration-slave-docker-1050: OFFLINE due to disk space [07:59:19] 10Continuous-Integration-Config, 10MediaWiki-extensions-WebDAV, 10PHP 7.0 support: WebDAV master is broken on PHP7.* CI: Use of undefined constant WEBDAV_AUTH_TOKEN - https://phabricator.wikimedia.org/T220888 (10hashar) That extension used to pass CI tests. In mediawiki/core there is a recentish structure t... [08:05:12] maintenance-disconnect-full-disks build 63585 integration-slave-docker-1037: OFFLINE due to disk space [08:05:13] maintenance-disconnect-full-disks build 63585 integration-slave-docker-1050: OFFLINE due to disk space [08:08:57] 10Continuous-Integration-Config, 10MediaWiki-extensions-WebDAV: WebDAV master is broken on PHP7.* CI: Use of undefined constant WEBDAV_AUTH_TOKEN - https://phabricator.wikimedia.org/T220888 (10hashar) And indeed the WebDAV extension is quite old. That was an effort by Jack Bates then a student, it might have b... [08:30:12] maintenance-disconnect-full-disks build 63590 integration-slave-docker-1037: OFFLINE due to disk space [08:30:13] maintenance-disconnect-full-disks build 63590 integration-slave-docker-1050: OFFLINE due to disk space [08:55:12] maintenance-disconnect-full-disks build 63595 integration-slave-docker-1037: OFFLINE due to disk space [08:55:12] maintenance-disconnect-full-disks build 63595 integration-slave-docker-1050: OFFLINE due to disk space [09:20:12] maintenance-disconnect-full-disks build 63600 integration-slave-docker-1037: OFFLINE due to disk space [09:20:13] maintenance-disconnect-full-disks build 63600 integration-slave-docker-1050: OFFLINE due to disk space [09:35:03] (03PS1) 10Umherirrender: [WebDAV] Switch to composer based quibble [integration/config] - 10https://gerrit.wikimedia.org/r/503727 (https://phabricator.wikimedia.org/T220888) [09:35:07] 10Continuous-Integration-Config, 10MediaWiki-extensions-WebDAV, 10Patch-For-Review: WebDAV master is broken on PHP7.* CI: Use of undefined constant WEBDAV_AUTH_TOKEN - https://phabricator.wikimedia.org/T220888 (10Umherirrender) The repo of this extension was created in February, so I would not assume this is... [09:45:12] maintenance-disconnect-full-disks build 63605 integration-slave-docker-1037: OFFLINE due to disk space [09:45:13] maintenance-disconnect-full-disks build 63605 integration-slave-docker-1050: OFFLINE due to disk space [10:10:13] maintenance-disconnect-full-disks build 63610 integration-slave-docker-1037: OFFLINE due to disk space [10:10:13] maintenance-disconnect-full-disks build 63610 integration-slave-docker-1050: OFFLINE due to disk space [10:14:32] 10Release-Engineering-Team (Kanban), 10Release Pipeline, 10Patch-For-Review: Determine a standard way of installing MediaWiki lib/extension dependencies within containers - https://phabricator.wikimedia.org/T193824 (10Legoktm) In general I think that's a good idea. Two things I think we'll need to do: 1. Ad... [10:35:11] maintenance-disconnect-full-disks build 63615 integration-slave-docker-1037: OFFLINE due to disk space [10:35:12] maintenance-disconnect-full-disks build 63615 integration-slave-docker-1050: OFFLINE due to disk space [11:00:13] maintenance-disconnect-full-disks build 63620 integration-slave-docker-1037: OFFLINE due to disk space [11:00:14] maintenance-disconnect-full-disks build 63620 integration-slave-docker-1050: OFFLINE due to disk space [11:25:13] maintenance-disconnect-full-disks build 63625 integration-slave-docker-1037: OFFLINE due to disk space [11:25:13] maintenance-disconnect-full-disks build 63625 integration-slave-docker-1050: OFFLINE due to disk space [11:50:13] maintenance-disconnect-full-disks build 63630 integration-slave-docker-1037: OFFLINE due to disk space [11:50:13] maintenance-disconnect-full-disks build 63630 integration-slave-docker-1050: OFFLINE due to disk space [12:15:12] maintenance-disconnect-full-disks build 63635 integration-slave-docker-1037: OFFLINE due to disk space [12:15:13] maintenance-disconnect-full-disks build 63635 integration-slave-docker-1050: OFFLINE due to disk space [12:40:12] maintenance-disconnect-full-disks build 63640 integration-slave-docker-1037: OFFLINE due to disk space [12:40:13] maintenance-disconnect-full-disks build 63640 integration-slave-docker-1050: OFFLINE due to disk space [13:05:13] maintenance-disconnect-full-disks build 63645 integration-slave-docker-1037: OFFLINE due to disk space [13:05:14] maintenance-disconnect-full-disks build 63645 integration-slave-docker-1050: OFFLINE due to disk space [13:18:30] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [13:30:13] maintenance-disconnect-full-disks build 63650 integration-slave-docker-1037: OFFLINE due to disk space [13:30:13] maintenance-disconnect-full-disks build 63650 integration-slave-docker-1050: OFFLINE due to disk space [13:55:12] maintenance-disconnect-full-disks build 63655 integration-slave-docker-1037: OFFLINE due to disk space [13:55:12] maintenance-disconnect-full-disks build 63655 integration-slave-docker-1050: OFFLINE due to disk space [14:20:14] maintenance-disconnect-full-disks build 63660 integration-slave-docker-1037: OFFLINE due to disk space [14:20:15] maintenance-disconnect-full-disks build 63660 integration-slave-docker-1050: OFFLINE due to disk space [14:45:12] maintenance-disconnect-full-disks build 63665 integration-slave-docker-1037: OFFLINE due to disk space [14:45:12] maintenance-disconnect-full-disks build 63665 integration-slave-docker-1050: OFFLINE due to disk space [15:10:12] maintenance-disconnect-full-disks build 63670 integration-slave-docker-1037: OFFLINE due to disk space [15:10:12] maintenance-disconnect-full-disks build 63670 integration-slave-docker-1050: OFFLINE due to disk space [15:35:12] maintenance-disconnect-full-disks build 63675 integration-slave-docker-1037: OFFLINE due to disk space [15:35:12] maintenance-disconnect-full-disks build 63675 integration-slave-docker-1050: OFFLINE due to disk space [16:00:12] maintenance-disconnect-full-disks build 63680 integration-slave-docker-1037: OFFLINE due to disk space [16:00:13] maintenance-disconnect-full-disks build 63680 integration-slave-docker-1050: OFFLINE due to disk space [16:04:46] 10Phabricator (Upstream), 10Browser-Support-Opera, 10Upstream: "Choose subtype" box does not disappear in Opera 58 after choosing an option when creating new subtask - https://phabricator.wikimedia.org/T220897 (10Aklapper) p:05Triage→03Lowest [16:25:12] maintenance-disconnect-full-disks build 63685 integration-slave-docker-1037: OFFLINE due to disk space [16:25:12] maintenance-disconnect-full-disks build 63685 integration-slave-docker-1050: OFFLINE due to disk space [16:50:12] maintenance-disconnect-full-disks build 63690 integration-slave-docker-1037: OFFLINE due to disk space [16:50:13] maintenance-disconnect-full-disks build 63690 integration-slave-docker-1050: OFFLINE due to disk space [16:52:22] 10Continuous-Integration-Config: Update quibble jobs to use phpunit-patch-coverage 0.0.10 - https://phabricator.wikimedia.org/T220917 (10Reedy) [16:52:29] 10Continuous-Integration-Config: Update quibble jobs to use phpunit-patch-coverage 0.0.10 - https://phabricator.wikimedia.org/T220917 (10Reedy) p:05Triage→03High [16:53:03] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Update quibble jobs to use phpunit-patch-coverage 0.0.10 - https://phabricator.wikimedia.org/T220917 (10Reedy) [16:54:26] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Update quibble jobs to use phpunit-patch-coverage 0.0.10 - https://phabricator.wikimedia.org/T220917 (10Reedy) [17:15:12] maintenance-disconnect-full-disks build 63695 integration-slave-docker-1037: OFFLINE due to disk space [17:15:12] maintenance-disconnect-full-disks build 63695 integration-slave-docker-1050: OFFLINE due to disk space [17:29:52] Hello! Could someone please tell me where can I find the php.ini used in CI? Thanks! [17:32:22] Daimona: It's probably whatever comes out of the apt packages... [17:32:47] Reedy thanks. And what if we have to change it? [17:32:54] What're you wanting to change? :P [17:33:37] Linking the task in a minute :) [17:33:47] Answer is possibly in operations/puppet [17:34:51] 10Continuous-Integration-Config: apc.enable_cli should be enabled in PHP jobs - https://phabricator.wikimedia.org/T220920 (10Daimona) [17:34:51] https://phabricator.wikimedia.org/T220920 [17:34:59] Eh darn bot [17:37:50] is it apc. or opcache.? [17:39:52] apc [17:40:13] maintenance-disconnect-full-disks build 63700 integration-slave-docker-1037: OFFLINE due to disk space [17:40:13] maintenance-disconnect-full-disks build 63700 integration-slave-docker-1050: OFFLINE due to disk space [17:40:53] I thought it was opcache in PHP7? [17:42:18] PHP7 has APCU, which we use, and is the replacement of APC [17:43:22] As CACHE_ACCEL I mean [18:05:13] maintenance-disconnect-full-disks build 63705 integration-slave-docker-1037: OFFLINE due to disk space [18:05:13] maintenance-disconnect-full-disks build 63705 integration-slave-docker-1050: OFFLINE due to disk space [18:25:53] Argument 1 passed to PhabricatorSearchField::getValueExistsInConduitRequest() must be of the type array, string given, called in /srv/deployment/phabricator/deployment/phabricator/src/applications/search/engine/PhabricatorApplicationSearchEngine.php on line 1171 and defined [18:26:35] this happens when I call https://phabricator.wikimedia.org/api/maniphest.search with "constraints": {"statuses": ["resolved"]}, [18:30:04] Krenair: Probably best just filing it in phab [18:30:12] maintenance-disconnect-full-disks build 63710 integration-slave-docker-1037: OFFLINE due to disk space [18:30:12] maintenance-disconnect-full-disks build 63710 integration-slave-docker-1050: OFFLINE due to disk space [18:50:00] true [18:55:12] maintenance-disconnect-full-disks build 63715 integration-slave-docker-1037: OFFLINE due to disk space [18:55:13] maintenance-disconnect-full-disks build 63715 integration-slave-docker-1050: OFFLINE due to disk space [19:20:14] maintenance-disconnect-full-disks build 63720 integration-slave-docker-1037: OFFLINE due to disk space [19:20:14] maintenance-disconnect-full-disks build 63720 integration-slave-docker-1050: OFFLINE due to disk space [19:34:21] 10Continuous-Integration-Config: apc.enable_cli should be enabled in PHP jobs - https://phabricator.wikimedia.org/T220920 (10Legoktm) IIRC apc has a history of causing weird bugs in CI with stuff not being invalidated properly. I don't know of those bugs still exist, but if you want a persistent cache to test wi... [19:45:14] maintenance-disconnect-full-disks build 63725 integration-slave-docker-1037: OFFLINE due to disk space [19:45:15] maintenance-disconnect-full-disks build 63725 integration-slave-docker-1050: OFFLINE due to disk space [20:01:08] 10Continuous-Integration-Config: apc.enable_cli should be enabled in PHP jobs - https://phabricator.wikimedia.org/T220920 (10Daimona) Maybe it has to do with the comment on top of the APCUBagOStuff file, mentioning bugs when serialization is set to default? [20:10:13] maintenance-disconnect-full-disks build 63730 integration-slave-docker-1037: OFFLINE due to disk space [20:10:13] maintenance-disconnect-full-disks build 63730 integration-slave-docker-1050: OFFLINE due to disk space [20:35:13] maintenance-disconnect-full-disks build 63735 integration-slave-docker-1037: OFFLINE due to disk space [20:35:13] maintenance-disconnect-full-disks build 63735 integration-slave-docker-1050: OFFLINE due to disk space [21:00:12] maintenance-disconnect-full-disks build 63740 integration-slave-docker-1037: OFFLINE due to disk space [21:00:13] maintenance-disconnect-full-disks build 63740 integration-slave-docker-1050: OFFLINE due to disk space [21:25:14] maintenance-disconnect-full-disks build 63745 integration-slave-docker-1037: OFFLINE due to disk space [21:25:14] maintenance-disconnect-full-disks build 63745 integration-slave-docker-1050: OFFLINE due to disk space [21:50:13] maintenance-disconnect-full-disks build 63750 integration-slave-docker-1037: OFFLINE due to disk space [21:50:13] maintenance-disconnect-full-disks build 63750 integration-slave-docker-1050: OFFLINE due to disk space [22:15:13] maintenance-disconnect-full-disks build 63755 integration-slave-docker-1037: OFFLINE due to disk space [22:15:13] maintenance-disconnect-full-disks build 63755 integration-slave-docker-1050: OFFLINE due to disk space [22:40:16] maintenance-disconnect-full-disks build 63760 integration-slave-docker-1037: OFFLINE due to disk space [22:40:17] maintenance-disconnect-full-disks build 63760 integration-slave-docker-1050: OFFLINE due to disk space [23:05:13] maintenance-disconnect-full-disks build 63765 integration-slave-docker-1037: OFFLINE due to disk space [23:05:13] maintenance-disconnect-full-disks build 63765 integration-slave-docker-1050: OFFLINE due to disk space [23:30:12] maintenance-disconnect-full-disks build 63770 integration-slave-docker-1037: OFFLINE due to disk space [23:30:12] maintenance-disconnect-full-disks build 63770 integration-slave-docker-1050: OFFLINE due to disk space [23:55:12] maintenance-disconnect-full-disks build 63775 integration-slave-docker-1037: OFFLINE due to disk space [23:55:12] maintenance-disconnect-full-disks build 63775 integration-slave-docker-1050: OFFLINE due to disk space