[00:21:36] System Info: Model: iPhone X · iOS: 11.2.5 · RAM: 2.72 GB · Disk Space: 59.55 GB · GPU: Apple Custom GPU · Screen Resolution (Portrait): 2436x1125 · Battery: 100% · Uptime: 10d · Client: Mutter 1.20.2 (Build 1800721) [01:24:52] 10Gerrit, 10BlueSpice, 10GitHub-Mirrors, 10Repository-Admins: extension-BlueSpiceMenues is empty - https://phabricator.wikimedia.org/T185662#3981274 (10SamanthaNguyen) @MarcoAurelio: Unfortunately I'm not sure, sorry. :/ I could try looking around for evidence on MediaWiki.org (maybe that new repositories... [02:54:39] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<11.11%) [03:13:24] 03:12:19 + docker run --rm --user=nobody -v /srv/jenkins-workspace/workspace/doxygen-publish:/workspace --entrypoint=/usr/bin/find docker-registry.wikimedia.org/wikimedia-stretch:latest /workspace/log -mindepth 1 -delete [03:13:24] 03:12:19 container_linux.go:262: starting container process caused "process_linux.go:261: applying cgroup configuration for process caused \"failed to write 4013 to cgroup.procs: write /sys/fs/cgroup/cpu,cpuacct/docker/16721df7d9033a541a92c58973a87c46414bca709bb6314645e44f755210bd1d/cgroup.procs: invalid argument\"" [03:13:24] 03:12:19 docker: Error response from daemon: oci runtime error: container_linux.go:262: starting container process caused "process_linux.go:261: applying cgroup configuration for process caused \"failed to write 4013 to cgroup.procs: write /sys/fs/cgroup/cpu,cpuacct/docker/16721df7d9033a541a92c58973a87c46414bca709bb6314645e44f755210bd1d/cgroup.procs: invalid argument\"". [04:20:33] Project mwext-phpunit-coverage-publish build #1146: 04FAILURE in 42 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/1146/ [04:39:30] Yippee, build fixed! [04:39:31] Project mwext-phpunit-coverage-publish build #1147: 09FIXED in 57 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/1147/ [04:51:29] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [10.0] [05:21:33] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [07:24:40] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [11:06:48] PROBLEM - Free space - all mounts on deployment-eventlog02 is CRITICAL: CRITICAL: deployment-prep.deployment-eventlog02.diskspace.root.byte_percentfree (<30.00%) [15:27:44] PROBLEM - Puppet errors on deployment-redis01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:33:42] PROBLEM - Puppet errors on deployment-redis02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [18:39:24] PROBLEM - Puppet errors on deployment-mx is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [19:07:30] RECOVERY - Mediawiki Error Rate on graphite-labs is OK: OK: Less than 1.00% above the threshold [1.0] [19:55:04] 10Differential, 10Analytics-Tech-community-metrics, 10Developer-Relations, 10DevRel-April-2016, 10DevRel-March-2016: Make MetricsGrimoire/korma support gathering Code Review statistics from Phabricator's Differential - https://phabricator.wikimedia.org/T118753#3981789 (10Aklapper) 05Open>03stalled p:... [20:12:56] 10Beta-Cluster-Infrastructure, 10MediaWiki-Maintenance-scripts, 10Privacy, 10Security, and 2 others: 'recentchanges' table on Beta Cluster wikis storing 'rc_ip' data older than 90 days - https://phabricator.wikimedia.org/T187656#3981821 (10MarcoAurelio) [20:13:13] 10Beta-Cluster-Infrastructure, 10MediaWiki-Maintenance-scripts, 10Privacy, 10Security, and 2 others: 'recentchanges' table on Beta Cluster wikis storing 'rc_ip' data older than 90 days - https://phabricator.wikimedia.org/T187656#3981831 (10MarcoAurelio) p:05Triage>03High [20:18:29] 10Phabricator (Upstream), 10Upstream: Add notification setting for pings - https://phabricator.wikimedia.org/T92860#3981849 (10Aklapper) 05Open>03declined `X-Phabricator-Mail-Stamps` are now available, see https://phabricator.wikimedia.org/T150766#3976238 So you could //filter// emails at least and throw a... [20:18:37] 10Beta-Cluster-Infrastructure, 10Operations, 10Puppet: Setup cron for foreachwikiindblist all-labs.dblist extensions/AbuseFilter/maintenance/purgeOldLogIPData.php on Beta - https://phabricator.wikimedia.org/T187658#3981851 (10MarcoAurelio) [20:22:23] 10Beta-Cluster-Infrastructure, 10MediaWiki-Maintenance-scripts, 10Privacy, 10Security, and 2 others: 'recentchanges' table on Beta Cluster wikis storing 'rc_ip' data older than 90 days - https://phabricator.wikimedia.org/T187656#3981862 (10MarcoAurelio) 05Open>03Invalid Sorry, wrong reading of the data... [20:23:19] 10Phabricator, 10WMSE-Bug-Reporting-and-Translation-2018: Trying to import workboard for some projects give error 400 - https://phabricator.wikimedia.org/T186189#3936461 (10Aklapper) There is a workboard now at https://phabricator.wikimedia.org/project/board/2480/ ? So I'm afraid the test case is gone here? [20:24:31] 10Phabricator: Rename @Gochiusa to @Hoto_Cocoa in Phabricator - https://phabricator.wikimedia.org/T182284#3981870 (10Aklapper) 05Open>03stalled p:05Triage>03Lowest Task blocked on reply from original reporter [20:25:53] 10Phabricator: Investigate why some widgets are not displayed in the search results for widgets to add to a dashboard - https://phabricator.wikimedia.org/T171889#3981876 (10Aklapper) [20:27:42] 10Phabricator, 10WMNO-General: Herald rule for WMNO - https://phabricator.wikimedia.org/T187539#3981879 (10Aklapper) 05Open>03Resolved a:03Aklapper Created H279 [20:37:39] !log Ran foreachwikiindblist all-labs.dblist extensions/AbuseFilter/maintenance/purgeOldLogIPData.php on Beta [20:37:45] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:39:02] PROBLEM - Puppet errors on deployment-mediawiki07 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [20:46:37] 10Beta-Cluster-Infrastructure: Spam/Spambot registration on deployment.beta is out of hand - https://phabricator.wikimedia.org/T187046#3981904 (10MarcoAurelio) ~80 accounts locked today, and similar batches in the previous days. Are we going to do something? [22:03:49] Project mwext-phpunit-coverage-publish build #1186: 04FAILURE in 39 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/1186/ [22:04:33] Yippee, build fixed! [22:04:34] Project mwext-phpunit-coverage-publish build #1187: 09FIXED in 43 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/1187/