[00:23:38] (03CR) 10Jforrester: "Thanks!" [integration/config] - 10https://gerrit.wikimedia.org/r/568600 (https://phabricator.wikimedia.org/T237707) (owner: 10Krinkle) [02:51:10] PROBLEM - Free space - all mounts on deployment-snapshot01 is CRITICAL: CRITICAL: deployment-prep.deployment-snapshot01.diskspace._data.byte_percentfree (No valid datapoints found)deployment-prep.deployment-snapshot01.diskspace.root.byte_percentfree (<11.11%) [03:01:09] RECOVERY - Free space - all mounts on deployment-snapshot01 is OK: OK: deployment-prep.deployment-snapshot01.diskspace._data.byte_percentfree (No valid datapoints found) [08:26:01] 10Continuous-Integration-Config, 10Wikidata, 10Wikidata-Campsite: Wikibase post-merge builds are failing - https://phabricator.wikimedia.org/T242617 (10Addshore) Looks like these are still failing https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-docker-publish/17540/console Still the same err... [09:58:10] (03PS1) 10Kosta Harlan: (wip/poc/avert your eyes): quibble drives docker [integration/quibble] - 10https://gerrit.wikimedia.org/r/568931 (https://phabricator.wikimedia.org/T234902) [09:58:41] (03PS2) 10Kosta Harlan: (wip/poc/avert your eyes): quibble drives docker [integration/quibble] - 10https://gerrit.wikimedia.org/r/568931 (https://phabricator.wikimedia.org/T234902) [09:59:35] (03CR) 10jerkins-bot: [V: 04-1] (wip/poc/avert your eyes): quibble drives docker [integration/quibble] - 10https://gerrit.wikimedia.org/r/568931 (https://phabricator.wikimedia.org/T234902) (owner: 10Kosta Harlan) [10:00:58] (03CR) 10Kosta Harlan: "I can clean this up over the coming weeks if there is interest in using this approach in CI; otherwise it was just an interesting experime" [integration/quibble] - 10https://gerrit.wikimedia.org/r/568931 (https://phabricator.wikimedia.org/T234902) (owner: 10Kosta Harlan) [10:32:08] PROBLEM - Free space - all mounts on deployment-snapshot01 is CRITICAL: CRITICAL: deployment-prep.deployment-snapshot01.diskspace._data.byte_percentfree (No valid datapoints found)deployment-prep.deployment-snapshot01.diskspace.root.byte_percentfree (<11.11%) [10:42:07] RECOVERY - Free space - all mounts on deployment-snapshot01 is OK: OK: deployment-prep.deployment-snapshot01.diskspace._data.byte_percentfree (No valid datapoints found) [10:58:19] o/ I'm trying to track down why some messages we've changed in master (of Wikibase) aren't being reflected on beta [10:58:27] I noticed errors like this: [10:58:38] https://www.irccloud.com/pastebin/E6ARZded/ [10:58:54] in the beta-scap-eqiad jobs [10:59:07] are those normal? [11:01:50] I dont think that is normla! [11:02:07] :D [11:30:20] PROBLEM - App Server Main HTTP Response on deployment-mediawiki-09 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [11:45:10] RECOVERY - App Server Main HTTP Response on deployment-mediawiki-09 is OK: HTTP OK: HTTP/1.1 200 OK - 48521 bytes in 0.795 second response time [13:28:01] (03Abandoned) 10Kosta Harlan: Add php-fpm/apache image for use with MediaWiki Dev Env [releng/dev-images] - 10https://gerrit.wikimedia.org/r/555914 (https://phabricator.wikimedia.org/T238224) (owner: 10Kosta Harlan) [13:59:33] 10Release-Engineering-Team, 10Release-Engineering-Team-TODO, 10Pywikibot: Share Appveyor account credentials with Release Engineering - https://phabricator.wikimedia.org/T104306 (10Dvorapa) Currently low priority as we managed to have team account at @Ladsgroup account [13:59:54] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10GitHub-Mirrors, and 2 others: AppVeyor test not running since months - https://phabricator.wikimedia.org/T183860 (10Dvorapa) p:05Normal→03Low Currently low priority as we managed to h... [14:17:19] o/ greg-g We would like to backport some things fixing 2 bugs in our wb_terms table related migration, according to Amir1 I might seek approval from you! [14:18:19] it might be a bit early yet (6:20 am in SF) [14:18:21] context https://phabricator.wikimedia.org/T243944 the ticket itself is high prio but avoiding more holes is UBN [14:18:37] very UBN :D [14:19:00] apergos: timezones always hurt me :) [14:19:08] me too :-D [14:19:23] I remember scheduling a lunch while in the UK with someone in Berlin, and then they turned up 1 hour early :D [14:19:45] were they an hour early or were you an hour late? :-D [14:19:50] in the email that google sent them it said something like 12:00 United Kingdom time, or something [14:19:52] hour early [14:20:10] i should have created the event pinned to to berlin time i guess [14:20:13] so partly my bad [14:20:21] but then the email did say United Kingdom time [14:20:38] uh ohes :-D [14:20:48] it should have said UCT of course [14:21:01] just keep countries out of it altogether! [15:35:48] 10Phabricator: Create new custom fundraising form - https://phabricator.wikimedia.org/T243961 (10DStrine) [15:56:31] 10Phabricator: Create new custom fundraising form - https://phabricator.wikimedia.org/T243961 (10Aklapper) a:0520after4→03None See https://www.mediawiki.org/wiki/Phabricator/Help/Forms linking to https://phabricator.wikimedia.org/maniphest/task/edit/form/17/?projects=Phabricator for info to provide in such r... [16:18:03] PROBLEM - SSH on deployment-ores01 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [16:22:51] RECOVERY - SSH on deployment-ores01 is OK: SSH OK - OpenSSH_7.4p1 Debian-10+deb9u6 (protocol 2.0) [17:00:41] 10Continuous-Integration-Config, 10BlueSpice, 10Patch-For-Review: Enable unit tests on BlueSpice* repos - https://phabricator.wikimedia.org/T130811 (10Pwirth) [17:06:24] 10Phabricator: Create new custom fundraising form - https://phabricator.wikimedia.org/T243961 (10Masumrezarock100) [17:47:52] PROBLEM - Parsoid on deployment-mediawiki-parsoid10 is CRITICAL: connect to address 172.16.0.141 and port 8000: Connection refused [17:47:52] PROBLEM - Parsoid on deployment-parsoid09 is CRITICAL: connect to address 172.16.5.63 and port 8000: Connection refused [19:16:04] 10Release-Engineering-Team, 10Recommendation-API, 10Services, 10serviceops, 10Patch-For-Review: Migrate recommendation-api to kubernetes - https://phabricator.wikimedia.org/T241230 (10Mholloway) [19:46:12] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations: Create Gerrit Administrator right policy - https://phabricator.wikimedia.org/T218686 (10JZ19775) [19:47:24] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations: Create Gerrit Administrator right policy - https://phabricator.wikimedia.org/T218686 (10Peachey88) [23:40:41] 10Phabricator, 10Release-Engineering-Team-TODO (2020-01 to 2020-03 (Q3)), 10Operations, 10Traffic, 10serviceops: Phabricator downtime due to aphlict and websockets (aphlict current disabled) - https://phabricator.wikimedia.org/T238593 (10mmodell) questions: why are there 2 yaml files for apache traffic... [23:41:04] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (2020-01 to 2020-03 (Q3)), 10Core Platform Team, 10CPT Initiatives (API Integration Tests), 10Code-Health: Run API integration tests suite in CI on core repo without extensions - https://phabricator.wikimedia.org/T243975 (10daniel) [23:44:07] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (2020-01 to 2020-03 (Q3)), 10Core Platform Team, 10CPT Initiatives (API Integration Tests), 10Code-Health: Run API integration tests suite in CI on core repo without extensions - https://phabricator.wikimedia.org/T243975 (10daniel) [23:47:25] 10Continuous-Integration-Config, 10Core Platform Team, 10CPT Initiatives (API Integration Tests), 10Code-Health: API integration tests should skip tests accourding to flags exposed by the SiteInfo API module - https://phabricator.wikimedia.org/T243976 (10daniel) [23:51:17] 10Release-Engineering-Team, 10Core Platform Team, 10CPT Initiatives (API Integration Tests), 10Code-Health: Expose flags for controlling API tests from the siteinfo API module - https://phabricator.wikimedia.org/T243977 (10daniel) [23:52:25] 10Continuous-Integration-Config, 10Core Platform Team, 10CPT Initiatives (API Integration Tests), 10Code-Health: API integration tests should skip tests accourding to flags exposed by the SiteInfo API module - https://phabricator.wikimedia.org/T243976 (10daniel) [23:54:27] 10Continuous-Integration-Config, 10Release-Engineering-Team-TODO (2020-01 to 2020-03 (Q3)), 10CPT Initiatives (API Integration Tests), 10Code-Health, and 2 others: In CI, configure MediaWiki to enable API testing - https://phabricator.wikimedia.org/T243978 (10daniel)