[00:20:22] Project beta-update-databases-eqiad build #17352: 04STILL FAILING in 21 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17352/ [00:22:53] greg-g: just wanted to let you know i forwarded your email to wikitech-l to wikipedia-l and ambassadors-l [01:20:25] Project beta-update-databases-eqiad build #17353: 04STILL FAILING in 25 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17353/ [01:28:25] !log Update mobileapps to db6493c [01:28:29] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [01:28:43] Jouncebot refresh [01:29:05] Is jouncebot dead? [01:30:02] Im stupid [01:30:04] Wrong channel [02:20:26] Project beta-update-databases-eqiad build #17354: 04STILL FAILING in 25 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17354/ [03:43:43] 10MediaWiki-Releasing, 10MediaWiki-Containers, 06Services, 15User-mobrovac: Ready-to-use Docker package for MediaWiki - https://phabricator.wikimedia.org/T92826#1140491 (10mschwarzer) >>! In T92826#3034057, @Dan.mulholland wrote: > ... > # adding containers for CirrusSearch/Elasticsearch, Graphoid, Mathoid... [03:47:17] (03CR) 10Kaldari: [C: 031] "I'm fine with the current implementation if others are. It's an improvement either way." [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/353623 (https://phabricator.wikimedia.org/T164649) (owner: 10Legoktm) [03:59:34] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<50.00%) [04:05:00] Project beta-update-databases-eqiad build #17355: 15ABORTED in 45 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17355/ [04:12:36] Project selenium-MultimediaViewer » firefox,beta,Linux,BrowserTests build #403: 04FAILURE in 16 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/403/ [04:21:00] Project beta-update-databases-eqiad build #17356: 04STILL FAILING in 1 min 0 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17356/ [04:32:33] 10Gerrit, 06Developer-Relations, 10GitHub-Mirrors, 06Repository-Admins: Add CODE_OF_CONDUCT.md to Wikimedia projects - https://phabricator.wikimedia.org/T165540#3269385 (10Mattflaschen-WMF) >>! In T165540#3273784, @Legoktm wrote: > I'm assuming this file is going to be as simple as "The development of Medi... [04:37:33] 10Gerrit, 06Developer-Relations, 10GitHub-Mirrors, 06Repository-Admins: Add CODE_OF_CONDUCT.md to Wikimedia projects - https://phabricator.wikimedia.org/T165540#3293800 (10Mattflaschen-WMF) Except "This project", not "The development of MediaWiki", since it applies regardless of what project it is. [05:04:50] 06Release-Engineering-Team (Kanban), 06Reading-Admin, 05Deployment Blockers, 05Release: MW-1.30.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T163512#3199999 (10KartikMistry) @greg cawiki and hewiki are on wmf.2, if needed they also need to revert. [05:20:54] Project beta-update-databases-eqiad build #17357: 04STILL FAILING in 54 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17357/ [06:21:00] Project beta-update-databases-eqiad build #17358: 04STILL FAILING in 1 min 0 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17358/ [06:58:47] Project selenium-Wikibase » chrome,beta,Linux,BrowserTests build #372: 04FAILURE in 2 hr 18 min: https://integration.wikimedia.org/ci/job/selenium-Wikibase/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/372/ [07:04:37] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [07:20:35] !log hacking on jobrunner02 in deployment-prep [07:20:38] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:20:58] Project beta-update-databases-eqiad build #17359: 04STILL FAILING in 58 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17359/ [07:46:41] 06Release-Engineering-Team (Kanban), 06Reading-Admin, 05Deployment Blockers, 05Release: MW-1.30.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T163512#3293969 (10hashar) >>! In T163512#3293804, @KartikMistry wrote: > @greg cawiki and hewiki are on wmf.2, if needed they also need to revert.... [08:21:00] Project beta-update-databases-eqiad build #17360: 04STILL FAILING in 1 min 0 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17360/ [09:18:52] 06Release-Engineering-Team (Kanban), 06Performance-Team, 10Phabricator, 15User-greg: Create Performance phame blog - https://phabricator.wikimedia.org/T166110#3294109 (10Gilles) 05Open>03Resolved It does, thanks. [09:20:29] !log installing hhvm_3.18.2+dfsg-1+wmf4+exp1_amd64.deb on jobrunner02 [09:20:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:21:04] Project beta-update-databases-eqiad build #17361: 04STILL FAILING in 1 min 3 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17361/ [09:22:58] PROBLEM - Puppet staleness on deployment-prometheus01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0] [09:27:49] 06Release-Engineering-Team (Kanban), 06Performance-Team, 10Phabricator, 06Project-Admins: Enable storage and display of OGV and WEBM videos on Phabricator - https://phabricator.wikimedia.org/T166112#3294116 (10Gilles) Fair enough, re: our information leaking to google. I think that's a clear deal breaker.... [09:53:28] 06Release-Engineering-Team (Kanban), 06Performance-Team, 10Phabricator: Custom domain/URL for performance blog - https://phabricator.wikimedia.org/T166374#3294149 (10Gilles) [09:57:50] 06Release-Engineering-Team (Kanban), 06Performance-Team, 10Phabricator: Custom domain/URL for phame performance blog - https://phabricator.wikimedia.org/T166374#3294166 (10Gilles) [10:20:54] Project beta-update-databases-eqiad build #17362: 04STILL FAILING in 54 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17362/ [10:22:26] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [10:45:55] PROBLEM - Puppet errors on deployment-imagescaler01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [10:59:32] PROBLEM - Puppet errors on deployment-pdf01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [11:20:56] Project beta-update-databases-eqiad build #17363: 04STILL FAILING in 56 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17363/ [11:55:32] 10Gerrit, 06Release-Engineering-Team (Kanban), 10MediaWiki-extensions-Other, 06Repository-Admins, and 2 others: Archive PageLanguageApi extension - https://phabricator.wikimedia.org/T160371#3294293 (10Mainframe98) [12:01:01] (03Abandoned) 10Zfilipin: 0.7.1: Increase log verbosity [ruby/api] - 10https://gerrit.wikimedia.org/r/304331 (https://phabricator.wikimedia.org/T142600) (owner: 10Gergő Tisza) [12:01:34] (03Abandoned) 10Zfilipin: 1.7.3: Add API log level to environment settings [selenium] - 10https://gerrit.wikimedia.org/r/304332 (https://phabricator.wikimedia.org/T142600) (owner: 10Gergő Tisza) [12:17:33] 10MediaWiki-Codesniffer: Question about sniff about Visibility must be declared on property - https://phabricator.wikimedia.org/T166381#3294334 (10Umherirrender) [12:20:58] Project beta-update-databases-eqiad build #17364: 04STILL FAILING in 58 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17364/ [12:25:02] 06Release-Engineering-Team, 06Language-Team, 06MediaWiki-Platform-Team, 10MediaWiki-extensions-WikimediaIncubator, and 2 others: Make creating a new Language project easier - https://phabricator.wikimedia.org/T165585#3294348 (10Benoit_Rochon) Hello. I agree with @Amqui, translatewiki.net should take advant... [13:21:11] Project beta-update-databases-eqiad build #17365: 04STILL FAILING in 1 min 11 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17365/ [13:27:41] RainbowSprinkles: o/ - I tried to write down my ramblings in https://phabricator.wikimedia.org/T125735#3294144 (connect timeouts to redis) - let me know what you think (if you have time) [13:29:35] 06Release-Engineering-Team (Kanban), 10Scap: scap: Investigate scaps handling of InitialiseSettings.php - https://phabricator.wikimedia.org/T166384#3294452 (10thcipriani) [13:30:00] 06Release-Engineering-Team (Kanban), 10Scap: scap: Investigate scaps handling of InitialiseSettings.php - https://phabricator.wikimedia.org/T166384#3294465 (10thcipriani) p:05Triage>03Normal [13:46:52] Yippee, build fixed! [13:46:53] Project selenium-VisualEditor » firefox,beta,Linux,BrowserTests build #409: 09FIXED in 2 min 52 sec: https://integration.wikimedia.org/ci/job/selenium-VisualEditor/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/409/ [14:20:54] Project beta-update-databases-eqiad build #17366: 04STILL FAILING in 54 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17366/ [14:39:35] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07Documentation, 07Easy, and 3 others: audit/update headers in files - https://phabricator.wikimedia.org/T69141#3294533 (10zeljkofilipin) [14:44:09] 06Release-Engineering-Team (Kanban), 06Collaboration-Team-Triage, 10Flow, 10MediaWiki-Vagrant, and 2 others: Ruby::Bundle fails on Jessie MediaWiki-Vagrant - https://phabricator.wikimedia.org/T161140#3294546 (10zeljkofilipin) a:03zeljkofilipin [14:45:06] (03CR) 10Krinkle: [C: 031] Disallow `and` and `or` [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/355182 (https://phabricator.wikimedia.org/T143888) (owner: 10Legoktm) [14:48:25] 06Release-Engineering-Team (Kanban), 06Collaboration-Team-Triage, 10Flow, 10MediaWiki-Vagrant, 15User-zeljkofilipin: Ruby::Bundle fails on Jessie MediaWiki-Vagrant - https://phabricator.wikimedia.org/T161140#3294556 (10zeljkofilipin) [14:53:01] 06Release-Engineering-Team (Kanban), 06Collaboration-Team-Triage, 10Flow, 10MediaWiki-Vagrant, 15User-zeljkofilipin: Ruby::Bundle fails on Jessie MediaWiki-Vagrant - https://phabricator.wikimedia.org/T161140#3294561 (10zeljkofilipin) 05Open>03Resolved Provisioning flow role works fine now at my machi... [14:59:07] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07Documentation, 07Easy, and 2 others: audit/update headers in files - https://phabricator.wikimedia.org/T69141#713980 (10zeljkofilipin) [15:02:26] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07Documentation, 07Easy, and 2 others: audit/update headers in files - https://phabricator.wikimedia.org/T69141#3294575 (10zeljkofilipin) 05Open>03Resolved This is resolved, as far as I can see: https://github.com/search?q=org%3Awik... [15:05:10] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07Easy: Use rspec-expectations "expect" syntax instead of "should" syntax - https://phabricator.wikimedia.org/T68369#3294580 (10zeljkofilipin) 05Open>03declined Because of {T139740} this is unlikely to ever get done. [15:07:29] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team, 06Reading-Web-Backlog, 07Ruby, 15User-zeljkofilipin: Run subset of MobileFrontend browser tests on merges in core - https://phabricator.wikimedia.org/T165940#3294586 (10zeljkofilipin) [15:07:31] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 13Patch-For-Review, 07Ruby, 15User-zeljkofilipin: mediawiki_selenium raises MediawikiSelenium::ConfigurationError if a configuration in environments.yml is empty string - https://phabricator.wikimedia.org/T128860#3294588 (10zeljkofilipi... [15:07:33] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 13Patch-For-Review, 07Ruby: New version of page-object Ruby gem is breaking CI - https://phabricator.wikimedia.org/T126354#3294589 (10zeljkofilipin) [15:07:35] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Next), 10Wikidata, 13Patch-For-Review, and 3 others: Increase in failures caused by Saucelabs - https://phabricator.wikimedia.org/T152963#3294587 (10zeljkofilipin) [15:07:37] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 10Wikidata, 07Ruby: headless browsertests: browser console output - https://phabricator.wikimedia.org/T118182#3294590 (10zeljkofilipin) [15:07:41] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 10Wikidata, 07Ruby: headless browsertests: assert absence of or log network errors - https://phabricator.wikimedia.org/T118179#3294591 (10zeljkofilipin) [15:07:44] 10Beta-Cluster-Infrastructure, 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07Ruby, and 2 others: Make selenium users use botflags at beta-cluster - https://phabricator.wikimedia.org/T116027#3294592 (10zeljkofilipin) [15:07:46] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07Easy, 07Ruby: Benchmark WebDriver via Browserstack for performance increase over SauceLabs - https://phabricator.wikimedia.org/T102282#3294594 (10zeljkofilipin) [15:07:48] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07Easy, 07Ruby: Remove regular expressions in browser tests page element selectors - https://phabricator.wikimedia.org/T112291#3294593 (10zeljkofilipin) [15:07:50] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07Documentation, 07Easy, and 3 others: Ruby gem documentation should state license - https://phabricator.wikimedia.org/T94001#3294595 (10zeljkofilipin) [15:07:52] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07Ruby: Remove "sleep" from tests - https://phabricator.wikimedia.org/T48887#3294597 (10zeljkofilipin) [15:07:54] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Watching / External), 06Wikipedia-Android-App-Backlog, 06Wikipedia-iOS-App-Backlog, 07Ruby: Create end-to-end automated test for Wikipedia native app(s) - https://phabricator.wikimedia.org/T90177#3294596 (10zeljkofilipin) [15:12:32] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 13Patch-For-Review, 07Ruby, 15User-zeljkofilipin: mediawiki_selenium raises MediawikiSelenium::ConfigurationError if a configuration in environments.yml is empty string - https://phabricator.wikimedia.org/T128860#3294639 (10zeljkofilipi... [15:20:56] Project beta-update-databases-eqiad build #17367: 04STILL FAILING in 56 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17367/ [15:29:37] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 13Patch-For-Review, 07Ruby: New version of page-object Ruby gem is breaking CI - https://phabricator.wikimedia.org/T126354#3294668 (10zeljkofilipin) p:05Normal>03Low [15:29:58] 10Beta-Cluster-Infrastructure, 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07Ruby, and 2 others: Make selenium users use botflags at beta-cluster - https://phabricator.wikimedia.org/T116027#3294669 (10zeljkofilipin) p:05Normal>03Low [15:30:40] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Watching / External), 06Wikipedia-Android-App-Backlog, 06Wikipedia-iOS-App-Backlog, 07Ruby: Create end-to-end automated test for Wikipedia native app(s) - https://phabricator.wikimedia.org/T90177#3294672 (10zeljkofilipin) p:05Normal>03Low [15:32:55] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Kanban), 13Patch-For-Review, 07WorkType-Maintenance: It takes about 20 seconds just to start a Sauce Labs browser - https://phabricator.wikimedia.org/T92613#3294690 (10zeljkofilipin) [15:32:57] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07Easy, 07Ruby: Remove regular expressions in browser tests page element selectors - https://phabricator.wikimedia.org/T112291#3294685 (10zeljkofilipin) 05Open>03declined p:05Normal>03Low Unlikely to ever be resolved because of {... [15:34:31] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07Ruby: Remove "sleep" from tests - https://phabricator.wikimedia.org/T48887#3294696 (10zeljkofilipin) 05Open>03declined Unlikely to ever be resolved because of {T139740}. [15:37:00] elukey: Wow. Assuming your observations are correct, that summary basically describes what we've been seeing the last ~year. [15:37:16] "I suspect that HHVM is not able to re-use a TCP connection between multiple RunJobs.php calls" [15:37:44] ^ Which would explain connection exhaustion, timeouts, job backlogs -- all without any/few actually failing jobs [15:42:08] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07Ruby, 15User-zeljkofilipin: Mediawiki Ruby gem incorrectly assumes path to index.php - https://phabricator.wikimedia.org/T149169#3294706 (10zeljkofilipin) p:05Normal>03Low [15:43:59] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 13Patch-For-Review, 15User-zeljkofilipin: mediawiki_selenium feature to show/capture Selenium WebDriver requests to remote browser. - https://phabricator.wikimedia.org/T94577#3294712 (10zeljkofilipin) 05Open>03declined p:05Normal>... [15:45:03] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07Ruby: Ruby mediawiki_api client hides error details - https://phabricator.wikimedia.org/T127786#3294720 (10zeljkofilipin) p:05Triage>03Low [15:45:29] RainbowSprinkles: this is the best explanation that I can come up with.. Let's see if Aaron agrees :) [15:46:01] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 15User-zeljkofilipin: Allow specifying required permissions for a new user - https://phabricator.wikimedia.org/T134529#3294721 (10zeljkofilipin) p:05Triage>03Low [15:47:11] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 05MW-1.27-release (WMF-deploy-2016-03-01_(1.27.0-wmf.15)), 05MW-1.27-release-notes, and 3 others: Remove LoginPage from mediawiki_selenium Ruby gem - https://phabricator.wikimedia.org/T127042#3294724 (10zeljkofilipin) 05Open>03decline... [15:50:16] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog): We should check for ResourceLoader errors after any page opens - https://phabricator.wikimedia.org/T59304#3294732 (10zeljkofilipin) 05Open>03declined Unlikely to ever be resolved because of {T139740}. [15:50:47] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog): browser tests should assert there are no pink error boxes on the page - https://phabricator.wikimedia.org/T63304#3294737 (10zeljkofilipin) 05Open>03declined Unlikely to ever be resolved because of {T139740}. [15:51:09] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 10CirrusSearch, 06Discovery, 06Discovery-Search: Support alternative API endpoints - https://phabricator.wikimedia.org/T99663#3294741 (10zeljkofilipin) p:05Normal>03Low [15:51:54] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog): Introduce @skip tag in mediawiki selenium - https://phabricator.wikimedia.org/T101062#3294746 (10zeljkofilipin) 05Open>03declined p:05Normal>03Low Unlikely to ever be resolved because of {T139740}. [15:52:51] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog): Given I am logged in step not generic enough - https://phabricator.wikimedia.org/T91671#3294753 (10zeljkofilipin) 05Open>03declined p:05Normal>03Low Unlikely to ever be resolved because of {T139740}. [15:58:28] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog): refactor "upload file" step to mediawiki_selenium gem - https://phabricator.wikimedia.org/T64888#3294769 (10zeljkofilipin) 05Open>03declined Unlikely to ever be resolved because of {T139740}. [16:00:23] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog): mediawiki_selenium does not timeout on sauce_api call - https://phabricator.wikimedia.org/T88221#3294779 (10zeljkofilipin) 05Open>03declined p:05Normal>03Low Unlikely to ever be resolved because of {T139740}. [16:01:07] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog): move UploadPage to mediawiki_selenium gem - https://phabricator.wikimedia.org/T65833#3294785 (10zeljkofilipin) 05Open>03declined Unlikely to ever be resolved because of {T139740}. [16:01:49] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog): Enable modification of Firefox (or browser-agnostic) request headers via framework if possible - https://phabricator.wikimedia.org/T88480#3294793 (10zeljkofilipin) 05Open>03declined p:05Normal>03Low Unlikely to ever be resolved becau... [16:02:47] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog): mediawiki selenium should retry sauce labs requests on timeout - https://phabricator.wikimedia.org/T72179#3294799 (10zeljkofilipin) 05Open>03declined Unlikely to ever be resolved because of {T139740}. [16:03:18] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog): Provide support for spoffing the physical geolocation on QA tests - https://phabricator.wikimedia.org/T60720#3294804 (10zeljkofilipin) 05Open>03declined Unlikely to ever be resolved because of {T139740}. [16:04:33] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog): mediawiki-api gem should have integration tests - https://phabricator.wikimedia.org/T1330#3294817 (10zeljkofilipin) 05Open>03declined Unlikely to ever be resolved because of {T139740}. [16:05:14] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 13Patch-For-Review: Improve password fallback for mediawiki_selenium - https://phabricator.wikimedia.org/T112279#3294822 (10zeljkofilipin) 05Open>03declined p:05Normal>03Low Unlikely to ever be resolved because of {T139740}. [16:06:41] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog): Use the API URL as a single entry point and infer the rest by querying siteinfo - https://phabricator.wikimedia.org/T103763#3294836 (10zeljkofilipin) 05Open>03declined Unlikely to ever be resolved because of {T139740}. [16:08:53] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Kanban), 10MediaWiki-Vagrant, 13Patch-For-Review, and 2 others: Vagrant command for running browser tests - https://phabricator.wikimedia.org/T96283#3294852 (10zeljkofilipin) 05Open>03declined Unlikely to ever be resolved because of {T139740}. [16:20:58] Project beta-update-databases-eqiad build #17368: 04STILL FAILING in 57 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17368/ [16:22:07] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 13Patch-For-Review, 07Upstream, 07WorkType-Maintenance: mediawiki_api gem recursion on log_in - https://phabricator.wikimedia.org/T111133#3294864 (10zeljkofilipin) 05Open>03stalled p:05Normal>03Low Still not fixed upstream. Th... [16:25:53] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Kanban), 10MediaWiki-Vagrant: Add role to run Virtualbox with vb.gui = true for testing without a X client on the host - https://phabricator.wikimedia.org/T53882#3294875 (10zeljkofilipin) 05Open>03Resolved a:03zeljkofilipin Selenium tests can... [16:26:24] 10Gerrit, 06Release-Engineering-Team (Watching / External), 06Performance-Team, 07Performance: Create a bot that automatically optimizes PNGs - https://phabricator.wikimedia.org/T127628#2049083 (10Gilles) p:05Triage>03Low [16:27:46] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 06Collaboration-Team-Triage, 10MediaWiki-Vagrant, and 2 others: Get PageTriage browser tests running on MediaWiki-Vagrant - https://phabricator.wikimedia.org/T90477#3294896 (10zeljkofilipin) 05Open>03declined p:05Normal>03Low Unl... [16:29:16] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07JavaScript, 06Maps (Kartographer), and 2 others: Set up Selenium tests for Kartographer extension - https://phabricator.wikimedia.org/T152683#3294903 (10zeljkofilipin) 05stalled>03declined Nobody is interested in this. [16:30:45] 10Browser-Tests-Infrastructure, 10Continuous-Integration-Config, 06Release-Engineering-Team (Kanban), 07Upstream, 15User-zeljkofilipin: Firefox v47 breaks mediawiki_selenium - https://phabricator.wikimedia.org/T137561#3294925 (10zeljkofilipin) [16:30:47] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 13Patch-For-Review, 15User-zeljkofilipin: Update mediawiki_selenium to use Marionette - https://phabricator.wikimedia.org/T137540#3294920 (10zeljkofilipin) 05Open>03declined p:05Normal>03Low Unlikely to ever be resolved because o... [16:31:45] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog): Links broken in UploadWizard-api-commons.wikimedia.beta.wmflabs.org job error report - https://phabricator.wikimedia.org/T66119#3294931 (10zeljkofilipin) 05Open>03Invalid No activity in years. [16:32:58] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Kanban), 13Patch-For-Review, 15User-zeljkofilipin: Make it possible to execute tests as a specific (new) MediaWiki user on beta cluster - https://phabricator.wikimedia.org/T152432#3294948 (10zeljkofilipin) 05Open>03Resolved p:05Normal>03Low... [16:45:23] RainbowSprinkles, hi, could you merge https://gerrit.wikimedia.org/r/#/c/355448/ please? [16:45:33] I was letting others review it [16:45:43] ok [16:59:52] 10Deployment-Systems, 06Release-Engineering-Team, 06Operations, 06Performance-Team, 07HHVM: Translation cache exhaustion caused by changes to PHP code in file scope - https://phabricator.wikimedia.org/T103886#3295034 (10Krinkle) p:05Triage>03High [17:05:40] PROBLEM - Puppet errors on buildlog is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [17:05:44] PROBLEM - Puppet errors on swift is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [17:07:19] PROBLEM - Puppet errors on swift-storage-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [17:09:40] (03PS1) 10Umherirrender: [WikimediaMaintenance] Add npm job [integration/config] - 10https://gerrit.wikimedia.org/r/355803 [17:20:47] Project beta-update-databases-eqiad build #17369: 04STILL FAILING in 47 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17369/ [17:22:49] RainbowSprinkles: ^ I saw you were working on that yesterday, did I miss a task about it? [17:23:47] There's a task, might not be assigned to me [17:24:10] T166266 [17:24:11] T166266: Beta update.php fails with Can't DROP 'flow_ext_ref_idx_v2'; - https://phabricator.wikimedia.org/T166266 [17:24:11] kk [17:24:17] ty [17:24:29] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team (Kanban), 06Collaboration-Team-Triage, 10Flow, and 2 others: Beta update.php fails with Can't DROP 'flow_ext_ref_idx_v2'; - https://phabricator.wikimedia.org/T166266#3295127 (10demon) a:03demon [17:24:46] greg-g: do you have a minute? [17:24:53] very few [17:25:02] Spoke in -editing and another review of the change will be done today :). [17:25:03] I'm incredibly busy today [17:25:34] greg-g: well me and paladox are hosting Icinga 2 instance, and we were wondering if maybe releng would like to semi-utilise it to see if they would like to use it as well? [17:25:42] 06Release-Engineering-Team (Kanban), 10Scap: scap: Investigate scaps handling of InitialiseSettings.php - https://phabricator.wikimedia.org/T166384#3294452 (10demon) Cf {D654} wherein I want to remove the idea of `--beta-only-change` as well. [17:25:51] Zppix: short answer: no. [17:26:16] greg-g: Is there a particular reason or is it more preference? [17:27:12] We won't depend on a monitoring service not maintained by Ops, basically. [17:27:40] greg-g: I understand... alright thanks for atleast considering it instead of straight up denying it [17:37:48] 06Release-Engineering-Team (Kanban), 06Reading-Admin, 05Deployment Blockers, 05Release: MW-1.30.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T163512#3295182 (10greg) Cross-posting from the blocking task: >>! In T166345#3295180, @greg wrote: > Status update from RelEng: > > We don't hav... [17:59:51] 06Release-Engineering-Team (Kanban), 10Wikimedia-Hackathon-2017: Building Better Software (Hack-a-thon session) - https://phabricator.wikimedia.org/T165729#3275211 (10greg) Any needed follow-ups from the session? [18:08:08] 06Release-Engineering-Team, 06Performance-Team, 10Phabricator: Custom domain/URL for phame performance blog - https://phabricator.wikimedia.org/T166374#3295343 (10greg) a:05mmodell>03None [18:31:18] Project beta-update-databases-eqiad build #17370: 15ABORTED in 11 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17370/ [18:31:36] That was me manually aborting, because it'll time out anyway ^ [18:31:45] (running last two manually right now, should resolve in next hour or 2) [18:42:12] RainbowSprinkles hopefully that will be fixed today :) [18:47:26] 06Release-Engineering-Team, 10Page-Previews, 06Reading-Web-Backlog: Generate compiled assets from continuous integration - https://phabricator.wikimedia.org/T158980#3295462 (10Jdlrobson) The main tension here seems to be rebasing at time of merge. I think the following are probably the best solutions to thi... [19:59:32] 06Release-Engineering-Team, 10Page-Previews, 06Reading-Web-Backlog: Generate compiled assets from continuous integration - https://phabricator.wikimedia.org/T158980#3295548 (10demon) >>! In T158980#3295462, @Jdlrobson wrote: > 1) Find some way to hook into the Gerrit rebase function and run `npm run build` a... [20:05:01] Project beta-update-databases-eqiad build #17371: 15ABORTED in 45 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17371/ [20:51:03] Yippee, build fixed! [20:51:03] Project beta-update-databases-eqiad build #17372: 09FIXED in 31 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17372/ [20:51:35] RainbowSprinkles how did you fix it [20:51:46] magic [20:51:53] :) [20:54:22] Actually: I went database by database and basically did the SQL cleanup by hand [20:54:30] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team (Kanban), 06Collaboration-Team-Triage, 10Flow, and 2 others: Beta update.php fails with Can't DROP 'flow_ext_ref_idx_v2'; - https://phabricator.wikimedia.org/T166266#3295687 (10Paladox) p:05Unbreak!>03High @demon fixed it so lowering priority. B... [20:54:33] Drop the index(es), do the field alteration, create the index [20:54:41] Rather time consuming [20:54:47] Ah [20:55:06] We need to get them to update flow.sql. Very much outdated. [20:55:15] anyways thanks for doing that :) [20:56:02] Well, I was the one who broke it :p [20:56:20] Oh how did you break it? [20:56:28] Truncating updatelog [20:56:32] You found a bug. [20:56:39] So it tried to re-run a schema migration again [20:56:46] Yeah, truncating updatelog exposed it [20:57:01] yep [20:57:06] Schema changes *should* be idempotent, but this one clearly wasn't [20:57:20] yep [20:59:33] (03PS1) 10Mattflaschen: Whitelist Foxy brown [integration/config] - 10https://gerrit.wikimedia.org/r/355862 [21:03:16] (03CR) 10Paladox: [C: 031] Whitelist Foxy brown [integration/config] - 10https://gerrit.wikimedia.org/r/355862 (owner: 10Mattflaschen) [21:17:32] 06Release-Engineering-Team, 07Jenkins: Update jerkins to 2.46.3 - https://phabricator.wikimedia.org/T166415#3295717 (10Paladox) [21:19:35] RainbowSprinkles lol in two weeks there have been 20 more fixes for stable-2.14 [21:19:42] bringing it to 70+ fixes. [21:40:27] 06Release-Engineering-Team (Kanban), 06Reading-Admin, 05Deployment Blockers, 05Release, 07User-notice: MW-1.30.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T163512#3295745 (10Elitre) Johan, adding it here as Tech News is likely wrong now. [22:04:21] 10MediaWiki-Codesniffer: Provide an MWCS rule to enforce "short" type definitions: int and bool, not integer and boolean - https://phabricator.wikimedia.org/T145162#2621760 (10Krinkle) Renaming task to avoid confusing with WMCS. [22:04:29] 10MediaWiki-Codesniffer: Provide a Codesniffer rule to enforce "short" type definitions: int and bool, not integer and boolean - https://phabricator.wikimedia.org/T145162#3295782 (10Krinkle) [22:22:20] RainbowSprinkles i wonder should we try out the new mariadb connector i added in 2.14 when we upgrade? [22:22:46] It has improvements over mysql one. One of the improvements is it does not support java 6 and is built with java 7. [22:30:55] 06Release-Engineering-Team (Kanban), 10Phabricator: Switch phabricator production to codfw - https://phabricator.wikimedia.org/T164810#3295808 (10mmodell) @dzahn tells me that we can set up temporary DNS records like `phabricator-new.wikimedia.org` in order to test the varnish config in codfw. [22:35:05] 06Release-Engineering-Team (Kanban), 10Phabricator: Switch phabricator production to codfw - https://phabricator.wikimedia.org/T164810#3295809 (10Dzahn) I saw we already have git-ssh in both eqiad and codfw, like so: git-ssh.codfw.wikimedia.org has address 208.80.153.250 git-ssh.codfw.wikimedia.org has IPv6 a... [22:54:00] (03CR) 10Catrope: [C: 031] Whitelist Foxy brown [integration/config] - 10https://gerrit.wikimedia.org/r/355862 (owner: 10Mattflaschen) [23:31:07] 06Release-Engineering-Team (Kanban), 06Reading-Admin, 05Deployment Blockers, 05Release, 07User-notice: MW-1.30.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T163512#3199999 (10Johan) Keeping track of it. Tentatively adding contradictory information to Tech News; will remove whatever's n... [23:57:06] PROBLEM - Puppet errors on deployment-kafka01 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0]