[03:18:17] 10Deployment-Systems, 03Scap3 (Scap3-Adoption-Phase1), 10scap, 10Analytics-Cluster, and 2 others: Deploy analytics-refinery with scap3 - https://phabricator.wikimedia.org/T129151#2529966 (10Milimetric) \o/ thanks everyone, refinery is quite painless to deploy this way. [03:36:37] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.28.0-wmf.14 deployment blockers - https://phabricator.wikimedia.org/T139217#2529978 (10Tgr) [03:36:40] 10Beta-Cluster-Infrastructure, 06Commons, 10MediaWiki-Authentication-and-authorization, 10MediaWiki-extensions-CentralAuth, 06Reading-Web-Backlog: Unable to log in on https://commons.m.wikimedia.beta.wmflabs.org/wiki/Special:UserLogin - https://phabricator.wikimedia.org/T142015#2529977 (10Tgr) [04:06:10] Yippee, build fixed! [04:06:10] Project selenium-MultimediaViewer » safari,beta,OS X 10.9,contintLabsSlave && UbuntuTrusty build #99: 09FIXED in 10 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=safari,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=OS%20X%2010.9,label=contintLabsSlave%20&&%20UbuntuTrusty/99/ [04:18:34] Yippee, build fixed! [04:18:35] Project selenium-MultimediaViewer » firefox,beta,Linux,contintLabsSlave && UbuntuTrusty build #99: 09FIXED in 22 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=contintLabsSlave%20&&%20UbuntuTrusty/99/ [05:06:23] 10Beta-Cluster-Infrastructure, 10Pywikibot-core, 07TestMe: Link.langlinkUnsafe does not work on Beta-Cluster wikis - https://phabricator.wikimedia.org/T112006#2530013 (10jayvdb) a:05XZise>03None [05:37:00] Project mediawiki-core-code-coverage build #2182: 04STILL FAILING in 2 hr 36 min: https://integration.wikimedia.org/ci/job/mediawiki-core-code-coverage/2182/ [06:58:15] PROBLEM - Puppet run on phab-beta is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [08:05:10] PROBLEM - English Wikipedia Main page on beta-cluster is CRITICAL: HTTP CRITICAL: HTTP/1.1 301 TLS Redirect - string 'Wikipedia' not found on 'http://en.wikipedia.beta.wmflabs.org:80/wiki/Main_Page?debug=true' - 567 bytes in 0.003 second response time [08:06:18] PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: HTTP CRITICAL: HTTP/1.1 301 TLS Redirect - string 'Wikipedia' not found on 'http://en.m.wikipedia.beta.wmflabs.org:80/wiki/Main_Page?debug=true' - 569 bytes in 0.003 second response time [08:09:56] 10Beta-Cluster-Infrastructure, 07Tracking: Consolidate or remove Beta Cluster instances to help with [[wikitech:Purge_2016]] - https://phabricator.wikimedia.org/T142288#2530145 (10greg) [08:13:26] 10Beta-Cluster-Infrastructure, 10Mathoid: Move mathoid to deployment-sca* hosts in Beta Cluster - https://phabricator.wikimedia.org/T142255#2530170 (10greg) [08:13:29] 10Beta-Cluster-Infrastructure, 10ContentTranslation-CXserver: Move apertium to deployment-sca* hosts in Beta Cluster - https://phabricator.wikimedia.org/T142152#2530171 (10greg) [08:13:32] 10Beta-Cluster-Infrastructure, 10Citoid, 10VisualEditor: Move citoid to deployment-sca* hosts in Beta Cluster - https://phabricator.wikimedia.org/T142150#2530172 (10greg) [08:13:34] 10Beta-Cluster-Infrastructure, 07Tracking: Consolidate or remove Beta Cluster instances to help with [[wikitech:Purge_2016]] - https://phabricator.wikimedia.org/T142288#2530169 (10greg) [08:32:19] 10Beta-Cluster-Infrastructure, 07WorkType-NewFunctionality: Enable GTID on beta cluster mariaDB once upgraded - https://phabricator.wikimedia.org/T139044#2530194 (10greg) [08:32:21] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 10DBA, 07WorkType-Maintenance: Upgrade mariadb in deployment-prep from Precise/MariaDB 5.5 to Jessie/MariaDB 5.10 - https://phabricator.wikimedia.org/T138778#2530193 (10greg) [08:32:29] 10Beta-Cluster-Infrastructure, 07WorkType-NewFunctionality: Enable GTID on beta cluster mariaDB once upgraded - https://phabricator.wikimedia.org/T139044#2417810 (10greg) [08:32:31] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team, 10DBA, 07WorkType-Maintenance: Upgrade mariadb in deployment-prep from Precise/MariaDB 5.5 to Jessie/MariaDB 5.10 - https://phabricator.wikimedia.org/T138778#2409864 (10greg) [08:35:38] 10Beta-Cluster-Infrastructure, 13Patch-For-Review: deployment-fluorine becomes unresponsive frequently - https://phabricator.wikimedia.org/T140313#2530197 (10greg) a:03thcipriani [08:44:05] 10Beta-Cluster-Infrastructure, 06Labs, 10Labs-Infrastructure, 06Operations: On deployment-prep, add warning text + labs Term of Uses link to the motd files - https://phabricator.wikimedia.org/T100837#2530201 (10greg) [08:53:17] 10Beta-Cluster-Infrastructure, 06Commons, 06Multimedia: Setup deployment-imagescaler host(s) in Beta Cluster - https://phabricator.wikimedia.org/T142289#2530205 (10greg) [09:32:39] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-ORES, 06Revision-Scoring-As-A-Service: Dashboard or pane for ORES failed jobs on beta - https://phabricator.wikimedia.org/T142119#2530286 (10Ladsgroup) I made this: https://grafana-labs-admin.wikimedia.org/dashboard/db/ores-extension-beta-cluster It seem... [09:34:20] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-ORES, 06Revision-Scoring-As-A-Service: Dashboard or pane for ORES failed jobs on beta - https://phabricator.wikimedia.org/T142119#2530302 (10Ladsgroup) (The exact same query doesn't work in grafana.wikimedia..org even though I selected labs-graphite as so... [09:44:20] 10Beta-Cluster-Infrastructure, 10ORES, 06Revision-Scoring-As-A-Service: Dashboard or pane for ORES service in beta - https://phabricator.wikimedia.org/T142294#2530311 (10Ladsgroup) [11:32:32] 10Beta-Cluster-Infrastructure, 07Tracking: Consolidate or remove Beta Cluster instances to help with [[wikitech:Purge_2016]] (tracking) - https://phabricator.wikimedia.org/T142288#2530361 (10Danny_B) [11:50:08] 10Browser-Tests-Infrastructure, 06Reading-Web-Backlog, 13Patch-For-Review, 03Reading-Web-Sprint-78-Terminal-Velocity, and 2 others: Upgrade Selenium gem for various reading web opened extensions - https://phabricator.wikimedia.org/T142141#2524100 (10dr0ptp4kt) Looks like @jdlrobson has a question on whethe... [11:50:29] PROBLEM - Puppet run on deployment-cache-text04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [12:00:06] !log restarting uwsgi-ores and celery-ores-worker in deployment-sca03 [12:00:10] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [12:12:48] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Phabricator, 07Zuul: Create a duplicate zuul type interface in phabricator - https://phabricator.wikimedia.org/T142296#2530384 (10Paladox) [12:13:27] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Phabricator, 07Zuul: Create a zuul type interface in phabricator - https://phabricator.wikimedia.org/T142296#2530396 (10Paladox) [12:28:27] !log cherry-picked 303356/1 into the puppetmaster [12:28:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [12:31:06] !log restarting uwsgi-ores and celery-ores-worker in deployment-sca03 [12:31:10] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL, Master [12:31:31] Aaand ORES beta is up: https://ores-beta.wmflabs.org/v1/scores/enwiki/damaging/?revids=216123|32423423|3243242|234324 [12:32:33] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-ORES, 06Revision-Scoring-As-A-Service: Dashboard or pane for ORES failed jobs on beta - https://phabricator.wikimedia.org/T142119#2530399 (10Ladsgroup) Okay, ORES beta is up and working. This puppet change (which got cherry-picked) solved this issue. http... [12:53:42] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Phabricator, 07Zuul: Create a zuul type interface in phabricator - https://phabricator.wikimedia.org/T142296#2530410 (10Aklapper) How is this different from T85123 and why do you propose to keep Zuul instead of trying Harbormaster? [13:52:21] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Phabricator, 07Zuul: Create a zuul type interface in phabricator - https://phabricator.wikimedia.org/T142296#2530424 (10Paladox) @aklapper the reason we should use Zuul over Harbormaster is because Zuul you get a nice layout and ca... [15:04:46] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Phabricator, 07Zuul: Create a zuul type interface in phabricator - https://phabricator.wikimedia.org/T142296#2530442 (10Aklapper) As I asked before: How is this task different from T85123 ? [15:10:12] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Phabricator, 07Zuul: Create a zuul type interface in phabricator - https://phabricator.wikimedia.org/T142296#2530443 (10Luke081515) [15:14:29] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Phabricator, 07Zuul: Create a zuul type interface in phabricator - https://phabricator.wikimedia.org/T142296#2530460 (10Paladox) Because that task discuss a integration plan, but looking at the description it doesn't discuss Zuul.... [16:00:35] (03CR) 10Paladox: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/227223 (https://phabricator.wikimedia.org/T105474) (owner: 10Hashar) [16:21:48] PROBLEM - Puppet run on deployment-changeprop is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [16:27:30] 06Release-Engineering-Team, 10Phabricator, 13Patch-For-Review: Create a continuous integration plan for Wikimedia Phabricator patches - https://phabricator.wikimedia.org/T85123#2530489 (10Paladox) [16:27:34] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 10Phabricator, 07Zuul: Create a zuul type interface in phabricator - https://phabricator.wikimedia.org/T142296#2530491 (10Paladox) [16:30:48] 06Release-Engineering-Team, 10Phabricator, 13Patch-For-Review: Create a continuous integration plan for Wikimedia Phabricator patches - https://phabricator.wikimedia.org/T85123#2530493 (10Paladox) [16:31:30] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 10Phabricator, 13Patch-For-Review, 07Zuul: Create a continuous integration plan for Wikimedia Phabricator patches - https://phabricator.wikimedia.org/T85123#939268 (10Paladox) [16:34:21] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 10Phabricator, 13Patch-For-Review, 07Zuul: Create a continuous integration plan for Wikimedia Phabricator patches - https://phabricator.wikimedia.org/T85123#2530495 (10Aklapper) [16:34:29] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 10Phabricator, 13Patch-For-Review: Create a continuous integration plan for Wikimedia Phabricator patches - https://phabricator.wikimedia.org/T85123#939268 (10Aklapper) [16:35:00] (03Draft2) 10Paladox: Test [integration/config] - 10https://gerrit.wikimedia.org/r/303368 [16:35:28] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 10Phabricator, 13Patch-For-Review: Create a continuous integration plan for Wikimedia Phabricator patches - https://phabricator.wikimedia.org/T85123#939268 (10Aklapper) @Paladox: Reverting latest edits as Zuul might be //one// outcome but... [16:38:29] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 10Phabricator, 13Patch-For-Review: Create a continuous integration plan for Wikimedia Phabricator patches - https://phabricator.wikimedia.org/T85123#2530498 (10Paladox) Oh sorry. Here's my personal opinion We should develop a Zuul like i... [16:53:23] (03Draft1) 10Paladox: Test [integration/config] - 10https://gerrit.wikimedia.org/r/303368 [16:55:26] (03Abandoned) 10Paladox: Test [integration/config] - 10https://gerrit.wikimedia.org/r/303368 (owner: 10Paladox) [18:01:47] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-ORES, 06Revision-Scoring-As-A-Service: Dashboard or pane for ORES failed jobs on beta - https://phabricator.wikimedia.org/T142119#2530528 (10Ladsgroup) [18:33:26] Yippee, build fixed! [18:33:27] Project mediawiki-core-code-coverage build #2183: 09FIXED in 3 hr 33 min: https://integration.wikimedia.org/ci/job/mediawiki-core-code-coverage/2183/ [19:12:16] PROBLEM - Host integration-slave-jessie-android is DOWN: CRITICAL - Host Unreachable (10.68.19.239) [19:42:06] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-ORES, 06Revision-Scoring-As-A-Service: Dashboard or pane for ORES failed jobs on beta - https://phabricator.wikimedia.org/T142119#2530681 (10Ladsgroup) Thanks to {T141891} getting solved, we have this in https://grafana.wikimedia.org/dashboard/db/ores-ext... [20:09:05] 10Beta-Cluster-Infrastructure, 10ORES, 06Revision-Scoring-As-A-Service: Dashboard or pane for ORES service in beta - https://phabricator.wikimedia.org/T142294#2530687 (10Ladsgroup) https://grafana.wikimedia.org/dashboard/db/ores-beta-cluster You might notice that some of them are empty. It's because the set...