[00:00:33] 10Continuous-Integration-Config, 06Release-Engineering-Team: CI failing on lack of test logs (mediawiki-core-qunit-selenium-jessie) - https://phabricator.wikimedia.org/T165447#3266091 (10Ejegg) [00:57:31] PROBLEM - Puppet staleness on deployment-logstash2 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0] [01:55:58] PROBLEM - Puppet errors on deployment-mira is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [02:10:58] RECOVERY - Puppet errors on deployment-mira is OK: OK: Less than 1.00% above the threshold [0.0] [03:19:42] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.30.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T163512#3266223 (10greg) [03:19:57] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.30.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T163512#3199999 (10greg) [03:20:37] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.30.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T163512#3199999 (10greg) [03:21:53] 06Release-Engineering-Team (Deployment-Blockers), 05Release: MW-1.30.0-wmf.2 deployment blockers - https://phabricator.wikimedia.org/T163512#3266228 (10greg) Added some blocking tasks that should be resolved before we branch/roll out wmf.2 (which caused production breakage last week). [03:26:54] 10Deployment-Systems, 06Release-Engineering-Team (Long-Lived-Branches): Static asset time on disk - https://phabricator.wikimedia.org/T140921#3266229 (10greg) [04:06:19] Project selenium-MultimediaViewer » safari,beta,OS X 10.9,BrowserTests build #393: 04FAILURE in 10 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=safari,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=OS%20X%2010.9,label=BrowserTests/393/ [06:44:36] Project selenium-Wikibase » chrome,beta,Linux,BrowserTests build #362: 04FAILURE in 2 hr 4 min: https://integration.wikimedia.org/ci/job/selenium-Wikibase/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/362/ [07:47:23] (03PS2) 10Hashar: Add npm jobs [integration/config] - 10https://gerrit.wikimedia.org/r/353583 (owner: 10Umherirrender) [07:50:06] (03CR) 10Hashar: [C: 032] Add npm jobs [integration/config] - 10https://gerrit.wikimedia.org/r/353583 (owner: 10Umherirrender) [07:51:04] (03Merged) 10jenkins-bot: Add npm jobs [integration/config] - 10https://gerrit.wikimedia.org/r/353583 (owner: 10Umherirrender) [09:09:44] 10Continuous-Integration-Infrastructure, 13Patch-For-Review: Allow use of ext-gmp on CI for composer tests - https://phabricator.wikimedia.org/T164977#3252974 (10hashar) php5-gmp is not available yet on the CI machine. The provisioning of Trusty images on Nodepool is broken due to an unrelated puppet mistake. [09:13:30] 10Continuous-Integration-Infrastructure: Puppet broken on CI Trusty nodes: Could not find dependency Apt::Pin[hhvm-from-experimental] - https://phabricator.wikimedia.org/T165462#3266571 (10hashar) [10:45:56] PROBLEM - Puppet errors on deployment-imagescaler01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [10:59:34] PROBLEM - Puppet errors on deployment-pdf01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [11:46:20] 10Continuous-Integration-Config, 06Release-Engineering-Team: CI failing on lack of test logs (mediawiki-core-qunit-selenium-jessie) - https://phabricator.wikimedia.org/T165447#3267243 (10hashar) We had a single job handling both qunit and selenium tests with 48a1955569bf9cca5bd4290526cb2ee5f9abe108 . Supposed... [11:48:51] 10Continuous-Integration-Config, 06Release-Engineering-Team: CI failing on lack of test logs (mediawiki-core-qunit-selenium-jessie) - https://phabricator.wikimedia.org/T165447#3267250 (10hashar) I have force regenerated the Jenkins job mediawiki-core-qunit-selenium-jessie . The web gui now shows: > Allow empt... [11:54:38] 10Continuous-Integration-Config, 06Release-Engineering-Team: CI failing on lack of test logs (mediawiki-core-qunit-selenium-jessie) - https://phabricator.wikimedia.org/T165447#3267266 (10hashar) 05Open>03Resolved a:03hashar I did a recheck on the Gerrit change https://gerrit.wikimedia.org/r/#/c/353929/ a... [11:59:34] 10Scap: Allow us to choose different targets on where to scrap to - https://phabricator.wikimedia.org/T165486#3267283 (10Paladox) [12:00:01] 10Scap: Allow us to choose different targets on where to scrap to - https://phabricator.wikimedia.org/T165486#3267271 (10Paladox) See T157414 [12:01:49] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team, 07Jenkins: Upgrade jenkins server and jenkins slaves to java 8 - https://phabricator.wikimedia.org/T162828#3267293 (10Paladox) [12:13:37] Yippee, build fixed! [12:13:37] Project selenium-CentralAuth » firefox,beta,Linux,BrowserTests build #397: 09FIXED in 1 min 26 sec: https://integration.wikimedia.org/ci/job/selenium-CentralAuth/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/397/ [12:23:36] 10Browser-Tests-Infrastructure: Could not find link to Sauce Labs job URL - https://phabricator.wikimedia.org/T165487#3267315 (10Tgr) [12:27:39] 10Scap: Allow to choose different targets on where to scap to - https://phabricator.wikimedia.org/T165486#3267343 (10Aklapper) [12:35:23] Yippee, build fixed! [12:35:23] Project selenium-QuickSurveys » chrome,beta,Linux,BrowserTests build #408: 09FIXED in 33 sec: https://integration.wikimedia.org/ci/job/selenium-QuickSurveys/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/408/ [13:21:59] PROBLEM - Puppet errors on deployment-mira is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [13:24:40] PROBLEM - Puppet errors on integration-slave-trusty-1006 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [13:27:13] 10Continuous-Integration-Config, 10MobileFrontend, 06Reading-Web-Backlog: Migrate experimental CI job mwext-MobileFrontend-npm-run-lint-modules to stable - https://phabricator.wikimedia.org/T163882#3267539 (10Jdlrobson) [13:38:51] PROBLEM - Puppet errors on integration-slave-trusty-1004 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [13:40:09] PROBLEM - Puppet errors on integration-slave-trusty-1001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [13:44:52] PROBLEM - Puppet errors on integration-slave-trusty-1003 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [14:02:00] RECOVERY - Puppet errors on deployment-mira is OK: OK: Less than 1.00% above the threshold [0.0] [14:50:25] 10Continuous-Integration-Infrastructure, 10MediaWiki-extensions-SecureSessions, 13Patch-For-Review: Secure Sessions ships with php-geoip, but test infrastructure has it already compiled, which gives failures - https://phabricator.wikimedia.org/T157814#3267781 (10Umherirrender) Now the composer is running on... [15:20:07] 10Gerrit, 07Upstream: Gerrit should feature customizable message on Login page (No 'Forgot password' link in the gerrit login page.) - https://phabricator.wikimedia.org/T60205#3267872 (10Paladox) What we could do is convert https://github.com/GerritCodeReview/gerrit/blob/461cff743afa2858e8f3ab7c51aacc698ab0fa5... [15:24:03] spam incoming... [15:35:17] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team (Watching / External): Beta cluster scap job ( beta-scap-eqiad ) fails due to puppet erasing /etc/ssh/ssh_known_hosts - https://phabricator.wikimedia.org/T159332#3267977 (10greg) [15:35:20] 06Release-Engineering-Team (Watching / External), 06Developer-Relations, 06WMF-Legal, 07RfC: Remove @author lines from code - https://phabricator.wikimedia.org/T139301#3267980 (10greg) [15:35:23] 06Release-Engineering-Team (Watching / External), 06Operations, 06Parsing-Team, 07HHVM, 07Wikimedia-Incident: API cluster failure / OOM - https://phabricator.wikimedia.org/T151702#3267979 (10greg) [15:35:25] 06Release-Engineering-Team (Watching / External), 10Security-Data-Mapping, 07Documentation, 15User-greg: Release Engineering Data Map - https://phabricator.wikimedia.org/T133239#3267981 (10greg) [15:35:27] 06Release-Engineering-Team (Watching / External), 10Fundraising-Backlog, 10fundraising-tech-ops, 07Spike: Spike: decide how payments-wiki deployment process should relate to the core MW deployment process - https://phabricator.wikimedia.org/T130658#3267982 (10greg) [15:35:31] 06Release-Engineering-Team (Watching / External), 07Performance: Create a bot that automatically optimizes PNGs - https://phabricator.wikimedia.org/T127628#3267986 (10greg) [15:35:36] 06Release-Engineering-Team (Watching / External), 10DBA, 06Operations: Audit all existing code to ensure that any extension currently or previously adding blobs to ExternalStore has been registering a reference in the text table (and fix up if wrong) - https://phabricator.wikimedia.org/T106388#3267987 (10greg) [15:35:37] 06Release-Engineering-Team (Watching / External), 10Pywikibot-General: Share Appveyor account credentials with Release Engineering - https://phabricator.wikimedia.org/T104306#3267988 (10greg) [15:35:40] 10MediaWiki-Releasing, 06Release-Engineering-Team (Watching / External), 10Architecture, 10Parsoid, and 2 others: Evaluate and decide on a distribution strategy targeted at VMs - https://phabricator.wikimedia.org/T87774#3267990 (10greg) [15:38:04] 06Release-Engineering-Team (Backlog), 10Phabricator, 06Project-Admins, 05Release: Decide whether to continue using deployment blocker tasks or combine them with the release milestones - https://phabricator.wikimedia.org/T164978#3267995 (10greg) [15:38:08] 06Release-Engineering-Team (Backlog), 06Operations, 07Security-General: setup mwreleases1001.eqiad.wmnet - https://phabricator.wikimedia.org/T164030#3267996 (10greg) [15:38:11] 10Gerrit, 06Release-Engineering-Team (Backlog), 10Phabricator, 07Technical-Debt: Replace deprecated phabricator conduit api calls in gerrit's its-phabricator plugin - https://phabricator.wikimedia.org/T159041#3267997 (10greg) [15:38:14] 10Gerrit, 06Release-Engineering-Team (Backlog), 10Scap: Deploy gerrit with scap3 - https://phabricator.wikimedia.org/T157414#3267999 (10greg) [15:38:16] 10Gerrit, 06Release-Engineering-Team (Backlog), 06Operations: Make sure replying to emails in gerrit 2.14 works - https://phabricator.wikimedia.org/T158915#3267998 (10greg) [15:38:19] 06Release-Engineering-Team (Backlog), 10Scap: Support shallow clones - https://phabricator.wikimedia.org/T157149#3268000 (10greg) [15:38:21] 10Gerrit, 06Release-Engineering-Team (Backlog), 13Patch-For-Review: Update gerrit to 2.14 - https://phabricator.wikimedia.org/T156120#3268001 (10greg) [15:38:23] 06Release-Engineering-Team (Backlog): make-wmf-branch doesn't seem to work with tags - https://phabricator.wikimedia.org/T153257#3268003 (10greg) [15:38:28] 10Gerrit, 06Release-Engineering-Team (Backlog), 10DBA, 06Operations, 13Patch-For-Review: Gerrit: Schedule downtime to migrate db to utf8mb4 - https://phabricator.wikimedia.org/T155764#3268002 (10greg) [15:38:29] 10Gerrit, 06Release-Engineering-Team (Backlog): Support redis as a cache store - https://phabricator.wikimedia.org/T152802#3268004 (10greg) [15:38:32] 06Release-Engineering-Team (Backlog), 06Operations, 10Phabricator: reinstall iridium (phabricator) as phab1001 with jessie - https://phabricator.wikimedia.org/T152129#3268005 (10greg) [15:38:45] 06Release-Engineering-Team (Backlog): Use pwstore (a shared gpg-encrypted password store) for Release Engineering related passwords - https://phabricator.wikimedia.org/T139093#3268012 (10greg) [15:38:45] 06Release-Engineering-Team (Backlog), 06Operations, 06Security-Team, 15User-greg: Determine a core set or a checklist of permissions for deployment purpose - https://phabricator.wikimedia.org/T140270#3268011 (10greg) [15:38:48] 06Release-Engineering-Team (Backlog), 15User-greg: Scope out "deployment-dashboard" project - https://phabricator.wikimedia.org/T137729#3268014 (10greg) [15:38:50] 06Release-Engineering-Team (Backlog), 15User-greg: Make a table of access levels per service RelEng maintains per person - https://phabricator.wikimedia.org/T135187#3268016 (10greg) [15:38:52] 06Release-Engineering-Team (Backlog): Reassess idea of CI point person role - https://phabricator.wikimedia.org/T129295#3268019 (10greg) [15:38:54] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Backlog), 07Documentation: Document actions not to take when responding to CI outages/issues - https://phabricator.wikimedia.org/T131778#3268017 (10greg) [15:38:56] 06Release-Engineering-Team (Backlog): investigate carbon aggregation of stats >1 month old behavior - https://phabricator.wikimedia.org/T129285#3268020 (10greg) [15:38:58] 06Release-Engineering-Team (Backlog), 15User-greg: Create person/project heat map - https://phabricator.wikimedia.org/T131764#3268018 (10greg) [15:39:00] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team (Backlog), 13Patch-For-Review: Beta puppetmaster cherry-pick process - https://phabricator.wikimedia.org/T135427#3268015 (10greg) [15:39:02] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Backlog), 07Ruby: Ruby mediawiki_api client hides error details - https://phabricator.wikimedia.org/T127786#3268022 (10greg) [15:39:04] 06Release-Engineering-Team (Backlog), 15User-greg: Expand Onboarding page - https://phabricator.wikimedia.org/T129050#3268021 (10greg) [15:39:06] 06Release-Engineering-Team (Backlog), 06Operations: Rewrite http://download.wikimedia.org/mediawiki/ -> https://releases.wikimedia.org/mediawiki in less than 3 redirects - https://phabricator.wikimedia.org/T119679#3268023 (10greg) [15:39:09] 10Beta-Cluster-Infrastructure, 06Release-Engineering-Team (Backlog), 07WorkType-NewFunctionality: Proposal: Force any WARNINGs on Beta Cluster to fail completely - https://phabricator.wikimedia.org/T115633#3268024 (10greg) [15:39:11] 06Release-Engineering-Team (Backlog): Repositories dashboard - https://phabricator.wikimedia.org/T112259#3268025 (10greg) [15:39:13] 06Release-Engineering-Team (Backlog), 05Testing-Initiative: Create QA Health scoreboard - https://phabricator.wikimedia.org/T108768#3268027 (10greg) [15:39:15] 06Release-Engineering-Team (Backlog), 10Scap: Scap should abort early when Keyholder is not armed - https://phabricator.wikimedia.org/T111062#3268026 (10greg) [15:39:17] 06Release-Engineering-Team (Backlog), 05Testing-Initiative, 07Tracking: Follow up workshop & brown bag ideas from Testing: Where does it hurt? (tracking) - https://phabricator.wikimedia.org/T108122#3268029 (10greg) [15:39:19] 10Deployment-Systems, 06Release-Engineering-Team (Backlog): Implement "new weekly release deploy duration" KPI - https://phabricator.wikimedia.org/T108742#3268028 (10greg) [15:39:21] 06Release-Engineering-Team (Backlog), 05Testing-Initiative: Guides for initializing a test suite: unit testing & browser testing - https://phabricator.wikimedia.org/T108107#3268030 (10greg) [15:39:23] 06Release-Engineering-Team (Backlog): Send email of last day's SAL entries to releng@ - https://phabricator.wikimedia.org/T106443#3268032 (10greg) [15:39:25] 06Release-Engineering-Team (Backlog), 05Testing-Initiative: Include links to unit testing (Emphasize testing documentation on mediawiki.org) - https://phabricator.wikimedia.org/T108105#3268031 (10greg) [15:39:27] 06Release-Engineering-Team (Backlog): "Welcome to Wikimedia development" information for new devs about releasing, CI, and QA - https://phabricator.wikimedia.org/T101031#3268034 (10greg) [15:39:29] 06Release-Engineering-Team (Backlog), 10MediaWiki-Vagrant, 07Documentation, 07WorkType-NewFunctionality: Document RSpec workflow on MediaWiki-Vagrant - https://phabricator.wikimedia.org/T97464#3268033 (10greg) [15:39:32] 06Release-Engineering-Team (Backlog): Design a Test-Driven Development (TDD) survey - https://phabricator.wikimedia.org/T94472#3268035 (10greg) [15:39:34] 06Release-Engineering-Team (Backlog), 10Staging, 10releng-201415-Q3, 07Tracking: [Quarterly Success Metric] Green nightly builds on the staging cluster (tracking) - https://phabricator.wikimedia.org/T88701#3268037 (10greg) [15:39:36] 10MediaWiki-Releasing, 06Release-Engineering-Team (Backlog), 06MediaWiki-Stakeholders-Group: Establish and consistently release schedule for MediaWiki releases - https://phabricator.wikimedia.org/T90249#3268036 (10greg) [15:43:09] 06Release-Engineering-Team (Watching / External), 10DBA, 06Operations: Audit all existing code to ensure that any extension currently or previously adding blobs to ExternalStore has been registering a reference in the text table (and fix up if wrong) - https://phabricator.wikimedia.org/T106388#3268061 (10jcre... [15:46:50] 10Scap (Scap3-Adoption-Phase1), 10scap2, 13Patch-For-Review: Scap3 needs to be deployed on RESTBase boxes and needs a group on tin - https://phabricator.wikimedia.org/T109862#3268087 (10thcipriani) 05Open>03Resolved RESTBase is now deployed via Scap3. Resolving. [15:47:46] 06Release-Engineering-Team, 10Scap: scap plugin to export prometheus metrics - https://phabricator.wikimedia.org/T151387#3268090 (10greg) a:05mmodell>03None [15:47:48] 10Beta-Cluster-Infrastructure, 10Scap: Automate beta scap3/keyholder setup - https://phabricator.wikimedia.org/T144647#3268091 (10greg) a:05thcipriani>03None [15:47:52] 10Deployment-Systems, 10Scap: puppet should run`deploy-init` command once after cloning the deploy repo - https://phabricator.wikimedia.org/T129906#3268093 (10greg) a:05thcipriani>03None [15:48:17] 10Scap (Scap3-Adoption-Phase2), 10scap2, 07WorkType-NewFunctionality: create a scap3 command to bootstrap a new deployment repo - https://phabricator.wikimedia.org/T118760#3268101 (10greg) a:05mmodell>03None [15:48:19] 10Scap (Scap3-Adoption-Phase1), 10scap2, 07Documentation, 15User-mobrovac: Document Scap3 config-deploy - https://phabricator.wikimedia.org/T116634#3268102 (10greg) a:05thcipriani>03None [15:48:23] 10Browser-Tests-Infrastructure, 13Patch-For-Review, 07WorkType-NewFunctionality: Auto retry failed browser tests to reduce false negatives - https://phabricator.wikimedia.org/T67773#3268104 (10greg) a:05dduvall>03None [15:48:25] 10Browser-Tests-Infrastructure, 10MediaWiki-Vagrant: Add role to run Virtualbox with vb.gui = true for testing without a X client on the host - https://phabricator.wikimedia.org/T53882#3268105 (10greg) a:05dduvall>03None [15:54:38] 06Release-Engineering-Team (Kanban), 10Release Pipeline: Move blubber to phabricator repo - https://phabricator.wikimedia.org/T164964#3268109 (10greg) [15:54:40] 06Release-Engineering-Team (Kanban), 10Phabricator: Switch phabricator production to codfw - https://phabricator.wikimedia.org/T164810#3268110 (10greg) [15:54:43] 10Continuous-Integration-Infrastructure (Little Steps Sprint), 06Release-Engineering-Team (Kanban), 15User-zeljkofilipin: For MediaWiki extensions, merge rubocop inside mwext-mw-selenium-jessie - https://phabricator.wikimedia.org/T164479#3268112 (10greg) [15:54:45] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Kanban), 13Patch-For-Review, 15User-zeljkofilipin: Run WebdriverIO tests in CI for extensions - https://phabricator.wikimedia.org/T164721#3268111 (10greg) [15:54:48] 06Release-Engineering-Team (Kanban), 10Phabricator: Custom task form for #Wikimedia-CentralNotice-Administration - https://phabricator.wikimedia.org/T164356#3268113 (10greg) [15:54:51] 06Release-Engineering-Team (Kanban), 06Operations, 07Security-General: setup mwreleases1001.eqiad.wmnet - https://phabricator.wikimedia.org/T164030#3268114 (10greg) [15:54:54] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Kanban), 07JavaScript, 13Patch-For-Review, and 2 others: Delete Ruby+Selenium Jenkins jobs - https://phabricator.wikimedia.org/T164004#3268115 (10greg) [15:54:56] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Kanban), 07JavaScript, 15User-zeljkofilipin: Update documentation for mediawiki-core-qunit-selenium-jessie Jenkins job - https://phabricator.wikimedia.org/T164002#3268116 (10greg) [15:54:58] 06Release-Engineering-Team (Kanban), 10CirrusSearch, 06Discovery, 06Discovery-Search, 15User-zeljkofilipin: Figure out why browser tests can't create suggestion box - https://phabricator.wikimedia.org/T162966#3268117 (10greg) [15:55:01] 06Release-Engineering-Team (Kanban), 10Phabricator: Custom task form for #mediawiki-extension-requests - https://phabricator.wikimedia.org/T160374#3268119 (10greg) [15:55:04] 06Release-Engineering-Team (Kanban), 10VisualEditor, 05MW-1.29-release (WMF-deploy-2017-04-25_(1.29.0-wmf.21)), 15User-Ryasmeen, 15User-zeljkofilipin: LanguageScreenshotBot trying to edit a non-existent page without signing in - https://phabricator.wikimedia.org/T162454#3268118 (10greg) [15:55:07] 06Release-Engineering-Team (Kanban), 10Wikimedia-Hackathon-2017, 07Browser-Tests, 07JavaScript, 15User-zeljkofilipin: Selenium/WebdriverIO tests in JavaScript/Node.js - https://phabricator.wikimedia.org/T159945#3268120 (10greg) [15:55:11] 06Release-Engineering-Team (Kanban), 10Release Pipeline, 13Patch-For-Review: Build testing k8s setup in ci-staging - https://phabricator.wikimedia.org/T159864#3268121 (10greg) [15:55:21] 10Gerrit, 06Release-Engineering-Team (Kanban), 06Operations, 07Beta-Cluster-reproducible, and 2 others: gerrit jgit gc caused mediawiki/core repo problems - https://phabricator.wikimedia.org/T151676#3268125 (10greg) [15:55:22] 06Release-Engineering-Team (Kanban): Audit of mw extensions' use of gating in CI - https://phabricator.wikimedia.org/T150701#3268127 (10greg) [15:55:24] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Kanban), 13Patch-For-Review: Set up experimental Docker CI slave - https://phabricator.wikimedia.org/T150502#3268128 (10greg) [15:55:26] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Kanban): Spike: Evaluate experimental Docker based CI w/ scap builds - https://phabricator.wikimedia.org/T150501#3268129 (10greg) [15:55:28] 06Release-Engineering-Team (Kanban), 10Scap: scap rollback behavior problems - https://phabricator.wikimedia.org/T150267#3268130 (10greg) [15:55:30] 06Release-Engineering-Team (Kanban), 10Scap, 13Patch-For-Review: scap deploy-local should make fewer assumptions about server/directories - https://phabricator.wikimedia.org/T146602#3268131 (10greg) [15:55:32] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Kanban), 13Patch-For-Review, 07Technical-Debt: Migrate CI labs slaves to use /srv instead of /mnt - https://phabricator.wikimedia.org/T146381#3268132 (10greg) [15:55:35] 06Release-Engineering-Team (Kanban), 10Phabricator, 07Availability, 07WorkType-NewFunctionality: Configure phabricator clustering for daemons and repositories - https://phabricator.wikimedia.org/T143175#3268133 (10greg) [15:55:38] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Kanban), 07Zuul: Write / update tutorial for Zuul Debian packaging - https://phabricator.wikimedia.org/T140912#3268134 (10greg) [15:55:40] 06Release-Engineering-Team (Kanban), 10Scap (Scap3-Adoption-Phase1): Make scap3 config deployment awesome - https://phabricator.wikimedia.org/T138452#3268136 (10greg) [15:55:42] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Kanban), 10MediaWiki-General-or-Unknown, 07JavaScript, and 5 others: Port Selenium tests from Ruby to Node.js - https://phabricator.wikimedia.org/T139740#3268135 (10greg) [15:55:45] 06Release-Engineering-Team (Kanban), 10Phabricator, 07Availability, 13Patch-For-Review, 07WorkType-NewFunctionality: Deploy phabricator to phab2001.codfw.wmnet - https://phabricator.wikimedia.org/T137928#3268137 (10greg) [15:55:49] 10MediaWiki-Releasing, 06Release-Engineering-Team (Kanban), 05MW-1.29-release-notes, 13Patch-For-Review: Include release extensions/skins as submodules of core (maybe vendor too?) - https://phabricator.wikimedia.org/T137564#3268138 (10greg) [15:55:52] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Kanban), 07Documentation: Document actions not to take when responding to CI outages/issues - https://phabricator.wikimedia.org/T131778#3268140 (10greg) [15:55:53] 10Deployment-Systems, 06Release-Engineering-Team (Kanban), 10Scap, 07WorkType-NewFunctionality: Scap3 submodule space issues - https://phabricator.wikimedia.org/T137124#3268139 (10greg) [15:56:04] 06Release-Engineering-Team (Kanban), 10Scap, 10scap2: Allow scap3 to read target host list from stdin - https://phabricator.wikimedia.org/T122913#3268146 (10greg) [15:56:04] 06Release-Engineering-Team (Kanban), 10Scap, 10scap2: scap3 should repack / pack-refs git repos under /srv/deployment - https://phabricator.wikimedia.org/T112509#3268148 (10greg) [15:56:07] 06Release-Engineering-Team (Kanban), 10Phabricator: Implement phabricator database clustering support - https://phabricator.wikimedia.org/T112776#3268147 (10greg) [15:56:10] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Kanban), 13Patch-For-Review, 07Upstream, 07WorkType-Maintenance: mediawiki_api gem recursion on log_in - https://phabricator.wikimedia.org/T111133#3268149 (10greg) [15:56:12] 06Release-Engineering-Team (Kanban), 10Education-Program-Dashboard, 13Patch-For-Review, 03Program Dashboard Sprint 1: Puppetize Dashboard application server - https://phabricator.wikimedia.org/T105967#3268150 (10greg) [15:56:15] 06Release-Engineering-Team (Kanban), 10Phabricator (Upstream), 07Upstream: allow users to export maniphest advanced search to csv - https://phabricator.wikimedia.org/T103009#3268151 (10greg) [15:56:17] 06Release-Engineering-Team (Kanban), 10MediaWiki-Vagrant, 07Documentation, 07WorkType-NewFunctionality: Document RSpec workflow on MediaWiki-Vagrant - https://phabricator.wikimedia.org/T97464#3268152 (10greg) [15:56:20] 10Browser-Tests-Infrastructure, 06Release-Engineering-Team (Kanban), 10MediaWiki-Vagrant, 13Patch-For-Review, and 2 others: Vagrant command for running browser tests - https://phabricator.wikimedia.org/T96283#3268153 (10greg) [15:56:23] 06Release-Engineering-Team (Kanban), 10Wikidata, 07Story: [Story] Use composer-merge-plugin to include Wikidata components in mediawiki-vendor - https://phabricator.wikimedia.org/T95663#3268154 (10greg) [15:56:26] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Kanban), 10Librarization, 07WorkType-NewFunctionality, 07Zuul: Zuul-cloner should check out submodules - https://phabricator.wikimedia.org/T84942#3268155 (10greg) [15:56:29] 10Deployment-Systems, 06Release-Engineering-Team (Kanban): Make make-wmf-branch able to branch extensions with replaced substring of the version of mediawiki being branched - https://phabricator.wikimedia.org/T51392#3268157 (10greg) [15:56:32] 10Continuous-Integration-Config, 06Release-Engineering-Team (Kanban), 06Operations, 13Patch-For-Review: Create a basic RSpec unit test for operations/puppet - https://phabricator.wikimedia.org/T78342#3268156 (10greg) [16:00:10] 10Continuous-Integration-Config, 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Backlog), 06Reading-Infrastructure-Team-Backlog, and 3 others: Setup trending service CI - https://phabricator.wikimedia.org/T151469#3268160 (10greg) [16:00:24] 10MediaWiki-Releasing, 06Release-Engineering-Team (Kanban), 06Security-Team: Streamline/automate MW tarball security release process - https://phabricator.wikimedia.org/T156445#3268162 (10greg) [16:00:38] 10Continuous-Integration-Infrastructure, 06Release-Engineering-Team (Backlog), 06Operations, 13Patch-For-Review: Secondary production Jenkins for CI - https://phabricator.wikimedia.org/T150771#3268164 (10greg) [16:00:43] 06Release-Engineering-Team (Backlog), 10Monitoring, 06Operations, 07Tracking, 07Wikimedia-Incident: Tracking: Monitoring and alerts for "business" metrics - https://phabricator.wikimedia.org/T140942#3268166 (10greg) [16:01:29] (done) [16:10:43] 06Release-Engineering-Team (Watching / External), 10DBA, 06Operations: Audit all existing code to ensure that any extension currently or previously adding blobs to ExternalStore has been registering a reference in the text table (and fix up if wrong) - https://phabricator.wikimedia.org/T106388#3268180 (10Jdfo... [16:19:03] 06Release-Engineering-Team (Watching / External), 10DBA, 06Operations: Audit all existing code to ensure that any extension currently or previously adding blobs to ExternalStore has been registering a reference in the text table (and fix up if wrong) - https://phabricator.wikimedia.org/T106388#3268191 (10jcre... [16:19:52] 06Release-Engineering-Team (Watching / External): Audit all existing code to ensure that any extension currently or previously adding blobs to ExternalStore has been registering a reference in the text table (and fix up if wrong) - https://phabricator.wikimedia.org/T106388#3268193 (10jcrespo) 05declined>03Ope... [16:20:01] 06Release-Engineering-Team (Watching / External): Audit all existing code to ensure that any extension currently or previously adding blobs to ExternalStore has been registering a reference in the text table (and fix up if wrong) - https://phabricator.wikimedia.org/T106388#3268203 (10jcrespo) p:05Normal>03Tri... [16:30:49] 06Release-Engineering-Team (Watching / External): Audit all existing code to ensure that any extension currently or previously adding blobs to ExternalStore has been registering a reference in the text table (and fix up if wrong) - https://phabricator.wikimedia.org/T106388#3268296 (10jcrespo) Despite my particul... [16:35:08] PROBLEM - Puppet errors on integration-slave-docker-1000 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [16:41:50] 06Release-Engineering-Team (Watching / External): Audit all existing code to ensure that any extension currently or previously adding blobs to ExternalStore has been registering a reference in the text table (and fix up if wrong) - https://phabricator.wikimedia.org/T106388#3268382 (10Jdforrester-WMF) >>! In T106... [17:05:40] PROBLEM - Puppet errors on buildlog is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [17:05:44] PROBLEM - Puppet errors on swift is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [17:07:18] PROBLEM - Puppet errors on swift-storage-01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [17:10:11] RECOVERY - Puppet errors on integration-slave-docker-1000 is OK: OK: Less than 1.00% above the threshold [0.0] [17:10:43] 10Gerrit: Gerrit code review view jumps/scrolls up and down when commenting - https://phabricator.wikimedia.org/T159919#3268451 (10Paladox) Hi, i believe this is the same problem as https://groups.google.com/forum/#!topic/repo-discuss/MI6fOXpo1Mw There workaround is to set rendering to slow instead of fast. [18:26:03] 10Continuous-Integration-Config, 10Page-Previews: Execute mwgate-composer workers in CI while building Popups extension - https://phabricator.wikimedia.org/T165521#3268886 (10pmiazga) [18:27:33] 10Continuous-Integration-Infrastructure, 10Page-Previews: Execute mwgate-composer workers in CI while building Popups extension - https://phabricator.wikimedia.org/T165521#3268899 (10pmiazga) [18:39:10] sorry, this may be offtopic here, but I do not know how to contact deployers [18:39:31] I am seeing 97f3ab1c77225e5535 merged but probably not deployed [18:40:19] I pinged the merger, but he doesn't answer [18:41:45] not sure in which state I should leve tin's repo? My patch cherry-picked? [20:59:06] 06Release-Engineering-Team (Watching / External), 06MediaWiki-Platform-Team: Audit all existing code to ensure that any extension currently or previously adding blobs to ExternalStore has been registering a reference in the text table (and fix up if wrong) - https://phabricator.wikimedia.org/T106388#3269253 (10... [21:24:17] 06Release-Engineering-Team (Kanban), 10Wikibugs, 15User-greg: Fix wikibugs config post #releng workboard refactor - https://phabricator.wikimedia.org/T165537#3269293 (10greg) [21:34:27] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [21:41:49] 06Release-Engineering-Team, 05Security: Review of reddit post about keyholder - https://phabricator.wikimedia.org/T161579#3269363 (10Bawolff) 05Open>03Resolved a:03Bawolff [21:41:51] 06Release-Engineering-Team, 05Security: Review of reddit post about keyholder - https://phabricator.wikimedia.org/T161579#3135889 (10Bawolff) a:05Bawolff>03None [21:41:53] 06Release-Engineering-Team, 05Security: Review of reddit post about keyholder - https://phabricator.wikimedia.org/T161579#3135889 (10Bawolff) [22:08:33] PROBLEM - Puppet staleness on deployment-ms-fe02 is CRITICAL: CRITICAL: 11.11% of data above the critical threshold [43200.0] [23:20:05] Project beta-update-databases-eqiad build #17147: 04FAILURE in 5 sec: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/17147/ [23:20:31] thats ^^ failing with [23:20:32] Exception: ('command: ', '/usr/local/bin/mwscript update.php --wiki=aawiki --quick', 'output: ', '#!/usr/bin/env php\nMediaWiki 1.30.0-alpha Updater\n\noojs/oojs-ui: 0.21.4 installed, 0.21.3 required.\nError: your composer.lock file is not up to date. Run "composer update" to install newer dependencies\n') [23:44:26] RECOVERY - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is OK: OK: Less than 100.00% above the threshold [0.0]