[00:01:08] Yippee, build fixed! [00:01:10] Project beta-scap-eqiad build #180820: 09FIXED in 7 min 27 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/180820/ [00:01:15] yay, thanks both!! [00:08:08] 10Scap: Port scap tests to pytest - https://phabricator.wikimedia.org/T179885#3739339 (10greg) [00:10:45] 10Gitblit-Deprecate, 10Epic, 10MW-1.30-release-notes (WMF-deploy-2017-07-18_(1.30.0-wmf.10)), 10MW-1.31-release-notes (WMF-deploy-2017-11-07 (1.31.0-wmf.7)), 10Patch-For-Review: Fix references to git.wikimedia.org in all repos - https://phabricator.wikimedia.org/T139089#3250363 (10demon) >>! In T139089#3... [00:25:43] 10Release-Engineering-Team (Watching / External), 10MediaWiki-Platform-Team, 10Wikidata: What happens if the Wikibase project specifies a version of a library outside of the range included in mediawiki-vendor? - https://phabricator.wikimedia.org/T179663#3739367 (10Legoktm) If Wikibase uses a feature of ~2.0... [00:32:46] PROBLEM - Puppet errors on deployment-puppetdb01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [00:53:15] 10MediaWiki-Releasing, 10Release-Engineering-Team: Removal of EOL release branches broke SemanticMediaWiki CI - https://phabricator.wikimedia.org/T179754#3739450 (10Legoktm) I filed: * https://github.com/SemanticMediaWiki/SemanticMediaWiki/issues/2809 - Use tags for CI testing against EOL MediaWiki branches *... [01:07:47] RECOVERY - Puppet errors on deployment-puppetdb01 is OK: OK: Less than 1.00% above the threshold [0.0] [01:09:35] landed D871, https://phabricator.wikimedia.org/rMSCA321b4a274f27d22b31d6cb2ff06555e36de58d5c [01:09:35] D871: Logging: Don't treat % as format string unless there are format args - https://phabricator.wikimedia.org/D871 [01:10:49] hopefully that resolves the format string error while rereverting the sh module changes [01:44:42] 10Release-Engineering-Team (Watching / External), 10MediaWiki-Platform-Team, 10Wikidata: What happens if the Wikibase project specifies a version of a library outside of the range included in mediawiki-vendor? - https://phabricator.wikimedia.org/T179663#3739502 (10bd808) @Legoktm nailed it. If wikibase is go... [03:59:40] Project selenium-MultimediaViewer » firefox,mediawiki,Linux,BrowserTests build #570: 04FAILURE in 3 min 39 sec: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=mediawiki,PLATFORM=Linux,label=BrowserTests/570/ [04:11:15] Project selenium-MultimediaViewer » safari,beta,OS X 10.9,BrowserTests build #570: 04FAILURE in 15 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=safari,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=OS%20X%2010.9,label=BrowserTests/570/ [04:19:42] Project selenium-MultimediaViewer » chrome,beta,OS X 10.9,BrowserTests build #570: 04FAILURE in 23 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=OS%20X%2010.9,label=BrowserTests/570/ [04:22:27] Project selenium-MultimediaViewer » firefox,beta,Linux,BrowserTests build #570: 04FAILURE in 26 min: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/570/ [05:42:14] 10Release-Engineering-Team (Backlog), 10Deployments, 10Operations, 10Beta-Cluster-reproducible, and 2 others: Switch mwscript from Zend PHP5 to default php alternative (e.g. HHVM or PHP7) - https://phabricator.wikimedia.org/T146285#3739730 (10Krinkle) >>! In T146285#3728364, @hashar wrote: > @Krinkle point... [06:20:22] 10Gerrit, 10Operations, 10Readers-Web-Backlog, 10Patch-For-Review, and 2 others: [subtask] Temporarily allow pushing large objects - https://phabricator.wikimedia.org/T178189#3739755 (10phuedx) >>! In T178189#3698691, @MoritzMuehlenhoff wrote: >> But the proper fix would be to run this service on stretch <... [06:53:26] 10commit-message-validator: Revert messages for valid commits still trigger validation failures - https://phabricator.wikimedia.org/T174136#3551961 (10Tgr) Shouldn't it ignore all rules? You are not really supposed to edit the description of commits you revert. The same probably goes for cherry-picks. [06:57:23] 10commit-message-validator: Make errors more colorful - https://phabricator.wikimedia.org/T179904#3739778 (10Tgr) [07:03:37] 10commit-message-validator: Make Change-Id optional - https://phabricator.wikimedia.org/T179905#3739796 (10Tgr) [07:48:38] 10Gerrit, 10Operations, 10Readers-Web-Backlog, 10Patch-For-Review, and 2 others: [subtask] Temporarily allow pushing large objects - https://phabricator.wikimedia.org/T178189#3739864 (10MoritzMuehlenhoff) Our scb* cluster currently runs jessie. I don't know the time frame for the new setup, but running the... [08:37:48] 10Gerrit, 10Operations, 10Readers-Web-Backlog, 10Patch-For-Review, and 2 others: [subtask] Temporarily allow pushing large objects - https://phabricator.wikimedia.org/T178189#3683839 (10mobrovac) While migrating SCB nodes to stretch will need to happen, I don't think we will have the bandwidth to do so soo... [08:41:27] (03PS1) 10Gergő Tisza: Allow _ in unit test method names [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/389668 [08:58:31] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Patch-For-Review: Install zip extension for CI - https://phabricator.wikimedia.org/T179772#3739982 (10hashar) a:03Umherirrender Patch https://gerrit.wikimedia.org/r/#/c/389512/ looks good and gets `zip` installed. [09:16:40] 10Gerrit, 10Operations, 10Readers-Web-Backlog, 10Patch-For-Review, and 2 others: [subtask] Temporarily allow pushing large objects - https://phabricator.wikimedia.org/T178189#3740018 (10phuedx) >>! In T178189#3739931, @mobrovac wrote: > (Also, this is highly off-topic for this ticket, we should probably cr... [09:33:18] 10Gerrit, 10Operations, 10Readers-Web-Backlog, 10Patch-For-Review, and 2 others: [subtask] Temporarily allow pushing large objects - https://phabricator.wikimedia.org/T178189#3683839 (10akosiaris) >>! In T178189#3739864, @MoritzMuehlenhoff wrote: > Our scb* cluster currently runs jessie. I don't know the t... [10:15:47] (03Draft1) 10MarcoAurelio: Archive the BookManagerv2 extension [integration/config] - 10https://gerrit.wikimedia.org/r/389680 (https://phabricator.wikimedia.org/T178938) [10:15:54] (03PS2) 10MarcoAurelio: Archive the BookManagerv2 extension [integration/config] - 10https://gerrit.wikimedia.org/r/389680 (https://phabricator.wikimedia.org/T178938) [10:34:57] 10Release-Engineering-Team (Watching / External), 10Discovery, 10Discovery-Analysis, 10Operations, and 2 others: Setup a mirror for R language dependencies (CRAN) - https://phabricator.wikimedia.org/T170995#3740493 (10Addshore) [10:36:29] 10Continuous-Integration-Config, 10Release-Engineering-Team (Watching / External), 10Discovery, 10Discovery-Analysis (Current work), 10Patch-For-Review: Add lint/CI to all wikimedia/discovery analytics repositories - https://phabricator.wikimedia.org/T153856#2893206 (10Addshore) So I setup a rough lintR... [10:51:26] (03CR) 10Hashar: [C: 032] Archive the BookManagerv2 extension [integration/config] - 10https://gerrit.wikimedia.org/r/389680 (https://phabricator.wikimedia.org/T178938) (owner: 10MarcoAurelio) [10:53:00] (03Merged) 10jenkins-bot: Archive the BookManagerv2 extension [integration/config] - 10https://gerrit.wikimedia.org/r/389680 (https://phabricator.wikimedia.org/T178938) (owner: 10MarcoAurelio) [10:54:58] (03CR) 10Hashar: [C: 032] Use voting jobs for HAWelcome [integration/config] - 10https://gerrit.wikimedia.org/r/389504 (owner: 10Umherirrender) [10:56:22] (03Merged) 10jenkins-bot: Use voting jobs for HAWelcome [integration/config] - 10https://gerrit.wikimedia.org/r/389504 (owner: 10Umherirrender) [10:58:19] 10Scap, 10User-Ladsgroup: Port scap tests to pytest - https://phabricator.wikimedia.org/T179885#3740580 (10Ladsgroup) a:03Ladsgroup [11:14:43] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: expired gpg key preventing new entries to releng pws - https://phabricator.wikimedia.org/T179824#3740607 (10zeljkofilipin) a:05zeljkofilipin>03hashar Thanks @thcipriani, I have done as you have suggested. @hashar please check if all is good now an... [11:45:45] PROBLEM - Puppet errors on integration-slave-jessie-android is CRITICAL: CRITICAL: 50.00% of data above the critical threshold [0.0] [11:50:40] PROBLEM - Puppet errors on webperformance is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [11:55:29] PROBLEM - Puppet errors on integration-slave-jessie-1001 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [12:16:45] (03CR) 10Hashar: [C: 031] Run search/mjolnir pyspark tests via tox (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/387658 (owner: 10EBernhardson) [12:17:02] (03PS7) 10Hashar: Run search/mjolnir pyspark tests via tox [integration/config] - 10https://gerrit.wikimedia.org/r/387658 (owner: 10EBernhardson) [12:17:58] (03CR) 10Hashar: "Rebuild and being uploaded with:" [integration/config] - 10https://gerrit.wikimedia.org/r/387658 (owner: 10EBernhardson) [12:20:44] RECOVERY - Puppet errors on integration-slave-jessie-android is OK: OK: Less than 1.00% above the threshold [0.0] [12:22:53] Yippee, build fixed! [12:22:53] Project selenium-GettingStarted » firefox,beta,Linux,BrowserTests build #579: 09FIXED in 52 sec: https://integration.wikimedia.org/ci/job/selenium-GettingStarted/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/579/ [12:23:06] PROBLEM - Puppet errors on jenkinstest is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [12:23:08] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [12:26:04] PROBLEM - Puppet errors on integration-slave-docker-1002 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [12:28:48] PROBLEM - Puppet errors on saucelabs-01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [12:29:48] (03CR) 10Hashar: [C: 032] "I have refreshed the Jenkins jobs and pull the Docker image on all slaves. Congrats on all the follow ups!" [integration/config] - 10https://gerrit.wikimedia.org/r/387658 (owner: 10EBernhardson) [12:30:13] PROBLEM - Puppet errors on integration-slave-docker-1005 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [12:30:29] RECOVERY - Puppet errors on integration-slave-jessie-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [12:30:43] RECOVERY - Puppet errors on webperformance is OK: OK: Less than 1.00% above the threshold [0.0] [12:31:20] (03Merged) 10jenkins-bot: Run search/mjolnir pyspark tests via tox [integration/config] - 10https://gerrit.wikimedia.org/r/387658 (owner: 10EBernhardson) [12:33:28] PROBLEM - Puppet errors on castor02 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [12:44:21] 10Gerrit, 10Operations, 10Readers-Web-Backlog, 10Patch-For-Review, and 2 others: [subtask] Temporarily allow pushing large objects - https://phabricator.wikimedia.org/T178189#3740805 (10phuedx) >>! In T178189#3740029, @akosiaris wrote: > Do we have a timeline for when we can (and want to) have this working... [13:03:04] RECOVERY - Puppet errors on jenkinstest is OK: OK: Less than 1.00% above the threshold [0.0] [13:03:46] RECOVERY - Puppet errors on saucelabs-01 is OK: OK: Less than 1.00% above the threshold [0.0] [13:06:03] RECOVERY - Puppet errors on integration-slave-docker-1002 is OK: OK: Less than 1.00% above the threshold [0.0] [13:08:27] RECOVERY - Puppet errors on castor02 is OK: OK: Less than 1.00% above the threshold [0.0] [13:10:16] RECOVERY - Puppet errors on integration-slave-docker-1005 is OK: OK: Less than 1.00% above the threshold [0.0] [13:11:43] (03PS1) 10Hashar: Support build varying timeout for tox-docker [integration/config] - 10https://gerrit.wikimedia.org/r/389704 [13:15:09] (03CR) 10Hashar: [C: 032] Support build varying timeout for tox-docker [integration/config] - 10https://gerrit.wikimedia.org/r/389704 (owner: 10Hashar) [13:16:18] (03Merged) 10jenkins-bot: Support build varying timeout for tox-docker [integration/config] - 10https://gerrit.wikimedia.org/r/389704 (owner: 10Hashar) [14:42:21] (03PS1) 10Hashar: Set operations/debs/varnish4 timeout to 60 minutes [integration/config] - 10https://gerrit.wikimedia.org/r/389717 [14:43:00] (03CR) 10Hashar: [C: 032] Set operations/debs/varnish4 timeout to 60 minutes [integration/config] - 10https://gerrit.wikimedia.org/r/389717 (owner: 10Hashar) [14:44:04] (03CR) 10jerkins-bot: [V: 04-1] Set operations/debs/varnish4 timeout to 60 minutes [integration/config] - 10https://gerrit.wikimedia.org/r/389717 (owner: 10Hashar) [14:45:06] (03CR) 10jerkins-bot: [V: 04-1] Set operations/debs/varnish4 timeout to 60 minutes [integration/config] - 10https://gerrit.wikimedia.org/r/389717 (owner: 10Hashar) [14:48:16] (03PS2) 10Hashar: Set operations/debs/varnish4 timeout to 60 minutes [integration/config] - 10https://gerrit.wikimedia.org/r/389717 [14:49:57] (03CR) 10Hashar: [C: 032] Set operations/debs/varnish4 timeout to 60 minutes [integration/config] - 10https://gerrit.wikimedia.org/r/389717 (owner: 10Hashar) [14:51:05] (03Merged) 10jenkins-bot: Set operations/debs/varnish4 timeout to 60 minutes [integration/config] - 10https://gerrit.wikimedia.org/r/389717 (owner: 10Hashar) [14:57:27] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Backlog), 10Wikidata-Sprint-2015-11-03: selenium fails to connect to firefox (headless not sauce) - https://phabricator.wikimedia.org/T117561#3741360 (10zeljkofilipin) 05Open>03declined Unlikely to ever be resolved because of {T139740}. [14:58:11] 10Continuous-Integration-Infrastructure, 10User-zeljkofilipin: Upgrade to Chromium 59 or newer on Debian Jessie in CI - https://phabricator.wikimedia.org/T170032#3741366 (10zeljkofilipin) [14:58:44] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikidata, 10MW-1.31-release-notes (WMF-deploy-2017-11-07 (1.31.0-wmf.7)), and 3 others: Run Wikibase daily browser tests on Jenkins - https://phabricator.wikimedia.org/T167432#3741370 (10zeljkofilipin) [14:59:38] 10Release-Engineering-Team (Backlog), 10Readers-Web-Backlog (Tracking), 10User-zeljkofilipin, 10Wikimedia-Incident: Run subset of MobileFrontend browser tests on merges in core - https://phabricator.wikimedia.org/T165940#3741389 (10zeljkofilipin) [15:00:34] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: expired gpg key preventing new entries to releng pws - https://phabricator.wikimedia.org/T179824#3741391 (10hashar) Solved :) thank you! [15:04:28] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Next), 10Wikidata, 10Patch-For-Review, and 3 others: Increase in failures caused by Saucelabs - https://phabricator.wikimedia.org/T152963#3741417 (10zeljkofilipin) 05Open>03declined The only thing we can do is avoid Sauce Labs: {T167432} [15:05:09] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Backlog), 10MediaWiki-extensions-GettingStarted, 10Jenkins, 10Upstream: Missing XML files cause "Publish Performance test result report" - https://phabricator.wikimedia.org/T164296#3741426 (10zeljkofilipin) [15:08:46] no_justification hi, how do you build gerrit 2.13 with buck? [15:08:59] i am getting buck errors [15:09:22] I think the command was `buck build :release` [15:09:34] (I need to re-clone gerrit's source, new laptop) [15:09:42] ah i see [15:09:54] hmm, buck fails when i try to build buck from source [15:10:01] ie checkout the version used in .buckversion [15:10:22] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Backlog), 10Jenkins, 10Readers-Web-Backlog (Tracking), and 3 others: MEDIAWIKI_URL may be set to incorrect value in mwext-mw-selenium job - https://phabricator.wikimedia.org/T144912#3741436 (10zeljkofilipin) 05Open>03declined Unlikely to ever... [15:10:52] ah [15:10:57] i found a working version [15:10:58] 19d299556d809e3e048125bd1cb5ff323929dab6 [15:11:06] 10Browser-Tests-Infrastructure, 10RelEng-Archive-FY201718-Q1, 10Epic, 10Patch-For-Review, 10Tracking: [EPIC] trigger browser tests from Gerrit (tracking) - https://phabricator.wikimedia.org/T55697#3741448 (10zeljkofilipin) [15:11:10] though i hope that version dosen't prevent using python imports [15:11:58] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Backlog), 10Jenkins, 10User-zeljkofilipin: Browser test Jenkins videos do not always play in-browser - https://phabricator.wikimedia.org/T155794#3741450 (10zeljkofilipin) 05stalled>03declined There is a workaround. [15:12:17] It seems buck is now broken if you use the one in .buckversion no_justification [15:12:23] * paladox submits a patch to update to 19d299556d809e3e048125bd1cb5ff323929dab6 [15:12:30] 10Release-Engineering-Team (Backlog): When beta cluster is down Jenkins jobs should be aborted and not trigger e-mail notifications - https://phabricator.wikimedia.org/T101563#3741452 (10zeljkofilipin) [15:13:57] 10Release-Engineering-Team: Nightly builds tested and used for production deployments - https://phabricator.wikimedia.org/T67126#3741457 (10zeljkofilipin) [15:13:59] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Backlog): Run browser tests against the nightly build version on Beta Cluster - https://phabricator.wikimedia.org/T67128#3741455 (10zeljkofilipin) 05stalled>03declined Replaced with #releasepipeline. [15:14:23] 10Release-Engineering-Team (Backlog), 10Deployments: Run @integration tests on new deploy branch creation - https://phabricator.wikimedia.org/T111545#3741459 (10zeljkofilipin) [15:14:40] 10Release-Engineering-Team (Backlog), 10Deployments: Display and/or announce build status of wmf branch cut tests (including @integration tests) - https://phabricator.wikimedia.org/T111823#3741460 (10zeljkofilipin) [15:14:54] Heh, I didn't even have the JDK installed yet [15:15:00] !log Created VPS account "PortalsBuilder" - T179694 [15:15:05] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:15:05] T179694: Create a Jenkins Job that builds the portal deployment artifacts in CI - https://phabricator.wikimedia.org/T179694 [15:15:20] lol [15:15:48] well now i am getting a bunch of errors due to deps. when building gerrit. [15:16:08] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team (Backlog): Investigate distribution of browser test run time - https://phabricator.wikimedia.org/T104396#3741464 (10zeljkofilipin) 05Open>03declined Unlikely to ever be resolved because of {T139740}. [15:16:19] aha [15:16:22] because i am sudo [15:16:31] sudo works with bazel :) [15:17:16] "The type java.lang.Object cannot be resolved" [15:17:21] That doesn't....sound right [15:17:47] are you using java 8? [15:17:53] Java 9 [15:18:48] ah [15:18:51] that will fail [15:19:03] someone already filled an issue for that [15:19:13] Oh yeah, JAVA_HOME isn't set, for one [15:19:46] 10Browser-Tests-Infrastructure, 10Release-Engineering-Team, 10Epic: Make browser tests voting for all repos of WMF deployed code - https://phabricator.wikimedia.org/T91669#3741473 (10zeljkofilipin) 05Open>03Resolved a:03zeljkofilipin This is implemented in Node.js Selenium framework (T139740). For all... [15:20:34] heh [15:20:36] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Discovery, 10Wikimedia-Portals, and 2 others: Create a Jenkins Job that builds the portal deployment artifacts in CI - https://phabricator.wikimedia.org/T179694#3741482 (10zeljkofilipin) [15:20:38] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: expired gpg key preventing new entries to releng pws - https://phabricator.wikimedia.org/T179824#3741481 (10zeljkofilipin) 05Open>03Resolved [15:22:02] 10Release-Engineering-Team (Backlog), 10Patch-For-Review, 10Upstream, 10WorkType-Maintenance: mediawiki_api gem recursion on log_in - https://phabricator.wikimedia.org/T111133#3741487 (10zeljkofilipin) [15:22:28] 10Release-Engineering-Team (Backlog), 10Ruby: Ruby mediawiki_api client hides error details - https://phabricator.wikimedia.org/T127786#3741488 (10zeljkofilipin) [15:22:38] 10Release-Engineering-Team (Backlog), 10Ruby: Mediawiki Ruby gem incorrectly assumes path to index.php - https://phabricator.wikimedia.org/T149169#3741492 (10zeljkofilipin) [15:22:44] 10Continuous-Integration-Infrastructure (shipyard), 10Operations, 10Patch-For-Review: wikimedia-jessie & wikimedia-stretch docker images don't have deb-src set for apt.wikimedia.org - https://phabricator.wikimedia.org/T179354#3741493 (10akosiaris) >>! In T179354#3733948, @hashar wrote: > `apt-get build-dep h... [15:23:00] 10Release-Engineering-Team (Backlog), 10Documentation, 10Ruby: Improve mediawiki_api documentation with inline yard - https://phabricator.wikimedia.org/T102726#3741495 (10zeljkofilipin) [15:23:20] 10Release-Engineering-Team (Watching / External), 10Wikipedia-Android-App-Backlog, 10Wikipedia-iOS-App-Backlog: Create end-to-end automated test for Wikipedia native app(s) - https://phabricator.wikimedia.org/T90177#3741496 (10zeljkofilipin) [15:27:54] Nov 07, 2017 3:24:51 PM java.util.prefs.FileSystemPreferences$1 run [15:27:54] INFO: Created user preferences directory. [15:30:53] yay buck works now. [15:31:02] But bazel shows me where the .war is [15:31:47] ah [15:31:58] buck requires 1.7 at the version 2.13 requires [15:33:45] yay there's /usr/bin/set-java.sh [15:34:36] (03PS1) 10Hashar: zuul: add comment about pbuilder parameters [integration/config] - 10https://gerrit.wikimedia.org/r/389723 [15:34:39] no_justification apparently buck is not compatible with java 8 until 19d299556d809e3e048125bd1cb5ff323929dab6 [15:34:48] /usr/bin/set-java.sh 7 works [15:35:44] (03PS1) 10Hashar: Build operations/debs/varnish4 with parallel=12 [integration/config] - 10https://gerrit.wikimedia.org/r/389725 [15:36:04] It doesn't? It's been working for me on my old machine [15:36:07] (and seems to be working now) [15:36:19] Hmm, buck fails for me on ubuntu 16.04 [15:36:25] but if i use 7 [15:36:26] it works [15:36:36] (03CR) 10Hashar: [C: 032] zuul: add comment about pbuilder parameters [integration/config] - 10https://gerrit.wikimedia.org/r/389723 (owner: 10Hashar) [15:36:37] Hmmm...it clearly worked before [15:36:43] (03CR) 10Hashar: [C: 032] Build operations/debs/varnish4 with parallel=12 [integration/config] - 10https://gerrit.wikimedia.org/r/389725 (owner: 10Hashar) [15:36:47] Because I *built* against java8 [15:37:16] hmm [15:37:55] (03Merged) 10jenkins-bot: zuul: add comment about pbuilder parameters [integration/config] - 10https://gerrit.wikimedia.org/r/389723 (owner: 10Hashar) [15:38:01] (03Merged) 10jenkins-bot: Build operations/debs/varnish4 with parallel=12 [integration/config] - 10https://gerrit.wikimedia.org/r/389725 (owner: 10Hashar) [15:38:20] it must be me then heh. [15:39:55] Now it's not working, ofc [15:40:32] tbh, we don't really need to build stable-2.13 again [15:40:47] errors like [15:40:48] 1. ERROR in /opt/buck-java8/src/com/facebook/buck/jvm/java/tracing/TranslatingJavacPhaseTracer.java (at line 63) [15:40:48] [javac] public class TranslatingJavacPhaseTracer implements JavacPhaseTracer, AutoCloseable { [15:41:12] yeh. bazel is so much easier, dosen't require us to build bazel :) [15:45:47] * paladox moves onto testing bazel now [15:46:35] Maven was just fine, for what it's worth :p [15:46:53] * no_justification had a longggggg discussion about this awhile back when they ditched maven for buck [15:47:48] heh [15:49:08] no_justification they are still using ubuntu 15.04 for ci lol. I am trying to get them to update as everything is going to break lol. Like the next time they update the image, chrome is going to break. [15:49:22] i've been trying to upgrade since april. [15:49:23] https://gerrit-review.googlesource.com/#/c/gerrit-ci-scripts/+/102155/ [15:49:44] What's wrong with ubuntu 15.04? [15:49:53] it's no longer supported [15:50:05] That doesn't make it a bad OS :) [15:50:06] so nodejs dosen't support it [15:50:14] * no_justification hates node anyway, so meh [15:50:16] :p [15:50:18] heh [15:50:24] polygerrit needs that to be built [15:50:40] * no_justification stays on gerrit 2.13.x forever [15:50:41] :p [15:50:44] then there's chrome which is also required by polygerrit to run tests. [15:50:46] lololololol [15:50:59] Hey, we stayed on 2.8.x welllllll past its EOL [15:51:03] 2.8.x was a good release [15:51:23] heh [15:52:40] 2.14 includes a security update for us :) (plugin wise). that i carn't say. [15:54:20] no_justification what do you think of https://docs.google.com/presentation/d/17q-ygGioZi_5DITLyELa8oaOr22e15AHy8cq6XTZ0nY/edit#slide=id.g27f16618ec_0_139 ? :) [16:11:07] PROBLEM - Puppet errors on deployment-cache-upload04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [16:13:03] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikidata, 10MW-1.31-release-notes (WMF-deploy-2017-11-07 (1.31.0-wmf.7)), and 3 others: Run Wikibase daily browser tests on Jenkins - https://phabricator.wikimedia.org/T167432#3741656 (10WMDE-leszek) So it seems @zeljkofilipin... [16:15:02] !log Created portalsbuilder in Gerrit, generated a ssh key pair for it and stored in Jenkins credentials store - T179694 [16:15:07] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:15:09] T179694: Create a Jenkins Job that builds the portal deployment artifacts in CI - https://phabricator.wikimedia.org/T179694 [16:23:49] PROBLEM - Puppet errors on deployment-cache-text04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [16:28:39] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Patch-For-Review: Install zip extension for CI - https://phabricator.wikimedia.org/T179772#3741705 (10Umherirrender) Now merged, the refresh of the nodepool snapshot is the next step [16:34:19] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Discovery, 10Wikimedia-Portals, and 2 others: Create a Jenkins Job that builds the portal deployment artifacts in CI - https://phabricator.wikimedia.org/T179694#3741765 (10debt) p:05Triage>03Normal [16:38:22] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Wikidata, 10MW-1.31-release-notes (WMF-deploy-2017-11-07 (1.31.0-wmf.7)), and 3 others: Run Wikibase daily browser tests on Jenkins - https://phabricator.wikimedia.org/T167432#3741774 (10WMDE-leszek) oh, wait. Looking a screen... [16:39:46] 10Gerrit, 10Operations, 10Readers-Web-Backlog, 10Patch-For-Review, and 2 others: [subtask] Temporarily allow pushing large objects - https://phabricator.wikimedia.org/T178189#3741790 (10akosiaris) >>! In T178189#3740805, @phuedx wrote: >>>! In T178189#3740029, @akosiaris wrote: >> Do we have a timeline for... [16:44:57] (03PS1) 10Chad: Remove VectorBeta, not installed anymore [tools/release] - 10https://gerrit.wikimedia.org/r/389733 [16:45:15] (03CR) 10Chad: [C: 032] Remove VectorBeta, not installed anymore [tools/release] - 10https://gerrit.wikimedia.org/r/389733 (owner: 10Chad) [16:45:47] (03Merged) 10jenkins-bot: Remove VectorBeta, not installed anymore [tools/release] - 10https://gerrit.wikimedia.org/r/389733 (owner: 10Chad) [16:52:26] jan_drewniak: something is building at https://integration.wikimedia.org/ci/job/wikimedia-portals-build/4/console will polish it up tomorrow [16:56:59] 10Gitblit-Deprecate, 10Epic, 10MW-1.30-release-notes (WMF-deploy-2017-07-18_(1.30.0-wmf.10)), 10MW-1.31-release-notes (WMF-deploy-2017-11-07 (1.31.0-wmf.7)), 10Patch-For-Review: Fix references to git.wikimedia.org in all repos - https://phabricator.wikimedia.org/T139089#3741810 (10Umherirrender) >>! In T... [16:57:20] (03PS2) 10Hashar: Job for building wikimedia portals [integration/config] - 10https://gerrit.wikimedia.org/r/389468 (https://phabricator.wikimedia.org/T179694) (owner: 10Jdrewniak) [17:00:43] addshore: Hey, question. I was building the wmf.7 branch, and I noticed the submodules in view/lib/* (yay) [17:00:50] But I noticed the submodules are from github (boo) [17:00:54] We don't deploy from github. [17:02:02] (03CR) 10Hashar: [C: 04-2] "a first build is https://integration.wikimedia.org/ci/job/wikimedia-portals-build/4/console" (038 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/389468 (https://phabricator.wikimedia.org/T179694) (owner: 10Jdrewniak) [17:03:38] Not only that we *don't*, we can't [17:03:44] tin can't hit the outside world :) [17:05:31] We *could* deploy from Phabricator -- which would make mirroring github back internally easier [17:05:57] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Discovery, 10Wikimedia-Portals, and 2 others: Create a Jenkins Job that builds the portal deployment artifacts in CI - https://phabricator.wikimedia.org/T179694#3741828 (10hashar) **Status** Jan and I had a chat yesterday about it. h... [17:06:25] (03CR) 10Umherirrender: [C: 031] Archive the VectorBeta extension [integration/config] - 10https://gerrit.wikimedia.org/r/389444 (https://phabricator.wikimedia.org/T178932) (owner: 10MarcoAurelio) [17:07:45] hey hey no_justification ! [17:08:11] :) [17:09:04] so, for the branch, the plan was to take it from master [17:09:12] but hmmm, yes, 3 of the submodules are indeed from github [17:09:31] Branching from master is what we did, that's not a problem. It's just that the repos are external :) [17:09:57] hmm, wait, so branching from Wikibase master right? [17:10:06] rather than wikidata (the build) [17:10:23] Looks like it, yeah [17:10:48] ack [17:11:14] Yep, master and wmf.7 both point to c4a2844b4acb8c2a2f0a126350ceb4394e91df4b [17:11:26] So yeh, "Wikibase" still isn't deployed yet, but yeh, when cutting the branch as it is in the script i guess thats where the issue is [17:12:11] "We *could* deploy from Phabricator -- which would make mirroring github back internally easier" how would that work? [17:12:25] We'd set up repos in Phabricator which poll from Github [17:12:33] So we'd be pulling master back in constantly [17:13:10] If we did it from Gerrit, it would require manual syncs of master @ github back to Gerrit [17:13:32] oooh, I didn't even know that pulling to phab from github was an option [17:13:46] Phab has the concept of "watching" a repo and just being a mirror [17:14:00] Can we run with that then? [17:14:19] We can try that route, yeah [17:15:12] 10Continuous-Integration-Infrastructure, 10Jenkins, 10Patch-For-Review: Install zip extension for CI - https://phabricator.wikimedia.org/T179772#3741865 (10hashar) The snapshot automatically get regenerated at 14:14 UTC. It failed though due to an unrelated issue: ``` Error: Execution of '/usr/bin/gem insta... [17:15:45] I'm not sure if the dev team want to move development to gerrit, but i guess that would be the other real option [17:15:49] !log Updating Nodepool snapshot to get php5.5-zip - T179772 [17:15:53] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:15:53] T179772: Install zip extension for CI - https://phabricator.wikimedia.org/T179772 [17:18:20] addshore: In an ideal/simple world from my end, yes. But I understand hesitation for that from your end :) [17:18:36] Here's the first repo, btw :) https://phabricator.wikimedia.org/source/wikibase-serialization/ [17:19:20] awesome, can I let you make the patch for wikibase updating the submodules URLS? and then I'll merge (and also make a new build) [17:19:21] 10Gitblit-Deprecate, 10Epic, 10MW-1.30-release-notes (WMF-deploy-2017-07-18_(1.30.0-wmf.10)), 10MW-1.31-release-notes (WMF-deploy-2017-11-07 (1.31.0-wmf.7)), 10Patch-For-Review: Fix references to git.wikimedia.org in all repos - https://phabricator.wikimedia.org/T139089#3741883 (10demon) Some of it. Not... [17:20:06] also no_justification would I be okay to use some of the train time today to start rattling through some config moves? [17:20:54] Yeah, a patch for the submodule urls @ master would be ideal [17:21:00] Then we'll update the wmf.7 branch of Wikibase [17:21:06] And should be good to go [17:24:07] And yeah, if we can get the submodule stuff sorted before the train window, we can use post-deploy time to futz with the config changes [17:24:33] awesome! [17:24:57] you to make the patch for wikibase master with the URLs or me? :) If me, give me a ping when they are setup, if you, give me a ping once i can +2 the patch! [17:25:36] got 13 config moving patches to slowly step through :) [17:25:48] Repo #2 importing now https://phabricator.wikimedia.org/source/datavalues-javascript/ [17:26:16] addshore hi, have you managed to get chrome working in docker? [17:26:26] did you have to do --security-opt seccomp:unconfined ? [17:27:30] !log Image snapshot-ci-jessie-1510074928 in wmflabs-eqiad is ready - T179772 [17:27:34] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:27:34] T179772: Install zip extension for CI - https://phabricator.wikimedia.org/T179772 [17:28:41] paladox: I have never tried :) [17:28:46] oh [17:29:36] Repo #3 importing https://phabricator.wikimedia.org/source/wikibase-data-model/ [17:29:41] So yay, that's the three of them [17:31:34] wheee [17:32:26] https://gerrit.wikimedia.org/r/#/c/389749/ bame [17:32:30] s/e// [17:33:20] sweet, will wait for CI, then merge, build and merge that! [17:33:42] Soon as it lands in master we'll cherry-pick to wmf.7 [17:33:44] :) [17:33:48] so phabricator get auto updated from whatever github has ? [17:34:05] iirc the main reason we have stuff on gerrit is that it is tightly controlled isn't it? [17:34:20] but yeah that will work :] [17:34:50] The submodules don't auto-update like this, so we're safe. The point isn't that it's tightly controlled, it's that /we/ control it [17:35:05] ok ok :]] [17:35:51] nb: I don't trust github ;-) [17:36:51] ebernhardson: good morning. Job merged :] [17:37:20] no_justification: yeah I was wondering what phabricator did :D [17:37:33] ebernhardson: so moljnir somehow get a job now! [17:37:43] hashar: Well I mean for another thing, tin can't hit the outside world [17:37:44] :) [17:37:54] hmm [17:38:20] http_proxy=webproxy.eqiad.wmnet:8080 git clone https://foo..x. [17:38:27] not sure how it would work via that proxy though [17:38:28] but yeah [17:38:34] got the point :] [17:38:42] dinner time! [17:40:07] addshore hasharAway if you ever get to migrating chrome tests to docker [17:40:09] you have to do [17:40:22] docker run --rm -it --security-opt seccomp:unconfined slave-bazel [17:40:28] something like ^^ [17:40:35] hasharAway: I know, but it sets a bad precedent :) [17:40:36] otherwise chrome just fails [17:42:59] 10Gerrit: Add mediawiki group to mediawiki/skins repo - https://phabricator.wikimedia.org/T179960#3742040 (10Umherirrender) [17:45:08] 10Gerrit: Add mediawiki group to mediawiki/skins repo - https://phabricator.wikimedia.org/T179960#3742057 (10demon) 05Open>03Resolved a:03demon [17:45:38] hasharAway: sweet, thanks! [17:51:25] 10Continuous-Integration-Config, 10Proton, 10Readers-Web-Backlog: Set up Jenkins for chromium-render and chromium-render-deploy repositories - https://phabricator.wikimedia.org/T179552#3742071 (10hashar) @phuedx yes that might be enough for chromium-render For the /deploy it is a bit more complicated. We do... [17:51:33] 17:48:29 ERROR: Error fetching remote repo 'origin' [17:51:35] 17:48:29 hudson.plugins.git.GitException: Failed to fetch from git://contint1001.wikimedia.org/mediawiki/core [17:51:56] eww [17:52:03] https://integration.wikimedia.org/ci/job/mediawiki-core-php55lint/32698/console [17:52:35] 10Continuous-Integration-Config, 10Proton, 10Readers-Web-Backlog: Set up Jenkins for chromium-render and chromium-render-deploy repositories - https://phabricator.wikimedia.org/T179552#3742077 (10phuedx) AFAIK the /deploy repo isn't set up yet. If we could get the chromium-render repo gating and submitting,... [17:52:50] no_justification: https://gerrit.wikimedia.org/r/#/c/389754/ [17:53:51] hasharAway ^^ [17:54:32] dinner time, ill be back for the train [17:55:07] https://integration.wikimedia.org/ci/job/mediawiki-core-jsduck/22367/console integration-slave-jessie-1003 is out of disk space? [17:55:45] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Intermittent tests failure - https://phabricator.wikimedia.org/T179962#3742101 (10Paladox) [17:55:53] /dev/mapper/vd-second--local--disk 21G 20G 0 100% /srv [17:56:18] 5.4G mwgate-php55lint [17:56:46] !log integration-slave-jessie-1003 /srv full, legoktm@integration-slave-jessie-1003:/srv/jenkins-workspace/workspace$ sudo rm -rf mwgate-* mediawiki-* [17:56:49] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:56:54] /dev/mapper/vd-second--local--disk 21G 8.5G 11G 45% /srv [17:57:09] addshore: Enjoy dinner, +2'd the wmf.7 backport [17:57:41] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Intermittent tests failure - https://phabricator.wikimedia.org/T179962#3742116 (10Paladox) p:05Triage>03High Making high as it affects some tests but not all. [17:57:50] 10Continuous-Integration-Infrastructure: mwgate-php55lint workspaces are getting huge - https://phabricator.wikimedia.org/T179963#3742118 (10Legoktm) [17:58:14] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team: Intermittent tests failure - https://phabricator.wikimedia.org/T179962#3742131 (10Legoktm) 05Open>03Resolved a:03Legoktm integration-slave-jessie-1003 /srv ran out of disk space. [17:58:22] Nikerabbit: should be fixed for now [17:58:42] 10Continuous-Integration-Infrastructure: mwgate-php55lint workspaces are getting huge - https://phabricator.wikimedia.org/T179963#3742134 (10Legoktm) [18:00:35] legoktm thanks [18:00:45] np [18:02:44] 10Continuous-Integration-Infrastructure: mwgate-php55lint workspaces are getting huge - https://phabricator.wikimedia.org/T179963#3742154 (10Legoktm) I had to do this a few days ago too: https://tools.wmflabs.org/sal/log/AV9_nwRdwg13V62859Va [18:03:56] 10Continuous-Integration-Infrastructure: mwgate-php55lint workspaces are getting huge - https://phabricator.wikimedia.org/T179963#3742158 (10Paladox) I guess we can setup a cron that deletes the repos every day? [18:04:08] PROBLEM - Puppet errors on deployment-netbox is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [18:05:50] RECOVERY - Free space - all mounts on integration-slave-jessie-1003 is OK: OK: All targets OK [18:07:05] twentyafterfour: thcipriani: beta-scap is failing, with > File "/usr/lib/python2.7/dist-packages/scap/git.py", line 39, in version [18:07:06] if scap.version.git_version is None: [18:07:07] AttributeError: 'module' object has no attribute 'git_version' [18:09:01] stupid --jobs [18:10:04] lol greg-g is optionally disgruntled [18:10:28] yup [18:12:59] 10Continuous-Integration-Infrastructure: mwgate-php55lint workspaces are getting huge - https://phabricator.wikimedia.org/T179963#3742181 (10Legoktm) No, we need to fix the root cause. This wasn't an issue until recently. [18:13:00] I'm guessing this is a combination of https://phabricator.wikimedia.org/D871 and the packaging code changing the version number in beta. [18:14:24] which is (I guess) a work around for having a global variable? [18:17:33] heh [18:19:02] but we still have the global variable: https://github.com/wikimedia/scap/blob/master/scap/git.py#L27 [18:24:37] 10Scap, 10Scoring-platform-team: Scap deploying arbitrary revision, incorrectly reporting correct revision - https://phabricator.wikimedia.org/T179965#3742204 (10awight) [18:25:26] 10Scap, 10Scoring-platform-team: Scap deploying arbitrary revision, incorrectly reporting correct revision - https://phabricator.wikimedia.org/T179965#3742217 (10awight) [18:26:01] thcipriani: Sounds like it’s not entirely trivial, and I should go ahead and paste the stack trace into a bug? [18:28:51] awight: yeah that'd be great! [18:29:54] * awight opens the floodgates :p [18:30:09] sorry about the trouble, but I want to get a good solution rather than just reverting stuff :( [18:30:12] awight: thcipriani: I can fix it [18:30:28] It’s no trouble! [18:31:01] I was just holding off in case this was a one-liner and not worth the overhead. [18:32:34] 10Scap: Scap glitch on beta tin: no attribute "git_version" - https://phabricator.wikimedia.org/T179966#3742243 (10awight) [18:40:06] 10Continuous-Integration-Config, 10TestMe: fix or mark as inactive extensions currently failing CI - https://phabricator.wikimedia.org/T134090#3742276 (10hashar) [18:40:08] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10Jenkins: Install zip extension for CI - https://phabricator.wikimedia.org/T179772#3742273 (10hashar) 05Open>03Resolved Fixed!!! Thanks @Umherirrender [18:40:43] 10Release-Engineering-Team (Kanban), 10Scap: Scap glitch on beta tin: no attribute "git_version" - https://phabricator.wikimedia.org/T179966#3742279 (10mmodell) a:03mmodell [18:45:56] !log cowboy-committed and pushed rMSCAc1f2ac2 to hopefully unbreak `scap deploy` in beta [18:46:00] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:46:25] 10Release-Engineering-Team (Kanban), 10Scap: Scap glitch on beta tin: no attribute "git_version" - https://phabricator.wikimedia.org/T179966#3742289 (10mmodell) 05Open>03Resolved [18:49:10] awight: how's `git deploy` now"? [18:49:18] err scap deploy [18:49:20] lol [18:49:27] trebuwhat [18:50:02] I tried the windows key and nothing is rebooting to the modem! [18:50:26] hahah [18:51:23] It progressed to the next error :) [18:51:24] 18:50:49 ['/usr/bin/scap', 'deploy-local', '-v', '--repo', 'ores/deploy', '-g', 'worker', 'fetch', '--refresh-config'] on deployment-sca03.deployment-prep.eqiad.wmflabs returned [70]: http://deployment-tin.deployment-prep.eqiad.wmflabs/ores/deploy/.git [18:54:12] awight: improvement! [18:54:31] Or, “better the devil I know” :p [18:59:21] ok I'll have it fixed asap but I have a meeting in 1 minute so I gotta do that first [19:05:41] twentyafterfour: JFYI I’m not blocked today [19:06:02] awight: awesome, thanks! [19:09:38] Project selenium-MinervaNeue » chrome,beta,Linux,BrowserTests build #189: 04FAILURE in 20 min: https://integration.wikimedia.org/ci/job/selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/189/ [19:18:26] 10Release-Engineering-Team (Kanban), 10Scap (Tech Debt Sprint FY201718-Q2): Audit dblists to see if they're all needed - https://phabricator.wikimedia.org/T179972#3742381 (10demon) [19:24:53] legoktm: tyvm [19:59:05] 10MediaWiki-Releasing, 10Release-Engineering-Team: Removal of EOL release branches broke SemanticMediaWiki CI - https://phabricator.wikimedia.org/T179754#3742523 (10greg) >>! In T179754#3739450, @Legoktm wrote: > On the MediaWiki/releng side I think we need to determine how long after a release goes unsupporte... [20:03:36] 10Release-Engineering-Team, 10GitHub-Mirrors: Audit @wikimedia GitHub org access (2017) - https://phabricator.wikimedia.org/T179462#3742531 (10greg) >>! In T179462#3738346, @demon wrote: > - The staff group is useless (it's never complete, and gives a false picture with former staff still in it). Suggest outri... [20:05:33] no_justification yay looks like upstream are going to work on plugin type support for tooltip for the commit msg [20:05:35] https://bugs.chromium.org/p/gerrit/issues/detail?id=7558#c1 [20:05:35] :) [20:05:54] T64053 [20:05:55] T64053: Provide description of linked Phabricator objects in tooltips - https://phabricator.wikimedia.org/T64053 [20:16:34] I'm getting a 403 error on phab: https://gist.github.com/pnorman/ee87dab0b40168b5a413164a0c31fee2 [20:18:26] pnorman hi, is your ip on https://github.com/wikimedia/puppet/blob/production/modules/phabricator/files/apache/phabbanlist.conf [20:18:27] ? [20:19:11] It hasn't been in the past, let me check since I did renew my DHCP lease on the weekend. [20:19:20] pnorman: looking at the logs, I can see your requests being denied... (AH01630: client denied by server configuration) [20:19:40] Yes, it's on the blacklist [20:19:46] Just see the x-client-ip in the gist [20:19:57] https://github.com/wikimedia/puppet/blob/production/modules/phabricator/files/apache/phabbanlist.conf#L63 [20:20:03] 154./8 being blocked [20:20:34] Require not ip 154.0.0.0/8 [20:20:52] 10Continuous-Integration-Config, 10Proton, 10Readers-Web-Backlog (Tracking): Set up Jenkins for chromium-render and chromium-render-deploy repositories - https://phabricator.wikimedia.org/T179552#3742612 (10phuedx) [20:21:15] Ah, the comment above then is wrong. 154.20.0.0/16 is allocated to Telus, which is in North America [20:21:58] hmm, i guess that should be discussed with ops then [20:22:29] cc twentyafterfour ^^ [20:23:00] it looks like that block has been there for 3 month already... [20:23:30] it was blocked due to spam a few months ago [20:23:43] but could it be possible the ip changed hands? [20:23:51] 154.0.0.0/8 is a big block [20:23:54] A /8 seems oddly broad [20:24:03] ^ yeah that [20:24:30] I'm asking my ops friends... [20:24:39] thanks gehel [20:26:02] I believe I've seen a 154 in the past, so that block was probably never right. [20:30:13] it looks like 154/8 is one of the AFRINIC subnets, but with the magic delegation, we're probably not blocking what we intended... [20:31:42] Ya, Telus has had the /16 since at least 2011. The /8 block is a large AfriNIC block, and since Africa isn't high impact the overly broad block appears not to have been noticed, or not been noticed by anyone willing to push the issue rather than giving up. [20:36:34] * pnorman prods his ipv6 [20:37:18] greg-g: can I let you follow up on that? I'm starving and need to get some diner... [20:37:34] gehel: sure, I need to step away to eat as well, but we'll figure it out [20:37:45] greg-g: great! thanks! [20:38:02] twentyafterfour: ^ :) I need to go find lunch them pick up Rowan [20:38:08] pnorman: good luck getting your access back! [20:38:32] An outage sounds like a good time to have lunch, since I can't work, and tunneling through labs or something sounds like a pain. [20:40:00] "Connected" under notifications on phabricator :) [20:40:57] paladox: yep :) [20:41:00] and i got my first live notification :) [20:58:10] 10Release-Engineering-Team (Kanban), 10Release Pipeline (Blubber): Blubber config input validation - https://phabricator.wikimedia.org/T175186#3742729 (10dduvall) 05Open>03Resolved [21:24:11] 10Release-Engineering-Team, 10Operations: Update Debian package for Blubber - https://phabricator.wikimedia.org/T179984#3742821 (10thcipriani) [21:25:49] 10Release-Engineering-Team, 10Operations, 10Release Pipeline: Update Debian package for Blubber - https://phabricator.wikimedia.org/T179984#3742833 (10thcipriani) adding @akosiaris and @Joe since they have the most background on Blubber: could one of you upload a new version of blubber to the apt repository/... [21:26:48] 10Continuous-Integration-Infrastructure: mwgate-php55lint workspaces are getting huge - https://phabricator.wikimedia.org/T179963#3742118 (10hashar) | 5.4G | mwgate-php55lint Sounds like that job clones the whole repo and with submodule processing. Which takes a bunch of space for a mediawiki/core patches @ som... [21:27:25] 10Release-Engineering-Team (Watching / External), 10Wikidata, 10Patch-For-Review, 10User-Addshore: Move config & loading logic out of Wikidata build and into mediawiki-config - https://phabricator.wikimedia.org/T176948#3742839 (10Addshore) So as far as I am aware this is all done now and we can remove what... [21:37:51] 10Scap, 10Scoring-platform-team: Deployment fails with something gitty - https://phabricator.wikimedia.org/T179987#3742873 (10awight) [22:07:29] 10Continuous-Integration-Infrastructure: mwgate-php55lint workspaces are getting huge - https://phabricator.wikimedia.org/T179963#3742937 (10Legoktm) > I guess we can switch to wiping the workspace and using a shallow clone. IIRC that was quite slow for mediawiki/core. We have mediawiki-core-php55lint as a sepa... [22:13:13] 10Continuous-Integration-Infrastructure: mwgate-php55lint workspaces are getting huge - https://phabricator.wikimedia.org/T179963#3742960 (10hashar) +1 +1. So yeah I guess we can switch to wiping the workspace and using a shallow clone. There might be a defaults for that in JJB already :)] [22:15:37] 10RelEng-Archive-FY201718-Q1, 10Operations, 10Phabricator, 10Patch-For-Review: setup/install phab1001.eqiad.wmnet - https://phabricator.wikimedia.org/T163938#3742971 (10mmodell) [22:17:21] 10Release-Engineering-Team (Kanban), 10Operations, 10Phabricator, 10Traffic: Verify that the codfw lvs is configured correctly for Phabricator - https://phabricator.wikimedia.org/T168699#3742977 (10mmodell) [22:17:48] 10Release-Engineering-Team (Kanban), 10Operations, 10Phabricator, 10Traffic: Verify that the codfw lvs is configured correctly for Phabricator - https://phabricator.wikimedia.org/T168699#3373930 (10mmodell) https://gerrit.wikimedia.org/r/#/c/389871/ [22:36:44] 10Release-Engineering-Team, 10GitHub-Mirrors: Audit @wikimedia GitHub org access (2017) - https://phabricator.wikimedia.org/T179462#3743013 (10Jdforrester-WMF) >>! In T179462#3742531, @greg wrote: >>>! In T179462#3738346, @demon wrote: >> - The staff group is useless (it's never complete, and gives a false pic... [22:38:09] 10Release-Engineering-Team (Kanban), 10Phabricator, 10Availability, 10Patch-For-Review, 10WorkType-NewFunctionality: Deploy phabricator to phab2001.codfw.wmnet - https://phabricator.wikimedia.org/T137928#3743016 (10mmodell) So BBlack and I went over this stuff today on IRC. I'll paste an excerpt from the... [23:02:02] (03CR) 10Hashar: [C: 032] Archive the VectorBeta extension [integration/config] - 10https://gerrit.wikimedia.org/r/389444 (https://phabricator.wikimedia.org/T178932) (owner: 10MarcoAurelio) [23:03:11] (03Merged) 10jenkins-bot: Archive the VectorBeta extension [integration/config] - 10https://gerrit.wikimedia.org/r/389444 (https://phabricator.wikimedia.org/T178932) (owner: 10MarcoAurelio) [23:13:45] https://gerrit.wikimedia.org/r/#/c/389888/ [23:13:56] greg-g: pnorman: ^ [23:16:39] twentyafterfour: that's delegated to a UK ISP [23:17:24] * paladox wonders what isp [23:17:34] sky, bt, vodaphone, virgin. [23:17:49] I mean, it'd fix *my* problems, but I suspect it's still wrong [23:17:50] telstra [23:18:19] https://ipinfo.io/AS1290/154.32.0.0/16 [23:18:20] pnorman: I can't actually figure out why that /8 was added, I was just trying to get you unblocked [23:18:30] * paladox has never herd of telstra as a uk isp. [23:19:10] is there any way to see a nice high level map of the delegations? [23:19:44] paladox: oh, my mistake, AU. [23:19:56] ah, there's one in au :) [23:20:50] twentyafterfour something like https://myip.ms/view/ip_owners/47855/Telstra_Limited.html map ? [23:20:51] :) [23:21:38] paladox: I mean to see all of 154 [23:21:45] oh i see [23:21:51] 154.0.0.0/8 is all of 154.everything [23:21:55] Do you know the original IP? [23:22:33] pnorman: no, I haven't even found the original commit / justification for the block. Maybe should just remove it entirely [23:22:36] * pnorman can't investigate very effectivly, being, well blocked from phab [23:22:45] I did find the original commit, and there were links to phab issues [23:23:44] https://github.com/wikimedia/puppet/commit/403fa47784473bc8c1d23be5778fef8354c8373e#diff-0f0cd2a406609e3f97265a81b1e93408 [23:24:06] twentyafterfour i think aklapper re enabled that block because of alot of spam [23:24:08] around that time [23:24:23] though i doint think the block for that ip is needed now [23:25:18] so I think it should be just reduced to a /16 [23:25:23] The original block was lifted then re-added, and that happened 3 months ago. So original block -12 mo, tested if necessary -3mo, probably still needed. But maybe 154/8 isn't, and that could be removed, and if the issue does reappear a more appropriate block could be done [23:28:30] twentyafterfour: original commit: https://gerrit.wikimedia.org/r/#/c/363001/ [23:28:48] oh, pnorman beat me [23:28:54] * greg-g finishes reading backscroll [23:30:58] 10Release-Engineering-Team (Watching / External), 10Operations, 10Release Pipeline: Update Debian package for Blubber - https://phabricator.wikimedia.org/T179984#3743157 (10greg) [23:32:02] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team (Kanban), 10Multimedia, 10Patch-For-Review, and 2 others: mediawiki-config changes not deployed automatically to deployment-videoscaler01 - https://phabricator.wikimedia.org/T179688#3743160 (10greg) 05Open>03Resolved This was merged, thanks @fgi... [23:32:10] I'm just gonna remove that one line and see what happens [23:35:33] no_justification heh upstream are trying to go with alphine for there ci. [23:48:12] pnorman: I tried to advocate for merging in -operations but so far no movement