[00:06:14] PROBLEM - Free space - all mounts on integration-slave-docker-1001 is CRITICAL: CRITICAL: integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_e6e01440f04a145c09affed5a3ca6f3b53daaab53d038a0d9a25a2321ad2e83e.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_368b9484735f938d1514e14cbeb21acea6bf49e46f5ad5b0ce103931de [00:06:14] e (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_542702f60914ef9f11b4fff9801fb50b98babed3d88dab01384772c8a8f1f043.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_382de2b48f7a102ecf17cbd12d459f88057d36dba96cedb922040e35e9b82d2e.byte_percentfree (No valid datapoints found) integrat [00:06:14] docker-1001.diskspace._var_lib_docker_devicemapper_mnt_244a2d354369e401b6c66d4bd544048c3840d932cf778fdf984537361725f816.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_d8fc462aeb76074a5f89517c31b54a82a6b9bee7edadd84f909f7bf192acd160.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_ [00:06:14] a7f3437c5df7fe9022aa85e5b8d3f8456090e82a1199b2b29dec9635842.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_1c0fb41f7a2181b123e5946fba45e85cdfef2556fc8597cb15f65675a1b19486.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_eed8308ecda3c515fc3eb746da906634666ed3ec1b4 [00:06:14] byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_31efaafd7825c53306a17741dcce1df6a28a339aadc13d4aa228986a0325c8a7.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_734cb090a18ec3f54bb70110faa369e4073fe6ad423521c364e403698f3ec8b1.byte_percentfree (No valid datapoints [00:06:14] egration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_858cef621f5c06ea809b2df2c7bc0dfc5382fcb3bfb5a1462c87c75499445363.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_bf3c5a6ecb338d0319bdcec67a168fc65691797a7bc5db4125a1a021b13529fa.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._ [00:06:14] apper_mnt_fa1007a5eb3003d50a1198bcce1298c1d88259f7fc12a91be365ec4904314699.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_5b935ee981f9b151da2acbcb7dd261ef3d0dbedd2ebbc3b805c1aa17a3ab22f8.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_a064dcbba5444f8accb682707765 [00:07:44] (03PS1) 10Thcipriani: docker: operations-puppet -> docker-pkg image [integration/config] - 10https://gerrit.wikimedia.org/r/391734 (https://phabricator.wikimedia.org/T177276) [00:10:37] !log removed old differential-docker-test images on integration-slave-docker-1001 [00:10:43] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:16:13] RECOVERY - Free space - all mounts on integration-slave-docker-1001 is OK: OK: integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_e6e01440f04a145c09affed5a3ca6f3b53daaab53d038a0d9a25a2321ad2e83e.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_368b9484735f938d1514e14cbeb21acea6bf49e46f5ad5b0ce103931ded95b02.byte [00:16:14] datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_542702f60914ef9f11b4fff9801fb50b98babed3d88dab01384772c8a8f1f043.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_382de2b48f7a102ecf17cbd12d459f88057d36dba96cedb922040e35e9b82d2e.byte_percentfree (No valid datapoints found) integration.integra [00:16:14] .diskspace._var_lib_docker_devicemapper_mnt_244a2d354369e401b6c66d4bd544048c3840d932cf778fdf984537361725f816.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_d8fc462aeb76074a5f89517c31b54a82a6b9bee7edadd84f909f7bf192acd160.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemappe [00:16:14] 7fe9022aa85e5b8d3f8456090e82a1199b2b29dec9635842.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_1c0fb41f7a2181b123e5946fba45e85cdfef2556fc8597cb15f65675a1b19486.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_eed8308ecda3c515fc3eb746da906634666ed3ec1b47ba65026819 [00:16:14] tfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_31efaafd7825c53306a17741dcce1df6a28a339aadc13d4aa228986a0325c8a7.byte_percentfree (No valid datapoints found) integration.integration-slave-docker-1001.diskspace._var_lib_docker_devicemapper_mnt_734cb090a18ec3f54bb70110faa369e4073fe6ad423521c364e403698f3ec8b1.byte_percentfree (No valid datapoints found) inte [00:17:52] (03CR) 10Thcipriani: [C: 032] "already live, seems working" [integration/config] - 10https://gerrit.wikimedia.org/r/391734 (https://phabricator.wikimedia.org/T177276) (owner: 10Thcipriani) [00:19:43] (03Merged) 10jenkins-bot: docker: operations-puppet -> docker-pkg image [integration/config] - 10https://gerrit.wikimedia.org/r/391734 (https://phabricator.wikimedia.org/T177276) (owner: 10Thcipriani) [00:31:42] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.8 deployment blockers - https://phabricator.wikimedia.org/T178635#3765269 (10Etonkovidova) [01:32:27] PROBLEM - Puppet errors on deployment-tin is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [02:10:22] 10Release-Engineering-Team (Kanban), 10Diffusion, 10Phabricator (Upstream), 10Upstream: Phabricator throws Unhandled Exception ("Exception") when querying Diffusion - https://phabricator.wikimedia.org/T180457#3765423 (10mmodell) 05Open>03Resolved [02:10:31] 10Release-Engineering-Team (Kanban), 10Diffusion, 10Phabricator (Upstream), 10Upstream: Phabricator throws Unhandled Exception ("Exception") when querying Diffusion - https://phabricator.wikimedia.org/T180457#3758579 (10mmodell) Patch deployed. [02:23:19] 10Gerrit, 10Operations, 10Phabricator, 10Traffic, 10periodic-update: Phabricator and Gerrit: Improve the way that maintenance downtime is communicated to users. - https://phabricator.wikimedia.org/T180655#3765427 (10mmodell) [02:23:26] 10Gerrit, 10Operations, 10Phabricator, 10Traffic, 10periodic-update: Phabricator and Gerrit: Improve the way that maintenance downtime is communicated to users. - https://phabricator.wikimedia.org/T180655#3765440 (10mmodell) p:05Triage>03Normal [02:27:16] 10Gerrit, 10Operations, 10Phabricator, 10Traffic, 10periodic-update: Phabricator and Gerrit: Improve the way that maintenance downtime is communicated to users. - https://phabricator.wikimedia.org/T180655#3765442 (10mmodell) [02:37:33] 10Continuous-Integration-Config: Jenkins isn't showing mediawiki-core-php70-phan-docker failure output - https://phabricator.wikimedia.org/T180630#3764103 (10Legoktm) Based on `19:30:00 [CHECKSTYLE] No files found. Configuration error?` your hunch seems correct that it's not writing to the XML file. [02:38:02] 10Continuous-Integration-Config: Jenkins isn't showing mediawiki-core-php70-phan-docker failure output - https://phabricator.wikimedia.org/T180630#3765460 (10Legoktm) p:05Triage>03Unbreak! This basically means all the phan jobs aren't actually doing anything. [02:39:25] 10Gerrit, 10Operations, 10Phabricator, 10Traffic, 10periodic-update: Phabricator and Gerrit: Improve the way that maintenance downtime is communicated to users. - https://phabricator.wikimedia.org/T180655#3765463 (10demon) Ideas for implementation from IRC: * Environment variables--might require Apache r... [02:58:02] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<40.00%) [06:53:00] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [06:55:58] Project selenium-Wikibase » chrome,beta,Linux,BrowserTests build #546: 04FAILURE in 2 hr 15 min: https://integration.wikimedia.org/ci/job/selenium-Wikibase/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/546/ [07:09:46] 10Continuous-Integration-Config, 10Librarization, 10Security, 10Security-General: Expand our usage of FriendsOfPHP/security-advisories - https://phabricator.wikimedia.org/T180278#3765618 (10Legoktm) [07:10:11] 10Continuous-Integration-Config, 10Librarization, 10Composer, 10Security, 10Security-General: Expand our usage of FriendsOfPHP/security-advisories - https://phabricator.wikimedia.org/T180278#3752217 (10Legoktm) [08:26:35] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.8 deployment blockers - https://phabricator.wikimedia.org/T178635#3765706 (10Legoktm) @demon did those two patches get cherry-picked into wmf.8? [08:55:09] 10Continuous-Integration-Config: Jenkins isn't showing mediawiki-core-php70-phan-docker failure output - https://phabricator.wikimedia.org/T180630#3765719 (10Addshore) >>! In T180630#3765460, @Legoktm wrote: > This basically means all the phan jobs aren't actually doing anything. Not quite right, the phan jobs... [09:25:54] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.8 deployment blockers - https://phabricator.wikimedia.org/T178635#3765780 (10Addshore) [09:44:24] Thanks for the rollout and back no_justification, I filed a ticket https://phabricator.wikimedia.org/T180665 and will get it fixed today so we can catchup again tonight! [10:26:20] 10Release-Engineering-Team, 10Wikidata, 10Epic, 10User-Addshore: [Epic] Kill the Wikidata build step - https://phabricator.wikimedia.org/T173818#3766054 (10Addshore) [10:45:39] 10Continuous-Integration-Infrastructure (shipyard), 10Operations, 10Patch-For-Review, 10User-Joe: Unify production and CI docker image build process - https://phabricator.wikimedia.org/T177276#3766088 (10Joe) Thanks @thcipriani for converting the use of ops/puppet already! [12:06:19] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.8 deployment blockers - https://phabricator.wikimedia.org/T178635#3766285 (10Addshore) [12:49:04] 10Continuous-Integration-Config: Whitelist @Daimona on integration/config - https://phabricator.wikimedia.org/T180683#3766400 (10Melos) [13:04:48] (03PS1) 10Melos: Add Daimona to trusted user list [integration/config] - 10https://gerrit.wikimedia.org/r/391812 (https://phabricator.wikimedia.org/T180683) [14:06:49] 10Release-Engineering-Team (Watching / External), 10MediaWiki-Platform-Team, 10Wikidata: What happens if the Wikibase project specifies a version of a library outside of the range included in mediawiki-vendor? - https://phabricator.wikimedia.org/T179663#3766581 (10Addshore) Just tested with the gate-submit j... [14:21:17] paladox: gerrit web ui should not allow invalid characters in the "topic" field [14:21:42] what i mean is.. it lets you change the topic to something, for example 'foo::bar' and it's fine in web ui [14:21:55] then when you try to download that to amend to it though... [14:22:17] "Cannot checkout to new branch" "fatal" "is not a valid branch name" "omg, exit code 128" , heh [14:23:48] (https://stackoverflow.com/questions/3651860/which-characters-are-illegal-within-a-branch-name , https://www.kernel.org/pub/software/scm/git/docs/git-check-ref-format.html) [14:24:26] (links from apergos, thanks :) [14:48:27] 10Gerrit, 10Operations, 10Readers-Web-Backlog, 10Patch-For-Review, and 3 others: [spike] Temporarily allow pushing large objects - https://phabricator.wikimedia.org/T178189#3766646 (10akosiaris) > If we're all agreed with this as a way forward, then this task and {T178570} can be resolved. Agreed on my pa... [15:34:37] no_justification: wikidatawiki should be all good to roll to .8 again fyi :) [15:35:34] I'll be around actively until about 17:50 UTC and then around not so actively till 2200utc ish [16:09:11] addshore: mmk [16:09:28] * addshore waves toward no_justification [16:15:54] 10Gerrit, 10Patch-For-Review: 404 downloading any changes with https remote url: "The requested URL /changes/ was not found" - https://phabricator.wikimedia.org/T154760#3766944 (10Paladox) Upstream have merged a possible fix for this. Though it dosent deal with /p/ prefix but a follow up will fix that. [16:20:04] (03CR) 10Chad: "That's also true, but this certainly won't hurt jobs that do need submodules." [integration/config] - 10https://gerrit.wikimedia.org/r/391712 (owner: 10Chad) [16:31:02] 10Continuous-Integration-Config, 10Analytics-Kanban: Add CI to all analytics/* repositories and archive obsolete ones - https://phabricator.wikimedia.org/T180301#3767004 (10fdans) [16:41:45] no_justification aha [16:41:52] i figured out why it's not working [16:42:00] lfs does not support digest auth [16:42:13] digest auth was removed in 2.14 which is also another difference [16:42:26] we switch a config in gerrit 2.13 to support basic auth [16:43:28] Will create a patch for this [16:43:35] though it may break tools that use digest [16:43:44] it will break tools anyways if we do the 2.14 update [16:43:48] + it's more secure [16:43:51] using basic auth [16:49:03] no_justification https://gerrit.wikimedia.org/r/#/c/391865/ :) [16:59:10] 10Release-Engineering-Team, 10Phabricator: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3767061 (10jcrespo) [17:01:17] 10Release-Engineering-Team, 10Phabricator: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3767061 (10Paladox) There was a phabricator update last night that required a reindex for search i think. [17:02:18] 10Gerrit, 10Release-Engineering-Team (Kanban), 10Scap (Tech Debt Sprint FY201718-Q2), 10ORES, and 3 others: Support git-lfs files in gerrit - https://phabricator.wikimedia.org/T171758#3767081 (10Paladox) LFS is now supported and enabled on a testing repo. As i found out today, lfs dosent seem to support d... [17:02:46] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.8 deployment blockers - https://phabricator.wikimedia.org/T178635#3767084 (10demon) @Legoktm They have now 😂 [17:03:19] 10Release-Engineering-Team, 10Phabricator: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3767088 (10jcrespo) If it is temporary, that is great news. This would be resolved if an email could be sent with details to wikitech/ops/engineering ? [17:17:45] no_justification i just read https://gerrit.wikimedia.org/r/Documentation/config-gerrit.html#auth.gitBasicAuthPolicy which we have set, so that wont take effect until we do https://gerrit.wikimedia.org/r/#/c/391865/ too apparently (sorry for another ping) [17:48:55] thanks no_justification [17:49:20] Yw [18:01:23] no_justification: legoktm I guess one of you has access to create gerrit user groups? [18:01:37] Obvs [18:01:38] :) [18:01:43] Whatcha need? [18:01:51] one called "mcr-reviewers" [18:02:16] and I need the ability to add people :) [18:09:32] bet you cant tell what it is for [18:14:38] 10Release-Engineering-Team, 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3767469 (10Aklapper) [18:18:21] 10Release-Engineering-Team, 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3767493 (10Aklapper) > For example, I would expect "puppet mysql" to show T162070 Works for me, it's just not among the first 100 results (pagination). Listed wh... [18:39:22] PROBLEM - Puppet errors on deployment-mx is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [19:16:14] Project selenium-MinervaNeue » chrome,beta,Linux,BrowserTests build #200: 04FAILURE in 27 min: https://integration.wikimedia.org/ci/job/selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/200/ [19:41:41] twentyafterfour: Hi, im wondering did we run the reindex for searching on phab last night? [19:43:29] twentyafterfour: was gonna ask too because search seems suboptimal [19:43:34] Project selenium-MinervaNeue » firefox,beta,Linux,BrowserTests build #200: 04FAILURE in 54 min: https://integration.wikimedia.org/ci/job/selenium-MinervaNeue/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/200/ [19:44:21] paladox: mutante: yeah it's reindexing [19:44:44] ok thanks :) [19:44:54] twentyafterfour see https://phabricator.wikimedia.org/T180706#3767088 :) [19:45:49] thanks ! [19:48:16] 10Release-Engineering-Team, 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3767061 (10Peachey88) those were mentioned in the ticket which causes reindexing iirc from last time we had search issues. [19:49:29] 10Release-Engineering-Team, 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3767881 (10mmodell) The reindex is almost finished, so quality should be restored to normal by now. If this is not the case then something else is wrong. [19:54:50] 10Release-Engineering-Team, 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3767899 (10jcrespo) Search is definitely still not working for me, I search db1022 and I cannot find the at least 3 tickets with it in the title (but a lot of unr... [19:58:19] 10Release-Engineering-Team, 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3767928 (10Aklapper) Please provide links to your search queries, so we can make sure that we talk about the same things. See T180706#3767493 for my links. Thanks! [19:59:33] 10Release-Engineering-Team, 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3767933 (10Paladox) @jcrespo we use elasticsearch for searches now so it's not going to be in the db. [20:02:22] 10Release-Engineering-Team, 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3767061 (10EBernhardson) Because of how phabricator is architected, it kinda does both. Elasticsearch seems to be used as a first pass filtering/sorting, but then... [20:06:51] 10Release-Engineering-Team, 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3767965 (10mmodell) Ok two things I think are affecting things: 1. I think the reindex just indexed a lot of commits that were previously not being seen by searc... [20:10:15] Yippee, build fixed! [20:10:16] Project selenium-MinervaNeue » chrome,beta,Linux,BrowserTests build #201: 09FIXED in 23 min: https://integration.wikimedia.org/ci/job/selenium-MinervaNeue/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/201/ [20:12:20] (03CR) 10Hashar: [C: 032] operations/dumps/dcat: npm for jsonlint/banana [integration/config] - 10https://gerrit.wikimedia.org/r/390993 (https://phabricator.wikimedia.org/T180328) (owner: 10Hashar) [20:13:37] (03Merged) 10jenkins-bot: operations/dumps/dcat: npm for jsonlint/banana [integration/config] - 10https://gerrit.wikimedia.org/r/390993 (https://phabricator.wikimedia.org/T180328) (owner: 10Hashar) [20:18:55] 10Continuous-Integration-Config, 10Dumps-Generation, 10Patch-For-Review: Add CI to all operations/dumps/* repositories and archive obsolete ones - https://phabricator.wikimedia.org/T180328#3768014 (10hashar) [20:19:07] 10Continuous-Integration-Config, 10Dumps-Generation, 10Patch-For-Review: Add CI to all operations/dumps/* repositories and archive obsolete ones - https://phabricator.wikimedia.org/T180328#3754037 (10hashar) In Gerrit, I have granted CI permission to submit patches on `operations/dumps` and its child. [20:19:49] 10Release-Engineering-Team, 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3768017 (10jcrespo) No, when I mean I search db1022, it means I literally search the word db1022, I do not think it is difficult to go to the search box and type... [20:22:08] 10Release-Engineering-Team, 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3768023 (10jcrespo) Now they seem much better, is the change immediate? [20:24:47] 10Release-Engineering-Team, 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3768034 (10jcrespo) I've tried now a couple of searches and this is for me fixed already- things like quality in a fine grained way is something that would take t... [20:25:53] Yippee, build fixed! [20:25:54] Project selenium-Wikibase-chrome » chrome,beta,Linux,DebianJessie && contintLabsSlave build #14: 09FIXED in 38 min: https://integration.wikimedia.org/ci/job/selenium-Wikibase-chrome/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=DebianJessie%20&&%20contintLabsSlave/14/ [20:25:59] (03CR) 10Hashar: [C: 032] operations/dumps/dcat: composer for php linting [integration/config] - 10https://gerrit.wikimedia.org/r/390998 (https://phabricator.wikimedia.org/T180328) (owner: 10Hashar) [20:26:07] (03CR) 10jerkins-bot: [V: 04-1] operations/dumps/dcat: composer for php linting [integration/config] - 10https://gerrit.wikimedia.org/r/390998 (https://phabricator.wikimedia.org/T180328) (owner: 10Hashar) [20:26:13] Yippee, build fixed! [20:26:13] Project selenium-MinervaNeue » firefox,beta,Linux,BrowserTests build #201: 09FIXED in 39 min: https://integration.wikimedia.org/ci/job/selenium-MinervaNeue/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/201/ [20:26:20] (03PS2) 10Hashar: operations/dumps/dcat: composer for php linting [integration/config] - 10https://gerrit.wikimedia.org/r/390998 (https://phabricator.wikimedia.org/T180328) [20:26:24] (03CR) 10Hashar: operations/dumps/dcat: composer for php linting [integration/config] - 10https://gerrit.wikimedia.org/r/390998 (https://phabricator.wikimedia.org/T180328) (owner: 10Hashar) [20:26:26] (03CR) 10Hashar: [C: 032] operations/dumps/dcat: composer for php linting [integration/config] - 10https://gerrit.wikimedia.org/r/390998 (https://phabricator.wikimedia.org/T180328) (owner: 10Hashar) [20:26:49] 10Continuous-Integration-Config, 10Dumps-Generation, 10Patch-For-Review: Add CI to all operations/dumps/* repositories and archive obsolete ones - https://phabricator.wikimedia.org/T180328#3768037 (10hashar) [20:27:43] (03Merged) 10jenkins-bot: operations/dumps/dcat: composer for php linting [integration/config] - 10https://gerrit.wikimedia.org/r/390998 (https://phabricator.wikimedia.org/T180328) (owner: 10Hashar) [20:29:17] 10Release-Engineering-Team, 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3768040 (10jcrespo) BTW, commit search is not a problem to me (my box already defaults to open tasks, and I change them to all tasks if necessary). [20:39:35] (03PS1) 10MarcoAurelio: Archive the ActivityMonitor extension [integration/config] - 10https://gerrit.wikimedia.org/r/391902 (https://phabricator.wikimedia.org/T180631) [21:02:45] 10Release-Engineering-Team (Watching / External), 10Operations, 10Release Pipeline: Update Debian package for Blubber - https://phabricator.wikimedia.org/T179984#3768097 (10dduvall) [21:02:47] 10Release-Engineering-Team (Kanban), 10Release Pipeline (Blubber): Use dep to manage vendor dependencies for blubber - https://phabricator.wikimedia.org/T180530#3768095 (10dduvall) 05Open>03Resolved [21:06:20] 10Release-Engineering-Team, 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3768104 (10greg) 05Open>03Resolved p:05Triage>03Unbreak! a:03mmodell Sorry and thanks all. [21:06:59] 10Release-Engineering-Team (Kanban), 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3768113 (10greg) [21:09:41] PROBLEM - Puppet staleness on deployment-changeprop is CRITICAL: CRITICAL: 10.00% of data above the critical threshold [43200.0] [21:11:43] 10Release-Engineering-Team, 10Wikidata, 10Epic, 10User-Addshore: [Epic] Kill the Wikidata build step - https://phabricator.wikimedia.org/T173818#3768140 (10Addshore) [21:12:19] *poke no_justification* any chance of that gerrit group? :D [21:13:39] Nope [22:10:39] * paladox renames projects to repo's in https://gerrit-review.googlesource.com/#/c/gerrit/+/142193/ [22:11:12] PROBLEM - Puppet staleness on deployment-redis06 is CRITICAL: CRITICAL: 37.50% of data above the critical threshold [43200.0] [22:29:12] (03CR) 10Greg Grossmeier: [C: 031] "works as expected" [tools/release] - 10https://gerrit.wikimedia.org/r/390429 (owner: 1020after4) [22:37:02] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.5 deployment blockers - https://phabricator.wikimedia.org/T174361#3768361 (10greg) [22:37:26] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.5 deployment blockers - https://phabricator.wikimedia.org/T174361#3559324 (10greg) 05Open>03Resolved [22:49:34] no_justification hi, question, how would i load external libs in gerrit please? [22:49:38] ie gelf. [22:51:22] They'd go in libs/ [22:51:31] Where we symlink the bcrypt libs rn [22:52:18] ah ok thanks. [23:10:31] (03CR) 10Krinkle: [C: 032] Archive the ActivityMonitor extension [integration/config] - 10https://gerrit.wikimedia.org/r/391902 (https://phabricator.wikimedia.org/T180631) (owner: 10MarcoAurelio) [23:10:34] (03CR) 10Krinkle: [C: 032] "Thanks!" [integration/config] - 10https://gerrit.wikimedia.org/r/391902 (https://phabricator.wikimedia.org/T180631) (owner: 10MarcoAurelio) [23:11:48] (03Merged) 10jenkins-bot: Archive the ActivityMonitor extension [integration/config] - 10https://gerrit.wikimedia.org/r/391902 (https://phabricator.wikimedia.org/T180631) (owner: 10MarcoAurelio) [23:16:21] !log Reloading Zuul to deploy https://gerrit.wikimedia.org/r/391902 [23:16:25] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [23:16:48] hmm fails [23:16:49] log4j:ERROR Could not instantiate appender named "gelf". [23:17:58] Nov 16 23:16:36 gerrit-test3 java[9703]: java.lang.ClassNotFoundException: biz.paluch.logging.gelf.log4j.GelfLogAppender [23:18:03] yet i have it in lib/ [23:18:09] and did -cp on java's command line [23:21:43] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.8 deployment blockers - https://phabricator.wikimedia.org/T178635#3768543 (10greg) [23:21:57] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.8 deployment blockers - https://phabricator.wikimedia.org/T178635#3698328 (10greg) Train is currently blocked by T180714. [23:31:20] 10Beta-Cluster-Infrastructure: Request to help out - https://phabricator.wikimedia.org/T180757#3768556 (10Sau226) [23:34:56] twentyafterfour hi, im wondering could you review https://phabricator.wikimedia.org/D869 please? :) [23:35:08] it reverts a change that broke login on test sites [23:35:11] that doint use ldap [23:35:51] 10Release-Engineering-Team (Kanban), 10Discovery, 10Discovery-Search (Current work), 10MW-1.31-release-notes (WMF-deploy-2017-10-24 (1.31.0-wmf.5)), 10Patch-For-Review: [Epic] Port Selenium tests from Ruby to Node.js for the Search Platform - https://phabricator.wikimedia.org/T174103#3768572 (10EBernhard... [23:36:46] 10Beta-Cluster-Infrastructure: Request to help out - https://phabricator.wikimedia.org/T180757#3768575 (10greg) 05Open>03stalled This is stalled due to previous interactions: T176213 @Sau226 let me know if you would like to talk about this via email or IRC private message. [23:58:22] 10Beta-Cluster-Infrastructure: Request to help out - https://phabricator.wikimedia.org/T180757#3768616 (10Sau226) @greg I am willing to talk using IRC as it is most convenient for me. Its breakfast for me very soon so let me get it first then we can discuss err what we are meant to discuss. Thank you for being m... [23:59:22] anyone know their irc nick so I can PM them? ^ [23:59:30] just pm it to me