[00:53:41] 10Release-Engineering-Team (Kanban), 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3811293 (10greg) status? If this isn't the same root cause, let's not reuse tasks. Especially since this one is UBN!. [03:59:13] Project selenium-MultimediaViewer » firefox,mediawiki,Linux,BrowserTests build #598: 04FAILURE in 3 min 13 sec: https://integration.wikimedia.org/ci/job/selenium-MultimediaViewer/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=mediawiki,PLATFORM=Linux,label=BrowserTests/598/ [04:16:14] 10MediaWiki-Codesniffer: Look for @parma typo (should be @param) - https://phabricator.wikimedia.org/T182057#3811450 (10Legoktm) [04:19:05] 10Release-Engineering-Team (Kanban), 10MediaWiki-Cache, 10MediaWiki-Vagrant, 10Performance-Team (Radar), 10User-zeljkofilipin: MediaWiki core Selenium tests fail when targeting Vagrant - https://phabricator.wikimedia.org/T180035#3811467 (10aaron) Does this still occur? [04:50:49] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [05:20:51] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [10.0] [06:37:42] !log Updated cxserver to 1693bcf [06:37:48] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [06:37:53] oh. it works :) [07:02:00] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [07:20:46] paladox: I see no gerrit logs in logstash [08:21:17] (03CR) 10Hashar: [C: 032] "Thanks Stas :)" [integration/config] - 10https://gerrit.wikimedia.org/r/395113 (https://phabricator.wikimedia.org/T155829) (owner: 10Hashar) [08:22:28] (03Merged) 10jenkins-bot: Remove wikidata/gremlin job [integration/config] - 10https://gerrit.wikimedia.org/r/395113 (https://phabricator.wikimedia.org/T155829) (owner: 10Hashar) [08:36:52] (03CR) 10Hashar: "The Docker containers are run with:" [integration/config] - 10https://gerrit.wikimedia.org/r/395093 (owner: 10Hashar) [08:49:57] (03Abandoned) 10Hashar: docker: set FORCE_COLOR for supports-color [integration/config] - 10https://gerrit.wikimedia.org/r/395093 (owner: 10Hashar) [08:51:56] !log jenkins: adding global property FORCE_COLOR=1 to https://integration.wikimedia.org/ci/configure . That forces webdriver.io to spurts color in the Jenkins console when not using a TTY [08:52:01] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:54:11] PROBLEM - Long lived cherry-picks on puppetmaster on deployment-puppetmaster02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [08:56:12] (03CR) 10Hashar: "Worked on https://integration.wikimedia.org/ci/job/mediawiki-core-qunit-selenium-jessie/11145/console and that also force the karma qunit" [integration/config] - 10https://gerrit.wikimedia.org/r/395093 (owner: 10Hashar) [09:03:28] 10Continuous-Integration-Infrastructure (shipyard), 10PAWS, 10monitoring, 10Graphite: Purge labs graphite metrics of Docker ephemeral partitions - https://phabricator.wikimedia.org/T181476#3811655 (10hashar) @fgiunchedi can you try deleting /srv/carbon/whisper/tools/tools-paws-*/diskspace again and see whe... [09:05:13] 10Gerrit: jenkins-bot ran gate-and-submit and voted V+2, but did not submit (merge) the change - https://phabricator.wikimedia.org/T155558#3811669 (10Nikerabbit) Sorry for the noise. I meant to put the bug into commit body. [09:08:00] PROBLEM - Free space - all mounts on deployment-videoscaler01 is CRITICAL: CRITICAL: deployment-prep.deployment-videoscaler01.diskspace.root.byte_percentfree (<10.00%) [09:18:37] 10MediaWiki-Codesniffer: Look for @parma typo (should be @param) - https://phabricator.wikimedia.org/T182057#3811450 (10Addshore) https://commons.wikimedia.org/wiki/Category:Prosciutto_di_Parma [09:28:39] addshore: more of a chicken parma person myself >.> / legoktm [09:33:00] PROBLEM - Free space - all mounts on deployment-videoscaler01 is CRITICAL: CRITICAL: deployment-prep.deployment-videoscaler01.diskspace.root.byte_percentfree (<10.00%) [09:34:45] Nikerabbit try type:"log4j" in the search bar [09:46:15] paladox: those are even less informative than what I get as a reply [09:47:19] I was expecting stacktraces or similar [09:51:36] Nikerrabit i havent really tested with any errors, but depends on the error (if it includes a stack trace) [09:51:42] Nikerabbit [09:56:15] (03PS1) 10Volans: Add tox job for operations/software/debmonitor [integration/config] - 10https://gerrit.wikimedia.org/r/395479 (https://phabricator.wikimedia.org/T167504) [10:08:55] (03CR) 10Paladox: [C: 031] Add tox job for operations/software/debmonitor [integration/config] - 10https://gerrit.wikimedia.org/r/395479 (https://phabricator.wikimedia.org/T167504) (owner: 10Volans) [10:12:21] PROBLEM - Puppet errors on deployment-videoscaler01 is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [10:13:12] 10Continuous-Integration-Infrastructure (shipyard), 10PAWS, 10monitoring, 10Graphite: Purge labs graphite metrics of Docker ephemeral partitions - https://phabricator.wikimedia.org/T181476#3811824 (10fgiunchedi) @hashar tried just now and yes they are being recreated [10:15:10] Nikerabbit anyways i am working on a update to gerrit that will fix the stack trace problem, but we wont be able to use it for a while (requires a gerrit update). (Also depends on if upstream merge it). [10:20:13] PROBLEM - Free space - all mounts on integration-slave-docker-1001 is CRITICAL: CRITICAL: integration.integration-slave-docker-1001.diskspace.root.byte_percentfree (<100.00%) [10:33:13] hashar: around? I want to know if something is possible in jjb :D [10:33:17] you might have the answer! [10:39:00] 10Release-Engineering-Team (Kanban), 10releng-201718-q1, 10MediaWiki-General-or-Unknown, 10Epic, and 5 others: Port Selenium tests from Ruby to Node.js - https://phabricator.wikimedia.org/T139740#3811883 (10zeljkofilipin) [10:39:03] 10Release-Engineering-Team (Kanban), 10MediaWiki-Cache, 10MediaWiki-Vagrant, 10Performance-Team (Radar), 10User-zeljkofilipin: MediaWiki core Selenium tests fail when targeting Vagrant - https://phabricator.wikimedia.org/T180035#3811881 (10zeljkofilipin) 05Open>03Resolved >>! In T180035#3811467, @aar... [10:52:18] (03CR) 10Ema: [C: 031] operations-puppet: install mtail [integration/config] - 10https://gerrit.wikimedia.org/r/394551 (https://phabricator.wikimedia.org/T181794) (owner: 10Filippo Giunchedi) [10:52:21] RECOVERY - Puppet errors on deployment-videoscaler01 is OK: OK: Less than 1.00% above the threshold [0.0] [10:57:45] zeljkof: good morning. I have made Jenkins to always inject the env variable FORCE_COLOR=1 [10:58:00] zeljkof: and that seems to have enabled color output for qunit and wdio tests in Jenkins !!! [10:58:07] hashar: I saw the commit [10:58:14] good work! [11:32:07] 10Continuous-Integration-Infrastructure (shipyard), 10PAWS, 10monitoring, 10Graphite: Purge labs graphite metrics of Docker ephemeral partitions - https://phabricator.wikimedia.org/T181476#3812081 (10hashar) I have checked tools-paws-worker-1002.tools.eqiad.wmflabs and puppet had just run. The Diamond con... [11:46:43] grbmbmbm [11:46:58] the slave-docker instances need disk space garbage collection eek [11:59:02] addshore: yeah I am around [11:59:15] though I went busy doing other stuff with the IRC client idling in the background :D [12:00:09] RECOVERY - Free space - all mounts on integration-slave-docker-1001 is OK: OK: All targets OK [12:00:13] 10Continuous-Integration-Config, 10Pywikibot-core: Jenkins output for pywikibot job is hard to read - https://phabricator.wikimedia.org/T117570#3812140 (10Dalba) [12:01:15] addshore: or just submit the half done patch :] [12:04:44] (03CR) 10Hashar: [C: 032] Add tox job for operations/software/debmonitor [integration/config] - 10https://gerrit.wikimedia.org/r/395479 (https://phabricator.wikimedia.org/T167504) (owner: 10Volans) [12:05:51] (03Merged) 10jenkins-bot: Add tox job for operations/software/debmonitor [integration/config] - 10https://gerrit.wikimedia.org/r/395479 (https://phabricator.wikimedia.org/T167504) (owner: 10Volans) [12:07:42] hashar: I dont even know how to make it to a half done patch :D [12:08:17] basically for beta updates of mediawiki-config I wanted beta-mediawiki-config-update-eqiad to trigger a scap of just wmf-config (nice and quick) [12:08:33] in order to get config changes on beta asap! rather than waiting for a fulll scap for them [12:09:18] However of course, we need to make sure only 1 scap runs at once, and I am aware of block-downstream and block-upstream, but if I made another scap job, this would actually be block-sideways ? [12:09:55] OR I modify beta-scap-eqiad to accept an env var somehow of the DIR to scap, and then by default that is "" and for config that is "wmf-config" ? [12:10:06] I'm sure you could figure out what to do :D [12:12:58] PROBLEM - Free space - all mounts on deployment-videoscaler01 is CRITICAL: CRITICAL: deployment-prep.deployment-videoscaler01.diskspace.root.byte_percentfree (<10.00%) [12:15:34] !log deployment-videoscaler01: apt-get clean to free up disk space [12:15:39] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:16:11] !log deployment-prep: sudo cumin --force '*' 'apt-get clean' [12:16:16] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:16:27] !log integration: sudo cumin --force '*' 'apt-get clean' [12:16:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:17:54] (03PS1) 10Zfilipin: Update RuboCop Ruby gem [ruby/api] - 10https://gerrit.wikimedia.org/r/395511 (https://phabricator.wikimedia.org/T180878) [12:18:38] !log deployment-videoscaler01: rm /var/log/hhvm/* /var/log/apache2/* . Restarted apache2/hhvm/syslog [12:18:43] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [12:21:00] addshore: so both the job that update mediawiki-config and the one that update code trigger beta-scap-eqiad and wait for [12:21:01] it [12:21:28] I am not sure it is worth optimizing but maybe it is hehe [12:21:45] so most probably you want a new job that just scap sync the config [12:21:50] well, it means if you merge a change in mediawiki-config that kills beta, the feedback loop is much shorter [12:21:53] (03PS1) 10Zfilipin: Update RuboCop Ruby gem [selenium] - 10https://gerrit.wikimedia.org/r/395513 (https://phabricator.wikimedia.org/T180878) [12:21:58] then the directory holding the config also hold the code [12:22:35] I think what you would need is a mutex between the current beta-scap-eqiad and a future beta-scap-config-eqiad maybe [12:22:44] what I am not quite sure though [12:22:53] Project selenium-GettingStarted » firefox,beta,Linux,BrowserTests build #607: 04FAILURE in 53 sec: https://integration.wikimedia.org/ci/job/selenium-GettingStarted/BROWSER=firefox,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/607/ [12:22:54] is the cod ei supdated every 10 minutes which then run scap [12:23:07] and while that is running, I dont think the mediawiki-config update job is able to run scap [12:23:13] (03CR) 10jerkins-bot: [V: 04-1] Update RuboCop Ruby gem [selenium] - 10https://gerrit.wikimedia.org/r/395513 (https://phabricator.wikimedia.org/T180878) (owner: 10Zfilipin) [12:23:21] beta-code-update-eqiad is every 10 mins [12:23:33] beta-mediawiki-config-update-eqiad is on patch merge [12:23:41] yeah and both jobs wait for scap-eqiad [12:24:21] Yup, and the scap takes 5-7 mins, but a scap of only wmf-config would only take around 30 seconds [12:24:31] then [12:24:42] if you already have a scap running due to the code update [12:24:51] you will not be able to run a second scap for the config files [12:25:02] yup, but then the job would fail [12:25:03] scap has a lock for that to prevent concurrent runs [12:25:19] it would be nicer to have them either wait, or only 1 job of that type be able to run at a time [12:25:38] there is only one run of scap-eqiad possible, it is not concurrently runnable [12:25:40] so you get [12:25:51] beta-code-update -> scap run for X minutes [12:26:02] if a config change is merged, it get deployed and then jenkins trigger the scap run [12:26:11] but that build waits till the scap run from beta-code-update is complete [12:27:10] yup [12:27:24] bah https://integration.wikimedia.org/ci/view/Beta/job/beta-scap-eqiad/buildTimeTrend [12:27:59] https://integration.wikimedia.org/ci/view/Beta/job/beta-scap-eqiad/184960/consoleFull is super verbose ( 6MBytes [12:28:01] RECOVERY - Free space - all mounts on deployment-videoscaler01 is OK: OK: All targets OK [12:28:09] i was thinking, if we limited deployment-beta to only have 1 executor, it would work, as a seperate job [12:28:36] anyway part of why scap is so long is [12:28:37] 00:01:37.665 11:55:25 Updating LocalisationCache for master using 6 thread(s) [12:28:37] 00:09:56.425 12:03:43 Generating JSON versions and md5 files [12:28:41] yup [12:28:55] which I guess because we build the l10n cache for every single extensions registered to mediawiki/extensions [12:29:03] yup [12:29:21] originally I made beta to clone mediawiki/extensions.git because I was too lazy to write a few lines to only clone the extensions we actually need [12:29:32] so maybe we can rework that part [12:29:45] less l10n files, faster l10n code generation, faster scap! ? [12:30:27] yup [12:31:54] let me throw up a patch quickly [12:32:00] k k :) [12:32:06] I am going to eat something though [12:35:00] hashar: idea 1, but i doubt that will work https://gerrit.wikimedia.org/r/#/c/395519/ [12:36:32] idea 2 https://gerrit.wikimedia.org/r/#/c/395520/ [12:39:54] meh, even when jsut doing scap sync-file for wmf-config it still does a bunch of spammy unexpected git stuff [12:42:15] 10Release-Engineering-Team (Kanban), 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3812348 (10mmodell) Yeah this probably deserves a new task. It doesn't appear to affect most queries. [12:44:21] 10Release-Engineering-Team (Kanban), 10Phabricator: Phabricator search returns low quality results for "db1110" and similar queries - https://phabricator.wikimedia.org/T182088#3812367 (10mmodell) [12:45:26] 10Release-Engineering-Team (Kanban), 10Phabricator, 10Regression: Phabricator search hugely degraded in quality - https://phabricator.wikimedia.org/T180706#3812396 (10jcrespo) 05Open>03Resolved Originally reported issue resolved. [12:45:36] 10Release-Engineering-Team (Kanban), 10Phabricator: Phabricator search returns low quality results for "db1110" and similar queries - https://phabricator.wikimedia.org/T182088#3812367 (10mmodell) Originally reported in {T180706} [12:46:44] 10Release-Engineering-Team (Kanban), 10Phabricator: Phabricator search returns low quality results for "db1110" and similar queries - https://phabricator.wikimedia.org/T182088#3812403 (10mmodell) p:05Triage>03High [12:48:29] 10Release-Engineering-Team (Kanban), 10Phabricator: Phabricator search degraded in quality for almost any query - https://phabricator.wikimedia.org/T182088#3812413 (10jcrespo) p:05High>03Triage [12:48:44] 10Release-Engineering-Team (Kanban), 10Phabricator: Phabricator search degraded in quality for almost any query - https://phabricator.wikimedia.org/T182088#3812367 (10jcrespo) p:05Triage>03High [12:50:09] 10Release-Engineering-Team (Kanban), 10Phabricator: Phabricator search degraded in quality for almost any query - https://phabricator.wikimedia.org/T182088#3812367 (10jcrespo) [12:52:50] 10Release-Engineering-Team (Kanban), 10Phabricator: Phabricator search degraded in quality for almost any query - https://phabricator.wikimedia.org/T182088#3812425 (10jcrespo) [12:56:24] 10Release-Engineering-Team (Kanban), 10Phabricator: Phabricator search degraded in quality for almost any query - https://phabricator.wikimedia.org/T182088#3812426 (10jcrespo) "search phabricator" is at the end of the 3rd page, which means there are 50-100 results before it: https://phabricator.wikimedia.org/s... [13:02:05] 10Release-Engineering-Team (Kanban), 10Phabricator: Phabricator search degraded in quality for almost any query - https://phabricator.wikimedia.org/T182088#3812443 (10jcrespo) I just tested mysql backend on my account, and it doesn't seem to work- so the issue is most likely at code level or data gathering/tok... [13:14:22] (03PS2) 10Hashar: Migrate some npm jobs to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/393246 [13:28:48] (03PS3) 10Hashar: Migrate some npm jobs to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/393246 [13:33:46] (03CR) 10Hashar: [C: 032] "Jobs deployed, tested each of the repos." [integration/config] - 10https://gerrit.wikimedia.org/r/393246 (owner: 10Hashar) [13:34:57] (03Merged) 10jenkins-bot: Migrate some npm jobs to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/393246 (owner: 10Hashar) [13:41:49] RECOVERY - Mediawiki Error Rate on graphite-labs is OK: OK: Less than 1.00% above the threshold [1.0] [14:13:22] (03CR) 10Hashar: "Deployed" [integration/config] - 10https://gerrit.wikimedia.org/r/393246 (owner: 10Hashar) [14:36:17] 10Beta-Cluster-Infrastructure, 10Scoring-platform-team, 10Wikimedia-Logstash, 10monitoring: Make an ORES service log dashboard for logstash-beta - https://phabricator.wikimedia.org/T182005#3812742 (10awight) 05Open>03Resolved a:03awight It needs refinement, but here's a rough pass which just shows al... [14:55:26] 10Release-Engineering-Team (Watching / External), 10Epic, 10MediaWiki-Platform-Team (MWPT-Q2-Oct-Dec-2017): Deploy refactored comment storage - https://phabricator.wikimedia.org/T166733#3812820 (10Anomie) My fault, I forgot the "git pull" before syncing the file out. [15:01:58] (03PS1) 10Hashar: Migrate oojs/core npm job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/395550 [15:08:27] (03CR) 10Hashar: [C: 032] Migrate oojs/core npm job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/395550 (owner: 10Hashar) [15:09:37] (03Merged) 10jenkins-bot: Migrate oojs/core npm job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/395550 (owner: 10Hashar) [15:14:26] (03PS1) 10Hashar: Migrate a VisualEditor npm job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/395553 [15:16:12] (03CR) 10Hashar: [C: 04-1] "gyp fails due to lack of python while building mmmagic" [integration/config] - 10https://gerrit.wikimedia.org/r/395553 (owner: 10Hashar) [15:21:58] Is something going on with CI...? [15:22:49] hashar: o/ I'm seeing the following CI failure: https://integration.wikimedia.org/ci/job/apps-android-wikipedia-npm-node-6-docker/8/console [15:26:59] dbrant: aRHGH [15:27:07] dbrant: yeah that is my fault sorry [15:38:02] dbrant: looking for a proper way to fix it [15:38:40] :thumbsup: [15:41:05] (03PS1) 10Hashar: Add python-minimal to npm base images [integration/config] - 10https://gerrit.wikimedia.org/r/395555 [15:46:25] (03PS1) 10Hashar: Revert "Migrate oojs/core npm job to Docker" [integration/config] - 10https://gerrit.wikimedia.org/r/395556 [15:46:29] (03PS1) 10Hashar: Revert "Migrate some npm jobs to Docker" [integration/config] - 10https://gerrit.wikimedia.org/r/395557 [15:47:07] (03PS2) 10Hashar: Revert "Migrate oojs/core npm job to Docker" [integration/config] - 10https://gerrit.wikimedia.org/r/395556 [15:47:09] (03PS2) 10Hashar: Revert "Migrate some npm jobs to Docker" [integration/config] - 10https://gerrit.wikimedia.org/r/395557 [15:47:30] dbrant: yeah hmm I am reverting my changes. They are wrong :( [15:47:39] some state is left between builds which is a nono [15:49:10] addshore: I think I gonna drop the ci-src-setup step [15:49:28] in favor of having a single container to do the source preparation + running the test command [15:49:45] which I guess is how we will end up running them later on on Kubernetes [15:50:05] (03CR) 10Hashar: [C: 032] Revert "Migrate oojs/core npm job to Docker" [integration/config] - 10https://gerrit.wikimedia.org/r/395556 (owner: 10Hashar) [15:50:07] (03CR) 10Hashar: [C: 032] Revert "Migrate some npm jobs to Docker" [integration/config] - 10https://gerrit.wikimedia.org/r/395557 (owner: 10Hashar) [15:52:21] (03Merged) 10jenkins-bot: Revert "Migrate oojs/core npm job to Docker" [integration/config] - 10https://gerrit.wikimedia.org/r/395556 (owner: 10Hashar) [15:52:22] (03Merged) 10jenkins-bot: Revert "Migrate some npm jobs to Docker" [integration/config] - 10https://gerrit.wikimedia.org/r/395557 (owner: 10Hashar) [16:14:17] hashar hi, im wondering how did you get chromium working in debian please? [16:21:48] paladox: using stretch ;D [16:21:59] yep /me is trying to use that :) [16:22:03] and passing -no-sandbox [16:23:29] i guess things like [16:23:30] ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM default [16:23:36] and [16:23:37] [9811:9831:1205/162233.470522:ERROR:bus.cc(395)] Failed to connect to the bus: Failed to connect to socket /var/run/dbus/system_bus_socket: No such file or directory [16:23:41] are normal? [16:23:57] (google-chrome:9811): LIBDBUSMENU-GLIB-WARNING **: Unable to get session bus: Unknown or unsupported transport 'disabled' for address 'disabled:' [16:52:14] (03PS1) 10Thcipriani: SSD: Switch Mathoid to service pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/395570 [17:07:58] 10Continuous-Integration-Config, 10Wikidata: Wikibase\Repo\Tests\Actions\ViewEntityActionTest::testShowDiff failing on CI - https://phabricator.wikimedia.org/T181928#3813582 (10thiemowmde) [17:08:34] 10Continuous-Integration-Config, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata: Wikibase\Repo\Tests\Actions\ViewEntityActionTest::testShowDiff failing on CI - https://phabricator.wikimedia.org/T181928#3806840 (10thiemowmde) p:05Triage>03High a:03thiemowmde [17:24:42] (03PS1) 10Florianschmidtwelzow: Whitelist GCI student Albert221 [integration/config] - 10https://gerrit.wikimedia.org/r/395575 [17:25:41] (03CR) 10Albert221: [C: 031] Whitelist GCI student Albert221 [integration/config] - 10https://gerrit.wikimedia.org/r/395575 (owner: 10Florianschmidtwelzow) [17:31:18] hashar: naaawww :( [17:31:18] hashar: it would be nice to have the setup or fetching of code optional in the images then [17:38:44] (03CR) 10Jforrester: [C: 031] "Time to merge this." [integration/config] - 10https://gerrit.wikimedia.org/r/394111 (https://phabricator.wikimedia.org/T174591) (owner: 10Zfilipin) [17:48:36] (03PS1) 10Ladsgroup: Remove Wikidata extension [integration/config] - 10https://gerrit.wikimedia.org/r/395581 (https://phabricator.wikimedia.org/T173818) [19:31:26] 10Scap: Support git-lfs - https://phabricator.wikimedia.org/T180627#3814038 (10mmodell) [19:48:15] 10Release-Engineering-Team (Kanban), 10Phabricator: Phabricator search degraded in quality for almost any query - https://phabricator.wikimedia.org/T182088#3814097 (10mmodell) @jcrespo: Indeed, something changed with the last update which broke tokenization. I'm working on it, sorry it's taking so long to figu... [19:49:49] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [19:56:21] 10Release-Engineering-Team (Kanban), 10User-greg: Create draft offsite schedule - https://phabricator.wikimedia.org/T180618#3814108 (10greg) 05Open>03Resolved Draft done [20:03:14] 10Release-Engineering-Team (Next), 10Scap, 10ORES, 10Operations, 10Scoring-platform-team: scap support for git-lfs - https://phabricator.wikimedia.org/T181855#3814132 (10greg) [20:06:42] 10Gerrit, 10Scap (Tech Debt Sprint FY201718-Q2), 10ORES, 10Operations, and 2 others: Plan migration of ORES repos to git-lfs - https://phabricator.wikimedia.org/T181678#3814151 (10greg) [20:06:56] 10Release-Engineering-Team (Watching / External), 10Scap, 10ORES, 10Operations, 10Scoring-platform-team: ORES should use a git large file plugin for storing serialized binaries - https://phabricator.wikimedia.org/T171619#3814156 (10Halfak) [20:16:26] 10Release-Engineering-Team (Kanban), 10Phabricator: Phabricator search degraded in quality for almost any query - https://phabricator.wikimedia.org/T182088#3814196 (10mmodell) @jcrespo: Your suggestions are all good ones, however, it's hard to tell what is happening currently in production. I need to add some... [20:18:59] (03CR) 10Dduvall: [C: 032] SSD: Switch Mathoid to service pipeline [integration/config] - 10https://gerrit.wikimedia.org/r/395570 (owner: 10Thcipriani) [20:19:09] Hey releng! Could someone if they have time, review https://gerrit.wikimedia.org/r/394872 [20:19:12] Thanks! [20:20:18] 10Release-Engineering-Team, 10User-greg: Create #wikimedia-releng-feed and move bots there - https://phabricator.wikimedia.org/T181582#3814203 (10greg) >>! In T181582#3809151, @zeljkofilipin wrote: > @greg I ignore all bots in `#wikimedia-releng`. As far as I am concerned, we can remove IRC ping from Selenium... [20:20:54] PROBLEM - Puppet errors on deployment-ms-be04 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [20:21:47] PROBLEM - Puppet errors on integration-slave-jessie-1003 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [20:22:29] PROBLEM - Puppet errors on integration-slave-jessie-1001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [20:28:56] Project selenium-Wikibase-chrome » chrome,beta,Linux,DebianJessie && contintLabsSlave build #33: 04FAILURE in 41 min: https://integration.wikimedia.org/ci/job/selenium-Wikibase-chrome/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=DebianJessie%20&&%20contintLabsSlave/33/ [20:28:59] (03PS1) 10Hashar: I DONT KNOW WHAT I AM DOING [integration/config] - 10https://gerrit.wikimedia.org/r/395610 [20:31:11] Can we bash that? [20:32:08] PROBLEM - Puppet errors on integration-slave-jessie-1004 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [20:33:57] Zppix: yes [20:34:10] Ok [20:34:14] Doing it [20:34:14] --volume /"$(pwd)"/cache://cache \ [20:34:14] --volume /"$(pwd)"/cache/node_modules://src/node_modules \ [20:34:20] being the craziness :D [20:34:42] but yeah feel free to bash the commit message above :] [20:34:51] Zppix: thank you! [20:35:43] https://tools.wmflabs.org/bash/quip/AWAoY_bBJfYMc0lpkgiu [20:38:25] (03PS1) 10Thcipriani: SSD: always pull FROM images for build [integration/config] - 10https://gerrit.wikimedia.org/r/395613 [20:41:50] Yippee, build fixed! [20:41:51] Project selenium-Echo » chrome,beta,Linux,BrowserTests build #600: 09FIXED in 50 sec: https://integration.wikimedia.org/ci/job/selenium-Echo/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=BrowserTests/600/ [20:42:11] Zppix: haha quip comes from bugzilla :] [20:42:17] I had completely forgotten about it [20:43:12] hashar: ive been reading it and i can tell that people have changed alot over the years [20:44:01] https://tools.wmflabs.org/bash/quip/AU7VVmnh6snAnmqnK_yr is definitely my favorite one [20:44:15] I should print it as a remember note to be nice with newcomers [20:44:47] Zppix: I started being involved in 2002 when I was barely 24 years old [20:45:00] I am close to my 40 now. Yeah we definitely change over time [20:46:20] I love that too [20:46:31] PROBLEM - Puppet errors on deployment-ms-be03 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [20:47:00] https://tools.wmflabs.org/bash/quip/AU7VT9KD6snAnmqnK_ql And this one [20:54:49] RECOVERY - Mediawiki Error Rate on graphite-labs is OK: OK: Less than 1.00% above the threshold [1.0] [20:55:54] (03CR) 10Thcipriani: [C: 032] SSD: always pull FROM images for build [integration/config] - 10https://gerrit.wikimedia.org/r/395613 (owner: 10Thcipriani) [20:57:07] (03Merged) 10jenkins-bot: SSD: always pull FROM images for build [integration/config] - 10https://gerrit.wikimedia.org/r/395613 (owner: 10Thcipriani) [21:12:25] 10Gerrit, 10Release-Engineering-Team, 10Cleanup, 10Wikidata, and 3 others: Mark extension-Wikidata & wikidata-build-resources on Gerrit as ARCHIVED - https://phabricator.wikimedia.org/T181838#3814325 (10greg) [21:12:31] 10Gerrit, 10Release-Engineering-Team (Someday), 10Cleanup, 10Wikidata, and 3 others: Mark extension-Wikidata & wikidata-build-resources on Gerrit as ARCHIVED - https://phabricator.wikimedia.org/T181838#3803897 (10greg) [21:22:05] (03PS2) 10Hashar: I DONT KNOW WHAT I AM DOING [integration/config] - 10https://gerrit.wikimedia.org/r/395610 [21:27:42] (03CR) 10Hashar: [C: 032] Whitelist GCI student Albert221 [integration/config] - 10https://gerrit.wikimedia.org/r/395575 (owner: 10Florianschmidtwelzow) [21:28:50] (03Merged) 10jenkins-bot: Whitelist GCI student Albert221 [integration/config] - 10https://gerrit.wikimedia.org/r/395575 (owner: 10Florianschmidtwelzow) [21:29:09] Lol [21:29:20] I somehow managed to commit the exact same thing... [21:38:58] (03PS9) 10Paladox: Backport fix "Fix change number extraction on new enough Gerrit master" [integration/zuul] (debian/jessie-wikimedia) - 10https://gerrit.wikimedia.org/r/356181 (https://phabricator.wikimedia.org/T158243) [21:38:58] (03CR) 10Reedy: [C: 031] I DONT KNOW WHAT I AM DOING [integration/config] - 10https://gerrit.wikimedia.org/r/395610 (owner: 10Hashar) [22:09:21] apt-get remove -y build-essential && apt-get autoremove -y && \ [22:09:23] .... [22:09:28] pfff [22:23:32] (03PS3) 10Hashar: I DONT KNOW WHAT I AM DOING [integration/config] - 10https://gerrit.wikimedia.org/r/395610 [22:23:34] (03PS1) 10Hashar: docker: keep build-essential in npm images [integration/config] - 10https://gerrit.wikimedia.org/r/395648 [22:28:32] (03PS2) 10Hashar: docker: keep build-essential in npm images [integration/config] - 10https://gerrit.wikimedia.org/r/395648 [22:28:34] (03PS4) 10Hashar: I DONT KNOW WHAT I AM DOING [integration/config] - 10https://gerrit.wikimedia.org/r/395610 [22:29:50] (03CR) 10Hashar: "If we want to keep the "npm" "npm-stretch" smaller, we could move jsduck and build-essential down to the "npm-test" and "npm-test-stretch"" [integration/config] - 10https://gerrit.wikimedia.org/r/395648 (owner: 10Hashar) [23:12:41] (03PS2) 10Hashar: Add python-minimal to npm base images [integration/config] - 10https://gerrit.wikimedia.org/r/395555 [23:12:43] (03PS3) 10Hashar: docker: keep build-essential in npm images [integration/config] - 10https://gerrit.wikimedia.org/r/395648 [23:12:45] (03PS5) 10Hashar: I DONT KNOW WHAT I AM DOING [integration/config] - 10https://gerrit.wikimedia.org/r/395610 [23:14:53] (03CR) 10Hashar: "node-gyp requires python. This change is similar to the child change https://gerrit.wikimedia.org/r/#/c/395648/ which adds build-essentia" [integration/config] - 10https://gerrit.wikimedia.org/r/395555 (owner: 10Hashar) [23:15:13] * hashar goes off [23:41:16] 10Release-Engineering-Team (Kanban), 10Phabricator: Phabricator search degraded in quality for almost any query - https://phabricator.wikimedia.org/T182088#3814699 (10mmodell) Ok I optimized the `ngram` index and reindexed maniphest tasks with `bin/search index --type task --force` and now results seem much im... [23:41:53] the fact that node-gyp depends on python is hilarious [23:44:03] 10Release-Engineering-Team (Kanban), 10Phabricator, 10monitoring, 10Browser-Tests: Develop tests for phabricator search to detect regressions / search quality issues - https://phabricator.wikimedia.org/T182160#3814704 (10mmodell) p:05Triage>03High [23:45:13] (03Abandoned) 10Zppix: Add GCI 2017 student: Albert221 to whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/394872 (owner: 10Zppix) [23:46:48] 10Release-Engineering-Team (Kanban), 10Phabricator: Phabricator search degraded in quality for almost any query - https://phabricator.wikimedia.org/T182088#3814727 (10mmodell) @jcrespo: I've taken your suggestions into a new task {T182160} [23:48:46] 10Release-Engineering-Team (Kanban), 10Phabricator, 10monitoring, 10Browser-Tests: Develop tests for phabricator search to detect regressions / search quality issues - https://phabricator.wikimedia.org/T182160#3814734 (10mmodell) [23:49:05] 10Release-Engineering-Team (Kanban), 10Phabricator: Phabricator search degraded in quality for almost any query - https://phabricator.wikimedia.org/T182088#3812367 (10mmodell) 05Open>03Resolved