[00:17:00] RECOVERY - Host integration-t102459 is UPING OK - Packet loss = 0%, RTA = 0.73 ms [00:22:27] 6Release-Engineering, 6Commons, 10MediaWiki-File-management, 10MediaWiki-Tarball-Backports, and 7 others: InstantCommons broken by switch to HTTPS - https://phabricator.wikimedia.org/T102566#1416235 (10Seb35) With HTTPS mandatory for InstantCommons, the php5-curl package / curl PHP extension becomes mandat... [00:28:19] PROBLEM - Free space - all mounts on deployment-bastion is CRITICAL deployment-prep.deployment-bastion.diskspace._var.byte_percentfree (<33.33%) [02:36:59] 6Release-Engineering, 6Project-Creators, 6Team-Practices: Create a generic "Release" tag in Phabricator - https://phabricator.wikimedia.org/T99143#1416369 (10Fjalapeno) Great - thanks! [06:38:19] RECOVERY - Free space - all mounts on deployment-bastion is OK All targets OK [08:02:06] 10Deployment-Systems, 6Release-Engineering, 6Performance-Team, 6operations, 7HHVM: Make scap able to depool/repool servers via the conftool API - https://phabricator.wikimedia.org/T104352#1416628 (10mmodell) @bd808: Note that #releng is working on the next-generation of deployment tooling, and I think we... [08:24:38] (03PS1) 10Hashar: Move GEM_HOME under $HOME [integration/config] - 10https://gerrit.wikimedia.org/r/222076 (https://phabricator.wikimedia.org/T101707) [08:42:40] (03CR) 10Hashar: "Refreshed all the browser test jobs as well as:" [integration/config] - 10https://gerrit.wikimedia.org/r/222076 (https://phabricator.wikimedia.org/T101707) (owner: 10Hashar) [08:45:50] (03CR) 10Zfilipin: [C: 032] Move GEM_HOME under $HOME [integration/config] - 10https://gerrit.wikimedia.org/r/222076 (https://phabricator.wikimedia.org/T101707) (owner: 10Hashar) [08:47:46] (03Merged) 10jenkins-bot: Move GEM_HOME under $HOME [integration/config] - 10https://gerrit.wikimedia.org/r/222076 (https://phabricator.wikimedia.org/T101707) (owner: 10Hashar) [08:52:25] zeljkof-meeting: https://wikitech.wikimedia.org/wiki/Salt :) [09:08:43] 10Continuous-Integration-Infrastructure, 6Labs: "puppet-compiler02" Jenkins slave is no more connected - https://phabricator.wikimedia.org/T104428#1416715 (10hashar) 3NEW a:3hashar [09:08:49] 10Continuous-Integration-Infrastructure, 6Labs: "puppet-compiler02" Jenkins slave is no more connected - https://phabricator.wikimedia.org/T104428#1416723 (10hashar) p:5Triage>3Unbreak! [09:10:48] 10Continuous-Integration-Infrastructure, 6Labs: "puppet-compiler02" Jenkins slave is no more connected - https://phabricator.wikimedia.org/T104428#1416715 (10hashar) Console has a bunch of entries related to mounting NFS labstore. Seems the instance hasn't been recovered following the recent labs NFS crash. R... [09:12:40] hashar: could we have a h-o re npm test jobs for services repos? [09:12:53] whenever is most convenient for you [09:13:12] (bonjour) [09:13:13] :P [09:14:24] 10Continuous-Integration-Infrastructure, 6Labs: "puppet-compiler02" Jenkins slave is no more connected - https://phabricator.wikimedia.org/T104428#1416727 (10hashar) labstore.svc.eqiad.wmnet: Temporary failure in name resolution Seems /etc/resolv.conf needs to be adjusted to the new DNS server and DNS sea... [09:16:42] 10Continuous-Integration-Infrastructure, 6Labs: "puppet-compiler02" Jenkins slave is no more connected - https://phabricator.wikimedia.org/T104428#1416728 (10hashar) a:5hashar>3None I can't ssh to the instance so need some labs root to connect to it and fix the instance :-/ [09:16:53] 10Continuous-Integration-Infrastructure, 6Labs, 10Labs-Infrastructure: "puppet-compiler02" Jenkins slave is no more connected - https://phabricator.wikimedia.org/T104428#1416730 (10hashar) [09:40:10] 10Continuous-Integration-Infrastructure, 7Upstream, 7Zuul: [upstream] Jobs are sometime no more being triggered by Zuul / Jenkins - https://phabricator.wikimedia.org/T65760#1416775 (10hashar) [09:40:29] mobrovac: bonjour :-} [09:40:43] mobrovac: sorry your IRC notification got lost in wikibugs spam [09:40:49] > could we have a h-o re npm test jobs for services repos? [09:40:52] what do you mean here? [09:40:56] yeah i figured [09:41:29] 6Release-Engineering, 6Commons, 10MediaWiki-File-management, 10MediaWiki-Tarball-Backports, and 7 others: InstantCommons broken by switch to HTTPS - https://phabricator.wikimedia.org/T102566#1416784 (10Bawolff) [09:41:29] hashar: h-o == hang-out, for setting up jobs to run tests on services gerrit repos [09:43:32] ahhhh [09:43:47] mobrovac: 2pm this afternoon ? [09:43:48] :P [09:43:57] hashar: pkoi pas [09:44:00] I am probably going to head lunch pretty soon with other folks from the coworking place [09:44:16] ah it's almost noon laready [09:44:17] true [09:44:18] kk [09:44:59] invite set :D [10:13:27] 10Continuous-Integration-Infrastructure, 6Release-Engineering, 7Jenkins, 7Upstream: [upstream] Jenkins Gearman plugin has deadlock on executor threads (was: Beta Cluster stopped receiving code updates (beta-update-databases-eqiad hung) - https://phabricator.wikimedia.org/T72597#1416824 (10hashar) >>! In T7... [10:20:59] 10Continuous-Integration-Infrastructure, 6Labs, 10Labs-Infrastructure: "puppet-compiler02" Jenkins slave is no more connected - https://phabricator.wikimedia.org/T104428#1416869 (10yuvipanda) See also T103760 [10:29:09] !log rebased puppet.git on integration-puppetmaster. Autoupdater was blocked by a couple 3-way merges. [10:29:12] Logged the message, Master [10:33:55] PROBLEM - Puppet failure on integration-slave-trusty-1016 is CRITICAL 40.00% of data above the critical threshold [0.0] [10:33:56] PROBLEM - Puppet failure on integration-slave-precise-1013 is CRITICAL 20.00% of data above the critical threshold [0.0] [10:33:58] PROBLEM - Puppet failure on integration-slave-trusty-1017 is CRITICAL 20.00% of data above the critical threshold [0.0] [10:35:36] PROBLEM - Puppet failure on integration-raita is CRITICAL 20.00% of data above the critical threshold [0.0] [10:36:00] bah :(- [10:36:13] !log integration: puppet now fails on instances :-/ [10:36:16] Logged the message, Master [10:36:58] !log Error: /Stage[main]/Ldap::Client::Utils/File[/usr/local/sbin/archive-project-volumes]: Could not evaluate: Could not retrieve information from environment production source(s) puppet:///modules/ldap/scripts/archive-project-volumes [10:37:01] Logged the message, Master [10:37:18] PROBLEM - Puppet failure on integration-puppetmaster is CRITICAL 20.00% of data above the critical threshold [0.0] [10:37:54] PROBLEM - Puppet failure on integration-t102108-trusty-new2 is CRITICAL 50.00% of data above the critical threshold [0.0] [10:38:49] !log restarted puppetmaster on integration [10:38:52] Logged the message, Master [10:38:52] PROBLEM - Puppet failure on integration-saltmaster is CRITICAL 60.00% of data above the critical threshold [0.0] [10:40:55] !log upgrading Jenkins gearman plugin from 0.1.1-8-gf2024bd to 0.1.1-9-g08e9c42-change_192429_2 https://phabricator.wikimedia.org/T72597#1416913 [10:40:57] 10Continuous-Integration-Infrastructure, 6Release-Engineering, 7Jenkins, 7Upstream: [upstream] Jenkins Gearman plugin has deadlock on executor threads (was: Beta Cluster stopped receiving code updates (beta-update-databases-eqiad hung) - https://phabricator.wikimedia.org/T72597#1416913 (10hashar) Rebuildin... [10:40:58] Logged the message, Master [10:41:46] !log restarting Jenkins [10:41:49] Logged the message, Master [10:42:20] PROBLEM - Puppet failure on integration-labsvagrant is CRITICAL 40.00% of data above the critical threshold [0.0] [10:44:06] PROBLEM - Puppet failure on integration-publisher is CRITICAL 66.67% of data above the critical threshold [0.0] [10:46:05] PROBLEM - Puppet failure on integration-slave-precise-1012 is CRITICAL 66.67% of data above the critical threshold [0.0] [10:47:13] PROBLEM - Puppet failure on integration-slave-trusty-1015 is CRITICAL 66.67% of data above the critical threshold [0.0] [10:47:47] !log puppet fixed by restarting the puppet master [10:47:49] Logged the message, Master [10:53:56] RECOVERY - Puppet failure on integration-slave-trusty-1016 is OK Less than 1.00% above the threshold [0.0] [11:02:54] RECOVERY - Puppet failure on integration-t102108-trusty-new2 is OK Less than 1.00% above the threshold [0.0] [11:03:57] RECOVERY - Puppet failure on integration-slave-precise-1013 is OK Less than 1.00% above the threshold [0.0] [11:04:01] RECOVERY - Puppet failure on integration-slave-trusty-1017 is OK Less than 1.00% above the threshold [0.0] [11:05:35] RECOVERY - Puppet failure on integration-raita is OK Less than 1.00% above the threshold [0.0] [11:07:16] RECOVERY - Puppet failure on integration-puppetmaster is OK Less than 1.00% above the threshold [0.0] [11:08:52] RECOVERY - Puppet failure on integration-saltmaster is OK Less than 1.00% above the threshold [0.0] [11:09:07] RECOVERY - Puppet failure on integration-publisher is OK Less than 1.00% above the threshold [0.0] [11:11:07] RECOVERY - Puppet failure on integration-slave-precise-1012 is OK Less than 1.00% above the threshold [0.0] [11:12:12] RECOVERY - Puppet failure on integration-slave-trusty-1015 is OK Less than 1.00% above the threshold [0.0] [11:12:18] RECOVERY - Puppet failure on integration-labsvagrant is OK Less than 1.00% above the threshold [0.0] [11:31:43] Project browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-10-sauce build #85: STILL FAILING in 43 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-en.wikipedia.beta.wmflabs.org-windows_8-internet_explorer-10-sauce/85/ [12:00:24] mobrovac: I am in at https://plus.google.com/hangouts/_/wikimedia.org/amusso-mobrovac?authuser=1 :D [12:25:51] (03PS1) 10Hashar: Polish up mathoid jobs [integration/config] - 10https://gerrit.wikimedia.org/r/222100 [12:34:02] (03CR) 10Hashar: [C: 032] Polish up mathoid jobs [integration/config] - 10https://gerrit.wikimedia.org/r/222100 (owner: 10Hashar) [12:34:34] PROBLEM - Puppet failure on deployment-memc04 is CRITICAL 50.00% of data above the critical threshold [0.0] [12:36:13] (03Merged) 10jenkins-bot: Polish up mathoid jobs [integration/config] - 10https://gerrit.wikimedia.org/r/222100 (owner: 10Hashar) [12:36:26] mobrovac: tutorial https://www.mediawiki.org/wiki/Continuous_integration/Jenkins_job_builder :D [12:38:54] PROBLEM - Puppet failure on deployment-pdf02 is CRITICAL 30.00% of data above the critical threshold [0.0] [12:39:12] PROBLEM - Puppet failure on deployment-restbase02 is CRITICAL 100.00% of data above the critical threshold [0.0] [12:39:22] PROBLEM - Puppet failure on deployment-pdf01 is CRITICAL 30.00% of data above the critical threshold [0.0] [12:41:40] PROBLEM - Puppet failure on deployment-parsoid05 is CRITICAL 30.00% of data above the critical threshold [0.0] [12:43:11] PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL 44.44% of data above the critical threshold [0.0] [12:44:35] PROBLEM - Puppet failure on deployment-memc02 is CRITICAL 50.00% of data above the critical threshold [0.0] [12:45:07] PROBLEM - Puppet failure on deployment-redis02 is CRITICAL 22.22% of data above the critical threshold [0.0] [12:45:25] PROBLEM - Puppet failure on deployment-stream is CRITICAL 20.00% of data above the critical threshold [0.0] [12:45:47] PROBLEM - Puppet failure on deployment-mediawiki03 is CRITICAL 60.00% of data above the critical threshold [0.0] [12:46:13] PROBLEM - Puppet failure on deployment-db1 is CRITICAL 33.33% of data above the critical threshold [0.0] [12:49:01] 6Release-Engineering, 6Commons, 10MediaWiki-File-management, 10MediaWiki-Tarball-Backports, and 7 others: InstantCommons broken by switch to HTTPS - https://phabricator.wikimedia.org/T102566#1417074 (10Tau) I have installed the php5-curl now but still Instantcommons isn't working properly. What next? [12:59:11] RECOVERY - Puppet failure on deployment-restbase02 is OK Less than 1.00% above the threshold [0.0] [13:04:34] RECOVERY - Puppet failure on deployment-memc04 is OK Less than 1.00% above the threshold [0.0] [13:06:18] Yippee, build fixed! [13:06:18] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #703: FIXED in 34 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/703/ [13:08:12] RECOVERY - Puppet failure on deployment-logstash1 is OK Less than 1.00% above the threshold [0.0] [13:10:42] RECOVERY - Puppet failure on deployment-mediawiki03 is OK Less than 1.00% above the threshold [0.0] [13:11:40] RECOVERY - Puppet failure on deployment-parsoid05 is OK Less than 1.00% above the threshold [0.0] [13:14:39] RECOVERY - Puppet failure on deployment-memc02 is OK Less than 1.00% above the threshold [0.0] [13:15:03] RECOVERY - Puppet failure on deployment-redis02 is OK Less than 1.00% above the threshold [0.0] [13:16:36] mobrovac: I posted a quick summary of our pairing session on the public QA list https://lists.wikimedia.org/pipermail/qa/2015-July/002304.html :D [13:17:44] nice hasharConfcall ! [13:17:44] :) [13:18:54] and thnx, it was really useful ;) [13:26:45] (03PS1) 10Hashar: Delete cxserver-source-npm (we use 'npm') [integration/config] - 10https://gerrit.wikimedia.org/r/222112 [13:26:56] (03CR) 10Hashar: [C: 032] Delete cxserver-source-npm (we use 'npm') [integration/config] - 10https://gerrit.wikimedia.org/r/222112 (owner: 10Hashar) [13:29:13] (03Merged) 10jenkins-bot: Delete cxserver-source-npm (we use 'npm') [integration/config] - 10https://gerrit.wikimedia.org/r/222112 (owner: 10Hashar) [13:35:25] RECOVERY - Puppet failure on deployment-stream is OK Less than 1.00% above the threshold [0.0] [13:35:35] 10Deployment-Systems, 10RESTBase: Setup staging for testing RESTBase deploys - https://phabricator.wikimedia.org/T104276#1417162 (10thcipriani) So, I moved the roles info into nodes/labs/staging.yaml and I've moved the hiera info to Hiera:Staging on wikitech so now any instance named `.*-restbase\d{2}` in the... [13:36:13] RECOVERY - Puppet failure on deployment-db1 is OK Less than 1.00% above the threshold [0.0] [13:48:58] RECOVERY - Puppet failure on deployment-pdf02 is OK Less than 1.00% above the threshold [0.0] [13:49:22] RECOVERY - Puppet failure on deployment-pdf01 is OK Less than 1.00% above the threshold [0.0] [13:58:24] (03PS1) 10Hashar: Migrate jshint/jsonlint/jslint jobs to wmflabs [integration/config] - 10https://gerrit.wikimedia.org/r/222115 (https://phabricator.wikimedia.org/T101966) [14:02:39] (03CR) 10Hashar: [C: 032] "Update a bunch of jobs: jshint jsonlint *-jslint" [integration/config] - 10https://gerrit.wikimedia.org/r/222115 (https://phabricator.wikimedia.org/T101966) (owner: 10Hashar) [14:03:45] 10Beta-Cluster, 10Traffic, 5Patch-For-Review: Puppet failing on deployment-prep caches - https://phabricator.wikimedia.org/T104076#1417236 (10akosiaris) 5Open>3Resolved https://gerrit.wikimedia.org/r/222099 fixed the issue for beta for now. It's providing dummy yaml structures to bypass the fact that the... [14:05:38] (03Merged) 10jenkins-bot: Migrate jshint/jsonlint/jslint jobs to wmflabs [integration/config] - 10https://gerrit.wikimedia.org/r/222115 (https://phabricator.wikimedia.org/T101966) (owner: 10Hashar) [14:55:18] (03PS1) 10Hashar: Migrate {name}-phpcs-HEAD to Precise labs [integration/config] - 10https://gerrit.wikimedia.org/r/222123 (https://phabricator.wikimedia.org/T101966) [14:55:22] sudo /etc/init.d/nginx start [14:55:22] Starting nginx: nginx: [emerg] SSL_CTX_use_PrivateKey_file("/etc/ssl/private/star.wmflabs.org.key") failed (SSL: error:0906D06C:PEM routines:PEM_read_bio:no start line error:140B0009:SSL routines:SSL_CTX_use_PrivateKey_file:PEM lib) [14:55:35] causing puppet failure on deployment-cache-bits01 [14:55:56] Krenair: there is a bug for it that is 1 + year old [14:56:25] so puppet has been broken on these machines for a year? [14:56:32] https://phabricator.wikimedia.org/T70387 T70387: Beta Cluster no longer listens for HTTPS [14:56:32] and https://phabricator.wikimedia.org/T50501 T50501: beta: Get SSL certificates for *.{projects}.beta.wmflabs.org [14:58:29] (03CR) 10Hashar: [C: 032] "Migrated. Confirmed in zuul diff the non voting jobs are still non voting. All of them now have the branch filter." [integration/config] - 10https://gerrit.wikimedia.org/r/222123 (https://phabricator.wikimedia.org/T101966) (owner: 10Hashar) [15:13:19] (03Merged) 10jenkins-bot: Migrate {name}-phpcs-HEAD to Precise labs [integration/config] - 10https://gerrit.wikimedia.org/r/222123 (https://phabricator.wikimedia.org/T101966) (owner: 10Hashar) [15:13:36] oops [15:14:00] !log https://integration.wikimedia.org/ci/job/mediawiki-core-phpcs-HEAD/ broken while cloning mediawiki/core :-( [15:14:03] Logged the message, Master [15:21:14] !log manually populating mediawiki/core on Precise instances for mediawiki-core-phpcs-HEAD job using: git config remote.origin.url https://gerrit.wikimedia.org/r/p/mediawiki/core git fetch [15:21:17] Logged the message, Master [15:23:03] poor instances [15:23:28] PROBLEM - Puppet failure on deployment-cache-bits01 is CRITICAL 100.00% of data above the critical threshold [0.0] [15:34:37] !log https://integration.wikimedia.org/ci/job/mediawiki-core-phpcs-HEAD/ is fixed. populated the git repos manually [15:34:40] Logged the message, Master [15:44:19] * hashar rolls the drums [15:44:26] !log Kunal awesome dashboard for repos https://www.mediawiki.org/wiki/User:Legoktm/ci [15:44:30] Logged the message, Master [15:59:51] mobrovac: Thoughts? https://phabricator.wikimedia.org/T104276#1417765 [16:01:21] ah yes [16:01:31] ostriches: will reply on the ticket [16:01:39] Thx! [16:06:02] "enable" means to "Enable one or more unit files" [16:06:06] Thanks systemctl [16:06:10] :) [16:17:48] twentyafterfour: could use some help on https://phabricator.wikimedia.org/T86655#1403352 [16:18:20] ostriches: looking [16:19:34] example url: https://phabricator.wikimedia.org/diffusion/SVN/browse/GOODBYE [16:19:43] ostriches: I have zero svn experience with phabricator [16:19:52] but I'll help debug as best I can [16:21:20] I don't see a GOODBYE repo? [16:21:55] GOODBYE is a file in the SVN repo :P [16:22:45] https://phabricator.wikimedia.org/diffusion/SVN/browse/GOODBYE [16:23:16] So what happened was we decom'd the old svn server. It went bam like reported. I swapped to locally hosted and you see ^ [16:23:50] https://phabricator.wikimedia.org/rSVN115794 is kind of silly and self-contradictory :p [16:25:32] is that really a valid svn path it's passing to the "svn cat" command? [16:25:38] I don't remember svn well enough [16:26:02] Yes. But the repo on disk is a bare repo. [16:26:06] I think I understand. [16:26:32] If it's remotely hosted: you have a working copy but need to fetch info from the remote host (because svn sucks like that) [16:26:37] if I remove the file:// part I get s "is not a working copy" message [16:26:45] Yeah [16:27:43] so how to give it a working copy to work with? [16:27:59] or tell it to treat it as bare [16:29:13] " Working Copy OK /srv/phab/repos/SVN/" [16:29:22] so phab thinks that it is a working copy [16:29:48] Yeah [16:35:57] twentyafterfour: open issues for Subversion + Diffusion: https://secure.phabricator.com/maniphest/query/N1jXKS2O4SfY/#R [16:36:03] ostriches: a svn clone doesn't contain the fulll revision history [16:36:21] Yeah but this isn't a clone [16:36:31] you sure it's the full repo? [16:36:32] It's also not a real repo. [16:36:36] Stupid SVN [16:36:45] demon@iridium:/srv/phab/repos/SVN$ du -sh . [16:36:46] 124K . [16:37:01] that doesn't sound like much [16:37:16] please tell me this wasn't the only copy ;) [16:37:29] We have backups :) [16:37:35] Plus the original hasn't gone anywhere [16:37:51] Just access to it [16:38:10] well what I'm reading seems to be telling me that svn _has_ to have a server no matter what? :-/ [16:38:46] I really hated svn. I am remembering why now [16:38:56] (03PS2) 10Addshore: Run mw-set-env for WikibaseQuality* tests [integration/config] - 10https://gerrit.wikimedia.org/r/221899 (https://phabricator.wikimedia.org/T103626) [16:39:38] We could do it locally on disk. No need for any access. Lemme test something with one of the smaller repos. [16:39:40] ostriches: we probably need to copy the files from the svn server? or from the backup? [16:39:44] (we have 3, thankfully) [16:40:02] I think if we convert it to "remotely" hosted, but point to a place on a disk, it could work. [16:40:09] 10Deployment-Systems, 6Release-Engineering, 6Services, 6operations: Streamline our service development and deployment process - https://phabricator.wikimedia.org/T93428#1417949 (10akosiaris) [16:40:47] oh good morning grrrit-wm, so kind of you to join us [16:40:48] (03CR) 10jenkins-bot: [V: 04-1] Run mw-set-env for WikibaseQuality* tests [integration/config] - 10https://gerrit.wikimedia.org/r/221899 (https://phabricator.wikimedia.org/T103626) (owner: 10Addshore) [16:41:13] (03PS3) 10Addshore: Run mw-set-env for WikibaseQuality* tests [integration/config] - 10https://gerrit.wikimedia.org/r/221899 (https://phabricator.wikimedia.org/T103626) [16:41:42] ostriches: I think it needs to be set to locally hosted but point to a path with the real svn server's copy rather than an svn checkout [16:42:26] Can try that too [16:42:52] (03CR) 10jenkins-bot: [V: 04-1] Run mw-set-env for WikibaseQuality* tests [integration/config] - 10https://gerrit.wikimedia.org/r/221899 (https://phabricator.wikimedia.org/T103626) (owner: 10Addshore) [16:43:08] 10Deployment-Systems, 10RESTBase: Setup staging for testing RESTBase deploys - https://phabricator.wikimedia.org/T104276#1417955 (10mobrovac) >>! In T104276#1417765, @demon wrote: > Had to `sudo systemctl enable restbase.service`, works now. Question is why puppet didn't do this for us? This is a safety measu... [16:43:32] We don't actually have real repos right now. I think the underlying bug here is: converting from remotely hosted (a working copy) to locally hosted (a repo) keeps all Diffusion commits intact but you hose the actual repo history (obviously, actually) [16:43:45] So yeah, lemme try rsyncing the data over and giving it a shot [16:44:49] (03CR) 10Addshore: "I have no clue re these failures, does anyone have any hints?" [integration/config] - 10https://gerrit.wikimedia.org/r/221899 (https://phabricator.wikimedia.org/T103626) (owner: 10Addshore) [16:46:59] addshore: your yaml is invalid [16:47:05] ostriches https://secure.phabricator.com/T6418 [16:47:07] my yaml is invalid :O [16:47:38] (03CR) 10Legoktm: [C: 04-1] Run mw-set-env for WikibaseQuality* tests (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/221899 (https://phabricator.wikimedia.org/T103626) (owner: 10Addshore) [16:47:50] addshore: yaml: ur doin iy rong? [16:47:53] it [16:48:05] typing: I'm doin it rong [16:48:09] :D [16:48:15] I think I am tired, [16:48:34] *copies and pastes what lego wrote...* [16:49:25] (03PS4) 10Addshore: Run mw-set-env for WikibaseQuality* tests [integration/config] - 10https://gerrit.wikimedia.org/r/221899 (https://phabricator.wikimedia.org/T103626) [16:49:29] ahhhhhh, it all makes sense now ;) [16:55:11] man gerrit has two horrible diff viewers, the 'old' and 'new' gerrit view both suck [16:55:51] addshore: I was completely stumped about your yaml problem until I switched it to side by side diff, and then forced gerrit to show all context, and then it made sense ;) [16:56:06] that's like, your opinion man [16:56:20] I use the side-by-side diff [16:56:22] legoktm: indeed it is. [16:56:31] er, I mean inline [16:56:31] the side-by-side definitely sucks less than the other one ;) [16:56:34] noo [16:56:37] but actually [16:56:37] inline is best :P [16:56:40] they both have problems [16:56:52] side by side doesn't like long lines at all [16:57:08] Yep, works. [16:57:11] and refuses to scroll horizontally (sometimes) [16:57:15] ostriches: yay [16:57:17] After importing the repo properly [16:57:23] Will do the other 2 now [16:57:25] cool [16:58:18] legoktm: are you saying you actually like gerrit's diff view though? have you used differential at all? it blows gerrit away IMO [16:59:25] twentyafterfour: I don't really have a preference, they all look the same to me [17:05:54] twentyafterfour: So for upstream...I figure the idea of moving a SVN repo from remote -> local should at least be documented as requiring a manual SVN repo dump import. [17:06:30] Or maybe it was obvious and I'm retarded :) [17:06:35] SVN is so stupid. [17:09:20] ostriches: it's not entirely obvious coming from the world of git [17:09:31] I've been out of SVN too long [17:09:37] if you'd done this back when svn was still the thing then it might have been kinda obvious [17:09:40] yeah [17:09:46] same here [18:35:22] chasemp, twentyafterfour: That MW SVN dump will take awhile to finish. It shouldn't be problematic, but if it is just shoot it and let me know. [18:35:38] (y) [18:35:42] ok [18:35:47] Only hits cpu and disk on iridium, no DB hits here. [18:49:45] 10Beta-Cluster: Lynx warns about invalid cookie for wmflabs.org from beta-labs - https://phabricator.wikimedia.org/T104478#1418325 (10Spage) 3NEW [18:59:54] 10Beta-Cluster: Lynx warns about invalid cookie for wmflabs.org from Beta Cluster - https://phabricator.wikimedia.org/T104478#1418347 (10greg) [19:07:35] 10Browser-Tests, 10VisualEditor: Update VisualEditor repository to mediawiki_selenium Ruby gem 1.1 - https://phabricator.wikimedia.org/T99661#1418352 (10Ryasmeen) a:3dduvall [19:20:24] 10Beta-Cluster, 6Release-Engineering, 10MediaWiki-User-login-and-signup, 10MediaWiki-extensions-CentralAuth: Login failing - The provided authentication token is either expired or invalid. - https://phabricator.wikimedia.org/T104212#1418384 (10Ryasmeen) None of those issues are occurring now. [19:32:56] 6Release-Engineering, 6Team-Practices: Do 90 day review of post-offsite actions in early August '15 - https://phabricator.wikimedia.org/T103917#1418444 (10Awjrichards) [20:09:29] (03PS11) 10Hashar: Configure npm for Metrolook and update tests [integration/config] - 10https://gerrit.wikimedia.org/r/221175 (owner: 10Paladox) [20:10:44] (03CR) 10Hashar: [C: 032] Configure npm for Metrolook and update tests [integration/config] - 10https://gerrit.wikimedia.org/r/221175 (owner: 10Paladox) [20:11:48] (03CR) 10Paladox: "Thanks." [integration/config] - 10https://gerrit.wikimedia.org/r/221175 (owner: 10Paladox) [20:12:43] (03Merged) 10jenkins-bot: Configure npm for Metrolook and update tests [integration/config] - 10https://gerrit.wikimedia.org/r/221175 (owner: 10Paladox) [20:19:08] (03CR) 10Hashar: "@Paladox congratulations, really :-}" [integration/config] - 10https://gerrit.wikimedia.org/r/221175 (owner: 10Paladox) [20:21:21] 10Beta-Cluster: Lynx warns about invalid cookie for wmflabs.org from Beta Cluster - https://phabricator.wikimedia.org/T104478#1418616 (10hashar) p:5Low>3Lowest @Spage feel free to reach out to Lynx developers. Maybe they want a wildcard or do not validate a sub sub sub domain [20:25:54] 10Beta-Cluster, 10Traffic, 5Patch-For-Review: Puppet failing on deployment-prep caches - https://phabricator.wikimedia.org/T104076#1418627 (10hashar) That seems to be a nice workaround @akosiaris. Thank you! [20:28:48] (03PS4) 10Legoktm: Run composer commands inside "src" for mediawiki-core [integration/config] - 10https://gerrit.wikimedia.org/r/193757 [20:39:28] (03CR) 10Paladox: "Ok thanks." [integration/config] - 10https://gerrit.wikimedia.org/r/221175 (owner: 10Paladox) [20:49:00] (03PS1) 10Dduvall: Fix headless setup for MW-Selenium [integration/jenkins] - 10https://gerrit.wikimedia.org/r/222186 [21:06:54] (03PS1) 10Hashar: Whitelist Paladox [integration/config] - 10https://gerrit.wikimedia.org/r/222187 [21:08:18] hashar: Dupe of I3e2f955a which was rejected… [21:08:38] (03CR) 10Hashar: [C: 031] "Jan proposed to whitelist Paradox during the CI weekly meeting." [integration/config] - 10https://gerrit.wikimedia.org/r/222187 (owner: 10Hashar) [21:10:04] (03CR) 10Alex Monk: [C: 04-1] "I can't -2 on this repository, but it's been pointed out that this is a duplicate of I3e2f955a" [integration/config] - 10https://gerrit.wikimedia.org/r/222187 (owner: 10Hashar) [21:10:45] cough, which was veto'd, cough [21:12:28] 10Browser-Tests, 10Continuous-Integration-Infrastructure, 6Release-Engineering, 7Jenkins: Jenkins: browser test host performance issue for timed builds - https://phabricator.wikimedia.org/T68449#1418793 (10hashar) [21:12:31] 10Browser-Tests, 6Release-Engineering: mediawiki_selenium always use the same default xvfb display 99 - https://phabricator.wikimedia.org/T73602#1418791 (10hashar) 5Resolved>3Open Reopening, that is actually a problem when we have multiple browser tests running on the same instance hitting a local mediawik... [21:15:58] 6Release-Engineering, 10Gather, 10MobileFrontend, 3Reading-Web: Ensure tests are green before they hit production - https://phabricator.wikimedia.org/T101865#1418807 (10Jdlrobson) [21:18:25] PROBLEM - Puppet failure on deployment-parsoidcache02 is CRITICAL 100.00% of data above the critical threshold [0.0] [21:19:25] 10Beta-Cluster: Lynx warns about invalid cookie for wmflabs.org from Beta Cluster - https://phabricator.wikimedia.org/T104478#1418819 (10Aklapper) Lynx 2.8.8 manpage says: > Send bug reports, comments, suggestions to after subscribing. [21:21:50] marxarelli: you might be interested in: https://phabricator.wikimedia.org/T101865 [21:24:33] (03CR) 10Hashar: "Bunch of weird 11pm notes :-}" (034 comments) [integration/jenkins] - 10https://gerrit.wikimedia.org/r/222186 (owner: 10Dduvall) [21:24:47] 10Browser-Tests, 6Release-Engineering: mediawiki_selenium always use the same default xvfb display 99 - https://phabricator.wikimedia.org/T73602#1418854 (10hashar) Being discussed on https://gerrit.wikimedia.org/r/#/c/222186/ [21:29:01] (03CR) 10Hashar: Fix headless setup for MW-Selenium (031 comment) [integration/jenkins] - 10https://gerrit.wikimedia.org/r/222186 (owner: 10Dduvall) [21:30:24] (03CR) 10Dduvall: Fix headless setup for MW-Selenium (031 comment) [integration/jenkins] - 10https://gerrit.wikimedia.org/r/222186 (owner: 10Dduvall) [21:30:29] legoktm: know someone who would be interested in doing https://phabricator.wikimedia.org/T104501? [21:32:58] (03CR) 10Hashar: "Ah good catch Alex sorry." [integration/config] - 10https://gerrit.wikimedia.org/r/222187 (owner: 10Hashar) [21:33:26] (03CR) 10Dduvall: [C: 032] "Merging per IRC discussion." [integration/jenkins] - 10https://gerrit.wikimedia.org/r/222186 (owner: 10Dduvall) [21:33:45] (03Abandoned) 10Hashar: Whitelist Paladox [integration/config] - 10https://gerrit.wikimedia.org/r/222187 (owner: 10Hashar) [21:34:00] (03Merged) 10jenkins-bot: Fix headless setup for MW-Selenium [integration/jenkins] - 10https://gerrit.wikimedia.org/r/222186 (owner: 10Dduvall) [21:35:27] marxarelli: we have a salt master to easily git pull the slave-scripts :-} [21:36:51] https://git.wikimedia.org/ has gotten extremely slow... is it just me? [21:37:05] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #687: FAILURE in 1 hr 11 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/687/ [21:37:35] 21:00 < icinga-wm> PROBLEM - git.wikimedia.org on antimony is CRITICAL - Socket timeout after 10 seconds [21:37:38] polybuildr: ^ [21:37:39] not just you [21:37:48] greg-g: alright. thanks! [21:37:56] marxarelli: sudo salt '*-slave*' cmd.run 'cd /srv/deployment/integration/slave-scripts && git rev-parse HEAD' [21:38:10] marxarelli: on integration-saltmaster , runs command as root :-/ [21:39:54] -rw-r----- 1 root root 2405310 Jul 1 21:39 /var/log/upstart/gitblit.log [21:40:04] greg-g: giblit needs root :-( [21:40:17] yeah, godo just bounced it [21:40:19] godog [21:41:19] [Wed Jul 1 13:18:46 2015] init: gitblit main process (15846) killed by KILL signal [21:41:19] [Wed Jul 1 14:06:14 2015] init: gitblit main process (28431) killed by KILL signal [21:41:20] [Wed Jul 1 15:07:01 2015] init: gitblit main process (31924) killed by KILL signal [21:41:20] :D [21:50:02] (03CR) 10Krinkle: Fix headless setup for MW-Selenium (031 comment) [integration/jenkins] - 10https://gerrit.wikimedia.org/r/222186 (owner: 10Dduvall) [21:50:27] hashar: if you know someone who'd be interested, please CC them at https://phabricator.wikimedia.org/T104501 [21:52:26] polybuildr: you might want to list examples in the task details [21:52:38] hashar: yeah, I probably should. :P I'm just too sleepy. [21:52:41] alright, will do. [21:52:44] polybuildr: as you said, it can easily breaks thing though [21:52:53] so I am not sure it is worth updating them just for the sake of it [21:53:00] we can probably whitelist the known ones [21:53:18] and reject anything not in that whitelist, that would prevent introduction of new globals that do not match the convention [21:53:37] I have no idea what the repercussions of changing each one would be, which is why I'd like someone else to check. [21:53:40] hmm, yeah, that's a good idea. [21:53:47] where, though? In codesniffer's code? [21:54:24] yeah in whatever sniff I wrote that check the globals [21:54:30] (if I am the one that wrote that sniff) [21:54:38] I am pretty sure I had some exceptions [21:55:22] polybuildr: maybe [21:55:23] ./MediaWiki/Sniffs/NamingConventions/PrefixedGlobalFunctionsSniff.php [21:55:23] ./MediaWiki/Sniffs/NamingConventions/ValidGlobalNameSniff.php [21:55:38] yeah [21:55:43] it has a list of PHP reserved variables [21:55:47] and a list of mediawiki ones [21:55:52] private static $mediaWikiValid = array( [21:55:52] '$messageMemc', [21:55:52] '$parserMemc', [21:55:52] '$IP', [21:56:01] so you can amend that mediaWikiValid [21:58:59] hashar: right, yeah. it does. okay, will do :) [21:59:22] polybuildr: I commented on the task :} [21:59:36] with pointer to the code [22:00:00] damn that code is from January 2012 :( [22:02:30] hashar: time flies, eh? :P [22:02:39] you git blamed? [22:03:13] polybuildr: yeah [22:03:17] i love git blame [22:03:40] 10Beta-Cluster, 10MediaWiki-API: mw: interwiki prefix missing on beta cluster, so API's "complete documentation" is a 404. - https://phabricator.wikimedia.org/T104504#1418952 (10Spage) 3NEW [22:03:47] hashar: why? :P [22:04:09] I like history? [22:04:48] good answer :D :P [22:05:16] you can look at the mediawiki/core commits from the early days [22:05:20] that is quite interesting [22:05:20] PROBLEM - Free space - all mounts on deployment-bastion is CRITICAL deployment-prep.deployment-bastion.diskspace._var.byte_percentfree (<40.00%) [22:05:28] sleeppp [22:13:38] 10Deployment-Systems, 6Release-Engineering, 7Epic: EPIC: The future of MediaWiki deployment: Tooling - https://phabricator.wikimedia.org/T94620#1418982 (10mmodell) [22:13:40] 10Deployment-Systems, 6Release-Engineering, 6Performance-Team, 6operations, 7HHVM: Make scap able to depool/repool servers via the conftool API - https://phabricator.wikimedia.org/T104352#1418983 (10mmodell) [22:16:09] 10Deployment-Systems, 6Release-Engineering, 7Epic: EPIC: The future of MediaWiki deployment: Tooling - https://phabricator.wikimedia.org/T94620#1418992 (10mmodell) [22:30:53] 6Release-Engineering, 6Commons, 10MediaWiki-File-management, 10MediaWiki-Tarball-Backports, and 7 others: InstantCommons broken by switch to HTTPS - https://phabricator.wikimedia.org/T102566#1419061 (10Bawolff) >>! In T102566#1417074, @Tau wrote: > I have installed the php5-curl now but still Instantcommon... [22:42:54] (03CR) 10Paladox: "Ok." [integration/config] - 10https://gerrit.wikimedia.org/r/222187 (owner: 10Hashar) [23:46:26] PROBLEM - Puppet failure on deployment-stream is CRITICAL 40.00% of data above the critical threshold [0.0] [23:57:16] (03PS1) 10Dduvall: Fix MEDIAWIKI_URL and enable screenshots [integration/jenkins] - 10https://gerrit.wikimedia.org/r/222219