[00:35:09] Project beta-scap-eqiad build #33053: FAILURE in 1 min 10 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/33053/ [00:55:17] Yippee, build fixed! [00:55:18] Project beta-scap-eqiad build #33055: FIXED in 1 min 19 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/33055/ [01:48:45] Yippee, build fixed! [01:48:46] Project browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce build #332: FIXED in 5 min 46 sec: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-mediawiki.org-linux-firefox-sauce/332/ [02:50:40] PROBLEM - Puppet failure on deployment-mx is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [02:51:00] PROBLEM - Puppet failure on deployment-sca-cache01 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [02:54:32] PROBLEM - Puppet failure on deployment-salt is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [02:55:06] PROBLEM - Puppet failure on deployment-db1 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [02:56:15] PROBLEM - Puppet failure on deployment-cache-upload02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [02:56:24] PROBLEM - Puppet failure on deployment-stream is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [02:56:39] Yippee, build fixed! [02:56:40] Project browsertests-VisualEditor-test2.wikipedia.org-linux-firefox-sauce build #351: FIXED in 1 hr 23 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-test2.wikipedia.org-linux-firefox-sauce/351/ [02:57:10] PROBLEM - Puppet failure on deployment-rsync01 is CRITICAL: CRITICAL: 37.50% of data above the critical threshold [0.0] [02:58:34] PROBLEM - Puppet failure on deployment-elastic05 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [02:59:50] PROBLEM - Puppet failure on deployment-db2 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [03:00:04] PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [03:00:32] PROBLEM - Puppet failure on deployment-cxserver03 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [03:02:10] PROBLEM - Puppet failure on deployment-redis02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [03:03:00] PROBLEM - Puppet failure on deployment-elastic06 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [03:03:48] PROBLEM - Puppet failure on deployment-mediawiki03 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [03:04:12] PROBLEM - Puppet failure on deployment-cache-mobile03 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [03:05:04] PROBLEM - Puppet failure on deployment-memc03 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [03:06:24] PROBLEM - Puppet failure on deployment-restbase03 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [03:06:30] PROBLEM - Puppet failure on deployment-cache-text02 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [03:06:38] PROBLEM - Puppet failure on deployment-parsoidcache02 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [03:07:13] PROBLEM - Puppet failure on deployment-elastic07 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [03:07:21] PROBLEM - Puppet failure on deployment-parsoid04 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [03:08:11] PROBLEM - Puppet failure on deployment-restbase02 is CRITICAL: CRITICAL: 62.50% of data above the critical threshold [0.0] [03:08:11] PROBLEM - Puppet failure on deployment-sca01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [03:08:15] PROBLEM - Puppet failure on deployment-upload is CRITICAL: CRITICAL: 37.50% of data above the critical threshold [0.0] [03:08:39] PROBLEM - Puppet failure on deployment-mathoid is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [03:08:41] PROBLEM - Puppet failure on deployment-elastic08 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [03:08:49] PROBLEM - Puppet failure on deployment-eventlogging02 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [03:09:08] PROBLEM - Puppet failure on deployment-fluoride is CRITICAL: CRITICAL: 44.44% of data above the critical threshold [0.0] [03:09:38] PROBLEM - Puppet failure on deployment-redis01 is CRITICAL: CRITICAL: 66.67% of data above the critical threshold [0.0] [03:10:44] PROBLEM - Puppet failure on deployment-pdf02 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [03:10:54] PROBLEM - Puppet failure on deployment-mediawiki01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [03:10:56] PROBLEM - Puppet failure on deployment-videoscaler01 is CRITICAL: CRITICAL: 30.00% of data above the critical threshold [0.0] [03:11:18] PROBLEM - Puppet failure on deployment-apertium01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [03:11:54] PROBLEM - Puppet failure on deployment-jobrunner01 is CRITICAL: CRITICAL: 70.00% of data above the critical threshold [0.0] [03:12:20] PROBLEM - Puppet failure on deployment-memc04 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [03:12:26] PROBLEM - Puppet failure on deployment-mediawiki02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [03:12:26] PROBLEM - Puppet failure on deployment-restbase01 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [03:13:06] PROBLEM - Puppet failure on deployment-memc02 is CRITICAL: CRITICAL: 75.00% of data above the critical threshold [0.0] [03:16:58] PROBLEM - Puppet failure on deployment-parsoid05 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [03:17:36] PROBLEM - Puppet failure on deployment-cache-bits01 is CRITICAL: CRITICAL: 55.56% of data above the critical threshold [0.0] [03:34:42] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce build #124: FAILURE in 32 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce/124/ [03:41:55] Yippee, build fixed! [03:41:55] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8.1-internet_explorer-11-sauce build #178: FIXED in 34 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8.1-internet_explorer-11-sauce/178/ [03:48:24] PROBLEM - Free space - all mounts on deployment-bastion is CRITICAL: CRITICAL: deployment-prep.deployment-bastion.diskspace._var.byte_percentfree.value (<55.56%) [04:35:37] RECOVERY - Puppet failure on deployment-mx is OK: OK: Less than 1.00% above the threshold [0.0] [04:36:00] RECOVERY - Puppet failure on deployment-sca-cache01 is OK: OK: Less than 1.00% above the threshold [0.0] [04:43:26] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #404: FAILURE in 1 hr 1 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/404/ [05:19:24] Yippee, build fixed! [05:19:25] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-monobook-sauce build #163: FIXED in 56 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-chrome-monobook-sauce/163/ [05:20:58] Project browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #53: FAILURE in 7 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/53/ [06:38:24] RECOVERY - Free space - all mounts on deployment-bastion is OK: OK: All targets OK [06:55:33] Project beta-scap-eqiad build #33093: FAILURE in 1 min 31 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/33093/ [07:12:00] PROBLEM - Puppet failure on deployment-sca-cache01 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [07:36:58] RECOVERY - Puppet failure on deployment-sca-cache01 is OK: OK: Less than 1.00% above the threshold [0.0] [07:40:23] Yippee, build fixed! [07:40:24] Project beta-scap-eqiad build #33095: FIXED in 26 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/33095/ [08:32:06] 3Release-Engineering: Add in Phabricator quarterly milestones for RelEng - https://phabricator.wikimedia.org/T75729#832985 (10hashar) 5Resolved>3Open Reopening, since that task is for our team to discuss about milestones. [08:32:15] 3Release-Engineering: Add in Phabricator quarterly milestones for RelEng - https://phabricator.wikimedia.org/T75729#832987 (10hashar) [11:00:56] Project browsertests-VisualEditor-test2.wikipedia.org-linux-firefox-sauce build #352: FAILURE in 1 hr 22 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-test2.wikipedia.org-linux-firefox-sauce/352/ [12:34:19] Project beta-scap-eqiad build #33123: FAILURE in 30 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/33123/ [12:46:30] Yippee, build fixed! [12:46:31] Project beta-scap-eqiad build #33125: FIXED in 2 min 28 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/33125/ [13:10:33] 3Beta-Cluster, Labs-Team: Core dumps fill up /var on labs instances - https://phabricator.wikimedia.org/T1259#833362 (10yuvipanda) [13:15:12] Project beta-scap-eqiad build #33128: FAILURE in 1 min 13 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/33128/ [13:35:15] Yippee, build fixed! [13:35:15] Project beta-scap-eqiad build #33130: FIXED in 1 min 13 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/33130/ [16:07:36] YuviPanda: I may need your root superpowers to fix deployment-salt. It is asking me for a password to use sudo and there are a lot of nslcd errors in /var/log/syslog. [16:07:46] bd808: bah, that. [16:07:51] bd808: see andrew’s email to labs-l [16:07:54] and yes, you do need root. [16:07:57] let me fix [16:08:06] * bd808 hasn't gotten to email yet today [16:09:32] doh. spooky puppet action at a distance [16:09:50] also what ubuntu base wouldn't have sudo installed? [16:09:56] (answer: none) [16:10:44] bd808: heh :) [16:12:42] bd808: try now? [16:13:13] YuviPanda: w00t. works on deployment-salt [16:13:35] I'll make sure Andrew's patch is there in puppet and would guess that will fix the rest eventually [16:13:37] bd808: cool! dpkg-remove didn’t actually work for me, I had to force an apt-get install of sudo-ldap [16:13:41] bd808: I rebased as well [16:13:57] Actually I should be able to force it via salt [16:14:01] yeah [16:19:21] !log forced install of sudo-ldap across beta with: salt '*' cmd.run 'env SUDO_FORCE_REMOVE=yes DEBIAN_FRONTEND=noninteractive apt-get -y install sudo-ldap' [16:19:25] Logged the message, Master [16:31:24] RECOVERY - Puppet failure on deployment-restbase03 is OK: OK: Less than 1.00% above the threshold [0.0] [16:31:26] RECOVERY - Puppet failure on deployment-cache-text02 is OK: OK: Less than 1.00% above the threshold [0.0] [16:31:41] RECOVERY - Puppet failure on deployment-parsoidcache02 is OK: OK: Less than 1.00% above the threshold [0.0] [16:32:19] RECOVERY - Puppet failure on deployment-parsoid04 is OK: OK: Less than 1.00% above the threshold [0.0] [16:33:01] RECOVERY - Puppet failure on deployment-elastic06 is OK: OK: Less than 1.00% above the threshold [0.0] [16:33:15] RECOVERY - Puppet failure on deployment-restbase02 is OK: OK: Less than 1.00% above the threshold [0.0] [16:33:15] RECOVERY - Puppet failure on deployment-upload is OK: OK: Less than 1.00% above the threshold [0.0] [16:33:41] RECOVERY - Puppet failure on deployment-mathoid is OK: OK: Less than 1.00% above the threshold [0.0] [16:33:41] RECOVERY - Puppet failure on deployment-elastic08 is OK: OK: Less than 1.00% above the threshold [0.0] [16:33:49] RECOVERY - Puppet failure on deployment-eventlogging02 is OK: OK: Less than 1.00% above the threshold [0.0] [16:33:57] !log restarted puppetmaster to pick up changes to custom functions [16:33:59] Logged the message, Master [16:34:06] RECOVERY - Puppet failure on deployment-fluoride is OK: OK: Less than 1.00% above the threshold [0.0] [16:34:36] RECOVERY - Puppet failure on deployment-redis01 is OK: OK: Less than 1.00% above the threshold [0.0] [16:35:52] RECOVERY - Puppet failure on deployment-mediawiki01 is OK: OK: Less than 1.00% above the threshold [0.0] [16:36:52] RECOVERY - Puppet failure on deployment-jobrunner01 is OK: OK: Less than 1.00% above the threshold [0.0] [16:37:12] RECOVERY - Puppet failure on deployment-elastic07 is OK: OK: Less than 1.00% above the threshold [0.0] [16:38:07] RECOVERY - Puppet failure on deployment-memc02 is OK: OK: Less than 1.00% above the threshold [0.0] [16:38:11] RECOVERY - Puppet failure on deployment-sca01 is OK: OK: Less than 1.00% above the threshold [0.0] [16:40:03] RECOVERY - Puppet failure on deployment-memc03 is OK: OK: Less than 1.00% above the threshold [0.0] [16:40:42] RECOVERY - Puppet failure on deployment-pdf02 is OK: OK: Less than 1.00% above the threshold [0.0] [16:40:56] RECOVERY - Puppet failure on deployment-videoscaler01 is OK: OK: Less than 1.00% above the threshold [0.0] [16:42:11] RECOVERY - Puppet failure on deployment-rsync01 is OK: OK: Less than 1.00% above the threshold [0.0] [16:42:24] RECOVERY - Puppet failure on deployment-memc04 is OK: OK: Less than 1.00% above the threshold [0.0] [16:42:26] RECOVERY - Puppet failure on deployment-mediawiki02 is OK: OK: Less than 1.00% above the threshold [0.0] [16:42:28] RECOVERY - Puppet failure on deployment-restbase01 is OK: OK: Less than 1.00% above the threshold [0.0] [16:42:36] RECOVERY - Puppet failure on deployment-cache-bits01 is OK: OK: Less than 1.00% above the threshold [0.0] [16:43:32] RECOVERY - Puppet failure on deployment-elastic05 is OK: OK: Less than 1.00% above the threshold [0.0] [16:44:32] RECOVERY - Puppet failure on deployment-salt is OK: OK: Less than 1.00% above the threshold [0.0] [16:45:06] RECOVERY - Puppet failure on deployment-db1 is OK: OK: Less than 1.00% above the threshold [0.0] [16:45:06] RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0] [16:45:30] RECOVERY - Puppet failure on deployment-cxserver03 is OK: OK: Less than 1.00% above the threshold [0.0] [16:46:14] RECOVERY - Puppet failure on deployment-cache-upload02 is OK: OK: Less than 1.00% above the threshold [0.0] [16:46:24] RECOVERY - Puppet failure on deployment-stream is OK: OK: Less than 1.00% above the threshold [0.0] [16:47:08] RECOVERY - Puppet failure on deployment-redis02 is OK: OK: Less than 1.00% above the threshold [0.0] [16:47:36] 3Release-Engineering: Create an outline of QA/Browser test workshops to give - https://phabricator.wikimedia.org/T425#833768 (10Cmcmahon) Notes from refactoring about confusion I see: * Proper use of Given, When, and Then, in particularly Then ** Then steps should always have an assertion, Given and When shou... [16:48:49] RECOVERY - Puppet failure on deployment-mediawiki03 is OK: OK: Less than 1.00% above the threshold [0.0] [16:49:49] RECOVERY - Puppet failure on deployment-db2 is OK: OK: Less than 1.00% above the threshold [0.0] [17:01:56] RECOVERY - Puppet failure on deployment-parsoid05 is OK: OK: Less than 1.00% above the threshold [0.0] [17:04:31] greg-g: is the hangout up? I don't see anyone. [17:05:16] yeah, we're here [17:07:13] weird [17:10:32] 3Release-Engineering, Deployment-Systems: Code Deploy Dashboard - https://phabricator.wikimedia.org/T280#833805 (10mmodell) [17:21:43] 3Scrum-of-Scrums, Beta-Cluster: Create labs ldap user and group "apache" to be used for NFS4 access in deployment-prep project - https://phabricator.wikimedia.org/T76086#833869 (10Andrew) The associated patch was merged. Can someone confirm that this is now resolved? [17:34:18] 3Continuous-Integration: [OPS] Jenkins: Slaves running Ubuntu Trusty should have hhvm installed - https://phabricator.wikimedia.org/T75356#833955 (10zeljkofilipin) [17:34:23] 3Continuous-Integration: [OPS] Jenkins: Slaves running Ubuntu Trusty should have hhvm installed - https://phabricator.wikimedia.org/T75356#833957 (10greg) [17:37:57] 3Release-Engineering: Share summary of MW-Vagrant survey with the org - https://phabricator.wikimedia.org/T430#833966 (10dduvall) [17:37:58] 3MediaWiki-Vagrant, Release-Engineering: Analyze MW-Vagrant qualitative survey - https://phabricator.wikimedia.org/T429#833964 (10dduvall) [18:01:07] 3MediaWiki-Core-Team, Release-Engineering: Make ::mediawiki::syslog and ::mediawiki::php logging destinations configurable via hiera - https://phabricator.wikimedia.org/T1295#833982 (10greg) [18:01:12] 3Release-Engineering: Submit "A Self-Translating User Manual" talk to a couple of FOSDEM devrooms - https://phabricator.wikimedia.org/T1389#833983 (10greg) [18:05:02] does the eventlogging data on beta go anywhere? the browsers request event.gif on bits, but not sure if that is put into a request log or even the full eventlogging stack where it ends up in a db [18:08:41] hmm, I... don't know [18:08:55] it's a bit early for an ori [18:11:44] (03PS1) 10Legoktm: Upgrading Composer to v1.0.0-alpha9 [integration/composer] - 10https://gerrit.wikimedia.org/r/178550 [18:26:01] Project browsertests-Echo-test2.wikipedia.org-linux-chrome-sauce build #215: FAILURE in 20 min: https://integration.wikimedia.org/ci/job/browsertests-Echo-test2.wikipedia.org-linux-chrome-sauce/215/ [18:27:40] greg-g: I straightened up the draft on the etherpad. I hope that wasn't too ridiculous. [18:28:36] chrismcmahon: :) we'll see where it goes [18:37:06] Yippee, build fixed! [18:37:06] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce build #125: FIXED in 35 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_7-internet_explorer-11-sauce/125/ [18:43:30] chrismcmalunch: i appreciated the soul searching. every year at NaNoWriMo we would sit down and parse our values, measuring our goals in relation to them. at times it would feel a little existential but it almost always yielded targeted reforms [18:44:07] ebernhardson: \/ [18:44:07] 13:42 < ori> greg-g: yes, they go to deployment-eventlogging02 [18:44:08] 13:42 < ori> nuria from analytics should be able to help [18:45:15] greg-g: awsome, thanks [18:52:15] (03CR) 10BryanDavis: [C: 031] Upgrading Composer to v1.0.0-alpha9 [integration/composer] - 10https://gerrit.wikimedia.org/r/178550 (owner: 10Legoktm) [19:03:05] 3Release-Engineering: Share summary of MW-Vagrant survey with the org - https://phabricator.wikimedia.org/T430#834139 (10dduvall) [19:35:36] (03CR) 10Legoktm: "Hashar: I've uploaded I84ae32b9fdf745111fbd0f83e80d181695253938 to set basepython = python3. I think that is clearer than renaming the env" [integration/config] - 10https://gerrit.wikimedia.org/r/175607 (owner: 10Legoktm) [19:37:41] PROBLEM - Puppet failure on deployment-pdf01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [19:38:28] it's going to be a fun day, isn't it [19:38:42] PROBLEM - Puppet failure on deployment-bastion is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [19:39:12] ugh, yep, it is [19:40:26] Project beta-scap-eqiad build #33165: FAILURE in 26 min: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/33165/ [19:52:18] Yippee, build fixed! [19:52:19] Project beta-scap-eqiad build #33167: FIXED in 6 min 6 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/33167/ [19:57:28] (03CR) 10Hashar: "IIRC the main impact is on the Wikidata related jobs which extensively use composer. Does anyone have an idea of an integration test to e" [integration/composer] - 10https://gerrit.wikimedia.org/r/178550 (owner: 10Legoktm) [20:03:27] Yippee, build fixed! [20:03:28] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #405: FIXED in 1 hr 4 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/405/ [20:32:44] Yippee, build fixed! [20:32:44] Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #234: FIXED in 15 min: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/234/ [21:13:53] marxarelli: if you have a moment to look this over, I'd appreciate it: https://gerrit.wikimedia.org/r/#/c/178205/ (I tried *everything* to get around that "sleep 1" but Chrome just won't dtrt there.) [21:15:11] (it's a span that contains two other spans. afact, Firefox waits for all three spans to arrive before acting, but Chrome just manipulates the outside span even though the test needs a value in an inside span.) [21:17:12] Yippee, build fixed! [21:17:12] Project browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #54: FIXED in 2 min 26 sec: https://integration.wikimedia.org/ci/job/browsertests-CentralNotice-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/54/ [21:29:43] Project browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce build #351: FAILURE in 12 min: https://integration.wikimedia.org/ci/job/browsertests-UploadWizard-commons.wikimedia.beta.wmflabs.org-linux-firefox-sauce/351/ [21:29:57] 3Continuous-Integration: [OPS] Jenkins: Slaves running Ubuntu Trusty should have hhvm installed - https://phabricator.wikimedia.org/T75356#834842 (10hashar) **On top of my head and from a thread I posted on the internal ops list. That is a bit messy but I think it covers the needs.** We want to use HHVM for the... [21:30:25] 3Continuous-Integration: [OPS] Jenkins: Slaves running Ubuntu Trusty should have hhvm installed - https://phabricator.wikimedia.org/T75356#834845 (10hashar) [21:31:04] 3Continuous-Integration: [OPS] Jenkins: Slaves running Ubuntu Trusty should have hhvm installed - https://phabricator.wikimedia.org/T75356#834853 (10hashar) [21:31:30] 3Continuous-Integration: Publish cdb documentation to doc.wikimedia.org - https://phabricator.wikimedia.org/T75530#834858 (10hashar) [21:32:21] 3Continuous-Integration: Publish cdb documentation to doc.wikimedia.org - https://phabricator.wikimedia.org/T75530#756588 (10hashar) [21:32:58] 3Continuous-Integration: Publish cdb documentation to doc.wikimedia.org - https://phabricator.wikimedia.org/T75530#756588 (10hashar) [21:36:50] 3Continuous-Integration: Jenkins: Set up lint and phpunit jobs for cdb repo - https://phabricator.wikimedia.org/T75541#834890 (10hashar) This is on hold while we figure out how to use composer, specially we need to determine a set of conventions that are used on all repositories which make it easy for developers... [21:42:52] 3Continuous-Integration: [OPS] Jenkins: Slaves running Ubuntu Trusty should have hhvm installed - https://phabricator.wikimedia.org/T75356#834916 (10bd808) >>! In T75356#834842, @hashar wrote: > We will probably want a `php` shell script wrapper inserted in the PATH environment, this way we can have Jenkins swit... [21:45:17] chrismcmahon: Hi Chris, AndyRussG and I were looking around for documentation about the browser test tags @firefox etc... [21:45:54] 3Continuous-Integration: [OPS] Jenkins: Slaves running Ubuntu Trusty should have hhvm installed - https://phabricator.wikimedia.org/T75356#834935 (10bd808) >>! In T75356#834842, @hashar wrote: >> @hashar wrote: >> We have a very [lame script] to run PHPUnit which invokes: >> >> hhvm -vEval.Jit=1 \ >> -vEval.... [21:48:35] hi awight, one moment [21:48:53] chrismcmahon: thanks! [21:49:06] awight: AndyRussG from the Cucumber wiki: https://github.com/cucumber/cucumber/wiki/Tags [21:50:03] awight: AndyRussG in particular, in the WMF universe, a .feature file must have at least one valid browser tag and at least one valid target environment in order to run automatically in the Jenkins build [21:50:45] chrismcmahon: Ah OK... cool... and where are the definitions of which specific tags are available? [21:51:09] AndyRussG: in Jenkins Job Builder [21:51:47] AndyRussG: http://www.mediawiki.org/wiki/Continuous_integration/Jenkins_job_builder [21:52:24] chrismcmahon: excellent... thanks much! :) [21:52:55] AndyRussG: most builds would have beta labs, firefox and chrome. if you want something other than that, let me know. [21:53:13] chrismcmahon: Right now we have this: "@chrome @en.m.wikipedia.beta.wmflabs.org @en.wikipedia.beta.wmflabs.org @firefox" [21:53:27] We've recently found some bugs only on versions of IE though [21:53:38] So it'd be nice to start automating tests there [21:54:06] Also I suppose we should be testing safari on desktop and mobile, and any other common mobile browsers... [21:54:33] AndyRussG: we have all those, e.g. https://integration.wikimedia.org/ci/view/BrowserTests/view/MultimediaViewer/ [21:55:15] cool! [21:55:24] 3Continuous-Integration: [OPS] Jenkins: Slaves running Ubuntu Trusty should have hhvm installed - https://phabricator.wikimedia.org/T75356#834989 (10bd808) [21:55:46] although the CN builds have not been passing for Chrome on Mobile web in some time https://integration.wikimedia.org/ci/view/BrowserTests/view/-All/job/browsertests-CentralNotice-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/ [21:56:25] 3Scrum-of-Scrums, Beta-Cluster: Create labs ldap user and group "apache" to be used for NFS4 access in deployment-prep project - https://phabricator.wikimedia.org/T76086#834996 (10hashar) Not yet On a Trusty instance which knows apache with uid 997 I create a file: ``` hashar@deployment-mediawiki01:~$ id apach... [21:57:15] chrismcmahon: oops! I knew desktop had started turning green again, hadn't seen that one... Could be due to setup of the site we're testing against [21:57:25] could be [22:03:11] 3Scrum-of-Scrums, Beta-Cluster: Create labs ldap user and group "apache" to be used for NFS4 access in deployment-prep project - https://phabricator.wikimedia.org/T76086#835034 (10bd808) >>! In T76086#834996, @hashar wrote: > Not yet > > On a Trusty instance which knows apache with uid 997 I create a file: > >... [22:07:29] chrismcmahon: cc AndyRussG: just to clarify, when we want to add more browsers, is it enough to include the tag in the .feature file, or should we call you about deploying new jenkins builds? [22:07:47] awight: it needs to be done in Jenkins [22:07:55] ok good to know, thanks! [22:08:19] awight: I can make you a quick-n-dirty Jenkins build, but it will disappear when Jenkins restarts unless it is in JJB [22:09:36] chrismcmahon: awight: though I guess the first step will be merging a patch with the new tags? [22:09:45] in the CentralNotice .feature file? [22:09:57] chrismcmahon: I see, so we add the tag in the .feature file, then define the job in jjb/browsertests.yaml [22:09:59] (and before that, fixing the tests that are currently failing of course) [22:10:50] 3Continuous-Integration: [OPS] Jenkins: Slaves running Ubuntu Trusty should have hhvm installed - https://phabricator.wikimedia.org/T75356#835062 (10hashar) [22:12:39] AndyRussG: yep so the mobile issue was just banners targeting the desktop device. I modified them to target all devices except "unknown". [22:15:37] awight: ah ok thanks, yeah I need to get the approriate rights on beta labs in fact... [22:16:17] 3Beta-Cluster: Renumber apache user/group to uid=48 on Trusty beta hosts - https://phabricator.wikimedia.org/T78076#835083 (10bd808) [22:17:23] AndyRussG: yep. I think people on wikimedia-operations randomly gave me that right. [22:17:44] Ah hmm at least sounds like there wasn't much bureaucracy :) [22:18:30] 3Scrum-of-Scrums, Beta-Cluster: Create labs ldap user and group "apache" to be used for NFS4 access in deployment-prep project - https://phabricator.wikimedia.org/T76086#835105 (10hashar) >>! In T76086#835034, @bd808 wrote: > The fact that you can touch the file now shows that the NFS server side is fixed. One... [22:18:39] just swashbuckling [22:23:50] 3Beta-Cluster: File upload area resorts to 0777 permissions to for uploaded content - https://phabricator.wikimedia.org/T75206#835132 (10bd808) [22:23:53] 3Scrum-of-Scrums, Beta-Cluster: Create labs ldap user and group "apache" to be used for NFS4 access in deployment-prep project - https://phabricator.wikimedia.org/T76086#835127 (10bd808) [22:25:59] chrismcmahon: cc awight: how do we get a unified dash of all CentralNotice tests like https://integration.wikimedia.org/ci/view/BrowserTests/view/MultimediaViewer/ ? [22:43:03] 3Scrum-of-Scrums, Beta-Cluster: Create labs ldap user and group "apache" to be used for NFS4 access in deployment-prep project - https://phabricator.wikimedia.org/T76086#835215 (10bd808) [23:14:14] 3Beta-Cluster: Renumber apache user/group to uid=48 on Trusty beta hosts - https://phabricator.wikimedia.org/T78076#835309 (10bd808) Running this on each of the effected servers should work I think: ``` sudo puppet --disable "renumbering apache user" sudo service apache stop sudo service hhvm stop sudo usermod... [23:46:30] PROBLEM - Free space - all mounts on deployment-cache-upload02 is CRITICAL: CRITICAL: deployment-prep.deployment-cache-upload02.diskspace._srv_vdb.byte_percentfree.value (<100.00%) [23:54:10] (03PS1) 10AndyRussG: Add browsers for CentralNotice [integration/config] - 10https://gerrit.wikimedia.org/r/178714 [23:57:16] (03CR) 10Awight: [C: 031] "Looking forward to it!" (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/178714 (owner: 10AndyRussG)