[00:02:06] !log Reestablished Jenkins-Gearman connection [00:02:08] Logged the message, Master [00:09:25] (03CR) 10Krinkle: [C: 032] "Deployed mediawiki-phpunit-zend and mediawiki-phpunit-hhvm." [integration/config] - 10https://gerrit.wikimedia.org/r/198424 (https://phabricator.wikimedia.org/T37912) (owner: 10Krinkle) [00:25:37] Krinkle: Gosh. [00:26:27] Krinkle: Does this mean https://phabricator.wikimedia.org/T37912 can be marked fixed too? [00:26:27] 10Continuous-Integration, 10MediaWiki-extensions-ImageMetrics, 6Multimedia: Karma failing on an ImageMetrics test referencing a value set in a head script - https://phabricator.wikimedia.org/T93459#1137592 (10Tgr) 3NEW [00:26:40] James_F: No, not by a long shot. [00:26:41] PROBLEM - Puppet staleness on deployment-restbase02 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [43200.0] [00:26:58] James_F: I'm still in for a lot of problems and migrations in jobs. There's 100s of them. [00:27:11] Oh, right. [00:27:12] And I'm seeeing a few regressions [00:27:25] https://phabricator.wikimedia.org/T22343 looks like it's further away, if we're abandoning SQLite. [00:27:56] James_F: that's a low-prio long-term issue for mediawiki-core (not CI) to run test against a matrix of database backends. [00:28:04] We're not abandoning SQlite [00:28:31] Didn't you just switch the RL stuff over to MySQL? [00:28:35] CI is switching the default because 1) SQLite is currently not working between mediawiki master and Ubuntu Trusty, 2) MySQL is more valuable as a test target (as long as we only have one) [00:28:56] IT is mwcore's problem to fix sqlite. [00:29:09] I filed a bug for the immediate issue. [00:29:12] Right. [00:29:13] OK. [00:29:56] We just happen to resolve on of its blockers by switching to mysql [00:34:56] (03Merged) 10jenkins-bot: mediawiki-phpunit-{phpflavour}: Use mysql instead of sqlite [integration/config] - 10https://gerrit.wikimedia.org/r/198424 (https://phabricator.wikimedia.org/T37912) (owner: 10Krinkle) [00:43:56] (03PS1) 10Krinkle: mw-phpunit-composer, mw-extensions, integration-phpunit: sqlite > msyql [integration/config] - 10https://gerrit.wikimedia.org/r/198444 (https://phabricator.wikimedia.org/T37912) [00:44:11] (03PS2) 10Krinkle: mw-phpunit-composer, mw-extensions, integration-phpunit: sqlite > mysyql [integration/config] - 10https://gerrit.wikimedia.org/r/198444 (https://phabricator.wikimedia.org/T37912) [00:54:48] (03PS1) 10BryanDavis: Make status output prettier for log files [tools/scap] - 10https://gerrit.wikimedia.org/r/198449 [00:55:15] 10Continuous-Integration, 10MediaWiki-Unit-tests, 7JavaScript, 7Tracking: Make QUnit tests run without installing MediaWiki - https://phabricator.wikimedia.org/T89433#1137643 (10Krinkle) [00:56:23] 10Continuous-Integration, 10MediaWiki-Unit-tests, 7Tracking: Make PHPUnit tests run without installing MediaWiki (tracking) - https://phabricator.wikimedia.org/T89432#1137645 (10Krinkle) [00:56:47] (03CR) 10BryanDavis: "Tired of seeing this in the l10nupdate log file:" [tools/scap] - 10https://gerrit.wikimedia.org/r/198449 (owner: 10BryanDavis) [01:02:41] (03CR) 10Krinkle: [C: 031] Make status output prettier for log files [tools/scap] - 10https://gerrit.wikimedia.org/r/198449 (owner: 10BryanDavis) [01:08:59] (03CR) 10Krinkle: "Deployed :" [integration/config] - 10https://gerrit.wikimedia.org/r/198444 (https://phabricator.wikimedia.org/T37912) (owner: 10Krinkle) [01:18:12] (03PS5) 10Krinkle: fab: Pause to let user "!log" the Zuul reload [integration/config] - 10https://gerrit.wikimedia.org/r/197380 [01:18:26] (03CR) 10Krinkle: [C: 032] "We'll remove it once the logbot works" [integration/config] - 10https://gerrit.wikimedia.org/r/197380 (owner: 10Krinkle) [01:19:35] (03Merged) 10jenkins-bot: fab: Pause to let user "!log" the Zuul reload [integration/config] - 10https://gerrit.wikimedia.org/r/197380 (owner: 10Krinkle) [01:38:45] 10Continuous-Integration, 10Flow: Database field 'workflow.workflow_wiki' too short (not compatible with mediawiki's iw_prefix) - https://phabricator.wikimedia.org/T93463#1137689 (10Krinkle) 3NEW [02:05:15] PROBLEM - Free space - all mounts on deployment-bastion is CRITICAL: CRITICAL: deployment-prep.deployment-bastion.diskspace._var.byte_percentfree.value (<12.50%) [02:09:00] 10Continuous-Integration, 10Flow: Database field 'workflow.workflow_wiki' too short (not compatible with mediawiki's iw_prefix) - https://phabricator.wikimedia.org/T93463#1137713 (10Krinkle) [02:10:11] RECOVERY - Free space - all mounts on deployment-bastion is OK: OK: All targets OK [02:11:38] 6076 Array key should be either a string or an integer in /srv/mediawiki/php-1.25wmf21/includes/libs/MapCacheLRU.php on line 77 [02:11:59] sometimes I fee like those counts fatalmonitor gives are random [02:12:03] (03CR) 10Krinkle: [C: 032] mw-phpunit-composer, mw-extensions, integration-phpunit: sqlite > mysyql [integration/config] - 10https://gerrit.wikimedia.org/r/198444 (https://phabricator.wikimedia.org/T37912) (owner: 10Krinkle) [02:13:42] Krenair: Hehe [02:13:50] Krenair: Why? [02:14:09] Isn't that only supposed to deal with up to 1000 entries at a time? [02:16:28] (03PS1) 10Krinkle: Temporarily remove Flow from mediawiki-extensions combo group [integration/config] - 10https://gerrit.wikimedia.org/r/198456 (https://phabricator.wikimedia.org/T93463) [02:16:39] Krenair: Hm.. not that I know of? [02:17:02] ok [02:17:08] Krenair: Ah< i see what you mean [02:17:10] https://github.com/wikimedia/operations-puppet/blob/fe693dccd2339faabed40ac66d95d33b145d3d95/files/misc/scripts/fatalmonitor#L10 [02:17:19] 13 Exécutable <code>texvccheck</code> absent ; veuillez lire math/README pour le configurer. in /srv/mediawiki/php-1.25wmf18/extensions/Math/MathInputCheckTexvc.php on line 64 [02:17:20] wat [02:17:23] french [02:17:24] wmf18 [02:17:33] texvccheck I thought we fixed already [02:17:44] Krenair: What server are you on? [02:17:52] when was /a/mw-log/hhvm.log last touched there [02:18:25] fluorine [02:18:54] (03PS3) 10Krinkle: mw-phpunit-composer, mw-extensions, integration-phpunit: sqlite > mysyql [integration/config] - 10https://gerrit.wikimedia.org/r/198444 (https://phabricator.wikimedia.org/T37912) [02:19:04] (03PS2) 10Krinkle: Temporarily remove Flow from mediawiki-extensions combo group [integration/config] - 10https://gerrit.wikimedia.org/r/198456 (https://phabricator.wikimedia.org/T93463) [02:21:12] Feb 28 08:27:11 mw1019: message repeated 12 times: [ #012Notice: Exécutable <code>texvccheck</code> absent ; veuillez lire math/README pour le configurer. in /srv/mediawiki/php-1.25wmf18/extensions/Math/MathInputCheckTexvc.php on line 64] [02:21:13] ok [02:21:14] it's old [02:21:22] from before https://phabricator.wikimedia.org/T92707 was fixed [02:21:28] I wasn't aware the log went back that far [02:24:44] (03CR) 10Krinkle: [C: 032] mw-phpunit-composer, mw-extensions, integration-phpunit: sqlite > mysyql [integration/config] - 10https://gerrit.wikimedia.org/r/198444 (https://phabricator.wikimedia.org/T37912) (owner: 10Krinkle) [02:27:45] (03CR) 10Krinkle: [C: 032] "Deployed mediawiki-extensions-hhvm, mediawiki-extensions-zend." [integration/config] - 10https://gerrit.wikimedia.org/r/198456 (https://phabricator.wikimedia.org/T93463) (owner: 10Krinkle) [02:29:27] (03Merged) 10jenkins-bot: mw-phpunit-composer, mw-extensions, integration-phpunit: sqlite > mysyql [integration/config] - 10https://gerrit.wikimedia.org/r/198444 (https://phabricator.wikimedia.org/T37912) (owner: 10Krinkle) [02:32:05] (03Merged) 10jenkins-bot: Temporarily remove Flow from mediawiki-extensions combo group [integration/config] - 10https://gerrit.wikimedia.org/r/198456 (https://phabricator.wikimedia.org/T93463) (owner: 10Krinkle) [04:05:18] Yippee, build fixed! [04:05:18] Project browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #440: FIXED in 8 min 41 sec: https://integration.wikimedia.org/ci/job/browsertests-Echo-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/440/ [04:15:04] (03CR) 1020after4: [C: 032] Make status output prettier for log files [tools/scap] - 10https://gerrit.wikimedia.org/r/198449 (owner: 10BryanDavis) [04:15:23] (03Merged) 10jenkins-bot: Make status output prettier for log files [tools/scap] - 10https://gerrit.wikimedia.org/r/198449 (owner: 10BryanDavis) [04:17:15] 10Continuous-Integration, 10MediaWiki-extensions-ImageMetrics, 6Multimedia: Karma failing on an ImageMetrics test referencing a value set in a head script - https://phabricator.wikimedia.org/T93459#1137792 (10Krinkle) [04:22:31] https://test.wikipedia.org/wiki/Special:AbuseLog/1 [04:22:32] sigh [04:22:33] really [04:36:25] 10Continuous-Integration, 10MediaWiki-extensions-ImageMetrics, 6Multimedia: Karma failing on an ImageMetrics test referencing a value set in a head script - https://phabricator.wikimedia.org/T93459#1137814 (10Krinkle) Using latest master of MediaWiki core and ImageMetrics (and EventLogging, otherwise it fail... [04:40:55] Krenair: wow [04:41:03] Krenair: That wasn't covered by a test? [05:43:26] (03CR) 10Krinkle: "The tools in this repo have been feature frozen for a while now and soft deprecated slated for replacement. If we had more time to investi" [integration/jenkins] - 10https://gerrit.wikimedia.org/r/197014 (owner: 10Adrian Lang) [05:43:30] (03Abandoned) 10Krinkle: Disable gzip in PhantomJS calls [integration/jenkins] - 10https://gerrit.wikimedia.org/r/197014 (owner: 10Adrian Lang) [06:32:45] 10Continuous-Integration, 10Flow: Database field 'workflow.workflow_wiki' too short (not compatible with mediawiki's iw_prefix) - https://phabricator.wikimedia.org/T93463#1137877 (10Krinkle) [06:35:41] 10Continuous-Integration, 5Patch-For-Review: Have jenkins jobs logrotate their build history - https://phabricator.wikimedia.org/T91396#1137883 (10Krinkle) >>! In T91396#1124540, @hashar wrote: > I have cleaned up a few more jobs. Still have to finish up the whole cleanup though. Maybe we can get a test to en... [06:41:23] 10Continuous-Integration, 6Release-Engineering: Regularly run mwext-{name}-testextension-* jobs to make sure they are still passing after core or dependency changes - https://phabricator.wikimedia.org/T93414#1137886 (10Krinkle) Extensions should encounter these naturally if they're being maintained. As a last... [06:41:32] 10Continuous-Integration, 6Release-Engineering: Regularly run mwext-{name}-testextension-* jobs to make sure they are still passing after core or dependency changes - https://phabricator.wikimedia.org/T93414#1137887 (10Krinkle) p:5Triage>3Low [06:51:20] 10Continuous-Integration: /tmp/npm-* directories left behind on Jenkins slaves - https://phabricator.wikimedia.org/T90836#1137893 (10Krinkle) This is similar to the garbage left behind by Doxygen in MediaWiki jobs. I solved that by giving builds their own "root" tmp to play with, by setting `$TMPDIR` to a subdir... [06:51:31] 10Continuous-Integration: /tmp/npm-* directories left behind on Jenkins slaves - https://phabricator.wikimedia.org/T90836#1137894 (10Krinkle) a:3Krinkle [07:11:37] RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0] [09:31:07] 7Blocked-on-RelEng, 6Release-Engineering, 6Multimedia, 6Scrum-of-Scrums, and 2 others: Create basic puppet role for Sentry - https://phabricator.wikimedia.org/T84956#1138011 (10Gilles) So far it was very little work and updating the packages I've put together could be fully automated. The rest of the packa... [10:52:58] RECOVERY - Puppet failure on deployment-cxserver03 is OK: OK: Less than 1.00% above the threshold [0.0] [12:37:34] PROBLEM - SSH on deployment-lucid-salt is CRITICAL: Connection refused [13:03:18] RECOVERY - Puppet failure on deployment-pdf02 is OK: OK: Less than 1.00% above the threshold [0.0] [13:09:38] 10Continuous-Integration, 10MediaWiki-Codesniffer, 10Possible-Tech-Projects, 3Google-Summer-of-Code-2015, 3Outreachy-Round-10: Improving static analysis tools for MediaWiki - https://phabricator.wikimedia.org/T89682#1138082 (10NiharikaKohli) We're holding an IRC meeting on [March 23, at 1700 UTC](http://... [14:45:33] Project browsertests-Wikidata-SmokeTests-linux-firefox-sauce build #193: FAILURE in 28 min: https://integration.wikimedia.org/ci/job/browsertests-Wikidata-SmokeTests-linux-firefox-sauce/193/ [15:19:22] 10Deployment-Systems, 6Services: Evaluate Ansible as a deployment tool - https://phabricator.wikimedia.org/T93433#1138132 (10mark) [15:20:12] 10Deployment-Systems, 6Services: Evaluate Ansible as a deployment tool - https://phabricator.wikimedia.org/T93433#1136938 (10mark) [15:20:13] 10Deployment-Systems, 6Release-Engineering, 6Services, 6operations: Streamline our service development and deployment process - https://phabricator.wikimedia.org/T93428#1138133 (10mark) [15:20:14] 10Deployment-Systems, 6Services, 6operations: Evaluate Docker as a container deployment tool - https://phabricator.wikimedia.org/T93439#1138135 (10mark) [16:47:30] 10Deployment-Systems, 6Release-Engineering, 6Services, 6operations: Streamline our service development and deployment process - https://phabricator.wikimedia.org/T93428#1138189 (10GWicke) [17:01:42] 10Deployment-Systems, 6Services, 6operations: Evaluate Docker as a container deployment tool - https://phabricator.wikimedia.org/T93439#1138203 (10GWicke) [17:05:09] 10Deployment-Systems, 6Services, 6operations: Evaluate Docker as a container deployment tool - https://phabricator.wikimedia.org/T93439#1138205 (10GWicke) [17:05:10] 10Deployment-Systems, 6Services: Evaluate Ansible as a deployment tool - https://phabricator.wikimedia.org/T93433#1138206 (10GWicke) [17:05:11] 10Deployment-Systems, 6Release-Engineering, 6Services, 6operations: Streamline our service development and deployment process - https://phabricator.wikimedia.org/T93428#1138204 (10GWicke) [17:06:38] 10Deployment-Systems, 6Release-Engineering, 6Services, 6operations: Streamline our service development and deployment process - https://phabricator.wikimedia.org/T93428#1136887 (10GWicke) [17:10:34] 10Deployment-Systems, 6Services, 6operations: Evaluate Docker as a container deployment tool - https://phabricator.wikimedia.org/T93439#1138212 (10ori) > Using init scripts from packages inside the container would require root How come? You mention that systemd can run docker images, which is interesting.... [17:44:15] (03PS1) 10Legoktm: Make mwext-JsonData-testextension-.* voting [integration/config] - 10https://gerrit.wikimedia.org/r/198503 [17:46:09] (03CR) 10Legoktm: [C: 032] Make mwext-JsonData-testextension-.* voting [integration/config] - 10https://gerrit.wikimedia.org/r/198503 (owner: 10Legoktm) [17:50:44] (03Merged) 10jenkins-bot: Make mwext-JsonData-testextension-.* voting [integration/config] - 10https://gerrit.wikimedia.org/r/198503 (owner: 10Legoktm) [17:52:57] !log deployed https://gerrit.wikimedia.org/r/198503 [17:53:02] Logged the message, Master [18:47:30] (03CR) 10Polybuildr: "Does anybody have any comments to give on this patch?" [tools/codesniffer] - 10https://gerrit.wikimedia.org/r/196872 (https://phabricator.wikimedia.org/T92749) (owner: 10Polybuildr) [19:15:10] 10Deployment-Systems, 6Services, 6operations: Evaluate Docker as a container deployment tool - https://phabricator.wikimedia.org/T93439#1138337 (10GWicke) > How does that work, exactly? Can systemd be made to invoke the init script somehow? Most docker applications aren't started using an init script execut... [20:03:16] 10Deployment-Systems, 6Services, 6operations: Evaluate Docker as a container deployment tool - https://phabricator.wikimedia.org/T93439#1138357 (10GWicke) [20:05:25] 10Deployment-Systems, 6Services, 6operations: Evaluate Docker as a container deployment tool - https://phabricator.wikimedia.org/T93439#1138358 (10GWicke) [20:08:01] 10Deployment-Systems, 6Services, 6operations: Evaluate Docker as a container deployment tool - https://phabricator.wikimedia.org/T93439#1138364 (10GWicke) [20:27:14] 10Continuous-Integration, 10Flow: Database field 'workflow.workflow_wiki' too short (not compatible with mediawiki's iw_prefix) - https://phabricator.wikimedia.org/T93463#1138378 (10hashar) @Krinkle SQLite does not limit the length of strings, so a `varchar(n)` is just a `text`. [20:29:35] 10Continuous-Integration, 10Flow: Database field 'workflow.workflow_wiki' too short (not compatible with mediawiki's iw_prefix) - https://phabricator.wikimedia.org/T93463#1138380 (10Krinkle) [20:39:15] 7Blocked-on-RelEng, 6Release-Engineering, 6Multimedia, 6Scrum-of-Scrums, and 2 others: Create basic puppet role for Sentry - https://phabricator.wikimedia.org/T84956#1138417 (10hashar) The use of dh-virtualenv is for Zuul T48552 . It is not validated by ops yet though. Note that whenever you update a pyth...