[00:08:08] (03CR) 10EBernhardson: "I'm not completely clear on how zuul and all this works, but looking over the provided patch for Translate extension it looks like Echo a" [integration/config] - 10https://gerrit.wikimedia.org/r/191086 (owner: 10EBernhardson) [00:12:36] (03PS4) 10EBernhardson: Add ParserFunctions to Echo phpunit dependencies [integration/config] - 10https://gerrit.wikimedia.org/r/191086 [00:12:57] (03CR) 10EBernhardson: "Ok actually i think i figured out what needs changes here, please review again :)" [integration/config] - 10https://gerrit.wikimedia.org/r/191086 (owner: 10EBernhardson) [00:15:29] (03Abandoned) 10Legoktm: Add experimental mediawiki-vendor-composer job [integration/config] - 10https://gerrit.wikimedia.org/r/192265 (https://phabricator.wikimedia.org/T74193) (owner: 10Legoktm) [01:03:25] 6Release-Engineering, 6Project-Creators: Add in Phabricator quarterly milestones for RelEng - https://phabricator.wikimedia.org/T75729#1061091 (10greg) 5stalled>3Resolved a:3greg First one created: # releng-201415-q3 Closing. [01:03:47] 6Release-Engineering, 10Staging, 3releng-201415-Q3: [Quarterly Success Metric] Green nightly builds on the staging cluster (tracking) - https://phabricator.wikimedia.org/T88701#1061094 (10greg) [01:03:57] 10Staging, 3releng-201415-Q3: [Quarterly Success Metric] Stable uptime metrics of the Staging cluster - https://phabricator.wikimedia.org/T88705#1061095 (10greg) [01:04:08] 6Release-Engineering, 10Staging, 3releng-201415-Q3: [Quarterly Success Metric] By team test history - https://phabricator.wikimedia.org/T88706#1061096 (10greg) [01:04:16] 6Release-Engineering, 5MW-1.25-release, 3releng-201415-Q3: Release MediaWiki 1.25 - https://phabricator.wikimedia.org/T88709#1061097 (10greg) [01:04:37] 10Continuous-Integration, 7Epic, 3releng-201415-Q3: Jenkins: Run jobs in disposable VMs - https://phabricator.wikimedia.org/T47499#1061098 (10greg) [01:04:46] 6Release-Engineering, 3releng-201415-Q3: [Quarterly Success Metric] RelEng+TPG process discussion and improvements (tracking) - https://phabricator.wikimedia.org/T88708#1061099 (10greg) [01:07:39] 10Continuous-Integration, 7Epic, 3releng-201415-Q3: Jenkins: Run jobs in disposable VMs - https://phabricator.wikimedia.org/T47499#1061107 (10greg) a:3hashar [01:12:09] (03PS1) 10Dduvall: Corrected Style/StringLiterals RuboCop offenses [selenium] - 10https://gerrit.wikimedia.org/r/192484 [01:12:43] (03PS2) 10Dduvall: Corrected Style/StringLiterals RuboCop offenses [selenium] - 10https://gerrit.wikimedia.org/r/192484 [01:50:42] Project beta-update-databases-eqiad build #7804: FAILURE in 30 min: https://integration.wikimedia.org/ci/job/beta-update-databases-eqiad/7804/ [02:52:34] PROBLEM - Puppet staleness on deployment-urldownloader is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [43200.0] [03:14:36] Project beta-scap-eqiad build #42944: FAILURE in 43 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/42944/ [03:34:53] Yippee, build fixed! [03:34:54] Project beta-scap-eqiad build #42946: FIXED in 54 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/42946/ [03:39:36] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8.1-internet_explorer-11-sauce build #333: FAILURE in 32 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8.1-internet_explorer-11-sauce/333/ [03:57:08] 10Continuous-Integration, 10MediaWiki-Unit-tests: Support running MediaWiki PHPUnit tests via composer - https://phabricator.wikimedia.org/T89626#1061273 (10Mattflaschen) 5declined>3Open >>! In T89626#1053416, @hashar wrote: > What Timo said :-]  The shared PHPUnit repo is still used to test REL1_19! Let... [04:36:53] Yippee, build fixed! [04:36:54] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #532: FIXED in 35 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/532/ [04:58:14] PROBLEM - Free space - all mounts on deployment-bastion is CRITICAL: CRITICAL: deployment-prep.deployment-bastion.diskspace._var.byte_percentfree.value (<37.50%) [05:31:42] PROBLEM - Host deployment-db1 is DOWN: PING CRITICAL - Packet loss = 100% [05:32:53] PROBLEM - Host deployment-logstash1 is DOWN: PING CRITICAL - Packet loss = 100% [05:33:36] PROBLEM - Host deployment-sca01 is DOWN: PING CRITICAL - Packet loss = 100% [05:34:14] PROBLEM - Host deployment-cxserver03 is DOWN: PING CRITICAL - Packet loss = 100% [05:35:56] PROBLEM - Host deployment-restbase02 is DOWN: PING CRITICAL - Packet loss = 100% [05:46:30] Project beta-cxserver-update-eqiad build #113: FAILURE in 13 min: https://integration.wikimedia.org/ci/job/beta-cxserver-update-eqiad/113/ [05:46:56] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #486: FAILURE in 36 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/486/ [05:51:29] YuviPanda: shinken is crying, whom to poke? [05:51:38] uh oh [05:51:43] are these hosts all really down? [05:51:48] Yes [05:52:22] ugh [05:52:37] kart_: looking. can you file a bug as well? [05:54:03] PROBLEM - English Wikipedia Main page on beta-cluster is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 MediaWiki exception - 1867 bytes in 1.640 second response time [05:54:32] right [05:54:57] PROBLEM - App Server Main HTTP Response on deployment-mediawiki01 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 MediaWiki exception - 1611 bytes in 2.096 second response time [05:55:41] PROBLEM - English Wikipedia Mobile Main page on beta-cluster is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 MediaWiki exception - 1872 bytes in 6.193 second response time [05:56:07] PROBLEM - App Server Main HTTP Response on deployment-mediawiki02 is CRITICAL: HTTP CRITICAL: HTTP/1.1 500 MediaWiki exception - 1611 bytes in 1.560 second response time [05:57:03] YuviPanda: which project should I add? Beta Cluster? [05:57:14] kart_: yes [05:59:07] 10Beta-Cluster: Beta: All hosts are down - https://phabricator.wikimedia.org/T90530#1061389 (10KartikMistry) 3NEW [05:59:13] https://phabricator.wikimedia.org/T90530 [05:59:18] YuviPanda: ^ [05:59:26] kart_: not all hosts :) [05:59:30] just a bunch of 'em [05:59:48] retitle please :) [06:00:40] 10Deployment-Systems: l10nupdate user can't access scap shared ssh key causing nightly l10nupdate sync process to fail - https://phabricator.wikimedia.org/T76061#1061401 (10mmodell) So, yes, you can authenticate with the remote server via the key on tin, however, the remote server then has to connect back to tin... [06:12:53] 10Deployment-Systems: l10nupdate user can't access scap shared ssh key causing nightly l10nupdate sync process to fail - https://phabricator.wikimedia.org/T76061#1061419 (10bd808) >>! In T76061#1061401, @mmodell wrote: > @bd808: So, yes, you can authenticate with the remote server via the key on tin, however, th... [06:15:29] 10Beta-Cluster, 6Labs, 10Tool-Labs, 6operations: A virt host seems down, taking down all instances with it - https://phabricator.wikimedia.org/T90530#1061420 (10yuvipanda) p:5High>3Unbreak! [06:19:16] kart_: have retitled [06:24:20] YuviPanda: nice! [06:38:12] RECOVERY - Free space - all mounts on deployment-bastion is OK: OK: All targets OK [06:46:26] Project UploadWizard-api-commons.wikimedia.beta.wmflabs.org build #1528: FAILURE in 25 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.beta.wmflabs.org/1528/ [07:52:44] English beta lab is down(http://en.wikipedia.beta.wmflabs.org/wiki/Main_Page) ? or is it just me ? [07:52:54] vikassy: it is down, we're working on it [07:53:08] YuviPanda: great ! thank you :) [07:54:46] 10Continuous-Integration, 10utfnormal: utfnormal tests failing on jenkins and travis-ci - https://phabricator.wikimedia.org/T90278#1061582 (10Legoktm) So here's what I discovered today: * Reliable utfnormal passes on PHP 5.6 (Fedora 21) and HHVM 3.3.1 (Ubuntu Trusty). Fail on PHP 5.5 (Ubuntu Trusty) * Core un... [08:00:34] RECOVERY - English Wikipedia Mobile Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 29242 bytes in 0.594 second response time [08:01:05] RECOVERY - App Server Main HTTP Response on deployment-mediawiki02 is OK: HTTP OK: HTTP/1.1 200 OK - 48375 bytes in 0.556 second response time [08:01:07] RECOVERY - Host deployment-db1 is UP: PING OK - Packet loss = 0%, RTA = 0.94 ms [08:03:04] RECOVERY - Host deployment-cxserver03 is UP: PING OK - Packet loss = 0%, RTA = 0.67 ms [08:03:05] kart_: ^ [08:04:01] RECOVERY - English Wikipedia Main page on beta-cluster is OK: HTTP OK: HTTP/1.1 200 OK - 48572 bytes in 0.564 second response time [08:04:57] RECOVERY - App Server Main HTTP Response on deployment-mediawiki01 is OK: HTTP OK: HTTP/1.1 200 OK - 48393 bytes in 0.587 second response time [08:10:10] PROBLEM - Puppet failure on deployment-db1 is CRITICAL: CRITICAL: 62.50% of data above the critical threshold [0.0] [08:12:02] 6Release-Engineering, 6Engineering-Community, 6Team-Practices, 10Wikimedia-Hackathon-2015, 3ECT-February-2015: RelEng team offsite - https://phabricator.wikimedia.org/T89036#1061592 (10Qgil) Even if this task is not a #Wikimedia-Hackathon-2015 activity, please allow me to associate it in order to keep tr... [08:13:13] YuviPanda: Thanks! [08:16:10] RECOVERY - Host deployment-logstash1 is UP: PING OK - Packet loss = 0%, RTA = 1.94 ms [08:16:22] ^demon|away: let me know when you've time, I need to fix my 'git pull' on tin. [08:19:10] Yippee, build fixed! [08:19:11] Project beta-cxserver-update-eqiad build #114: FIXED in 5.8 sec: https://integration.wikimedia.org/ci/job/beta-cxserver-update-eqiad/114/ [08:21:55] PROBLEM - Content Translation Server on deployment-cxserver03 is CRITICAL: Connection refused [08:27:39] PROBLEM - Puppet failure on deployment-logstash1 is CRITICAL: CRITICAL: 62.50% of data above the critical threshold [0.0] [08:30:10] RECOVERY - Puppet failure on deployment-db1 is OK: OK: Less than 1.00% above the threshold [0.0] [08:33:18] RECOVERY - Host deployment-sca01 is UP: PING OK - Packet loss = 0%, RTA = 0.50 ms [08:35:45] RECOVERY - Host deployment-restbase02 is UP: PING OK - Packet loss = 0%, RTA = 0.97 ms [08:36:57] RECOVERY - Content Translation Server on deployment-cxserver03 is OK: HTTP OK: HTTP/1.1 200 OK - 1103 bytes in 0.019 second response time [08:37:43] PROBLEM - Puppet failure on deployment-restbase02 is CRITICAL: CRITICAL: 25.00% of data above the critical threshold [0.0] [08:47:39] RECOVERY - Puppet failure on deployment-logstash1 is OK: OK: Less than 1.00% above the threshold [0.0] [08:52:44] RECOVERY - Puppet failure on deployment-restbase02 is OK: OK: Less than 1.00% above the threshold [0.0] [08:53:55] 10Deployment-Systems: l10nupdate user can't access scap shared ssh key causing nightly l10nupdate sync process to fail - https://phabricator.wikimedia.org/T76061#1061675 (10mmodell) Ok $SSH_AUTH_SOCK doesn't seem to be getting passed on to the sync targets. Don't we need to enable agent forwarding in ssh config... [09:01:46] 10Deployment-Systems: l10nupdate user can't access scap shared ssh key causing nightly l10nupdate sync process to fail - https://phabricator.wikimedia.org/T76061#1061690 (10mmodell) hmmm I guess it must be something in scap then. I don't know how to get more debugging info from scap, adding --verbose didn't chan... [09:05:10] 10Quality-Assurance, 6Release-Engineering, 10MediaWiki-Vagrant, 10MediaWiki-extensions-PageCuration: Get PageTriage browser tests running on MediaWiki-Vagrant - https://phabricator.wikimedia.org/T90477#1061693 (10zeljkofilipin) [09:13:45] (03CR) 10Zfilipin: [C: 04-1] "I vote for converting to single quotes, all other projects use it. It would be strange to me to land in this repo and see double quotes us" [selenium] - 10https://gerrit.wikimedia.org/r/192484 (owner: 10Dduvall) [09:18:10] 10Continuous-Integration, 10Wikidata: fix the qunit tests for wikidata: mwext-Wikibase-qunit - https://phabricator.wikimedia.org/T74184#1061699 (10adrianheine) >>! In T74184#1060104, @JanZerebecki wrote: > […] Probably should add the apache error log to the build artifacts to be able to find out why. Who is g... [09:19:12] 10Quality-Assurance, 6Release-Engineering: mediawiki_selenium always use the same default xvfb display 99 - https://phabricator.wikimedia.org/T73602#1061708 (10Amire80) [09:19:13] 6Release-Engineering, 10VisualEditor, 7Browser-test-bug: browsertests-VisualEditor-language-screenshot jenkins job failing - https://phabricator.wikimedia.org/T1321#1061704 (10Amire80) 5Open>3Resolved p:5Lowest>3Normal a:3zeljkofilipin [09:19:37] 6Release-Engineering, 10VisualEditor, 7Browser-test-bug: browsertests-VisualEditor-language-screenshot jenkins job failing - https://phabricator.wikimedia.org/T1321#23157 (10Amire80) It was moved to run on Mac, and now runs successfully. [09:20:44] 6Release-Engineering, 10VisualEditor, 7Browser-test-bug: Fix browsertests-VisualEditor-language-screenshot-windows_8.1-firefox Jenkins job - https://phabricator.wikimedia.org/T76133#1061711 (10Amire80) 5Open>3Resolved [09:25:11] 6Release-Engineering, 6Project-Creators: Add in Phabricator quarterly milestones for RelEng - https://phabricator.wikimedia.org/T75729#1061745 (10Aklapper) >>! In T75729#1061091, @greg wrote: > First one created: # releng-201415-q3 I've added "(January-March 2015)" to the description. [09:34:44] Project beta-scap-eqiad build #42982: FAILURE in 45 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/42982/ [09:48:13] hallo zeljkof [09:48:20] hi aharoni [09:48:22] is there a bug about Jenkins changing its language?. [09:48:31] aharoni: in phab? [09:48:35] yes [09:48:35] not sure, but probably [09:48:39] I couldn't find one [09:48:49] time to create one then :) [09:51:05] 10Continuous-Integration: Jenkins sporadically changes its UI language - https://phabricator.wikimedia.org/T90558#1061887 (10Amire80) 3NEW [09:51:11] https://phabricator.wikimedia.org/T90558 [09:55:20] Yippee, build fixed! [09:55:20] Project beta-scap-eqiad build #42984: FIXED in 1 min 5 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/42984/ [10:01:52] zeljkof: hi!! [10:02:02] joakino: hi [10:02:32] I'd love to pair with you on setting the tests up for gather [10:02:38] zeljkof: whenever you can [10:02:40] no rush [10:03:06] joakino: I am free now [10:03:17] I just need a few minutes to get ready [10:03:25] ok, same here [10:03:30] should I set up a hangout for 11:10? [10:03:30] :D [10:03:38] zeljkof: cool with me [10:03:49] joakino: will do, see you in 5 minutes then :) [10:04:13] 👍 [10:13:20] 10Continuous-Integration: Jenkins sporadically changes its UI language - https://phabricator.wikimedia.org/T90558#1061948 (10yuvipanda) It is a test from God specifically targeted at you, @Amire80 [10:13:48] (03PS2) 10Adrian Lang: Ignore HEADLESS and KEEP_BROWSER_OPEN for phantomjs [selenium] - 10https://gerrit.wikimedia.org/r/191552 [10:14:47] (03CR) 10Adrian Lang: "Addressed issue. I could definitely need some guidance for testing this. I don't really know how to get a test double in place for spying " (031 comment) [selenium] - 10https://gerrit.wikimedia.org/r/191552 (owner: 10Adrian Lang) [10:51:04] Yippee, build fixed! [10:51:04] Project browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox » en,contintLabsSlave && UbuntuTrusty build #20: FIXED in 58 min: https://integration.wikimedia.org/ci/job/browsertests-VisualEditor-language-screenshot-os_x_10.10-firefox/LANGUAGE_SCREENSHOT_CODE=en,label=contintLabsSlave%20&&%20UbuntuTrusty/20/ [11:01:28] legoktm: https://github.com/search?q=%22mediawiki%2Fmediawiki-codesniffer%22+@wikimedia&type=Code&utf8=%E2%9C%93 [11:01:37] legoktm: col [11:01:38] cool * [11:35:49] 6Release-Engineering, 10Wikimedia-Hackathon-2015: Hackathon Proposal: Wikimedia Site Requests Sprint - https://phabricator.wikimedia.org/T90468#1062029 (10TTO) Thanks Chad, this work is sorely needed. I went through the Site Requests component in Bugzilla some time ago and was dismayed at the lack of progress... [11:57:26] 10Staging, 6Labs: Increase Security Groups quota on Wikitech staging project - https://phabricator.wikimedia.org/T90473#1062066 (10faidon) [11:57:48] 10Beta-Cluster, 6Labs, 10Tool-Labs, 6operations: A virt host seems down, taking down all instances with it - https://phabricator.wikimedia.org/T90530#1062069 (10yuvipanda) 5Open>3Resolved a:3yuvipanda Fixed now - the instances were all running but no network access. Restarting nova network was of no... [11:58:31] 10Beta-Cluster, 6Labs, 10Tool-Labs, 6operations: Investigate and do incident report for strange virt1012 issues - https://phabricator.wikimedia.org/T90566#1062073 (10yuvipanda) 3NEW a:3yuvipanda [12:00:22] 10Beta-Cluster, 6Labs, 10Tool-Labs, 6operations: Investigate and do incident report for strange virt1012 issues - https://phabricator.wikimedia.org/T90566#1062073 (10yuvipanda) a:5yuvipanda>3Andrew [12:01:35] 10Beta-Cluster, 6Labs, 10Tool-Labs, 6operations: A virt host seems down, taking down all instances with it - https://phabricator.wikimedia.org/T90530#1062087 (10yuvipanda) [12:16:30] 10Deployment-Systems, 6Release-Engineering: update wikitech trebuchet instructions - https://phabricator.wikimedia.org/T90571#1062142 (10fgiunchedi) 3NEW [12:46:55] Yippee, build fixed! [12:46:55] Project UploadWizard-api-commons.wikimedia.beta.wmflabs.org build #1529: FIXED in 31 sec: https://integration.wikimedia.org/ci/job/UploadWizard-api-commons.wikimedia.beta.wmflabs.org/1529/ [12:55:50] @zeljkof:Hi! [12:56:05] hi Jagori [12:57:31] @zeljkof:I had doubts still on the long external link code one commit [12:57:39] 6Release-Engineering, 10Wikimania-Hackathon-2015, 10Wikimedia-Hackathon-2015, 7I18n: Load i18n messages from MediaWiki to browser tests - https://phabricator.wikimedia.org/T90577#1062230 (10Jhernandez) 3NEW [12:57:44] should I remove the Gemfile.lock? [12:58:39] @zeljkof: This is wrt https://gerrit.wikimedia.org/r/#/c/122400/ [12:59:38] Jagori: no, I do not think you should remove the file [13:00:16] zeljkof:Okay,so should I just recheck code and keep it as it is and remove just WIP? [13:00:18] zeljkof: ^ task created [13:00:32] Jagori: yes, I think so [13:00:39] Jagori: great, thanks! [13:00:47] zeljkof:Okay would do that [13:00:53] joakino: great, thanks :) [13:00:55] no problem:) [13:01:09] joakino: I will add it to my wikimania submission :) [13:01:29] zeljkof: hehe :) [13:03:44] zeljkof:Also,in https://gerrit.wikimedia.org/r/#/c/186947/ view_history.feature [13:04:01] where I had removed a line [13:04:28] zeljkof: Is it correct to put in should within 'When' statements? [13:04:48] I thoughen [13:05:04] joakino: done, that increases my chances of going to wikimania :) [13:05:14] zeljkof:sorry typo..:) I was writing I thought it should be part of 'Then' [13:05:30] great [13:05:43] joakino: you might be interested in this https://phabricator.wikimedia.org/T90177 [13:05:45] i'm adding it into additional stuff for the hackathons [13:06:20] Jagori: but you should create a separate commit for that [13:06:22] zeljkof: i'm mobile web, have no idea about android :( [13:06:32] joakino: ignore then :) [13:06:47] zeljkof:Okay ,I will remove the same from here [13:06:51] Thanks [13:06:54] Jagori: the commit you are working on is about converting rspec 2 to 3, not about fixing something else [13:07:04] yes understood [13:07:57] zeljkof:will make the changes and recommit [13:08:00] thanks [13:08:19] Jagori: great, thanks [13:09:12] zeljkof:also,could you assign me some tasks ? [13:10:03] Jagori: did you take a look at phabricator? [13:10:12] all we have should be there [13:10:33] will check [13:22:54] (03CR) 10Zfilipin: [C: 031] Ignore HEADLESS and KEEP_BROWSER_OPEN for phantomjs [selenium] - 10https://gerrit.wikimedia.org/r/191552 (owner: 10Adrian Lang) [13:26:31] 10Deployment-Systems, 6Release-Engineering, 7Documentation: update wikitech trebuchet instructions which still mention deployment::target - https://phabricator.wikimedia.org/T90571#1062310 (10Aklapper) [13:47:46] 10Continuous-Integration, 6Release-Engineering, 7Jenkins: JJB installation problem - https://phabricator.wikimedia.org/T90434#1062334 (10zeljkofilipin) ``` $ python Python 2.7.6 (default, Sep 9 2014, 15:04:36) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin Type "help", "copyright", "credi... [14:04:29] Tobi_WMDE_SW: around? [14:04:52] where do you accept tickets for wikidata browser tests? github or phab? [14:08:34] zeljkof: hey [14:09:45] Tobi_WMDE_SW: https://phabricator.wikimedia.org/T90582 :) [14:10:16] zeljkof: on phabricator, yes [14:10:23] here's the tracking bug: https://phabricator.wikimedia.org/T88541 [14:11:24] zeljkof: I'll add it! thx! [14:11:51] Tobi_WMDE_SW: I think I have managed to add it myself [14:12:03] found it while looking for something else [14:12:13] I have finally had some time to take a look at your commits [14:12:14] yay, thx! [14:12:21] I will leave a few comments [14:13:09] zeljkof: I think they got merged in the meantime [14:13:34] but if you have comments, you can add them though [14:14:06] Tobi_WMDE_SW: yes, everything is merged already :( but I will leave a couple of comments for future reference :) [14:17:07] Tobi_WMDE_SW: ok, done, not sure if github will send mails, if you do not receive a couple of e-mails today, let me know :) [14:18:08] zeljkof: did you comment on the pull requests? [14:18:27] zeljkof: ok, found it [14:18:38] I'll poke christoph about it [14:21:57] Tobi_WMDE_SW: yes, comments on the pull requests [14:23:12] 10Continuous-Integration, 10ContentTranslation-Deployments, 10MediaWiki-extensions-ContentTranslation, 5ContentTranslation-Release4, and 2 others: Enable Debian CI tests on all Apertium packages - https://phabricator.wikimedia.org/T87607#1062479 (10Pginer-WMF) [14:44:18] someone fiddled with the jenkins language or something: תוצרים של בניה מוצלחת אחרונה [14:47:40] can someone check for me why e.g. https://integration.wikimedia.org/ci/job/mwext-Wikibase-qunit/7336/console or https://integration.wikimedia.org/ci/job/mwext-Wikibase-qunit-experiment/5/consolehttps://integration.wikimedia.org/ci/job/mwext-Wikibase-qunit-experiment/5/console shows that the webserver request made for qunit have the response body empty? perhaps by looking at the apache error logs? [14:47:54] or give me permission to check myself? [14:58:50] 10Beta-Cluster, 7Tracking: automatically import some content from production into Beta-Cluster (tracking) - https://phabricator.wikimedia.org/T54382#1062578 (10Aklapper) [15:11:43] aharoni: zeljkof:Can you review https://github.com/amire80/screenshot/pull/8, when you have time ? [15:12:12] vikassy: sure, it is on my list, I hope to get to it today :) [15:12:23] zeljkof:awesome :) [15:12:25] is Jenkins displaying in Hebrew for other people? [15:12:29] Krinkle, zeljkof ^^? [15:12:33] chrismcmahon: yes [15:13:01] <^demon|away> bleh, someone just needs to reset it again [15:13:04] <^demon|away> such a silly bug [15:15:06] chrismcmahon, jzerebecki, ^demon|away: I think amir created a phab ticket for it today :) [15:15:15] should I fix jenkins, or is something on it? [15:15:30] em, somebody on it [15:16:01] <^demon|away> Usually all you need to do is set the language to 'en' again [15:16:32] can someone check for me why e.g. https://integration.wikimedia.org/ci/job/mwext-Wikibase-qunit/7336/console or https://integration.wikimedia.org/ci/job/mwext-Wikibase-qunit-experiment/5/consolehttps://integration.wikimedia.org/ci/job/mwext-Wikibase-qunit-experiment/5/console shows that the webserver request made for qunit have the response body empty? perhaps by looking at the apache error logs? [15:16:46] <^demon|away> it's english again for me yay [15:16:59] fixed [15:17:16] for future reference, log in, go to https://integration.wikimedia.org/ci/configure and click save [15:17:18] that is all [15:17:23] <^demon|away> Yeah [15:17:39] <^demon|away> The bug happens when someone with a non-english language set as their preference restarts jenkins iirc [17:56:20] twentyafterfour: sure, is that ok with you? [17:56:20] greg-g: that's fine though, I just need to do some cleanup at my shop and thursday is gonna be better weather here anyway. I asked for friday because you already set up that meeting for thursday [17:56:20] so yeah either day works for me [17:56:20] cool [18:01:39] (03PS2) 10Werdna: Add OOUIPlayground extension to Jenkins config [integration/config] - 10https://gerrit.wikimedia.org/r/191888 [18:01:48] (03PS1) 10Legoktm: Add '{name}-composer-security' template and use it for mediawiki/vendor [integration/config] - 10https://gerrit.wikimedia.org/r/192564 (https://phabricator.wikimedia.org/T74193) [18:05:06] (03CR) 10Dduvall: "I think you're right to want to stay consistent between our own projects. I'll push a new PS." [selenium] - 10https://gerrit.wikimedia.org/r/192484 (owner: 10Dduvall) [18:05:46] * greg-g has to run to an appt [18:06:28] <^d> bd808: Gah, yes. That too [18:07:20] That's where I got stuck when trying to figure out how Ryan's trebuchet deploy for MediaWiki would work in practice. [18:07:39] Because he was using a similar "slots" model [18:07:41] (03PS3) 10Dduvall: Corrected Style/StringLiterals RuboCop offenses [selenium] - 10https://gerrit.wikimedia.org/r/192484 [18:07:52] which I think can be made to work but things will need to change in the URLs [18:08:00] <^d> We could go back to not having 3 versions deployed :) [18:09:02] 1 branch update about every 6 months works for me ;) [18:09:56] Making static content (which gets versioned) separate from php code on the prod cluster would make things easier I think [18:09:59] in a lot of ways [18:10:37] I think it would be possible to manage that as a build step on the deploy server [18:10:44] <^d> Or we could just get rid of caching and buy a metric shitton of apaches [18:10:50] heh [18:11:48] the urls get written to bits already so somewhere we know what is static and what isn't [18:12:27] I don't think I've seen versioned URLs that aren't handled by bits, but I may e missing something [18:12:50] is that all done by RL? [18:13:02] RECOVERY - Puppet failure on deployment-cache-mobile03 is OK: OK: Less than 1.00% above the threshold [0.0] [18:13:58] <^d> I don't think everything on bits is RL [18:14:21] I'm not sure I understand why static content complicates this? my proposal would still have versions released with each change to the deployed code, but it would use tags for that instead of branches [18:15:45] So there would still be multiple php-1.XwmfY directories on the MW servers? [18:16:17] <^d> How do backports & such fit into this? Do you have to make a new tag? [18:16:18] I took the rolling branch strategy to mean that there would be fewer checked out versions [18:16:36] 1.25wmf12.2? [18:16:49] bd808: there would always be 3 directories checked out [18:17:14] RECOVERY - Puppet failure on deployment-pdf02 is OK: OK: Less than 1.00% above the threshold [0.0] [18:17:17] so that's where the static assets come into the discussion [18:17:18] and backports would append to the tag version yes [18:17:42] <^d> Wouldn't that make bits messier? [18:17:55] I know nothing about bits [18:18:06] less than nothing [18:18:18] today the versioned static assets are served by apache out of the checked out php-1.25wmfX branches [18:18:25] which would need to change [18:18:36] and I think that would be a good change actually [18:19:38] I could make symlinks to each tag that gets deployed I suppose but that gets into the mess we are already in with the current setup [18:19:42] <^d> What would that look like ideally? [18:19:49] <^d> bd808: ^ [18:20:09] what is bits [18:20:37] ^d: I think a static asset collection that is separated from the php code tree at deploy time [18:20:41] nvm found docs [18:21:06] <^d> twentyafterfour: tldr: static assets like CSS/JS/images for UI things [18:21:10] so that the bits vhost is completely independent of the release branche php data [18:21:18] <^d> Heavily cached behind varnish [18:21:59] My assumption is that the static assets are a small part of the footprint of a given release branch [18:22:00] the way we avoided such problems at deviantart is strict rules against static assets in the php repo [18:22:07] +1 [18:22:25] but MediaWiki ain't built that way [18:22:37] <^d> Actually, isn't it mostly images these days? [18:22:42] so we need to separate the assets at deploy time [18:22:46] <^d> bits.wm.o traffic for CSS/JS should all be RL [18:23:39] when we wanted to reference a static asset that snuck into the repo... we had to append a unique ?sequential-identifier to the url wherever we were referencing it ...and increment that identifier every time we committed an update to the asset. then we depended on the cdn cache to work things out from there [18:24:05] <^d> We used to do something along those lines, pre-resourceloader [18:24:59] The true madness in a generic MW deploy comes in the form of extensions which can literally do any damn thing they want [18:25:01] <^d> Oh what the hell was that thing called? [18:25:10] <^d> It was in the freaking "how to commit code" doc [18:25:14] For the WMF cluster though I think we can set some rules [18:25:46] * bd808 is supposed to be thinking about authn refactor [18:25:57] <^d> Ahhhh [18:25:57] <^d> https://www.mediawiki.org/wiki/Manual:$wgStyleVersion [18:26:22] <^d> Actually docs hella wrong [18:26:26] <^d> That doesn't exist anymore [18:26:45] jzerebecki: Found the issue [18:26:54] [18:25 UTC] krinkle at integration-slave1004.eqiad.wmflabs in /mnt/jenkins-workspace/workspace/mwext-Wikibase-qunit [18:26:54] $ php index.php [18:26:54] PHP Parse error: syntax error, unexpected '<' in /mnt/jenkins-workspace/workspace/mwext-Wikibase-qunit/LocalSettings.php on line 377 [18:27:12] The local settings part is a series of concats [18:27:19] except the part from wikibase has no closing bit [18:27:45] ow [18:27:51] https://github.com/wikimedia/integration-jenkins/blob/master/mediawiki/conf.d/00_dev_settings.php [18:28:12] jzerebecki: the ?> shouldn't be in the raw php file either, it's inserted by the concatenator [18:28:18] so e.g. dev_settings.php does not have it [18:28:25] but it gets one at run time [18:28:51] good rule of thumb: never use the ?> in pure-php source files [18:28:52] jzerebecki: it is also setting $wgDebugLogFile = "mw-debug.log"; which is causing the log not to be populated [18:28:58] because that's in the wrong place [18:29:11] Jenkins already sets $wgDebugLogFile earlier to log/mw-debug...log [18:29:16] doh! [18:29:19] Where is that snippet coming from [18:29:33] It's also setting $wgShowExceptionDetails etc. [18:29:37] all that stuff shouldn' be there [18:29:46] so bd808 & ^d: is resource loader used for absolutely everything now? or do extensions not use that at all? [18:29:59] <^d> CSS/JS [18:30:04] and if extensions do wtf ever they want, can we change that [18:30:16] <^d> Every extension we deploy should be using resource loader [18:30:16] and i though meh should post a patch to actualy copy the modified LocalSettings.php to the artifacts... because currently the one in the artifacts is copied before the append [18:30:17] Ah, mw-apply-wb-settings.sh [18:30:31] <^d> twentyafterfour: And if it's not, some developers need whacking with a cluebat :) [18:30:41] jzerebecki: Depends. mw-apply-wb-settings.sh is arguably running too late [18:30:50] That doesn't exist in teh standard job [18:30:54] it's definitely customised in some way [18:31:29] so we can modify the deployed code layout as long as we update resourceloader to be aware of it? [18:31:38] yea the job is put together quite differently because we need to run composer, and the others don't yet [18:31:45] https://github.com/wikimedia/integration-config/blob/98f4cf8028d9fe92add8c26ee9a7651e29aaff3d/jjb/wikidata.yaml#L53-L70 [18:31:50] Aha [18:31:54] so it's not using the qunit macro :) [18:31:59] well, the macro but not the template [18:31:59] it is [18:32:13] twentyafterfour: RL handles CSS/JS but not static assets (images, fonts, ..?). But if we could standardize the location of static assets in skins/extensions or at least require them to be listed in a config file they shouldn't be too hard to move around. [18:32:19] The mwext-{}-qunit [18:32:21] template [18:32:26] <^d> What bd808 said [18:32:30] which makes sense, but I thought it was a standard job [18:32:33] hence looking in the wrong place [18:33:05] jzerebecki: I'll file a task to collect LocalSettings in teardown [18:33:36] Krinkle: and apache error logs? [18:33:42] well I mean, how does it work now with them being in a subdirectory with some 1.2XwmfXX subdirectory [18:33:47] jzerebecki: Not accessible, and not feasible to do so. [18:34:05] jzerebecki: Also not needed, because when configured properly, it will be accessible from the artefacts [18:34:13] images don't use bits? [18:35:14] jzerebecki: and meta debugging of ci jobs themselves is not a use case for what Jenkins jobs can provide in their build. When developping jobs themselves, ci admins have access to the server [18:35:21] (03PS3) 10Dduvall: Run CentralAuth browser tests at en.m.wikipedia.beta.wmflabs.org [integration/config] - 10https://gerrit.wikimedia.org/r/191798 [18:35:51] yea [18:37:34] <^d> twentyafterfour: served via bits but not via ResourceLoader [18:37:42] <^d> (hence the versioned directories) [18:39:02] Krinkle: thank you for finding out [18:39:15] where is the part that controls what urls are generated? like how does en.wikipedia.org know that it needs image tags to use a prefix of 1.25wmf17/ [18:40:56] <^d> https://www.mediawiki.org/wiki/Manual:$wgExtensionAssetsPath is part of it [18:41:03] !log Running `jenkins-jobs update` to create browsertests-CentralAuth-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce [18:41:06] Logged the message, Master [18:41:09] Yippee, build fixed! [18:41:10] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8.1-internet_explorer-11-sauce build #334: FIXED in 34 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-windows_8.1-internet_explorer-11-sauce/334/ [18:41:27] <^d> Then in wmf-config [18:41:28] <^d> $wgExtensionAssetsPath = "//{$wmfHostnames['bits']}/static-$wmfVersionNumber/extensions"; [18:41:30] <^d> etc [18:41:59] ok no problem then [18:42:42] each release tag just gets a corresponding static-$tagname directory [18:43:08] <^d> Wouldn't that lead to even more versioned directories than we have now? [18:43:09] if we wanna be fancy we can prune php files, hard link identical files, etc... [18:43:43] ^d: the versioned directories wouldn't have to be managed manually [18:44:06] $tagname is generated from the state of the repo [18:44:21] jzerebecki: Hm.. mw-apply-wb-settings.sh is not in integration/jenkins.git [18:44:23] <^d> Yeah no I get that. I'm just thinking about the impact on the varnish there [18:44:25] Ah its part of the extension? [18:44:38] Krinkle: yes [18:44:48] ^d: there isn't really any way to solve that problem is there? [18:45:20] <^d> Perhaps not. Just trying to think through all the angles of why we ended up where we are [18:45:24] Don't allow non-backwards compatible static asset changes in a branch [18:45:40] jzerebecki: That makes it a lot easier [18:46:10] jzerebecki: If it was generic (not jenkins specific) then it'd be difficult since we generally don't want to put ?> in files. But then neither would you put bd808: Nice idea. Does that mean anything involving a static asset change would be summarily excluded from swat? [18:46:37] jzerebecki: Right now mw-apply-wb-settings.sh hardcodes echoing Which means it's fine for it to echo ?> after wards [18:46:43] jzerebecki: Can you write a patch for that? [18:46:57] Krinkle: yes, will do [18:47:22] ^d: yes, unless it was shown to be ok fulfilling for prior requests in the same branch [18:47:26] jzerebecki: i'd also recommend removing these https://github.com/wikimedia/mediawiki-extensions-Wikibase/blob/master/build/jenkins/mw-apply-wb-settings.sh#L52-L57 except for LANG maybe [18:47:44] making the green greener or whatever would be ok [18:47:49] They are obsolete (and conflict) with the dev_settings already included by default [18:47:53] as would adding a missing asset [18:48:05] but deleting an asset would be disallowed [18:48:30] <^d> hmmm [18:49:13] It would be simpler to say no changes at all of course [18:49:28] but that will led to a a problem that has to be addressed at some point [18:49:34] *lead [18:49:34] <^d> "never delete anything!" [18:49:52] yeah, well we've done that haven't we [18:50:03] look at the number of branches deployed on any given day [18:50:25] the first time I cleaned up branches it had not been done for something like 3 months [18:50:46] and the dumps server disks were full of stupid dead code [18:55:40] why do old versions need to hang around, anyway? if the php code that generates the links to said code is updated, where would the requests be coming from? [18:57:02] Varnish cache [18:57:41] An given page request by an anon could be served from Varnish for up to 30 days after it was last generated by PHP [18:57:43] which can't be invalidated easily? [19:00:31] I think I could make a script that generates a static-whatever for each tag. prunes php files, then deduplicates identical files and deletes any tag > 30 days old... then the whole process would be optimal and automatic [19:05:52] Invalidation is hard because there is no way to enumerate the Varnish cache. We send purges from the MW backend when a page is edited but we can't get a list of all the pages that are in cache across all of the Varnish instances and there is no way to tell a prior what the contents of those cached assets are. [19:06:08] *a priori [19:06:33] the only way to see what is in cache is to ask for something and see if you get a cache hit response [19:10:30] what if the endpoint for those varnish requests was a script that simply queried git and returned the correct object for the version in question. then we wouldn't need a checkout a t all, just a .git object store [19:11:42] it could even be a simple/fast C program that would simply parse the url into $version/$path and use https://libgit2.github.com/ to resolve the file data. could even be a custom apache module for maximum performance [19:13:18] ^d & bd808: ^ I can't think of any reason not to do this [19:13:24] what am I missing? ;) [19:14:39] I... feel dumb for not thinking of it [19:15:01] something like https://github.com/mbostock/git-static [19:15:24] it would need to be fast and light, but those are just implementation constraints. [19:15:36] Project browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-monobook-sauce build #332: FAILURE in 40 min: https://integration.wikimedia.org/ci/job/browsertests-Flow-en.wikipedia.beta.wmflabs.org-linux-firefox-monobook-sauce/332/ [19:15:49] twentyafterfour: you just trolled me with a link to assetoid ;) [19:15:51] fast and light is why I said apache module... but it might be possible to do it several other ways and remain fast enough [19:16:06] assetoid? [19:16:23] most of the nodejs services are named oid [19:16:35] parsoid started it and then it became a "thing" [19:16:40] lol [19:17:08] * ^d grabs something stabby shaped [19:17:09] https://github.com/googoid/assetoid [19:17:15] parsoid, mathoid, citoid, graphoid all exist today [19:17:20] ok I gotta deploy.. fun [19:37:41] so far I can only think of one thing that will complicate the process of directly serving bare git directories: it'd have to be aware of submodules and recursively process teh extensions .git repos [19:37:53] still quite doable though' [19:40:22] PROBLEM - Puppet failure on deployment-pdf01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [19:41:48] mh it seems integration-slave1006 hangs a few jobs [19:47:27] Krinkle: could you look into that? [19:48:41] Project browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #533: FAILURE in 39 min: https://integration.wikimedia.org/ci/job/browsertests-MobileFrontend-en.m.wikipedia.beta.wmflabs.org-linux-chrome-sauce/533/ [19:52:48] (03CR) 10Dduvall: [C: 032] Run CentralAuth browser tests at en.m.wikipedia.beta.wmflabs.org [integration/config] - 10https://gerrit.wikimedia.org/r/191798 (owner: 10Dduvall) [20:00:03] (03Merged) 10jenkins-bot: Run CentralAuth browser tests at en.m.wikipedia.beta.wmflabs.org [integration/config] - 10https://gerrit.wikimedia.org/r/191798 (owner: 10Dduvall) [20:24:11] Project browsertests-GettingStarted-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce build #359: FAILURE in 44 sec: https://integration.wikimedia.org/ci/job/browsertests-GettingStarted-en.wikipedia.beta.wmflabs.org-linux-firefox-sauce/359/ [20:39:18] uhm.. help? merge failed, operations channel hasn't offered any help: https://gerrit.wikimedia.org/r/#/c/192611/ [20:39:28] is this just a problem with jenkins? [20:41:42] arg jenkins/gerrit are sooooo slow as usual at this time of day [20:43:26] twentyafterfour: no this is not a normal jenkins problem, seems like a real failure [20:44:12] yeah seems like it but I can't see how it possibly had anything to do with my tiny javascript change [20:44:43] so that is an actual bug ask, on #wikimedia-collaboration? (although why Flow tests were running for that commit is kind of weird) [20:44:47] twentyafterfour: ^^ [20:45:15] I need to merge the commit, running out of deployment window [20:46:46] chrismcmahon: is it not supposed to run all tests because its a deploy branch of core? [20:47:00] yeah, probably [20:47:50] but how did it go from passing to failing without a relevant change to any of the code? non-deterministic test? [20:47:58] heisenbug? [20:48:17] perhaps, sent recheck [20:48:18] I suspect a bad test. try 'recheck'? [20:48:39] is that in gerrit or jenkins? [20:49:01] already did so in gerrit [20:50:32] (03CR) 10Jdlrobson: [C: 031] "If I understand correctly this will make" [integration/config] - 10https://gerrit.wikimedia.org/r/191046 (https://phabricator.wikimedia.org/T74794) (owner: 10Hashar) [20:52:14] heisenbug. hmm [20:52:37] I asked in the -collaboration channel. smells like a race condition to me. [20:53:44] so jzerebecki you just reply with "recheck" in the comment to request that zuul resubmit the test? [20:54:13] twentyafterfour: yes [20:54:24] but your email needs to whitelisted for that [20:54:31] to work [20:55:11] Yippee, build fixed! [20:55:11] Project browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce build #487: FIXED in 25 min: https://integration.wikimedia.org/ci/job/browsertests-MultimediaViewer-en.wikipedia.beta.wmflabs.org-linux-chrome-sauce/487/ [20:59:36] all @wikimedia.org emails are automatically whitelisted [21:31:59] chrismcmahon: marxarelli|lunch please make sure https://phabricator.wikimedia.org/T89180 is on the SOS agenda tomorrow [21:32:14] ^d: So… we should probably tweak the name and description of https://phabricator.wikimedia.org/project/profile/1076/ given neither of them now apply. [21:32:43] greg-g: will do [21:32:54] ty [21:33:02] * greg-g just added the "blocked-by-mwcore" project [21:33:10] <^d> James_F: go for it? [21:34:02] ^d: What should it be called? [21:34:59] greg-g: needed to add the Scrum-of-scrums tag to that one, I did it just now [21:35:06] chrismcmahon: gotcha [21:36:22] ^d: Have gone with "Repository-Ownership-Requests" and "Some but not all members of this group can grant the permission." [21:39:17] Also, any idea why the log on https://phabricator.wikimedia.org/project/profile/1076/ doesn't give a "details" link to show my edit to the description? [21:41:53] bug/feature request, probably [21:41:55] :) [21:42:52] (03PS1) 10Gergő Tisza: Add Buggy to CI config [integration/config] - 10https://gerrit.wikimedia.org/r/192699 [21:49:45] (03CR) 10jenkins-bot: [V: 04-1] Add Buggy to CI config [integration/config] - 10https://gerrit.wikimedia.org/r/192699 (owner: 10Gergő Tisza) [21:55:59] greg-g: Ha. :-) [21:58:03] (03PS2) 10Gergő Tisza: Add Buggy to CI config [integration/config] - 10https://gerrit.wikimedia.org/r/192699 [22:04:50] (03CR) 10jenkins-bot: [V: 04-1] Add Buggy to CI config [integration/config] - 10https://gerrit.wikimedia.org/r/192699 (owner: 10Gergő Tisza) [22:06:55] Yippee, build fixed! [22:06:55] Project beta-parsoid-update-eqiad build #747: FIXED in 2.2 sec: https://integration.wikimedia.org/ci/job/beta-parsoid-update-eqiad/747/ [22:07:04] (03CR) 10Legoktm: "Maybe use the normal extension-unittests template but make jslint intentionally non-voting?" [integration/config] - 10https://gerrit.wikimedia.org/r/192699 (owner: 10Gergő Tisza) [22:10:21] twentyafterfour: when you need to force something past Jenkins from gerrit, remove jenkins from the commit (little grey x by it's name), review with +2 verified and +2 code-review and hit "publish and submit" instead of "publish comments" [22:10:32] quite a dance I know [22:11:07] it can be necessary on backports sometimes [22:11:17] (03PS3) 10Gergő Tisza: Add Buggy to CI config [integration/config] - 10https://gerrit.wikimedia.org/r/192699 [22:20:38] bd808: thanks [22:21:08] I'd imagine that knowledge is "on a wiki somehwhere" but I'm not sure where [22:48:47] (03PS1) 10Legoktm: Make utfnormal-composer voting [integration/config] - 10https://gerrit.wikimedia.org/r/192713 (https://phabricator.wikimedia.org/T90278) [22:53:43] (03PS3) 10Dduvall: Ignore HEADLESS and KEEP_BROWSER_OPEN for phantomjs [selenium] - 10https://gerrit.wikimedia.org/r/191552 (owner: 10Adrian Lang) [23:13:37] * chrismcmahon and ryasmeen filed ALL THE BUGS [23:14:02] (03CR) 10Dduvall: [C: 031] "This one was a little tricky to test, but I left some comments for you if you're interested." (036 comments) [selenium] - 10https://gerrit.wikimedia.org/r/191552 (owner: 10Adrian Lang) [23:14:27] where's wikibugs? [23:14:33] legoktm: where's wikibugs? [23:14:47] o.O [23:15:15] legoktm: sorry for bugging you (hah, get it?) you're just who I think of for that bot [23:16:08] it's in -labs [23:16:09] it died when labs was in maintenance? [23:16:10] weird. [23:17:36] it rejoinged labs but not here :( [23:17:49] I didn't ban it, I promise [23:19:14] also not in operations [23:20:42] Or anywhere. [23:20:57] Well. It's in -labs, -dev and -parsoid for some reason. [23:33:32] it's lazy and joins channels only when it needs to send a message (except for -labs which it is always in) [23:36:50] 6Release-Engineering, 10VisualEditor, 7Browser-Tests: Selenium bug with Firefox causes VE test failure - https://phabricator.wikimedia.org/T90651#1064254 (10greg) [23:36:53] taha [23:39:40] greg-g: I am going to have to get used to that tag [23:41:50] ^demon|busy: do you want to attend that Hack talk? It's during our 1:1, but we can move that easily. [23:44:58] 10Deployment-Systems, 6operations: generate dsh nodegroups out of puppet data - https://phabricator.wikimedia.org/T79126#1064294 (10Dzahn) [23:47:54] <^demon|busy> greg-g: hack talk? [23:48:12] ^demon|busy: [Wikitech-l] Tech Talk: Hack - An Evolution of PHP: March 4th [23:48:20] <^demon|busy> oh dur, i have e-mail [23:48:24] :) [23:49:47] <^demon|busy> probably wouldn't hurt to go [23:50:16] thought you might be interested, I'll probably watch it with one eye [23:51:36] 6Release-Engineering, 6operations: Thumbnails error on foreign repo config - https://phabricator.wikimedia.org/T84647#1064367 (10Dzahn) [23:52:11] ^ old but new :p [23:53:53] 6Release-Engineering, 6operations: Thumbnails error on foreign repo config - https://phabricator.wikimedia.org/T84647#1064378 (10greg) [23:56:35] 6Release-Engineering, 6operations: Thumbnails error on foreign repo config - https://phabricator.wikimedia.org/T84647#1064401 (10greg) 5Open>3Resolved a:3greg Those files are producing correct thumbnails for me now (even random sized thumbails I was throwing at it). I hope this was fixed a while ago :) [23:56:42] 6Release-Engineering, 6operations: Thumbnails error on foreign repo config - https://phabricator.wikimedia.org/T84647#1064404 (10greg) a:5greg>3None [23:57:33] :)