[00:01:10] PROBLEM - puppet last run on contint2001 is CRITICAL: CRITICAL: Catalog fetch fail. Either compilation failed or puppetmaster has issues [00:03:25] no_justification: pg bug? [00:03:52] Maybe file a bug? [00:04:26] no_justification: ah [00:04:46] You mean the diff preference dosent show the direct info after changing? [00:04:54] Unless you refresh the page? [00:05:18] who is the op in charge for puppet at this time? [00:05:28] icinga reporting errors all over -operations [00:07:20] no_justification: did you have that problem on gerrit.wikimedia.org? [00:07:21] paladox: No, I mean that if you click into that preference box, and click "Delete" on your keyboard to remove the characters, the whole box disappears [00:07:27] Yes, on gerrit.wm.o [00:08:25] Ah [00:08:43] I think that may be fixed in either 2.15 or master [00:31:11] RECOVERY - puppet last run on contint2001 is OK: OK: Puppet is currently enabled, last run 2 minutes ago with 0 failures [00:49:03] no_justification: https://groups.google.com/forum/m/#!topic/repo-discuss/fQJAQiwFIsY Lol [00:49:40] It is problem with the switch in caches (ie switched from using guva to caffeine in some places) [00:49:59] Heh [00:51:48] no_justification: only took me to find the problem in 2.15, Alice first saw the problem in master. [00:52:22] I wanted to use the recent changes that were merged (I backport we) to fix the look of how disabled buttons look like in pg [00:52:30] (Becky fixed that on master) [01:41:51] 10Beta-Cluster-Infrastructure, 10Community-Tech, 10MediaWiki-extensions-GlobalPreferences, 10Patch-For-Review: Deploy GlobalPreferences on beta cluster - https://phabricator.wikimedia.org/T184668#4008922 (10TBolliger) [01:42:30] PROBLEM - Mediawiki Error Rate on graphite-labs is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [10.0] [01:47:30] RECOVERY - Mediawiki Error Rate on graphite-labs is OK: OK: Less than 1.00% above the threshold [1.0] [01:57:48] PROBLEM - Free space - all mounts on deployment-mediawiki04 is CRITICAL: CRITICAL: deployment-prep.deployment-mediawiki04.diskspace.root.byte_percentfree (<11.11%) [02:19:39] PROBLEM - Free space - all mounts on deployment-fluorine02 is CRITICAL: CRITICAL: deployment-prep.deployment-fluorine02.diskspace._srv.byte_percentfree (<22.22%) [02:43:12] mediawiki04 lacks a /srv/ partition, so it's all on / [02:49:39] RECOVERY - Free space - all mounts on deployment-fluorine02 is OK: OK: All targets OK [04:51:02] 10Gerrit, 10Wikibugs, 10Security: wikibugs IRC bot can see private commits in Gerrit? - https://phabricator.wikimedia.org/T185970#4009139 (10Legoktm) [05:31:09] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T183962#4009169 (10Legoktm) [05:39:10] legoktm: should we rollback from mw.org asap? [05:46:31] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T183962#4009184 (10greg) Reverted back to wmf.22: > 05:43:37 +logmsgbot | !log demon@tin rebuilt and synchronized wikiversions files: (no justification provided) [05:48:47] Rel-engs: As planned CLL migration script is running. See: https://phabricator.wikimedia.org/T187677#4009011 (Just heads up :)) [05:51:13] kart_: that won't be effected by the rollback that just happened, right? [05:51:25] No. [05:51:38] cool :) [05:51:41] I'm running script from wmf.22, so no worries. [05:51:46] * greg-g nods [06:42:10] 10Phabricator: Integrate login with SUL - https://phabricator.wikimedia.org/T179124#4009237 (10Kerry_Raymond) 05declined>03Open Having finally found out how to get back here - the notification I received did not provide any link back here. What further information did you want from me? I did not see a speci... [06:51:34] 10Continuous-Integration-Infrastructure: Test against PHP extensions in 32-bit - https://phabricator.wikimedia.org/T188482#4009257 (10Legoktm) [07:29:36] !log mass queuing jenkins jobs again [07:29:41] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [07:54:21] (03PS1) 10Samwilson: Add TemplateWizard extension [integration/config] - 10https://gerrit.wikimedia.org/r/415237 (https://phabricator.wikimedia.org/T188324) [08:01:28] 10Gerrit, 10Commons, 10MediaWiki-Special-pages, 10Multimedia, and 2 others: Special:Upload prepending "== {{int:filedesc}} ==" twice (Commons) - https://phabricator.wikimedia.org/T187302#4009330 (10Jeff_G) If there is no other section, why do we even need this section header at all? [08:08:02] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Wikidata: Request access to beta cluster for Lucas Werkmeister (Cloud VPS project deployment-prep) - https://phabricator.wikimedia.org/T188427#4007356 (10Addshore) [08:14:07] 10Gerrit, 10Commons, 10MediaWiki-Special-pages, 10Multimedia, and 2 others: Special:Upload prepending "== {{int:filedesc}} ==" twice (Commons) - https://phabricator.wikimedia.org/T187302#3971207 (10Fae) As far as I am aware, there is no community consensus on Wikimedia Commons that every image page should... [08:17:35] (03CR) 10Hashar: [C: 032] Add TemplateWizard extension [integration/config] - 10https://gerrit.wikimedia.org/r/415237 (https://phabricator.wikimedia.org/T188324) (owner: 10Samwilson) [08:19:04] (03Merged) 10jenkins-bot: Add TemplateWizard extension [integration/config] - 10https://gerrit.wikimedia.org/r/415237 (https://phabricator.wikimedia.org/T188324) (owner: 10Samwilson) [08:22:43] (03CR) 10Hashar: "Yup sure, but does phan pass on CentraLogging? :]" [integration/config] - 10https://gerrit.wikimedia.org/r/415099 (owner: 10MarcoAurelio) [08:29:10] RECOVERY - Puppet errors on integration-slave-jessie-1004 is OK: OK: Less than 1.00% above the threshold [0.0] [08:29:58] RECOVERY - Puppet errors on integration-slave-jessie-1001 is OK: OK: Less than 1.00% above the threshold [0.0] [08:42:01] !log queued more jenkins jobs (last for tonight) [08:42:06] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [08:50:09] PROBLEM - Puppet errors on integration-slave-jessie-1004 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [0.0] [08:50:59] PROBLEM - Puppet errors on integration-slave-jessie-1001 is CRITICAL: CRITICAL: 22.22% of data above the critical threshold [0.0] [09:03:03] 10Gerrit, 10Wikibugs, 10Security: wikibugs IRC bot can see private commits in Gerrit? - https://phabricator.wikimedia.org/T185970#3929842 (10Joe) Please note that the user used for pulling the repo to production falls under that category too, and this change just broken the update procedure in production. [09:03:15] 10Gerrit, 10Release-Engineering-Team (Kanban), 10Wikibugs, 10Security: wikibugs IRC bot can see private commits in Gerrit? - https://phabricator.wikimedia.org/T185970#4009430 (10Joe) 05Resolved>03Open [09:03:30] 10Gerrit, 10Release-Engineering-Team (Kanban), 10Wikibugs, 10Security: wikibugs IRC bot can see private commits in Gerrit? - https://phabricator.wikimedia.org/T185970#4009432 (10Joe) p:05Triage>03Normal [09:05:04] 10Gerrit, 10Release-Engineering-Team (Kanban), 10Wikibugs, 10Security: wikibugs IRC bot can see private commits in Gerrit? - https://phabricator.wikimedia.org/T185970#4009434 (10demon) Well then the should be in the existing transparency group or we create a new group and add it to the ACL. Using "Non Inte... [09:11:17] PROBLEM - App Server Main HTTP Response on deployment-mediawiki05 is CRITICAL: CRITICAL - Socket timeout after 10 seconds [09:16:12] RECOVERY - App Server Main HTTP Response on deployment-mediawiki05 is OK: HTTP OK: HTTP/1.1 200 OK - 47158 bytes in 6.026 second response time [09:16:12] 10Continuous-Integration-Infrastructure (shipyard): Castor rsync causes: rsync: failed to set times on "/cache/.": Operation not permitted (1) - https://phabricator.wikimedia.org/T188488#4009453 (10hashar) [09:16:19] 10Continuous-Integration-Infrastructure (shipyard): Castor rsync causes: rsync: failed to set times on "/cache/.": Operation not permitted (1) - https://phabricator.wikimedia.org/T188488#4009463 (10hashar) p:05Triage>03Low [09:37:50] 10Phabricator, 10MediaWiki-extensions-Translate, 10translatewiki.net, 10I18n: Improvements for automatic reporting of tasks from translatewiki to Phabricator - https://phabricator.wikimedia.org/T188379#4009485 (10Aklapper) What "multiplication"? Discussion here mentions creating 1 additional tag, based on... [09:52:28] 10Phabricator, 10MediaWiki-extensions-Translate, 10translatewiki.net, 10I18n: Improvements for automatic reporting of tasks from translatewiki to Phabricator - https://phabricator.wikimedia.org/T188379#4009509 (10Amire80) >>! In T188379#4009485, @Aklapper wrote: > What "multiplication"? Discussion here men... [10:03:37] (03PS1) 10Hashar: docker: npm-browser-test now pass args to /run.sh [integration/config] - 10https://gerrit.wikimedia.org/r/415250 [10:03:39] (03PS1) 10Hashar: docker: polish up npm-test-oojsui [integration/config] - 10https://gerrit.wikimedia.org/r/415251 [10:07:19] (03CR) 10Hashar: [C: 032] docker: npm-browser-test now pass args to /run.sh [integration/config] - 10https://gerrit.wikimedia.org/r/415250 (owner: 10Hashar) [10:11:02] (03PS2) 10Hashar: docker: polish up npm-test-oojsui [integration/config] - 10https://gerrit.wikimedia.org/r/415251 [10:11:12] (03CR) 10Hashar: [C: 032] docker: polish up npm-test-oojsui [integration/config] - 10https://gerrit.wikimedia.org/r/415251 (owner: 10Hashar) [10:12:07] (03Merged) 10jenkins-bot: docker: npm-browser-test now pass args to /run.sh [integration/config] - 10https://gerrit.wikimedia.org/r/415250 (owner: 10Hashar) [10:12:44] (03Merged) 10jenkins-bot: docker: polish up npm-test-oojsui [integration/config] - 10https://gerrit.wikimedia.org/r/415251 (owner: 10Hashar) [10:15:59] 10Phabricator: Integrate login with SUL - https://phabricator.wikimedia.org/T179124#4009546 (10Aklapper) Could you provide a list, step by step, plus expected and actual outcome that allows others to reproduce the problem? My understanding is that this task includes two issues: Something about being sent in circ... [10:37:54] 10Release-Engineering-Team (Kanban), 10MW-1.31-release-notes (WMF-deploy-2018-02-27 (1.31.0-wmf.23)), 10Patch-For-Review, 10User-zeljkofilipin: Update README file for Selenium tests - https://phabricator.wikimedia.org/T187862#4009587 (10zeljkofilipin) [10:38:18] 10Release-Engineering-Team (Kanban), 10MW-1.31-release-notes (WMF-deploy-2018-02-27 (1.31.0-wmf.23)), 10Patch-For-Review, 10User-zeljkofilipin: Q3 Selenium framework improvements - https://phabricator.wikimedia.org/T182421#4009589 (10zeljkofilipin) [10:38:21] 10Release-Engineering-Team (Kanban), 10MW-1.31-release-notes (WMF-deploy-2018-02-27 (1.31.0-wmf.23)), 10Patch-For-Review, 10User-zeljkofilipin: Update README file for Selenium tests - https://phabricator.wikimedia.org/T187862#3988462 (10zeljkofilipin) 05Open>03Resolved [10:40:05] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10Patch-For-Review, 10User-zeljkofilipin, 10Wikimedia-Incident: Create selenium-core-jessie daily Jenkins job - https://phabricator.wikimedia.org/T185011#3903268 (10zeljkofilipin) a:03zeljkofilipin [10:42:39] !log build docker-registry.discovery.wmnet/releng/npm-browser-test:0.1.2 and docker-registry.discovery.wmnet/releng/npm-test-oojsui:0.1.1 [10:42:45] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:49:58] PROBLEM - Host deployment-eventlogging05 is DOWN: CRITICAL - Host Unreachable (10.68.23.51) [10:51:31] !log integration-slave-jessie-android killed stall qemu-system-i386 process [10:51:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:00:35] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Investigate replacing nodemw with mwbot - https://phabricator.wikimedia.org/T181284#4009649 (10zeljkofilipin) @EBernhardson @Jdrewniak um, [[ https://github.com/Fannon/mwbot | mwbot ]] does not have a license?! Isn't that a problem in case we want to f... [11:01:02] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10User-zeljkofilipin: Investigate replacing nodemw with mwbot - https://phabricator.wikimedia.org/T181284#4009651 (10zeljkofilipin) [11:01:15] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10User-zeljkofilipin: Investigate replacing nodemw with mwbot - https://phabricator.wikimedia.org/T181284#3785137 (10zeljkofilipin) a:03zeljkofilipin [11:13:13] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10User-zeljkofilipin: Investigate replacing nodemw with mwbot - https://phabricator.wikimedia.org/T181284#4009703 (10zeljkofilipin) [[ https://github.com/Fannon/mwbot/blob/master/package.json#L61 | package.json ]] says `"license": "MIT"` 😅 but ther... [11:21:04] (03CR) 10MarcoAurelio: "I think it does :) --> https://gerrit.wikimedia.org/r/#/c/414122/" [integration/config] - 10https://gerrit.wikimedia.org/r/415099 (owner: 10MarcoAurelio) [11:21:08] (03PS3) 10MarcoAurelio: Run phan for mediawiki/extensions/CentralLogging [integration/config] - 10https://gerrit.wikimedia.org/r/415099 [11:27:33] PROBLEM - Puppet errors on deployment-eventlog05 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [11:29:45] I am working on --^, plus eventlog02 has almost its disk saturated.. will fix it in ~2h max, need to go now [11:30:24] it was weird to see that the 80G of space reserved for the large instance were almost entirely in a lvm pvs not allocated [11:30:50] Is there a way to make the root 80G and not 20G from horizon's launhcer? [11:30:53] *launcher [11:31:08] otherwise I'll simply add a new partition manually via lvm [11:49:13] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10Patch-For-Review, 10User-zeljkofilipin: Update page object pattern in Selenium tests - https://phabricator.wikimedia.org/T185094#4009794 (10zeljkofilipin) ``` git clone https://github.com/webdriverio/webdriverio.git cd webdriverio npm install n... [12:15:47] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Wikidata, 10User-Ladsgroup: Request access to beta cluster for Lucas Werkmeister (Cloud VPS project deployment-prep) - https://phabricator.wikimedia.org/T188427#4009876 (10Ladsgroup) 05Open>03Resolved a:03Ladsgroup Added [12:27:18] (03PS1) 10Hashar: Migrate oojs npm job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/415270 (https://phabricator.wikimedia.org/T187797) [12:29:17] (03CR) 10Hashar: "The job looks fine, I gave it a try on a dummy change https://gerrit.wikimedia.org/r/#/c/285972/ by commenting "check experimental"." [integration/config] - 10https://gerrit.wikimedia.org/r/415270 (https://phabricator.wikimedia.org/T187797) (owner: 10Hashar) [12:30:16] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q4, 10Patch-For-Review: Migrate leftover Nodepool jobs to Docker - https://phabricator.wikimedia.org/T187797#4009931 (10hashar) [12:50:25] RECOVERY - Free space - all mounts on integration-slave-jessie-1001 is OK: OK: integration.integration-slave-jessie-1001.diskspace._mnt.byte_percentfree (No valid datapoints found) [12:55:04] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Wikidata, 10User-Ladsgroup: Request access to beta cluster for Lucas Werkmeister (Cloud VPS project deployment-prep) - https://phabricator.wikimedia.org/T188427#4010050 (10Lucas_Werkmeister_WMDE) Thanks! [13:01:10] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10Patch-For-Review, 10User-zeljkofilipin: Update page object pattern in Selenium tests - https://phabricator.wikimedia.org/T185094#4010076 (10zeljkofilipin) Looks like it works, but [[ https://babeljs.io/ | Babel ]] is required. https://github.c... [13:04:24] 10Phabricator, 10Release-Engineering-Team (Kanban), 10Phlogiston: Phlogiston reports don't have new data since mid-February - https://phabricator.wikimedia.org/T188149#4010091 (10Aklapper) Where to see the problem? Screenshot says "iOS Backlog by count" but "iOS" link on http://phlogiston.wmflabs.org/ going... [13:07:17] 10Beta-Cluster-Infrastructure, 10Wikimedia-Logstash: logstash-beta.wmflabs.org default dashboard missing - https://phabricator.wikimedia.org/T184602#4010112 (10Addshore) p:05Triage>03Low [13:25:36] Hi releng team [13:25:45] hashar: would you have a minute? [13:26:04] joal: good morning :) [13:26:12] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10Patch-For-Review, 10User-zeljkofilipin: Update page object pattern in Selenium tests - https://phabricator.wikimedia.org/T185094#4010191 (10zeljkofilipin) Done: https://github.com/zeljkofilipin/webdriverio/commit/5c4a8503dc9a9494e329b10d5080926... [13:26:12] or whatever time of day it is really [13:26:14] hashar: afternoon for me ;) [13:26:16] :) [13:26:18] yeah same [13:26:21] I am jetlagged [13:26:24] Arf [13:26:35] how can we help? [13:26:40] hashar: a lot of travel lately? [13:27:10] hashar: analytics-team uses Jenkins to build java/scala code [13:27:30] hashar: I have some patches that regularly fal due to memory issue [13:27:51] ahhh [13:28:00] hashar: The code we develop uses Spark, and tests are kinda super heavy [13:28:01] which repo / jobs? [13:28:10] 10Release-Engineering-Team (Kanban), 10MediaWiki-Core-Tests, 10Patch-For-Review, 10User-zeljkofilipin: Update page object pattern in Selenium tests - https://phabricator.wikimedia.org/T185094#4010195 (10zeljkofilipin) How to test the above code: ``` git clone git@github.com:zeljkofilipin/webdriverio.git c... [13:28:17] hashar: https://integration.wikimedia.org/ci/job/analytics-refinery-maven-java8-docker/74/console [13:29:06] Erik Bernhardson also added a job for search/mjolnir which relies on spark as I understand it [13:29:39] : System memory 468189184 must be at least 471859200 :( [13:29:40] hashar: it had same kind of issue? [13:29:47] maybe it will [13:30:06] hashar: funny thing is, sometimes it succeed - I think it depends on the machine load [13:31:00] yeah so that job runs on instances that have 4GBytes of memory [13:31:07] but there can be two jobs running in parallel [13:32:15] AH NO [13:32:22] joal: yeah the instance only has 2GBytes of memory [13:32:42] https://grafana-labs.wikimedia.org/dashboard/db/labs-project-board?panelId=37&fullscreen&orgId=1&var-project=integration&var-server=integration-slave-docker-1003 [13:32:56] with 500MBytes taken by existing process, there is 1.5Gbytes left [13:33:46] hashar: Weird that it fails to get 500M [13:33:50] hm [13:34:05] I will route it to a slave with 4GB [13:35:40] Many thanks for that hashar [13:35:53] hashar: Let me know when done, I'll push a dummy patch to recheck [13:38:56] joal: working on it :) [13:39:12] addshore: thank you so much for the m4executor Jenkins label :D [13:39:31] :D [13:39:52] your welcome :) [13:40:16] (03PS1) 10Hashar: Tie maven docker jobs to 4GBytes RAM instances [integration/config] - 10https://gerrit.wikimedia.org/r/415280 [13:40:24] joal: job updated [13:40:51] hashar: many thansk, pushing a patch to recheck [13:41:19] joal: it needs deploying first; ) [13:41:27] Ahhh - ok ! [13:41:32] Waiting then :) [13:42:05] hashar: I really need to spend some time on learning our releng processes - I'm such ignorant in that field [13:42:12] I have deployed the jobs :D [13:42:27] oh yeah ignorance is my fault really [13:42:27] Before merging? :P [13:42:39] I never went the extra step to present what CI does :D [13:42:58] Reedy: yeah I typically deploy a job from my local machine, recheck, and if that fail amend locally rinse repease [13:43:03] repeate [13:43:04] lol [13:43:07] then sent to gerrit and +2 :D [13:44:20] joal: so now the job is tied to instances that have 4GB of RAM and can only run a single job [13:44:27] so you should get ~ 3.5GB of RAM [13:44:44] That should be way enough [13:44:58] and if you ever need more memory [13:45:09] I guess you can rewrite refinery in C / Assembly :D [13:45:25] (more seriously, labs has 8GBytes instances) [13:45:26] :D [13:45:47] hashar: I do hope 4gb is enough (it's already a lot for just testing) [13:46:04] if you reach that, we can spawn a 8G instance :d [13:47:45] joal: did a recheck and it is running on https://integration.wikimedia.org/ci/job/analytics-refinery-maven-java8-docker/75/console [13:47:53] the first line saying Building remotely on integration-slave-docker-1005 (DebianJessieDocker m4executor) [13:48:38] This is great hashar :) I'm following the run now [13:48:38] WOOO 8GB :D [13:49:00] addshore: 4G is most probably going to be enough :] [13:49:15] (03CR) 10Bartosz Dziewoński: [C: 031] Migrate oojs npm job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/415270 (https://phabricator.wikimedia.org/T187797) (owner: 10Hashar) [13:52:36] hashar: SUCCESS ! \o/ [13:52:42] Many thanks again :) [13:57:19] joal: yeah it will probably be fine from now on [14:27:13] (03PS1) 10Hashar: Migrate oojs/ui coverage and demos job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/415295 [14:27:44] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q4, 10Patch-For-Review: Migrate leftover Nodepool jobs to Docker - https://phabricator.wikimedia.org/T187797#4010406 (10hashar) [14:32:55] hashar: https://gerrit.wikimedia.org/r/#/c/415099/ --> I replied there, but later rebased the patch, so you might have missed it. [14:35:16] (03CR) 10Jdrewniak: "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/393252 (https://phabricator.wikimedia.org/T180777) (owner: 10Jdrewniak) [14:39:30] (03PS2) 10Jdrewniak: Setting portals JJ to commit to portals/deploy [integration/config] - 10https://gerrit.wikimedia.org/r/393252 (https://phabricator.wikimedia.org/T180777) [14:41:48] RECOVERY - Free space - all mounts on deployment-eventlog02 is OK: OK: deployment-prep.deployment-eventlog02.diskspace._mnt_mysql.byte_percentfree (No valid datapoints found) [14:47:26] (03CR) 10Hashar: [C: 032] Run phan for mediawiki/extensions/CentralLogging [integration/config] - 10https://gerrit.wikimedia.org/r/415099 (owner: 10MarcoAurelio) [14:47:28] Hauskatze: indeed :] [14:48:36] (03Merged) 10jenkins-bot: Run phan for mediawiki/extensions/CentralLogging [integration/config] - 10https://gerrit.wikimedia.org/r/415099 (owner: 10MarcoAurelio) [14:49:25] Hauskatze: deployed!!! [14:49:33] and I am rushing out [14:49:35] Merci bien [15:17:33] RECOVERY - Puppet errors on deployment-eventlog05 is OK: OK: Less than 1.00% above the threshold [0.0] [15:27:44] PROBLEM - Puppet errors on deployment-redis01 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:30:53] zeljkof: You know stuff about our selenium tests, right? [15:31:40] Reedy: I speak selenium like I've lived there! ;) [15:31:46] https://integration.wikimedia.org/ci/job/mediawiki-core-qunit-selenium-jessie/15685/console [15:31:57] I'm guessing https://gerrit.wikimedia.org/r/#/c/414603/ has broken those tests... [15:32:20] Presumably... Because there's a trivially simple password somewhere in the mix [15:32:22] Actually [15:32:24] Easier way to fix this [15:32:49] Reedy: never seen that one before! [15:33:05] Hmm. Hang on. I didn't enable it by default [15:33:39] specs/user.js passed, but specs/page.js failed [15:33:41] PROBLEM - Puppet errors on deployment-redis02 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [15:33:58] specs.page only does browser.url( '/index.php?title=' + path ); [15:34:35] 10Gerrit, 10Release-Engineering-Team (Kanban), 10Wikibugs, 10Security: wikibugs IRC bot can see private commits in Gerrit? - https://phabricator.wikimedia.org/T185970#4010717 (10demon) Added "private-static-site" to the ACL. [15:34:40] 10Gerrit, 10Release-Engineering-Team (Kanban), 10Wikibugs, 10Security: wikibugs IRC bot can see private commits in Gerrit? - https://phabricator.wikimedia.org/T185970#4010718 (10demon) 05Open>03Resolved [15:34:58] and passwords are really simple, if that is something that you are checking, in CI the password is something like wikipass [15:35:13] Yeah, that's what I was wondering [15:36:13] $ grep wiki 10_million_password_list_top_100000.txt [15:36:13] kiwikiwi [15:36:13] wiking [15:36:22] Reedy: it's actually testpass :) https://phabricator.wikimedia.org/source/mediawiki/browse/master/tests/selenium/wdio.conf.jenkins.js;c9e13ac9f8a6a47faddeab957649a654d626db8b$3 [15:36:37] aha [15:36:41] That's definitely in there [15:37:15] you should talk with hashar or grep integration/config to see where it is configured [15:37:24] I'm guessing that means we do an install.php passing that password somewhere [15:37:49] probably [15:38:38] Ok, let me file a bug [15:38:39] Thanks! :D [15:38:54] Reedy: no problemo! :) [15:39:07] 10Beta-Cluster-Infrastructure: Create mediawiki::maintenance server (ala terbium) in deployment-prep - https://phabricator.wikimedia.org/T187826#3987182 (10Dzahn) But don't call it "deployment-terbium", use "maintenance-server" or something generic. The prod host name will change at some point. [15:45:45] 10Continuous-Integration-Config: Use a more complex password for WikiAdmin in selenium tests - https://phabricator.wikimedia.org/T188520#4010763 (10Reedy) [15:48:55] 10Continuous-Integration-Config, 10MediaWiki-Core-Tests: Use a more complex password for WikiAdmin in selenium tests - https://phabricator.wikimedia.org/T188520#4010789 (10Reedy) [15:53:30] 10Continuous-Integration-Config, 10MediaWiki-Core-Tests: Use a more complex password for WikiAdmin in selenium tests - https://phabricator.wikimedia.org/T188520#4010807 (10Reedy) https://github.com/wikimedia/integration-jenkins/blob/ddcc42e3f6e069ba7dcf24ca07122c4e68659db9/bin/mw-install-sqlite.sh#L12 https://... [15:55:37] (03PS1) 10Reedy: Use a more complex password for wikiadmin user [integration/jenkins] - 10https://gerrit.wikimedia.org/r/415312 (https://phabricator.wikimedia.org/T188520) [15:55:58] 10Phabricator (Upstream), 10Upstream: Project tags can not be found via the fulltext search index - https://phabricator.wikimedia.org/T92630#4010811 (10thiemowmde) Three years later I'm still using Google to search Phabricator: * https://www.google.com/search?q=site:phabricator.wikimedia.org+wikidata+arrayobje... [15:57:32] (03CR) 10VolkerE: [C: 031] Migrate oojs npm job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/415270 (https://phabricator.wikimedia.org/T187797) (owner: 10Hashar) [15:58:17] 10Continuous-Integration-Config, 10MediaWiki-Core-Tests, 10Patch-For-Review: Use a more complex password for WikiAdmin in selenium tests - https://phabricator.wikimedia.org/T188520#4010814 (10Reedy) Suggesting we use `testwikijenkinspass` unless we have any other objections/suggestions (I'm as happy to use s... [16:07:42] RECOVERY - Puppet errors on deployment-cache-upload04 is OK: OK: Less than 1.00% above the threshold [0.0] [16:13:42] hasharAway: for fun, how hard/practical do you think https://phabricator.wikimedia.org/T188482 is? [16:17:21] we'd need some 32bit vms [16:20:10] Reedy: https://forums.docker.com/t/support-for-32-bit-images-containers-on-64-bit-hosts/10010 suggests that if you get a 32-bit image it would just work [16:20:26] Yeah, why wouldn't it? lol [16:29:21] 10Continuous-Integration-Infrastructure (shipyard), 10MediaWiki-Platform-Team (MWPT-Q3-Jan-Mar-2018), 10Patch-For-Review: Run MediaWiki tests on PHP 7 - https://phabricator.wikimedia.org/T144962#4010889 (10Legoktm) [16:29:38] (03PS1) 10Ejegg: Run SmashPig tests on PHP5.6 and 7.0 [integration/config] - 10https://gerrit.wikimedia.org/r/415325 [16:31:27] !log manually queued jenkins jobs [16:31:33] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:33:12] hi releng! [16:33:21] fr-tech is leaving PHP5.5 behind [16:33:39] and we'd like to test our payments lib on 5.6 and 7.0 instead [16:33:57] How does this patch look? [16:34:14] https://gerrit.wikimedia.org/r/415325 [16:34:22] 7 should be ok... Not sure about 5.6 [16:34:40] Hmm. Apparently we d [16:34:41] do [16:34:44] nice [16:36:27] Kinda looks like both exist [16:37:50] Yippee, build fixed! [16:37:50] Project selenium-Wikibase-chrome » chrome,beta,Linux,DebianJessie && contintLabsSlave build #125: 09FIXED in 42 min: https://integration.wikimedia.org/ci/job/selenium-Wikibase-chrome/BROWSER=chrome,MEDIAWIKI_ENVIRONMENT=beta,PLATFORM=Linux,label=DebianJessie%20&&%20contintLabsSlave/125/ [16:41:08] ejegg: should be fine, how long do you plan on supporting 5.6 for? [16:42:44] legoktm: looks like we need to get the payments cluster on 7.x sometime this year [16:43:38] will the php5.6 CI images be going away around the same time the security updates run out? [16:50:16] thanks for taking a look! much appreciated when someone gets a chance to +2 / deploy. [16:52:22] no_justification https://gerrit-review.googlesource.com/q/topic:%22Add+plugin+defined+presubmit+rules%22+(status:open%20OR%20status:merged) :) [16:52:40] (they started work on that plugin thingy, so we doint need to use prolog) [16:52:45] PROBLEM - Puppet errors on deployment-cache-text04 is CRITICAL: CRITICAL: 60.00% of data above the critical threshold [0.0] [16:53:18] 10Release-Engineering-Team (Kanban), 10User-zeljkofilipin: Video recording for Selenium tests in Node.js - https://phabricator.wikimedia.org/T179188#4010965 (10greg) p:05Normal>03High [16:53:31] 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10User-zeljkofilipin: Upgrade WebdriverIO to 4.9 - https://phabricator.wikimedia.org/T180144#4010966 (10zeljkofilipin) p:05Low>03High [16:53:34] ejegg: I'm not sure, we currently depend on Debian for security updates, and I think jessie might be EOL at that point. [16:53:45] my question was mostly out of curiosity w/r to php 7.0 migration [16:55:09] ah yeah, I think cwd and Jeff_Green have plans to get php 7 up on paymentswiki sometime in the next few months [16:57:59] no_justification oh, re the bug you reported last night. [16:58:10] when you press the back space bar font size disapears. [16:58:18] is that the behavour your seeing? [17:05:22] legoktm: 32 bits I am not sure :] [17:09:06] 10Continuous-Integration-Infrastructure: Test against PHP extensions in 32-bit - https://phabricator.wikimedia.org/T188482#4010996 (10hashar) Is it doable with a Docker container? I am not sure. Potentially we could use autopkgtest and have the Debian package build on a i386 arch. We had support for that in pa... [17:10:55] 10Continuous-Integration-Infrastructure (shipyard), 10Release-Engineering-Team (Kanban), 10releng-201718-q4, 10Patch-For-Review: Migrate leftover Nodepool jobs to Docker - https://phabricator.wikimedia.org/T187797#4011004 (10greg) [17:12:43] RECOVERY - Puppet errors on deployment-cache-text04 is OK: OK: Less than 1.00% above the threshold [0.0] [17:17:29] 10Continuous-Integration-Infrastructure: Test against PHP extensions in 32-bit - https://phabricator.wikimedia.org/T188482#4011036 (10Legoktm) https://forums.docker.com/t/support-for-32-bit-images-containers-on-64-bit-hosts/10010 suggested that if we had a 32-bit container it would just work. I'm not sure autop... [17:18:42] PROBLEM - Puppet errors on deployment-cache-text04 is CRITICAL: CRITICAL: 20.00% of data above the critical threshold [0.0] [17:24:27] legoktm: I am fine keeping that task around but I am not sure it should be a priority. Mostly because I have no clue how to implement it :] [17:25:51] hashar: yeah, same [17:25:59] I think manual testing will be fine for now [17:26:12] 10Continuous-Integration-Infrastructure: Test against PHP extensions in 32-bit - https://phabricator.wikimedia.org/T188482#4011082 (10Legoktm) p:05Triage>03Lowest [17:26:17] but we can have the debian package build on our infra using an i386 architecture [17:26:30] and if it triggers autopkgtest that would catch the issue [17:27:24] or maybe we can have an i386 debian container [17:28:04] When any +2ers get a chance, fr-tech would love to get these tests changed to PHP5.6. It's what we're running on prod, and mepps has just written some tests that don't seem to work on php5.5. [17:28:08] https://gerrit.wikimedia.org/r/415325 [17:28:59] +2'ing is easy [17:29:01] deploying is hard [17:29:02] xD [17:29:40] scap do it for you [17:29:42] ahhh. I'd be happy to help if anyone feels like granting me privs! [17:29:56] 10Continuous-Integration-Infrastructure: Test against PHP extensions in 32-bit - https://phabricator.wikimedia.org/T188482#4011116 (10hashar) Potentially we could run the autopkgtest on our infra using an i386 architecture. package_builder used to provision such arch: https://gerrit.wikimedia.org/r/#/c/384999/... [17:30:05] scap is easy [17:30:08] it's a jenkins-job-builder thing? [17:30:08] reviewing is hard :D [17:30:30] ah yeah smashpig [17:31:01] one of our funny little snowflake repos [17:31:07] ejegg: no hhvm either? [17:31:30] nah, never used it in the payments cluster [17:31:37] and we're not planning to go hack-only [17:31:41] (03CR) 10Hashar: [C: 032] "It is magic! Well done." [integration/config] - 10https://gerrit.wikimedia.org/r/415325 (owner: 10Ejegg) [17:31:52] TY muchly! [17:32:27] ejegg: I will eventually have to have a close look at how we test DonationInterface :D [17:32:42] hashar: that one's mostly standard [17:32:54] (03Merged) 10jenkins-bot: Run SmashPig tests on PHP5.6 and 7.0 [integration/config] - 10https://gerrit.wikimedia.org/r/415325 (owner: 10Ejegg) [17:32:59] just on the old LTS version of mediawiki [17:33:26] patch deployed [17:33:29] ejegg: hashar in the meanwhile https://gerrit.wikimedia.org/r/#/c/407275/ :P [17:33:41] thanks again hashar, re-running those tests [17:33:42] those REL1_27 tests are breaking the merge [17:33:54] ejegg: I think the DonationInterface has a few glitches here and there. Such as installing smashpig from composer instead of using mediawiki/vendor [17:34:21] hashar: ah yeah, we should be using the composer merge plugin [17:34:28] "smashpig" lol [17:35:02] i thought it was an anti-capitalist thing for the longest time [17:35:24] it isn't? [17:35:26] till K4-713 explained it was just about breaking your piggy bank to get the money out [17:35:30] :P [17:38:15] Hauskatze: if that SPDX id isn't urgent, we'll be moving payments-wiki to 1.31 soon after the official LTS branch is cut [17:38:33] (and with it, the DonationInterface tests) [17:38:56] ejegg: not urgent at all, just wanting to clear that backlog :) [17:39:36] ejegg: will you have DonationInterface@master to keep supporting both 1.27 and 1.31 ? [17:40:29] hashar: we don't plan on keeping 1.27 support after we've migrated ourselves off it [17:40:58] sadly, it's still single-installation 'open source' software [17:41:03] :D [17:41:18] whatever you end up doing, loop Tyler and I for the CI front and we can assist [17:41:27] though creating a job for REL1_31 should be straightforward [17:41:28] thanks! [17:41:33] we could even add it now as a non voting job [17:42:00] Looks like LTS release is planned for June [17:43:45] RECOVERY - Puppet errors on deployment-cache-text04 is OK: OK: Less than 1.00% above the threshold [0.0] [17:44:30] ejegg: traditionally after the European hackathon [17:44:56] gotcha [17:53:36] ejegg: I'm curious what the motivation is for LTS on paymentswiki. Is it upgrade cycles/time? Stability of the code base? [17:54:32] I've understood the desire to not follow wmf/* branches, but why not latest stable? [17:54:57] no_justification: yeah, stability and upgrade cycles / time [17:55:29] I'll note that the latest stable is just as stable as LTS. There's no real other backporting going on [17:55:33] the payment processors have a lot of deadlines for API upgrades, product changes, etc [17:56:06] so we're happy to only have to worry about changing our mw version every 2 years [17:56:36] Fair enough. It's no extra work to support for Fundraising if we already do for third parties 😊 [17:56:42] :) [18:00:44] also, saying you only run LTS releases looks good for PCI compliance probably :P [18:39:23] PROBLEM - Puppet errors on deployment-mx is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [18:44:40] PROBLEM - Puppet errors on deployment-imagescaler01 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [0.0] [19:00:52] 10Release-Engineering-Team (Kanban), 10Wiki-Setup (Close): Close chairwiki - https://phabricator.wikimedia.org/T184961#4011434 (10demon) Heya, wonder if you had a chance to look at this? [19:19:21] 10Phabricator (Upstream), 10Upstream: Project tags can not be found via the fulltext search index - https://phabricator.wikimedia.org/T92630#1116730 (10mmodell) This should have been resolved by the elasticsearch upgrade. I'll look into this, that should definitely be found. [19:19:51] 10Phabricator, 10Release-Engineering-Team (Kanban), 10monitoring, 10Browser-Tests, 10User-zeljkofilipin: Develop tests for phabricator search to detect regressions / search quality issues - https://phabricator.wikimedia.org/T182160#4011458 (10mmodell) [19:19:54] 10Phabricator (Upstream), 10Upstream: Project tags can not be found via the fulltext search index - https://phabricator.wikimedia.org/T92630#4011457 (10mmodell) [19:24:39] RECOVERY - Puppet errors on deployment-imagescaler01 is OK: OK: Less than 1.00% above the threshold [0.0] [19:59:02] (03CR) 10Umherirrender: [C: 031] Run phan for Gadgets [integration/config] - 10https://gerrit.wikimedia.org/r/414962 (owner: 10Legoktm) [20:12:49] 10Phabricator, 10Release-Engineering-Team (Kanban), 10Phlogiston: Phlogiston reports don't have new data since mid-February - https://phabricator.wikimedia.org/T188149#4011689 (10JAufrecht) All of the current reports are failing every day because they basically have no data and Phlogiston (or R) doesn't have... [20:30:46] thcipriani: the betacluster seems to be down again. can you do that thing you do? [20:31:57] niedzielski: What part? [20:31:59] https://en.wikipedia.beta.wmflabs.org/wiki/Main_Page WFM [20:32:02] niedzielski: the api? [20:33:28] thcipriani: no i'm getting errors just on https://en.m.wikipedia.beta.wmflabs.org/wiki/Barack_Obama [20:33:34] weirdly i can access the main page [20:34:08] Why is everything redlink? [20:34:16] sorry thcipriani , it seems to be working now :| [20:34:20] i was getting a 503 [20:34:23] hrm, seems working to me aside from the redlink thing [20:34:28] weird :\ [20:34:54] Oh.. Redlinks would make sense [20:34:58] Most articles not imported. duh [20:39:02] PROBLEM - Puppet errors on deployment-mediawiki07 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [0.0] [20:40:23] finally, a little bit more colour on the change list http://gerrit-new.wmflabs.org/r/q/status:open [20:51:05] 10Continuous-Integration-Config, 10MediaWiki-Core-Tests, 10Patch-For-Review: Use a more complex password for WikiAdmin in selenium tests - https://phabricator.wikimedia.org/T188520#4010763 (10hashar) I am confused. Your patch https://gerrit.wikimedia.org/r/#/c/414603/ runs Selenium tests https://integration.... [20:55:25] 10Continuous-Integration-Config, 10MediaWiki-Core-Tests, 10Patch-For-Review: Use a more complex password for WikiAdmin in selenium tests - https://phabricator.wikimedia.org/T188520#4011818 (10Reedy) See the tests on PS5 - https://integration.wikimedia.org/ci/job/mediawiki-core-qunit-selenium-jessie/15499/con... [20:58:08] 10Continuous-Integration-Config, 10MediaWiki-Core-Tests, 10Patch-For-Review: Use a more complex password for WikiAdmin in selenium tests - https://phabricator.wikimedia.org/T188520#4011835 (10hashar) Ah good thanks :] Lets sync all of that with Zeljko and I tomorrow! [21:34:37] paladox: https://gerrit.wikimedia.org/r/plugins/gitiles/All-Projects/+/cbbe2a6d62c1b6f57ce9fe124e8493c771e43a56%5E%21/ [21:34:53] no_justification :) [21:35:07] thanks [21:36:48] no_justification /me wonders what "Gerrit Managers" is? [21:37:56] ah https://gerrit.wikimedia.org/r/#/admin/groups/119,members [21:38:07] should the group own it's self? [21:38:09] I just folded "Project and Group Creators" and "gerrit" into one group [21:38:11] No real reason [21:38:13] To have more [21:38:30] ah [21:39:14] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T183962#4011971 (10thcipriani) [21:45:24] no_justification /me wishes upstream changes library's for the diff screen in pg. [21:45:34] They made it so complicated to switch to something else now. [21:51:51] Project mwext-phpunit-coverage-publish build #1604: 04FAILURE in 41 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/1604/ [21:52:32] Yippee, build fixed! [21:52:32] Project mwext-phpunit-coverage-publish build #1605: 09FIXED in 39 sec: https://integration.wikimedia.org/ci/job/mwext-phpunit-coverage-publish/1605/ [21:54:26] no_justification ah we need to change the group on https://gerrit.wikimedia.org/r/#/admin/projects/operations/software/gerrit/plugins/wikimedia,access [21:55:10] Fixed [21:55:19] thanks [22:03:45] heya Krenair [22:07:12] hi [22:11:13] Hauskatze? [22:11:56] Krenair: oh, didn't wanted nothing, just saying hi to you [22:12:41] oh hey [22:12:41] have been having connection problems today for some reason [22:13:30] I think at one point freenode actually killed the connection to my bouncer [22:16:42] I dunno. Never used a bouncer :) [22:17:03] Hauskatze: ygm :) [22:17:11] ygm? [22:17:36] you've got mail! [22:17:43] OK [22:17:43] classic movie too, https://en.wikipedia.org/wiki/You%27ve_Got_Mail [22:17:47] no_justification / paladox - the new Gerrit UI (PolyGerrit) doesn't have that gear next to the project name allowing you to create a new patch [22:17:52] can we add it? [22:17:55] * Hauskatze checks [22:17:57] Hauskatze that is known [22:18:09] jesus this month I'm like Royal Mail, I have mail elsewhere [22:18:20] Hauskatze inline editing is not supported in gerrit 2.14 [22:18:21] or 2.15 [22:18:22] but is added in 2.16 [22:18:27] /me renames to PostCat [22:18:40] In 2.15 you can create a change [22:18:51] you just carn't edit it, you have to go into the old ui to edit it [22:20:08] legoktm: read, will think about it, but talking about classic movies, all Humphrey Bogart ones will work for me [22:20:16] :) [22:20:22] Casablanca, To have or not to have... [22:21:19] paladox: okay, so I'll stick with the old ui [22:21:24] I like it better tbh [22:21:33] Hauskatze https://docs.google.com/presentation/d/17q-ygGioZi_5DITLyELa8oaOr22e15AHy8cq6XTZ0nY/edit#slide=id.g27f16618ec_0_139 [22:23:24] 10Release-Engineering-Team (Kanban), 10Release, 10Train Deployments: 1.31.0-wmf.23 deployment blockers - https://phabricator.wikimedia.org/T183962#4012117 (10thcipriani) [22:25:48] 10Gerrit, 10Upstream: PolyGerrit has wrong URLs in links in "Your" menu - https://phabricator.wikimedia.org/T187347#4012120 (10Paladox) [22:25:50] 10Gerrit, 10Release-Engineering-Team (Next): Upgrade gerrit from 2.14.6 to 2.14.7 - https://phabricator.wikimedia.org/T186135#4012119 (10Paladox) [22:27:24] no_justification im guessing i can close this T38937 in favour of T188196 ? [22:27:24] T188196: Create a self-service portal for trusted users to easily create new Gerrit repos - https://phabricator.wikimedia.org/T188196 [22:27:24] T38937: Allow for group admins to create project repositories inside a parent project - https://phabricator.wikimedia.org/T38937 [22:31:03] we should probably ask gerrit upstream to allow deleting groups as well [22:31:57] Hauskatze that should be possible soon. [22:32:06] good then [22:32:06] But currently not supported [22:45:15] hey, are there any plans to make (non-voting) php 7 tests (not hhvm but actual php 7.0.x) run for php extensions? [22:46:37] legoktm ^^ [22:47:24] SMalyshev: yes, it should be for most [22:47:55] it should already be there? Doesn't happen for CirrusSearch as far as I can see [22:48:05] is there some config that needs to be enabled? [22:48:19] oh, yeah CirrusSearch is one of the slightly different extensions [22:48:26] I'll try and fix it tomorrow [22:48:33] legoktm: cool, thanks! [22:49:06] the specific task is https://phabricator.wikimedia.org/T144962 [22:50:36] 10Continuous-Integration-Infrastructure (shipyard), 10MediaWiki-Platform-Team (MWPT-Q3-Jan-Mar-2018), 10Patch-For-Review: Run MediaWiki tests on PHP 7 - https://phabricator.wikimedia.org/T144962#4012182 (10Smalyshev) [22:52:22] oh https://gerrit-review.googlesource.com/q/status:open the columns follow you now [22:52:25] heh [23:00:36] paladox: I think we'll end up doing both actually [23:00:46] no_justification both? [23:01:13] There's the plugin that lets you do in-tree project creation [23:01:20] I think that should be allowed [23:01:30] ah [23:01:32] The self-service portal is for /all/ users to create different styles of projects [23:01:37] i see [23:01:48] I will keep it open then :) [23:01:51] So some developer can say "I wanna make a new __extension__" [23:01:54] Or __skin__ [23:01:55] :) [23:01:57] heh [23:03:30] (03Draft1) 10Paladox: Test: DO NOT MERGE [labs/tools/wikibugs2] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/415476 [23:03:33] yay [23:03:34] no_justification ^^ [23:03:44] Awesome [23:03:49] (03Abandoned) 10Paladox: Test: DO NOT MERGE [labs/tools/wikibugs2] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/415476 (owner: 10Paladox) [23:09:46] pg should add support for hooking into gr-router [23:09:51] to create your own page heh [23:10:04] something like /r/awesome heh [23:12:41] snow storm from london is here now. [23:46:38] paladox: Wow. https://gerrit.googlesource.com/gitiles/+/master/java/com/google/gitiles/PathServlet.java#76