[00:15:26] 10Beta-Cluster-Infrastructure: Steward on the Beta Cluster - https://phabricator.wikimedia.org/T183790#3866112 (10Glendales) @Aklapper despite not being urgent, can you or someone else at least acknowledge the existence of this ticket and give information about how to proceed? [00:17:22] 10Beta-Cluster-Infrastructure: Steward on the Beta Cluster - https://phabricator.wikimedia.org/T183790#3863790 (10Legoktm) Most people are on holiday and aren't necessarily checking their bug mail. In any case, I don't think this is a valid use of the beta cluster, which is intended for developers testing code. [00:18:27] 10Beta-Cluster-Infrastructure: Steward on the Beta Cluster - https://phabricator.wikimedia.org/T183790#3866115 (10Glendales) @Legoktm understood. However, if that is the case, can you please point me to some other location where testing of global account tools is available (either hosted by WMF or not, it doesn’... [00:28:15] 10MediaWiki-Codesniffer: Require indentation of CASE statements in PHP code - https://phabricator.wikimedia.org/T182546#3866124 (10Huji) a:05Huji>03None [00:47:06] 10Beta-Cluster-Infrastructure: Special pages is not working on the beta cluster - https://phabricator.wikimedia.org/T183883#3866127 (10Paladox) [00:52:35] 10Beta-Cluster-Infrastructure: Special pages is not working on the beta cluster - https://phabricator.wikimedia.org/T183883#3866140 (10Paladox) I think this https://github.com/wikimedia/mediawiki-extensions-VisualEditor/commit/df29d96c3ef1b381b65a77df0986b78e2ad8a88f#diff-9b460944e50ea44a950aa5f49cd3b1b0 broke i... [00:54:54] 10Beta-Cluster-Infrastructure, 10VisualEditor: Special pages is not working on the beta cluster - https://phabricator.wikimedia.org/T183883#3866141 (10Paladox) [00:56:13] 10Beta-Cluster-Infrastructure, 10VisualEditor: Special pages is not working on the beta cluster - https://phabricator.wikimedia.org/T183883#3866127 (10Paladox) [00:56:39] 10Beta-Cluster-Infrastructure, 10VisualEditor: Special pages is not working on the beta cluster - https://phabricator.wikimedia.org/T183883#3866146 (10Paladox) p:05Triage>03Unbreak! Triaging as unbreak as it will block the train for tomaror T183883 [01:36:36] 10Beta-Cluster-Infrastructure, 10VisualEditor, 10Patch-For-Review, 10User-Ryasmeen: Special pages is not working on the beta cluster - https://phabricator.wikimedia.org/T183883#3866158 (10Paladox) 05Open>03Resolved a:03Paladox [01:37:43] 10Beta-Cluster-Infrastructure, 10VisualEditor, 10User-Ryasmeen: Special pages is not working on the beta cluster - https://phabricator.wikimedia.org/T183883#3866162 (10Paladox) [08:50:07] 10Continuous-Integration-Config, 10Gerrit, 10Cleanup, 10GitHub-Mirrors, and 6 others: Archive mediawiki/extensions/Collection and others - https://phabricator.wikimedia.org/T183891#3866385 (10hashar) [08:51:03] 10Continuous-Integration-Config, 10Gerrit, 10Cleanup, 10GitHub-Mirrors, and 6 others: Archive mediawiki/extensions/Collection and others - https://phabricator.wikimedia.org/T183891#3866385 (10hashar) [08:52:10] 10Continuous-Integration-Config, 10Gerrit, 10Cleanup, 10GitHub-Mirrors, and 6 others: Archive mediawiki/extensions/Collection and others - https://phabricator.wikimedia.org/T183891#3866385 (10Legoktm) Err, Extension:Collection is definitely still used and deployed. Only the OCG service was sunset. [08:52:20] 10Continuous-Integration-Config, 10Gerrit, 10Cleanup, 10GitHub-Mirrors, and 6 others: Archive mediawiki/extensions/Collection and others - https://phabricator.wikimedia.org/T183891#3866398 (10hashar) [08:53:40] 10Continuous-Integration-Config, 10Gerrit, 10Cleanup, 10GitHub-Mirrors, and 6 others: Archive mediawiki/extensions/Collection and others - https://phabricator.wikimedia.org/T183891#3866385 (10hashar) Argh I copy pasted too many of them. So I guess we keep `Collection` but can archive the `mediawiki/extensi... [08:56:08] 10Continuous-Integration-Config, 10Gerrit, 10Cleanup, 10GitHub-Mirrors, and 6 others: Archive mediawiki/extensions/Collection and others - https://phabricator.wikimedia.org/T183891#3866423 (10Legoktm) I think so :) [08:57:14] 10Continuous-Integration-Config, 10Gerrit, 10Cleanup, 10GitHub-Mirrors, and 6 others: Archive mediawiki/extensions/Collection and others - https://phabricator.wikimedia.org/T183891#3866424 (10MarcoAurelio) [08:57:26] 10Continuous-Integration-Config, 10Gerrit, 10Cleanup, 10GitHub-Mirrors, and 6 others: Archive mediawiki/extensions/Collection and others - https://phabricator.wikimedia.org/T183891#3866385 (10MarcoAurelio) Task description editted. [08:58:52] 10Release-Engineering-Team (Watching / External), 10MediaWiki-Core-Tests, 10User-zeljkofilipin: WebdriverIO should run Chrome headlessly - https://phabricator.wikimedia.org/T167507#3866428 (10hashar) I guess we want to migrate the job mediawiki-core-qunit-selenium-jessie to run in a docker container such as... [09:04:58] 10Continuous-Integration-Infrastructure, 10Wikidata Query UI, 10Jenkins: CI fails but jenkins gives +2 - https://phabricator.wikimedia.org/T183831#3866439 (10hashar) The repository is configured in CI to use the generic npm test that relies on the Docker container wmfreleng/npm-test. That containers lacks we... [09:07:39] 10Continuous-Integration-Infrastructure, 10Wikidata Query UI, 10Jenkins: wikidata/query/gui CI job lacks PhantomJS / proper browsers - https://phabricator.wikimedia.org/T183831#3866440 (10hashar) [09:09:18] (03PS1) 10Hashar: Point wikidata/query/gui to a npm browser job [integration/config] - 10https://gerrit.wikimedia.org/r/401475 (https://phabricator.wikimedia.org/T183831) [09:12:24] (03CR) 10Hashar: [C: 032] Point wikidata/query/gui to a npm browser job [integration/config] - 10https://gerrit.wikimedia.org/r/401475 (https://phabricator.wikimedia.org/T183831) (owner: 10Hashar) [09:13:33] (03Merged) 10jenkins-bot: Point wikidata/query/gui to a npm browser job [integration/config] - 10https://gerrit.wikimedia.org/r/401475 (https://phabricator.wikimedia.org/T183831) (owner: 10Hashar) [09:25:06] 10Continuous-Integration-Infrastructure, 10Wikidata Query UI, 10Jenkins, 10Patch-For-Review: wikidata/query/gui CI job lacks PhantomJS / proper browsers - https://phabricator.wikimedia.org/T183831#3866475 (10hashar) Gave it a try on a dummy change https://gerrit.wikimedia.org/r/#/c/401476/ the [[ https://i... [12:08:28] 10Scap: Scap MediaWiki canaries should prompt to continue - https://phabricator.wikimedia.org/T173146#3519703 (10jcrespo) I suffered a similar issue: ``` WARNING: Check 'Logstash Error rate for mw1276.eqiad.wmnet' failed: ERROR: 50% OVER_THRESHOLD (Avg. Error rate: Before: 0.03, After: 2.00, Threshold: 1.00) ``... [12:14:05] 10Gerrit, 10Release-Engineering-Team (Kanban), 10Patch-For-Review, 10Zuul: Update zuul to upstream master - https://phabricator.wikimedia.org/T158243#3866887 (10Paladox) We will want to drop the precise branch too. :) [12:20:37] hashar: hey, happy new year, can we pick this up? https://phabricator.wikimedia.org/T166672 [12:20:45] We need to push the whole history [12:22:51] Amir1 you could request it here https://www.mediawiki.org/wiki/Gerrit/New_repositories/Requests [12:24:57] or https://phabricator.wikimedia.org/project/view/85/ [12:29:40] 10Beta-Cluster-Infrastructure: Steward on the Beta Cluster - https://phabricator.wikimedia.org/T183790#3863790 (10Krenair) I don't think one is provided, most probably just set up their own MW installs [12:31:21] paladox: it's different because it needs history merge AFAIK [12:31:35] Oh, history merge? [12:34:45] Is it on purpose that there is no morning swat (but EU + evening) today? [12:35:36] moving history I meant [12:43:04] ok [12:56:34] 10Continuous-Integration-Infrastructure, 10Wikidata Query UI, 10Jenkins, 10Patch-For-Review: wikidata/query/gui CI job lacks PhantomJS / proper browsers - https://phabricator.wikimedia.org/T183831#3864785 (10Lucas_Werkmeister_WMDE) (I’ve had the “problem parsing d=…” error locally for months, probably unre... [13:31:54] 10Beta-Cluster-Infrastructure: Request for steward rights on the Beta Cluster - https://phabricator.wikimedia.org/T183790#3867012 (10Aklapper) [14:22:56] 10Continuous-Integration-Infrastructure, 10Wikidata Query UI, 10Jenkins, 10Patch-For-Review: wikidata/query/gui CI job lacks PhantomJS / proper browsers - https://phabricator.wikimedia.org/T183831#3867121 (10hashar) I would guess that grunt-contrib-qunit does not handle the error? I haven't looked at the m... [15:34:59] 10Gerrit, 10Developer-Relations (Jan-Mar-2018), 10Documentation: [[mw:Gerrit/Tutorial]] is way too much information for new contributors - https://phabricator.wikimedia.org/T161901#3867357 (10Aklapper) p:05Low>03Lowest [15:35:18] 10Gerrit, 10Developer-Relations (Jan-Mar-2018), 10Documentation: [[mw:Gerrit/Tutorial]] is way too much information for new contributors - https://phabricator.wikimedia.org/T161901#3146787 (10Aklapper) p:05Lowest>03Low [16:06:20] (03PS1) 10Hashar: Migrate VisualEditor 'npm doc' job to Docker [integration/config] - 10https://gerrit.wikimedia.org/r/401530 [16:10:37] Oh gerrit has analytics for gerrit-review now https://gerrit-analytics.gerritforge.com/ [16:14:38] (03PS1) 10Hashar: Override npm run-script by passing an argument [integration/config] - 10https://gerrit.wikimedia.org/r/401532 [16:15:11] (03CR) 10Hashar: "Mostly reverted by https://gerrit.wikimedia.org/r/401532 in favor of passing the script as an argument to the docker run command." [integration/config] - 10https://gerrit.wikimedia.org/r/397576 (owner: 10Hashar) [16:19:38] (03CR) 10Hashar: [C: 032] Override npm run-script by passing an argument [integration/config] - 10https://gerrit.wikimedia.org/r/401532 (owner: 10Hashar) [16:22:24] (03Merged) 10jenkins-bot: Override npm run-script by passing an argument [integration/config] - 10https://gerrit.wikimedia.org/r/401532 (owner: 10Hashar) [16:26:03] (03Abandoned) 10Thcipriani: docker-pkg: use `run` to update docker image [integration/config] - 10https://gerrit.wikimedia.org/r/398086 (owner: 10Thcipriani) [16:26:51] (03PS2) 10Hashar: VisualEditor 'npm doc' job on Docker [integration/config] - 10https://gerrit.wikimedia.org/r/401530 [16:27:46] (03CR) 10Hashar: [C: 032] "We had an issue with the 'doc' job actually running 'npm test'. That was due to some parameter not being passed." [integration/config] - 10https://gerrit.wikimedia.org/r/401530 (owner: 10Hashar) [16:29:05] (03Merged) 10jenkins-bot: VisualEditor 'npm doc' job on Docker [integration/config] - 10https://gerrit.wikimedia.org/r/401530 (owner: 10Hashar) [16:34:02] 10Continuous-Integration-Infrastructure, 10Wikidata Query UI, 10Jenkins, 10Patch-For-Review: wikidata/query/gui CI job lacks PhantomJS / proper browsers - https://phabricator.wikimedia.org/T183831#3867633 (10Lucas_Werkmeister_WMDE) Okay, regular test failures are picked up at least… [16:55:02] 10Release-Engineering-Team, 10Wikidata, 10Epic, 10Patch-For-Review, 10User-Addshore: [Epic] Kill the Wikidata build step - https://phabricator.wikimedia.org/T173818#3867744 (10Addshore) [16:55:10] 10Release-Engineering-Team, 10Wikidata, 10Epic, 10Patch-For-Review, 10User-Addshore: [Epic] Kill the Wikidata build step - https://phabricator.wikimedia.org/T173818#3642465 (10Addshore) 05Open>03Resolved [16:55:18] 10Release-Engineering-Team, 10Wikidata, 10Epic, 10Patch-For-Review, 10User-Addshore: [Epic] Kill the Wikidata build step - https://phabricator.wikimedia.org/T173818#3645462 (10Addshore) Wheeee =] [17:03:11] 10Release-Engineering-Team (Kanban), 10MediaWiki-Vagrant, 10Security, 10User-zeljkofilipin: Security vulnerabilities in yard - https://phabricator.wikimedia.org/T183440#3853068 (10Bawolff) >>! In T183440#3856612, @zeljkofilipin wrote: > Not sure why phab/gerrit did not automatically link to [[ https://gerr... [17:03:14] 10Release-Engineering-Team (Kanban), 10MediaWiki-Vagrant, 10Security, 10User-zeljkofilipin: Security vulnerabilities in yard - https://phabricator.wikimedia.org/T183440#3867815 (10Bawolff) [17:16:55] 10Scap: Scap MediaWiki canaries should prompt to continue - https://phabricator.wikimedia.org/T173146#3867897 (10thcipriani) [17:17:02] (03PS3) 10Addshore: DNM Remove Wikidata build from make-wmf-branch script [tools/release] - 10https://gerrit.wikimedia.org/r/394288 (https://phabricator.wikimedia.org/T173818) [17:17:11] (03PS4) 10Addshore: Remove Wikidata build from make-wmf-branch script [tools/release] - 10https://gerrit.wikimedia.org/r/394288 (https://phabricator.wikimedia.org/T173818) [17:17:49] (03CR) 10Chad: [C: 032] Remove Wikidata build from make-wmf-branch script [tools/release] - 10https://gerrit.wikimedia.org/r/394288 (https://phabricator.wikimedia.org/T173818) (owner: 10Addshore) [17:19:06] 10Scap, 10MediaWiki-Platform-Team (MWPT-Q3-Jan-Mar-2018), 10Patch-For-Review: 'sql' script doesn't support multi-instance hosts - https://phabricator.wikimedia.org/T182713#3867906 (10CCicalese_WMF) [17:26:48] (03PS1) 10Thcipriani: Comment typo fixes [integration/config] - 10https://gerrit.wikimedia.org/r/401546 [17:30:23] 10Scap, 10MediaWiki-Platform-Team (MWPT-Q3-Jan-Mar-2018), 10Patch-For-Review: 'sql' script doesn't support multi-instance hosts - https://phabricator.wikimedia.org/T182713#3867968 (10CCicalese_WMF) a:03Anomie [17:51:34] (03CR) 10Thcipriani: [C: 032] Comment typo fixes [integration/config] - 10https://gerrit.wikimedia.org/r/401546 (owner: 10Thcipriani) [17:52:42] (03Merged) 10jenkins-bot: Comment typo fixes [integration/config] - 10https://gerrit.wikimedia.org/r/401546 (owner: 10Thcipriani) [18:08:14] 10RelEng-Archive-FY201718-Q2, 10MediaWiki-Vagrant: Match puppet version in Gemfile and actual box - https://phabricator.wikimedia.org/T183575#3868092 (10greg) [18:08:17] 10RelEng-Archive-FY201718-Q2, 10Project-Admins: Create Phabricator project docker-pkg - https://phabricator.wikimedia.org/T183544#3868093 (10greg) [18:08:19] 10RelEng-Archive-FY201718-Q2, 10MediaWiki-Vagrant, 10Security, 10User-zeljkofilipin: Security vulnerabilities in yard - https://phabricator.wikimedia.org/T183440#3868094 (10greg) [18:08:22] 10RelEng-Archive-FY201718-Q2, 10Scap (Tech Debt Sprint FY201718-Q2): Audit dblists to see if they're all needed - https://phabricator.wikimedia.org/T179972#3868096 (10greg) [18:08:27] 10RelEng-Archive-FY201718-Q2, 10Phabricator, 10Patch-For-Review: Make the phabricator OAuth login UI more prominent - https://phabricator.wikimedia.org/T963#3868097 (10greg) [18:08:31] (sorry, forgot to use @phabricator_maintenance) [18:10:04] 10Release-Engineering-Team (Kanban), 10User-greg: End of Q2 grooming - https://phabricator.wikimedia.org/T183427#3868101 (10greg) >>! In T183427#3855119, @greg wrote: >>>! In T183427#3854918, @greg wrote: >> * Closed, assigned to people in RelEng, not in Q1 archive project, updated after Sep 30, 2017: >> ** ht... [18:10:47] 10Release-Engineering-Team (Kanban), 10User-greg: End of Q2 grooming - https://phabricator.wikimedia.org/T183427#3868107 (10greg) >>! In T183427#3855119, @greg wrote: >>>! In T183427#3854918, @greg wrote: >> * Closed, assigned to people in RelEng, not in -kanban, not in Q1 archive project, updated before July... [18:14:38] (03CR) 10Chad: [C: 032] Remove DataTypes extension from make-wmf-branch script [tools/release] - 10https://gerrit.wikimedia.org/r/394286 (https://phabricator.wikimedia.org/T180455) (owner: 10Addshore) [18:15:20] (03Merged) 10jenkins-bot: Remove DataTypes extension from make-wmf-branch script [tools/release] - 10https://gerrit.wikimedia.org/r/394286 (https://phabricator.wikimedia.org/T180455) (owner: 10Addshore) [18:15:50] (03Merged) 10jenkins-bot: Remove Wikidata build from make-wmf-branch script [tools/release] - 10https://gerrit.wikimedia.org/r/394288 (https://phabricator.wikimedia.org/T173818) (owner: 10Addshore) [18:37:45] (03PS1) 10Chad: Stop branching vendor [tools/release] - 10https://gerrit.wikimedia.org/r/401574 (https://phabricator.wikimedia.org/T172927) [18:38:36] (03PS1) 10Chad: Remove support for mediawiki < 1.27 [tools/release] - 10https://gerrit.wikimedia.org/r/401577 [18:39:51] thcipriani: thoughts on https://phabricator.wikimedia.org/T182266#3855630 ? [18:40:13] * thcipriani reads [18:40:16] * greg-g was going through the UBN!s [18:41:00] oh good. [18:41:21] * thcipriani rebuilds some docker images [18:44:32] (03Abandoned) 10Reedy: We need a vendor repo for older versions with no submodules [tools/release] - 10https://gerrit.wikimedia.org/r/391295 (owner: 10Reedy) [18:48:45] 10Release-Engineering-Team, 10Scap, 10Wikimedia-Incident: Investigate deployment that caused high error-rate but wasn't prevented by Scap - https://phabricator.wikimedia.org/T183952#3868385 (10Krinkle) [18:49:49] 10Release-Engineering-Team, 10Scap, 10Wikimedia-Incident: Investigate deployment that caused high error-rate but wasn't prevented by Scap - https://phabricator.wikimedia.org/T183952#3868395 (10Krinkle) [18:50:08] zeljkof: no_justification: ^ [18:51:00] Bleh [18:52:43] Krinkle https://phabricator.wikimedia.org/T173146 [18:55:29] 10Release-Engineering-Team, 10Scap, 10Wikimedia-Incident: Investigate deployment that caused high error-rate but wasn't prevented by Scap - https://phabricator.wikimedia.org/T183952#3868385 (10Paladox) See T173146 [18:56:22] Related.... thcipriani I just accepted D939 [18:56:22] D939: Log message to explain canary behavior - https://phabricator.wikimedia.org/D939 [18:57:03] no_justification: thanks [19:04:24] 10Release-Engineering-Team, 10Scap, 10Wikimedia-Incident: Investigate deployment that caused high error-rate but wasn't prevented by Scap - https://phabricator.wikimedia.org/T183952#3868385 (10thcipriani) Logstash graph based on the query looks like it should have caught it: https://logstash.wikimedia.org/go... [19:06:20] 10Release-Engineering-Team, 10Operations, 10Ops-Access-Requests: Allow "releasers-mediawiki" sudo rights to manage Jenkins - https://phabricator.wikimedia.org/T183972#3868664 (10demon) [19:08:05] 10Release-Engineering-Team, 10Scap, 10Wikimedia-Incident: Investigate deployment that caused high error-rate but wasn't prevented by Scap - https://phabricator.wikimedia.org/T183952#3868684 (10thcipriani) Well. Scap does seem to have failed along with that error spike: http://tools.wmflabs.org/sal/log/AWC3S... [19:12:52] HI, any is not ok with phabricator. I no see pictuers there [19:13:24] When I click on any of pictuers, I got error with black wmf logo and: Error Our servers are currently under maintenance or experiencing a technical problem. Please try again in a few minutes. See the error message at the bottom of this page for more information. If you report this error to the Wikimedia System Administrators, please include the details below. Request from 109.245.159.208 via cp3010 frontend, Varni [19:13:58] Same is problem and in mozilla and in chrome and in edge and in internet explorer.. [19:14:06] Zoranzoki21: wfm, you can't see eg https://phab.wmfusercontent.org/file/data/fvjs4shqlwzt4f6lojo4/PHID-FILE-7nt7o34nd6cgzw2ipdow/profile ? [19:14:16] Zoranzoki21 do you use wikipedia zero? [19:14:25] no [19:14:31] I have not wikipedia zero [19:14:57] greg-g: I can not see pictuers.. Nothing of pictuers on phabricator [19:15:19] Does your ip match https://github.com/wikimedia/puppet/blob/d8fc61177ba6465fc34f409d486f958c1afa4f26/modules/phabricator/files/apache/phabbanlist.conf ? [19:15:22] Zoranzoki21: didn't you have a similar issue relatively recently? or was that someone else? [19:15:38] greg-g: Yes, I reported it alreadz [19:15:41] *already [19:15:48] task? [19:16:05] he reported it on irc. [19:16:10] I doint think there was a task. [19:16:16] Yes, I reported here.. [19:16:23] I am not on black list [19:16:38] I disabled adblock for all wmf projects [19:18:18] What can be problem? [19:18:38] When I reported, I had internet in oriontelekom.rs [19:18:47] I migrated me on telenor internet and same is [19:19:55] Zoranzoki21: what's the task you reported? [19:20:13] I no reported it on phabricator.. I will do it now [19:20:58] greg-g: Which project to I add? [19:21:10] #phabricator [19:28:12] greg-g: OK, I created task.. https://phabricator.wikimedia.org/T183980 [19:29:14] 10Scap: Scap MediaWiki canaries should prompt to continue - https://phabricator.wikimedia.org/T173146#3868826 (10thcipriani) 05Open>03Resolved [19:29:36] greg-g: Where I can find help about this https://phabricator.wikimedia.org/T183933 [19:30:05] Zoranzoki21 https://blog.wikimedia.org/2013/11/07/wikipedia-zero/ [19:30:15] i think your provider may be using wikipedia zero [19:30:30] I will contact my provider [19:30:36] But, I have not free wikipedia [19:30:59] I have telenor home which have not included free wikipedia [19:31:42] yep, but it seems they use wikipedia zero and i doint see your ip on https://github.com/wikimedia/puppet/blob/d8fc61177ba6465fc34f409d486f958c1afa4f26/modules/phabricator/files/apache/phabbanlist.conf [19:32:00] and we block wikipedia zero traffic in varnish to phab or we block in ^^ [19:32:07] I will contact telenor [19:32:11] now [19:32:14] ok [19:35:24] Zoranzoki21: regarding pip, please use the #wikimedia-cloud channel [19:35:46] Ok.. Telenor Serbia no use wikipedia zero [19:36:03] Orion Telekom no use wikipedia zero too [19:36:14] Zoranzoki21: also, just to be sure, there are things like "/24" in that ip ban list, so you might not find your exact IP, but it might be included due to it being in a blcok [19:36:20] a block of IPs, that is. [19:37:21] When I reported first time, I had internet in orion telekom. There IP address are 178.250.xxx.xxx [19:37:50] But in black list I no see IP addresses with 178.xxx.xxx.xxx [19:40:53] OK, I created task [19:45:06] 10Continuous-Integration-Config, 10RelEng-Archive-FY201718-Q2, 10Release-Engineering-Team (Kanban), 10Composer, 10Patch-For-Review: Composer\Downloader\TransportException in mwext-testextension-hhvm-jessie - https://phabricator.wikimedia.org/T182266#3868897 (10thcipriani) p:05Unbreak!>03Normal It loo... [19:48:18] Is the wiki list for group0 at https://tools.wmflabs.org/versions/ broken? It lists a bunch of wikis that I didn't think/realize were in group0 and that are also listed for group1 [19:48:40] Hmm, no, I'm wrong [19:48:45] Were closed wikis moved to group0 or something? [19:49:42] Yup [19:50:39] yeah, recently [19:50:50] * greg-g is having a hard time defining "recently" right now [19:56:06] 10Release-Engineering-Team (Kanban), 10User-greg: End of Q2 grooming - https://phabricator.wikimedia.org/T183427#3868961 (10greg) 05Open>03Resolved This is done for Q2, open question for @mmodell above, but that's super low priority. [19:58:03] 10Release-Engineering-Team (Kanban), 10User-greg, 10User-zeljkofilipin: Create #wikimedia-releng-feed and move bots there - https://phabricator.wikimedia.org/T181582#3868976 (10greg) The channel is looking a lot better lately, maybe it was mostly the browser tests? [19:58:10] 10Release-Engineering-Team (Kanban), 10User-greg: End of Q2 grooming - https://phabricator.wikimedia.org/T183427#3868977 (10mmodell) @greg: I don't have a strong opinion one way or the other. Closed tasks cease to have much visibility so I don't know if there is much value in organizing them. [19:59:42] 10Release-Engineering-Team (Kanban), 10User-greg: End of Q2 grooming - https://phabricator.wikimedia.org/T183427#3852738 (10greg) Yeah, definitely has little benefit; was mostly me being slightly OCD :) [20:20:09] Is there a way to track who created a column on a workboard? [20:20:23] It seems an unnamed column was created on #performance-team https://phabricator.wikimedia.org/project/board/1212/column/10564/ [20:20:27] But no idea by whom or why [20:23:05] I don't think so. Same issue for the left sidebar links... we got some spam there [20:56:34] 10Continuous-Integration-Config, 10Release-Engineering-Team (Kanban), 10Discovery, 10Wikimedia-Portals, and 2 others: Create a Jenkins Job that builds the portal deployment artifacts in CI - https://phabricator.wikimedia.org/T179694#3869236 (10debt) Sweet, @hashar! I think we'll need to take a look at this... [21:01:05] 10Continuous-Integration-Config, 10MediaWiki-General-or-Unknown, 10Multi-Content-Revisions, 10User-Addshore: mediawiki tests fail in mediawiki code coverage test - https://phabricator.wikimedia.org/T183777#3869263 (10Addshore) [21:21:55] 10Release-Engineering-Team (Kanban), 10Scap: Scap canary has a shifting baseline - https://phabricator.wikimedia.org/T183999#3869380 (10thcipriani) [21:24:01] (03CR) 10Chad: [C: 032] Stop branching vendor [tools/release] - 10https://gerrit.wikimedia.org/r/401574 (https://phabricator.wikimedia.org/T172927) (owner: 10Chad) [21:24:48] (03Merged) 10jenkins-bot: Stop branching vendor [tools/release] - 10https://gerrit.wikimedia.org/r/401574 (https://phabricator.wikimedia.org/T172927) (owner: 10Chad) [21:24:57] 10Release-Engineering-Team, 10Scap, 10Wikimedia-Incident: Investigate deployment that caused high error-rate but wasn't prevented by Scap - https://phabricator.wikimedia.org/T183952#3869400 (10thcipriani) I filed {T183999} to address the main problem I see here, which is that a deployment that spikes the err... [21:25:34] (03CR) 10Chad: [C: 032] Remove support for mediawiki < 1.27 [tools/release] - 10https://gerrit.wikimedia.org/r/401577 (owner: 10Chad) [21:26:49] (03Merged) 10jenkins-bot: Remove support for mediawiki < 1.27 [tools/release] - 10https://gerrit.wikimedia.org/r/401577 (owner: 10Chad) [21:26:57] Krinkle: huh, looks like the creation is logged like renames are? https://phabricator.wikimedia.org/project/board/1212/column/1786/ [21:27:15] greg-g: Ah, naming vs creation. [21:27:24] Initial name is optional, apparently. [21:28:13] aren't* [21:28:16] (for what I said) [21:28:41] Right. [21:28:43] Yeah [21:28:46] odd. [21:29:32] https://phabricator.wikimedia.org/project/board/1212/column/10564/ [21:29:36] Added a name :) [21:30:06] odd, you'd think creation would be a logable event :) [21:31:57] thcipriani: Regarding T183952, are you suggesting the rate didn't increase by 10x during the sync of InitialiseSettings, but rahter by a smaller factor that slowly increased with various deployments? Or is T183999 unrelated? [21:31:57] T183952: Investigate deployment that caused high error-rate but wasn't prevented by Scap - https://phabricator.wikimedia.org/T183952 [21:31:57] T183999: Scap canary has a shifting baseline - https://phabricator.wikimedia.org/T183999 [21:32:13] I'm confused given the quote that said "average error rate on 6/11 canaries increased by 10x " [21:32:42] Krinkle: it went up, and the deployment failed just like it should have [21:32:58] from the looks of it, the patch was then sync'd again [21:33:18] thcipriani: Ah, so you mean the baseline also shifts after an unsuccessful deployment? [21:33:26] and since it didn't fail the second time, since we don't rollback on the canaries [21:33:44] hm.. why did it not rollback? [21:34:01] we don't have a way to rollback mediawiki deployments [21:34:13] since mediawiki still uses rsync [21:34:20] Right [21:34:58] so the error rate stayed high, and when the canary check compared the error rate to the error rate before the 2nd deployment it let the patch be deployed [21:35:09] So the deployer did stop and not continue to rest of prod, during which canaries continued with the broken code, and thus on re-try the attempt was a no-op for canaries that didn't cause a (relative) regression. [21:35:13] Aye, I understand. [21:35:26] yep, hence T183999 [21:36:06] Right, but solving that might be hard, although maybe you have ideas already :) [21:36:50] I was thinking that it's also possible the deployer didn't know it stayed on the canaries. E.g. that basically to do it "right" in the current system, you have to undo the staged change on tin, re-deploy that first, and then try again. instead of re-trying from the same staging. [21:37:12] Also, rollback with rsync would be possible if we have two copies with symlink switching. [21:37:27] I vaguely thought we had adopted that at some point in the last 2 years, but maybe I was dreaming :) [21:37:31] yeah, that's how we do rollback with scap3 [21:37:36] right [21:37:50] ah, okay. Not with git checkout of earlier hash? [21:37:50] and we have an epic for moving mediawiki to that model [21:37:53] but it [21:38:03] 's not on the short list of projects :) [21:38:12] I assume that would be based on /srv/mediawiki becoming a flat git tree? [21:38:29] yep, which we have, but it seems to grow pretty quickly [21:38:41] it's behind a feature flag for production [21:38:45] it's live in beta [21:38:49] hm.. ok [21:39:33] Yeah, I can see how indefinite history of full MW copies might not scale well. [21:39:34] Im not quite sure where to ask, but im running hosts on google cloud and was wondering if its possible for me to use wmf apts as a source (for jessie & ubuntu 16.04)? If so, how does one do this? [21:40:11] anyway, my ideas for the near-term for canaries would be either stashing the baseline error rate somewhere for some period of time, so subsequent deploys over the course of an hour or so all have the same baseline, or using some absolute threshold past which all deployments will require --force [21:40:24] Zppix: The WMF apt repo is public, so yes [21:40:29] Ah i found the answer nevermind [21:42:10] also it's a good idea to tell deployers that this code will stay on canaries and rollback is currently manual. [22:09:11] (03PS1) 10Chad: Remove pre-SemVer support [tools/release] - 10https://gerrit.wikimedia.org/r/401623 [22:09:13] (03PS1) 10Chad: Do composer update after cloning [tools/release] - 10https://gerrit.wikimedia.org/r/401624 [22:17:33] (03PS2) 10Chad: Remove pre-SemVer support [tools/release] - 10https://gerrit.wikimedia.org/r/401623 [22:19:01] (03CR) 10jerkins-bot: [V: 04-1] Do composer update after cloning [tools/release] - 10https://gerrit.wikimedia.org/r/401624 (owner: 10Chad) [22:22:58] (03CR) 10jerkins-bot: [V: 04-1] Remove pre-SemVer support [tools/release] - 10https://gerrit.wikimedia.org/r/401623 (owner: 10Chad) [22:23:31] 10Release-Engineering-Team, 10Scap, 10Wikimedia-Incident: Investigate deployment that caused high error-rate but wasn't prevented by Scap - https://phabricator.wikimedia.org/T183952#3868385 (10mmodell) Maybe the baseline should be 24 hours in the past? [22:25:45] (03CR) 10Pppery: "... and needed for like three other patches as well." [integration/config] - 10https://gerrit.wikimedia.org/r/401407 (owner: 10Florianschmidtwelzow) [22:30:01] (03PS3) 10Chad: Remove pre-SemVer support [tools/release] - 10https://gerrit.wikimedia.org/r/401623 [22:32:27] (03CR) 10Chad: [C: 032] Remove pre-SemVer support [tools/release] - 10https://gerrit.wikimedia.org/r/401623 (owner: 10Chad) [22:33:12] (03Merged) 10jenkins-bot: Remove pre-SemVer support [tools/release] - 10https://gerrit.wikimedia.org/r/401623 (owner: 10Chad) [22:34:09] (03PS2) 10Chad: Do composer update after cloning [tools/release] - 10https://gerrit.wikimedia.org/r/401624 [22:35:01] (03PS1) 10Divadsn: Add BlueSpicePageAccess extension to zuul/layout.yaml [integration/config] - 10https://gerrit.wikimedia.org/r/401627 (https://phabricator.wikimedia.org/T183674) [22:37:30] (03PS1) 10Divadsn: Add BlueSpiceNamespaceCSS extension to zuul/layout.yaml [integration/config] - 10https://gerrit.wikimedia.org/r/401628 (https://phabricator.wikimedia.org/T183674) [22:37:57] (03CR) 10Chad: [C: 032] Do composer update after cloning [tools/release] - 10https://gerrit.wikimedia.org/r/401624 (owner: 10Chad) [22:38:38] (03Merged) 10jenkins-bot: Do composer update after cloning [tools/release] - 10https://gerrit.wikimedia.org/r/401624 (owner: 10Chad) [22:38:53] 10Continuous-Integration-Config, 10MediaWiki-extensions-General, 10Google-Code-in-2017, 10Jenkins, 10Patch-For-Review: Add MediaWiki extensions to zuul/layout.yaml in so Jenkins can run builds - https://phabricator.wikimedia.org/T183674#3869857 (10divadsn) [22:40:55] (03CR) 10Phantom42: [C: 031] Add BlueSpicePageAccess extension to zuul/layout.yaml [integration/config] - 10https://gerrit.wikimedia.org/r/401627 (https://phabricator.wikimedia.org/T183674) (owner: 10Divadsn) [22:41:16] (03CR) 10Phantom42: [C: 031] Add BlueSpiceNamespaceCSS extension to zuul/layout.yaml [integration/config] - 10https://gerrit.wikimedia.org/r/401628 (https://phabricator.wikimedia.org/T183674) (owner: 10Divadsn) [22:48:05] (03PS1) 10Chad: Migrate to pytest, remove some junk old tests [tools/release] - 10https://gerrit.wikimedia.org/r/401630 [23:00:52] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban), 10MediaWiki-Vagrant, 10User-zeljkofilipin: Continuous integration for mediawiki-vagrant - https://phabricator.wikimedia.org/T183456#3869921 (10greg) p:05Normal>03Low I'm going to set this to Low priority, what with minikube/d... [23:01:17] 10Release-Engineering-Team (Kanban), 10MediaWiki-Vagrant, 10Patch-For-Review, 10User-zeljkofilipin: Create a test suite that compiles mediawiki-vagrant puppet manifests - https://phabricator.wikimedia.org/T183570#3869926 (10greg) p:05Normal>03Low I'm going to set this to Low priority, what with minikub... [23:03:17] 10Gerrit, 10Developer-Relations (Jan-Mar-2018), 10Documentation: [[mw:Gerrit/Tutorial]] is way too much information for new contributors - https://phabricator.wikimedia.org/T161901#3869930 (10Aklapper) 05Open>03Resolved https://www.mediawiki.org/w/index.php?title=Gerrit/Tutorial&oldid=2677976 and https:/... [23:05:39] (03PS2) 10Chad: make-release: Actually make snapshot names sensible [tools/release] - 10https://gerrit.wikimedia.org/r/399889 [23:10:31] (03CR) 10Chad: [C: 032] make-release: Actually make snapshot names sensible [tools/release] - 10https://gerrit.wikimedia.org/r/399889 (owner: 10Chad) [23:11:07] (03Merged) 10jenkins-bot: make-release: Actually make snapshot names sensible [tools/release] - 10https://gerrit.wikimedia.org/r/399889 (owner: 10Chad) [23:21:11] 10Gerrit, 10Diffusion: Commits merged in Gerrit should appear near-instantly in Phabricator - https://phabricator.wikimedia.org/T183792#3869985 (10mmodell) It looks like the most straightforward way to do this would be to use the `hooks` plugin for gerrit which provides a `change-merged` hook. ===Relevant Do... [23:21:20] 10Gerrit, 10Diffusion: Commits merged in Gerrit should appear near-instantly in Phabricator - https://phabricator.wikimedia.org/T183792#3869986 (10mmodell) p:05Triage>03Normal [23:22:27] 10Gerrit, 10Diffusion: Commits merged in Gerrit should appear near-instantly in Phabricator - https://phabricator.wikimedia.org/T183792#3869991 (10Paladox) We will need to reinstall the hooks plugin. CC'ing @demon as he removed the plugin. [23:31:53] 10Release-Engineering-Team (Kanban), 10Scap: Scap canary has a shifting baseline - https://phabricator.wikimedia.org/T183999#3869380 (10mmodell) Should we delay the baseline by 24 hours to avoid recent changes from affecting it? [23:38:59] (03CR) 10Chad: [C: 032] Migrate to pytest, remove some junk old tests [tools/release] - 10https://gerrit.wikimedia.org/r/401630 (owner: 10Chad) [23:39:03] (03PS2) 10Divadsn: Add BlueSpiceNamespaceCSS extension to zuul/layout.yaml [integration/config] - 10https://gerrit.wikimedia.org/r/401628 (https://phabricator.wikimedia.org/T183674) [23:39:56] (03Merged) 10jenkins-bot: Migrate to pytest, remove some junk old tests [tools/release] - 10https://gerrit.wikimedia.org/r/401630 (owner: 10Chad)