[00:49:20] (03CR) 10Jeena Huneidi: [V: 03+2 C: 03+2] "LGTM" [releng/local-charts] - 10https://gerrit.wikimedia.org/r/546990 (owner: 10Zfilipin) [00:54:20] (03PS2) 10Jeena Huneidi: TLDR section in README [releng/local-charts] - 10https://gerrit.wikimedia.org/r/546990 (owner: 10Zfilipin) [00:55:28] (03CR) 10Jeena Huneidi: [V: 03+2 C: 03+2] TLDR section in README [releng/local-charts] - 10https://gerrit.wikimedia.org/r/546990 (owner: 10Zfilipin) [02:20:10] (03CR) 10Jeena Huneidi: [C: 04-1] "You can see what is used to create the the mediawiki/core image that is currently being built by CI here: https://gerrit.wikimedia.org/r/p" [releng/dev-images] - 10https://gerrit.wikimedia.org/r/555750 (owner: 10Kosta Harlan) [05:28:50] (03PS1) 10Zoranzoki21: Add Pppery in CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/556316 [06:25:37] PROBLEM - Puppet staleness on deployment-hadoop-test-1 is CRITICAL: (Service Check Timed Out) [07:06:49] Greetings! I am working on wikidata/query/blazegraph and wikidata/query/rdf projects, but Jenkins does not run CI on my 'recheck' comments. I assume, I have to be added to https://gerrit.wikimedia.org/r/#/admin/groups/995,members Is there a chance it might be done? Should I open a ticket? [07:07:26] gerrit username igorkim78, email igorkim78@gmail.com [07:54:46] (03PS1) 10Awight: Add ParserFunctions dependency for integration testing [integration/config] - 10https://gerrit.wikimedia.org/r/556328 (https://phabricator.wikimedia.org/T240345) [08:08:39] ok we are starting a round of uploads/captions/depicts with 7 seconds between edits instead of the usual 8, but also with replag checks. I don't expect any issues. [08:15:51] (03CR) 10Hashar: [C: 03+2] Add ParserFunctions dependency for integration testing [integration/config] - 10https://gerrit.wikimedia.org/r/556328 (https://phabricator.wikimedia.org/T240345) (owner: 10Awight) [08:16:33] apergos: hi, I am not sure what those captions things are? :] [08:16:40] (03Merged) 10jenkins-bot: Add ParserFunctions dependency for integration testing [integration/config] - 10https://gerrit.wikimedia.org/r/556328 (https://phabricator.wikimedia.org/T240345) (owner: 10Awight) [08:16:42] heh [08:17:17] I guess that is a notice for something targetting production and you just pass us (well me at this time) the info? :) [08:17:23] I've been doing this for weeks: generate a bunch of little icon-sized images containing characters from a font [08:17:39] oh [08:17:47] with different background colors and borders, upload to beta, then add a caption and a depliscts statement, all via script [08:18:01] awight: Cite now has ParserFunctions injected as a dependency in CI [08:18:03] the goal is to end up with about 50k items in commons beta [08:18:14] that can be used for structured data testing of various sorts [08:18:35] when i started out there were a paltry few hundred items with structured data as I understand it [08:19:04] ordinarily I don't even announce here but since I'm shrinking the number of seconds between edits, figured it was worth reporting [08:19:24] ;] [08:19:34] and you can !log it as well [08:19:47] that ends up in https://tools.wmflabs.org/sal/releng [08:19:48] I can but I'm here during the duration of the run [08:19:59] so if there's anything needed a ping is the fastest [08:20:04] definitely [08:20:19] (I also check the channel regularly during the day0 [08:20:20] ) [08:23:41] Greetings! I am working on wikidata/query/blazegraph and wikidata/query/rdf projects, but Jenkins does not run CI on my 'recheck' comments. I assume, I have to be added to https://gerrit.wikimedia.org/r/#/admin/groups/995,members Is there a chance it might be done? Should I open a ticket? gerrit username igorkim78, email igorkim78@gmail.com [08:24:48] hey igor_kim! I wanted to chat with you anyways about structured data testing on beta when you get a chance [08:25:06] Oh, sure. [08:25:08] hashar: can he just be added? he's a contractor on the sdc stuff [08:25:16] (if that's the problem) [08:25:36] Guillaume is ready to approve the access if needed. [08:25:49] igor_kim: is there an irc channel for the sdc stuff where folks hang out? [08:27:00] Not, that I am aware of. probably PM for now? [08:27:23] ok! [08:32:32] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10ContentTranslation, 10Patch-For-Review: debian-glue fails with: jessie no longer support backports - https://phabricator.wikimedia.org/T240175 (10hashar) p:05Triage→03Normal [08:33:32] (03CR) 10Hashar: [C: 03+2] "apertium-apy targets jessie which no more has backports." [integration/config] - 10https://gerrit.wikimedia.org/r/555917 (https://phabricator.wikimedia.org/T240175) (owner: 10Hashar) [08:34:21] (03Merged) 10jenkins-bot: apertium-apy drop backports [integration/config] - 10https://gerrit.wikimedia.org/r/555917 (https://phabricator.wikimedia.org/T240175) (owner: 10Hashar) [08:35:26] igor_kim: hello! The CI system that process events from Gerrit requires your acccount to be added to a whitelist [08:35:34] igor_kim: apergos: i will just do it [08:35:49] ah that's awesome, thank you! [08:37:36] (03PS1) 10Hashar: Whitelist Igor Kim [integration/config] - 10https://gerrit.wikimedia.org/r/556331 [08:38:31] igor_kim: apergos: gehel: Igo would be whitelist via https://gerrit.wikimedia.org/r/#/c/integration/config/+/556331/ [08:38:50] I am just assuming wikidata/query/blazegraph and wikidata/query/rdf are for the structured data project hehe [08:39:20] hashar: please don't identify Structured Data project with WDQS! [08:39:23] :) [08:39:29] data data [08:39:55] I'll send a correction... [08:40:14] ;] [08:40:22] (03CR) 10Zoranzoki21: [C: 03+1] "No problem, LGTM!" [integration/config] - 10https://gerrit.wikimedia.org/r/556331 (owner: 10Hashar) [08:40:34] heh [08:41:15] (03PS2) 10Gehel: Whitelist Igor Kim [integration/config] - 10https://gerrit.wikimedia.org/r/556331 (owner: 10Hashar) [08:41:44] (03CR) 10Hashar: [C: 03+2] "Magic :]" [integration/config] - 10https://gerrit.wikimedia.org/r/556331 (owner: 10Hashar) [08:41:44] * gehel is now reading the backlog [08:42:23] yep, Igor is part of the Search Platform team, so the channel would be #wikimedia-discovery (yes, that's confusing as well) [08:42:33] (03Merged) 10jenkins-bot: Whitelist Igor Kim [integration/config] - 10https://gerrit.wikimedia.org/r/556331 (owner: 10Hashar) [08:42:43] hashar, apergos: thanks! [08:43:03] * gehel was going to ask if this needed a more formal approval from me, but since it's already merged... [08:43:21] it sure is, given that it's about dumps of structured data and how to use them for wdqs :-/ [08:43:22] (03PS2) 10Zoranzoki21: Add Pppery in CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/556316 [08:43:33] like, how is any of that search? :-D [08:44:42] (03CR) 10Hashar: "I will amend it :)" (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/556316 (owner: 10Zoranzoki21) [08:45:29] (03PS3) 10Hashar: Add Pppery in CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/556316 (owner: 10Zoranzoki21) [08:45:43] (03CR) 10Hashar: [C: 03+2] Add Pppery in CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/556316 (owner: 10Zoranzoki21) [08:45:44] there are 2 hard problems: cache invalidation, naming things and off-by-one-errors [08:45:51] Thank you! I assume it needs some time to apply, though? The group has not changed https://gerrit.wikimedia.org/r/#/admin/groups/995,members [08:46:00] little secret: Search Platform team isn't actually about search [08:46:11] igor_kim: you should be able to have CI to trigger by sending a new patchset or comment in Gerrit "recheck" which CI recognizes as an event to run jobs [08:46:36] (03Merged) 10jenkins-bot: Add Pppery in CI whitelist [integration/config] - 10https://gerrit.wikimedia.org/r/556316 (owner: 10Zoranzoki21) [08:46:36] Oh, great. That was what I am looking for [08:47:15] (03CR) 10Hashar: [C: 03+2] "Deployed :)" [integration/config] - 10https://gerrit.wikimedia.org/r/556316 (owner: 10Zoranzoki21) [08:47:21] well i shouldn't talk, it's not like dumps has anything to do with sre either [08:48:03] right, 'search' is 'discovery' isn't it [08:49:35] so gehel I don't see a good irc channel: https://meta.wikimedia.org/wiki/IRC/Channels#General_development_and_technical_discussion [08:50:17] apergos: you're looking for a Structured Data channel? [08:50:21] no [08:50:23] :-D [08:50:58] I'm looking for the 'where do I talk about the wdqs and structured dumps stuff with igor or his colleagues' channel [08:51:45] Igor is actually probably not the right person for this (well, depends on the question) [08:51:53] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10ContentTranslation, 10Patch-For-Review: debian-glue fails with: jessie no longer support backports - https://phabricator.wikimedia.org/T240175 (10hashar) In CI I have replaced the backports job since there is no... [08:52:10] it's his/their questions, not mine :-d [08:52:19] what's the question? [08:52:23] * gehel is missing context here... [08:53:23] lol [08:53:50] * gehel should really try to understand the question before giving stupid answers [08:53:51] it's just discussion of getting mediainfo data out of beta commons and into wdqs [08:53:53] apologies... [08:54:18] so where can we talk about outstanding tasks, issues around it, how scripts run, etc without cluttering up this channel right here [08:54:40] the "getting it into wdqs" part would be igor_kim and dcausse, but the export side should be cormacparle [08:54:58] you're welcome to chat about this in #wikimedia-discovery [08:55:59] as far as I know, the Structured Data team does not have an IRC channel, but cormacparle probably knows better [08:56:07] it's not cormac, it's me :-D [08:56:30] and the channel is all we need... so yeah I'll move the conv there if that's ok [08:56:42] sure, let's continue on the other side [09:14:07] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201912), 10Packaging: debian-glue does not need to run step "debian/rules clean" - https://phabricator.wikimedia.org/T240423 (10hashar) [09:17:55] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201912), 10Packaging: debian-glue does not need to run step "debian/rules clean" - https://phabricator.wikimedia.org/T240423 (10hashar) [09:33:52] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Patch-For-Review: Investigate gerrit session expiration - https://phabricator.wikimedia.org/T222472 (10Aklapper) Not seeing this issue either anymore, proposing to close task as `resolved` if nobody states that... [09:46:07] (03PS1) 10Hashar: debian-glue: do not clean when building [integration/config] - 10https://gerrit.wikimedia.org/r/556339 (https://phabricator.wikimedia.org/T240423) [09:51:03] (03CR) 10Hashar: "deployed :)" [integration/config] - 10https://gerrit.wikimedia.org/r/556339 (https://phabricator.wikimedia.org/T240423) (owner: 10Hashar) [09:54:05] (03CR) 10Hashar: [C: 03+2] debian-glue: do not clean when building [integration/config] - 10https://gerrit.wikimedia.org/r/556339 (https://phabricator.wikimedia.org/T240423) (owner: 10Hashar) [09:54:58] (03Merged) 10jenkins-bot: debian-glue: do not clean when building [integration/config] - 10https://gerrit.wikimedia.org/r/556339 (https://phabricator.wikimedia.org/T240423) (owner: 10Hashar) [09:55:14] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201912), 10Packaging, 10Patch-For-Review: debian-glue does not need to run step "debian/rules clean" - https://phabricator.wikimedia.org/T240423 (10hashar) 05Open→03Resolved a... [09:56:44] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10ContentTranslation: debian-glue fails with: jessie no longer support backports - https://phabricator.wikimedia.org/T240175 (10hashar) The CI jobs no more run `debian/rules clean`. [10:04:34] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO: debian-glue jobs ignored error messages about libeatmydata.so in LD_PRELOAD - https://phabricator.wikimedia.org/T240430 (10hashar) [10:57:33] 10Phabricator, 10Mobile: URLs for "Related Gerrit Patches" are cut off on mobile and not shown - https://phabricator.wikimedia.org/T240407 (10Aklapper) [10:59:14] 10Phabricator, 10Code-Review-Workgroup, 10GerritBot: Enable semantic relationship between code review changesets and maniphest tasks in phabricator (show "Related Gerrit Patches") - https://phabricator.wikimedia.org/T229934 (10Aklapper) [11:04:21] 10Phabricator, 10Mobile: URLs for "Related Gerrit Patches" are cut off on mobile and not shown - https://phabricator.wikimedia.org/T240407 (10Aklapper) It's a table, as it is tabular data. :) Code is in https://phabricator.wikimedia.org/rPHEX7a526af8b46c0b9a0aacd3270a3bc3323ce51a2c I wonder if we could somewh... [11:43:12] 10Release-Engineering-Team (Code Health), 10Code-Stewardship-Reviews, 10MediaWiki-extensions-ShortUrl: Code Stewardship Review: ShortUrl Extension - https://phabricator.wikimedia.org/T187045 (10Aklapper) [11:48:17] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Patch-For-Review: Investigate gerrit session expiration - https://phabricator.wikimedia.org/T222472 (10LarsWirzenius) I last had to log in on Nov 15. I propose that if I'm still logged in on Monday, Dec 16, we c... [12:22:22] (03PS1) 10Jakob: Publish wikibase-components storybook on doc.wikimedia.org [integration/config] - 10https://gerrit.wikimedia.org/r/556359 (https://phabricator.wikimedia.org/T240445) [12:44:55] 10Release-Engineering-Team, 10Core Platform Team, 10Operations, 10Wikimedia-Rdbms: WikiPage::updateCategoryCounts putting heavy load on commonswiki - https://phabricator.wikimedia.org/T240405 (10jcrespo) Please note the tendril results are rolling (the .5 is a relative range that is now meaningless). There... [12:45:43] 10Release-Engineering-Team, 10Core Platform Team, 10Operations, 10Wikimedia-Rdbms: WikiPage::updateCategoryCounts causing replication lag due to long-running writes on commonswiki - https://phabricator.wikimedia.org/T240405 (10jcrespo) [12:48:08] 10Release-Engineering-Team, 10Core Platform Team, 10Operations, 10Wikimedia-Rdbms: WikiPage::updateCategoryCounts causing replication lag due to long-running writes on commonswiki - https://phabricator.wikimedia.org/T240405 (10jcrespo) updating title as load was not really high, in fact there was lower loa... [12:50:53] 10Phabricator, 10WMSE-Wikispeech-Speech-Data-Collector-2019, 10User-Sebastian_Berlin-WMSE, 10Wikispeech-jobrunner (Sprint): Automatically add tasks in some Wikispeech projects to column of another Wikispeech project - https://phabricator.wikimedia.org/T239390 (10Aklapper) Sure! Created H339 [12:55:51] 10Phabricator, 10WMSE-Wikispeech-Speech-Data-Collector-2019, 10User-Sebastian_Berlin-WMSE, 10Wikispeech-jobrunner (Sprint): Automatically add tasks in some Wikispeech projects to column of another Wikispeech project - https://phabricator.wikimedia.org/T239390 (10Sebastian_Berlin-WMSE) Thank you, @Aklapper. [13:19:48] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO, 10Operations, 10SRE-Access-Requests, 10User-greg: Grant "contint-roots" and "releasers-mediawiki" to user brennen - https://phabricator.wikimedia.org/T240382 (10jcrespo) a:03greg Assigning to @greg for approval, as "service owner... [13:42:24] 10Release-Engineering-Team, 10Gerrit-Privilege-Requests, 10Discovery-Search (Current work): Grant merge permission to discovery-team members on https://gerrit.wikimedia.org/r/admin/projects/wikidata/query/deploy - https://phabricator.wikimedia.org/T239389 (10Aklapper) @Gehel, @hashar, @Mathew.onipe: Is there... [13:43:09] 10Release-Engineering-Team, 10Gerrit-Privilege-Requests, 10Discovery-Search (Current work): Grant merge permission to discovery-team members on https://gerrit.wikimedia.org/r/admin/projects/wikidata/query/deploy - https://phabricator.wikimedia.org/T239389 (10Gehel) 05Open→03Resolved a:03Gehel [13:44:32] (03PS1) 10Hashar: debian-glue: force colors in pbuilder [integration/config] - 10https://gerrit.wikimedia.org/r/556374 [13:48:27] (03CR) 10Hashar: [C: 03+2] debian-glue: force colors in pbuilder [integration/config] - 10https://gerrit.wikimedia.org/r/556374 (owner: 10Hashar) [13:49:19] (03Merged) 10jenkins-bot: debian-glue: force colors in pbuilder [integration/config] - 10https://gerrit.wikimedia.org/r/556374 (owner: 10Hashar) [14:34:15] (03CR) 10Jakob: "This change is ready for review." [integration/config] - 10https://gerrit.wikimedia.org/r/556359 (https://phabricator.wikimedia.org/T240445) (owner: 10Jakob) [14:45:53] 10Release-Engineering-Team, 10Core Platform Team, 10Operations, 10Wikimedia-Rdbms: WikiPage::updateCategoryCounts causing replication lag due to long-running writes on commonswiki - https://phabricator.wikimedia.org/T240405 (10Anomie) I think that {8acea5491d} fixes this. That patch is in wmf.10, but wasn'... [14:46:07] 10Release-Engineering-Team, 10Operations, 10Wikimedia-Rdbms, 10Core Platform Team Workboards (Clinic Duty Team): WikiPage::updateCategoryCounts causing replication lag due to long-running writes on commonswiki - https://phabricator.wikimedia.org/T240405 (10Anomie) [15:15:38] 10Release-Engineering-Team, 10Operations, 10Wikimedia-Rdbms, 10Core Platform Team Workboards (Clinic Duty Team): WikiPage::updateCategoryCounts causing replication lag due to long-running writes on commonswiki - https://phabricator.wikimedia.org/T240405 (10jcrespo) @Anomie I can confim it was commonswiki,... [15:16:55] (03CR) 10Addshore: [C: 03+2] Publish wikibase-components storybook on doc.wikimedia.org [integration/config] - 10https://gerrit.wikimedia.org/r/556359 (https://phabricator.wikimedia.org/T240445) (owner: 10Jakob) [15:17:15] * addshore looks to see if that needs a zuul reload / how to do that now [15:17:46] (03Merged) 10jenkins-bot: Publish wikibase-components storybook on doc.wikimedia.org [integration/config] - 10https://gerrit.wikimedia.org/r/556359 (https://phabricator.wikimedia.org/T240445) (owner: 10Jakob) [15:20:28] !log reloaded zuul for https://gerrit.wikimedia.org/r/556359 T240445 (also pulled in 2 more changes?) [15:20:31] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [15:20:31] T240445: Publish storybook on doc.wikimedia.org - https://phabricator.wikimedia.org/T240445 [15:24:33] (03CR) 10WMDE-leszek: Publish wikibase-components storybook on doc.wikimedia.org (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/556359 (https://phabricator.wikimedia.org/T240445) (owner: 10Jakob) [15:26:54] (03PS1) 10Awight: Edit Project Config [extensions/Cite] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/556400 [15:32:14] (03CR) 10Addshore: [C: 03+2] Publish wikibase-components storybook on doc.wikimedia.org (031 comment) [integration/config] - 10https://gerrit.wikimedia.org/r/556359 (https://phabricator.wikimedia.org/T240445) (owner: 10Jakob) [15:37:08] dammit, i got a ping the other day about docker and devn envs [15:37:14] and now i dont remember where or what it said :( [15:56:37] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10ContentTranslation: debian-glue fails with: jessie no longer support backports - https://phabricator.wikimedia.org/T240175 (10akosiaris) I guess https://gerrit.wikimedia.org/r/555917 and >>! In T240175#5731186, @... [15:59:37] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO, 10Operations, 10SRE-Access-Requests, 10User-greg: Grant "contint-roots" and "releasers-mediawiki" to user brennen - https://phabricator.wikimedia.org/T240382 (10brennen) [16:01:59] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO, 10Operations, 10SRE-Access-Requests, 10User-greg: Grant "contint-roots" and "releasers-mediawiki" to user brennen - https://phabricator.wikimedia.org/T240382 (10Jdforrester-WMF) Service owner is @thcipriani, not Greg. [16:05:25] 10Release-Engineering-Team-TODO, 10DiscussionTools, 10Editing-team, 10Wikimedia-Site-requests: Deploy DiscussionTools to Wikimedia production - https://phabricator.wikimedia.org/T240470 (10Jdforrester-WMF) 05Open→03Stalled [16:13:00] 10Release-Engineering-Team-TODO, 10DiscussionTools, 10Editing-team, 10Wikimedia-Site-requests: Deploy DiscussionTools to Wikimedia production - https://phabricator.wikimedia.org/T240470 (10Jdforrester-WMF) [16:19:34] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO, 10Operations, 10SRE-Access-Requests, 10User-greg: Grant "contint-roots" and "releasers-mediawiki" to user brennen - https://phabricator.wikimedia.org/T240382 (10jcrespo) > Service owner is @thcipriani, not Greg. Sorry about that.... [16:19:48] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO, 10Operations, 10SRE-Access-Requests, 10User-greg: Grant "contint-roots" and "releasers-mediawiki" to user brennen - https://phabricator.wikimedia.org/T240382 (10jcrespo) [16:28:22] (03CR) 10Jforrester: "What's the plan for this? Generally rights changes for production code (let alone tarball code) should go through a Phabricator task…" [extensions/Cite] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/556400 (owner: 10Awight) [16:33:46] (03CR) 10Jforrester: [C: 04-1] "Probably not worth it as there aren't any node selenium tests yet, and I'm not sure they're going to get written?" [integration/config] - 10https://gerrit.wikimedia.org/r/556253 (owner: 10Jforrester) [16:57:59] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO, 10Operations, 10SRE-Access-Requests: Grant "contint-roots" and "releasers-mediawiki" to user brennen - https://phabricator.wikimedia.org/T240382 (10jcrespo) a:05greg→03thcipriani Assigning to @thcipriani for approval, as "service... [17:05:37] 10Release-Engineering-Team-TODO (201912), 10Code-Review-Workgroup: Define code review metrics - https://phabricator.wikimedia.org/T229510 (10Aklapper) Re to the "Metrics" in T229510#5455245 , I think they are possible but some of the "cuts" are not available in wikimedia.biterg.io: * Submitters of patches: htt... [17:27:48] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO, 10Operations, 10SRE-Access-Requests: Grant "contint-roots" and "releasers-mediawiki" to user brennen - https://phabricator.wikimedia.org/T240382 (10jcrespo) p:05Triage→03Normal [17:47:52] PROBLEM - Parsoid on deployment-parsoid09 is CRITICAL: connect to address 172.16.5.63 and port 8000: Connection refused [17:47:52] PROBLEM - Parsoid on deployment-mediawiki-parsoid10 is CRITICAL: connect to address 172.16.0.141 and port 8000: Connection refused [17:56:03] 10Release-Engineering-Team, 10MediaWiki-Core-Testing, 10Readers-Web-Backlog, 10Technical-Debt: Upgrade Sinon.JS - https://phabricator.wikimedia.org/T239271 (10Jdlrobson) [17:56:16] 10Release-Engineering-Team, 10MediaWiki-Core-Testing, 10Readers-Web-Backlog (Tracking), 10Technical-Debt: Upgrade Sinon.JS - https://phabricator.wikimedia.org/T239271 (10Jdlrobson) [17:56:32] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Release-Engineering-Team-TODO, 10MobileFrontend, 10Readers-Web-Backlog (Tracking): Add curl support to mwext-node10-rundoc-docker - https://phabricator.wikimedia.org/T239246 (10Jdlrobson) [17:57:00] btw today's uploads/captions/dispatches batch concluded without incident so I will continue at 7 secs for another week and not bother to announce [18:13:50] 10Release-Engineering-Team-TODO, 10Documentation: Consolidate / write "runbooks" for Release Engineering into our centralised list - https://phabricator.wikimedia.org/T240483 (10Jdforrester-WMF) [18:15:48] 10Release-Engineering-Team-TODO: Explore tools for remote pairing - https://phabricator.wikimedia.org/T240484 (10brennen) [18:33:37] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Release-Engineering-Team-TODO, 10MobileFrontend, 10Readers-Web-Backlog (Tracking): Add curl support to mwext-node10-rundoc-docker - https://phabricator.wikimedia.org/T239246 (10Krinkle) >>! In T239246#5715404, @Jdlrobson wrote: > I dont mind... [18:39:49] (03PS1) 10Krinkle: Add 'curl' to node10-test and bump downstream images [integration/config] - 10https://gerrit.wikimedia.org/r/556444 (https://phabricator.wikimedia.org/T239246) [18:45:54] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team-TODO, 10Operations, 10SRE-Access-Requests: Grant "contint-roots" and "releasers-mediawiki" to user brennen - https://phabricator.wikimedia.org/T240382 (10thcipriani) a:05thcipriani→03jcrespo Approved, thank you @jcrespo ! [18:52:42] (03CR) 10Jforrester: "I still think this is a bad idea, as explained." [integration/config] - 10https://gerrit.wikimedia.org/r/556444 (https://phabricator.wikimedia.org/T239246) (owner: 10Krinkle) [19:05:19] 10Phabricator, 10Security-Team: PoC for security@ email tracking - https://phabricator.wikimedia.org/T239144 (10chasemp) [19:06:07] 10Phabricator, 10Security-Team: PoC for security@ email tracking - https://phabricator.wikimedia.org/T239144 (10chasemp) [19:09:43] 10Phabricator, 10Security-Team: PoC for security@ email tracking - https://phabricator.wikimedia.org/T239144 (10chasemp) [19:09:57] 10Phabricator, 10Security-Team: PoC for security@ email tracking - https://phabricator.wikimedia.org/T239144 (10chasemp) [19:12:06] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Release-Engineering-Team-TODO, 10MobileFrontend, and 2 others: Add curl support to mwext-node10-rundoc-docker - https://phabricator.wikimedia.org/T239246 (10Krinkle) >>! In T239246#5693999, @Jdforrester-WMF wrote: > With the immediate proposal,... [19:12:11] (03CR) 10Krinkle: "I've elaborated on the task, perhaps there was a misunderstanding before. – https://phabricator.wikimedia.org/T239246#5733355" [integration/config] - 10https://gerrit.wikimedia.org/r/556444 (https://phabricator.wikimedia.org/T239246) (owner: 10Krinkle) [19:18:06] 10MediaWiki-Codesniffer, 10Security-team-backlog, 10Security: Security related Static Analysis - https://phabricator.wikimedia.org/T192207 (10chasemp) [19:24:16] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Release-Engineering-Team-TODO, 10MobileFrontend, and 2 others: Add curl support to mwext-node10-rundoc-docker - https://phabricator.wikimedia.org/T239246 (10Jdforrester-WMF) >>! In T239246#5733355, @Krinkle wrote: > Does that addresses your con... [19:50:57] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Release-Engineering-Team-TODO, 10MobileFrontend, and 2 others: Add curl support to mwext-node10-rundoc-docker - https://phabricator.wikimedia.org/T239246 (10Krinkle) >>! In T239246#5733419, @Jdforrester-WMF > https://gerrit.wikimedia.org/r/c/m... [20:04:50] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Release-Engineering-Team-TODO, 10MobileFrontend, and 2 others: Add curl support to mwext-node10-rundoc-docker - https://phabricator.wikimedia.org/T239246 (10Jdlrobson) I'm lost now. From my point of view the answer is simple - you either give m... [20:24:44] (03CR) 10Jforrester: [C: 03+2] Add 'curl' to node10-test and bump downstream images [integration/config] - 10https://gerrit.wikimedia.org/r/556444 (https://phabricator.wikimedia.org/T239246) (owner: 10Krinkle) [20:25:04] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Release-Engineering-Team-TODO, 10MobileFrontend, and 2 others: Add curl support to mwext-node10-rundoc-docker - https://phabricator.wikimedia.org/T239246 (10Jdforrester-WMF) Fine, I'll merge and deploy. [20:25:34] (03Merged) 10jenkins-bot: Add 'curl' to node10-test and bump downstream images [integration/config] - 10https://gerrit.wikimedia.org/r/556444 (https://phabricator.wikimedia.org/T239246) (owner: 10Krinkle) [20:26:39] !log Docker: Pushing node10-test:0.6.0 and friends [20:26:40] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:30:41] (03PS1) 10Jforrester: jjb: Use the new version of node10-test/etc. with curl [integration/config] - 10https://gerrit.wikimedia.org/r/556469 (https://phabricator.wikimedia.org/T239246) [20:41:03] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/556469 (https://phabricator.wikimedia.org/T239246) (owner: 10Jforrester) [20:41:52] (03Merged) 10jenkins-bot: jjb: Use the new version of node10-test/etc. with curl [integration/config] - 10https://gerrit.wikimedia.org/r/556469 (https://phabricator.wikimedia.org/T239246) (owner: 10Jforrester) [20:44:48] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Release-Engineering-Team-TODO (201912), 10MobileFrontend, and 2 others: Add curl support to mwext-node10-rundoc-docker - https://phabricator.wikimedia.org/T239246 (10Jdforrester-WMF) [20:45:02] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Release-Engineering-Team-TODO (201912), 10MobileFrontend, and 2 others: Add curl support to mwext-node10-rundoc-docker - https://phabricator.wikimedia.org/T239246 (10Jdforrester-WMF) 05Open→03Resolved [21:15:09] 10Release-Engineering-Team-TODO, 10Core Platform Team, 10Front-end-Standards-Group, 10Product-Infrastructure-Team-Backlog, 10Readers-Web-Backlog: Should npm packages maintained by Wikimedia be scoped or unscoped? - https://phabricator.wikimedia.org/T239742 (10daniel) [21:19:21] (03CR) 1020after4: [C: 03+1] "I guess the release notes automation will have to be changed to deal with the branch creation..." [tools/release] - 10https://gerrit.wikimedia.org/r/556242 (owner: 10Dduvall) [21:51:25] (03PS2) 10Dduvall: Remove experimental warning from README [blubber] - 10https://gerrit.wikimedia.org/r/549900 [22:13:06] hi greg-g, I'm suddenly getting a gerrit ssh error: sign_and_send_pubkey: signing failed: agent refused operation [22:13:08] andyrussg@gerrit.wikimedia.org: Permission denied (publickey). [22:13:49] brennen: hashar: jrbranaa: liw: longma: thcipriani: twentyafterfour: zeljkof: marxarelli ^ [22:14:24] sorry for the massive ping 8p [22:14:53] AndyRussG: that probably needs someone from sre to look into it? I don't think I have enough privs to be helpful in debugging the issue [22:15:00] likely needs a keyholder rearm [22:15:57] twentyafterfour: thanks! [22:16:37] AndyRussG: maybe your ssh agent lost track of your key somehow? [22:16:43] what does `ssh-add -l` say? [22:19:08] AndyRussG: also, debugging with `ssh -v andyrussg@gerrit.wikimedia.org -p 29418` might give you a better view into where exactly the connection fails [22:21:33] hmm they key does seem to be there [22:21:55] proper username? :) [22:22:16] well, I haven't changed anything... [22:23:41] marxarelli: thanks! I don't see anything that helps me figure this out when I try that, even with -vv [22:23:55] the `ssh -v` output should at least tease out whether your client is offering your key, whether it's a client or server issue [22:24:08] http://gerrit.wikimedia.org/r/accounts/andyrussg shows the username his using is correct. [22:24:14] *he [22:24:27] no changes in user preferences beside the downloadscheme [22:24:40] AndyRussG: is this on the deployment server, or local, or ... ? [22:25:15] just doing my normal git pull form gerrit [22:25:46] on your local dev machine? [22:26:21] Do you have your ssh key under your preference? [22:26:22] https://gerrit.wikimedia.org/r/#/settings/ssh-keys [22:26:55] I would guess the ssh client present the wrong key [22:27:08] so what marxarelli proposed: ssh -v andyrussg@gerrit.wikimedia.org -p 29418 [22:27:14] that should indicates the local key that is proposed [22:27:23] and one can then compare versus the keys stored in Gerrit [22:27:30] yes [22:27:32] sorry I misunderstood the problem (thought it was keyholder problems on deployment server ... my bad) [22:27:47] ssh -i ~/.ssh/foobar_id_rsa <-- to force usage of a specific key when you have severals [22:28:43] hashar: twentyafterfour: marxarelli the key is being offered and it's in gerrit [22:28:48] https://gerrit.wikimedia.org/r/settings/ssh-keys [22:29:18] "signing failed: agent refused operation" seems to me that something is wrong with the ssh agent (which is why I thought keyholder) [22:29:26] and those keys have not changed (I have checked in All-Users.git ) [22:29:38] twentyafterfour: so that would be the local agent? ;) [22:30:09] AndyRussG: check permissions on your keys? it won't sign with a key that has too open of privs [22:30:18] hashar: yes local agent [22:30:43] twentyafterfour: ok hmmm [22:30:45] ssh -o IdentityAgent=none [22:31:02] maybe `ssh-add -l` will tell something useful [22:31:08] that should disable the agent [22:31:26] at least on the server side I don't see anything beside the initial connection [22:31:47] there does not seem to be any rejection from gerrit sshd ( can be checked on gerrit1001 in /var/log/gerrit/sshd_log ) [22:32:32] pretty sure this is a client-side issue [22:33:03] also, is the bare ssh command (not via git) connecting successfully or failing as well? if it's successful, that could mean git's ssh invocation is not talking to the agent properly or in the same way for whatever reason [22:33:16] bare ssh is also failing [22:33:20] sounds like a client issue to me as well, albeit a very frustrating one [22:33:22] ok [22:33:46] you can debug git ssh issues with `GIT_SSH_COMMAND='ssh -vv' git ...` [22:34:44] if bare ssh is also failing, can you share (privately) the output of `ssh -v andyrussg@gerrit.wikimedia.org -p 29418` or similar? maybe a private paste on phab? [22:36:53] twentyafterfour: ok sure! [22:37:48] twentyafterfour: marxarelli: hashar: I have to kinda deal with this later, though... I was able to work around doing stuff via Gerrit ui... I'll post like that and ping a bit later, if that's ok... thanks so much!!!!! [22:47:50] 10Continuous-Integration-Config, 10Release-Engineering-Team, 10Release-Engineering-Team-TODO (201912), 10MobileFrontend, and 2 others: Add curl support to mwext-node10-rundoc-docker - https://phabricator.wikimedia.org/T239246 (10hashar) For reference, locally I have used: `lang=bash export ZUUL_URL=https:/... [22:52:33] I am off ;) [23:38:08] 10Release-Engineering-Team-TODO, 10DiscussionTools, 10Editing-team, 10Wikimedia-Site-requests: Deploy DiscussionTools to Wikimedia production - https://phabricator.wikimedia.org/T240470 (10ppelberg) [23:49:41] 10Release-Engineering-Team, 10Operations, 10Wikimedia-Rdbms, 10Core Platform Team Workboards (Clinic Duty Team): WikiPage::updateCategoryCounts causing replication lag due to long-running writes on commonswiki - https://phabricator.wikimedia.org/T240405 (10Jdforrester-WMF) Production Commons is now running...