[00:05:05] https://www.mediawiki.org/wiki/MediaWiki_1.35/Roadmap - wmf.6 and wmf.7 are missing? [02:36:02] 10Gerrit: Can't change email in Gerrit - https://phabricator.wikimedia.org/T237211 (10Nullzero) [03:15:53] 10Beta-Cluster-Infrastructure, 10Pywikibot: Add beta cluster families to Pywikibot - https://phabricator.wikimedia.org/T237215 (10Tgr) [04:08:32] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10Parsoid-PHP, 10Patch-For-Review: Installing composer modules for deployment - https://phabricator.wikimedia.org/T213494 (10ssastry) [04:37:50] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201909), 10Patch-For-Review: Investigate Docker slowness between 18.06.2 and 18.09.7 - https://phabricator.wikimedia.org/T236675 (10bd808) [05:47:22] 10Beta-Cluster-Infrastructure, 10Pywikibot: Add beta cluster families to Pywikibot - https://phabricator.wikimedia.org/T237215 (10Zoranzoki21) Which wikis should be added? [07:04:48] 10Beta-Cluster-Infrastructure, 10Pywikibot: Add beta cluster families to Pywikibot - https://phabricator.wikimedia.org/T237215 (10Dvorapa) [07:06:10] 10Beta-Cluster-Infrastructure, 10Pywikibot: Add beta cluster families to Pywikibot - https://phabricator.wikimedia.org/T237215 (10Dvorapa) @Zoranzoki21 cca 35 of them, see https://deployment.wikimedia.beta.wmflabs.org/wiki/Special:SiteMatrix [07:49:42] 10Beta-Cluster-Infrastructure, 10Growth-Team, 10User-Ladsgroup: Set up beta cswiki - https://phabricator.wikimedia.org/T237077 (10Tgr) Hm, it looks like [[https://deployment.wikimedia.beta.wmflabs.org/wiki/Special:SiteMatrix|Special:SiteMatrix]] does not recognize the wiki as a Wikipedia. >>! In T237077#56... [08:21:08] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team-TODO, 10Wikimedia-Logstash, 10observability: logstash-beta.wmflabs.org does not receive any mediawiki events - https://phabricator.wikimedia.org/T233134 (10fgiunchedi) re: mediawiki events, searching for `type:mediawiki` does yield results for me (e... [08:29:07] 10Gerrit: Can't change email in Gerrit - https://phabricator.wikimedia.org/T237211 (10Aklapper) @Nullzero: Have you tried changing your email address on https://wikitech.wikimedia.org/wiki/Special:Preferences instead? [09:06:51] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201911), 10ci-test-error: Docker image missing for php-compile-php72-docker - https://phabricator.wikimedia.org/T237207 (10hashar) That has been broken since October 15th due to https://ger... [09:08:29] (03PS1) 10Hashar: Fix unexisting image for php72-compile [integration/config] - 10https://gerrit.wikimedia.org/r/548143 (https://phabricator.wikimedia.org/T237207) [09:08:51] (03CR) 10Hashar: [C: 03+2] "Job updated :)" [integration/config] - 10https://gerrit.wikimedia.org/r/548143 (https://phabricator.wikimedia.org/T237207) (owner: 10Hashar) [09:11:08] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201911), 10Patch-For-Review, 10ci-test-error: Docker image missing for php-compile-php72-docker - https://phabricator.wikimedia.org/T237207 (10hashar) 05Open→03Resolved a:03hashar D... [09:11:34] (03Merged) 10jenkins-bot: Fix unexisting image for php72-compile [integration/config] - 10https://gerrit.wikimedia.org/r/548143 (https://phabricator.wikimedia.org/T237207) (owner: 10Hashar) [09:31:53] 10Release-Engineering-Team, 10Operations, 10Prod-Kubernetes, 10Release Pipeline, 10serviceops: Allow testing of feature-flag-protected features in deployment-charts CI - https://phabricator.wikimedia.org/T236899 (10Joe) 05Open→03Resolved The CI is far from perfect, but it catches the most mundane iss... [09:39:05] (03CR) 10Mvolz: [C: 03+1] Update RESTBase link [integration/docroot] - 10https://gerrit.wikimedia.org/r/547846 (https://phabricator.wikimedia.org/T227766) (owner: 10Ammarpad) [10:06:29] (03PS1) 10Hashar: prometheus-ipsec-exporter make debian glue voting [integration/config] - 10https://gerrit.wikimedia.org/r/548219 [10:12:20] (03PS3) 10Hashar: Drop HHVM support [integration/quibble] - 10https://gerrit.wikimedia.org/r/545968 (https://phabricator.wikimedia.org/T236019) (owner: 10Awight) [10:12:29] (03CR) 10Hashar: "Attached to T236019" [integration/quibble] - 10https://gerrit.wikimedia.org/r/545968 (https://phabricator.wikimedia.org/T236019) (owner: 10Awight) [10:12:51] (03CR) 10Hashar: [C: 04-1] phpunit: Drop --debug-tests command, killed off in PHPUnit 8 [integration/quibble] - 10https://gerrit.wikimedia.org/r/547843 (https://phabricator.wikimedia.org/T192167) (owner: 10Jforrester) [10:13:39] (03PS2) 10Hashar: Chronometer emits folding markers [integration/quibble] - 10https://gerrit.wikimedia.org/r/545972 (https://phabricator.wikimedia.org/T220586) (owner: 10Awight) [10:18:26] (03CR) 10Awight: Chronometer emits folding markers (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/545972 (https://phabricator.wikimedia.org/T220586) (owner: 10Awight) [10:26:35] (03CR) 10Hashar: [C: 03+2] "I guess that is a good first step not sure whether that would ease anything with the Jenkins collapsing plugin though :]" [integration/quibble] - 10https://gerrit.wikimedia.org/r/545972 (https://phabricator.wikimedia.org/T220586) (owner: 10Awight) [10:27:28] (03Merged) 10jenkins-bot: Chronometer emits folding markers [integration/quibble] - 10https://gerrit.wikimedia.org/r/545972 (https://phabricator.wikimedia.org/T220586) (owner: 10Awight) [10:46:29] (03CR) 10Hashar: [C: 04-1] "Good overall, there is some more cleanup to be done in the test suite. Notably php_is_hhvm() is gone so we no more need to mock subprocess" (035 comments) [integration/quibble] - 10https://gerrit.wikimedia.org/r/545968 (https://phabricator.wikimedia.org/T236019) (owner: 10Awight) [10:47:06] 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201911), 10Quibble, 10HHVM, 10Patch-For-Review: Remove HHVM support from Quibble - https://phabricator.wikimedia.org/T236019 (10hashar) a:03awight [10:48:28] (03CR) 10Hashar: "recheck" [integration/quibble] - 10https://gerrit.wikimedia.org/r/546124 (owner: 10Awight) [10:48:38] (03CR) 10jerkins-bot: [V: 04-1] Drop --dry-run parameter [integration/quibble] - 10https://gerrit.wikimedia.org/r/546124 (owner: 10Awight) [10:50:04] (03PS5) 10Hashar: Drop --dry-run parameter [integration/quibble] - 10https://gerrit.wikimedia.org/r/546124 (owner: 10Awight) [10:50:53] (03CR) 10Hashar: [C: 03+1] "Rebased since that conflicted with the change that adds --color ;]" [integration/quibble] - 10https://gerrit.wikimedia.org/r/546124 (owner: 10Awight) [10:51:57] (03PS4) 10Hashar: Move logger to module [integration/quibble] - 10https://gerrit.wikimedia.org/r/545552 (owner: 10Awight) [10:52:37] (03CR) 10Hashar: [C: 03+2] Move logger to module [integration/quibble] - 10https://gerrit.wikimedia.org/r/545552 (owner: 10Awight) [11:11:36] (03CR) 10Hashar: "Gotta test it. It is a large change :-\" (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/545309 (owner: 10Awight) [11:18:10] (03CR) 10Hashar: Move class state into local variables (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/545309 (owner: 10Awight) [11:18:52] (03CR) 10Awight: Move class state into local variables (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/545309 (owner: 10Awight) [11:26:02] (03CR) 10Hashar: Move class state into local variables (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/545309 (owner: 10Awight) [11:26:46] (03CR) 10Awight: Move class state into local variables (031 comment) [integration/quibble] - 10https://gerrit.wikimedia.org/r/545309 (owner: 10Awight) [11:45:39] (03PS4) 10Awight: Drop HHVM support [integration/quibble] - 10https://gerrit.wikimedia.org/r/545968 (https://phabricator.wikimedia.org/T236019) [11:45:48] (03CR) 10Awight: "Thanks for the cleanup suggestions!" (035 comments) [integration/quibble] - 10https://gerrit.wikimedia.org/r/545968 (https://phabricator.wikimedia.org/T236019) (owner: 10Awight) [11:51:23] (03CR) 10Awight: [C: 03+1] "Manual rebase looks correct." [integration/quibble] - 10https://gerrit.wikimedia.org/r/546124 (owner: 10Awight) [12:08:20] (03PS1) 10Awight: Enable test coverage jobs for mw-ext-Cite [integration/config] - 10https://gerrit.wikimedia.org/r/548235 [12:22:14] 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201910), 10local-charts, 10User-zeljkofilipin: Can not log in to mediawiki - https://phabricator.wikimedia.org/T236776 (10zeljkofilipin) 05Open→03Resolved [12:42:05] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+1] "Yes, please. We expect this to be 0%, because there are no PHPUnit tests in this extension yet. It's all tested via citeParserTests.txt, w" [integration/config] - 10https://gerrit.wikimedia.org/r/548235 (owner: 10Awight) [12:51:57] (03CR) 10WMDE-Fisch: [C: 03+1] Enable test coverage jobs for mw-ext-Cite [integration/config] - 10https://gerrit.wikimedia.org/r/548235 (owner: 10Awight) [13:01:40] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+2] Drop --dry-run parameter [integration/quibble] - 10https://gerrit.wikimedia.org/r/546124 (owner: 10Awight) [13:02:54] (03Merged) 10jenkins-bot: Drop --dry-run parameter [integration/quibble] - 10https://gerrit.wikimedia.org/r/546124 (owner: 10Awight) [13:02:56] (03Merged) 10jenkins-bot: Move logger to module [integration/quibble] - 10https://gerrit.wikimedia.org/r/545552 (owner: 10Awight) [13:04:46] (03CR) 10Mobrovac: [C: 04-1] Update RESTBase link (031 comment) [integration/docroot] - 10https://gerrit.wikimedia.org/r/547846 (https://phabricator.wikimedia.org/T227766) (owner: 10Ammarpad) [13:30:55] I'll start a run of bulk additions of depicts statements in about 4.5 hours from now. (I have meetings coming up in about half an hour that run almost straight through until then.) [13:31:33] Also, may I please gripe about how we got 'depicts' to be P180 on both WMCS and prod... but not deployment-prep >_< [13:44:13] 10Gerrit: Can't change email in Gerrit - https://phabricator.wikimedia.org/T237211 (10Paladox) This is totally expected. Gerrit dosen't allow you to do that when the auth type is LDAP. Please follow @Aklapper suggestion. [14:16:38] (03CR) 10Hashar: [C: 03+2] "Great :]" [integration/quibble] - 10https://gerrit.wikimedia.org/r/545968 (https://phabricator.wikimedia.org/T236019) (owner: 10Awight) [14:17:32] (03Merged) 10jenkins-bot: Drop HHVM support [integration/quibble] - 10https://gerrit.wikimedia.org/r/545968 (https://phabricator.wikimedia.org/T236019) (owner: 10Awight) [15:00:52] * awight enters git rebase escape pod [15:02:25] 10Gerrit, 10Gerrit-Privilege-Requests, 10Pywikibot: Give access to l10n-bot-watcher group - https://phabricator.wikimedia.org/T237263 (10abi_) [15:02:56] (03PS8) 10Awight: Move class state into local variables [integration/quibble] - 10https://gerrit.wikimedia.org/r/545309 [15:02:58] (03PS7) 10Awight: Move argument parsing to module scope [integration/quibble] - 10https://gerrit.wikimedia.org/r/546125 [15:03:00] (03PS26) 10Awight: Commands expand recursively [integration/quibble] - 10https://gerrit.wikimedia.org/r/519776 [15:03:02] (03PS3) 10Awight: Cleanup: require all command parameter keywords [integration/quibble] - 10https://gerrit.wikimedia.org/r/546469 [15:03:04] (03PS5) 10Awight: Merge utility functions into calling modules [integration/quibble] - 10https://gerrit.wikimedia.org/r/546434 [15:03:06] (03PS13) 10Awight: Clean up logging [integration/quibble] - 10https://gerrit.wikimedia.org/r/546126 [15:03:08] (03PS9) 10Awight: Replace argparse with docopt [integration/quibble] - 10https://gerrit.wikimedia.org/r/546167 [15:03:10] (03PS10) 10Awight: Prepare to run browser suites in parallel [integration/quibble] - 10https://gerrit.wikimedia.org/r/545661 (https://phabricator.wikimedia.org/T226869) [15:05:09] 10Gerrit, 10Gerrit-Privilege-Requests, 10Pywikibot, 10User-MarcoAurelio: Give access to l10n-bot-watcher group - https://phabricator.wikimedia.org/T237263 (10MarcoAurelio) a:03MarcoAurelio @abi_ Would you like this just for the` pywikibot/i18n.git` or would you prefer to have it for all `pywikibot/.*` re... [15:07:30] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (201907), 10Patch-For-Review: Investigate gerrit session expiration - https://phabricator.wikimedia.org/T222472 (10LarsWirzenius) In the hope it's useful, I'll mention I'm still logged in to Gerrit, since my previo... [15:19:49] 10Beta-Cluster-Infrastructure, 10Growth-Team, 10Notifications, 10CPT Initiatives (Multi-DC Echo Notification Storage), 10Core Platform Team Workboards (Green): Provision deployment-prep instance of echostore - https://phabricator.wikimedia.org/T235920 (10CCicalese_WMF) 05Open→03Resolved [15:35:34] 10Gerrit, 10Gerrit-Privilege-Requests, 10Pywikibot, 10User-MarcoAurelio: Give access to l10n-bot-watcher group - https://phabricator.wikimedia.org/T237263 (10abi_) I think just `pywikibot/i18n` is fine for now. [15:45:46] (03PS1) 10MarcoAurelio: Give access to l10n-bot-watcher group [pywikibot/i18n] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/548295 (https://phabricator.wikimedia.org/T237263) [15:46:12] (03CR) 10MarcoAurelio: [V: 03+2] Give access to l10n-bot-watcher group [pywikibot/i18n] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/548295 (https://phabricator.wikimedia.org/T237263) (owner: 10MarcoAurelio) [15:47:41] (03CR) 10MarcoAurelio: [V: 03+2] "If you approve, please C+2 and Submit (jenkins do not operate on refs/meta/config branches), thanks." [pywikibot/i18n] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/548295 (https://phabricator.wikimedia.org/T237263) (owner: 10MarcoAurelio) [16:04:44] <_joe_> thcipriani: I'm sorry clearly my email wasn't clear, my questions were to all of us, not to releng specifically [16:05:13] <_joe_> I didn't expect you folks to solve the problem of "network security model for CI in the WMF infra" [16:11:51] but surely we can try, if we want to? it's a very intresting problem to think about [16:13:37] _joe_: ah, thanks, wasn't sure if they were unanswered in the proposal or in general [16:14:10] <_joe_> more in general, I was trying to list open questions [16:14:36] <_joe_> thcipriani: my biggest worry with argo is... it's big, and it has a lot of containers we won't rebuild I guess [16:15:08] I was assuming we would rebuild all the container, unless it becomes too much work [16:15:19] <_joe_> so that is something I want to discuss with moritz and john (SRE security) [16:15:30] <_joe_> liw: it's a lot of work [16:15:41] <_joe_> esp to keep up with new versions [16:16:09] <_joe_> i will have a clearer picture once i figure out what images they use from their helm charts [16:16:17] I was assuming we'd use CI to build and test it [16:16:29] but we'll see [16:16:44] so my biggest doubt is one that you've hit on already, which is: what is the long-term viability of this solution. One of the arguments from marxarelli was that it's fairly small in terms of actual code. [16:17:12] <_joe_> well the problem is, if we don't use our whole pipeline of images, we have no control on security updates when the next heartbleed comes along [16:18:06] right, trade offs, keeping up with upstream vs our standards and ability to respond quickly [16:18:32] <_joe_> yeah it's a tradeoff we're not willing to make usually, so that needs to be determined too [16:18:46] indeed [16:19:13] <_joe_> I am not against considering a third-party infra in absolute, but then the problem of trusted builds becomes larger [16:19:36] yeah, I was caught off guard by that question initially [16:19:42] there's a ton to consider in that case [16:20:03] my response was: it's possible, but I have no idea if we want to do that [16:20:32] SLAs cover some of my concern [16:21:03] but from a trust standpoint...well...I guess I could just link to "reflections on trusting trust" :) [16:26:04] <_joe_> i'll try to take the time to think about the open questions on the SRE side [16:26:35] great! [16:28:32] I think I'll try to come up with a shared place to talk about this rather than Google docs/random email threads (of which there are now a couple happening :)) [16:31:27] thcipriani, +1 [16:35:16] 10Release-Engineering-Team-TODO (201911): Upgrade PHP 7.2.x CI images to PHP 7.2.24 to align with production - https://phabricator.wikimedia.org/T237275 (10Jdforrester-WMF) [16:35:25] 10Release-Engineering-Team-TODO (201911): Upgrade PHP 7.2.x CI images to PHP 7.2.24 to align with production - https://phabricator.wikimedia.org/T237275 (10Jdforrester-WMF) [16:35:32] 10Release-Engineering-Team-TODO (201911): Upgrade PHP 7.2.x CI images to PHP 7.2.24 to align with production - https://phabricator.wikimedia.org/T237275 (10Jdforrester-WMF) [16:41:08] thcipriani: fwiw, i consider comments in the google doc to be the most helpful place for discussion atm [16:41:10] 10Gerrit: Can't change email in Gerrit - https://phabricator.wikimedia.org/T237211 (10Nullzero) 05Open→03Declined Cool! Closing then. [16:41:33] keep discussion close to the design doc [16:41:38] it *keeps* [16:42:03] marxarelli: I have Feelings™ about this :) [16:42:09] unless we want to move the entire doc to a wiki and use discussion pages [16:43:03] which would be fine by me. but let's not fragment the discussion further by seeking the one true place for discussion [16:43:36] thcipriani: can you articulate them? :) [16:43:37] on the one hand it's nice to be able to talk about specific lines in the doc directly on the doc. On the other hand, meta discussion is kinda clunky on google docs. [16:43:59] (03CR) 10Jforrester: [C: 03+2] Enable test coverage jobs for mw-ext-Cite [integration/config] - 10https://gerrit.wikimedia.org/r/548235 (owner: 10Awight) [16:44:17] google docs is great for refining a doc, but once it's in a state where you want feedback my impression is that google docs is less-great [16:44:20] hmm, yeah re: meta discussion [16:44:42] this is why meta discussion has moved to email and there are specific questions in the doc [16:45:45] (03Merged) 10jenkins-bot: Enable test coverage jobs for mw-ext-Cite [integration/config] - 10https://gerrit.wikimedia.org/r/548235 (owner: 10Awight) [16:46:30] thcipriani: right. that seems sufficient to me is i guess what i'm saying [16:47:56] my problem with email is that it's ephemeral and not a shared artifact; i.e., we have the same discussion happening in 2 email threads and the google doc (re:airflow) [16:48:23] haha, ok. well, i'm not sure why airflow is even in the discussion honestly [16:48:47] heh, well, let's not have that discussion on IRC as well :P [16:48:54] true true [16:48:56] marxarelli, some people are already looking into it and if it would work for CI as well, it's duplication of effort, I [16:49:00] think [16:49:21] !log Zuul: Add coverage reporting to Cite [16:49:23] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:49:40] my thinking if is there's a low friction way to have this discussion in a collaborative way it would help folks organize their thoughts [16:49:50] (03CR) 10Jforrester: [C: 03+2] prometheus-ipsec-exporter make debian glue voting [integration/config] - 10https://gerrit.wikimedia.org/r/548219 (owner: 10Hashar) [16:51:21] thcipriani, you're a manager now, so you should be thinking in terms of committees, subcommittees, task forces, working groups, and white papers [16:51:30] * James_F shudders. ;-) [16:51:37] working groups... that sounds familiar [16:51:42] not sure why [16:51:42] (03Merged) 10jenkins-bot: prometheus-ipsec-exporter make debian glue voting [integration/config] - 10https://gerrit.wikimedia.org/r/548219 (owner: 10Hashar) [16:52:10] !log Zuul: Make debian-glue voting for prometheus-ipsec-exporter [16:52:11] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [16:53:57] marxarelli, you've met a manager once? [16:55:01] 10Continuous-Integration-Config, 10CPT Initiatives (API Integration Tests), 10Code-Health: Enable API integration tests in CI for MediaWiki core - https://phabricator.wikimedia.org/T236680 (10daniel) @zeljkofilipin suggested to move only the tests proper into the core repo, and leave the framework in a separ... [16:56:55] I think that wikis and phab meet the criteria and google docs and one-off email threads come up a bit short. I'm willing to be overruled by you smart folks liw and marxarelli I am now off to a meeting and leave the real work to you all #JustManagerStuff [16:57:38] * liw side-delegates to marxarelli :) [16:57:49] I'm ok with phab [17:03:59] phab seems better for discussion than a wiki [17:06:20] i'd prefer to leave it where it is (google doc) and if wiki+phab is more appropriate in general, we should establish a policy around it and follow that policy for the next project [17:06:55] I can live with that, I guess [17:19:07] (03CR) 10Jforrester: ""Cannot open file "…/tests/phpunit.php"."" [integration/config] - 10https://gerrit.wikimedia.org/r/548235 (owner: 10Awight) [17:56:43] 10Release-Engineering-Team-TODO (201911), 10MediaWiki-General, 10MediaWiki-Page-Diffs, 10wikidiff2, 10ci-test-error: DifferenceEngineTest and ApiComparePagesTest failures when wikidiff2 is installed - https://phabricator.wikimedia.org/T237049 (10WDoranWMF) @Jdforrester-WMF Yeah, that was for our team to... [17:58:43] 10Release-Engineering-Team-TODO (201911), 10MediaWiki-General, 10MediaWiki-Page-Diffs, 10wikidiff2, 10ci-test-error: DifferenceEngineTest and ApiComparePagesTest failures when wikidiff2 is installed - https://phabricator.wikimedia.org/T237049 (10Jdforrester-WMF) >>! In T237049#5632579, @WDoranWMF wrote:... [18:01:24] starting bulk additions of depicts statements to 1k of the images I uploaded to beta commons last week, 15 seconds between edits as usual [18:03:26] thcipriani: liw marxarelli I know that gehel and ottomata are also interested in the airflow/job management discussion (as am I) and whether we might have enough common requirements to find a solution that meets all of our needs [18:04:07] this is by no means a given, but it might be worth someone from each group writing out two paragraphs of requirements and then having a 30 minute irc meeting to see if there's enough common ground or not [18:04:29] I plan to send an email to this effect to: liw ottomata gehel [18:04:34] :) [18:05:01] i think argo ci makes sense from what I can tell, at least compared to airflow. Airflow is just DAG pipelines, just like argo [18:05:07] the two paragraphs could live on a phab ticket, and the summary of any such discussion could also live on same ticket (are there any candidates?) [18:05:10] but there is no airflow-ci. Argo-CI looks nice [18:05:28] its more that there are more needs in analytics and sre prod for dag workflow job pipelines in k8s [18:05:34] let's not try to decide it here (but save that for the 30 minute irc-meet if we have it) ;-) [18:05:35] and it might be nice to unify those things if we could [18:05:39] if we can't that's ok, but we should consider it [18:05:56] agree, let's see if unification is reasonably possible or not [18:06:23] if other people want/should be in this short meetup, please chime in now (gehel and I have already talked out of band about it) [18:06:25] re the google doc discussion. agree google doc is great for initial design discussion and document for a limited number of parties. once questions there have been settled it is probably best to move to phab and continue discussions there [18:06:27] apergos: loop me in as well, please [18:06:38] {{done}} [18:06:44] e.g. [18:06:44] https://phabricator.wikimedia.org/T233634 [18:06:47] https://phabricator.wikimedia.org/T233634 [18:06:48] mail will be sent tomorrow morning my (EET) time [18:06:52] https://docs.google.com/document/d/1dpCo33RpZAbQG15nM_GcZ_zqA3sj0S4h_0CQ4Fsahkg/edit#heading=h.vpanev2oq14b [18:07:15] 10MediaWiki-Releasing, 10Core Platform Team, 10MW-1.34-notes, 10MW-1.34-release: Release MW 1.34.0-rc.1 - https://phabricator.wikimedia.org/T235922 (10Reedy) [18:10:41] apergos: thanks for the coordination! [18:11:01] yeah well, don't thank me until it's actually coordinated :-D [18:26:18] 10Release-Engineering-Team-TODO (201911), 10MediaWiki-General, 10MediaWiki-Page-Diffs, 10wikidiff2, 10ci-test-error: DifferenceEngineTest and ApiComparePagesTest failures when wikidiff2 is installed - https://phabricator.wikimedia.org/T237049 (10Jdforrester-WMF) Actually, no. I've spent three hours tryin... [18:28:34] 10Release-Engineering-Team-TODO (201911), 10Core Platform Team Workboards, 10MediaWiki-General, 10MediaWiki-Page-Diffs, and 2 others: DifferenceEngineTest and ApiComparePagesTest failures when wikidiff2 is installed - https://phabricator.wikimedia.org/T237049 (10Jdforrester-WMF) [18:29:18] apergos, looking forward to email [18:30:19] 10Release-Engineering-Team-TODO (201911), 10MediaWiki-General, 10MediaWiki-Page-Diffs, 10wikidiff2, and 2 others: DifferenceEngineTest and ApiComparePagesTest failures when wikidiff2 is installed - https://phabricator.wikimedia.org/T237049 (10daniel) >>! In T237049#5632750, @Jdforrester-WMF wrote: > @Danie... [18:37:04] 10Continuous-Integration-Config, 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (201911), 10Scap, and 5 others: Define variant Wikimedia production config in compiled, static files - https://phabricator.wikimedia.org/T223602 (10Jdforrester-WMF) [18:43:34] (03CR) 10Jforrester: "> Patch Set 1: Code-Review-1" [integration/quibble] - 10https://gerrit.wikimedia.org/r/547843 (https://phabricator.wikimedia.org/T192167) (owner: 10Jforrester) [18:48:49] cool cool [19:00:37] 10Release-Engineering-Team, 10Wikimedia-General-or-Unknown: "No localisation cache found for English. Please run maintenance/rebuildLocalisationCache.php." - https://phabricator.wikimedia.org/T237305 (10Reedy) [19:01:01] 10Release-Engineering-Team, 10Wikimedia-General-or-Unknown: "No localisation cache found for English. Please run maintenance/rebuildLocalisationCache.php." - https://phabricator.wikimedia.org/T237305 (10Reedy) [19:12:01] 10Release-Engineering-Team, 10MediaWiki-User-management, 10User-DannyS712: User rights validation is sometimes malfunctioning (with FlaggedRevs only?) - https://phabricator.wikimedia.org/T234743 (10Amorymeltzer) Noting this here since I don't think protection itself has been mentioned (here or in T233561), b... [19:15:19] 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201910), 10Jenkins, 10Upstream: JJB is too slow - https://phabricator.wikimedia.org/T236746 (10hashar) Both changes got merged. Pending release. [19:54:41] (03PS2) 10Krinkle: Update RESTBase link [integration/docroot] - 10https://gerrit.wikimedia.org/r/547846 (https://phabricator.wikimedia.org/T227766) (owner: 10Ammarpad) [19:58:59] 10Release-Engineering-Team-TODO, 10Developer Productivity, 10local-charts: mediawiki/core/.pipeline/dev_prereq.sh: fails if VisualEditor lib/ve submodule is at a commit not present in shallow clone - https://phabricator.wikimedia.org/T237310 (10brennen) [19:59:24] 10Release-Engineering-Team-TODO, 10Developer Productivity, 10local-charts: mediawiki/core/.pipeline/dev_prereq.sh: fails if VisualEditor lib/ve submodule is at a commit not present in shallow clone - https://phabricator.wikimedia.org/T237310 (10brennen) [20:00:12] 10Release-Engineering-Team, 10Wikimedia-General-or-Unknown: "No localisation cache found for English. Please run maintenance/rebuildLocalisationCache.php." - https://phabricator.wikimedia.org/T237305 (10Krinkle) Which wikis. Which servers. Such server should be depooled immediately and later investigated why i... [20:00:41] 10Release-Engineering-Team, 10Wikimedia-General-or-Unknown, 10serviceops: "No localisation cache found for English. Please run maintenance/rebuildLocalisationCache.php." - https://phabricator.wikimedia.org/T237305 (10Krinkle) [20:00:53] 10Release-Engineering-Team, 10Wikimedia-General-or-Unknown, 10serviceops: Some servers CLI fatal "No localisation cache found for English." - https://phabricator.wikimedia.org/T237305 (10Krinkle) [20:06:20] 10Project-Admins, 10Wikipedia-iOS-App-Backlog, 10iOS-app-feature-WP0, 10User-DannyS712: Archive #iOS-app-feature-WP0 component - https://phabricator.wikimedia.org/T236932 (10LGoto) p:05Triage→03Normal [20:23:15] 10Release-Engineering-Team, 10Wikimedia-General-or-Unknown, 10serviceops: Some servers CLI fatal "No localisation cache found for English." - https://phabricator.wikimedia.org/T237305 (10Reedy) On `deploy1001` - All wikis should fail, or non should. They're all on the same MW version `lines=10 reedy@deploy1... [20:31:39] 10Release-Engineering-Team, 10Wikimedia-General-or-Unknown, 10serviceops: Some wikis on CLI deploy1001 fatal with "No localisation cache found for English." - https://phabricator.wikimedia.org/T237305 (10Reedy) [20:53:38] 10Release-Engineering-Team-TODO (201911), 10MediaWiki-General, 10MediaWiki-Page-Diffs, 10wikidiff2, and 2 others: DifferenceEngineTest and ApiComparePagesTest failures when wikidiff2 is installed - https://phabricator.wikimedia.org/T237049 (10hashar) `$wgExternalDiffEngine` defaults to false which also mea... [20:56:38] 10Project-Admins, 10Wikipedia-iOS-App-Backlog, 10iOS-app-feature-WP0, 10User-DannyS712: Archive #iOS-app-feature-WP0 component - https://phabricator.wikimedia.org/T236932 (10JMinor) Yes, this can be archived/removed. [21:04:11] 10Release-Engineering-Team, 10Wikimedia-General-or-Unknown, 10serviceops: Some wikis on CLI deploy1001 fatal with "No localisation cache found for English." - https://phabricator.wikimedia.org/T237305 (10Krinkle) (On mobile) Do the files exist on disk? Are they different on mwmaint in terms of content or mo... [21:04:22] 10Release-Engineering-Team-TODO (201911), 10MediaWiki-General, 10MediaWiki-Page-Diffs, 10wikidiff2, and 2 others: DifferenceEngineTest and ApiComparePagesTest failures when wikidiff2 is installed - https://phabricator.wikimedia.org/T237049 (10Reedy) >>! In T237049#5633455, @hashar wrote: > The task descrip... [21:04:27] 10Release-Engineering-Team, 10Wikimedia-General-or-Unknown: Some wikis on CLI deploy1001 fatal with "No localisation cache found for English." - https://phabricator.wikimedia.org/T237305 (10Krinkle) [21:04:55] 10Continuous-Integration-Config, 10CPT Initiatives (API Integration Tests), 10Code-Health: Enable API integration tests in CI for MediaWiki core - https://phabricator.wikimedia.org/T236680 (10hashar) Once `api-testing` is made a npm package, each extension interested in having such tests would add the module... [21:07:10] 10Release-Engineering-Team-TODO (201911), 10MediaWiki-General, 10MediaWiki-Page-Diffs, 10wikidiff2, and 2 others: DifferenceEngineTest and ApiComparePagesTest failures when wikidiff2 is installed - https://phabricator.wikimedia.org/T237049 (10Jdforrester-WMF) Given other pressing needs, I'd recommend rever... [21:16:24] 10Release-Engineering-Team, 10Wikimedia-General-or-Unknown: Some wikis on CLI deploy1001 fatal with "No localisation cache found for English." - https://phabricator.wikimedia.org/T237305 (10Reedy) >>! In T237305#5633518, @Krinkle wrote: > Do the files exist on disk? Are they different on mwmaint in terms of co... [21:17:55] 10Release-Engineering-Team-TODO (201911), 10MediaWiki-General, 10MediaWiki-Page-Diffs, 10wikidiff2, and 2 others: DifferenceEngineTest and ApiComparePagesTest failures when wikidiff2 is installed - https://phabricator.wikimedia.org/T237049 (10Tgr) The traditional hack has been to set the diff engine to som... [21:20:14] about halfway through tonight's bulk adds of depicts statements, ~500 to go and I"ll probably go to sleep before it's done [21:32:18] 10Release-Engineering-Team-TODO (201911), 10MediaWiki-General, 10MediaWiki-Page-Diffs, 10wikidiff2, and 2 others: DifferenceEngineTest and ApiComparePagesTest failures when wikidiff2 is installed - https://phabricator.wikimedia.org/T237049 (10WDoranWMF) [22:16:19] lol @ logo at https://integration.wikimedia.org/ci/configureSecurity/ [22:16:24] if you're logged out [22:17:04] xDD [22:47:33] (03PS1) 10Zoranzoki21: Disable running selenium tests for AnonPrivacy extension because of timeouts and too this extension isn't big for much tests [integration/config] - 10https://gerrit.wikimedia.org/r/548542 [22:49:39] (03CR) 10jerkins-bot: [V: 04-1] Disable running selenium tests for AnonPrivacy extension because of timeouts and too this extension isn't big for much tests [integration/config] - 10https://gerrit.wikimedia.org/r/548542 (owner: 10Zoranzoki21) [22:52:58] 10Continuous-Integration-Config, 10User-Zoranzoki21: quibble-vendor-mysql-php72-docker fails for AnonPrivacy because of selenium timeouts - https://phabricator.wikimedia.org/T237335 (10Zoranzoki21) [22:53:36] (03PS2) 10Zoranzoki21: Disable running selenium tests for AnonPrivacy extension [integration/config] - 10https://gerrit.wikimedia.org/r/548542 (https://phabricator.wikimedia.org/T237335) [23:07:18] paladox: we do all this stuff in gerrit::jetty that isnt even related to the webserver [23:07:27] yeh [23:07:28] some time i want to refactor [23:07:32] maybe time for a split [23:07:36] yea, i want to [23:07:51] and then puppetize 2 things: [23:07:55] - that tmp dir from Friday [23:08:23] - a symlink from /etc/gerrit/ to /var/lib/gerrit2/review_site/etc for convenience because how many times have people looked for that [23:08:38] oh! [23:08:43] yeh [23:10:16] ok, just changing the order of those things [23:10:57] paladox: do you have any Gerrits that are not buster yet? [23:11:05] yup [23:11:07] gerrit-test5 [23:11:31] would you want to upgrade? [23:11:35] yup [23:11:45] flamethrower it instead [23:11:49] i need to retest the gerrit upgrade [23:11:50] * mutante makes another patch to remove support for pre-buster [23:12:00] per discusion with tyler [23:12:12] I need to git mirror mediawiki/core :P [23:12:21] and the puppet repo [23:12:29] paladox: and gerrit-replica is also migrated? [23:12:41] yup [23:12:42] it's been moved to buster [23:12:47] Good, good [23:12:49] * paladox need's to retest 2.15 -> 2.16 [23:13:00] hauskater: happened on Friday [23:13:06] Yeah that task is growing weeds [23:13:20] hauskater what task? the upgrade to 2.16? [23:13:23] nice expression :p [23:13:24] Yup [23:13:33] mutante: not that weed :P [23:13:49] oh, we now have a tentative date. [23:14:33] and it's 2.16 -> 3.x ? [23:16:06] Well that'll be next year :P [23:16:15] then 3.x to 3.1 :P [23:16:22] *3.0 [23:16:33] 2.16 will be this year [23:17:17] well, on one hand I'd rather keep what we have now, because it works [23:17:44] heh, it goes EOL in a week :( [23:18:00] yeah, that's the other hand [23:18:18] paladox: we really need to warn people the UI is changing [23:18:25] the default will be polygerrit [23:18:28] yeh [23:18:35] i think i've sent emails on that [23:18:35] that [23:18:37] and i am sure there are a lot of people left who dont use that [23:18:46] * hauskater whistles [23:19:07] but in my case because Polygerrit in our install does not support create/edit patches [23:19:07] sent on the 7th of may [23:19:14] paladox: and what about the things where the solution is "switch to old UI" [23:19:19] like editing in browser [23:19:34] Oh, you can use the new UI for editing in the browser. [23:19:35] and the admin UI [23:19:46] 2.16 includes the missing features [23:19:54] just saying because recently there was a case where i had to switch back, remember [23:20:03] ok! :) cool [23:20:15] mutante hauskater https://lists.wikimedia.org/pipermail/wikitech-l/2019-May/092057.html [23:20:23] yup [23:20:31] Yeah I read that mail [23:20:45] I later unsubscribed from wikitech-l [23:20:52] some weeks later [23:25:26] paladox: wanna do a change in the gerrit deploy repo? [23:25:33] sure [23:25:34] to add the mysql client jar [23:25:37] oh! [23:25:39] yes [23:25:40] i am doing the puppet tmp dir [23:25:47] thanks [23:26:03] mutante are you doing that? [23:26:31] paladox: i am doing the puppet changes and asking for the gerrit repo part [23:26:37] ah ok [23:26:42] * paladox finds the driver [23:27:12] paladox: let me copy it [23:27:18] ok [23:27:57] https://people.wikimedia.org/~dzahn/mysql-connector-java.jar [23:28:15] thanks! [23:29:36] paladox: so deployment should copy it to /usr/share/java/ and puppet can symlink to /var/lib/gerrit2/review_site/lib i guess [23:29:39] doing the change now [23:29:58] i doin't think deployment can copy it to /usr [23:29:59] well.. i mean [23:30:02] but [23:30:13] lib/ is symnlinked to the lib folder of the deployment repo [23:30:24] we need a new link from /srv/deployment to /usr/share/java or directly to /var/lib/gerrti2 [23:30:39] ok! [23:31:48] (03PS1) 10Paladox: Add mysql-connector-java [software/gerrit] (deploy/wmf/stable-2.15) - 10https://gerrit.wikimedia.org/r/548549 [23:31:52] thcipriani mutante ^ [23:36:14] oh! [23:36:20] it's not symnlinked [23:36:44] mutante wondering should we symnlink the folder or should we just add the mysql symnlink to the puppet repo? [23:37:23] paladox: ehm.. puppet repo. already has other links [23:38:16] (03CR) 10Dzahn: [C: 03+1] Add mysql-connector-java [software/gerrit] (deploy/wmf/stable-2.15) - 10https://gerrit.wikimedia.org/r/548549 (owner: 10Paladox) [23:38:55] or that :) [23:39:19] heh [23:47:12] Good night mutante & paladox [23:47:19] and you too :) [23:47:33] thanks bud [23:49:22] good night, gato [23:52:40] mutante: y'know we use the same word for cats and https://commons.wikimedia.org/wiki/File:Jackscrew.jpg ? :) [23:53:08] that device is also a 'gato' [23:53:26] TIL [23:53:35] 70 # TEMP - revert and always enable once T176532 is resolved [23:53:35] 71 $ensure_monitor_process = $replica ? { [23:53:35] T176532: Gerrit is failing to connect to db on gerrit2001 thus preventing systemd from working - https://phabricator.wikimedia.org/T176532 [23:53:37] paladox: ^ [23:53:46] that ticket is resolved but we did not revert [23:53:52] oh! [23:53:54] yeh let [23:53:58] revert that [23:54:04] ok :) making it [23:54:31] thanks!