[00:04:17] 10Beta-Cluster-Infrastructure, 10Analytics, 10Event-Platform, 10MW-1.36-notes (1.36.0-wmf.20; 2020-12-01), and 2 others: Server returned error: HTTP 500 appears while trying to open VE or reply to a comment on Beta cluster - https://phabricator.wikimedia.org/T268184 (10Jdforrester-WMF) 05Open→03Resolved... [00:25:33] !log shutting off deployment-aqs* instances (T257118) [00:25:37] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [00:25:37] T257118: Beta cluster has reached its quota - https://phabricator.wikimedia.org/T257118 [01:10:49] 10VPS-project-codesearch, 10GitLab: Figure out the future of codesearch in a GitLab world - https://phabricator.wikimedia.org/T268196 (10Legoktm) [01:11:25] 10VPS-project-codesearch, 10GitLab: Figure out the future of codesearch in a GitLab world - https://phabricator.wikimedia.org/T268196 (10Legoktm) [01:17:27] 10VPS-project-codesearch, 10GitLab: Figure out the future of codesearch in a GitLab world - https://phabricator.wikimedia.org/T268196 (10Legoktm) > Once all repositories have moved to GitLab It would be good to have an estimate of when this is expected to happen. [01:31:24] 10VPS-project-codesearch: Reduce user impact of codesearch's nightly restarts - https://phabricator.wikimedia.org/T268198 (10Legoktm) [01:44:04] 10VPS-project-codesearch, 10Operations: Graduate codesearch to production - https://phabricator.wikimedia.org/T268199 (10Legoktm) [01:44:30] 10VPS-project-codesearch, 10Operations: Graduate codesearch to production - https://phabricator.wikimedia.org/T268199 (10Legoktm) [01:45:56] 10VPS-project-codesearch, 10Operations: Graduate codesearch to production - https://phabricator.wikimedia.org/T268199 (10Legoktm) [01:45:59] 10Beta-Cluster-Infrastructure, 10Operations, 10User-DannyS712: Beta cluster logstash down - https://phabricator.wikimedia.org/T268200 (10DannyS712) [01:47:10] 10Beta-Cluster-Infrastructure, 10Release-Engineering-Team, 10Operations, 10User-DannyS712: Beta cluster logstash down - https://phabricator.wikimedia.org/T268200 (10DannyS712) [04:11:04] 10VPS-project-codesearch, 10GitLab: Figure out the future of codesearch in a GitLab world - https://phabricator.wikimedia.org/T268196 (10greg) >>! In T268196#6632441, @Legoktm wrote: >> Once all repositories have moved to GitLab > > It would be good to have an estimate of when this is expected to happen. We... [08:25:41] 10Release-Engineering-Team (Pipeline), 10Release Pipeline (Blubber), 10User-Pablo-WMDE: Blubber: runuser permissions to node_modules/.cache - https://phabricator.wikimedia.org/T255434 (10Pablo-WMDE) [08:27:33] more gerrit woes: https://integration.wikimedia.org/ci/job/wmf-quibble-selenium-php72-docker/68748/console [08:27:50] 10:09:56 stderr: 'fatal: unable to access 'https://gerrit.wikimedia.org/r/mediawiki/extensions/AbuseFilter/': The requested URL returned error: 502' [08:50:34] 10Diffusion, 10Diffusion-Repository-Administrators, 10Wikidata: Diffusion: wikibase-termbox gerrit mirror connection time out - https://phabricator.wikimedia.org/T218780 (10Pablo-WMDE) 05Open→03Resolved a:03Pablo-WMDE This should have been closed a long time ago. Thanks! [09:18:18] (03PS1) 10Hashar: Merge tag 'v3.2.5' into wmf/stable-3.2 [software/gerrit] (wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/641925 (https://phabricator.wikimedia.org/T268009) [09:18:58] (03CR) 10jerkins-bot: [V: 04-1] Merge tag 'v3.2.5' into wmf/stable-3.2 [software/gerrit] (wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/641925 (https://phabricator.wikimedia.org/T268009) (owner: 10Hashar) [09:50:30] (03PS1) 10Hashar: dockerfiles: use Bazel 3.7.0 [integration/config] - 10https://gerrit.wikimedia.org/r/641934 [09:50:32] (03PS1) 10Hashar: jjb: use Gerrit upstream submodules [integration/config] - 10https://gerrit.wikimedia.org/r/641935 [09:50:34] (03PS1) 10Hashar: jjb: update gerrit-docker job to bazel 3.7.0 [integration/config] - 10https://gerrit.wikimedia.org/r/641936 [09:50:45] (03CR) 10Hashar: [C: 03+2] dockerfiles: use Bazel 3.7.0 [integration/config] - 10https://gerrit.wikimedia.org/r/641934 (owner: 10Hashar) [09:52:04] (03Merged) 10jenkins-bot: dockerfiles: use Bazel 3.7.0 [integration/config] - 10https://gerrit.wikimedia.org/r/641934 (owner: 10Hashar) [09:52:22] !log Building newer Docker images for Bazel / Gerrit [09:52:24] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [09:56:42] 10Release-Engineering-Team, 10Product-Infrastructure-Team-Backlog, 10Recommendation-API, 10Services, and 2 others: Migrate recommendation-api to kubernetes - https://phabricator.wikimedia.org/T241230 (10akosiaris) [09:59:20] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10Operations, 10Release Pipeline, and 2 others: Migrate production services to kubernetes using the pipeline - https://phabricator.wikimedia.org/T198901 (10akosiaris) [09:59:50] 10Release-Engineering-Team, 10Product-Infrastructure-Team-Backlog, 10Recommendation-API, 10Services, and 2 others: Migrate recommendation-api to kubernetes - https://phabricator.wikimedia.org/T241230 (10akosiaris) 05Open→03Resolved p:05Triage→03Medium a:03akosiaris The service has been deployed y... [10:17:01] (03CR) 10Hashar: [C: 03+2] jjb: use Gerrit upstream submodules [integration/config] - 10https://gerrit.wikimedia.org/r/641935 (owner: 10Hashar) [10:18:04] (03CR) 10Hashar: [C: 03+2] jjb: update gerrit-docker job to bazel 3.7.0 [integration/config] - 10https://gerrit.wikimedia.org/r/641936 (owner: 10Hashar) [10:18:19] (03Merged) 10jenkins-bot: jjb: use Gerrit upstream submodules [integration/config] - 10https://gerrit.wikimedia.org/r/641935 (owner: 10Hashar) [10:19:38] (03Merged) 10jenkins-bot: jjb: update gerrit-docker job to bazel 3.7.0 [integration/config] - 10https://gerrit.wikimedia.org/r/641936 (owner: 10Hashar) [10:26:55] hashar: any idea how easy it would be to add / get the mediawiki debug logs in jobs like https://integration.wikimedia.org/ci/job/selenium-daily-beta-WikibaseLexeme/833/ ? [10:39:58] addshore: it running against the beta cluster instances, so the logs should be on one of those instances [10:40:09] aaah yes [10:40:11] addshore: possibly in the logstash beta if it is still up ;) [10:40:14] good pint *facepalm* :P [10:40:19] *point [11:11:16] (03PS1) 10Hashar: dockerfiles: gerrit: allow changing bazel targets [integration/config] - 10https://gerrit.wikimedia.org/r/641957 [11:12:53] (03CR) 10Hashar: [C: 03+2] dockerfiles: gerrit: allow changing bazel targets [integration/config] - 10https://gerrit.wikimedia.org/r/641957 (owner: 10Hashar) [11:13:57] (03Merged) 10jenkins-bot: dockerfiles: gerrit: allow changing bazel targets [integration/config] - 10https://gerrit.wikimedia.org/r/641957 (owner: 10Hashar) [11:19:54] 10Project-Admins: Project (+workboard) for Koninklijke Bibliotheek (National Library of The Netherlands) - https://phabricator.wikimedia.org/T268222 (10DanielleJWiki) [11:27:31] 10Project-Admins: New project: Wiki-Techstorm-2021 - https://phabricator.wikimedia.org/T268223 (10Ecritures) [11:31:46] 10Project-Admins: Tag:#KB-TeamWiki - https://phabricator.wikimedia.org/T268224 (10DanielleJWiki) [11:31:52] 10Gerrit, 10Release-Engineering-Team (Development services): Switch Gerrit from Java 8 to Java 11 - https://phabricator.wikimedia.org/T268225 (10hashar) [11:35:53] (03PS1) 10Hashar: jjb: build multiple gerrit targets [integration/config] - 10https://gerrit.wikimedia.org/r/641966 [13:04:48] (03CR) 10Hashar: [C: 03+2] "Works!" [integration/config] - 10https://gerrit.wikimedia.org/r/641966 (owner: 10Hashar) [13:06:10] (03Merged) 10jenkins-bot: jjb: build multiple gerrit targets [integration/config] - 10https://gerrit.wikimedia.org/r/641966 (owner: 10Hashar) [14:12:13] (03PS1) 10Hashar: Bump gerrit.war to 3.2.5 [software/gerrit] (deploy/wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/641991 (https://phabricator.wikimedia.org/T268009) [14:20:56] (03CR) 10Hashar: "recheck I have updated CI to use Bazel 3.7.0" [software/gerrit] (wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/641925 (https://phabricator.wikimedia.org/T268009) (owner: 10Hashar) [14:24:44] (03CR) 10Hashar: "I have more or less followed the documentation at https://wikitech.wikimedia.org/wiki/Gerrit#Build_Gerrit" [software/gerrit] (deploy/wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/641991 (https://phabricator.wikimedia.org/T268009) (owner: 10Hashar) [14:39:21] (03CR) 10Hashar: [C: 04-1] "I have cherry picked this change on the instance gerrit-prod-1001.devtools.eqiad.wmflabs and restarted gerrit there https://gerrit.devtool" [software/gerrit] (deploy/wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/641991 (https://phabricator.wikimedia.org/T268009) (owner: 10Hashar) [15:09:52] thcipriani: paladox: lets speak there cause I cant really multitasks so many things :) [15:10:08] the patch gerrit is at https://gerrit.devtools.wmflabs.org/r/ :] [15:10:16] great! [15:10:47] I saw that, but you mentioned the version # looks wrong. I have never before used the tools/workspace_status_release.py script and am reading it now [15:11:10] here is the diff https://phabricator.wikimedia.org/P13336 [15:11:34] hashar did you create a master tag? [15:11:54] and I have build the releases.war using the info from https://wikitech.wikimedia.org/wiki/Gerrit#Build_Gerrit [15:11:59] bazel build --workspace_status_command="python ./tools/workspace_status_release.py" release [15:12:21] which seems right since this is a realease :) [15:14:13] but it also looks like the other one gives you the correct version # [15:16:42] ah my master branch is dirty [15:17:14] * thcipriani goes to take care of morning things [15:17:27] back in 30 [15:27:51] rebuilding without --workspace_status_command="python ./tools/workspace_status_release.py" [15:27:58] I guess it assumes the master branch points at a specific tag [15:28:10] or whatever [15:29:15] $ java -jar gerrit.war version [15:29:15] gerrit version 3.2.5-2-gade85f3c32 [15:29:17] better [15:33:07] (03PS2) 10Hashar: Bump gerrit.war to 3.2.5 [software/gerrit] (deploy/wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/641991 (https://phabricator.wikimedia.org/T268009) [15:45:05] (back) [15:47:25] is there a way to login on the devtools gerrit? [15:47:55] (03PS3) 10Hashar: Bump gerrit.war to 3.2.5 [software/gerrit] (deploy/wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/641991 (https://phabricator.wikimedia.org/T268009) [15:52:34] (03CR) 10Hashar: "I have rebuild it withouth `--workspace_status_command="python ./tools/workspace_status_release.py"`" [software/gerrit] (deploy/wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/641991 (https://phabricator.wikimedia.org/T268009) (owner: 10Hashar) [15:54:27] oh my god [15:54:32] thcipriani: I hvae no idea :( [15:54:47] I cant upload to archive again: error 409 [15:55:25] hashar: I wonder if you have to use a different tag? [15:55:31] yeah I guess [15:55:37] 3.2.5~1? [15:55:58] yeah doing that [15:56:04] the version tag being irrelevant I imagine? [15:56:54] for us, yeah [15:57:14] since git fat uses the hash and scap uses git fat [15:57:24] it's for humans, I guess [15:57:29] as soon as the uploaded has completed I will update devtools [15:57:50] (not that I've ever actually tried to download anything for gerrit directly from archiva) [15:59:38] oh [15:59:54] and the mysterious java error you had is that the gerrit-3.2.5.war upstream release is for java 11 [16:00:51] yeah, I saw the mailing list, evidently that was a mistake :P [16:01:38] I guess I have a bad habit of assuming that if it doesn't work for me it's my problem [16:07:37] --version=3.2.5~1 worked [16:11:48] (03CR) 10Ejegg: [C: 03+1] "Also working for me under Linux and MacOS" (031 comment) [releng/dev-images] - 10https://gerrit.wikimedia.org/r/632173 (https://phabricator.wikimedia.org/T262976) (owner: 10AndyRussG) [16:12:30] (03CR) 10Ahmon Dancy: [C: 04-1] feat: ./train-dev save and restore subcommands (032 comments) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/641787 (owner: 10Lars Wirzenius) [16:12:49] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10MW-on-K8s: Convert static mediawiki configuration to form more suitable for containers - https://phabricator.wikimedia.org/T263166 (10BPirkle) As usual, one of the challenges is that whatever we do shoul... [16:15:41] (03CR) 10Lars Wirzenius: "Fixing, re-testing, pushing new patch set soon." (032 comments) [tools/train-dev] - 10https://gerrit.wikimedia.org/r/641787 (owner: 10Lars Wirzenius) [16:20:26] fetched on the devtool machine [16:20:29] and ran git fat pull [16:20:40] gerrit restarted there [16:20:55] works! [16:20:59] but there is no auth system there [16:21:04] at least I don't have an account :] [16:21:20] ohhh [16:21:24] but we have gitiles https://gerrit.devtools.wmflabs.org/g/hello-world [16:21:29] let msee if i can fix that [16:21:44] i think mutante and me agreed to use the developer auth rather than ldap. [16:21:50] and gitiles gets the fonts from the instance all good [16:22:09] well maybe we can hook it with some oauth provider [16:22:46] so I think it is fine [16:23:04] 10Release-Engineering-Team-TODO, 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T263184 (10Lucas_Werkmeister_WMDE) {T267668} seems to be cropping up again. [16:23:32] (03Restored) 10Jforrester: [DNM] layout: [mediawiki/services/recommendation-api] Drop npm-docker [integration/config] - 10https://gerrit.wikimedia.org/r/516635 (https://phabricator.wikimedia.org/T248355) (owner: 10Mholloway) [16:23:34] (03Restored) 10Jforrester: layout: [mediawiki/services/recommendation-api/deploy] Archive [integration/config] - 10https://gerrit.wikimedia.org/r/516718 (https://phabricator.wikimedia.org/T248355) (owner: 10Jforrester) [16:23:42] (03Restored) 10Jforrester: jjb: Drop recommendation API node 6 jobs, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/516719 (https://phabricator.wikimedia.org/T248355) (owner: 10Jforrester) [16:24:42] (03PS7) 10Jforrester: layout: [mediawiki/services/recommendation-api] Drop npm-docker [integration/config] - 10https://gerrit.wikimedia.org/r/516635 (https://phabricator.wikimedia.org/T248355) (owner: 10Mholloway) [16:24:44] (03PS5) 10Jforrester: layout: [mediawiki/services/recommendation-api/deploy] Archive [integration/config] - 10https://gerrit.wikimedia.org/r/516718 (https://phabricator.wikimedia.org/T248355) [16:24:46] (03PS7) 10Jforrester: jjb: Drop recommendation API node 6 jobs, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/516719 (https://phabricator.wikimedia.org/T248355) [16:25:09] (03CR) 10Hashar: [C: 03+2] "Battle tested and looks like the basics are definitely working." [software/gerrit] (wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/641925 (https://phabricator.wikimedia.org/T268009) (owner: 10Hashar) [16:26:17] dancy: I am going to upgrade Gerrit now I believe [16:26:29] ok. Have at it. [16:27:06] * James_F waits on the CI config change, then. [16:27:29] 10Release-Engineering-Team-TODO, 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T263184 (10dancy) >>! In T263184#6634017, @Lucas_Werkmeister_WMDE wrote: > {T267668} seems to be cropping up again. Thanks for the note. Is this a req... [16:31:39] (03Merged) 10jenkins-bot: Merge tag 'v3.2.5' into wmf/stable-3.2 [software/gerrit] (wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/641925 (https://phabricator.wikimedia.org/T268009) (owner: 10Hashar) [16:32:12] 10Release-Engineering-Team-TODO, 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T263184 (10Lucas_Werkmeister_WMDE) I’m not the right person to decide that, sorry – I just saw a few reports of the few issue and thought it was worth m... [16:35:47] (03CR) 10Hashar: [V: 03+2 C: 03+2] "At least the basic testing works. gitiles is not updated so we still have https://gerrit-review.googlesource.com/c/plugins/gitiles/+/2733" [software/gerrit] (deploy/wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/641991 (https://phabricator.wikimedia.org/T268009) (owner: 10Hashar) [16:39:31] starting with gerrit2001 [16:47:05] upgraded! [16:49:07] Nice work [16:50:51] 👍 [16:51:41] https://gerrit-replica.wikimedia.org confirmed up [16:52:55] I am holding a bit, will do gerrit1001 soonish [16:52:59] <3 [16:54:32] (03PS2) 10Lars Wirzenius: feat: ./train-dev save and restore subcommands [tools/train-dev] - 10https://gerrit.wikimedia.org/r/641787 [16:54:47] I need two or three extra monitors [17:05:35] 10Release-Engineering-Team-TODO, 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T263184 (10dancy) [17:06:39] 10Release-Engineering-Team-TODO, 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T263184 (10dancy) The train has been rolled back to group0 (i.e., group0 = 1.36.0-wmf.18, group1 and group 2 = 1.36.0-wmf.16). T267668 is now blocking... [17:08:39] 10Release-Engineering-Team (Pipeline), 10Release Pipeline (Blubber), 10User-Pablo-WMDE: Blubber: runuser permissions to node_modules/.cache - https://phabricator.wikimedia.org/T255434 (10dduvall) I'm not knowledgable of the initiative or NPM package, and Blubber doesn't really have an opinion about this othe... [17:10:54] (03CR) 10Ahmon Dancy: [C: 03+2] feat: ./train-dev save and restore subcommands [tools/train-dev] - 10https://gerrit.wikimedia.org/r/641787 (owner: 10Lars Wirzenius) [17:11:08] (03Merged) 10jenkins-bot: feat: ./train-dev save and restore subcommands [tools/train-dev] - 10https://gerrit.wikimedia.org/r/641787 (owner: 10Lars Wirzenius) [17:14:39] hello all! I'm new to this particular test but this looks to me like a Jenkins failure to me: [17:14:39] https://integration.wikimedia.org/ci/job/operations-puppet-logstash-filter-verifier/52/console [17:15:05] If it's actually just rejecting my patch in a correct but cryptic way then I guess I need someone to explain :( [17:17:24] Very cryptic. [17:18:28] hashar hi, could you review https://gerrit.wikimedia.org/r/c/operations/puppet/+/641778 please? :) [17:18:39] paladox: not today, too busy :-\ [17:18:44] ok [17:18:46] unless that is super trivial hehe [17:18:53] it's super trivial [17:18:57] noop on gerrit1001 [17:18:58] oh that is for labs [17:18:59] hehe [17:19:01] yeh [17:19:05] yeah we can do that later tonight I guess [17:19:19] I guess if the compiler is a noop for gerrit1001 we can land it [17:19:25] yeh [17:20:02] andrewbogott: I'll look around a bit to see if I can help [17:20:27] hashar i've enabled it on the dev site. [17:20:33] dancy: I'm getting help in another channel, I think it's a legit failure (with bad reporting) [17:20:35] thanks though [17:20:39] ok. [17:20:41] one problem i found is after clicking login :80 is added to the url. [17:27:48] I am reviewing the change but said "no merge during their upgrade" [17:27:57] already talking about code though [17:28:06] for cloud and to validate the auth_type [17:47:13] (03CR) 10Jforrester: [C: 03+2] layout: [mediawiki/services/recommendation-api] Drop npm-docker [integration/config] - 10https://gerrit.wikimedia.org/r/516635 (https://phabricator.wikimedia.org/T248355) (owner: 10Mholloway) [17:48:31] (03Merged) 10jenkins-bot: layout: [mediawiki/services/recommendation-api] Drop npm-docker [integration/config] - 10https://gerrit.wikimedia.org/r/516635 (https://phabricator.wikimedia.org/T248355) (owner: 10Mholloway) [17:50:52] 10LibUp: Add a list of failing repos on libup web - https://phabricator.wikimedia.org/T268254 (10Umherirrender) [17:51:12] !log Zuul:[mediawiki/services/recommendation-api] Drop npm-docker T248355 [17:51:15] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [17:51:20] T248355: Archive mediawiki/services/recommendation-api/deploy once it's no longer used - https://phabricator.wikimedia.org/T248355 [17:52:16] 10LibUp: Add a list of failing repos or an indicator in lists for failing repos on libup web - https://phabricator.wikimedia.org/T268254 (10Umherirrender) [17:52:25] (03CR) 10Jforrester: "Want to land the actual emptying-out first: I34188e3." [integration/config] - 10https://gerrit.wikimedia.org/r/516718 (https://phabricator.wikimedia.org/T248355) (owner: 10Jforrester) [17:52:38] 10Gerrit, 10Security-Team, 10Security: Gerrit security advisory 2020-11-17 - https://phabricator.wikimedia.org/T268009 (10Reedy) [18:00:12] 10LibUp: Add a list of failing repos or an indicator in lists for failing repos on libup web - https://phabricator.wikimedia.org/T268254 (10Legoktm) See https://libraryupgrader2.wmcloud.org/errors (sorry, I thought you already knew about that page) I like putting an icon in the lists as well. [18:02:00] 10LibUp: Add a list of failing repos or an indicator in lists for failing repos on libup web - https://phabricator.wikimedia.org/T268254 (10Umherirrender) It seems not linked from anywhere or I missed it. But thanks for the link I usually looking at the package gets updated and than all repos not having the new... [18:02:02] (03CR) 10Jforrester: [C: 03+2] layout: [mediawiki/services/recommendation-api/deploy] Archive [integration/config] - 10https://gerrit.wikimedia.org/r/516718 (https://phabricator.wikimedia.org/T248355) (owner: 10Jforrester) [18:02:47] legoktm: Huh, I never knew about the error page either. [18:03:05] ://// [18:03:18] (03Merged) 10jenkins-bot: layout: [mediawiki/services/recommendation-api/deploy] Archive [integration/config] - 10https://gerrit.wikimedia.org/r/516718 (https://phabricator.wikimedia.org/T248355) (owner: 10Jforrester) [18:03:30] Making it more visible might help with you feeling like you have to fix every repo. :-( [18:03:56] !log Zuul: [mediawiki/services/recommendation-api/deploy] Archive T248355 [18:03:58] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [18:03:59] T248355: Archive mediawiki/services/recommendation-api/deploy once it's no longer used - https://phabricator.wikimedia.org/T248355 [18:05:23] (03CR) 10Jforrester: [C: 03+2] "Deployed." [integration/config] - 10https://gerrit.wikimedia.org/r/516719 (https://phabricator.wikimedia.org/T248355) (owner: 10Jforrester) [18:06:09] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10Operations, 10Release Pipeline, and 2 others: Migrate production services to kubernetes using the pipeline - https://phabricator.wikimedia.org/T198901 (10Jdforrester-WMF) [18:06:43] (03Merged) 10jenkins-bot: jjb: Drop recommendation API node 6 jobs, now unused [integration/config] - 10https://gerrit.wikimedia.org/r/516719 (https://phabricator.wikimedia.org/T248355) (owner: 10Jforrester) [18:07:26] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10JavaScript, 10Patch-For-Review: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10Jdfo... [18:07:40] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10JavaScript, 10Patch-For-Review: Upgrade all CI jobs from node6/npm3 to node10/npm6 across all projects - https://phabricator.wikimedia.org/T211784 (10Jdfo... [18:07:45] 10Release-Engineering-Team (Pipeline), 10Release-Engineering-Team-TODO, 10Operations, 10Release Pipeline, and 2 others: Migrate production services to kubernetes using the pipeline - https://phabricator.wikimedia.org/T198901 (10Mholloway) [18:20:19] 10Gerrit, 10Regression: URLs in format https://gerrit.wikimedia.org/r/123456 end up on port 80, leading to error SSL_ERROR_RX_RECORD_TOO_LONG - https://phabricator.wikimedia.org/T268260 (10Aklapper) p:05Triage→03High [18:21:23] 10Gerrit, 10Regression, 10Upstream: URLs in format https://gerrit.wikimedia.org/r/123456 end up on port 80, leading to error SSL_ERROR_RX_RECORD_TOO_LONG - https://phabricator.wikimedia.org/T268260 (10hashar) [18:21:25] (03Abandoned) 10Jforrester: Add CheckUser as a integration dependency of WikimediaMessages [integration/config] - 10https://gerrit.wikimedia.org/r/617832 (https://phabricator.wikimedia.org/T256586) (owner: 10Dbarratt) [18:21:38] 10Gerrit, 10Security-Team, 10Security: Gerrit security advisory 2020-11-17 - https://phabricator.wikimedia.org/T268009 (10hashar) An aftermath reported by some persons on IRC is that an URL such as https://gerrit.wikimedia.org/r/123 ends up redirected to an url including `:80` which breaks it. T268260 and... [18:21:40] 10Gerrit, 10Regression, 10Upstream: URLs in format https://gerrit.wikimedia.org/r/123456 end up on port 80, leading to error SSL_ERROR_RX_RECORD_TOO_LONG - https://phabricator.wikimedia.org/T268260 (10hashar) [18:21:42] 10Gerrit, 10Security-Team, 10Security: Gerrit security advisory 2020-11-17 - https://phabricator.wikimedia.org/T268009 (10hashar) [18:22:17] (03Abandoned) 10Jforrester: Add CirrusSearch as a GrowthExperiments dependency [integration/config] - 10https://gerrit.wikimedia.org/r/579230 (owner: 10Gergő Tisza) [18:23:57] 10Gerrit, 10Regression, 10Upstream: URLs in format https://gerrit.wikimedia.org/r/123456 end up on port 80, leading to error SSL_ERROR_RX_RECORD_TOO_LONG - https://phabricator.wikimedia.org/T268260 (10Aklapper) [18:24:00] 10Gerrit, 10Security-Team, 10Security: Gerrit security advisory 2020-11-17 - https://phabricator.wikimedia.org/T268009 (10Aklapper) [18:24:13] 10Gerrit, 10Regression, 10Upstream: URLs in format https://gerrit.wikimedia.org/r/123456 end up on port 80, leading to error SSL_ERROR_RX_RECORD_TOO_LONG - https://phabricator.wikimedia.org/T268260 (10Aklapper) [18:24:15] 10Gerrit, 10Security-Team, 10Security: Gerrit security advisory 2020-11-17 - https://phabricator.wikimedia.org/T268009 (10Aklapper) [18:26:39] 10Gerrit, 10Regression, 10Upstream: URLs in format https://gerrit.wikimedia.org/r/123456 end up on port 80, leading to error SSL_ERROR_RX_RECORD_TOO_LONG - https://phabricator.wikimedia.org/T268260 (10Paladox) Filed upstream at https://bugs.chromium.org/p/gerrit/issues/detail?id=13705 [18:32:33] (03PS1) 10Ahmon Dancy: deploy-promote: poll SCM for merge of wikiversions commit [tools/release] - 10https://gerrit.wikimedia.org/r/642076 [18:36:50] (03CR) 1020after4: [C: 03+2] "Very cool!" [tools/release] - 10https://gerrit.wikimedia.org/r/642076 (owner: 10Ahmon Dancy) [18:37:56] (03Merged) 10jenkins-bot: deploy-promote: poll SCM for merge of wikiversions commit [tools/release] - 10https://gerrit.wikimedia.org/r/642076 (owner: 10Ahmon Dancy) [18:44:12] (03PS2) 10Jforrester: dockerfiles: Provide composer, to replace composer-php70 [integration/config] - 10https://gerrit.wikimedia.org/r/634811 [18:44:14] (03PS2) 10Jforrester: dockerfiles: [phpmetrics] Switch parent image composer-php70 to composer-php72 [integration/config] - 10https://gerrit.wikimedia.org/r/634812 [18:44:16] (03PS2) 10Jforrester: dockerfiles: [composer-php*] Switch from composer-php70 to use composer [integration/config] - 10https://gerrit.wikimedia.org/r/634813 [18:44:18] (03PS2) 10Jforrester: dockerfiles: [composer-package*] Switch from composer-php70 to use composer [integration/config] - 10https://gerrit.wikimedia.org/r/634814 [18:44:20] (03PS2) 10Jforrester: dockerfiles: [quibble*] Switch from composer-php70 to use composer [integration/config] - 10https://gerrit.wikimedia.org/r/634815 [18:44:22] (03PS2) 10Jforrester: dockerfiles: Drop composer-package-php70, unused [integration/config] - 10https://gerrit.wikimedia.org/r/634816 [18:44:24] (03PS2) 10Jforrester: dockerfiles: Drop composer-php70, unused [integration/config] - 10https://gerrit.wikimedia.org/r/634817 [18:44:26] (03PS2) 10Jforrester: dockerfiles: Drop php70, unused [integration/config] - 10https://gerrit.wikimedia.org/r/634818 [18:44:28] (03PS2) 10Jforrester: dockerfiles: Drop php71, unused [integration/config] - 10https://gerrit.wikimedia.org/r/634819 [18:47:02] (03CR) 10Jforrester: "> Patch Set 1:" [integration/config] - 10https://gerrit.wikimedia.org/r/634811 (owner: 10Jforrester) [18:50:34] 10Gerrit, 10Patch-For-Review, 10Regression, 10Upstream: URLs in format https://gerrit.wikimedia.org/r/123456 end up on port 80, leading to error SSL_ERROR_RX_RECORD_TOO_LONG - https://phabricator.wikimedia.org/T268260 (10Dzahn) This should be fixed now. Please confirm. [18:50:51] 10Gerrit, 10Patch-For-Review, 10Regression, 10Upstream: URLs in format https://gerrit.wikimedia.org/r/123456 end up on port 80, leading to error SSL_ERROR_RX_RECORD_TOO_LONG - https://phabricator.wikimedia.org/T268260 (10Dzahn) bug and solution came from: https://bugs.chromium.org/p/gerrit/issues/detail?id... [18:56:51] 10Gerrit, 10Patch-For-Review, 10Regression, 10Upstream: URLs in format https://gerrit.wikimedia.org/r/123456 end up on port 80, leading to error SSL_ERROR_RX_RECORD_TOO_LONG - https://phabricator.wikimedia.org/T268260 (10Aklapper) Works; thanks a lot for the quick handling and patch! [19:01:04] 10Gerrit, 10Release-Engineering-Team (Development services), 10Patch-For-Review, 10Regression, 10Upstream: URLs in format https://gerrit.wikimedia.org/r/123456 end up on port 80, leading to error SSL_ERROR_RX_RECORD_TOO_LONG - https://phabricator.wikimedia.org/T268260 (10hashar) 05Open→03Resolved a:... [19:01:59] 10Release-Engineering-Team, 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10GitLab, 10User-brennen: Document GitLab Development Kit setup on-wiki - https://phabricator.wikimedia.org/T268269 (10brennen) [19:03:15] 10Gerrit, 10Security-Team, 10Security: Gerrit security advisory 2020-11-17 - https://phabricator.wikimedia.org/T268009 (10hashar) Gerrit has been upgraded. Next thing is to find out whether we can drop the work around we have applied to All-Projects / All-Users. Notably the READ permission for refs/* was qu... [19:19:36] 10LibUp: Add a list of failing repos or an indicator in lists for failing repos on libup web - https://phabricator.wikimedia.org/T268254 (10Umherirrender) Can you also look at https://libraryupgrader2.wmcloud.org/r/mediawiki/extensions/EventLogging where the web ui crashed? [19:22:00] 10LibUp: Enable xdebug on libup to run tests for mediawiki/libs/Equivset - https://phabricator.wikimedia.org/T268271 (10Umherirrender) [19:33:16] 10LibUp: Add a list of failing repos or an indicator in lists for failing repos on libup web - https://phabricator.wikimedia.org/T268254 (10Umherirrender) The following repositories have some kind of error (as of now). `mediawiki/extensions/BlueSpiceBookshelfUI` -> to archive, see T268085 `mediawiki/extensions/... [19:34:08] 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10Patch-For-Review, 10Platform Team Workboards (Clinic Duty Team): Setup Git repo and CI for shellbox - https://phabricator.wikimedia.org/T263295 (10Reedy)... [20:19:07] I'm unable to clone a repo on stats1008 from gerrit, is this intentional? [20:30:29] !log delete deployment-aqs* instances (T257118) [20:30:32] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [20:30:32] T257118: Beta cluster has reached its quota - https://phabricator.wikimedia.org/T257118 [20:30:39] !log shut down deployment-sca04 (T257118) [20:30:42] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [21:01:26] 10VPS-project-codesearch: Reduce user impact of codesearch's nightly restarts - https://phabricator.wikimedia.org/T268198 (10Ladsgroup) Is there a way for the write config not to restart if configs are the same? for example, it can load the config and check if they are the same and not write it if they are the s... [21:24:59] kostajh: like can't reach gerrit over the network? [21:25:34] brennen: yeah. Or maybe the port is blocked I don’t know. [21:26:30] brennen: maybe has something to do with https://wikitech.wikimedia.org/wiki/HTTP_proxy ? [21:28:36] kostajh: hrm, maybe? i'm trying to figure out where i'd even look for this. [21:30:32] brennen: do you have SSH access to the stats machines? I can poke around some more tomorrow, going to sleep now. [21:30:55] kostajh: don't believe so, but i'll see what i can learn. have a good night! [21:46:20] 10VPS-project-codesearch, 10Operations: Graduate codesearch to production - https://phabricator.wikimedia.org/T268199 (10Ladsgroup) In general I like the idea but I'm slightly worried with the limited time and energy we have and given that we would eventually migrate to gitlab, we could use this time and energ... [21:46:57] 10Release-Engineering-Team, 10Release-Engineering-Team-TODO, 10GitLab: Figure out submodule updating in GitLab - https://phabricator.wikimedia.org/T268283 (10brennen) [22:03:12] 10Release-Engineering-Team, 10Release-Engineering-Team-TODO, 10GitLab, 10User-brennen: Figure out submodule updating in GitLab - https://phabricator.wikimedia.org/T268283 (10brennen) [22:04:13] 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10GitLab, 10User-brennen: Figure out submodule updating in GitLab - https://phabricator.wikimedia.org/T268283 (10brennen) [22:05:16] 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10GitLab, 10User-brennen: Document GitLab Development Kit setup on-wiki - https://phabricator.wikimedia.org/T268269 (10brennen) [22:05:19] 10Gerrit, 10Release-Engineering-Team (Development services): Gerrit: remote unpack failed: Missing tree / blob - https://phabricator.wikimedia.org/T268119 (10hashar) 05Stalled→03Open T268009 is now public. That was a nasty security issue in Gerrit which is addressed by our upgrade to Gerrit version 3.2.5.... [22:05:20] 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)), 10GitLab, 10User-brennen: Figure out submodule updating in GitLab - https://phabricator.wikimedia.org/T268283 (10brennen) [22:41:27] 10Gerrit, 10Release-Engineering-Team, 10Analytics: Unable to clone git repo from stat1008 - https://phabricator.wikimedia.org/T268290 (10brennen) [22:41:53] 10Gerrit, 10Release-Engineering-Team (Development services), 10Analytics: Unable to clone git repo from stat1008 - https://phabricator.wikimedia.org/T268290 (10brennen)