[01:10:01] <James_F>	 Lovely, phan failure on only the second 0.10.4 upgrade LibUp attempts: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/MassMessage/+/641580
[01:10:22] <James_F>	 Ideally we'd move phan jobs into composer test so LibUp spots them and knows not to push.
[03:58:42] <tgr_>	 this is new
[03:58:56] <tgr_>	 $ git review
[03:59:06] <tgr_>	 error: remote unpack failed: error Missing blob 4966483667bd342047378b6c9d90f860d8e1ae86
[04:00:20] <tgr_>	 what does that even mean? I have something in the git parent tree that has been deleted from the central repo?
[04:33:42] <Reedy>	 Gerrit seems a bit unhappy atm
[07:04:06] <wikibugs>	 10Release-Engineering-Team-TODO, 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T263184 (10Tgr) We found the underlying issue (it was specific to the extension) so the logspam will be fixed soon.
[08:28:41] <hashar>	 damn git
[08:28:51] <hashar>	 I have learned yet another sub command:  git bundle
[09:14:50] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)): Gerrit link to Sonar cloud reports are broken - https://phabricator.wikimedia.org/T267028 (10awight) Verified working in production :-)
[09:31:31] <wikibugs>	 10Gerrit: Gerrit: remote unpack failed: Missing tree / blob - https://phabricator.wikimedia.org/T268119 (10Tgr)
[10:04:47] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services): Gerrit: remote unpack failed: Missing tree / blob - https://phabricator.wikimedia.org/T268119 (10hashar) I believe the server side stacktrace is: `counterexample [2020-11-18 09:23:31,379] [SSH git-receive-pack /mediawiki/core.git (tgr)] ERROR com.goo...
[10:20:01] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services): Gerrit: remote unpack failed: Missing tree / blob - https://phabricator.wikimedia.org/T268119 (10hashar) 05Open→03Stalled a:03hashar The first error occurred at 2020-11-17 15:02:35 UTC. And there is no other match when I crawl all the way up to...
[10:20:18] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services): Gerrit: remote unpack failed: Missing tree / blob - https://phabricator.wikimedia.org/T268119 (10hashar)
[10:25:53] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services): Gerrit: remote unpack failed: Missing tree / blob - https://phabricator.wikimedia.org/T268119 (10Daimona) FTR, I experienced this error when pushing to core as well, but it was fixed after running `git fetch --all`. HTH
[10:32:11] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (2020-10-01 to 2020-12-31 (Q2)): Gerrit link to Sonar cloud reports are broken - https://phabricator.wikimedia.org/T267028 (10hashar) 05Open→03Resolved Thank you so much @Dzahn for the merge and @awight for the v...
[14:40:26] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services): Gerrit: remote unpack failed: Missing tree / blob - https://phabricator.wikimedia.org/T268119 (10daniel) Me too:  ` error: remote unpack failed: error Missing tree 42ec2840a99f9e73c57781e9ac77bf7ac153e104 fatal: Unpack error, check server log `  On a...
[14:53:16] <wikibugs>	 (03CR) 10Jakob: "This change is ready for review." [releng/dev-images] - 10https://gerrit.wikimedia.org/r/641739 (owner: 10Jakob)
[15:45:16] <wikibugs>	 10Release-Engineering-Team-TODO, 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.18 deployment blockers - https://phabricator.wikimedia.org/T263184 (10dancy) >>! In T263184#6629462, @Tgr wrote: > We found the underlying issue (it was specific to the extension) so the logspam will be fixed so...
[16:27:22] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services): Gerrit: remote unpack failed: Missing tree / blob - https://phabricator.wikimedia.org/T268119 (10thcipriani) >>! In T268119#6629949, @hashar wrote: > The devil is finding out what `Missing tree 303f2006e325c9665c23e722fccf5c83f1c97d3b` is about. >  >...
[16:36:42] <wikibugs>	 (03CR) 10Lucas Werkmeister (WMDE): Anchor regex for MW_SCRIPT_PATH change from `/` to `/w` (031 comment) [releng/dev-images] - 10https://gerrit.wikimedia.org/r/641739 (owner: 10Jakob)
[16:51:13] <wikibugs>	 (03PS2) 10Jakob: Anchor regex for MW_SCRIPT_PATH change from `/` to `/w` [releng/dev-images] - 10https://gerrit.wikimedia.org/r/641739
[16:51:30] <wikibugs>	 (03CR) 10Jakob: Anchor regex for MW_SCRIPT_PATH change from `/` to `/w` (031 comment) [releng/dev-images] - 10https://gerrit.wikimedia.org/r/641739 (owner: 10Jakob)
[19:48:48] <wikibugs>	 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO, 10Operations, 10serviceops: Upgrade MediaWiki appservers to Debian Buster (debian 10) - https://phabricator.wikimedia.org/T245757 (10jijiki)
[19:50:18] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services): Remove jsonevent-layout library from our Gerrit fork - https://phabricator.wikimedia.org/T268020 (10hashar) From Luca, we can put the extra `.jar` in `$GERRIT_SITE/lib` like we did previously for the mysql connector.  Gerrit adds that path to CLASSPA...
[20:03:15] <James_F>	 Daimona_: Hey, am I missing something or are some of the "sub" tasks of T213006 actually parent tasks, and once we've finished the run and dropped the back-compat. they'll be done? E.g. T34478 T110854 etc.?
[20:03:16] <stashbot>	 T34478: AbuseFilter not setting utf-8 flag - https://phabricator.wikimedia.org/T34478
[20:03:17] <stashbot>	 T110854: Blank afl_var_dump on 9 testwiki abuse_filter_log entries from July 2012 causes exceptions - https://phabricator.wikimedia.org/T110854
[20:03:18] <stashbot>	 T213006: Create a script to update afl_var_dump, drop back-compat code - https://phabricator.wikimedia.org/T213006
[20:58:08] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services): Remove jsonevent-layout library from our Gerrit fork - https://phabricator.wikimedia.org/T268020 (10QChris) >>! In T268020#6631592, @hashar wrote: > From Luca, we can put the extra `.jar` in `$GERRIT_SITE/lib` like we did previously for the mysql con...
[21:07:49] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services): Remove jsonevent-layout library from our Gerrit fork - https://phabricator.wikimedia.org/T268020 (10QChris) >>! In T268020#6631592, @hashar wrote: > From Luca, we can put the extra `.jar` in `$GERRIT_SITE/lib` like we did previously for the mysql con...
[21:13:47] <hashar>	 qchris: thank you so much for preventing me to break the world :-]
[21:14:44] * qchris humbly bows to hashar who obviously has the power break the world :-)
[21:16:46] <qchris>	 I mean, the jar loading might work with Flogger being somewhat lazy. So it might be worth trying.
[21:25:53] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services): Remove jsonevent-layout library from our Gerrit fork - https://phabricator.wikimedia.org/T268020 (10hashar) Thank you for mentioning load order might be an issue and actually has hit us in the past. I will give it a try at least for educational purpo...
[21:50:11] <wikibugs>	 10Beta-Cluster-Infrastructure, 10VisualEditor: Server returned error: HTTP 500 appears while trying to open VE or reply to a comment on Beta cluster - https://phabricator.wikimedia.org/T268184 (10Ryasmeen)
[21:58:03] <Daimona_>	 James_F: There's some confusion with those tasks. The main problem being that T213006 was originally just a "create the script", but has become "create the script, run it, and drop BC code". So the relation with other tasks depend on how you interpret it
[21:58:03] <stashbot>	 T213006: Create a script to update afl_var_dump, drop back-compat code - https://phabricator.wikimedia.org/T213006
[21:58:54] <Daimona_>	 And BTW, thank you for the CR!
[22:12:31] <DannyS712>	 logstash-beta appears down (502 Bad Gateway nginx/1.13.6) - is this a known issue / has it already been reported?
[22:14:33] <James_F>	 Daimona_: Happy to help. For the tasks that can't be fixed until the code is dropped, they should be marked as blocked by T213006, not the other way around. If we have to do one of the tasks before we can finish running the script, let's get them done, but AFAICR there's nothing left beyond running it?
[22:14:33] <stashbot>	 T213006: Create a script to update afl_var_dump, drop back-compat code - https://phabricator.wikimedia.org/T213006
[22:16:52] <Daimona_>	 Good point, let me reorganize things a bit
[22:17:10] <James_F>	 Sure, or I can. :-)
[22:17:25] <Daimona_>	 And yes, running the script is the very last step. Then we only have to kill BC code with fire
[22:17:32] <James_F>	 Hurrah.
[22:17:47] <James_F>	 Daimona_: https://gerrit.wikimedia.org/r/c/mediawiki/extensions/AbuseFilter/+/547724 can be merged too?
[22:17:54] <James_F>	 (Also this is the wrong channel for this, but oh well.)
[22:18:57] <Daimona_>	 Yes, it should be ready, thank you
[22:19:47] <wikibugs>	 10Beta-Cluster-Infrastructure, 10VisualEditor: Server returned error: HTTP 500 appears while trying to open VE or reply to a comment on Beta cluster - https://phabricator.wikimedia.org/T268184 (10DannyS712) Not specific to VE / discussion tools - it seems all api calls on the beta cluster are returning 500 (eg...
[22:19:58] <Daimona_>	 Actually, that BC code will just become dead code once the script has finished, so the removal isn't blocking anything AFAICT
[22:20:00] <wikibugs>	 10Beta-Cluster-Infrastructure: Server returned error: HTTP 500 appears while trying to open VE or reply to a comment on Beta cluster - https://phabricator.wikimedia.org/T268184 (10DannyS712) p:05Triage→03High
[22:29:51] <wikibugs>	 10VPS-project-codesearch, 10WikimediaDebug, 10Performance-Team (Radar): Add performance/WikimediaDebug to Codesearch - https://phabricator.wikimedia.org/T268187 (10Krinkle)
[22:37:08] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services): Gerrit: remote unpack failed: Missing tree / blob - https://phabricator.wikimedia.org/T268119 (10Tgr) I did not enable the v2 protocol.
[22:39:22] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services): Gerrit: remote unpack failed: Missing tree / blob - https://phabricator.wikimedia.org/T268119 (10hashar) >>! In T268119#6632153, @Tgr wrote: > I did not enable the v2 protocol.  Great thank you for confirming. I guess that rules out a protocol issue....
[23:07:45] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Analytics, 10Event-Platform, 10Patch-For-Review: Server returned error: HTTP 500 appears while trying to open VE or reply to a comment on Beta cluster - https://phabricator.wikimedia.org/T268184 (10Jdforrester-WMF)
[23:25:37] <wikibugs>	 10VPS-project-codesearch, 10WikimediaDebug, 10Performance-Team (Radar), 10User-Ladsgroup: Add performance/WikimediaDebug to Codesearch - https://phabricator.wikimedia.org/T268187 (10Ladsgroup) 05Open→03Resolved a:03Ladsgroup Done now: https://codesearch.wmcloud.org/search/?q=mwdebug1001&i=nope&files=...