[00:57:41] 10LibUp, 10User-DannyS712: Automatically bump minimum version of mediawiki for deployed extensions after new branch cut - https://phabricator.wikimedia.org/T279032 (10Krinkle) I mentioned to Danny in CR as an idea and thus support the proposal to bump these as part of existing libup sweeps (no separate commit... [01:31:36] gerrit just got really slow loading for me - is it just me? [01:34:10] 10Release-Engineering-Team (Deployment services), 10Release-Engineering-Team-TODO: Automatically tag the release task in the commit message of the weekly branch cut - https://phabricator.wikimedia.org/T278994 (10brennen) > We'd need a way to find out what the next branch task is; https://train-blockers.toolfor... [01:58:06] It's slow for me, but not excessively [01:58:57] it started working again after a few minutes, the problem was probably on my end [03:09:21] 10MediaWiki-Codesniffer, 10MediaWiki-Documentation: Standardize return type hint spacing - https://phabricator.wikimedia.org/T220719 (10DannyS712) >>! In T220719#5704728, @Reedy wrote: >>>! In T220719#5704619, @Lucas_Werkmeister_WMDE wrote: >> Can we avoid having the discussion in two places, please? There’s a... [03:20:28] 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-WikimediaMaintenance: Error while create a new wiki in beta cluster - https://phabricator.wikimedia.org/T188353 (10Reedy) 05Open→03Declined [03:22:32] 10MediaWiki-Codesniffer, 10MediaWiki-Documentation: Standardize return type hint spacing - https://phabricator.wikimedia.org/T220719 (10Reedy) p:05Triage→03Low [04:18:58] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T278343 (10DannyS712) Filed {T279127} - might be an issue with code that went out with the train [05:33:57] 10MediaWiki-Codesniffer, 10MediaWiki-Documentation: Standardize return type hint spacing - https://phabricator.wikimedia.org/T220719 (10Jdforrester-WMF) >>! In T220719#6967168, @DannyS712 wrote: >>>! In T220719#5704728, @Reedy wrote: >>>>! In T220719#5704619, @Lucas_Werkmeister_WMDE wrote: >>> Can we avoid hav... [08:52:39] (03CR) 10Hashar: "That solved the issue. The integration fullrun jobs runs under Stretch which has python 3.5 ;)" [integration/quibble] - 10https://gerrit.wikimedia.org/r/676452 (owner: 10Hashar) [09:07:48] (03PS1) 10Hashar: Merge tag 'v3.2.8' into wmf/stable-3.2 [software/gerrit] (wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/676551 (https://phabricator.wikimedia.org/T273223) [09:30:34] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T278343 (10hashar) [09:34:30] 10Beta-Cluster-Infrastructure, 10MediaWiki-Export-or-Import, 10Beta-Cluster-reproducible: Special:Export broken on beta cluster: XML Parsing Error: no root element found - https://phabricator.wikimedia.org/T278579 (10hashar) [09:35:34] 10Beta-Cluster-Infrastructure, 10MediaWiki-Export-or-Import, 10Beta-Cluster-reproducible: Special:Export broken on beta cluster: XML Parsing Error: no root element found - https://phabricator.wikimedia.org/T278579 (10hashar) [09:35:38] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T278343 (10hashar) [09:36:23] 10Beta-Cluster-Infrastructure, 10MediaWiki-Export-or-Import, 10Beta-Cluster-reproducible: Special:Export broken on beta cluster: XML Parsing Error: no root element found - https://phabricator.wikimedia.org/T278579 (10hashar) p:05Triage→03Unbreak! That should have been flagged as a train blocker for T2783... [09:36:45] 10Beta-Cluster-Infrastructure, 10MediaWiki-Export-or-Import, 10Beta-Cluster-reproducible: Special:Export broken: XML Parsing Error: no root element found - https://phabricator.wikimedia.org/T278579 (10Majavah) [09:38:00] 10Beta-Cluster-Infrastructure, 10MediaWiki-Export-or-Import, 10Beta-Cluster-reproducible: Special:Export broken: always generates an empty file - https://phabricator.wikimedia.org/T278579 (10hashar) [09:41:04] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T278343 (10hashar) [09:51:51] 10Beta-Cluster-Infrastructure, 10MediaWiki-Export-or-Import, 10Beta-Cluster-reproducible, 10Patch-For-Review: Special:Export broken: always generates an empty file - https://phabricator.wikimedia.org/T278579 (10hashar) I have rolled Mediawiki deployment from 1.36.0-wmf.37 to 1.36.0-wmf.36 and Special:Expor... [10:07:13] (03PS2) 10Hashar: Merge tag 'v3.2.8' into wmf/stable-3.2 [software/gerrit] (wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/676551 (https://phabricator.wikimedia.org/T278990) [10:08:53] 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)): Upgrade Gerrit to 3.2.7 - https://phabricator.wikimedia.org/T273223 (10hashar) ^ ignore the commit. The proper task is T278990 [10:09:03] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T278343 (10hashar) I have rolled back group 0 wikis as well since https://www.mediawiki.org/wiki/Special:Export/Main_Pag... [10:17:17] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T278343 (10Ladsgroup) [10:47:17] !log update web proxy parsoid-beta.wmflabs.org to point to deployment-parsoid12 [10:47:19] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [10:53:59] !log change deployment-wikifeeds01 config to use deployment-mediawiki11 [10:54:01] Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL [11:21:46] 10Beta-Cluster-Infrastructure, 10MediaWiki-Export-or-Import, 10Beta-Cluster-reproducible: Special:Export broken: always generates an empty file - https://phabricator.wikimedia.org/T278579 (10Ladsgroup) Update: removing `wfResetOutputBuffers` from SpecialExport fixes the issue. Don't know why it has broken in... [11:47:30] 10Beta-Cluster-Infrastructure, 10MediaWiki-Export-or-Import, 10Beta-Cluster-reproducible: Special:Export broken: always generates an empty file - https://phabricator.wikimedia.org/T278579 (10Ladsgroup) It's caused by https://gerrit.wikimedia.org/r/c/mediawiki/core/+/674196 and it has also broken using gzip i... [12:13:52] 10Beta-Cluster-Infrastructure, 10MediaWiki-Export-or-Import, 10Beta-Cluster-reproducible: Special:Export broken: always generates an empty file - https://phabricator.wikimedia.org/T278579 (10Reedy) >>! In T278579#6967485, @hashar wrote: > That should have been flagged as a train blocker for T278343 since tha... [12:14:54] 10Beta-Cluster-Infrastructure, 10MediaWiki-Export-or-Import, 10Beta-Cluster-reproducible, 10MW-1.35-release: Special:Export broken: always generates an empty file - https://phabricator.wikimedia.org/T278579 (10Reedy) >>! In T278579#6967698, @Ladsgroup wrote: > It's caused by https://gerrit.wikimedia.org/r/... [12:31:59] 10Beta-Cluster-Infrastructure, 10MediaWiki-Export-or-Import, 10Beta-Cluster-reproducible, 10MW-1.35-release, 10Patch-For-Review: Special:Export broken: always generates an empty file - https://phabricator.wikimedia.org/T278579 (10Reedy) I thought the revert would be worse than it was as I thought https:/... [12:52:48] 10Beta-Cluster-Infrastructure, 10Cloud-VPS (Debian Jessie Deprecation): Migrate deployment-prep away from Debian Jessie to Debian Stretch/Buster - https://phabricator.wikimedia.org/T218729 (10Krenair) Thanks @Majavah [13:19:02] 10Beta-Cluster-Infrastructure, 10MediaWiki-Export-or-Import, 10Beta-Cluster-reproducible, 10MW-1.35-notes, and 2 others: Special:Export broken: always generates an empty file - https://phabricator.wikimedia.org/T278579 (10Reedy) Confirmed fixed on beta [13:30:24] (03CR) 10Hashar: [C: 03+2] Merge tag 'v3.2.8' into wmf/stable-3.2 [software/gerrit] (wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/676551 (https://phabricator.wikimedia.org/T278990) (owner: 10Hashar) [13:35:03] (03Merged) 10jenkins-bot: Merge tag 'v3.2.8' into wmf/stable-3.2 [software/gerrit] (wmf/stable-3.2) - 10https://gerrit.wikimedia.org/r/676551 (https://phabricator.wikimedia.org/T278990) (owner: 10Hashar) [13:43:20] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T278343 (10hashar) The Special:Export issue has been figured out and confirmed to be fixed on beta. So I have rolled all... [13:44:35] 10Release-Engineering-Team-TODO (2021-01-01 to 2021-03-31 (Q3)), 10Patch-For-Review, 10Release, 10Train Deployments: 1.36.0-wmf.37 deployment blockers - https://phabricator.wikimedia.org/T278343 (10hashar) [13:44:39] 10Beta-Cluster-Infrastructure, 10MediaWiki-Export-or-Import, 10Beta-Cluster-reproducible, 10MW-1.35-notes, and 2 others: Special:Export broken: always generates an empty file - https://phabricator.wikimedia.org/T278579 (10hashar) 05Open→03Resolved Confirmed to be fixed on production as well. Congratula... [13:46:08] Ty hashar [13:51:00] \o/ [13:57:53] what's the policy around https://wikitech.wikimedia.org/wiki/Deployment_pipeline deployments? [13:58:31] if I have a quick fix for a service which doesn't get production traffic yet, is it OK to deploy it on Friday? [13:59:24] hashar: it's nice when go for a coffee and come back to a fix [14:35:13] RhinosF1: definitely yes [14:35:38] tr [15:34:42] tgr|away: late response, there's not much of a policy just yet. I'm ok with the Friday deploy in principle (in terms of it seems low risk from the service perspective); however, not sure the risk to cluster -- would be good to get an sre +1 [18:24:33] 10Release-Engineering-Team (Logspam), 10VisualEditor, 10Editing-team (Tracking), 10JavaScript, 10Wikimedia-production-error: Uncaught TypeError: Cannot read property 'node' of null (in importTable ) - https://phabricator.wikimedia.org/T262842 (10Jdlrobson) The logstash board has been updated to filter th... [19:26:42] (03Abandoned) 10Zabe: Edit Repo Config [sandbox] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/676514 (owner: 10Zabe) [19:28:19] 10Continuous-Integration-Config, 10Better Use Of Data, 10Product-Data-Infrastructure: Add CI jobs for mediawiki/libs/metrics-platform - https://phabricator.wikimedia.org/T279180 (10Mholloway) [20:30:57] (03PS1) 10QChris: Allow “Gerrit Managers” to import history [gadgets/TwinkleCore] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/676661 [20:30:59] (03CR) 10QChris: [V: 03+2 C: 03+2] Allow “Gerrit Managers” to import history [gadgets/TwinkleCore] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/676661 (owner: 10QChris) [20:31:05] (03PS1) 10QChris: Import done. Revoke import grants [gadgets/TwinkleCore] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/676662 [20:31:07] (03CR) 10QChris: [V: 03+2 C: 03+2] Import done. Revoke import grants [gadgets/TwinkleCore] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/676662 (owner: 10QChris) [20:34:16] (03PS1) 10QChris: Allow “Gerrit Managers” to import history [debs/postorius] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/676665 [20:34:19] (03CR) 10QChris: [V: 03+2 C: 03+2] Allow “Gerrit Managers” to import history [debs/postorius] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/676665 (owner: 10QChris) [20:34:31] (03PS1) 10QChris: Import done. Revoke import grants [debs/postorius] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/676666 [20:34:34] (03CR) 10QChris: [V: 03+2 C: 03+2] Import done. Revoke import grants [debs/postorius] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/676666 (owner: 10QChris) [20:36:36] (03PS1) 10QChris: Allow “Gerrit Managers” to import history [debs/mailman3] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/676667 [20:36:39] (03CR) 10QChris: [V: 03+2 C: 03+2] Allow “Gerrit Managers” to import history [debs/mailman3] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/676667 (owner: 10QChris) [20:36:44] (03PS1) 10QChris: Import done. Revoke import grants [debs/mailman3] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/676668 [20:36:47] (03CR) 10QChris: [V: 03+2 C: 03+2] Import done. Revoke import grants [debs/mailman3] (refs/meta/config) - 10https://gerrit.wikimedia.org/r/676668 (owner: 10QChris) [22:21:15] 10Release-Engineering-Team (Logspam), 10MediaWiki-extensions-Graph, 10Reading-Web-Local-Wiki-Issues, 10Wikimedia-production-error: Error: Bad constructor arguments (Asynchronously loaded graphs error) - https://phabricator.wikimedia.org/T277906 (10Jdlrobson) Fixed on multiple wikis with this kind of edit t... [22:21:18] 10Release-Engineering-Team (Logspam), 10MediaWiki-extensions-Graph, 10Reading-Web-Local-Wiki-Issues, 10Wikimedia-production-error: Error: Bad constructor arguments (Asynchronously loaded graphs error) - https://phabricator.wikimedia.org/T277906 (10Jdlrobson) will close next week once i've confirmed the iss... [22:30:37] 10Release-Engineering-Team (Logspam), 10VisualEditor, 10Editing-team (Tracking), 10JavaScript, 10Wikimedia-production-error: Uncaught Error: Remover past end at VeDmTreeModifier.ve.dm.TreeModifier.processRetain - https://phabricator.wikimedia.org/T264956 (10Jdlrobson) Now treating this as logspam and fil... [22:35:41] 10Release-Engineering-Team (Logspam), 10Growth-Team, 10MediaWiki-Interface, 10MediaWiki-User-login-and-signup, and 2 others: Uncaught TypeError: Cannot read property 'slice' of undefined (during account creation) - https://phabricator.wikimedia.org/T264960 (10Jdlrobson) Filtering by default as logspam {F34... [22:50:10] brennen: fyi https://gerrit.wikimedia.org/r/c/labs/tools/wikibugs2/+/676691/ [23:12:14] 10Release-Engineering-Team (Logspam), 10ContentTranslation, 10JavaScript, 10Wikimedia-production-error: Uncaught TypeError: Cannot read property 'offsetHeight' of null - https://phabricator.wikimedia.org/T261992 (10Jdlrobson) [23:12:50] 10Release-Engineering-Team (Logspam), 10MediaWiki-extensions-Graph, 10Reading-Web-Local-Wiki-Issues, 10User-Jdlrobson, 10Wikimedia-production-error: Error: Bad constructor arguments (Asynchronously loaded graphs error) - https://phabricator.wikimedia.org/T277906 (10Jdlrobson) a:03Jdlrobson [23:13:27] 10Release-Engineering-Team (Logspam), 10VisualEditor, 10Editing-team (Tracking), 10JavaScript, and 2 others: Cannot read property 'focus' of undefined on mobile at VeCeSurface.ve.ce.Surface.focus - https://phabricator.wikimedia.org/T265156 (10Jdlrobson) [23:13:41] 10Release-Engineering-Team (Logspam), 10Editing-team, 10WikiEditor, 10JavaScript, 10Wikimedia-production-error: Uncaught TypeError: Cannot read property 'end' of null - https://phabricator.wikimedia.org/T264959 (10Jdlrobson) [23:14:03] 10Release-Engineering-Team (Logspam), 10Kaltura player, 10JavaScript, 10Wikimedia-production-error: TypeError: embedPlayer.controlBuilder is undefined / self is undefined / TypeError: embedPlayer is undefined / TypeError: undefined is not an object (evaluating 'e... - https://phabricator.wikimedia.org/T262623 [23:14:32] 10Release-Engineering-Team (Logspam), 10Editing-team, 10VisualEditor, 10Wikimedia-production-error: Error: Unbalanced set of replace operations found - https://phabricator.wikimedia.org/T277594 (10Jdlrobson) [23:14:51] 10Release-Engineering-Team (Logspam), 10VisualEditor, 10Wikimedia-production-error: TypeError: Cannot read property 'node' of undefined / Error: domNode has no ancestor with a .data( 'view' ) - https://phabricator.wikimedia.org/T279169 (10Jdlrobson)