[02:04:12] <wikibugs>	 10Beta-Cluster-Infrastructure: Global developer for DannyS712 on beta cluster - https://phabricator.wikimedia.org/T235650 (10DannyS712) >>! In T235650#5579876, @MarcoAurelio wrote: > Global permissions are granted through [[ https://deployment.wikimedia.beta.wmflabs.org/wiki/Special:GlobalUserRights | Special:Gl...
[05:52:38] <wikibugs>	 10Release-Engineering-Team, 10MediaWiki-User-management, 10MediaWiki-extensions-FlaggedRevs, 10User-DannyS712: Pending changes: autoreview randomly fails - https://phabricator.wikimedia.org/T233561 (10binbot) Other phenomena in huwiki not mentioned by this time:    - the "undo" link is randomly missing   -...
[07:22:45] <wikibugs>	 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-WikibaseClient, 10Wikidata, 10Wikidata-Campsite: beta testwiki site is broken: "Cannot access the database: Unknown error" - https://phabricator.wikimedia.org/T235970 (10Lydia_Pintscher)
[07:31:50] <awight>	 Something is corrupted in the deployments calendar template: the EU "SWAT" group was pasted into every SWAT window.
[07:33:49] <awight>	 Actually, the template is fine but the current calendar is wrong.
[07:35:06] <awight>	 thcipriani: Looks like the deployments-calendar tool is glitching: https://wikitech.wikimedia.org/w/index.php?title=Deployments&diff=1841526&oldid=1841525
[08:13:38] <wikibugs>	 (03PS2) 10Hashar: Add labs/tools/watch-translations [integration/config] - 10https://gerrit.wikimedia.org/r/544412 (owner: 10Urbanecm)
[08:13:59] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] "Updated with a more specific job: labs-tools-watch-translations-tox-docker" [integration/config] - 10https://gerrit.wikimedia.org/r/544412 (owner: 10Urbanecm)
[08:16:42] <wikibugs>	 (03Merged) 10jenkins-bot: Add labs/tools/watch-translations [integration/config] - 10https://gerrit.wikimedia.org/r/544412 (owner: 10Urbanecm)
[08:41:50] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] layout: [skins/MonoBook] Run phan [integration/config] - 10https://gerrit.wikimedia.org/r/544385 (owner: 10Daimona Eaytoy)
[08:43:59] <wikibugs>	 (03Merged) 10jenkins-bot: layout: [skins/MonoBook] Run phan [integration/config] - 10https://gerrit.wikimedia.org/r/544385 (owner: 10Daimona Eaytoy)
[08:54:30] <wikibugs>	 10Phabricator, 10User-DannyS712: Form 60 has extraneous whitespace at the top - https://phabricator.wikimedia.org/T236018 (10DannyS712)
[09:00:11] <James_F>	 hashar: You on https://gerrit.wikimedia.org/r/c/integration/config/+/544245 ?
[09:00:22] <hashar>	 hmm
[09:00:29] <hashar>	 James_F: yeah will do it today :]
[09:01:18] <hashar>	 also have to look at removing the debug log spam caused by DevWebServer 
[09:01:23] <hashar>	 and dish out hhvm from quibble
[09:03:47] <wikibugs>	 10Release-Engineering-Team (CI & Testing services), 10Quibble, 10HHVM: Remove HHVM support from Quibble - https://phabricator.wikimedia.org/T236019 (10hashar)
[09:03:55] <wikibugs>	 10Release-Engineering-Team (CI & Testing services), 10Quibble, 10HHVM: Remove HHVM support from Quibble - https://phabricator.wikimedia.org/T236019 (10hashar) p:05Triage→03Normal
[09:04:50] <hashar>	 !log Restarting CI Jenkins for plugin upgrades
[09:04:51] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[09:08:09] * James_F nods.
[09:10:12] <wikibugs>	 (03PS2) 10Hashar: jjb: Bump all quibble jobs to 0.0.39 [integration/config] - 10https://gerrit.wikimedia.org/r/544245 (owner: 10Jforrester)
[09:10:17] <hashar>	 James_F: doing it now :]
[09:12:14] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] "The job for api-testing required a bump from 0.0.39 to 0.0.39-s1 . Beside that: all good :]" [integration/config] - 10https://gerrit.wikimedia.org/r/544245 (owner: 10Jforrester)
[09:12:35] <hashar>	 !log update Quibble jobs to container 0.0.39-s1
[09:12:36] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[09:14:35] <wikibugs>	 (03Merged) 10jenkins-bot: jjb: Bump all quibble jobs to 0.0.39 [integration/config] - 10https://gerrit.wikimedia.org/r/544245 (owner: 10Jforrester)
[09:16:44] <wmf-insecte>	 Yippee, build fixed!
[09:16:45] <wmf-insecte>	 Project beta-scap-eqiad build #272137: 09FIXED in 2 min 15 sec: https://integration.wikimedia.org/ci/job/beta-scap-eqiad/272137/
[09:23:49] <wikibugs>	 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO: Check whether subversion is actually needed on Phabricator hosts - https://phabricator.wikimedia.org/T236026 (10hashar)
[09:27:19] <wikibugs>	 (03CR) 10Hashar: [C: 03+1] layout: add publication jobs for restbase dev image [integration/config] - 10https://gerrit.wikimedia.org/r/540976 (https://phabricator.wikimedia.org/T234580) (owner: 10Jeena Huneidi)
[09:29:57] <wikibugs>	 10Continuous-Integration-Config, 10Browser-Tests, 10Wikimedia-production-error (Shared Build Failure): Selenium tests failing for PluggableAuth plugins - https://phabricator.wikimedia.org/T236030 (10Daimona)
[09:36:33] <wikibugs>	 (03PS1) 10Jforrester: layout: [wikimedia/textcat] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544808
[09:36:38] <wikibugs>	 (03PS1) 10Jforrester: layout: [purtle] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544809
[09:36:40] <wikibugs>	 (03PS1) 10Jforrester: layout: [utfnormal] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544810
[09:36:42] <wikibugs>	 (03PS1) 10Jforrester: layout: [php-sesson-serlizer] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544811
[09:36:44] <wikibugs>	 (03PS1) 10Jforrester: layout: [testing-access-wrapper] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544812
[09:36:46] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/tools/minus-x] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544813
[09:36:48] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/tools/phpunit-patch-coverage] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544814
[09:36:50] <wikibugs>	 (03PS1) 10Jforrester: layout: [wikimedia/lucene-explain-parser] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544815
[09:36:52] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/services/jobrunner] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544816
[09:36:53] <wikibugs>	 10Phabricator, 10User-DannyS712: Form 60 has extraneous whitespace at the top - https://phabricator.wikimedia.org/T236018 (10Aklapper) 05Open→03Resolved a:03Aklapper
[09:36:56] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/oauthclient-php] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544817
[09:36:58] <wikibugs>	 (03PS1) 10Jforrester: layout: [cdb] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544818
[09:37:00] <wikibugs>	 (03PS1) 10Jforrester: layout: [at-ease] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544819
[09:37:02] <wikibugs>	 (03PS1) 10Jforrester: layout: [base-convert] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544820
[09:37:04] <wikibugs>	 (03PS1) 10Jforrester: layout: [css-sanitzer] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544821
[09:37:06] <wikibugs>	 (03PS1) 10Jforrester: layout: [AhoCorasick] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544822
[09:37:08] <wikibugs>	 (03PS1) 10Jforrester: layout: [RunningStat] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544823
[09:37:10] <wikibugs>	 (03PS1) 10Jforrester: layout: [IPSet] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544824
[09:37:12] <wikibugs>	 (03PS1) 10Jforrester: layout: [RelPath] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544825
[09:37:14] <wikibugs>	 (03PS1) 10Jforrester: layout: [WrappedString] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544826
[09:37:16] <wikibugs>	 (03PS1) 10Jforrester: layout: [CLDRPluralRuleParser] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544827
[09:37:18] <wikibugs>	 (03PS1) 10Jforrester: layout: [HtmlFormatter] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544828
[09:39:16] <Daimona>	 Poor zuul
[09:40:10] <hashar>	 James_F: what would happen to the old mediawiki versions such as REL1_31 ? 
[09:40:21] <hashar>	 should we stop testing 7.0/7.1 ?
[09:40:35] <James_F>	 No, we'll keep that for now.
[09:40:46] <James_F>	 But all our libraries are for use with MW master.
[09:40:53] <James_F>	 We don't do old branches of anything other than MW.
[09:41:05] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/libs/Assert] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544830
[09:41:07] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/libs/Equivset] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544831
[09:41:09] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/libs/etcd] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544832
[09:41:10] <hashar>	 I am too much in paranoia maybe
[09:41:11] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/libs/IPUtils] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544833
[09:41:13] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/libs/ObjectFactory] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544834
[09:41:15] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/libs/PasswordBlacklist] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544835
[09:41:17] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/libs/RemexHtml] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544836
[09:41:19] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/libs/ScopedCallback] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544837
[09:41:21] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/libs/Services] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544838
[09:41:22] <hashar>	 but imagine  IPSet get some php7.2+ code and end up being bumped in REL1_31
[09:41:23] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/libs/Timestamp] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544839
[09:41:25] <hashar>	 that might cause issue
[09:41:25] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/libs/WaitConditionLoop] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544840
[09:41:27] <wikibugs>	 (03PS1) 10Jforrester: layout: [mediawiki/libs/XMPReader] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544841
[09:41:29] <wikibugs>	 (03PS1) 10Jforrester: layout: [wikimedia/slimapp] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544842
[09:41:31] <wikibugs>	 (03PS1) 10Jforrester: layout: [avro-php] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544843
[09:41:35] <James_F>	 hashar: In that extremely exceptional case, we'll see it when we test it.
[09:41:39] <wikibugs>	 10Phabricator (Upstream), 10Developer-Advocacy, 10Legalpad, 10translatewiki.net, and 3 others: Use translatewiki.net to localize Phabricator - https://phabricator.wikimedia.org/T225 (10Aklapper) That is a question for a TranslateWiki support channel, I'd say...
[09:41:39] <hashar>	 though we can just make all those php libs to have require php>7.2
[09:41:43] <James_F>	 Composer will error in update.php
[09:41:46] <James_F>	 That's the plan, yes.
[09:41:50] <hashar>	 ok ;]
[09:41:52] <James_F>	 See https://gerrit.wikimedia.org/r/c/wikimedia/textcat/+/544798 etc.
[09:42:27] <hashar>	 we really need a global CI job that enforces a bunch of generic rules :-\
[09:42:42] <hashar>	 ah 
[09:42:47] <James_F>	 Initially prompted by LibUp being unable to upgrade MW-codesniffer in a bunch of repos.
[09:42:51] <hashar>	 composer.json changes are all other the place. That is great
[09:42:53] <James_F>	 Yeah, maybe.
[09:43:25] <Daimona>	 Unrelated, hashar, James_F: would it be feasible to have some system in CI that would run against the master of all existing repos, say once a week? There are so many extensions where the build is failing on master
[09:44:24] <Daimona>	 Knowing it in a timely manner would make it easier to fix them, instead of having to understand what happened 3 months ago that broke the (unmaintained) extension
[09:44:34] <James_F>	 Yeah, was thinking about that.
[09:44:37] <hashar>	 Daimona: yeah Reedy has filled a task about it a while back
[09:44:47] <James_F>	 It'd be do-able, except with self-managed CI it'll be harder to do.
[09:44:54] <hashar>	 Daimona: I gave it a try but given we have a thousand of repositories , that is a bit though :\
[09:44:57] <Daimona>	 Ohhhhh interesting
[09:45:03] <Daimona>	 Yeah, well
[09:45:15] <wikibugs>	 10Beta-Cluster-Infrastructure, 10MediaWiki-extensions-WikibaseClient, 10Wikidata, 10Wikidata-Campsite: beta testwiki site is broken: "Unknown database 'testwikidatawiki'" - https://phabricator.wikimedia.org/T235970 (10Jdforrester-WMF)
[09:45:18] <hashar>	 I think I even wrote an implementation with a Zuul pipeline that is triggers once per week
[09:45:20] <Daimona>	 One could limit that to repos where no tests have been run in the past week, but that would still be the majority of repos I guess
[09:45:25] <hashar>	 and bound to X changes max 
[09:45:29] <James_F>	 It would, yes.
[09:45:40] <hashar>	 and gave up when I realized that eg at 3am on Sunday we would enqueue a thousand of changes
[09:45:43] <James_F>	 In practice, LibUp finds error en-mass every time we upgrade. :-(
[09:46:00] <hashar>	 so I don't think Zuul is adequate there :-\   We would probably want a build bot
[09:46:02] <James_F>	 hashar: See https://gerrit.wikimedia.org/r/q/owner:tools.libraryupgrader%2540tools.wmflabs.org+is:open
[09:46:23] <James_F>	 All the failures are "master is broken" failures (or "master is still trying to run PHP7.0/7.1").
[09:46:52] <Daimona>	 Yeah, the queue would explode very quickly I believe
[09:47:32] <icinga-wm>	 PROBLEM - Work requests waiting in Zuul Gearman server on contint1001 is CRITICAL: CRITICAL: 71.43% of data above the critical threshold [140.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1
[09:48:00] <Daimona>	 ^ Heh, by the way :)
[09:48:32] <hashar>	 !log jenkins-job-builder : remove cherry-pick which is no more needed (to support multiple macro in conditional-step builder https://review.openstack.org/#/c/471030/1 ) == rollback from a06d173e98c to tag 2.0.2
[09:48:34] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[09:48:59] <hashar>	 one less hack
[09:50:31] <hashar>	 I should really write tasks
[09:50:57] <hashar>	 Daimona: so yeah what could be done is to create a bot that list all repos,  then run the quibble and phan job and report whatever faied
[09:51:06] <hashar>	 with the library upgrader more or less serving that purposes now
[09:51:09] <hashar>	 and ideally
[09:51:24] <hashar>	 I would love to split Wikimedia deployed extension to their own namespace in Gerrit
[09:51:52] <Daimona>	 Oh, a stand-alone system, so
[09:52:33] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Wikimedia-maintenance-script-run: Run maintenance/cleanupUsersWithNoId.php on beta testwiki - https://phabricator.wikimedia.org/T235971 (10MarcoAurelio) Ping to @Anomie as I think he has the most experience with this task.
[09:53:45] <Daimona>	 Well, actually, one could use smarter scheduling. E.g. one repo every X time, but making it so that it's once a week for every repo. Maybe doing it off-peak
[09:54:16] <Daimona>	 Dunno if that would be easier to do than a standalone bot, tho
[09:54:19] <wikibugs>	 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Jenkins: Align jenkins-job-builder with upstream - https://phabricator.wikimedia.org/T236036 (10hashar)
[09:54:20] <James_F>	 hashar: If you're going to cut a new version Quibble, T236030 might be worth doing there?
[09:54:21] <stashbot>	 T236030: Selenium tests failing for PluggableAuth plugins - https://phabricator.wikimedia.org/T236030
[09:55:00] <hashar>	 James_F: no idea :\
[09:55:02] <James_F>	 And yes, run it at ~ 03:00 UTC on Saturdays. Very quiet time for gerrit/CI.
[09:55:07] <hashar>	 we can't quite inject config settings now
[09:55:34] <wikibugs>	 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Jenkins, 10Technical-Debt: Align jenkins-job-builder with upstream - https://phabricator.wikimedia.org/T236036 (10hashar)
[09:55:39] <hashar>	 anyway
[09:55:48] <hashar>	 gotta deliver some outfit to one of my kid nearby
[09:55:51] <hashar>	 brb :]
[10:05:09] <hashar>	 Daimona: James_F: and that thousand of repo would probably need to have jobs run for each of the supported branches :\
[10:06:04] <Daimona>	 Well... Jobs for non-master branches can be more relaxed, e.g. once per month
[10:06:29] <James_F>	 Or never.
[10:06:36] <Daimona>	 That would solve it :D
[10:06:42] <wikibugs>	 (03PS1) 10Hashar: Use jenkins-job-builder from pypi [integration/config] - 10https://gerrit.wikimedia.org/r/544851 (https://phabricator.wikimedia.org/T236036)
[10:07:36] <hauskater>	 lol
[10:08:19] <Daimona>	 Uhm, what is zuul doing right now?
[10:11:13] <hashar>	 you have killed it
[10:11:13] <wikibugs>	 10MediaWiki-Codesniffer, 10LibUp: Upgrade PHPCS to 28.0.0 in all repos - https://phabricator.wikimedia.org/T235113 (10Jdforrester-WMF) Status update: Per [[https://libraryupgrader2.wmflabs.org/library/composer/mediawiki/mediawiki-codesniffer | LibUp's tracking]], 177 extensions and 9 skins (including all of Wi...
[10:13:19] <Daimona>	 RIP :[
[10:14:52] <hauskater>	 Guys zuul is a bit... ehm... crowded?
[10:15:45] <hashar>	 !log contint1001 / contint2001 : marking integration/config zuul merger repo readonly:   sudo chown -R root:root /srv/zuul/git/integration/config
[10:15:47] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[10:16:14] <James_F>	 Stack a bit too tall, I guess.
[10:17:46] <Daimona>	 Future idea: implement distributed computing in CI!
[10:18:07] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [wikimedia/textcat] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544808 (owner: 10Jforrester)
[10:18:19] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [purtle] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544809 (owner: 10Jforrester)
[10:18:21] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [utfnormal] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544810 (owner: 10Jforrester)
[10:18:25] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [php-sesson-serlizer] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544811 (owner: 10Jforrester)
[10:18:27] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [testing-access-wrapper] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544812 (owner: 10Jforrester)
[10:18:29] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/tools/minus-x] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544813 (owner: 10Jforrester)
[10:18:31] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/tools/phpunit-patch-coverage] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544814 (owner: 10Jforrester)
[10:18:35] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [wikimedia/lucene-explain-parser] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544815 (owner: 10Jforrester)
[10:18:37] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/services/jobrunner] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544816 (owner: 10Jforrester)
[10:18:39] <hauskater>	 https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/481903/ skipped PS3 for some reason
[10:18:41] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/oauthclient-php] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544817 (owner: 10Jforrester)
[10:18:43] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [cdb] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544818 (owner: 10Jforrester)
[10:18:45] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [at-ease] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544819 (owner: 10Jforrester)
[10:18:47] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [base-convert] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544820 (owner: 10Jforrester)
[10:18:50] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [css-sanitzer] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544821 (owner: 10Jforrester)
[10:18:53] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [AhoCorasick] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544822 (owner: 10Jforrester)
[10:18:59] <hashar>	 James_F: ^^ ^ that is expected
[10:19:03] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [RunningStat] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544823 (owner: 10Jforrester)
[10:19:05] <hashar>	 I have marked the repo read only on the zuul merger
[10:19:07] <hashar>	 to purge the queue 
[10:19:09] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [IPSet] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544824 (owner: 10Jforrester)
[10:19:11] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [RelPath] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544825 (owner: 10Jforrester)
[10:19:15] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [WrappedString] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544826 (owner: 10Jforrester)
[10:19:17] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [CLDRPluralRuleParser] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544827 (owner: 10Jforrester)
[10:19:21] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [HtmlFormatter] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544828 (owner: 10Jforrester)
[10:19:48] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/libs/Assert] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544830 (owner: 10Jforrester)
[10:19:58] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/libs/Equivset] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544831 (owner: 10Jforrester)
[10:20:02] <James_F>	 Yeah.
[10:20:07] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/libs/etcd] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544832 (owner: 10Jforrester)
[10:20:10] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/libs/IPUtils] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544833 (owner: 10Jforrester)
[10:20:13] <James_F>	 Clever quick way to make it do that.
[10:20:15] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/libs/ObjectFactory] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544834 (owner: 10Jforrester)
[10:20:21] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/libs/PasswordBlacklist] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544835 (owner: 10Jforrester)
[10:20:27] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/libs/RemexHtml] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544836 (owner: 10Jforrester)
[10:20:33] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/libs/ScopedCallback] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544837 (owner: 10Jforrester)
[10:20:40] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/libs/Services] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544838 (owner: 10Jforrester)
[10:20:46] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/libs/Timestamp] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544839 (owner: 10Jforrester)
[10:20:52] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/libs/WaitConditionLoop] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544840 (owner: 10Jforrester)
[10:21:00] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [mediawiki/libs/XMPReader] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544841 (owner: 10Jforrester)
[10:21:06] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [wikimedia/slimapp] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544842 (owner: 10Jforrester)
[10:21:13] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] layout: [avro-php] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544843 (owner: 10Jforrester)
[10:23:32] <hashar>	 !log Restored integration/config on zuul-mergers : sudo chown -R zuul:zuul /srv/zuul/git/integration/config
[10:23:33] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[10:25:10] <wikibugs>	 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Jenkins, 10Patch-For-Review, 10Technical-Debt: Align jenkins-job-builder with upstream - https://phabricator.wikimedia.org/T236036 (10hashar)
[10:25:38] <wikibugs>	 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Jenkins, 10Patch-For-Review, 10Technical-Debt: Align jenkins-job-builder with upstream - https://phabricator.wikimedia.org/T236036 (10hashar) jenkins-job-builder 2.0.3 fails due to our outdated usage of postbuildscript...
[10:29:28] <wikibugs>	 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Jenkins, 10Technical-Debt: integration-config-jjb-diff-docker should diff using the proper jjb version - https://phabricator.wikimedia.org/T236046 (10hashar)
[10:31:16] <wikibugs>	 (03PS2) 10Hashar: Use jenkins-job-builder from pypi [integration/config] - 10https://gerrit.wikimedia.org/r/544851 (https://phabricator.wikimedia.org/T236036)
[10:31:18] <wikibugs>	 (03PS1) 10Hashar: Run integration-config-jjb-diff-docker on jjb version change [integration/config] - 10https://gerrit.wikimedia.org/r/544857 (https://phabricator.wikimedia.org/T236046)
[10:31:39] <wikibugs>	 (03CR) 10Hashar: [C: 04-2] "The diff is wrong due to  T236046" [integration/config] - 10https://gerrit.wikimedia.org/r/544851 (https://phabricator.wikimedia.org/T236036) (owner: 10Hashar)
[10:32:06] <hashar>	 lunch time
[11:16:42] <wikibugs>	 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201910), 10Move-Files-To-Commons, 10TCB-Team, and 5 others: Test WikiTextEditor class with browser tests. - https://phabricator.wikimedia.org/T190829 (10zeljkofilipin)
[11:24:36] <wmf-insecte>	 Project beta-code-update-eqiad build #268761: 04FAILURE in 1 min 35 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/268761/
[11:25:38] <wikibugs>	 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201910), 10Browser-Tests, and 2 others: Selenium tests failing for PluggableAuth plugins - https://phabricator.wikimedia.org/T236030 (10zeljkofilipin)
[11:25:57] <wikibugs>	 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO (201910), 10Browser-Tests, and 2 others: Selenium tests failing for PluggableAuth plugins - https://phabricator.wikimedia.org/T236030 (10zeljkofilipin) cc @hashar
[11:30:49] <hashar>	 00:00:09.773 error: cannot update the ref 'refs/remotes/origin/wmf/1.35.0-wmf.2': unable to append to '.git/logs/refs/remotes/origin/wmf/1.35.0-wmf.2': Permission denied
[11:30:50] <hashar>	 :(
[11:31:25] <Reedy>	 It's permissions
[11:31:37] <Reedy>	 I changed the remotes last night to fix the /r/p gerrit issue
[11:31:46] <Reedy>	 But some had to be done with sudo, because permission denied
[11:32:12] <Reedy>	 But it was working fine after...
[11:32:31] <hashar>	 seems transient
[11:32:37] <hashar>	 I triggered a new build
[11:32:54] <hashar>	 00:00:01.112 From https://gerrit.wikimedia.org/r/mediawiki/core
[11:32:54] <hashar>	 00:00:01.112  ! cd8dee3c05..1c3b474966  wmf/1.35.0-wmf.2 -> origin/wmf/1.35.0-wmf.2  (unable to update local ref)
[11:32:54] <hashar>	 00:00:01.139 INFO:mwcore:Exit code: 1
[11:32:55] <hashar>	 bah
[11:33:24] <hauskater>	 ?
[11:33:46] <wmf-insecte>	 Project beta-code-update-eqiad build #268762: 04STILL FAILING in 1 min 25 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/268762/
[11:34:23] <hauskater>	 what a silly error
[11:34:44] <hauskater>	 the origin clone url is okay, why is it refusing to clone?
[11:35:05] <Reedy>	 file permissions on disk for the git repo
[11:35:11] <wmf-insecte>	 Yippee, build fixed!
[11:35:11] <wmf-insecte>	 Project beta-code-update-eqiad build #268763: 09FIXED in 1 min 24 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/268763/
[11:35:19] <Reedy>	 I've just re-chown'd some files
[11:35:23] <Reedy>	 And lo and behold
[11:35:49] <hauskater>	 :)
[11:45:38] <wikibugs>	 10Release-Engineering-Team (Unit & Int & System Tooling), 10Release-Engineering-Team-TODO (201910), 10MediaWiki-Core-Testing, 10MediaWiki-Vagrant, 10User-zeljkofilipin: mediawiki/core Selenium tests fail when targeting mediawiki/vagrant VM - https://phabricator.wikimedia.org/T233820 (10zeljkofilipin) 05...
[11:45:42] <James_F>	 Success.
[11:57:40] <Nikerabbit>	 https://integration.wikimedia.org/zuul/ codehealth and postmerge pipelines stuck?
[11:58:39] <hashar>	 Nikerabbit: indeed :/
[12:02:38] <hashar>	 Nikerabbit: they will run eventually
[12:03:20] <Nikerabbit>	 hashar: what are they waiting for?
[12:03:37] <hashar>	 a shit ton of things for integration/config repo
[12:03:49] <Nikerabbit>	 hmm okay
[12:04:08] <Nikerabbit>	 not visible on that page then
[12:04:36] <hashar>	 !log Draining merger:merge request. On contint1001  : sudo chown -R root:root /srv/zuul/git/integration/config
[12:04:38] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[12:04:51] <hashar>	 Nikerabbit: that is the graph "Gearman job queue" at the bottom
[12:06:24] <hashar>	 !log Draining merger:merge request. On contint2001  : sudo chown -R root:root /srv/zuul/git/integration/config
[12:06:25] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[12:06:32] <hashar>	 400 left 
[12:07:10] <Nikerabbit>	 hashar: I'm learning new stuff every time :D
[12:10:06] <hashar>	 Nikerabbit: we noticed it eariler today, but I have failed to purge it all :(
[12:13:11] <hashar>	 Nikerabbit: unblocked :]
[12:14:25] <Nikerabbit>	 hashar: yay, thanks for the magic
[12:18:41] <icinga-wm>	 PROBLEM - zuul_merger_service_running on contint1001 is CRITICAL: PROCS CRITICAL: 0 processes with regex args ^/usr/share/python/zuul/bin/python /usr/bin/zuul-merger https://www.mediawiki.org/wiki/Continuous_integration/Zuul
[12:21:01] <paladox>	 https://gerrit-review.googlesource.com/c/gerrit/+/240629 \o/
[12:21:17] <paladox>	 no more errors when you click on the autocomplete for example project:
[12:27:56] <icinga-wm>	 ACKNOWLEDGEMENT - Work requests waiting in Zuul Gearman server on contint1001 is CRITICAL: CRITICAL: 100.00% of data above the critical threshold [140.0] amusso too many changes https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1
[12:27:56] <icinga-wm>	 ACKNOWLEDGEMENT - zuul_merger_service_running on contint1001 is CRITICAL: PROCS CRITICAL: 0 processes with regex args ^/usr/share/python/zuul/bin/python /usr/bin/zuul-merger amusso too many changes https://www.mediawiki.org/wiki/Continuous_integration/Zuul
[12:33:03] <icinga-wm>	 RECOVERY - zuul_merger_service_running on contint1001 is OK: PROCS OK: 1 process with regex args ^/usr/share/python/zuul/bin/python /usr/bin/zuul-merger https://www.mediawiki.org/wiki/Continuous_integration/Zuul
[12:35:29] <icinga-wm>	 PROBLEM - zuul_merger_service_running on contint2001 is CRITICAL: PROCS CRITICAL: 0 processes with regex args ^/usr/share/python/zuul/bin/python /usr/bin/zuul-merger https://www.mediawiki.org/wiki/Continuous_integration/Zuul
[12:38:39] <icinga-wm>	 RECOVERY - zuul_merger_service_running on contint2001 is OK: PROCS OK: 1 process with regex args ^/usr/share/python/zuul/bin/python /usr/bin/zuul-merger https://www.mediawiki.org/wiki/Continuous_integration/Zuul
[12:40:21] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] Run integration-config-jjb-diff-docker on jjb version change [integration/config] - 10https://gerrit.wikimedia.org/r/544857 (https://phabricator.wikimedia.org/T236046) (owner: 10Hashar)
[12:40:28] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Run integration-config-jjb-diff-docker on jjb version change [integration/config] - 10https://gerrit.wikimedia.org/r/544857 (https://phabricator.wikimedia.org/T236046) (owner: 10Hashar)
[12:41:32] <wikibugs>	 (03CR) 10Hashar: [C: 03+2] Run integration-config-jjb-diff-docker on jjb version change [integration/config] - 10https://gerrit.wikimedia.org/r/544857 (https://phabricator.wikimedia.org/T236046) (owner: 10Hashar)
[12:43:08] <wikibugs>	 (03Merged) 10jenkins-bot: Run integration-config-jjb-diff-docker on jjb version change [integration/config] - 10https://gerrit.wikimedia.org/r/544857 (https://phabricator.wikimedia.org/T236046) (owner: 10Hashar)
[12:44:25] <wikibugs>	 (03CR) 10Hashar: [C: 04-2] "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/544851 (https://phabricator.wikimedia.org/T236036) (owner: 10Hashar)
[12:48:47] <James_F>	 hashar: Nice.
[12:49:38] <wikibugs>	 (03CR) 10Jforrester: [C: 03+1] Use jenkins-job-builder from pypi [integration/config] - 10https://gerrit.wikimedia.org/r/544851 (https://phabricator.wikimedia.org/T236036) (owner: 10Hashar)
[12:50:18] <hashar>	 almost
[12:50:22] <hashar>	 I wanna fix the diff
[12:50:33] <hashar>	 it always use the version of jjb that comes from the change/patch
[12:51:45] <James_F>	 Oh, should it do before/after diff?
[12:53:51] <hashar>	 hmm
[12:53:55] <hashar>	 not in this specific case :]
[12:54:12] <hashar>	 it is a noop, I checked manually locally
[12:54:17] <hashar>	 but the CI job is wrong
[12:54:40] <hashar>	 the jenkins-jobs environment is created from  the change version
[12:54:51] <hashar>	 and the same environment is used to generate both versions ;D
[12:58:33] <icinga-wm>	 RECOVERY - Work requests waiting in Zuul Gearman server on contint1001 is OK: OK: Less than 30.00% above the threshold [90.0] https://www.mediawiki.org/wiki/Continuous_integration/Zuul https://grafana.wikimedia.org/dashboard/db/zuul-gearman?panelId=10&fullscreen&orgId=1
[13:04:02] <wikibugs>	 (03PS1) 10Thcipriani: deployments-calendar: fix US vs EU SWATters [tools/release] - 10https://gerrit.wikimedia.org/r/544899
[13:14:31] <wikibugs>	 (03PS3) 10Hashar: Use jenkins-job-builder from pypi [integration/config] - 10https://gerrit.wikimedia.org/r/544851 (https://phabricator.wikimedia.org/T236036)
[13:14:33] <wikibugs>	 (03PS1) 10Hashar: Use proper jjb version when diffing [integration/config] - 10https://gerrit.wikimedia.org/r/544906 (https://phabricator.wikimedia.org/T236046)
[13:15:34] <hashar>	 note how Tyler hacks json for breakfast
[13:20:54] <wikibugs>	 (03CR) 10Hashar: [C: 04-2] "recheck" [integration/config] - 10https://gerrit.wikimedia.org/r/544851 (https://phabricator.wikimedia.org/T236036) (owner: 10Hashar)
[13:21:38] <wikibugs>	 (03PS1) 10Hashar: Update jjb to 2.0.3 [integration/config] - 10https://gerrit.wikimedia.org/r/544907 (https://phabricator.wikimedia.org/T188398)
[13:22:47] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Update jjb to 2.0.3 [integration/config] - 10https://gerrit.wikimedia.org/r/544907 (https://phabricator.wikimedia.org/T188398) (owner: 10Hashar)
[13:27:06] <wikibugs>	 (03CR) 10Hashar: Use proper jjb version when diffing (033 comments) [integration/config] - 10https://gerrit.wikimedia.org/r/544906 (https://phabricator.wikimedia.org/T236046) (owner: 10Hashar)
[13:29:48] <wikibugs>	 (03PS1) 10Hashar: Try jenkins-job-builder==2.0.1 [integration/config] - 10https://gerrit.wikimedia.org/r/544910
[13:30:00] <wikibugs>	 (03CR) 10Hashar: [C: 04-2] Try jenkins-job-builder==2.0.1 [integration/config] - 10https://gerrit.wikimedia.org/r/544910 (owner: 10Hashar)
[13:33:23] <wikibugs>	 (03CR) 10Hashar: "Parent change https://gerrit.wikimedia.org/r/#/c/integration/config/+/544906/  setup jjb using the parent tox.ini / requirement  when gene" [integration/config] - 10https://gerrit.wikimedia.org/r/544851 (https://phabricator.wikimedia.org/T236036) (owner: 10Hashar)
[13:33:50] <wikibugs>	 (03CR) 10Hashar: "The dummy change https://gerrit.wikimedia.org/r/#/c/integration/config/+/544910/  switches from 2.0.2 to 2.0.1 and the console shows a dif" [integration/config] - 10https://gerrit.wikimedia.org/r/544906 (https://phabricator.wikimedia.org/T236046) (owner: 10Hashar)
[13:33:58] <wikibugs>	 (03Abandoned) 10Hashar: Try jenkins-job-builder==2.0.1 [integration/config] - 10https://gerrit.wikimedia.org/r/544910 (owner: 10Hashar)
[13:34:19] <hashar>	 James_F: so essentially :  https://gerrit.wikimedia.org/r/#/c/integration/config/+/544906/
[13:34:42] <hashar>	 when generating the JJB config for the parent change,  that also install a fresh jjb using whtaver tox.ini / jenkins-requirements.txt   was in the parent
[13:34:43] <hashar>	 so 
[13:34:52] <hashar>	 we can then just bump jjb in the requirements.txt file
[13:34:58] <hashar>	 and see whether that introduces any diff
[13:38:35] <wikibugs>	 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Jenkins, and 2 others: integration-config-jjb-diff-docker should diff using the proper jjb version - https://phabricator.wikimedia.org/T236046 (10hashar) p:05Triage→03Normal a:03hash...
[13:38:45] <wikibugs>	 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Jenkins, 10Patch-For-Review, 10Technical-Debt: Align jenkins-job-builder with upstream - https://phabricator.wikimedia.org/T236036 (10hashar) p:05Triage→03Normal a:03hashar
[13:59:08] <wikibugs>	 10Release-Engineering-Team-TODO, 10Code-Health-Metrics, 10MediaWiki-Core-Testing, 10Code-Health, and 2 others: Unit tests are not being run for extensions under PHPUnit 4.x (HHVM) - https://phabricator.wikimedia.org/T229220 (10kostajh) 05Open→03Declined HHVM is gone, so I'm marking this as declined.
[13:59:35] <wikibugs>	 (03CR) 10Jforrester: [C: 03+2] Use proper jjb version when diffing [integration/config] - 10https://gerrit.wikimedia.org/r/544906 (https://phabricator.wikimedia.org/T236046) (owner: 10Hashar)
[14:00:11] <wikibugs>	 (03CR) 10Jforrester: [C: 03+2] Use jenkins-job-builder from pypi [integration/config] - 10https://gerrit.wikimedia.org/r/544851 (https://phabricator.wikimedia.org/T236036) (owner: 10Hashar)
[14:01:29] <wikibugs>	 (03PS1) 10Daimona Eaytoy: Revert "layout: [skins/MonoBook] Run phan" [integration/config] - 10https://gerrit.wikimedia.org/r/544924
[14:01:52] <wikibugs>	 (03CR) 10Jforrester: [C: 03+2] "Ha." [integration/config] - 10https://gerrit.wikimedia.org/r/544924 (owner: 10Daimona Eaytoy)
[14:02:09] <wikibugs>	 (03Merged) 10jenkins-bot: Use proper jjb version when diffing [integration/config] - 10https://gerrit.wikimedia.org/r/544906 (https://phabricator.wikimedia.org/T236046) (owner: 10Hashar)
[14:02:46] <wikibugs>	 (03Merged) 10jenkins-bot: Use jenkins-job-builder from pypi [integration/config] - 10https://gerrit.wikimedia.org/r/544851 (https://phabricator.wikimedia.org/T236036) (owner: 10Hashar)
[14:03:08] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Revert "layout: [skins/MonoBook] Run phan" [integration/config] - 10https://gerrit.wikimedia.org/r/544924 (owner: 10Daimona Eaytoy)
[14:03:32] <wikibugs>	 (03PS2) 10Jforrester: Revert "layout: [skins/MonoBook] Run phan" [integration/config] - 10https://gerrit.wikimedia.org/r/544924 (owner: 10Daimona Eaytoy)
[14:04:53] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Revert "layout: [skins/MonoBook] Run phan" [integration/config] - 10https://gerrit.wikimedia.org/r/544924 (owner: 10Daimona Eaytoy)
[14:06:17] <wikibugs>	 (03PS3) 10Jforrester: Revert "layout: [skins/MonoBook] Run phan" [integration/config] - 10https://gerrit.wikimedia.org/r/544924 (owner: 10Daimona Eaytoy)
[14:06:31] <wikibugs>	 (03CR) 10Jforrester: [C: 03+2] "Yay gerrit commit message length requirement…" [integration/config] - 10https://gerrit.wikimedia.org/r/544924 (owner: 10Daimona Eaytoy)
[14:08:07] <Reedy>	 I do wonder if we should be testing PHP 7.3 on REL1_31
[14:08:16] <Reedy>	 Considering numerous extensions don't have PHPCS that works
[14:08:25] <wikibugs>	 (03Merged) 10jenkins-bot: Revert "layout: [skins/MonoBook] Run phan" [integration/config] - 10https://gerrit.wikimedia.org/r/544924 (owner: 10Daimona Eaytoy)
[14:08:43] <James_F>	 Reedy: Hmm. :-(
[14:09:00] <Reedy>	 Rather, phpcs that fails on PHP 7.3 due to a continue issue
[14:09:14] <Reedy>	 ala https://integration.wikimedia.org/ci/job/quibble-composer-mysql-php73-docker/2144/console
[14:09:24] <Reedy>	 14:51:01 PHP Fatal error:  Uncaught PHP_CodeSniffer\Exceptions\RuntimeException: "continue" targeting switch is equivalent to "break". Did you mean to use "continue 2"? in /workspace/src/extensions/HitCounters/vendor/squizlabs/php_codesniffer/src/Files/File.php on line 1351 in 
[14:10:40] <James_F>	 Oh, yeah, that.
[14:10:47] <Daimona>	 Or PHPCS could be updated, version 19.0.0 is enough I think
[14:11:00] <James_F>	 Yeah, we're upgrading PHPCS everywhere.
[14:11:10] <Daimona>	 Unless the majority of repos have 18.0.0 in their REL1_31 branch
[14:11:23] <Daimona>	 In which case, well, let's just stop testing on PHP73
[14:11:35] <Reedy>	 https://github.com/wikimedia/mediawiki-extensions-HitCounters/blob/REL1_31/composer.json#L6
[14:11:42] <Reedy>	 indeed 18.0.0
[14:12:15] <Reedy>	 How about we make 7.3 just non voting?
[14:12:23] <Daimona>	 Huh actually... Maybe it was fixed in 21.0.0
[14:12:26] <Reedy>	 It seems useful if it passes
[14:12:28] <Daimona>	 ^ Sounds good
[14:13:28] <Reedy>	 If an extension is actively maintained... they can backport PHPCS if they want
[14:14:51] <James_F>	 Yeah. If.
[14:15:17] <James_F>	 Getting LibUp to operate on non-master branches to mass-fix 1.31 would be nice.
[14:15:27] <wikibugs>	 (03PS2) 10Hashar: Upgrade PostBuildScript and Update jjb to 2.0.3 [integration/config] - 10https://gerrit.wikimedia.org/r/544907 (https://phabricator.wikimedia.org/T188398)
[14:15:44] <wikibugs>	 (03CR) 10Hashar: [C: 04-2] "WIP, most probably broken." [integration/config] - 10https://gerrit.wikimedia.org/r/544907 (https://phabricator.wikimedia.org/T188398) (owner: 10Hashar)
[14:15:52] <wikibugs>	 10Continuous-Integration-Config, 10MW-1.31-release: Make PHP 7.3 non voting on REL1_31 - https://phabricator.wikimedia.org/T236069 (10Reedy)
[14:16:42] <Reedy>	 Hmm
[14:16:48] <Reedy>	 1.31 is to June 2021
[14:16:51] <wikibugs>	 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Jenkins, 10Technical-Debt: Align jenkins-job-builder with upstream - https://phabricator.wikimedia.org/T236036 (10hashar) We now use 2.0.2 from pypi.  2.0.3 upgrade is blocked on upgrading the PostBuildScript plugin.
[14:16:56] <wikibugs>	 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Jenkins, 10Technical-Debt: Align jenkins-job-builder with upstream - https://phabricator.wikimedia.org/T236036 (10hashar)
[14:16:58] <wikibugs>	 10Continuous-Integration-Config, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Jenkins, 10Technical-Debt: integration-config-jjb-diff-docker should diff using the proper jjb version - https://phabricator.wikimedia.org/T236046 (10hashar) 05Open→03Resolved
[14:17:03] <wikibugs>	 10Continuous-Integration-Config, 10MW-1.31-release: Make PHP 7.3 non voting on REL1_31 - https://phabricator.wikimedia.org/T236069 (10Jdforrester-WMF) I'm not enthusiastic about blanket making it non-voting for all repos on the REL1_31 branch; repos that have tried and succeeded to achieve compatibility/testin...
[14:17:03] <Reedy>	 *if* we can get libup to do it as a longer term fix... I imagine that's going to be useful
[14:17:31] <Reedy>	 I imagine it'll be a problem we'll keep experiencing as we drop support and new PHP comes out for testing
[14:18:00] <Reedy>	 Is there a task about libup branch support?
[14:18:00] <wikibugs>	 (03CR) 10Hashar: [C: 04-2] "I have to review the diff, craft a copy of those jobs and test them out." [integration/config] - 10https://gerrit.wikimedia.org/r/544907 (https://phabricator.wikimedia.org/T188398) (owner: 10Hashar)
[14:18:55] <James_F>	 Reedy: No, don't think so.
[14:19:12] * Reedy files one
[14:21:19] <hashar>	 for phpcs I had to do some upgrade of phpcs due to php7.2 support
[14:21:31] <hashar>	 and iirc in those case I have just went with 0.19.1 or something like that
[14:21:37] <hashar>	 and ignored all new rules
[14:21:51] <hashar>	 php 7.3 is I guess a different horse
[14:21:55] <wikibugs>	 10Continuous-Integration-Config, 10MW-1.31-release: Make PHP 7.3 non voting on REL1_31 - https://phabricator.wikimedia.org/T236069 (10Reedy) I wonder if any have explicitly :)  {T236070} would definitely be a better and longer term solution to this
[14:22:19] <hashar>	 oh
[14:22:49] <hashar>	 Reedy: Daimona James_F   so for phpcs in old branches, one has to bump it to 19.1.0
[14:23:03] <hashar>	 and I guess just ignore the new rules
[14:23:04] <hashar>	 https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/CentralAuth/+/538973/
[14:23:09] <James_F>	 Right.
[14:23:12] <hashar>	 an example for REL1_31
[14:23:13] <Daimona>	 Yeah, exactly that one
[14:23:29] <hashar>	 the difficulty is that the code is "old"
[14:23:43] <Reedy>	 hashar: Yeah, exactly. I don't see any point fixing phpcs failures, just add the ignores
[14:23:47] <hashar>	 or at least worked for a different environemtn (php 5.6? / php 7.0 / hhvmm )
[14:23:56] <hashar>	 and suddenly we changed to also have to support 7.2/7.3
[14:24:00] <Reedy>	 If someone again cares about said branch of the extension, they can fix the exclusions at a later date
[14:24:11] <hashar>	 yeah
[14:24:24] <hashar>	 and supporting the latest rules mean having to change code
[14:24:39] <hashar>	 and really we should not bother changing code in old release branches
[14:24:47] <hashar>	 unless that is a security fix I guess or an important bug fix
[14:25:49] <Reedy>	 If we can have LibUp do it... Human labour should be minimal
[14:25:59] <Reedy>	 I'm not saying we should be doing it for the sake of it, but there's definitely cases
[14:38:58] <wikibugs>	 (03CR) 10Jforrester: [C: 03+2] layout: [wikimedia/textcat] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544808 (owner: 10Jforrester)
[14:41:16] <wikibugs>	 (03Merged) 10jenkins-bot: layout: [wikimedia/textcat] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544808 (owner: 10Jforrester)
[14:52:40] <wikibugs>	 (03CR) 10Jforrester: "304 jobs changed, so… 62 didn't. Now I'm curious about those 62 jobs. :-)" [integration/config] - 10https://gerrit.wikimedia.org/r/544907 (https://phabricator.wikimedia.org/T188398) (owner: 10Hashar)
[14:53:58] <wikibugs>	 (03CR) 10Hashar: [C: 04-2] "Nicely pointed, indeed that looks suspicious bah :-\   One sure thing that upgrade is not trivial, I gotta revisit the diff and try it out" [integration/config] - 10https://gerrit.wikimedia.org/r/544907 (https://phabricator.wikimedia.org/T188398) (owner: 10Hashar)
[15:04:07] <wikibugs>	 (03CR) 10Thcipriani: [C: 03+2] deployments-calendar: fix US vs EU SWATters [tools/release] - 10https://gerrit.wikimedia.org/r/544899 (owner: 10Thcipriani)
[15:04:56] <wikibugs>	 (03Merged) 10jenkins-bot: deployments-calendar: fix US vs EU SWATters [tools/release] - 10https://gerrit.wikimedia.org/r/544899 (owner: 10Thcipriani)
[15:05:07] <wikibugs>	 (03CR) 10Jforrester: [C: 03+2] layout: [utfnormal] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544810 (owner: 10Jforrester)
[15:05:29] <wikibugs>	 (03CR) 10Jforrester: [C: 03+2] layout: [purtle] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544809 (owner: 10Jforrester)
[15:07:42] <wikibugs>	 (03Merged) 10jenkins-bot: layout: [purtle] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544809 (owner: 10Jforrester)
[15:09:33] <wikibugs>	 (03PS1) 10Hashar: jjb: archive section is invalid, use "publishers" instead [integration/config] - 10https://gerrit.wikimedia.org/r/544948
[15:12:58] <wikibugs>	 (03PS1) 10Hashar: Add missing docker-cleanup macros [integration/config] - 10https://gerrit.wikimedia.org/r/544952
[15:13:21] <wikibugs>	 (03PS3) 10Hashar: Upgrade PostBuildScript and Update jjb to 2.0.3 [integration/config] - 10https://gerrit.wikimedia.org/r/544907 (https://phabricator.wikimedia.org/T188398)
[15:13:28] <wikibugs>	 (03CR) 10Jforrester: "Whoops. Presumably we made the mistake once and copy-pasted it to the others?" [integration/config] - 10https://gerrit.wikimedia.org/r/544948 (owner: 10Hashar)
[15:15:59] <wikibugs>	 (03CR) 10Hashar: [C: 04-2] "Some of the jobs used:" [integration/config] - 10https://gerrit.wikimedia.org/r/544907 (https://phabricator.wikimedia.org/T188398) (owner: 10Hashar)
[15:16:27] <paladox>	 marxarelli would this https://gerrit-review.googlesource.com/c/plugins/go-import/+/241958 fix your issue?
[15:17:59] <James_F>	 hashar: Should it do `git diff -w` to hide whitespace-only changes?
[15:19:10] <wikibugs>	 (03CR) 10Jforrester: [C: 03+2] layout: [utfnormal] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544810 (owner: 10Jforrester)
[15:21:02] <wikibugs>	 (03Merged) 10jenkins-bot: layout: [utfnormal] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544810 (owner: 10Jforrester)
[15:22:51] <wikibugs>	 (03PS2) 10Jforrester: layout: [php-session-serlizer] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544811
[15:23:04] <wikibugs>	 (03PS3) 10Jforrester: layout: [php-session-serializer] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544811
[15:25:26] <wikibugs>	 10Gerrit: Lots of "Skipping change xxx because the corresponding repository was not found" in the logs - https://phabricator.wikimedia.org/T233989 (10hashar) Eventually that surfaced on the Gerrit mailing list: https://groups.google.com/forum/#!msg/repo-discuss/BgH2pG4kuyA/pvLxryvBDgAJ  > > WARN  com.google.gerr...
[15:26:26] <hashar>	 James_F: I just diff locally
[15:26:51] <hashar>	 James_F: really the devil will be to figure out whether the new postbuildscript config matches the old one :-\
[15:26:58] * James_F nods.
[15:27:03] <hashar>	 and the current/old syntax was a bit annoying
[15:32:47] <wikibugs>	 10Continuous-Integration-Config, 10MW-1.31-release: Make PHP 7.3 non voting on REL1_31 - https://phabricator.wikimedia.org/T236069 (10hashar) 05Open→03Declined That is a non issue with CodeSniffer.  The fix is to upgrade `mediawiki/mediawiki-codesniffer` to `19.1.0` and ignore new rules in `.phpcs.xml`. Ex...
[15:33:21] <wikibugs>	 10MediaWiki-Codesniffer: Codesniffer warnings sometimes don't show up in the commit that enforces them - https://phabricator.wikimedia.org/T236083 (10Krinkle)
[15:34:07] <wikibugs>	 10MediaWiki-Codesniffer: Codesniffer warnings sometimes don't show up in the commit that enforces them - https://phabricator.wikimedia.org/T236083 (10Krinkle)
[15:34:24] <wikibugs>	 10MediaWiki-Codesniffer, 10Developer Productivity: Codesniffer warnings sometimes don't show up in the commit that enforces them - https://phabricator.wikimedia.org/T236083 (10Krinkle)
[15:34:47] <wikibugs>	 10MediaWiki-Codesniffer, 10Developer Productivity: Codesniffer warnings sometimes don't show up in the commit that enforces them - https://phabricator.wikimedia.org/T236083 (10Jdforrester-WMF) 05Open→03Invalid The commit that causes the warnings for `wgContLang` is that one – specifically, requiring MW 1.3...
[15:35:08] <James_F>	 Krinkle: extension.json is effectively part of the PHP code.
[15:36:39] <Krinkle>	 James_F: Right, but I find it counter-intuitive that codesniffer 28.0 is not only variable by version number of it but also of the MW version
[15:36:49] <Krinkle>	 Anyway, will have to get used to I guess..
[15:37:41] <James_F>	 Krinkle: If you try an old checkout with CS 24 on TMH and bump MW requires it'll also error.
[15:37:50] <Krinkle>	 Is that to cater to things like MLEB? I imagine that gets akward when it the method in question has been removed since. There isn't an upper bound. Anyway, if it doesn't happen too often, can be ignored I guess..
[15:38:17] <Krinkle>	 James_F: Yeah, I don't mean v28 specifically, but I was expecting in general for rules to be tied to the rule version, not based on a field somewhere in the repo.
[15:38:34] <Krinkle>	 E.g. that if you don't want PHP 7.2 stuff yet or MW 1.34 to retain an older codesniffer version
[15:38:57] <Krinkle>	 like we would for eslint and most other tools I guess. It's a bit magical, but I get it.
[15:40:09] <wikibugs>	 10MediaWiki-Codesniffer, 10Developer Productivity: Codesniffer warnings sometimes don't show up in the commit that enforces them - https://phabricator.wikimedia.org/T236083 (10hashar) That is due to: ` lang=diff,name=extension.json {   "requires": { -               "MediaWiki": ">= 1.32.0" +               "Med...
[15:40:16] <James_F>	 It's for badly-versioned extensions like MLEB, yes.
[15:40:17] <wikibugs>	 10MediaWiki-Codesniffer, 10Developer Productivity: Codesniffer warnings sometimes don't show up in the commit that enforces them - https://phabricator.wikimedia.org/T236083 (10hashar) 05Invalid→03Open
[15:40:33] <Krinkle>	 maybe libUp could also auto-require the current release cycle version by default for any tarball/wmf extension that isn't in MLEB. WE don't support mixing with older versions, so whenever it isn't auto-bumped, we would without hesitation on a later commit, might as well automate.
[15:40:35] <James_F>	 Note that Wikibase, because it doesn't have an extension.json file, has hundreds of MWCS violations. :-(
[15:40:48] <James_F>	 That might be an idea, yeah.
[15:40:48] <hashar>	 Krinkle: James_F oops sorry I messed up on that task :/
[15:40:57] <Krinkle>	 That way, we don't discover these issues on random commits like mine, but rather as part of the LibUp and then fix or disable warnings as needed. 
[15:41:02] <hashar>	 so yeah nice to see mediawiki-codesniffer can behave differently based on the MW version requirement
[15:41:04] <James_F>	 hashar: It's fine, Phabricator edit conflicts aren't great.
[15:41:07] <hashar>	 it is a bit too magic though
[15:41:33] <Krinkle>	 It's only magical because we change extension.json#requires.MediaWiki very often by hand.
[15:41:43] <Krinkle>	 If we automate that, we don't experience the issue for the most part.
[15:42:02] <Lucas_WMDE>	 James_F: I missed some context, but what does extension.json have to do with CodeSniffer?
[15:42:08] <Lucas_WMDE>	 we don’t even use the default MediaWiki config
[15:42:27] <Lucas_WMDE>	 (CodeSniffer config, that is)
[15:42:44] <James_F>	 Lucas_WMDE: That doesn't help either.
[15:42:49] <Krinkle>	 mediawiki-codesniffer reads $dest/extension.json and only warns for known deprecation of certain global variables if you require a version newer than the version that deprecated it.
[15:42:56] <James_F>	 Yup.
[15:43:00] <Lucas_WMDE>	 ah, ok
[15:43:08] <Lucas_WMDE>	 good to know, thanks
[15:43:08] <James_F>	 And as Wikibase doesn't have one yet (it has four WIPs)…
[15:43:15] <hashar>	 I am heading out, kids / dinner etc
[15:43:21] <James_F>	 See you tomorrow.
[15:43:36] <hashar>	 i shall be there tonight for the gerrit upgrade
[15:43:55] <Krinkle>	 So if your code uses wgContLang and you (unrelatedly) bump that extension.json back to the current release cycle to match our actual support, it only starts complaining then. which means we probably have more use of deprecated code in prod because most repos won't be forced to bump that requirement until they need something of it.
[15:44:05] <Krinkle>	 that might be a better reason to automate it
[15:44:12] <Krinkle>	 so that we discover it sooner
[15:45:00] <James_F>	 Yeah, see also all the failing LibUp patches over the weekend for repos that now don't work in master.
[16:05:11] <wikibugs>	 10MediaWiki-Codesniffer, 10Developer Productivity: Codesniffer warnings sometimes don't show up in the commit that enforces them - https://phabricator.wikimedia.org/T236083 (10Jdforrester-WMF) 05Open→03Invalid
[16:23:28] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10MediaWiki-Internationalization, 10I18n, 10Patch-For-Review: Ensure constructed i18n messages document all possible used strings in comments in JS - https://phabricator.wikimedia.org/T235502 (10Jdforrester-WMF)...
[16:45:55] <wikibugs>	 10Release-Engineering-Team, 10Operations, 10serviceops, 10Performance-Team (Radar), and 2 others: PHP Fatal error:  The UdpSocket to 127.0.0.1:10514 has been closed (from Monolog/SyslogUdp on mwdebug1002) - https://phabricator.wikimedia.org/T214734 (10jijiki) Server will be reimaged, I will ping here when...
[17:16:33] <wikibugs>	 (03CR) 10Jforrester: [C: 03+2] layout: [php-session-serializer] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544811 (owner: 10Jforrester)
[17:18:49] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10Paladox)
[17:25:34] <wikibugs>	 10Release-Engineering-Team, 10Repository-Admins, 10Product-Analytics (Kanban): Create a repository and user for Product Analytics Oozie jobs? - https://phabricator.wikimedia.org/T230743 (10kzimmerman) p:05Triage→03Normal a:03kzimmerman
[17:27:33] <Daimona>	 WTF? What does "unstable" means? https://gerrit.wikimedia.org/r/#/c/mediawiki/extensions/MultiMaps/+/544983/
[17:28:09] <Daimona>	 Oh, risky tests I guess
[17:29:00] <James_F>	 Daimona: For our shared jobs it's meaningless.
[17:29:07] <James_F>	 It's a measure of how often this test has failed recently.
[17:29:26] <James_F>	 But of course our use of jenkins for CI doesn't match the concept of one job <-> one repo.
[17:29:27] <Daimona>	 Ah cool. Never seen it before
[17:29:44] <James_F>	 Yeah, it means something like "at least 5 of the last 10 runs have failed".
[17:32:30] <wikibugs>	 (03PS2) 10Jforrester: layout: [testing-access-wrapper] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544812
[17:32:40] <wikibugs>	 (03CR) 10Jforrester: [C: 03+2] layout: [testing-access-wrapper] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544812 (owner: 10Jforrester)
[17:34:33] <wikibugs>	 (03Merged) 10jenkins-bot: layout: [testing-access-wrapper] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544812 (owner: 10Jforrester)
[17:37:39] <wikibugs>	 (03PS2) 10Jforrester: layout: [mediawiki/tools/minus-x] Drop PHP7.0/PHP7.1 testing [integration/config] - 10https://gerrit.wikimedia.org/r/544813
[17:39:04] <wikibugs>	 (03PS1) 10Phedenskog: fresnel: Update docker image to install Fresnel 0.5.0 [integration/config] - 10https://gerrit.wikimedia.org/r/544992
[17:49:04] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO: Re-enable use of Gerrit HTTP token to push patchsets - https://phabricator.wikimedia.org/T218750 (10thcipriani)
[18:03:50] <subbu>	 anyone know what this is about?
[18:03:51] <subbu>	 18:02:54 ['/usr/bin/scap', 'deploy-local', '-v', '--repo', 'parsoid/deploy', '-g', 'default', 'fetch', '--refresh-config'] on deployment-parsoid09.deployment-prep.eqiad.wmflabs returned [255]: sign_and_send_pubkey: signing failed: agent refused operation
[18:03:51] <subbu>	 Permission denied (publickey).
[18:09:46] <wikibugs>	 10Gerrit, 10Release-Engineering-Team, 10Operations, 10Wikimedia Design Style Guide: Automatic pickup of Gerrit clone master doesn't happen (due to git-lfs not installed on production misc) - https://phabricator.wikimedia.org/T235677 (10Volker_E) @thcipriani Who could help here further?
[18:16:15] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10Paladox)
[18:22:46] <subbu>	 James_F, thcipriani ^^^ anyone know .. see my qn. about scap deploy?
[18:22:59] <subbu>	 arlo and i both ran into this.
[18:34:21] <mutante>	 heads-up: In half an hour Gerrit migration starts. It's moving to a new server (with more RAM) and OS version, expect some downtime during the switch. If you must merge stuff please consider that. New hardware / moar RAM will hopefully be good for the user experience.
[18:34:34] <thcipriani>	 subbu|lunch: the "agent refused operation" thing is typically keyholder
[18:35:06] <thcipriani>	 i.e., you don't have permission to use the key according to keyholder
[18:36:59] <mutante>	 maybe related to the recent issue/work on deployment keys 
[18:37:10] <thcipriani>	 could be
[18:37:52] <wikibugs>	 (03PS2) 10Phedenskog: fresnel: Update docker image to install Fresnel 0.5.0 [integration/config] - 10https://gerrit.wikimedia.org/r/544992 (https://phabricator.wikimedia.org/T235195)
[18:38:18] <hashar>	 !log Jenkins: upgraded logstash plugin to 2.3.1 , overriding a custom build snapshot of 1.2.1.  # T78705 
[18:38:20] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[18:38:20] <stashbot>	 T78705: Send Jenkins build log and results to ElasticSearch - https://phabricator.wikimedia.org/T78705
[18:39:18] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] fresnel: Update docker image to install Fresnel 0.5.0 [integration/config] - 10https://gerrit.wikimedia.org/r/544992 (https://phabricator.wikimedia.org/T235195) (owner: 10Phedenskog)
[18:40:10] <wikibugs>	 (03PS3) 10Phedenskog: fresnel: Update docker image to install Fresnel 0.5.0 [integration/config] - 10https://gerrit.wikimedia.org/r/544992 (https://phabricator.wikimedia.org/T235195)
[18:40:17] <wikibugs>	 (03CR) 10Krinkle: [C: 03+2] fresnel: Update docker image to install Fresnel 0.5.0 [integration/config] - 10https://gerrit.wikimedia.org/r/544992 (https://phabricator.wikimedia.org/T235195) (owner: 10Phedenskog)
[18:41:48] <wikibugs>	 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (CI & Testing services), 10Release-Engineering-Team-TODO, 10Wikimedia-Logstash, and 3 others: Send Jenkins build log and results to ElasticSearch - https://phabricator.wikimedia.org/T78705 (10hashar) I have upgraded the jenkins Logstash pl...
[18:42:28] <wikibugs>	 (03Merged) 10jenkins-bot: fresnel: Update docker image to install Fresnel 0.5.0 [integration/config] - 10https://gerrit.wikimedia.org/r/544992 (https://phabricator.wikimedia.org/T235195) (owner: 10Phedenskog)
[18:42:54] <Krinkle>	 !log Updating docker-pkg files on contint1001 for https://gerrit.wikimedia.org/r/544992
[18:42:55] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[18:43:02] <wikibugs>	 10Deployments, 10Release-Engineering-Team, 10VisualEditor, 10Wikimedia-Logstash, and 3 others: Logstash discards messages from MediaWiki if they contain uncommon keys in the $context array - https://phabricator.wikimedia.org/T234564 (10Urbanecm) >>! In T234564#5592733, @gerritbot wrote: > Change 544958 **m...
[18:44:02] <wikibugs>	 (03PS1) 10Phedenskog: jjb: use Fresnel 0.5.0 [integration/config] - 10https://gerrit.wikimedia.org/r/545005 (https://phabricator.wikimedia.org/T235195)
[18:47:18] <wikibugs>	 (03PS2) 10Krinkle: jjb: use Fresnel 0.5.0 [integration/config] - 10https://gerrit.wikimedia.org/r/545005 (https://phabricator.wikimedia.org/T235195) (owner: 10Phedenskog)
[18:47:58] <wikibugs>	 10Deployments, 10Release-Engineering-Team, 10VisualEditor, 10Wikimedia-Logstash, and 3 others: Logstash discards messages from MediaWiki if they contain uncommon keys in the $context array - https://phabricator.wikimedia.org/T234564 (10matmarex) I tested that on mwdebug1001, triggered the case that should...
[18:55:05] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10thcipriani)
[18:55:58] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10Paladox)
[18:59:03] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10Paladox)
[19:00:31] <Krinkle>	 !log Pruning old image from contint1001: 0.0.38-s1                     2b39d2f00de3
[19:00:33] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[19:00:52] <Krinkle>	 !log Pruning old image from contint1001:  0.0.38                        073a49221d74
[19:00:53] <stashbot>	 Logged the message at https://wikitech.wikimedia.org/wiki/Release_Engineering/SAL
[19:21:31] <wikibugs>	 10MediaWiki-Codesniffer, 10LibUp: Upgrade PHPCS to 28.0.0 in all repos - https://phabricator.wikimedia.org/T235113 (10Legoktm) >>! In T235113#5569899, @Daimona wrote: > And also, please upgrade wikibase/wikibase-codesniffer to 1.0.0, which brings in the 28.0.0 version of mw-codesniffer.  That will need its own...
[19:46:42] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Wikimedia-maintenance-script-run: Run maintenance/cleanupUsersWithNoId.php on beta testwiki - https://phabricator.wikimedia.org/T235971 (10Anomie) You need to supply `--prefix`, and you might want to use `--assign` too.  `--prefix` is used to prefix the usernames for the proble...
[20:09:27] <wikibugs>	 10Release-Engineering-Team: Phatility: Include server field in "Request URL" field - https://phabricator.wikimedia.org/T235945 (10mmodell) a:03mmodell
[20:12:19] <wikibugs>	 10Phabricator, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO: Check whether subversion is actually needed on Phabricator hosts - https://phabricator.wikimedia.org/T236026 (10mmodell) 05Open→03Resolved a:03mmodell Subversion is needed, yes, because phabricator hosts...
[20:15:27] <wikibugs>	 10Phabricator (Upstream), 10Upstream: Bundle web notifications from Phab - https://phabricator.wikimedia.org/T235789 (10mmodell)  >>! In T235789#5586172, @Waddie96 wrote: > Thanks I will test the browser extension and revert back to you it’s usefulness. And, if it’s useful, can it be included in Phab as a ‘Gad...
[20:16:16] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10Dzahn)
[20:27:46] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10Dzahn)
[20:34:10] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10Dzahn)
[20:38:35] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10thcipriani)
[20:40:50] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10thcipriani)
[20:45:23] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10thcipriani)
[20:47:58] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10Dzahn)
[20:54:23] <wmf-insecte>	 Project beta-code-update-eqiad build #268785: 04FAILURE in 1 min 22 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/268785/
[20:54:42] <icinga-wm>	 PROBLEM - Check systemd state on contint2001 is CRITICAL: CRITICAL - degraded: The system is operational but one or more units failed. https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
[20:55:07] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10Dzahn)
[20:55:48] <icinga-wm>	 PROBLEM - Check systemd state on contint1001 is CRITICAL: CRITICAL - degraded: The system is operational but one or more units failed. https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
[20:56:42] <icinga-wm>	 ACKNOWLEDGEMENT - Check systemd state on contint1001 is CRITICAL: CRITICAL - degraded: The system is operational but one or more units failed. daniel_zahn gerrit migration https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
[20:56:42] <icinga-wm>	 ACKNOWLEDGEMENT - Check systemd state on contint2001 is CRITICAL: CRITICAL - degraded: The system is operational but one or more units failed. daniel_zahn gerrit migration https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
[20:58:07] <shinken-wm>	 PROBLEM - Puppet errors on integration-agent-stretch-1001 is CRITICAL: CRITICAL: 1.11% of data above the critical threshold [3.0]
[21:04:21] <wmf-insecte>	 Project beta-code-update-eqiad build #268786: 04STILL FAILING in 1 min 21 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/268786/
[21:10:12] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Page Content Service, 10Product-Infrastructure-Team-Backlog: Beta Cluster deployment failed: sign_and_send_pubkey: signing failed: agent refused operation - https://phabricator.wikimedia.org/T236103 (10Mholloway)
[21:11:40] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Page Content Service, 10Product-Infrastructure-Team-Backlog: Beta Cluster deployment failed: sign_and_send_pubkey: signing failed: agent refused operation - https://phabricator.wikimedia.org/T236103 (10Krenair) a:03Krenair I have a likely fix when gerrit is back up, for the...
[21:14:21] <wmf-insecte>	 Project beta-code-update-eqiad build #268787: 04STILL FAILING in 1 min 21 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/268787/
[21:19:46] <icinga-wm>	 RECOVERY - Check systemd state on contint1001 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
[21:22:16] <icinga-wm>	 RECOVERY - Check systemd state on contint2001 is OK: OK - running: The system is fully operational https://wikitech.wikimedia.org/wiki/Monitoring/check_systemd_state
[21:24:25] <wmf-insecte>	 Yippee, build fixed!
[21:24:26] <wmf-insecte>	 Project beta-code-update-eqiad build #268788: 09FIXED in 1 min 25 sec: https://integration.wikimedia.org/ci/job/beta-code-update-eqiad/268788/
[21:30:12] <wikibugs>	 10Deployments, 10Release-Engineering-Team, 10serviceops, 10Performance-Team (Radar): Cache of wmf-config/InitialiseSettings often 1 step behind - https://phabricator.wikimedia.org/T236104 (10Krinkle)
[21:30:15] <hauskater>	 so this new gerrit1001 server is it more powerful? cooks sandwiches for coders?...
[21:30:23] <wikibugs>	 10Deployments, 10Release-Engineering-Team, 10serviceops, 10Performance-Team (Radar): Cache of wmf-config/InitialiseSettings often 1 step behind - https://phabricator.wikimedia.org/T236104 (10Krinkle)
[21:30:26] <hauskater>	 or it's just a minor improvement?
[21:32:35] <wikibugs>	 10Deployments, 10Release-Engineering-Team, 10serviceops, 10Performance-Team (Radar): Cache of wmf-config/InitialiseSettings often 1 step behind - https://phabricator.wikimedia.org/T236104 (10Krinkle)
[21:33:38] <wikibugs>	 10Beta-Cluster-Infrastructure, 10Wikimedia-maintenance-script-run: Run maintenance/cleanupUsersWithNoId.php on beta testwiki - https://phabricator.wikimedia.org/T235971 (10MarcoAurelio) Thanks @Anomie for the detailed reply. I think we should wait until {T235970} is resolved as the wiki is not readable at this...
[21:34:01] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, 10serviceops: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10Dzahn)
[21:35:54] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, 10serviceops: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10CDanis) something to note/fix for future migrations: one option for...
[21:36:06] <paladox>	 hauskater yes, more powerful
[21:36:18] <paladox>	 (32 cores over 16 cores) and 64gb of ram over 32gb
[21:39:28] <mutante>	 hauskater: 64GB RAM is the bigger deal
[21:39:33] <mutante>	 afaict
[21:45:41] <hauskater>	 still no sandwiches
[21:46:46] <bd808>	 hauskater: heh. Mostly we are going from "large laptop" to "tiny server" class hardware :)
[21:47:08] <bd808>	 maybe "small" rather than "tiny"
[21:47:24] <hauskater>	 at least it's progress :)
[22:01:51] <wikibugs>	 10Gerrit: Gerrit crashed due to out of Heap - https://phabricator.wikimedia.org/T225166 (10Dzahn) Gerrit is on a new server now with 64GB RAM but we still need to tune settings to make use of that.  https://gerrit.wikimedia.org/r/monitoring?part=graph&graph=usedMemory
[23:40:17] <wikibugs>	 10Gerrit: check and fix some Gerrit revs - https://phabricator.wikimedia.org/T236114 (10Dzahn)
[23:44:46] <wikibugs>	 10Gerrit: check and fix some Gerrit revs - https://phabricator.wikimedia.org/T236114 (10Dzahn)
[23:48:40] <wikibugs>	 10Gerrit: check and fix some Gerrit revs - https://phabricator.wikimedia.org/T236114 (10Dzahn)
[23:51:04] <wikibugs>	 10Gerrit: check and fix some Gerrit revs - https://phabricator.wikimedia.org/T236114 (10Dzahn)
[23:51:13] <wikibugs>	 10Gerrit, 10Release-Engineering-Team (Development services), 10Release-Engineering-Team-TODO, 10Operations, and 2 others: Gerrit Hardware Upgrade (+ upgrade from jessie to stretch or buster) - https://phabricator.wikimedia.org/T222391 (10Dzahn)
[23:58:20] <wikibugs>	 10Gerrit: check and fix some Gerrit revs - https://phabricator.wikimedia.org/T236114 (10thcipriani)
[23:59:59] <wikibugs>	 10Gerrit: check and fix some Gerrit revs - https://phabricator.wikimedia.org/T236114 (10thcipriani) p:05Triage→03Normal I've updated all of these to match `cobalt` unless they had changes that were newer than cobalt's that looked normal.  We should also check other repos that are fairly active.  I generated...