[04:58:58] <wikibugs>	 10DBA, 10MediaWiki-Database, 10Multi-Content-Revisions (MCR-SDC phase 1), 10Patch-For-Review, 10Wikimedia-log-errors: Rising lock wait timeout SQL errors upon 1.32.0-wmf.10 group1 deployment - https://phabricator.wikimedia.org/T198350 (10Marostegui) >>! In T198350#4323205, @dduvall wrote: > @tgr's fix wo...
[05:11:05] <wikibugs>	 10DBA, 10Epic, 10Wikimedia-Incident: Improve regular production database backups handling - https://phabricator.wikimedia.org/T138562 (10Marostegui)
[05:11:23] <wikibugs>	 10DBA, 10Operations, 10monitoring, 10Patch-For-Review: Create script to monitor db dumps for backups are successful - https://phabricator.wikimedia.org/T151999 (10Marostegui) 05Open>03Resolved a:03jcrespo I am going to close this as resolved as per T151999#4038297 and going to create a task for the "...
[05:12:00] <wikibugs>	 10DBA, 10Goal: Monitor backup generation for failure or incorrect generation - https://phabricator.wikimedia.org/T198447 (10Marostegui)
[05:12:25] <wikibugs>	 10DBA, 10Goal: Monitor backup generation for failure or incorrect generation - https://phabricator.wikimedia.org/T198447 (10Marostegui) p:05Triage>03Normal
[05:13:16] <wikibugs>	 10DBA, 10Goal: Monitor backup generation for failure or incorrect generation - https://phabricator.wikimedia.org/T198447 (10Marostegui)
[05:13:46] <wikibugs>	 10DBA, 10Epic, 10Wikimedia-Incident: Improve regular production database backups handling - https://phabricator.wikimedia.org/T138562 (10Marostegui)
[05:30:50] <wikibugs>	 10DBA, 10Continuous-Integration-Infrastructure, 10MediaWiki-Database, 10Quibble, 10Release-Engineering-Team (Kanban): Enable MariaDB/MySQL strict mode on CI db hosts - https://phabricator.wikimedia.org/T119371 (10jcrespo) `ONLY_FULL_GROUP_BY` on MariaDB (see bug https://jira.mariadb.org/browse/MDEV-11588...
[05:54:05] <wikibugs>	 10DBA, 10MediaWiki-Database, 10Multi-Content-Revisions (MCR-SDC phase 1), 10Patch-For-Review, 10Wikimedia-log-errors: Rising lock wait timeout SQL errors upon 1.32.0-wmf.10 group1 deployment - https://phabricator.wikimedia.org/T198350 (10jcrespo) > I'm not sure how much can be done about those  May I sug...
[14:20:11] <jynus>	 manuel, are you around?
[14:20:21] <marostegui>	 yep
[14:20:54] <jynus>	 can you check s8 load balancing
[14:21:01] <marostegui>	 checking
[14:21:21] <jynus>	 i dont think is a huge issue
[14:21:24] <marostegui>	 There is one slave depooled for an alter
[14:21:27] <marostegui>	 which is finishing
[14:21:42] <jynus>	 but it seems a bit asymetrical
[14:21:53] <jynus>	 ok if you are aware of it
[14:21:57] <jynus>	 np
[14:22:00] <marostegui>	 Yeah
[14:22:05] <jynus>	 just a heads up
[14:22:11] <marostegui>	 db1104 is more loaded, as it is handling api + 200 of main
[14:22:25] <marostegui>	 I reduced it from 300 to 200, but maybe I can give it 100 of main
[14:22:35] <jynus>	 in case it wasnt planned or intended
[14:22:55] <marostegui>	 The alter should be done in 15-20 mins or so, so things will go back as normal :)
[14:23:07] <jynus>	 np if you don plan to leave it all weekend
[14:23:14] <marostegui>	 no way :-)
[14:23:29] <jynus>	 i was worriec you may have not seen it
[14:23:38] <marostegui>	 Hehe no, i was aware :)
[14:23:42] <marostegui>	 Thanks for double checking
[14:27:05] <marostegui>	 Alter done! :)
[14:34:15] <wikibugs>	 10DBA, 10MediaWiki-extensions-WikibaseClient, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, 10Performance: [Task] Use an own load section for Wikibase DB queries - https://phabricator.wikimedia.org/T97414 (10Lydia_Pintscher) @hoo Is this still needed? If so it'd be great to have some more detail...
[14:35:04] <jynus>	 remember you told me you prefer me me to be extra pedantic than not pointing things and later have issues 0:-)
[14:35:14] <marostegui>	 of course!!
[14:35:18] <marostegui>	 That is why I thanked you :)
[14:35:40] <jynus>	 I also would like the same so less window for mistakes
[14:35:56] <marostegui>	 Indeed :)
[14:36:08] <marostegui>	 Happy to do it too!
[21:10:14] <wikibugs>	 10DBA, 10Cloud-Services, 10Cloud-VPS, 10Epic, 10Tracking: Labs databases rearchitecture (tracking) - https://phabricator.wikimedia.org/T140788 (10bd808)
[21:10:20] <wikibugs>	 10DBA, 10Data-Services, 10Goal, 10Patch-For-Review, 10cloud-services-team (FY2017-18): Migrate all users to new Wiki Replica cluster and decommission old hardware - https://phabricator.wikimedia.org/T142807 (10bd808) 05Open>03Resolved a:03bd808 We have several follow up tasks still pending, but the...