[00:43:06] <wikibugs>	 10DBA, 07Tracking: Cleanup x1 database connection patterns - https://phabricator.wikimedia.org/T164504#3242333 (10Krinkle)
[00:43:43] <wikibugs>	 10DBA, 10MediaWiki-Logging: Evaluate the need for FORCE INDEX (ls_field_val) [now IGNORE INDEX (ls_log_id)], delete the index hint if not needed anymore - https://phabricator.wikimedia.org/T164382#3242335 (10Krinkle)
[00:44:37] <wikibugs>	 10DBA, 10MediaWiki-Change-tagging, 07Schema-change: change_tag table needs redesign - https://phabricator.wikimedia.org/T164167#3242336 (10Krinkle)
[00:53:21] <wikibugs>	 07Blocked-on-schema-change, 06Community-Tech, 10MediaWiki-Database, 07Hindi-Sites, and 3 others: Allow comments longer than 255 bytes - https://phabricator.wikimedia.org/T6715#3242345 (10Krinkle)
[01:32:16] <wikibugs>	 10DBA, 10Wikimedia-Site-requests, 07Tracking: Database table cleanup (tracking) - https://phabricator.wikimedia.org/T18660#3242466 (10Krinkle)
[06:02:39] <wikibugs>	 10DBA, 06Operations: Create less overhead on bacula jobs when dumping production databases - https://phabricator.wikimedia.org/T162789#3242585 (10Marostegui) @jcrespo if you want to test with s4 feel free to pick any host from: T161088 (only db1081 was done)
[06:05:25] <wikibugs>	 10DBA, 06Operations, 10ops-eqiad, 13Patch-For-Review: Reset db1070 idrac - https://phabricator.wikimedia.org/T160392#3242587 (10Marostegui) Thanks @Cmjohnson for all the help.  We can test with db1070 for as much as you like during the week (I just don't like leaving hosts down for the weekend) just let me...
[06:07:18] <wikibugs>	 10DBA, 10Wikidata, 13Patch-For-Review, 07Schema-change: Drop the useless wb_terms keys "wb_terms_entity_type" and "wb_terms_type" on "wb_terms" table - https://phabricator.wikimedia.org/T163548#3242588 (10Marostegui) db2045 is done: ``` root@neodymium:~# mysql --skip-ssl -hdb2045.codfw.wmnet wikidatawiki -...
[06:08:22] <wikibugs>	 07Blocked-on-schema-change, 10DBA, 10Wikidata, 13Patch-For-Review, 03Wikidata-Sprint: Deploy schema change for adding term_full_entity_id column to wb_terms table - https://phabricator.wikimedia.org/T162539#3242589 (10Marostegui) db2045 is done: ``` root@neodymium:~# mysql --skip-ssl -hdb2045.codfw.wmnet...
[06:44:54] <wikibugs>	 10DBA, 06Operations, 10ops-eqiad: Decommission db1024 - https://phabricator.wikimedia.org/T164702#3242621 (10Marostegui)
[06:45:00] <wikibugs>	 10DBA, 06Operations, 13Patch-For-Review: Decommission old coredb machines (<=db1050) - https://phabricator.wikimedia.org/T134476#3242638 (10Marostegui)
[06:45:02] <wikibugs>	 10DBA, 06Operations, 10ops-eqiad: Decommission db1024 - https://phabricator.wikimedia.org/T164702#3242637 (10Marostegui)
[08:49:05] <jynus>	 I will show you something nice on our meeting :-)
[08:50:13] <marostegui>	 \o/
[08:50:21] <marostegui>	 does it has to be with mydumper?
[08:50:27] <marostegui>	 dbstore running fully innodb maybe?
[08:50:28] <jynus>	 nope
[08:50:29] <marostegui>	 XD
[08:50:36] <jynus>	 easy, tiger
[08:50:48] <jynus>	 don't get too excited, it is a 2-hour work
[08:50:56] <marostegui>	 haha
[08:51:00] <marostegui>	 ok ok!
[08:59:26] <marostegui>	 jynus: you playing with db1041 from s7?
[08:59:47] <jynus>	 41?
[08:59:54] <marostegui>	 yep
[08:59:57] <marostegui>	 it was lagging
[09:00:01] <jynus>	 do you see something bad or just want to do something with it
[09:00:05] <jynus>	 not that I know
[09:00:07] <marostegui>	 it is lagging a bit
[09:00:07] <marostegui>	 ok
[09:00:08] <marostegui>	 i will check
[09:00:10] <jynus>	 I think I depooled it
[09:00:14] <marostegui>	 yeah
[09:00:15] <marostegui>	 it is depooled
[09:00:17] <jynus>	 which could be related
[09:00:33] <jynus>	 if it is depooled or has 0 weight
[09:00:45] <jynus>	 it is not "waited" on batched jobs
[09:00:46] <marostegui>	 yeah, it is depooled
[09:00:49] <marostegui>	 ah :)
[09:00:56] <marostegui>	 yeah checking the graph it shows constant spikes
[09:00:58] <jynus>	 and as it probably has a larger master
[09:01:08] <marostegui>	 that is correct
[09:01:12] <jynus>	 it could explain it
[09:01:15] <marostegui>	 hehe and all that from memory!
[09:01:15] <marostegui>	 :)
[09:01:25] <jynus>	 well, all slaves have a larger slave
[09:01:34] <jynus>	 not too difficult to say
[09:01:46] <marostegui>	 :)
[09:01:50] <jynus>	 I would give a quick look at its disk
[09:01:50] <marostegui>	 it is gone now
[09:01:53] <marostegui>	 yeah
[09:01:55] <marostegui>	 iu am checking that
[09:02:04] <jynus>	 but I would not worry until it is pooled
[09:02:09] <jynus>	 and lagging
[09:02:29] <jynus>	 we should get rid of the weaker slves ASAP
[09:02:39] <marostegui>	 yeah
[09:02:46] <jynus>	 they either lag or drag the whole thing down
[09:03:38] <jynus>	 is that alter going to finish any time?
[09:04:24] <marostegui>	 haha
[09:04:25] <marostegui>	 I know...
[09:04:30] <marostegui>	 Hopefully today or tomorrow!
[09:04:47] <jynus>	 going to take a large  break, see you later
[09:04:53] <marostegui>	 o/
[10:41:23] <addshore>	 jynus: let me know when today you would like to try turning cognate on for the remaining sites!
[10:42:57] <marostegui>	 we are in a meeting right now
[10:43:00] <marostegui>	 will ping you later
[10:43:04] <addshore>	 okay! :)
[12:00:11] <wikibugs>	 10DBA, 10Wikidata, 07Performance, 15User-Daniel: DispatchChanges: Avoid long-lasting connections to the master DB - https://phabricator.wikimedia.org/T151681#3243421 (10Ladsgroup)
[12:00:13] <wikibugs>	 10DBA, 10Wikidata, 07Performance, 15User-Daniel, and 2 others: Use redis-based lock manager in dispatch changes in production - https://phabricator.wikimedia.org/T159826#3079907 (10Ladsgroup) 05Open>03Resolved
[13:17:27] <wikibugs>	 10DBA, 10Wikidata, 07Performance, 15User-Daniel: DispatchChanges: Avoid long-lasting connections to the master DB - https://phabricator.wikimedia.org/T151681#3243701 (10Ladsgroup) Okay, it's up and running. What's next?
[13:24:47] <wikibugs>	 10DBA, 10Wikidata, 07Performance, 15User-Daniel: DispatchChanges: Avoid long-lasting connections to the master DB - https://phabricator.wikimedia.org/T151681#3243748 (10daniel) @Ladsgroup You could look into {T162557} and {T162556}. Please talk to @hoo about it.
[13:59:33] <jynus>	 https://phabricator.wikimedia.org/P5395
[14:02:38] <marostegui>	 much better! :)
[14:05:03] <jynus>	 also I have added the read only parameter: https://phabricator.wikimedia.org/P5395#28982
[14:05:31] <marostegui>	 ah great! :)
[14:07:53] <wikibugs>	 10DBA: Run pt-table-checksum on s7 - https://phabricator.wikimedia.org/T163190#3243958 (10Marostegui) I have updated the dsns_s7 table to so it has the latest changes on that shard (new master)
[14:19:57] <wikibugs>	 10DBA: Run pt-table-checksum on s7 - https://phabricator.wikimedia.org/T163190#3244016 (10Marostegui) a:03Marostegui
[15:55:05] <wikibugs>	 07Blocked-on-schema-change, 10DBA, 10Wikidata, 13Patch-For-Review, 03Wikidata-Sprint: Deploy schema change for adding term_full_entity_id column to wb_terms table - https://phabricator.wikimedia.org/T162539#3244391 (10Marostegui) db2038 is done: ``` root@neodymium:/home/marostegui/git/software/dbtools# m...
[15:55:47] <wikibugs>	 10DBA, 10Wikidata, 13Patch-For-Review, 07Schema-change: Drop the useless wb_terms keys "wb_terms_entity_type" and "wb_terms_type" on "wb_terms" table - https://phabricator.wikimedia.org/T163548#3244393 (10Marostegui) db2038 is done: ``` root@neodymium:/home/marostegui/git/software/dbtools#  mysql --skip-ss...
[22:35:29] <wikibugs>	 10DBA, 10RESTBase: Investigate requirements for MySQL access in RESTBase - https://phabricator.wikimedia.org/T164805#3245958 (10Tgr)
[22:35:45] <wikibugs>	 10DBA, 10RESTBase, 10Reading List Service: Investigate requirements for MySQL access in RESTBase - https://phabricator.wikimedia.org/T164805#3245970 (10Tgr)
[23:30:12] <wikibugs>	 10DBA, 10RESTBase, 10Reading List Service: Investigate requirements for MySQL access in RESTBase - https://phabricator.wikimedia.org/T164805#3246089 (10mobrovac) To be more concrete, the scope of this ticket is the Reading Lists functionality. As such, we are not asking whether a Node.JS service can directly...
[23:31:04] <wikibugs>	 10DBA, 10RESTBase, 10Reading List Service, 06Services (watching), 15User-mobrovac: Investigate requirements for MySQL access in RESTBase - https://phabricator.wikimedia.org/T164805#3246091 (10mobrovac)