[01:47:49] 10DBA, 10MediaWiki-General-or-Unknown, 10Patch-For-Review, 10Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q2): [Bug] Update old nonuniformly distributed page_random values - https://phabricator.wikimedia.org/T208909 (10Krenair) >>! In T208909#4732695, @jcrespo wrote: > I have no idea what this... [02:04:58] 10DBA, 10MediaWiki-General-or-Unknown, 10Patch-For-Review, 10Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q2): [Bug] Update old nonuniformly distributed page_random values - https://phabricator.wikimedia.org/T208909 (10Tbayer) >>! In T208909#4727881, @tstarling wrote: >>>! In T208909#4727845, @... [02:32:38] 10DBA, 10MediaWiki-General-or-Unknown, 10Patch-For-Review, 10Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q2): [Bug] Update old nonuniformly distributed page_random values - https://phabricator.wikimedia.org/T208909 (10Tbayer) >>! In T208909#4733549, @Niedzielski wrote: > Thanks for the feedbac... [02:36:42] 10DBA, 10MediaWiki-General-or-Unknown, 10Patch-For-Review, 10Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q2): [Bug] Update old nonuniformly distributed page_random values - https://phabricator.wikimedia.org/T208909 (10Tbayer) >>! In T208909#4733549, @Niedzielski wrote: > Thanks for the feedbac... [09:02:22] 10DBA, 10Patch-For-Review, 10User-Banyek: Missing row in enwiki.archive on sanitarium - https://phabricator.wikimedia.org/T208954 (10Banyek) [09:15:14] o/ banyek [09:15:27] i'm cooking breakfast but available whenever you've got cycles [09:16:06] ok [09:16:08] hey [09:17:28] I need time b/c I am fixing the labs [09:18:07] acknowledged :) [09:20:40] I am downtimed db1106 and db1107 [09:20:48] I am downtimed db1106 and db1124 (sorry, fix) [09:25:14] 10DBA, 10Patch-For-Review, 10User-Banyek: Missing row in enwiki.archive on sanitarium - https://phabricator.wikimedia.org/T208954 (10Banyek) creating dump of enwiki.archive on db1106 with: ` mysqldump enwiki archive --skip-ssl --no-create-info > enwiki.archive.sql ` [09:46:45] 10DBA, 10Patch-For-Review, 10User-Banyek: Missing row in enwiki.archive on sanitarium - https://phabricator.wikimedia.org/T208954 (10Banyek) the table was copied to db1124 with `transfer.py db1106.eqiad.wmnet:/home/banyek/enwiki.archive.sql db1124.eqiad.wmnet:/home/banyek/` [10:17:19] 10DBA, 10Patch-For-Review, 10User-Banyek: Missing row in enwiki.archive on sanitarium - https://phabricator.wikimedia.org/T208954 (10Banyek) loading dump in db1124 with: ` mysql -S /run/mysqld/mysqld.s1.sock --skip-ssl enwiki hey, banyek. i'll be breaking for lunch/exercise in about an hour. how's labs looking? [11:16:17] the data is getting loaded [11:16:47] I am now reading the ticket you showed me but I didn't get to that part to see what I shall do with that [11:21:40] there's a proposed script to fix the problem: https://gerrit.wikimedia.org/r/#/c/mediawiki/core/+/472596/ [11:23:40] Ok, I see [11:23:59] so you need a CR (+1) from me on your script to run and fix the db? [11:24:28] I can't give that to you, sorry [11:24:31] I mean not now [11:25:14] -> I am pretty new here, and I am not confident enough [11:29:51] phudex^ ? [11:32:42] well, i'm not entirely sure when and where to run the script in the first place ;) [11:34:04] but yes, we were looking for feedback from a dba on the script [11:37:45] as far as I know we have hosts named mwmaint* which are made for running mediawiki maintenance scripts, but I don't know if it applies to ad-hoc scripts, or just ones which running all the time etc. [11:37:59] So I might be not help to you, BUT [11:38:12] I recommend to go to #wikimedia-operations [11:38:22] and ask for help there [11:38:28] but if I were you [11:39:05] I'd wait until Monday, because the two other DBA's will be here and they're here for a long time [11:40:25] I also subscribed to the task to see what will be the conclusion, and I hope I can help next time [11:40:38] alright. well, if you spot any glaring errors in the task or in the script, then lmk :) [11:40:54] 👍 indeed [11:41:22] thanks for your time though :) [11:41:39] sorry I wasn't more helpful :) [11:41:59] I mean :( I just can't type [11:42:58] the data loading in db1124 finished, now I am restarting replication on db1106 [11:43:01] banyek: thanks for pointing at mwmaint*, i've just found this: https://wikitech.wikimedia.org/wiki/Heterogeneous_deployment#Run_a_maintenance_script_on_a_wiki [11:45:34] 10DBA, 10Patch-For-Review, 10User-Banyek: Missing row in enwiki.archive on sanitarium - https://phabricator.wikimedia.org/T208954 (10Banyek) the replication filters were set back to the original ones [12:33:32] The replication catched up, I repool the host [12:34:17] 10DBA, 10Patch-For-Review, 10User-Banyek: Missing row in enwiki.archive on sanitarium - https://phabricator.wikimedia.org/T208954 (10Banyek) replication catched up [13:04:38] 10DBA, 10Patch-For-Review, 10User-Banyek: Missing row in enwiki.archive on sanitarium - https://phabricator.wikimedia.org/T208954 (10Banyek) 05Open>03Resolved a:03Banyek I re-run the check_private_data script on db1124 and the sanitarium is clean. [15:16:19] 10DBA, 10Operations, 10ops-codfw, 10Patch-For-Review, 10User-Banyek: rack/setup/install pc2007-pc2010 - https://phabricator.wikimedia.org/T207259 (10Papaul) a:05Papaul>03Banyek If this is in testing mode , I think it needs to be assigned to DB. [15:17:48] 10DBA, 10Operations, 10ops-codfw, 10Patch-For-Review, 10User-Banyek: rack/setup/install pc2007-pc2010 - https://phabricator.wikimedia.org/T207259 (10Banyek) Ah sorry we were talking about this off-phab: the stripe size will be good [15:23:05] 10DBA, 10MediaWiki-General-or-Unknown, 10Patch-For-Review, 10Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q2): [Bug] Update old nonuniformly distributed page_random values - https://phabricator.wikimedia.org/T208909 (10Anomie) >>! In T208909#4733549, @Niedzielski wrote: > 1. We don’t know how l... [15:36:28] 10DBA, 10MediaWiki-General-or-Unknown, 10Patch-For-Review, 10Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q2): [Bug] Update old nonuniformly distributed page_random values - https://phabricator.wikimedia.org/T208909 (10Niedzielski) @anomie, thank you for this thoughtful feedback. It's very much... [16:57:43] 10DBA, 10MediaWiki-General-or-Unknown, 10Patch-For-Review, 10Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q2): [Bug] Update old nonuniformly distributed page_random values - https://phabricator.wikimedia.org/T208909 (10Anomie) I don't see anything where your stuff would interact with my stuff. [17:28:56] 10DBA, 10Analytics, 10Data-Services: Not able to scoop coment table in labs for mediawiki reconstruction process. - https://phabricator.wikimedia.org/T209031 (10Nuria) [17:43:22] 10DBA, 10Analytics, 10Data-Services: Not able to scoop coment table in labs for mediawiki reconstruction process. - https://phabricator.wikimedia.org/T209031 (10Nuria) [19:31:40] 10DBA, 10Analytics, 10Data-Services: Not able to scoop comment table in labs for mediawiki reconstruction process - https://phabricator.wikimedia.org/T209031 (10Reedy) [19:47:39] 10DBA, 10Analytics, 10Data-Services: Not able to scoop comment table in labs for mediawiki reconstruction process - https://phabricator.wikimedia.org/T209031 (10Milimetric) @Krenair: there were a handful of discussions, but the gist of the reasoning is that we want to pull data *after* it's sanitized. If we... [20:08:31] 10DBA, 10Analytics, 10Data-Services: Not able to scoop comment table in labs for mediawiki reconstruction process - https://phabricator.wikimedia.org/T209031 (10Milimetric) @Marostegui and @jcrespo, I want to touch base with one or both of you on this, it turns out to be a tricky problem from an analytics po... [20:54:30] 10DBA, 10Analytics, 10Data-Services: Not able to scoop comment table in labs for mediawiki reconstruction process - https://phabricator.wikimedia.org/T209031 (10Nuria) pinging @JAllemandou and @elukey Let's please follow up with dbas early next week on this ticket. [23:21:59] 10DBA, 10MediaWiki-General-or-Unknown, 10Patch-For-Review, 10Readers-Web-Backlog (Readers-Web-Kanbanana-Board-2018-19-Q2): [Bug] Update old nonuniformly distributed page_random values - https://phabricator.wikimedia.org/T208909 (10ovasileva) p:05Normal>03High