[01:39:07] 10DBA, 10XTools: Xtools Beta showing incorrect graph - https://phabricator.wikimedia.org/T172045#3484915 (10Matthewrbowker) A quick Quarry look shows that you do indeed have 437 edits without summary. Link: https://quarry.wmflabs.org/query/20618 XTools is showing what it knows to be accurate data. I'm handi... [02:39:52] 10DBA, 10XTools: Xtools Beta showing incorrect graph - https://phabricator.wikimedia.org/T172045#3484944 (10MusikAnimal) So looks like the colours for summaries vs no summaries are inversed! I'm guessing that is what PokestarFan was referring to. Here we thought the numbers were being disputed :) This an easy... [02:43:02] 10DBA, 10XTools: Colours of edit summary graph are inverted - https://phabricator.wikimedia.org/T172045#3484946 (10MusikAnimal) [05:13:51] 10DBA, 10Operations, 10ops-eqiad: Degraded RAID on db1068 - https://phabricator.wikimedia.org/T171723#3485006 (10Marostegui) 05Open>03Resolved a:03Cmjohnson This is now fixed: ``` root@db1068:~# megacli -LDPDInfo -aAll Adapter #0 Number of Virtual Disks: 1 Virtual Drive: 0 (Target Id: 0) Name... [05:14:20] 10DBA, 10Cloud-Services, 10User-MarcoAurelio: Prepare and check storage layer for hi.wikiversity - https://phabricator.wikimedia.org/T171829#3485010 (10Marostegui) [07:54:52] 10DBA, 10MediaWiki-Watchlist, 10Wikimedia-General-or-Unknown, 10Wikimedia-log-errors: User with 40000 entries in their Watchlist cannot access it on Commons anymore: Database error - https://phabricator.wikimedia.org/T171898#3479604 (10Marostegui) Do you happen to have the query handy? I am guessing that... [07:57:00] 10DBA, 10Analytics, 10Analytics-EventLogging, 10Community-Tech, 10User-Elukey: Drop CookieBlock* tables from EventLogging DB - https://phabricator.wikimedia.org/T171883#3485104 (10Marostegui) Will #analytics handle this? I would suggest to rename the tables on all the hosts and leave them with a differe... [07:57:09] 10DBA, 10Analytics, 10Analytics-EventLogging, 10Community-Tech, 10User-Elukey: Drop CookieBlock* tables from EventLogging DB - https://phabricator.wikimedia.org/T171883#3485106 (10Marostegui) p:05Triage>03Normal [07:59:05] 10DBA, 10Wikimedia-General-or-Unknown, 10Performance: Clean up skin properties - https://phabricator.wikimedia.org/T171643#3471975 (10Marostegui) > After solving the subtask, we could run the following query on all wikis: I would prefer if this gets run by a maintenance script using mediawiki, we normally d... [08:00:53] 10DBA, 10Analytics, 10Analytics-EventLogging, 10Community-Tech, 10User-Elukey: Drop CookieBlock* tables from EventLogging DB - https://phabricator.wikimedia.org/T171883#3479233 (10elukey) Sure we can handle it, renaming sounds good. As far as I can understand dropping the renamed table after some days se... [08:02:35] 10DBA, 10Analytics, 10Analytics-EventLogging, 10Community-Tech, 10User-Elukey: Drop CookieBlock* tables from EventLogging DB - https://phabricator.wikimedia.org/T171883#3485112 (10Marostegui) >>! In T171883#3485110, @elukey wrote: > Sure we can handle it, renaming sounds good. As far as I can understand... [08:39:53] 10DBA, 10MediaWiki-extensions-ClickTracking, 10Operations: Drop the tables old_growth, hitcounter, click_tracking, click_tracking_user_properties from enwiki, maybe other schemas - https://phabricator.wikimedia.org/T115982#3485197 (10Marostegui) The empty table old_growth has been dropped from enwiki. [08:39:57] 10DBA, 10MediaWiki-extensions-ClickTracking, 10Operations: Drop the tables old_growth, hitcounter, click_tracking, click_tracking_user_properties from enwiki, maybe other schemas - https://phabricator.wikimedia.org/T115982#3485198 (10Marostegui) [08:46:57] 10DBA: Investigate (and if possible, drop) _counters - https://phabricator.wikimedia.org/T145487#3485204 (10TerraCodes) [08:50:36] 10DBA, 10MediaWiki-extensions-ClickTracking, 10Operations: Drop the tables old_growth, hitcounter, click_tracking, click_tracking_user_properties from enwiki, maybe other schemas - https://phabricator.wikimedia.org/T115982#3485206 (10Marostegui) On `db1089` (enwiki) I have renamed `click_tracking` and `click... [09:12:31] can I do a db edit on db1102:3317? [09:12:37] (change a grant) [09:12:43] yep! [09:13:04] do I do it with or without binlog? [09:13:13] do it with [09:13:20] so it gets replicate as i just finished the mydumper [09:13:23] ok [09:13:24] and will start with myloader [09:13:28] on labs [09:13:49] I may have already run it on labs, not sure [09:13:56] but it was missing from 1102 [09:14:13] ah [09:14:24] multi-instance prometheus assumes passwordless authentication [09:14:34] based on uid [09:14:46] collections were failing on prometheus [09:15:02] it is a grant, so it can be executed multiple times [09:15:09] yeah [09:15:14] no harm in using binlog :) [09:15:29] actually, there is [09:15:34] will tell you later [09:15:38] oki! [09:16:22] now it starts working: https://grafana.wikimedia.org/dashboard/db/mysql-aggregated?panelId=4&fullscreen&orgId=1 [09:17:31] I think db1096 is there by mistake [09:46:23] 10DBA, 10Cloud-Services, 10Toolforge: labsdb1001 and labsdb1003 short on available space - https://phabricator.wikimedia.org/T132431#3485437 (10Marostegui) [09:46:29] 10DBA, 10Cloud-Services, 10Tool-Erwin's-tools, 10Toolforge, 10Tools: s51362 has been rate limited to 2 concurrent connections for creating hundreds of 1400-second queries to labsdb1001 and labsdb1003 every 10 seconds - https://phabricator.wikimedia.org/T162519#3485434 (10Marostegui) 05Open>03Resolved... [09:47:54] 10DBA, 10Data-Services, 10Security-Team, 10WMF-Legal, and 5 others: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3485449 (10Marostegui) Any update about this? [14:44:55] 10DBA, 10Data-Services, 10Security-Team, 10WMF-Legal, and 5 others: Make wbqc_constraints table available on Quarry et al. - https://phabricator.wikimedia.org/T170927#3486381 (10bd808) ping for @Bawolff, @dpatrick, @Reedy. If #security-team can confirm that this is already public data on wiki I think we ca... [14:59:38] 10DBA, 10Patch-For-Review: Finish dbstore2002 migration to multi-instance - https://phabricator.wikimedia.org/T171321#3486447 (10Marostegui) a:05jcrespo>03Marostegui [15:03:02] 10DBA, 10Patch-For-Review: Productionize 22 new codfw database servers - https://phabricator.wikimedia.org/T170662#3486453 (10Marostegui) [15:15:53] 10DBA, 10Scoring-platform-team, 10articlequality-modeling, 10artificial-intelligence: [Discuss] Hosting the monthly article quality dataset on labsDB - https://phabricator.wikimedia.org/T146718#3486484 (10Halfak) 05Resolved>03Open Looks like I'm failing to create one important index on the table. ```... [15:19:03] 10DBA, 10Scoring-platform-team, 10articlequality-modeling, 10artificial-intelligence: [Discuss] Hosting the monthly article quality dataset on labsDB - https://phabricator.wikimedia.org/T146718#3486489 (10Marostegui) >>! In T146718#3486484, @Halfak wrote: > Looks like I'm failing to create one important in... [15:21:06] 10DBA, 10Analytics, 10Contributors-Analysis, 10Chinese-Sites: Data Lake edit data missing for many wikis - https://phabricator.wikimedia.org/T165233#3486496 (10Nuria) Ping @Marostegui: do we have an ETA on when these wikis will be available on labs new db hosts? Is it still end of Q1? @Neil_P._Quinn_WMF T... [15:22:47] 10DBA, 10Analytics, 10Contributors-Analysis, 10Chinese-Sites: Data Lake edit data missing for many wikis - https://phabricator.wikimedia.org/T165233#3486500 (10Marostegui) >>! In T165233#3486496, @Nuria wrote: > Ping @Marostegui: do we have an ETA on when these wikis will be available on labs new db hosts?... [15:24:46] 10DBA, 10Analytics, 10Contributors-Analysis, 10Chinese-Sites: Data Lake edit data missing for many wikis - https://phabricator.wikimedia.org/T165233#3486505 (10Nuria) Excellent, so we can count on all being available by end of this quarter! [15:25:22] 10DBA, 10Analytics, 10Contributors-Analysis, 10Chinese-Sites: Data Lake edit data missing for many wikis - https://phabricator.wikimedia.org/T165233#3486506 (10Marostegui) >>! In T165233#3486505, @Nuria wrote: > Excellent, so we can count on all being available by end of this quarter! We are hoping to hav... [15:31:06] 10DBA, 10Scoring-platform-team, 10articlequality-modeling, 10artificial-intelligence: [Discuss] Hosting the monthly article quality dataset on labsDB - https://phabricator.wikimedia.org/T146718#3486557 (10Halfak) Maybe? ``` $ host enwiki.labsdb enwiki.labsdb has address 10.64.4.11 ``` [15:32:29] 10DBA, 10Scoring-platform-team, 10articlequality-modeling, 10artificial-intelligence: [Discuss] Hosting the monthly article quality dataset on labsDB - https://phabricator.wikimedia.org/T146718#3486575 (10Marostegui) >>! In T146718#3486557, @Halfak wrote: > Maybe? > > ``` > $ host enwiki.labsdb > enwiki... [15:35:31] 10DBA, 10Analytics-EventLogging, 10Analytics-Kanban, 10Community-Tech, 10User-Elukey: Drop CookieBlock* tables from EventLogging DB - https://phabricator.wikimedia.org/T171883#3486618 (10mforns) [16:43:22] 10DBA, 10Scoring-platform-team, 10articlequality-modeling, 10artificial-intelligence: [Discuss] Hosting the monthly article quality dataset on labsDB - https://phabricator.wikimedia.org/T146718#3486863 (10Marostegui) 05Open>03Resolved ``` mysql:root@localhost [u2041__ores_p]> CREATE INDEX page_idx ON m... [17:01:59] 10DBA, 10Scoring-platform-team, 10articlequality-modeling, 10artificial-intelligence: [Discuss] Hosting the monthly article quality dataset on labsDB - https://phabricator.wikimedia.org/T146718#3486911 (10Halfak) <3! [17:30:59] 10DBA, 10Reading-Web-Backlog (Tracking): Drop WikiGrok tables from production - https://phabricator.wikimedia.org/T172020#3486996 (10Jdlrobson) [17:53:50] 10DBA, 10MediaWiki-extensions-WikibaseClient, 10Wikidata, 10Patch-For-Review, and 3 others: wikibase-addUsagesForPage doesn't batch properly - https://phabricator.wikimedia.org/T172015#3482899 (10Ladsgroup) Let's keep it open as reminder to check logstash when it got deployed. [18:01:56] 10DBA, 10MediaWiki-Watchlist, 10Wikimedia-General-or-Unknown, 10Wikimedia-log-errors: User with 40000 entries in their Watchlist cannot access it on Commons anymore: Database error - https://phabricator.wikimedia.org/T171898#3487116 (10Catrope) >>! In T171898#3485101, @Marostegui wrote: > Do you happen to... [19:47:13] 10DBA, 10Analytics, 10Contributors-Analysis, 10Chinese-Sites: Data Lake edit data missing for many wikis - https://phabricator.wikimedia.org/T165233#3487492 (10Neil_P._Quinn_WMF) >>! In T165233#3486496, @Nuria wrote: > @Neil_P._Quinn_WMF Team agreed to only do prod snapshots ad hoc as we hope they are not... [19:48:51] 10DBA, 10Analytics, 10Contributors-Analysis, 10Chinese-Sites: Data Lake edit data missing for many wikis - https://phabricator.wikimedia.org/T165233#3487509 (10Nuria) @Neil_P._Quinn_WMF we are going to be short 2 people in the team in the upcoming days, i cannot promise we can get this done in the next cou... [20:55:20] 10DBA, 10Cloud-Services: Prepare and check storage layer for hi.wikiversity - https://phabricator.wikimedia.org/T171829#3477438 (10MarcoAurelio) [21:17:41] 10DBA, 10Analytics, 10Contributors-Analysis, 10Chinese-Sites: Data Lake edit data missing for many wikis - https://phabricator.wikimedia.org/T165233#3487721 (10Neil_P._Quinn_WMF) >>! In T165233#3487509, @Nuria wrote: > @Neil_P._Quinn_WMF we are going to be short 2 people in the team in the upcoming days, i... [22:22:45] 10DBA, 10Wikimedia-General-or-Unknown, 10Performance: Clean up skin properties - https://phabricator.wikimedia.org/T171643#3487918 (10demon) >>! In T171643#3485107, @Marostegui wrote: >> After solving the subtask, we could run the following query on all wikis: > > I would prefer if this gets run by a mainte...