[07:45:19] DBA, Labs, Patch-For-Review: Make watchlist table available on labs - https://phabricator.wikimedia.org/T59617#2370748 (jcrespo) > The Toolserver previously used MySQL views for this An escalation of privileges vulnerability would expose user private data. It also exposes the threat of someone poten... [08:21:44] DBA, Operations, Wikidata, Performance: EntityUsageTable::getUsedEntityIdStrings query on wbc_entity_usage table is sometimes fast, sometimes slow - https://phabricator.wikimedia.org/T116404#2370849 (jcrespo) This is not a problem with the servers, the query planner, or the indexing: ``` MariaDB... [09:08:15] DBA, Labs, Labs-Infrastructure, Operations, Blocked-on-Operations: No replica for adywiki - https://phabricator.wikimedia.org/T135029#2371001 (Gehel) some reverse engineering has already been done by @jcrespo, documented on [[ https://wikitech.wikimedia.org/wiki/MariaDB/Sanitarium_and_Labsdbs... [09:35:34] DBA, Labs, Operations: disk failure on labsdb1002 - https://phabricator.wikimedia.org/T126946#2371014 (jcrespo) It is scheduled. It is difficult to give an estimation, but it can be done after enwiki is finished, so 3-6 months? Labs hosts, by its own nature cannot and will probably not be 100% ever... [09:55:08] DBA, Operations, Wikidata, Performance: EntityUsageTable::getUsedEntityIdStrings query on wbc_entity_usage table is sometimes fast, sometimes slow - https://phabricator.wikimedia.org/T116404#2371028 (daniel) @jcrespo @hoo: ick, 14 million rows? And this isn't optimized away because of the DISTINC... [10:20:52] DBA, Operations, Wikidata, Performance: EntityUsageTable::getUsedEntityIdStrings query on wbc_entity_usage table is sometimes fast, sometimes slow - https://phabricator.wikimedia.org/T116404#2371094 (jcrespo) @daniel I think this is a case of [[ http://ubiquity.acm.org/article.cfm?id=1513451 | pr... [10:51:32] DBA, Labs, Operations: disk failure on labsdb1002 - https://phabricator.wikimedia.org/T126946#2371192 (Blahma) Thank you. I did not realize there were also SQL dumps, not only XML. Would it perhaps be possible to have the latest dump readily available on an SQL server? That could be an alternative fo... [10:51:55] DBA, Operations, ops-codfw: es2017 and es2019 crashed with no logs - https://phabricator.wikimedia.org/T130702#2371193 (jcrespo) This is ok, just ping me today early or next week to get it done. [11:03:08] DBA, Operations, Wikidata, Performance: EntityUsageTable::getUsedEntityIdStrings query on wbc_entity_usage table is sometimes fast, sometimes slow - https://phabricator.wikimedia.org/T116404#2371207 (daniel) @jcrespo so we should do one query per ID, with limit 1? ok! [11:10:39] DBA, Wikidata, Patch-For-Review, Wikidata-Sprint-2016-05-24: Wikidata master database connection issue - https://phabricator.wikimedia.org/T136598#2371228 (daniel) [11:11:10] DBA, Wikidata, Patch-For-Review, Wikidata-Sprint-2016-05-24: Wikidata master database connection issue - https://phabricator.wikimedia.org/T136598#2340469 (daniel) >>! In T136598#2369366, @hoo wrote: > The fix for this triggered {T116404}, not sure how to continue from here. We could potentially... [11:17:39] DBA, Labs, Operations: disk failure on labsdb1002 - https://phabricator.wikimedia.org/T126946#2371239 (jcrespo) @Blahma what do you want me to do? I can load those files, but those would be out of sync as soon as they are imported, and impossible to get updated. You can load those tables to the same... [12:05:33] DBA, Labs, Operations: disk failure on labsdb1002 - https://phabricator.wikimedia.org/T126946#2371339 (Blahma) @jcrespo Thanks for staying on the constructive line. FYI, the output in question is https://cs.wikipedia.org/wiki/Wikipedie:%C3%9Adr%C5%BEba/Nekategorizovan%C3%A9_%C4%8Dl%C3%A1nky_s_ohledem... [12:46:09] DBA, Labs, Operations: disk failure on labsdb1002 - https://phabricator.wikimedia.org/T126946#2371458 (jcrespo) This is very easy to fix- tell your users to mark those that are incorrect, and exclude them from your query- that is very easy to do and doesn't require waiting. [13:12:29] DBA, Operations, ops-codfw: db2034 crash - https://phabricator.wikimedia.org/T137084#2371534 (jcrespo) [13:12:31] DBA, Operations, ops-codfw: db2034 degraded RAID - https://phabricator.wikimedia.org/T136583#2371536 (jcrespo) [13:16:09] DBA, Labs, Operations: disk failure on labsdb1002 - https://phabricator.wikimedia.org/T126946#2371555 (MZMcBride) >>! In T126946#2371458, @jcrespo wrote: > This is very easy to fix- tell your users to mark those that are incorrect, and exclude them from your query- that is very easy to do and doesn't... [13:18:56] DBA, Labs, Operations: disk failure on labsdb1002 - https://phabricator.wikimedia.org/T126946#2371577 (jcrespo) > The latest "solution" to the constant stream of data integrity issues on Wikimedia Labs database replicas is to further inconvenience volunteers? No, the latest solution is the reimport... [16:06:38] DBA, MediaWiki-extensions-ORES: Review ORES extension for data leaks - https://phabricator.wikimedia.org/T137567#2372053 (Halfak) [16:07:24] DBA, MediaWiki-extensions-ORES: Review ORES extension for data leaks - https://phabricator.wikimedia.org/T137567#2372075 (Halfak) [16:08:20] DBA, MediaWiki-extensions-ORES: Review ORES extension for data leaks - https://phabricator.wikimedia.org/T137567#2372053 (Halfak) [16:10:29] DBA, MediaWiki-extensions-ORES: Review ORES extension for data leaks - https://phabricator.wikimedia.org/T137567#2372083 (jcrespo) Please share the gerrit commit link with the update tables script. [16:12:01] DBA, Wikidata, Patch-For-Review, Wikidata-Sprint-2016-05-24: Wikidata master database connection issue - https://phabricator.wikimedia.org/T136598#2372093 (hoo) [16:12:04] DBA, Operations, Wikidata, Performance: EntityUsageTable::getUsedEntityIdStrings query on wbc_entity_usage table is sometimes fast, sometimes slow - https://phabricator.wikimedia.org/T116404#2372089 (hoo) Open>declined We'll do {T137539} instead. [16:26:17] DBA, MediaWiki-extensions-ORES: Review ORES extension for data leaks - https://phabricator.wikimedia.org/T137567#2372152 (Ladsgroup) @jcrespo [[https://github.com/wikimedia/mediawiki-extensions-ORES/tree/master/sql | This is the schema change]]. Do you need anything else? The gerrit patch is [[https://ge... [16:27:33] DBA, MediaWiki-extensions-ORES, Revision-Scoring-As-A-Service: Review ORES extension for data leaks - https://phabricator.wikimedia.org/T137567#2372170 (Ladsgroup) [16:28:43] DBA, MediaWiki-extensions-ORES, Revision-Scoring-As-A-Service: Review ORES extension for data leaks - https://phabricator.wikimedia.org/T137567#2372053 (Ladsgroup) Oh I need to note, these schema changes are being invoked using [[https://github.com/wikimedia/mediawiki-extensions-ORES/blob/master/incl... [16:40:47] DBA, MediaWiki-extensions-ORES, Revision-Scoring-As-A-Service: Review ORES extension for data leaks - https://phabricator.wikimedia.org/T137567#2372228 (jcrespo) What is the privacy requirements of these score tables? Are they all public? Is their privacy dependent on the revision they refer to? What... [16:55:18] DBA, MediaWiki-extensions-ORES, Revision-Scoring-As-A-Service: Review ORES extension for data leaks - https://phabricator.wikimedia.org/T137567#2372283 (Ladsgroup) All of these data came from a public endpoint (the ORES service). So no private dependencies. We do not support private wikis (yet) becau... [17:10:02] DBA, MediaWiki-extensions-ORES, Revision-Scoring-As-A-Service: Review ORES extension for data leaks - https://phabricator.wikimedia.org/T137567#2372370 (Ladsgroup) Sure, Thanks :) [17:51:21] DBA, Tracking: Database tables to be dropped on Wikimedia wikis and other WMF databases (tracking) - https://phabricator.wikimedia.org/T54921#2372501 (Nemo_bis) > tables available on a wiki multiply its storage needs by 20- 40 Maybe a list of table names sorted by combined storage needs across all wikis... [19:23:00] DBA, Operations, hardware-requests, ops-codfw, Patch-For-Review: Decommission es2005-es2010 - https://phabricator.wikimedia.org/T134755#2372708 (Papaul) [19:34:57] DBA, Operations, hardware-requests, ops-codfw, Patch-For-Review: Decommission es2005-es2010 - https://phabricator.wikimedia.org/T134755#2372731 (Papaul) [19:36:16] DBA, Operations, hardware-requests, ops-codfw, Patch-For-Review: Decommission es2005-es2010 - https://phabricator.wikimedia.org/T134755#2275897 (Papaul) a:Papaul>RobH @Robh all steps complete you good to remove the entries on the switches.