[00:03:29] 10DBA, 10Community-Tech, 10MediaWiki-General-or-Unknown, 10Operations, and 3 others: Regularly purge expired temporary userrights from DB tables - https://phabricator.wikimedia.org/T176754#3663102 (10EddieGP) >>! In T176754#3659854, @jcrespo wrote: > Let's stop meta-talking about the issue, and start worki... [00:21:35] 10DBA, 10Community-Tech, 10MediaWiki-General-or-Unknown, 10Operations, and 3 others: Regularly purge expired temporary userrights from DB tables - https://phabricator.wikimedia.org/T176754#3635590 (10Krinkle) >>! In T176754#3658385, @kaldari wrote: >>If purging is undesirable on production, which is someth... [03:22:54] 10DBA, 10cloud-services-team: Request to increase active connection quota for user s51187 on analytics.db.svc.eqiad.wmflabs - https://phabricator.wikimedia.org/T177570#3663283 (10MusikAnimal) [03:23:03] 10DBA, 10XTools, 10cloud-services-team: Request to increase active connection quota for user s51187 on analytics.db.svc.eqiad.wmflabs - https://phabricator.wikimedia.org/T177570#3663297 (10MusikAnimal) [03:35:06] 10DBA, 10XTools, 10cloud-services-team: Request to increase active connection quota for user s51187 on analytics.db.svc.eqiad.wmflabs - https://phabricator.wikimedia.org/T177570#3663307 (10MusikAnimal) Or perhaps that's just it... the connection limit was higher on `tools.labsdb` where the temporary tables w... [03:56:38] 10DBA, 10Data-Services, 10XTools, 10cloud-services-team: Request to increase active connection quota for user s51187 on analytics.db.svc.eqiad.wmflabs - https://phabricator.wikimedia.org/T177570#3663345 (10bd808) [05:30:24] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Drop now redundant indexes from pagelinks and templatelinks - https://phabricator.wikimedia.org/T174509#3663396 (10Marostegui) [05:31:42] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Drop now redundant indexes from pagelinks and templatelinks - https://phabricator.wikimedia.org/T174509#3642638 (10Marostegui) [05:43:22] 10DBA, 10Epic, 10Patch-For-Review, 10codfw-rollout: Database maintenance scheduled while eqiad datacenter is non primary (after the DC switchover) - https://phabricator.wikimedia.org/T155099#3663411 (10Marostegui) [05:43:26] 10DBA, 10MediaWiki-Database, 10Patch-For-Review, 10PostgreSQL, 10Schema-change: Some tables lack unique or primary keys, may allow confusing duplicate data - https://phabricator.wikimedia.org/T17441#3663412 (10Marostegui) [05:43:28] 10Blocked-on-schema-change, 10DBA: Convert unique keys into primary keys for some wiki tables on s3-eqiad and s3-codfw - https://phabricator.wikimedia.org/T163912#3663409 (10Marostegui) 05Open>03Resolved codfw is now done [05:43:35] 10Blocked-on-schema-change, 10DBA: Convert unique keys into primary keys for some wiki tables on s3-eqiad and s3-codfw - https://phabricator.wikimedia.org/T163912#3663413 (10Marostegui) [05:56:22] 10DBA, 10Cloud-Services, 10cloud-services-team: Prepare and check storage layer for amwikimedia - https://phabricator.wikimedia.org/T176043#3663430 (10Marostegui) Sanitarium host was sanitized and check_private_data reported no issues. So this is ready for #cloud-services-team to do their magic and resolve t... [06:02:38] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Drop now redundant indexes from pagelinks and templatelinks - https://phabricator.wikimedia.org/T174509#3663436 (10Marostegui) [06:02:59] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Drop now redundant indexes from pagelinks and templatelinks - https://phabricator.wikimedia.org/T174509#3642654 (10Marostegui) [06:03:41] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Drop now redundant indexes from pagelinks and templatelinks - https://phabricator.wikimedia.org/T174509#3642672 (10Marostegui) [06:22:43] 10DBA, 10Data-Services, 10XTools, 10cloud-services-team: Request to increase active connection quota for user s51187 on analytics.db.svc.eqiad.wmflabs - https://phabricator.wikimedia.org/T177570#3663283 (10Marostegui) This user has no connection limits on the old servers (I guess for historic reasons?). Ri... [06:23:00] 10DBA, 10Patch-For-Review: Productionize 11 new eqiad database servers - https://phabricator.wikimedia.org/T172679#3663447 (10Marostegui) [06:23:56] 10DBA, 10Patch-For-Review: Productionize 11 new eqiad database servers - https://phabricator.wikimedia.org/T172679#3663449 (10Marostegui) 05Open>03Resolved a:03Marostegui All these hosts have been productionized already. I am going to close this task as resolved as they are now all in production. They m... [06:25:38] 10DBA, 10Cloud-Services, 10Toolforge: What would be the preferred way to run a series of read queries on DB replica, each on two *_p databases ? - https://phabricator.wikimedia.org/T124805#3663454 (10Marostegui) 05Open>03Resolved Going to mark this as resolved for now as it is an old one and as Jaime sai... [06:29:47] 10DBA: Analyze memory usage on db1011 - https://phabricator.wikimedia.org/T131146#3663458 (10Marostegui) 05Open>03Resolved Unfortunately db1011 has these issues quite often but I don't think anyone is taking a look at this as it is an old ticket :-) This server will be decommissioned anyways as part of: T134... [06:52:10] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Apply schema change to add 3D filetype for STL files - https://phabricator.wikimedia.org/T168661#3663479 (10Marostegui) 05stalled>03Open [06:55:45] 10DBA, 10Gerrit, 10Operations, 10Patch-For-Review, 10Release-Engineering-Team (Backlog): Gerrit shows HTTP 500 error when pasting extended unicode characters - https://phabricator.wikimedia.org/T145885#3663481 (10Marostegui) >>! In T145885#3555784, @Paladox wrote: > I doint think this is worth it now sin... [07:21:28] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Apply schema change to add 3D filetype for STL files - https://phabricator.wikimedia.org/T168661#3663516 (10Legoktm) OK, I tested it by putting Commons into read-only mode using the same method that is planned. My hunch was correct, and that adding a fil... [07:21:37] marostegui: ^ [07:31:45] lovely... [07:32:47] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Apply schema change to add 3D filetype for STL files - https://phabricator.wikimedia.org/T168661#3663518 (10Marostegui) Thanks @Legoktm for testing this. @aaron @Bawolff any advise? [07:33:31] legoktm: I assume this isn't easy to fix right? [07:34:01] well I'm not really sure what the proper behavior should be for the globalusage issue [07:34:26] I'm figuring out why the read only message isn't showing up, that should be easy [07:34:41] yeah, that would be really useful indeed [07:34:43] thanks a lot [07:35:25] I suppose we'd just tell people they can't add images while in read only? but that could cause some messy vandalism where people delete images and now you can't undo their edit...so we might as well just make all the wikis read only [07:35:52] That would be a pretty big maintenance I guess [07:47:49] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Apply schema change to add 3D filetype for STL files - https://phabricator.wikimedia.org/T168661#3663591 (10Legoktm) Once an administrator implements https://commons.wikimedia.org/wiki/MediaWiki_talk:Readonlywarning that will fix the issue of the read on... [07:48:55] legoktm: nice finding :) [07:49:58] legoktm: just to be clear, there nothing from our (dbas) side to be done, right? that is to be done by the wikicommons admins? [07:50:06] yep [07:50:14] \o/ [07:50:34] I should have clarified "Commons administrator" :p [07:50:46] haha [07:57:42] I've added the request to https://commons.wikimedia.org/wiki/Commons:Administrators%27_noticeboard#Please_see_mediawiki_message_change_request_on_MediaWiki_talk:Readonlywarning [08:01:33] :) [08:01:39] thanks jynus! [08:02:22] any other suggestions for things to test? (I've just been live hacking db-eqiad.php on mwdebug1002 to make Commons read-only) [08:03:02] 10DBA, 10Analytics: Drop MoodBar tables from all wikis - https://phabricator.wikimedia.org/T153033#3663654 (10Marostegui) I have backuped the tables on: ``` root@dbstore1001:/srv/tmp/T153033# pwd /srv/tmp/T153033 root@dbstore1001:/srv/tmp/T153033# ls -lh total 16M -rw-r--r-- 1 root root 14M Oct 6 07:56 moodb... [08:16:44] 10DBA, 10Operations, 10Availability (Multiple-active-datacenters), 10Performance-Team (Radar): Make apache/maintenance hosts TLS connections to mariadb work - https://phabricator.wikimedia.org/T175672#3663663 (10jcrespo) I will setup that and ping you. Independently of this, not supporting TLS 1.2 (if that... [08:20:34] 10DBA, 10Analytics: Drop MoodBar tables from all wikis - https://phabricator.wikimedia.org/T153033#3663680 (10Marostegui) a:03Marostegui I have imported those tables into db1047 and dbstore1002 with the name of the wiki at the start: ``` root@EVENTLOGGING m4[staging]> select @@hostname; +------------+ | @@ho... [08:29:23] 10DBA, 10Wikidata: Migrate wb_terms to using prefixed entity IDs instead of numeric IDs - https://phabricator.wikimedia.org/T114903#3663724 (10Ladsgroup) [08:29:45] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Drop now redundant indexes from pagelinks and templatelinks - https://phabricator.wikimedia.org/T174509#3663725 (10Marostegui) [08:42:09] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Apply schema change to add 3D filetype for STL files - https://phabricator.wikimedia.org/T168661#3663735 (10jcrespo) I would compress the steps: > Set commonswiki on read-only by merging: https://gerrit.wikimedia.org/r/#/c/382379/1 > Double check indeed... [08:49:50] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Apply schema change to add 3D filetype for STL files - https://phabricator.wikimedia.org/T168661#3663748 (10Marostegui) >>! In T168661#3663735, @jcrespo wrote: > Into >> apt update && apt full-upgrade >> Set commonswiki on read-only by merging: https://g... [08:51:10] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Apply schema change to add 3D filetype for STL files - https://phabricator.wikimedia.org/T168661#3663749 (10jcrespo) > you would still want to run mysql_upgrade after read-only is gone? Yes, precisely because it will be noop, so no need to block on that. [08:53:03] 10Blocked-on-schema-change, 10DBA, 10Patch-For-Review: Apply schema change to add 3D filetype for STL files - https://phabricator.wikimedia.org/T168661#3663752 (10Marostegui) Cool! Fine by me, so let's do that procedure then - thanks!: apt update && apt full-upgrade Set commonswiki on read-only by merging:... [09:01:42] legoktm: question- shouldn't be globalimagelinks done asnychronously? [09:03:05] it kind of is - MW has the concept of deferred updates which run in the same web request, but only after the response has been sent to the user [09:03:07] maybe it is a stupid suggestion, I just thought those kind of cross-upgrades were all through the job queue [09:03:54] allowing them to fail a bit [09:05:01] I'm actually not sure why it doesn't go through the job queue...hmm [09:05:58] of course, not something to do in 1 week [09:06:12] but maybe a ticket to discuss,etc. [09:06:21] maybe there is a reason for it [09:06:52] actually, regarding the jobqueue, that will mostly not respect the read only, right? [09:08:16] should we do anything about dbstore1002's swap ? [09:08:31] elukey: we can restart? maybe tune down some buffers? [09:08:56] pretty sure that we can restart now [09:09:07] I'm not sure how the job runner will behave..these are all great questions for Aaron :) [09:09:08] ok, let me check the config [09:09:15] legoktm: he he [09:09:47] all right, if you are ok I can downtime + stop el_sync [09:10:10] (and the mysql replicas as well ) [09:10:59] let's do that, but let me see if I have to merge some puppet code first [09:11:10] also we can use it to move the socket location [09:11:59] sure [09:16:29] marostegui: there is no longer dbstore2 hosts, right? [09:18:35] what do you mean? [09:18:47] the puppet role? [09:18:51] I am preparing a patch your way [09:18:56] for you to review [09:19:02] ok :) [09:36:16] 10DBA, 10Gerrit, 10Operations, 10Patch-For-Review, 10Release-Engineering-Team (Backlog): Gerrit shows HTTP 500 error when pasting extended unicode characters - https://phabricator.wikimedia.org/T145885#3663819 (10Paladox) >>! In T145885#3663481, @Marostegui wrote: >>>! In T145885#3555784, @Paladox wrote:... [09:40:55] elukey: the patches needs also your ok so we can restart [09:45:31] 10DBA, 10Gerrit, 10Operations, 10Patch-For-Review, 10Release-Engineering-Team (Backlog): Gerrit shows HTTP 500 error when pasting extended unicode characters - https://phabricator.wikimedia.org/T145885#3663883 (10Paladox) I guess we can remove DBA now as there is nothing here now for the DBA to do? [09:46:27] the only one that I don't get (because of admitted ignorance) is https://gerrit.wikimedia.org/r/#/c/382673/1/modules/role/templates/mariadb/mysqld_config/dbstore.my.cnf.erb [09:46:33] what is the reasoning behind those numbers? [09:46:46] they are lower than the previous ones [09:47:31] I assume someone stopped replication on dbstore1002? [09:47:42] not me [09:47:45] I did [09:47:50] ok [09:48:15] log when those happen (maybe I didn't see it) [09:48:30] so we do not try to stop it twice [09:48:43] we need to merge and run puppet when it goes down [09:50:48] sorry I've only written those info in here, should have logged [09:52:11] so if you lead the maintenance let's do it [09:53:22] all right, if you want to merge your changes I'll run puppet and restart mysql [09:55:59] no [09:56:20] mysql has to be down first to change the socket [09:58:44] sure, didn't get it at first, I can stop mysql now then [09:58:51] thanks [09:59:26] doing it now, will tell you when it will be done [09:59:41] we have a non-written agreement between DBAs that only one person touches a server under maintenance, unless it is asked to do so [09:59:48] as it can lead to problems [10:00:20] too many cooks [10:01:29] I can handover the host to you when mysql it is down, so you'll be able to double check that your changes are ok [10:01:45] and then restart it [10:02:01] whatever works [10:02:07] it is just a puppet merge, nothing else [10:02:16] that may start eventlogging? [10:02:43] good point, it will (now puppet is disabled) [10:03:49] I am going to merge [10:03:56] +1 [10:04:11] tell me how you want to do dbstore1002 [10:04:26] run puppet, kill eventlogging, disable puppet? [10:04:42] fine for me! [10:05:00] you do or I do? [10:05:06] if it is only a matter of running puppet I can do it [10:05:10] sure [10:05:32] just 1 run as soon as mysql is confirmed down [10:05:54] mysql is down now, clean shutdown + I don't see any process running [10:06:06] cool, go on, I merged [10:06:12] enventlogging should fail [10:06:25] so just need to stop it [10:06:59] maybe we can add a watchdog that if it cannot connect, just kill itself [10:07:04] at another time :-) [10:08:00] elukey: we are going to make a great DBA out of you :-D [10:08:11] ahahhaha [10:08:22] looking forward for it :P [10:08:28] all right puppet has run, all good [10:08:44] it showed some diff on my.cnf, right? [10:09:13] yep [10:09:23] wait [10:09:30] I may have done it wrong [10:09:36] -port = 3306 [10:09:36] -socket = /tmp/mysql.sock [10:09:37] +socket = /run/mysqld.sock [10:09:48] /run is the right place [10:09:52] but under a subdir [10:09:55] let me fix this [10:09:59] before we restart mysql [10:10:07] sure [10:10:13] sorry about that [10:10:48] 3 people reviewed this and they gave me too much confidence :-) [10:12:51] https://gerrit.wikimedia.org/r/#/c/382678/ before I do it wrongly again [10:13:43] ahhh [10:14:09] not a big deal [10:14:12] but all others are there [10:14:20] so it would have been strange [10:14:57] This is part of https://phabricator.wikimedia.org/T148507 [10:15:22] sorry, elukey we need to run puppet again, then we can finally start mysql [10:16:07] -socket = /run/mysqld.sock [10:16:08] +socket = /run/mysqld/mysqld.sock [10:16:11] there you go [10:16:12] yay [10:16:46] /run/mysqld/ was created automatically, so we are good to start mysql [10:16:56] all right, starting mysql [10:17:25] we should upgrade to stretch at some point [10:17:31] obviusly not today [10:17:49] yep agreed [10:17:53] mysql is up [10:18:06] not yet [10:18:13] it needs to open its socket [10:18:17] can take a while [10:18:54] sorry I meant that service mysql start worked :) [10:19:48] uhg, this is an issue, it may take a while to start [10:21:26] the server is cool, but we saw this in other cases, for some reason, some hosts (probably those with lots of tables) take way too much to start [10:21:50] but it is ok, it will eventually fo up [10:21:52] now [10:22:05] opt/wmf-mariadb10/bin/mysqld: ready for connections [10:23:03] we can start replication, unles you want to do something first [10:23:10] e.g. with eventlogging [10:24:12] +1 [10:24:26] all right starting with the mysql slaves [10:24:32] ok [10:24:46] we are a bit over an hour behind [10:25:19] I was surprised about the stop, because we normally try to minimize the amount of time down [10:26:34] makes sense [10:26:42] so it caught my by surprise [10:26:54] we hadn't fully prepared the patches, etc [10:27:20] with replication, masters will write stuff, we are prepared or not :-) [10:28:47] looking good: https://grafana.wikimedia.org/dashboard/db/server-board?refresh=1m&orgId=1&var-server=dbstore1002&var-network=eth0&from=now-3h&to=now [10:30:40] the query patterns are interesting: https://grafana.wikimedia.org/dashboard/db/mysql?orgId=1&var-dc=eqiad%20prometheus%2Fops&var-server=dbstore1002&var-port=9104&from=now-7d&to=now [10:30:54] check the purge lag and the change buffer [10:31:16] elukey: did you start deleting/deleting faster since 10/2 ? [10:32:00] otherwise we could be having an issue [10:33:21] The only thing that I did recently, but I don't remember exactly when, is stopping the eventlogging_cleaner because we had to delete some tables [10:33:24] it was this week [10:33:50] mmm [10:33:58] we should be keeping an eye on that [10:34:30] other than that, let's wait a bit, check that replicas catch up normally, evenlogging is good and close the ticket [10:35:17] we need to restart puppet, too [10:36:31] I haven't restarted eventlogging_sync, will do it in a bit [11:30:27] going to lunch + a little errand, but dbstore1002 should be ok now (el sync started, puppet enabled) [13:06:42] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, 10Wikidata-Sprint: Deploy dropping wb_entity_per_page table - https://phabricator.wikimedia.org/T177601#3664286 (10Ladsgroup) [13:14:06] jynus: hey, regarding https://phabricator.wikimedia.org/T177601 is there anything else needed? Do you need a .sql file too? [13:15:11] Amir1: Convention is to usually include the bit of SQL we run in the auto updater [13:15:40] probably mostly to make it clear what exactly is supposed to happen [13:15:49] hoo: it's there https://gerrit.wikimedia.org/r/382693 already, but it's so tiny that happens in php [13:16:10] so the auto updater can take care of it already [13:17:42] sorry, I do not understand that [13:18:22] wb_entity_per_page is a wikibase created table? [13:18:52] is that ready? [13:20:21] I do not think sql is need, it is prefered even if it is trivial [13:20:45] but for WMF we do not ever run it, so not something I need [13:21:14] the thing is- if it has the blocked-on-schema-change, it means I can drop it right now [13:21:30] but I see the patch is not merged, that is a bit contradictory? [13:22:20] can you clarify that and either remove the blocked on schema change or confirm it is good to go? [14:12:21] 10DBA, 10Data-Services, 10XTools, 10cloud-services-team: Request to increase active connection quota for user s51187 on analytics.db.svc.eqiad.wmflabs - https://phabricator.wikimedia.org/T177570#3664436 (10chasemp) p:05Triage>03Normal We did profile things and put in per user limits on the new setup fo... [14:24:27] 10DBA, 10Data-Services, 10XTools, 10cloud-services-team: Request to increase active connection quota for user s51187 on analytics.db.svc.eqiad.wmflabs - https://phabricator.wikimedia.org/T177570#3663283 (10jcrespo) Note *.analytics.db.svc.eqiad.wmflabs hosts are supposed to be for very long-running connect... [14:40:45] 10DBA, 10Wikidata: Migrate wb_terms to using prefixed entity IDs instead of numeric IDs - https://phabricator.wikimedia.org/T114903#3664609 (10WMDE-leszek) [16:26:08] 10DBA, 10Data-Services, 10XTools, 10cloud-services-team: Request to increase active connection quota for user s51187 on analytics.db.svc.eqiad.wmflabs - https://phabricator.wikimedia.org/T177570#3664959 (10MusikAnimal) >>! In T177570#3664528, @jcrespo wrote: > Note *.analytics.db.svc.eqiad.wmflabs hosts ar... [16:59:17] 10DBA, 10Operations, 10ops-eqiad: check db1052 power supply redundancy - https://phabricator.wikimedia.org/T177627#3665092 (10herron) [17:01:49] 10DBA, 10Operations, 10ops-eqiad: check db1054 power supply redundancy - https://phabricator.wikimedia.org/T177628#3665107 (10herron) [17:04:03] 10DBA, 10Operations, 10ops-eqiad: check db1080 power supply redundancy - https://phabricator.wikimedia.org/T177630#3665141 (10herron) [17:45:56] 10DBA, 10Operations, 10ops-eqiad: check db1054 power supply redundancy - https://phabricator.wikimedia.org/T177628#3665356 (10Marostegui) p:05Triage>03High This is s2 master so raising it to high. @Cmjohnson does this host have a led indicator on the PSU so we can see whether this is true or a false posi... [17:46:14] 10DBA, 10Operations, 10ops-eqiad: check db1052 power supply redundancy - https://phabricator.wikimedia.org/T177627#3665359 (10Marostegui) p:05Triage>03High This is s1 master so raising it to high. @Cmjohnson does this host have a led indicator on the PSU so we can see whether this is true or a false posi... [17:47:28] 10DBA, 10Operations, 10ops-eqiad: check db1080 power supply redundancy - https://phabricator.wikimedia.org/T177630#3665364 (10Marostegui) p:05Triage>03Normal @Cmjohnson this host is easy to depool so we can test if this is true or a false positive. We can reboot it, change its PDU socket, cable whatever... [17:52:32] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, and 3 others: Deploy dropping wb_entity_per_page table - https://phabricator.wikimedia.org/T177601#3664286 (10Marostegui) Hi @Ladsgroup - I am a bit confused with this ticket. Is this something you will do? Or somethi... [17:59:08] 10DBA, 10Operations, 10ops-eqiad: check db1052 power supply redundancy - https://phabricator.wikimedia.org/T177627#3665403 (10Marostegui) According to the logs it is the PSU number 2. ``` /admin1/system1/logs1/log1-> show record5 properties CreationTimestamp = 20170124114652.000000-360 ElementName = Sy... [18:01:29] 10DBA, 10Operations, 10ops-eqiad: check db1054 power supply redundancy - https://phabricator.wikimedia.org/T177628#3665413 (10Marostegui) Looks like PSU #2 as per the logs: ``` /admin1/system1/logs1/log1-> show record1 properties CreationTimestamp = 20170610221238.000000-300 ElementName = System Event... [18:03:28] 10DBA, 10Operations, 10ops-eqiad: check db1080 power supply redundancy - https://phabricator.wikimedia.org/T177630#3665424 (10Marostegui) Looks like PSU #2 as per the logs: ``` status=0 status_tag=COMMAND COMPLETED Fri Oct 6 17:45:40 2017 /system1/log1/record8 Targets Properties number=8 sev... [18:14:09] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, and 3 others: Deploy dropping wb_entity_per_page table - https://phabricator.wikimedia.org/T177601#3665456 (10Ladsgroup) This is something for the DBAs, this patch only does it for the beta cluster, I was thinking do... [18:15:36] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, and 3 others: Deploy dropping wb_entity_per_page table - https://phabricator.wikimedia.org/T177601#3665465 (10Marostegui) >>! In T177601#3665456, @Ladsgroup wrote: > This is something for the DBAs, this patch only doe... [18:25:41] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, and 3 others: Deploy dropping wb_entity_per_page table - https://phabricator.wikimedia.org/T177601#3665510 (10Ladsgroup) It's merged but not deployed, we should wait for a week if train doesn't get stuck (more if it d... [18:33:49] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, and 3 others: Deploy dropping wb_entity_per_page table - https://phabricator.wikimedia.org/T177601#3665591 (10Marostegui) >>! In T177601#3665510, @Ladsgroup wrote: > It's merged but not deployed, we should wait for a... [19:04:23] 10Blocked-on-schema-change, 10DBA, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, and 3 others: Deploy dropping wb_entity_per_page table - https://phabricator.wikimedia.org/T177601#3665725 (10Ladsgroup) Sure. Thanks! [20:28:31] 10DBA, 10MediaWiki-extensions-WikibaseRepository, 10Wikidata, 10MW-1.31-release-notes (WMF-deploy-2017-10-10 (1.31.0-wmf.3)), and 2 others: Deploy dropping wb_entity_per_page table - https://phabricator.wikimedia.org/T177601#3666079 (10jcrespo) Add #blocked-on-schema-change, that way we will know it is rea... [22:38:40] 10DBA, 10Operations, 10Patch-For-Review, 10Wiki-Setup (Create): Create elections committee private wiki - https://phabricator.wikimedia.org/T174370#3666404 (10jrbs) Hey - just wondering what the status is here. Is there anything we can do to push this along? [23:42:38] 10DBA, 10Operations, 10Patch-For-Review, 10Wiki-Setup (Create): Create elections committee private wiki - https://phabricator.wikimedia.org/T174370#3666568 (10Reedy) Just needs scheduling, patches rebasing and someone to deploy it