[03:51:09] <wikibugs_>	 10DBA, 10Edit-Review-Improvements-RC-Page, 06Collaboration-Team-Triage (Collab-Team-Q2-Oct-Dec-2016), 05MW-1.29-release-notes, 13Patch-For-Review: Implement functionality for RC page 'Experience level' filters - https://phabricator.wikimedia.org/T149637#3073640 (10Mattflaschen-WMF) > If NEITHER A, B nor...
[03:51:39] <wikibugs_>	 10DBA, 10Edit-Review-Improvements-RC-Page, 06Collaboration-Team-Triage (Collab-Team-Q2-Oct-Dec-2016), 05MW-1.29-release-notes, 13Patch-For-Review: Implement functionality for RC page 'Experience level' filters - https://phabricator.wikimedia.org/T149637#3073642 (10Mattflaschen-WMF)
[05:04:08] <wikibugs_>	 10DBA, 10Edit-Review-Improvements-RC-Page, 06Collaboration-Team-Triage (Collab-Team-Q2-Oct-Dec-2016), 05MW-1.29-release-notes, 13Patch-For-Review: Implement functionality for RC page 'Experience level' filters - https://phabricator.wikimedia.org/T149637#3073671 (10Mattflaschen-WMF)
[06:28:55] <wikibugs_>	 10DBA, 10Edit-Review-Improvements-RC-Page, 06Collaboration-Team-Triage (Collab-Team-Q2-Oct-Dec-2016), 05MW-1.29-release-notes, 13Patch-For-Review: Implement functionality for RC page 'Experience level' filters - https://phabricator.wikimedia.org/T149637#3073688 (10Mattflaschen-WMF) Also, "all" (or listin...
[12:19:17] <wikibugs_>	 10DBA, 06Labs, 10Labs-Infrastructure: Data integrity issue with enwiki_p user_groups on Wikimedia Tool Labs (missing rows) - https://phabricator.wikimedia.org/T159493#3073791 (10jcrespo) >>! In T159493#3071970, @MZMcBride wrote: > Related question: in scripts, I connect to `enwiki_p` for the database name an...
[12:26:15] <wikibugs_>	 10DBA, 13Patch-For-Review: Install and reimage dbstore1001 as jessie - https://phabricator.wikimedia.org/T153768#3073795 (10jcrespo) Yes, but let's try forcing/managing bacula first, local backups are not that useful.
[12:26:33] <wikibugs_>	 10DBA, 13Patch-For-Review: Install and reimage dbstore1001 as jessie - https://phabricator.wikimedia.org/T153768#3073796 (10jcrespo) p:05Normal>03High
[12:31:14] <wikibugs_>	 10DBA, 06Labs: labsdb1004 MySQL crash - https://phabricator.wikimedia.org/T159572#3073825 (10jcrespo) @Marostegui That log by itself should not make the server crash. s51412__data having wrong permissions is something we did on purpose during the failover, so I would expect some of those happening in the past.
[12:38:44] <wikibugs_>	 10DBA, 06Labs: labsdb1004 MySQL crash - https://phabricator.wikimedia.org/T159572#3073826 (10jcrespo) I have restarted the slave ignoring a couple of extra heavy hitters, that should avoid crashes for now. I have also enabled GTID to mitigate problems on crash.
[12:59:12] <wikibugs_>	 10DBA, 06Labs: labsdb1004 MySQL crash - https://phabricator.wikimedia.org/T159572#3073827 (10jcrespo) BTW, labsdb1005 run out of /tmp space during the weekend- we need to move it to /srv, somewhere. Both cases are probably the same origin, too much load from 1 or several users.
[14:53:19] <wikibugs_>	 10DBA, 06Labs: labsdb1004 MySQL crash - https://phabricator.wikimedia.org/T159572#3073901 (10Marostegui) >>! In T159572#3073825, @jcrespo wrote: > @Marostegui That log by itself should not make the server crash. s51412__data having wrong permissions is something we did on purpose during the failover, so I woul...