[00:04:49] <bobwest>	 that someone is me... ;/
[00:05:04] <bobwest>	 it was in an attempt to solve the issue with sqoop
[00:05:17] <bobwest>	 but the error also happens when those jopbs arent running
[00:05:20] <bobwest>	 (i killed them, btw)
[00:08:32] <springle>	 yeah, it's all making the box replication lag, and so the slow queries are killed off
[00:09:21] <springle>	 let's leave my local dump running, and allow replication to catch up (several hours behind by now)
[00:09:48] <springle>	 bobwest: if the dump is successful, i'll put it on stat1002 to unblock you
[00:10:46] <springle>	 as for analytics-store, this may simply be another indication we need to get some of analytics traffic into CODFW
[00:11:47] <bobwest>	 ok, thanks a lot, @springle, i appreciate it!
[00:46:49] <springle>	 bobwest: where on stat1002 should the dump go?
[00:47:21] <springle>	 er... that is, i assumed stat1002 for some reason. don't recall why :)
[00:53:47] <bobwest>	 could you put it into stat1002:/home/west1/, please?
[00:53:59] <bobwest>	 thanks so much for your help!!
[01:20:28] <springle>	 bobwest: /home/west1/enwiki_revision.sql.gz (your sed filter not yet run, just the plain dump)
[01:28:30] <bobwest>	 wohoo -- thanks!
[03:20:34] <springle>	 db1045 has an enormous old ibdata1. time to shrink it i guess
[06:03:26] <jynus>	 answering from top to bottom: db1018, not db1008, the only production machine with P_S activated
[06:07:43] <jynus>	 I havn't checked, but db1047 will probably need the same treatment as db1045 due to the long running transaction
[06:12:49] <jynus>	 I have stopped the profiling on db1018
[08:32:28] <jynus>	 pt-osc failed on db1016, will do ALTER TABLE + master-master failover
[11:21:43] <jynus>	 etherpad and db1001/db1016 bath to its original state (after the successful alter table)
[11:21:57] <jynus>	 repooling es1002, depooling es1003
[11:22:52] <jynus>	 did you run any check on es1002 during the nigh?, there is activity not from me there (like a pt-checksum or sth.)
[11:30:16] <jynus>	 db1049 is generating more errors than the average, probably related to the pool_size =8 //TODO
[15:35:37] <jynus>	 sorry, I downtime'd the wrong host
[15:35:57] <jynus>	 1003 instead of 1004, soory for the alert
[16:25:08] <jynus>	 winding up for today
[16:26:19] <jynus>	 I think I will let es1004 up and ready to be repooled (except for the bp warning up)
[16:26:37] <jynus>	 s/let/leave
[16:54:44] <leila>	 hi springle. thank you so much for helping Bob yesterday. we're unblocked. :-)
[17:09:27] <jynus>	 1047 is probably not useful anymore (disabled lag alert), but let's keep it on while can use it