[00:00:37] <milimetric>	 madhuvishy: I worked on the other one I am doing with Marcel
[00:00:58] <milimetric>	 I don't know how to test on beta cluster though, because it doesn't seem to have zookeeper
[00:01:07] <madhuvishy>	 milimetric: alright. let's catch up tomorrow and pair on our thing.
[00:01:08] <madhuvishy>	 aah
[00:01:19] <madhuvishy>	 milimetric: I'm trying to test the oozie thing i wrote
[00:01:23] <milimetric>	 sure, let's do the config one tomorrow
[00:01:25] <milimetric>	 feel better
[00:01:32] <madhuvishy>	 milimetric: its blah
[00:02:16] <madhuvishy>	 milimetric: oh also, is there a way to get staging wikimetrics database on local?
[00:03:41] <milimetric>	 madhuvishy: what do you mean?
[00:03:47] <milimetric>	 download all the data?
[00:03:56] <madhuvishy>	 milimetric: yeah
[00:04:08] <milimetric>	 why do you need that?
[00:04:35] <milimetric>	 sorry, yes, there's a way, but I'm just making sure there's not an easier way to do what you want
[00:04:55] <madhuvishy>	 milimetric: i thought it might be nice to test the validate again change on that
[00:04:59] <milimetric>	 to dump a mysql db, you can just do mysqldump [options] -Dwikimetrics > wikimetrics.backup.sql
[00:05:13] <milimetric>	 oh, madhuvishy, yeah, then just deploy it to staging
[00:05:20] <milimetric>	 no need to copy the db locally
[00:05:31] <milimetric>	 it wouldn't help you anyway, because you don't have access to labsdb
[00:05:35] <madhuvishy>	 milimetric: alright.
[00:05:48] <madhuvishy>	 how do i deploy to staging - is there a wiki on that
[00:06:07] <milimetric>	 there... might be?  But it's easy
[00:06:15] <milimetric>	 wikimetrics is deployed to /srv/wikimetrics
[00:06:25] <milimetric>	 so just go there, sudo git checkout your gerrit change
[00:06:39] <madhuvishy>	 milimetric: okay
[00:06:43] <madhuvishy>	 will do
[00:06:47] <milimetric>	 run whatever you need to run (alembic upgrade, install script, etc.)
[00:06:47] <madhuvishy>	 thanks :)
[00:06:51] <milimetric>	 hang on, not done :)
[00:06:55] <madhuvishy>	 milimetric: aaah
[00:06:56] <madhuvishy>	 :D
[00:06:58] <madhuvishy>	  okay
[00:07:23] <milimetric>	 then you restart the services like wikimetrics-queue, wikimetrics-scheduler, and apache.
[00:08:04] <milimetric>	 if you need to change the configuration in /etc/wikimetrics then let me know I have to show you how to disable puppet (because it will overwrite the config every few minutes when it's running)
[00:08:49] <milimetric>	 and that should be it, you can tail the logs in /var/log/apache2 and /var/log/upstart/wikimetrics-queue
[00:08:49] <madhuvishy>	 milimetric: no config changes necessary i think - in this case - but useful to know that.
[00:08:59] <madhuvishy>	 milimetric: Awesome.
[00:09:07] <milimetric>	 k, let me know if you run into trouble
[00:09:14] <milimetric>	 oh, madhuvishy
[00:09:21] <milimetric>	 one thing when you rebase
[00:09:23] <milimetric>	 there's a hidden conflict
[00:09:35] <milimetric>	 because Marcel's code that got merged while yours was in review has an alembic migration
[00:09:48] <milimetric>	 so you'll have to update your migration to have his as the "down" revision
[00:09:52] <madhuvishy>	 milimetric: aah
[00:10:19] <milimetric>	 i recommend rebasing before deploying to staging
[00:10:28] <madhuvishy>	 milimetric: yeah i'll do that
[00:10:41] <milimetric>	 but then if something breaks and it ends up being Marcel's code, just yell at me for it 'cause I merged it
[00:11:33] <madhuvishy>	 ha ha okay
[00:14:04] <matt_flaschen>	 milimetric, do you know if External Store is available on stat1003?
[00:14:41] <milimetric>	 matt_flaschen: i don't know what External Store is
[00:14:46] <milimetric>	 oh, where rev text is?
[00:14:50] <halfak>	 milimetric, got attacked by a yuvipanda
[00:14:58] <milimetric>	 omg halfak are you ok?
[00:15:05] <halfak>	 I'll survive. :)
[00:15:09] <milimetric>	 phew
[00:15:18] <matt_flaschen>	 milimetric, yeah.  https://wikitech.wikimedia.org/wiki/External_storage
[00:15:40] <halfak>	 So, we should probably delay a conversation about VE work until tomorrow given that (a) it's late in EDT and (b) I have to prepare to give a presentation tomorrow.
[00:15:49] <milimetric>	 matt_flaschen: that's like a black box to me, I don't know if it's replicated
[00:16:07] <matt_flaschen>	 milimetric, okay, no problem.
[00:16:10] <milimetric>	 matt_flaschen: thanks for pointing me to the article, I never knew what this thing was called
[00:16:30] <milimetric>	 halfak: cool, I've got a little vega presentation I'm trying to help with too.  let's talk tomorrow
[00:16:33] <matt_flaschen>	 I just like to use research databases for doing reads in production, just in case.  I'm assuming these are read-only...
[00:16:39] <halfak>	 milimetric, I found a slot on our calendars.
[00:16:46] <halfak>	 5:30 EDT
[00:16:47] <halfak>	 OK?
[00:16:51] <milimetric>	 halfak: works
[00:16:57] <milimetric>	 matt_flaschen: read only, yes
[00:16:58] <halfak>	 Great.  Thanks
[00:17:02] <matt_flaschen>	 halfak, do you ever do any research with the DB that involves page text?
[00:17:11] <matt_flaschen>	 Because that must involve these tables
[00:18:47] <milimetric>	 matt_flaschen: yes he does, he gets the text from dumps though
[00:18:57] <matt_flaschen>	 Yeah, so not direct DB access.
[00:18:58] <milimetric>	 we were told External Store was a beast to access
[00:19:11] <matt_flaschen>	 It may not be replicated.
[00:19:19] <milimetric>	 except it was more like "That ^&* &#*#O* no way man &**((&"
[00:19:27] <matt_flaschen>	 milimetric, LOL.
[00:19:56] <matt_flaschen>	 Not sure why, other than the cluster name changing every so often, it should still be replicatable.  Though there are some other complexities with decoding it (gzip, etc.).
[00:24:26] <milimetric>	 from what I understand if you want a specific revision and it's far back in the past, it's stored together with other revisions in blobs and there's magic code that knows how to get it out, but I was told that code was pretty hard to port / understand
[00:24:40] <milimetric>	 I doubted this, because I can take some guesses as to who wrote that code and they're all fine engineers
[00:24:51] <milimetric>	 but I never looked at it
[01:11:56] <grrrit-wm>	 (PS1) Madhuvishy: Add generic oozie component for emailing data [analytics/refinery] - https://gerrit.wikimedia.org/r/210632
[01:15:40] <grrrit-wm>	 (PS2) Madhuvishy: Add generic oozie component for emailing data [analytics/refinery] - https://gerrit.wikimedia.org/r/210632
[08:47:59] <wikibugs>	 Analytics, Engineering-Community, ECT-May-2015: Analytics Team Offsite - Before Wikimania - https://phabricator.wikimedia.org/T90602#1281713 (Qgil)
[09:46:30] <grrrit-wm>	 (CR) Mforns: Permit rerun of failed report (1 comment) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/208017 (https://phabricator.wikimedia.org/T88610) (owner: Mforns)
[09:49:46] <grrrit-wm>	 (CR) Mforns: Add error message to failed reports (1 comment) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/203241 (https://phabricator.wikimedia.org/T88610) (owner: Mforns)
[11:13:03] <joal|night>	 milimetric: Hi
[11:13:09] <joal|night>	 Let me know when you're around
[11:13:14] <milimetric>	 hi joal|night
[11:13:24] <joal|night>	 AH, I am still nighty
[11:13:27] <joal|night>	 Let's change that
[11:13:27] <milimetric>	 :)
[11:13:44] * joal feels awake now
[11:13:50] <milimetric>	 heh
[11:13:58] <joal>	 I have a zookeeper address for you :)
[11:14:01] <milimetric>	 yay!
[11:14:10] <joal>	 And a few hints as well :)
[11:14:18] <milimetric>	 yay^2
[11:14:22] <joal>	 huhu
[11:14:48] <joal>	 I wonder, does yay^2 == yayyay ?
[11:14:58] <joal>	 :)
[11:15:18] <joal>	 So, kafka in betalabs
[11:15:28] <joal>	 --> deployment-zookeeper01:2181/kafka/deployment-kafka
[11:15:43] <joal>	 sorry, zookerper in betalabd, for jafja
[11:16:17] <milimetric>	 I'm assuming jafja is kafka
[11:16:36] <joal>	 Yes, you are right :)
[11:16:44] <joal>	 I like jafja as well :)
[11:16:45] <milimetric>	 what's the /kafka/deployment-kafka thing?
[11:16:54] <milimetric>	 jafja should totally be a thing, we should build it
[11:17:11] <joal>	 It's the zookeeper inner path for kafka values
[11:17:40] <milimetric>	 ah, I see, so I pass this to --zookeeper deployment-zookeeper01:2181/kafka/deployment-kafka
[11:17:42] <milimetric>	 like that right?
[11:17:48] <joal>	 correct
[11:17:50] <milimetric>	 k, adding to wikitech
[11:18:09] <joal>	 hmmm ! didin't know there was a wikitech page on this !
[11:18:11] <milimetric>	 https://wikitech.wikimedia.org/wiki/EventLogging/Testing/BetaLabs#Kafka
[11:18:17] <joal>	 Nice !
[11:18:32] <milimetric>	 well, I didn't know the root password for the beta cluster db was there
[11:18:38] <joal>	 Another way to have the same effect : export ZOOKEEPER_URL=deployment-zookeeper01:2181/kafka/deployment-kafka
[11:18:48] <milimetric>	 btw, I was told it's "Beta Cluster" not "Beta Labs" because it's managed by different people that aren't labs ops
[11:18:54] <milimetric>	 (release eng)
[11:19:49] <joal>	 I think we should rephrase the kafka stuff here
[11:20:20] <joal>	 When using kafka on betalabs: deployment-kafka02.eqiad.wmflabs
[11:20:30] <milimetric>	 beta cluster :)
[11:20:39] <joal>	 Oops, sorry
[11:20:43] <joal>	 buffer not updated yet
[11:20:46] <joal>	 :)
[11:20:47] <milimetric>	 heh
[11:21:07] <joal>	 When using kafka on betalabs: deployment-kafka02.eqiad.wmflabs, using zookeeper deployment-zookeeper01:2181/kafka/deployment-kafka
[11:21:28] <milimetric>	 I'm gonna take out the trash, feed the animals, and do other morning things.  I'll be back to try out my new zookeeper, thanks for that
[11:21:38] <joal>	 no prob
[11:22:11] <joal>	 Let me know when you come back, I will show you where we are with Marcel (if you wish
[14:06:26] <milimetric>	 joal, I'm a little confused about the data flow on beta
[14:06:37] <joal>	 hm
[14:06:41] <milimetric>	 i see a trickle of server side events
[14:06:42] <joal>	 batcave ?
[14:06:45] <milimetric>	 sure
[15:01:42] <halfak>	 o/ joal
[15:01:52] <joal>	 Hi halfak
[15:02:08] <halfak>	 Now a good time to chat?
[15:02:13] <joal>	 I am in a talk with Dan :)
[15:02:26] <halfak>	 OK.  I have the next hour blocked off to sync up.
[15:02:34] <halfak>	 Could you ping me when you are ready?
[15:02:57] <joal>	 Ready :)
[15:03:07] <joal>	 halfak: --^
[15:03:19] <joal>	 Where do we meet ?
[15:03:55] <halfak>	 batcave?
[15:04:29] <joal>	 Sure
[15:04:31] <joal>	 OMW
[15:19:58] <joal>	 milimetric: fighting git
[15:24:12] <milimetric>	 joal: I can help with that
[15:24:14] <milimetric>	 to the batcave!
[15:24:19] <joal>	 OMWn
[15:43:48] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban: ContentTranslationError event logging table is not receiving new events - https://phabricator.wikimedia.org/T98842#1282359 (ggellerman) a:Milimetric
[15:47:37] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban, Patch-For-Review: Troubleshoot EL performance problems on 2015-05-06 - https://phabricator.wikimedia.org/T98588#1282384 (mforns)
[15:47:38] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban: ContentTranslationError event logging table is not receiving new events - https://phabricator.wikimedia.org/T98842#1282383 (mforns)
[16:50:43] <wikibugs>	 Analytics-Kanban: Deal with non-timeboxed queries recomputing too much data - https://phabricator.wikimedia.org/T98979#1282555 (Milimetric) NEW a:Milimetric
[16:51:11] <grrrit-wm>	 (PS2) Milimetric: Disable 19 queries from the scheduler [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/210364 (https://phabricator.wikimedia.org/T98979)
[16:59:03] <lzia>	 hello milimetric. do you know if we will be having the Analytics checkpoint tomorrow? :-)
[16:59:29] <lzia>	 I'd like to attend it, but if it's not happening, I need to schedule another meeting at that time.
[16:59:59] <milimetric>	 lzia: joal says he can be there and hang out with you guys, but maybe it's up to you
[17:00:01] <milimetric>	 Andrew won't be around
[17:01:44] <lzia>	 it's good to hangout with joal. okay, I'll plan to attend the first half hour. thanks milimetric.
[17:12:41] <yurik>	 milimetric, hi, did you talk to veag?
[17:12:42] <yurik>	 vega
[17:12:58] <yurik>	 milimetric, want to present something yourself tomorrow?
[17:13:06] <yurik>	 hehe, two q at once
[17:13:42] <milimetric>	 yes, and maybe?  I've been looking at lyra
[17:14:23] <milimetric>	 yurik: http://idl.cs.washington.edu/projects/lyra/
[17:14:37] <milimetric>	 I wanted to basically walk people through looking at the same data in a few different ways
[17:14:51] <milimetric>	 and try to make it as smooth as possible.  And as usual, I got stuck on some Vega syntax
[17:15:11] <milimetric>	 so I was thinking this tool might be useful as it generates vega definitions
[17:15:35] <milimetric>	 it's pretty well thought out, but still confusing :)
[17:15:46] <yurik>	 WOWOOWOWOW
[17:15:52] <milimetric>	 so basically, right now, I'd have nothing great to show but I can show this
[17:16:04] <yurik>	 visual :)
[17:16:09] <milimetric>	 it's nice, yea
[17:16:17] <yurik>	 nah, don't, because its not generating vegas yet, right?
[17:16:23] <milimetric>	 it is generating vega
[17:16:26] <milimetric>	 you just go "export"
[17:16:40] <milimetric>	 you can also import, of course
[17:16:47] <yurik>	 it does?
[17:16:49] <yurik>	 holy crap
[17:16:51] <milimetric>	 yes
[17:16:52] <milimetric>	 :)
[17:16:58] <milimetric>	 don't get excited
[17:17:02] <yurik>	 where do you find this stuff
[17:17:02] <milimetric>	 it's pretty confusing
[17:17:06] <milimetric>	 less confusing than vega
[17:17:11] <milimetric>	 oh, Jane told me about it in passing
[17:17:20] <milimetric>	 we were talking about better ways to debug vega
[17:18:31] <yurik>	 so wait, why can't we use this?
[17:20:35] <yurik>	 so its not ther competitor - its the same university
[17:21:48] <yurik>	 and how can i import
[17:27:22] <yurik>	 milimetric, do you know how to import there?
[17:31:05] <milimetric>	 yurik: in scrum of scrums and other meetings, I'll look in a bit
[17:31:25] <mutante>	 kafka.server.BrokerTopicMetrics.AllTopicsMessagesInPerSec.FifteenMinuteRate CRITICAL: 7.06787035605e-16
[17:32:16] <milimetric>	 yurik: weird! they don't have import
[17:32:30] <yurik>	 milimetric, exactly - they say they do in the readme/notes
[17:32:37] <milimetric>	 mutante: thanks, where is that?  (sorry our ops guy is not around)
[17:35:50] <mutante>	 milimetric: it's on analytics1021  did you meant that? or where i saw the message? that was on icinga web ui
[17:36:12] <milimetric>	 thanks mutante, that's what I meant
[17:36:16] <milimetric>	 hm.... pooop...
[17:36:33] <milimetric>	 jgage: can you help?
[17:37:10] <mutante>	 milimetric: yw. i just had no idea what it means in practice
[17:37:37] <milimetric>	 mutante: sounds like we're getting less data than we expect maybe
[17:38:49] <mutante>	 ok, *nod*
[17:40:12] <madhuvishy>	 mforns: I kee p getting this - ProgrammingError: (_mysql_exceptions.ProgrammingError) (1146, "Table 'wikimetrics_testing.task_error' doesn't exist") [SQL: u'DELETE FROM task_error'] when running tests. Do you have any insight into this error?
[17:40:28] <mforns>	 madhuvishy, mmmm
[17:40:44] <mforns>	 madhuvishy, how are your migrations behaving?
[17:40:59] <mforns>	 madhuvishy, what happens if you execute alembic upgrade head?
[17:41:08] <joal>	 milimetric: EL on beta cluster seems to work now (valid and invalid in kafka)
[17:41:17] <madhuvishy>	 mforns: It complained that there are two heads
[17:41:22] <jgage>	 hi
[17:41:28] <joal>	 hi jgage
[17:41:32] <madhuvishy>	 mforns: so i ran them one by one
[17:41:36] <mforns>	 madhuvishy, that is a good candidate
[17:41:42] <mforns>	 madhuvishy, aha
[17:41:59] <madhuvishy>	 mforns: do I merge both migrations?
[17:42:06] <mforns>	 madhuvishy, mmm
[17:42:18] <mforns>	 madhuvishy,  what patch are you trying?
[17:42:22] <mforns>	 what task?
[17:43:33] <madhuvishy>	 mforns: I am doing my validate cohort one. I rebased my changes against master - which now has your recent patches
[17:43:47] <mforns>	 madhuvishy, I see
[17:44:09] <mforns>	 so before rebasing, you had your validate again migration on top of previous master
[17:44:10] <madhuvishy>	 mforns: alembic branches now gives me this -
[17:44:16] <madhuvishy>	 https://www.irccloud.com/pastebin/Y9CQfE3C
[17:44:43] <milimetric>	 madhuvishy: yep, so you have to set 4a2162ae3e84 as the down revision for 35adbe20f3d
[17:44:57] <madhuvishy>	 mforns: aah
[17:45:55] <mforns>	 madhuvishy, the normal workflow should be alembic downgrade before rebase, rebase, and then alembic upgrade
[17:46:51] <mforns>	 madhuvishy, it's confusing.. I've messed up alembic lots of times xD
[17:47:15] <madhuvishy>	 mforns: ha ha, i foresee messing it up a lot of times
[17:48:04] <mforns>	 madhuvishy, so now I don't know in which state are your migrations
[17:48:16] <jgage>	 mutante & milimetrric: ok, analytics1021 problem is fixed. it's a longstanding and mysterious problem we have with this host where its connection between kafka and zookeeper times out and so it stops being a leader for kafka partitions.
[17:48:20] <jgage>	 gage@analytics1021:~$ kafka topic --describe | grep "Leader: 21" | wc -l
[17:48:23] <jgage>	 0
[17:48:25] <jgage>	 gage@analytics1021:~$ kafka preferred-replica-election
[17:48:28] <jgage>	 gage@analytics1021:~$ kafka topic --describe | grep "Leader: 21" | wc -l
[17:48:31] <jgage>	 24
[17:48:32] <madhuvishy>	 mforns: hmmm let me try setting yours as the downgrade
[17:48:39] <mforns>	 madhuvishy, and as you rebased, I don't know how you can revert the changes
[17:49:03] <mforns>	 madhuvishy, what happens if you try alembic downgrade base?
[17:49:03] <joal>	 jgage: Weirdddd :)
[17:49:05] <jgage>	 (no data was lost, we just had a period of 3 active kafka brokers instead of 4)
[17:49:17] <mutante>	 jgage: thanks!!
[17:49:21] <jgage>	 :D
[17:50:08] <jgage>	 the phab task about it: https://phabricator.wikimedia.org/T71667
[17:50:33] <jgage>	 https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Kafka/Administration#Replica_Elections
[17:50:36] <madhuvishy>	 mforns: hmm it seems to be fine - but i think i understand - your migration for the task error table dint show up
[17:50:52] <jgage>	 i don't mind fixing this when it happens, just sharing the knowledge..
[17:51:00] <mforns>	 madhuvishy, aha
[17:51:16] <madhuvishy>	 mforns: so i have to merge first and then upgrade
[17:51:49] <mforns>	 madhuvishy, do you need to merge?
[17:51:58] <mforns>	 madhuvishy, you mean rebase?
[17:52:18] <madhuvishy>	 mforns: uhhh i dont know. reading alembic docs
[17:52:53] <mforns>	 madhuvishy, ok
[17:53:23] <madhuvishy>	 git log
[17:53:44] <madhuvishy>	 gah i'm typing in the wrong place
[17:53:53] <mforns>	 madhuvishy, do you want to try solving it in the batcave?
[17:53:54] <milimetric>	 madhuvishy: all i meant is literally go into your revision and type in marcel's revision as your "down"
[17:54:09] <milimetric>	 that's much simpler than doing it "the right way"
[17:54:10] <milimetric>	 :)
[17:54:46] <yurik>	 milimetric, i will put that ide on wmflabs
[17:54:55] <milimetric>	 jgage: awesome, thanks much
[17:54:57] <yurik>	 this way ppl can import data
[17:55:00] <yurik>	 from wiki
[17:55:15] <milimetric>	 yurik: cool, send me the link, I'd like to demo it from there then
[17:55:24] <milimetric>	 I got one night to get good at it :)
[17:55:42] <yurik>	 milimetric,  lets hope we find a way to import :)
[17:56:50] <madhuvishy>	 milimetric: mforns thanks - think its fixed now
[17:57:14] <mforns>	 madhuvishy, cool :]
[17:58:26] <milimetric>	 mforns: ok, so the processor seems fine now, and we should pair up on the config
[17:58:37] <milimetric>	 I have a meeting in 30 minutes
[17:58:46] <mforns>	 milimetric, aha
[17:58:50] <milimetric>	 and I want to eat some lunch, when are you done for the day?
[17:59:19] <mforns>	 milimetric, today at 22h -> 16h Phili
[18:00:06] <milimetric>	 ok, then I'll ping you as soon as I'm done with my next meeting, which is 15:00 here and 21:00 there
[18:00:37] <joal>	 milimetric, mforns : I'll push a review for puppet config tomorrow
[18:00:43] <joal>	 for the forwarder
[18:00:46] <mforns>	 milimetric, ok
[18:00:47] <mforns>	 joal, ok
[18:00:56] <joal>	 Please let me know if you have one already
[18:01:20] <joal>	 Done for today !
[18:01:27] <joal>	 Have a good end of day folks
[18:02:57] <mforns>	 joal|night, good night!
[18:17:33] <wikibugs>	 Analytics-Kanban, VisualEditor, Editing Department 2014/15 Q4 blockers: Schema:Edit seems to incorrectly set users as anonymous {lion} - https://phabricator.wikimedia.org/T92596#1282915 (Jdforrester-WMF) Note: Discussed at the weekly Editing triage on 2015-05-13 but not accepted or rejected whilst we f...
[19:00:46] <milimetric>	 mforns: ping :)
[19:00:49] <milimetric>	 to the batcave!
[19:02:58] <mforns>	 milimetric, going
[19:27:56] <grrrit-wm>	 (PS8) Madhuvishy: WIP: Fix validate again functionality on cohort display page [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/206346 (https://phabricator.wikimedia.org/T78339)
[19:27:58] <grrrit-wm>	 (PS2) Madhuvishy: Fix models to match migrations [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/206343 (owner: Milimetric)
[19:34:01] <madhuvishy>	 mforns/milimetric: what's the staging server called?
[19:34:14] <madhuvishy>	 for wikimetrics
[19:34:23] <milimetric>	 wikimetrics-staging1.eqiad.wmflabs
[19:34:35] <madhuvishy>	 cool thanks
[19:59:30] <milimetric>	 mforns: you'll love what it was doing
[19:59:38] <mforns>	 milimetric, ?
[19:59:42] <milimetric>	 instead of pasting, it was trying to paste 98779 times
[19:59:48] <milimetric>	 because it was interpreting what I pasted as commands
[19:59:49] <mforns>	 milimetric, xDDD
[20:00:26] <milimetric>	 i opened the recovered file and it was full of 98779 copies of the bug number, so funny :)
[20:00:27] <mforns>	 milimetric, again: vi is for crazy people :]
[20:00:34] <mforns>	 xD
[20:04:50] <milimetric>	 :) I rarely screw up anymore, but when I do, I do it big
[20:05:11] <mforns>	 bye everyone! see you tomorrow
[20:05:13] <milimetric>	 it's mostly because of the inconsistencies, the "gui" version and the "cli" version never agree on settings
[20:05:15] <milimetric>	 nite!
[20:05:30] <mforns>	 milimetric, aha
[20:05:54] <mforns>	 g'night
[21:08:16] <yurik>	 milimetric, didn't have enough time to set up the lily. If you can, could you install it somewhere? possibly on a separate instance?
[21:08:32] <yurik>	 i'm off to bed, dead tired, and will need to make a presentation tomorrow morning
[21:09:24] <milimetric>	 I'll give it a shot, yurik, good night
[21:09:48] <yurik>	 milimetric, thx!  make sure you add me as an admin there :)
[21:09:58] <yurik>	 i mean - install it in a group that has both of us as admins
[21:10:13] <madhuvishy>	 milimetric: tested my change on staging - it seems to work fine!
[21:10:45] <madhuvishy>	 milimetric: although my css change is not showing up.
[21:12:04] <milimetric>	 madhuvishy: that's great
[21:12:12] <milimetric>	 CSS might be a caching issue - did you clear?
[21:12:20] <milimetric>	 hm, no, but we have cache busting
[21:12:35] <madhuvishy>	 milimetric: its not even CSS - its html
[21:13:38] <milimetric>	 madhuvishy: i see the validate again button, what html change?
[21:15:01] <madhuvishy>	 milimetric: hmmm i spaced the buttons. i think the code is not pulled right. i dont see it on the staging server code. checking what happened
[21:15:33] <milimetric>	 yeah, if it's html you don't even have to restart anything
[21:15:58] <milimetric>	 how'd you grab the code?  The easiest way is sudo-ing the command that gerrit lets you copy/paste for checkout
[21:16:14] <milimetric>	 and use the anonymous http version
[21:18:20] <madhuvishy>	 milimetric: aah. that's what i was missing. keep getting permission denied
[21:26:58] <madhuvishy>	 milimetric: hmmm i think it's fine now
[21:27:20] <milimetric>	 great work.  I have to look at it in a bit or tomorrow because I have a meeting coming up
[21:27:35] <milimetric>	 but if you want to look at the puppet stuff I did in a half hour or so, I can do that
[21:27:52] <madhuvishy>	 milimetric: yeah no problem
[21:27:57] <madhuvishy>	 sure we can do that
[22:42:18] <grrrit-wm>	 (PS3) Madhuvishy: Add generic oozie component for emailing data [analytics/refinery] - https://gerrit.wikimedia.org/r/210632
[22:46:53] <milimetric>	 madhuvishy: ok, free
[22:48:07] <madhuvishy>	 milimetric: cool. let me just get some water. then batcave?
[22:49:46] <wikibugs>	 Analytics-Tech-community-metrics, ECT-May-2015: Provide list of open Gerrit changesets with most activity which aren't -1/-2'ed - https://phabricator.wikimedia.org/T94036#1283775 (Dicortazar) Hi again,  I already fixed the query to get the results.  This is the top 10 (well, with the last version I've loc...
[22:50:03] <milimetric>	 to the batcave!
[23:19:34] <wikibugs>	 Analytics-Kanban, VisualEditor, Editing Department 2014/15 Q4 blockers: Schema:Edit seems to incorrectly set users as anonymous {lion} - https://phabricator.wikimedia.org/T92596#1283855 (Halfak) Open>Resolved a:Halfak Upon re-review with DAndreescu, it looks like this field is set appropriatel...
[23:42:13] <wikibugs>	 Analytics-Kanban, VisualEditor, Editing Department 2014/15 Q4 blockers: Schema:Edit seems to incorrectly set users as anonymous {lion} - https://phabricator.wikimedia.org/T92596#1283891 (ggellerman) Thanks, @Halfak!