[13:02:33] <grrrit-wm>	 (CR) QChris: Add scheduler mode to wikimetrics (7 comments) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/111623 (owner: Milimetric)
[13:21:30] <grrrit-wm>	 (CR) Nuria: Add scheduler mode to wikimetrics (1 comment) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/111623 (owner: Milimetric)
[13:29:55] <grrrit-wm>	 (CR) Nuria: [WIP] Run recurring reports using the scheduler (4 comments) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/112165 (owner: Milimetric)
[14:00:38] <qchris>	 average, csalvia: We're in the appear.in hangout for standup
[14:30:18] <milimetric>	 csalvia: nuria: I froze on the hangout
[14:30:25] <milimetric>	 sorry, but we can finish talking here quickly
[14:30:37] <milimetric>	 so nuria, you were saying it's ok if I comment out the delete stuff?
[14:30:37] <nuria>	 ok, very well
[14:30:57] <nuria>	 sure, we can "move it" to a different changeset
[14:31:06] <nuria>	 so as to reduce scope of the change
[14:31:21] <milimetric>	 ok, cool
[14:31:56] <milimetric>	 so this change is supposed to make sure that the recurrent / public notions make sense together
[14:32:03] <milimetric>	 I think that means we need to merge csalvia's change into it
[14:32:04] <nuria>	 yes
[14:32:11] <nuria>	 and that they rae tested
[14:32:13] <milimetric>	 but besides that, I think we're done
[14:32:16] <milimetric>	 oh right
[14:32:20] <milimetric>	 and add a shitton of tests :)
[14:32:25] <nuria>	 csalvia did you tested your change in vagrant?
[14:33:24] <nuria>	 what about the other two changesets you have?
[14:35:00] <nuria>	 sorry milimetric, what about the other two (smaller) changes?
[14:35:40] <milimetric>	 I think those are more or less fine.  After qchris gets done reviewing them, we can just merge those
[14:36:20] <milimetric>	 but i mean, i don't think they're missing anything fundamental, probably just need to be cleaned up
[14:37:06] <qchris>	 No need to wait for me. If someone else reviews stuff, it would make my life simpler.
[14:37:23] <qchris>	 So just review and merge! :-)
[14:37:31] <nuria>	 the only remaining thing on those (that I can see)
[14:37:47] <nuria>	 is that they are creating files on source
[14:37:59] <nuria>	 which will also not work in production
[14:38:19] <nuria>	 but I think qchris had another work item
[14:38:30] <milimetric>	 oh, the scheduled_tasks file?
[14:38:55] <nuria>	 of making that very change work in vagrant/puppet setup
[14:38:55] <milimetric>	 yeah, the path to that is configurable but we have to add the setting to the puppet config
[14:38:55] <nuria>	 yes
[14:39:17] <milimetric>	 we basically just have to update the puppet template for the queue config to match the one in the Change
[14:40:40] <nuria>	 yes, that and create the directory where scheduling would go via puppet
[14:41:02] <nuria>	 then, are we doing that with your change or after?
[14:41:47] <milimetric>	 well isn't it a separate change anyway nuria?  I mean, it's a puppet change that can't be linked to this, right?
[14:42:56] <grrrit-wm>	 (PS31) Milimetric: [WIP] Run recurring reports using the scheduler [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/112165
[14:43:00] <grrrit-wm>	 (CR) jenkins-bot: [V: -1] [WIP] Run recurring reports using the scheduler [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/112165 (owner: Milimetric)
[14:43:35] <milimetric>	 weird, flake8 shows fine on my machine but jenkins -1ed me
[14:44:22] <milimetric>	 well, nuria, I pushed my latest to that change, so feel free to clean up / hack at it, it's all yours until you're done for today
[14:44:29] <nuria>	 milimetric see this one: https://gerrit.wikimedia.org/r/#/c/111623/13/wikimetrics/configurables.py
[14:44:51] <milimetric>	 but push what you have at the end and I'll try to finish it tonight like I promised :)
[14:46:03] <nuria>	 Will do.
[14:46:49] <nuria>	 As for teh other change you are right, rest of changes are on puppet minus the one mentioned by christian on configurables.py
[14:46:53] <milimetric>	 hm, that's an interesting point, nuria you meant qchris's comment on that configurables?
[14:47:03] <nuria>	 right
[14:47:21] <milimetric>	 so if we put that in config, it means we have to somehow put that big dictionary into yaml and read it out properly
[14:47:23] <milimetric>	 does that work?
[14:49:40] <milimetric>	 like, I mean we'd need this in yaml somehow:
[14:49:41] <milimetric>	 	CELERYBEAT_SCHEDULE = {	3
[14:49:41] <milimetric>	     # Executes every day at midnight 	4
[14:49:41] <milimetric>	     'update-daily-recurring-reports': {	5
[14:49:41] <milimetric>	         'task': 'wikimetrics.schedules.daily.recurring_reports',	6
[14:49:42] <milimetric>	         'schedule': crontab(minute=0, hour=0),	7
[14:49:44] <milimetric>	     },	8
[14:49:46] <milimetric>	 }
[14:50:15] <nuria>	 mmm..
[14:51:06] <milimetric>	 ok, gotta get ready to walk to the office, but i'll leave irc on
[14:51:18] <milimetric>	 thanks for the comments qchris, I'll try to address today
[14:51:23] <nuria>	 ok, let's tak later
[14:51:38] <qchris>	 milimetric: yw :-)
[14:52:31] <qchris>	 milimetric: about the yaml thing above. You mean my request about reordering so we can get rid of the coupling?
[14:53:22] <milimetric>	 not reordering, you mentioned trying to put CELERYBEAT_SCHEDULE in the configuration so that it's loaded and we don't have to load it in code
[14:54:01] <milimetric>	 I admit that would be better but I'm not sure how to technically do it
[14:54:38] <qchris>	 Technically, it could just be merging dictionaries.
[14:54:56] <qchris>	 First loading some (maybe hard-coded) defaults into the dict.
[14:55:10] <qchris>	 Then merging in user supplied key/values.
[14:55:24] <qchris>	 Then merging in overrides.
[14:56:02] <qchris>	 (Currently we have loading user supplied values. Then merging overrides. Then merging the schedule)
[15:01:41] <milimetric>	 oh, ok, so we'd still have code interact with the CELERYBEAT_SCHEDULE setting, but in a generic way and not a specific way
[15:01:45] <milimetric>	 cool, I like it, will make it so
[15:02:02] <milimetric>	 the other part I wasn't sure I understood was the scheduled_tasks controversy :)
[15:02:58] <milimetric>	 so, I'll fix the gitignore to only expect those files at the root of the repository, but why is that "poisoned"?  wouldn't putting them somewhere else in the filesystem have to necessarily assume some file permissions we're not guaranteed to have?
[15:10:58] <qchris>	 milimetric: About scheduled_tasks. Yes, having those files in a well defined place would be great.
[15:11:19] <milimetric>	 like, is ./scheduled_tasks ok?
[15:11:22] <qchris>	 I am not sure root would be the best place. Some projects have a special directory where generated files go to.
[15:11:26] <milimetric>	 oh ok
[15:11:28] <milimetric>	 i see
[15:11:39] <milimetric>	 that's a good idea, let's make a generated folder
[15:11:39] <qchris>	 Then the main directory and the sources etc stay clean
[15:11:52] <milimetric>	 i like it, we do that
[15:12:09] <qchris>	 So if some process dies horribly, you can just kill the generated directory, and you are back to normal.
[15:12:12] <qchris>	 Cool \o/
[15:12:15] <qchris>	 Thanks.
[15:12:31] <milimetric>	 yeah, in general if I'm doing something dumb like this you can feel free to suggest a solution
[15:12:37] <milimetric>	 it's probably because I don't know any better :)
[15:12:56] <qchris>	 :-P
[15:12:57] <milimetric>	 ok, gonna go grab breakfast and I'll be back online in an hour or so
[15:13:02] <milimetric>	 do you guys need anything from me?
[15:13:03] <qchris>	 Meh. Both solutions work.
[15:13:14] <qchris>	 Enjoy your breakfast!
[15:13:26] <milimetric>	 k :)
[15:34:15] <grrrit-wm>	 (PS1) Nuria: Adding coding guidelines to README [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/120541
[15:34:19] <grrrit-wm>	 (CR) jenkins-bot: [V: -1] Adding coding guidelines to README [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/120541 (owner: Nuria)
[15:34:42] <nuria>	 opppps sorry
[15:36:41] <grrrit-wm>	 (Abandoned) Nuria: Adding coding guidelines to README [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/120541 (owner: Nuria)
[15:38:42] <grrrit-wm>	 (PS1) Nuria: Adding coding guidelines to README.md file [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/120542
[15:41:36] <nuria>	 milimetric: a generated directory sounds good. Same aproach we are using with public reports
[15:42:11] <nuria>	 I agree with qchris, we should not be creating files on source
[16:34:31] <Niharika>	 Hello. I was looking for some stats on a beta feature usage across all wikis(or as many as possible). Is there anything for that?
[16:41:56] <grrrit-wm>	 (CR) Nuria: [WIP] Run recurring reports using the scheduler (1 comment) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/112165 (owner: Milimetric)
[16:42:39] <milimetric>	 hi nuria
[16:42:41] <milimetric>	 we're in the office
[16:43:41] <nuria>	 ok, very well. I updated README , send CR to qchris and looked at your patch #31. from what i can see what is left to be done is writing tests
[16:44:09] <nuria>	 is there something else you had in mind?
[16:46:15] <milimetric>	 nope, just checking if you want to hang out before you leave for the day
[16:46:58] <milimetric>	 thanks for taking a look at #31, I'll write tests all day :)
[17:34:32] <terrrydactyl>	 hi, is anyone around that can help me with a sqlalchemy/database problem?
[17:45:56] <grrrit-wm>	 (PS2) Nuria: Adding coding guidelines to README.md file [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/120542
[17:49:48] <nuria>	   Hi, terrrydactyl with wikimetrics?
[17:50:42] <terrrydactyl>	 yeah. i'm getting some trouble with filtering through my tables
[17:50:43] <terrrydactyl>	 https://gerrit.wikimedia.org/r/#/c/119343/3/tests/test_controllers/test_cohorts.py
[17:50:57] <terrrydactyl>	 i get an ObjectDeletedError when i try to run my tests.
[17:52:04] <terrrydactyl>	 any ideas nuria?
[17:54:07] <ottomata>	 milimetric: is all of limn and wikimetrics now fully migrated to eqiad labs?
[17:54:43] <nuria>	 terrrydactyl
[17:55:02] <nuria>	 you have a missmatch between the data you shold in your orm until you commit your session
[17:55:10] <nuria>	 and what is on the db
[17:55:28] <nuria>	 I think it will help to separate the code a little bit and test in smaller units
[17:55:49] <nuria>	 like separate controller logic from storage logic and test those separately 1st
[17:56:32] <milimetric>	 yes ottomata, all migrated
[17:57:09] <terrrydactyl>	 nuria: do you mean deleting each element in a separate function and testing that?
[17:57:49] <nuria>	 terrrydactyl actually separating your testing logic in smaller units
[17:58:03] <nuria>	 but not on separate functions
[17:59:10] <terrrydactyl>	 the current testing logic is to test if the cohort is deleted if there is an owner and no viewers, test if cohort i deleted if there is an owner and viewers, and test if cohort is deleted from a viewer's personal list. i'm not sure how you mean to separate it out more.
[17:59:44] <nuria>	 sorry, let me rexplain: so you are not testing the controllers and storage at the same time
[18:00:18] <nuria>	 just test storage operations on db separated from what is going on at the controller layer, amkes sense?
[18:00:23] <nuria>	 *makes sense?
[18:01:54] <terrrydactyl>	 yeah i think so
[18:02:00] <terrrydactyl>	 let me mull over it for a bit
[18:03:33] <nuria>	 i will be offline but dan is in SF , he can help you to separate logic a bit which will uncover where is the error coming from
[18:07:12] <terrrydactyl>	 okay, thanks nuria
[18:17:38] <ottomata>	 milimetric: just to be double sure, it is ok to delete the pmtpa instances, right?
[18:56:19] <Ironholds>	 anyone remember the URL to the page explaining the sampled requestlogs format?
[18:57:30] <Ironholds>	 milimetric, around?
[18:57:35] <Ironholds>	 the analysts/researchers are grabbing lunch
[18:58:15] <milimetric>	 hey Ironholds
[18:58:20] <milimetric>	 yes, we're just wrapping up a meeting
[18:58:25] <milimetric>	 let's all do lunch
[18:58:39] <milimetric>	 ottomata: sorry, in meeting but yes, safe to delete all instances, I care not about any of them
[18:58:51] <milimetric>	 *all pmtpa instances :)
[19:01:31] <ottomata>	 Ironholds: found it
[19:01:31] <ottomata>	 https://www.mediawiki.org/wiki/Analytics/Kraken/Data_Formats#Web_Request_Format
[19:02:59] <Ironholds>	 ottomata, ta!
[19:03:01] <Ironholds>	 milimetric, cool!
[19:19:01] <matanya>	 DarTar: any news : https://rt.wikimedia.org/Ticket/Display.html?id=6845 ?
[20:10:37] <DarTar>	 matanya: no, I applied for a 1-week extension :p
[20:10:51] <DarTar>	 but I’ll send out a reminder
[20:10:55] <matanya>	 haven't it passed? :)
[20:11:00] <DarTar>	 hmm I don’t think so
[20:11:43] <matanya>	 was due to march 13 +7 days = 20
[20:11:50] <matanya>	 it is 24 now :P
[20:12:42] <DarTar>	 shite, ok I thought I had it for Wed/Thurs this week
[20:14:01] <DarTar>	 matanya: I’ll send out a reminder in a moment, I am sorry about the delay
[20:14:33] <matanya>	 no worries, just a friendly reminder
[20:17:46] <ottomata>	 milimetric: can I delete hadoop-test1 and 2, and hypercube?
[20:17:52] <milimetric>	 yes
[20:17:57] <ottomata>	 anyone know anything about foundry or smartdoc instances?
[20:18:28] <ottomata>	 those are the only two instances we have left in pmtpa!
[20:20:25] <Ironholds>	 tetragenic!
[20:20:46] <Ironholds>	 mutagens cause active genetic damage, tetragens damage an embroyo directly without altering the genetic code
[20:20:49] <Ironholds>	 (sorry; debate at lunch)
[21:09:38] <milimetric>	 ottomata: nobody spoke up about foundry or smartdoc, so let's AXE them :)
[21:09:55] <ottomata>	 k i mean, where did they come from?
[21:09:56] <ottomata>	 what are they?
[21:09:59] <ottomata>	 those are ones I didn't know anything bout
[21:16:18] <qchris>	 Ironholds: I updated the page you got referred to with the maintained version of the webrequest log format:
[21:16:19] <qchris>	 https://wikitech.wikimedia.org/wiki/Cache_log_format
[21:22:11] <Ironholds>	 qchris, thanks!
[21:27:09] <milimetric>	 ottomata: i know nothing about those instances, I looked and couldn't figure out what they were for
[21:27:22] <ottomata>	 k i'll wait and ask in standup tomorrow just in case
[21:27:22] <milimetric>	 I'm certain that if neither you or me care/know about them, we're safe
[21:27:27] <milimetric>	 ok
[21:27:29] <ottomata>	 no reason not to
[21:39:29] <average>	 how's appear.in ? is it stable ? can you guys use it ?
[21:39:40] <average>	 I might jump in too tommorow
[21:46:02] <DarTar>	 matanya: reminders sent, I have one account confirmed for removal, I hope to hear about the remaining ones withint 24-48 hrs
[21:54:55] <qchris>	 average: Yes, It has been working fine so far (at least for me)
[21:55:18] <qchris>	 We had the last couple of standup using appear.in, and worked well.
[21:55:33] <qchris>	 And you can do without Google plugins :-)
[21:56:19] <average>	 very nice :)
[21:59:10] <matanya>	 thanks DarTar please update that account and i'll preper a patch for it
[22:02:23] <matanya>	 *the ticket
[22:05:51] <milimetric>	 hey, that'd be awesome average :)
[22:06:16] <milimetric>	 and yeah, so you know the batcave address right?  appear.in/wmf-analytics-batcave
[22:41:11] <sumanah>	 halfak: hey there, got a moment?
[23:13:06] <halfak>	 sumanah, sorry to miss you.  What's up?
[23:14:03] <sumanah>	 oh halfak in #wikimedia-research I asked: I could swear that, back in 2011 or 2012, I saw our research about how new people hit Edit, see wikitext, and bounce off.
[23:14:23] <sumanah>	 halfak: And I'm giving a speech tomorrow and I thought I'd mention the bounce rate but I can't find it!
[23:14:44] <halfak>	 Hmm...  Nothing I'm aware of.  There could have been user studies that were done before I came on.
[23:14:50] <halfak>	 Sorry to not be able to help more.
[23:15:01] <halfak>	 Did you ask DarTar?
[23:15:50] <DarTar>	 hey sumanah
[23:15:54] <sumanah>	 Hi DarTar!
[23:16:03] <sumanah>	 halfak: thanks for the thought!
[23:16:10] <DarTar>	 was that the Trevor experiment?
[23:16:33] <sumanah>	 Could have been
[23:17:21] <sumanah>	 Maybe I've been searching on the wrong keywords, because I do not think I've found it, DarTar, and would welcome any pointers
[23:17:50] <DarTar>	 that was about making the edit icon larger
[23:18:17] <sumanah>	 oh, moving the edit icon to the left -- well, that might help illustrate my point too
[23:18:19] <DarTar>	 but as a result produced some data about the click-through and completion rate
[23:18:56] <DarTar>	 I can try and dig it up, I don’t think it;s on Meta
[23:21:04] <sumanah>	 https://meta.wikimedia.org/wiki/Change_to_section_edit_links
[23:21:15] <sumanah>	 "In 2009, the Wikipedia Usability Initiative proposed an approach where section edit links would be "moved next to the title" and icon would be added. Years later this was tested on a subset of all anonymous or registered editors to English Wikipedia, with promising results: the number of clicks on section edit links increased by 116%!"
[23:25:16] <sumanah>	 https://meta.wikimedia.org/wiki/Research:Section_edit_modification
[23:25:22] <sumanah>	 thanks DarTar, I think I can use this
[23:27:58] <DarTar>	 sumanah: yes, keep in mind that was never run again as a controlled test in Growth/E3, it’s something I feel we should try to do at some point
[23:28:49] <DarTar>	 and assuming Growth (or VE or whoever wants to test any modification of the UI) has bandwidth
[23:29:15] <sumanah>	 Right, got it
[23:33:39] <grrrit-wm>	 (CR) QChris: [C: -1] Adding coding guidelines to README.md file (15 comments) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/120542 (owner: Nuria)
[23:34:38] <milimetric>	 qchris: go to sleep! :)
[23:34:43] <milimetric>	 man, you work way too hard
[23:34:58] <qchris>	 Argh ... it's late. You're right.
[23:35:03] <qchris>	 :-)
[23:35:09] <qchris>	 Ok. You convinced me.
[23:35:13] <qchris>	 Good night.
[23:35:45] <grrrit-wm>	 (PS14) Milimetric: Add scheduler mode to wikimetrics [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/111623
[23:35:47] <grrrit-wm>	 (CR) jenkins-bot: [V: -1] Add scheduler mode to wikimetrics [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/111623 (owner: Milimetric)
[23:36:32] <grrrit-wm>	 (CR) Milimetric: "patchset 14 addresses the things I claim to address in these comments." (9 comments) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/111623 (owner: Milimetric)