[00:32:23] Ironholds: I see many edits on the PageViewReview that are notes/todos, heads up that I'm going to edit this page boldly as we're using it not just to brainstorm about possible causes/solutions but also to summarize findings and communicate them [00:41:17] Ironholds: I'm actually going to move all your notes to the talk page, hope you don't mind. The first section should really be about data quality and there should be another one about possible internal causes, separate from data quality [00:57:17] DarTar, no problemo [01:53:46] dennyvrandecic: hi, I'm Dan Andreescu, just replied to your comment on the reportcard wikidata bug thing [01:53:52] let me know if any of that was confusing :) [01:54:44] and I, for one, would love to see WikiData get mentioned in the next meeting. The increase in use has been nothing short of amazing [13:00:42] [travis-ci] master/9a5cd3f (#154 by Dan Andreescu): The build passed. http://travis-ci.org/wikimedia/limn/builds/16930972 [14:40:34] morning ottomata [14:40:54] so do we know exactly what time_firstbyte is measuring? [14:41:09] time when the client received the first byte or time when the first byte started to be sent? [14:43:19] milimetric: https://www.varnish-cache.org/trac/changeset/7754eb3baecceadb9051dc4f768e09c6ce0ef176 [14:44:02] awesome, thanks Snaps [14:44:08] so when the byte is sent [14:44:14] interesting [14:47:47] moorning! [14:49:55] Hi [14:50:18] Poor qchris is again getting no love from google. [14:51:22] I am on the invite ... but I cannot join :-( [14:51:44] I tried deleting myself and adding again, but it does not help either. [14:51:58] Could some of you invite me to the hangout again, once you joined [14:52:07] (That worked the last few times) [14:56:08] done [15:07:59] ... [15:08:42] heyaa [15:08:44] ok yeah [15:08:50] do you have a google email we could invite you with? [15:09:02] i think that might be the problem [15:09:02] Dwan^ [15:09:04] yes [15:09:09] charles.a.salvia@gmail.com [15:09:31] ok try now [15:51:58] nuria / Dwan, wikimetrics kick off now-ish? [15:52:00] quick break? [15:52:09] ok [15:52:15] sounds good [15:53:26] ok, I'm in the batcave whenever you're ready [15:53:35] nuria / Dwan: https://plus.google.com/hangouts/_/calendar/d2lraW1lZGlhLm9yZ19jYjM3bXU0OGNuaHRkN2hybmE4czI3b25hb0Bncm91cC5jYWxlbmRhci5nb29nbGUuY29t.c6j7qidqs491nhi7ovk9pi4h14 [15:53:41] :) [15:53:43] damn [15:56:45] Dwan: wanna join us back in the hangout? [15:56:52] we'll get started with wikimetrics stuff [16:48:31] Hey Toby... do you expect that my email address should be available today? I want to generate a public key and associate it with the address so I can connect to the various servers [16:49:56] I'll ask [16:55:10] ok thanks [17:01:56] Dwan, you need an account in office wiki to do that [17:02:10] so, both e-mail address and account if office are needed [17:02:35] they should come about the same time [19:04:42] ottomata, milimetric, I'm getting the runaround in -labs about how to set up a limn instance on multimedia-metrics.wmflabs.org, can you point at a puppet module that's helpful? [19:04:58] oh, i can help you with that marktraceur [19:05:05] Wunderbar. [19:05:07] I've just gotta finish up a meeting with nuria [19:05:13] do you have time a bit later? [19:05:15] 'kay no rush [19:05:15] i'll poke you [19:05:16] k [19:05:59] marktraceur: runaround is a type of puzzle... you hunt? :) [19:06:00] marktraceur: https://github.com/wikimedia/operations-puppet/tree/production/modules/limn [19:06:16] jeremyb: price has been on me to come with him this weekend [19:06:25] idk who that is [19:06:25] I'm not sure I'll have time! But I'm considering it. [19:06:30] but ok :) [19:06:40] He's on Metaphysical Plant [19:06:43] i think i may hunt part time [19:06:49] Formerly Atlas Shrugged [19:07:05] you should join codex... [19:07:15] ottomata: I was looking at that but it doesn't look like the limn class is helpful [19:07:26] * jeremyb runs away [19:07:30] no? [19:07:48] limn::instance { 'multimedia-metrics': } doesn't work? [19:07:58] (note you have to clone limn into /usr/local/share/limn yourself) [19:08:04] I'm not editing a puppet file by hand, I can't write that AFAIK [19:08:19] I'm editing the puppet config via wikitech [19:08:36] Not that it seems to be having any bloody effect [19:08:49] ahhh hmm [19:08:55] I'm still getting " Could not find class misc::limn::instance " after changing it to just use the limn class [19:09:02] yeah i think we don't have it there, i think we did self hosted puppet and modified manually [19:09:49] hmm, misc::limn::instance is a define [19:09:51] how are you using it? [19:28:59] marktraceur: sorry! done [19:28:59] and ready to help [19:28:59] so... ottomata, let's just put it up on limn0 [19:28:59] objections? [19:29:00] i'm migrating all those instances to use mw oauth anyway [19:29:01] while I'm in there I'll add a new instance [19:29:46] milimetric: I'm nearly there, I'm just not able to find where the mod_proxy definition should go I think [19:30:47] hm, but there are several reasons why you would want to just be on limn0 [19:31:06] 1. i'll upgrade your instance with the rest [19:31:15] 2. i'll fix any problems in one spot as opposed to many [19:31:37] 3. won't need a separate IP address [19:31:40] Guh [19:31:49] milimetric: -alpha already has an IP address, so it's fine [19:31:58] The others are helpful I guess [19:32:09] milimetric: OK I'm game [19:32:16] and most important 4. I'm not sure what mod_proxy is :) [19:32:40] :/ [19:32:44] like, i don't want to kill your steam here, if you're actually really close then I can try helping [19:32:51] where are you seeing this mod proxy business? [19:33:16] milimetric: OK, so, a node process like limn can't talk to a webserver directly [19:33:36] milimetric: It uses a reverse proxy over HTTP locally to accomplish basically the same goal [19:33:55] oh that thing [19:34:05] no idea how that's set up, yuvi and otto did it [19:34:34] Yeah [19:34:42] and I never understood why you need that if you have apache configured to hit the node process? [19:34:43] hmmmmmmmmm [19:34:55] i think that was before you could do arbitrary port mapping in labs [19:34:57] Never mind [19:35:11] milimetric: Just stick it on limn0 [19:35:14] k :) [19:35:15] happily [19:35:16] ah yeah [19:35:17] now there is this [19:35:18] https://wikitech.wikimedia.org/wiki/Special:NovaProxy [19:35:21] this didn't used to exist [19:35:48] Better than continuing to fuck around with puppet [19:35:51] haha [19:35:55] you were almost there :p [19:36:08] "almost" [19:36:09] sorry marktraceur, we made that limn puppet stuff a long time ago and it has basically only been used by dan [19:36:21] i betcha you could skip misc::limn::instance [19:36:24] and use limn::instance directly [19:36:26] oh hm... [19:36:28] forget the proxy [19:36:36] wait ottomata [19:36:39] ottomata: "yes" but I don't super like that plan anyway [19:36:40] and then in Special:NovaProxy set up a name map [19:36:41] he's just gonna put it on limn0 [19:36:45] yea yeah ye [19:36:47] that's fine :) [19:36:50] now, the problem [19:37:06] how's config.json handle by puppet, I forget all this stuff [19:37:15] hangout ottomata? [19:37:31] you're welcome to join marktraceur: https://plus.google.com/hangouts/_/calendar/d2lraW1lZGlhLm9yZ19jYjM3bXU0OGNuaHRkN2hybmE4czI3b25hb0Bncm91cC5jYWxlbmRhci5nb29nbGUuY29t.c6j7qidqs491nhi7ovk9pi4h14 [19:37:54] * marktraceur should really start putting pointy objects into his eyes first [19:38:27] I think I'll move to other tasks, let me know if I can help, but basically I don't understand this puppet thing very well [19:42:49] you were so close :p ! [19:42:49] :) [19:43:05] milimetric: btw, ori's mediaiwiki vagrant is so slick [19:43:10] :) [19:43:11] I know [19:43:15] might try to add wikimetrics and limn roles there too, eh?!: [19:43:22] that'd be great [19:43:24] not sure about lim [19:43:26] *limn [19:43:30] maybe pause on that for now [19:43:30] limn would be easier [19:43:32] k [19:43:41] well, i'm not doing anything else til ori merges my patches :p [19:43:43] ok ok [19:43:44] time to run [19:44:21] and with wikimetrics, maybe it'd be a good thing for us to do with charles [19:44:21] k [19:44:21] i just mean we may very well end up not using limn the way it is now [19:44:22] so i'd hate if you put too much work into it [19:48:33] Ya i vote for ading wikimetrics [19:48:38] let's hold on lim [19:49:27] hm, puppetd -tv didn't apply the puppet config there, there must be some other way to do it when it's self hosted [19:52:22] hmm no that should be it [19:52:23] hm [19:52:38] going to run puppet and check [20:01:01] marktraceur: what repository is hosting your limn graphs/datasources/etc? [20:02:23] ok milimetric, there was some weird unlreated puppet error with /usr/local/bin/position-of-the-moon :/ [20:02:28] i just symlinked /bin/true to that [20:02:33] and then puppet ran [20:02:34] :) [20:02:36] and put the instance in place [20:02:46] i didn't know stuff like that affected the rest of the puppet run [20:02:50] thanks! [20:02:54] sorry to keep you [20:02:57] I didn't ping your name on purpose [20:04:23] well, it usually doesn't [20:04:37] but for some reason this was causing it to fail [20:04:38] funno [20:04:39] whatever [20:04:44] s'ok, i am getting ready to leave [20:04:46] it is raining outside [20:04:48] putting on rain pants! [20:05:16] :) [20:24:57] milimetric: for https://wikimedia.mingle.thoughtworks.com/projects/analytics/cards/1302 / https://bugzilla.wikimedia.org/show_bug.cgi?id=58129 I'm guessing the answer is "send a note to Toby Negrin" right? [20:25:10] (on the About page, "it would probably be a good idea to tell people [20:25:10] *how they can gain access* to WikiMetrics, whether it is "talk to WMF (email [20:25:10] here)" or or something else?") [20:27:38] sumanah: no, the answer is that anyone can just log in [20:28:00] drdee pinged me about this when you sent it to him and I meant to do it but it got brushed under the rug [20:28:03] sorry about that [20:28:11] milimetric: no prob, I'll update the bug & patch accordingly [20:28:32] yeah, totally, you're welcome to add any words to the effect of "Wikimetrics welcomes all :)" [20:36:25] (PS1) Gerrit Patch Uploader: adding info on how to get access to wikimetrics [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/107431 [20:36:31] (CR) Gerrit Patch Uploader: "This commit was uploaded using the Gerrit Patch Uploader [1]." [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/107431 (owner: Gerrit Patch Uploader) [20:42:41] (CR) Milimetric: "Thanks! Just one nitpick." (1 comment) [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/107431 (owner: Gerrit Patch Uploader) [20:53:10] (PS1) Gerrit Patch Uploader: adding info on how to get access to wikimetrics [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/107470 [20:53:14] (CR) Gerrit Patch Uploader: "This commit was uploaded using the Gerrit Patch Uploader [1]." [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/107470 (owner: Gerrit Patch Uploader) [20:54:42] (CR) Sharihareswara: "Please abandon in favor of https://gerrit.wikimedia.org/r/#/c/107470/ ." [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/107431 (owner: Gerrit Patch Uploader) [21:08:37] back back back [21:08:41] not wet at all ha haa! [21:13:30] heya average, you around? [21:18:11] milimetric: I don't have them anywhere, just locally and on my labs instance [21:22:37] hey milimetric [21:22:44] what would be the proper place to drop some tests in this repo? [21:22:44] https://github.com/wikimedia/operations-software-ganglios [21:22:48] src/tests? [21:23:27] oo, maybe just test.py [21:25:45] (CR) Milimetric: [C: 2 V: 2] adding info on how to get access to wikimetrics [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/107470 (owner: Gerrit Patch Uploader) [21:26:20] (Abandoned) Milimetric: adding info on how to get access to wikimetrics [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/107431 (owner: Gerrit Patch Uploader) [21:29:36] thanks sumanah, sorry I didn't know you didn't have --amend [21:29:37] I'll patch stuff manually in the future instead of bugging you to abandon / resubmit [21:29:38] ottomata: nosetests searches by default for anything with "test" in the path and classname [21:29:38] so tests.py [21:29:39] or tests/test_file_1.py [21:29:40] those are all fine [21:29:41] I wouldn't put them in src as that's usually just source [21:29:41] and I wouldn't have an src in the first place as the convention in python is to name your root folder after your module's name [21:29:42] eg. wikimetrics has a tests folder and a wikimetrics folder [21:29:53] oh no prob milimetric I am in the process of getting --amend to work right now! [21:31:01] marktraceur: mind making a repository of the metadata files with the following folders: [21:31:12] yeah, ganglios does have a ganglios folder, but meh [21:31:12] hm [21:31:18] thanks milimetric [21:31:20] graphs/ [21:31:20] datasources/ [21:31:20] datafiles/ [21:31:20] dashboards/ [21:31:28] So...the var directory? [21:31:38] var/data [21:32:00] Right. [21:32:18] the repo can be anywhere you'd like, but you can put it in gerrit and add me as a reviewer if you want [21:32:38] it just makes deployments easier [21:32:40] I'll just stick it on Gitorious for now [21:35:29] milimetric: https://gitorious.org/analytics/multimedia-limn [21:35:35] cool, thx [21:35:57] (PS1) Sharihareswara: "Wiki Metrics" -> "Wikimetrics" consistently [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/107486 [21:41:51] marktraceur: https://github.com/wikimedia/limn-deploy/blob/master/fabfile/stages.py#L359 [21:41:56] that's the stage I added to the deployer [21:42:11] I can now deploy changes you make in that repo with "fab multimedia deploy.only_data" [21:42:24] and upgrade limn with "fab multimedia deploy.only_code" [21:44:07] and I can show you how as well, you just need to clone that repo and pip install its dependencies [21:45:45] 'kay [21:46:07] milimetric: Do we have a limn instance up, then? [21:46:23] milimetric: not to be a total pain, but you're talking about installing something with pip just on his dev env, right? (IIUC Ops doesn't want us to pip install stuff) [21:46:33] everything seems ok, except the routing to the expected address seems broken [21:46:48] just on dev of course [21:46:50] :) [21:47:06] deployer is installed locally [21:47:10] and deploys over ssh [21:47:33] * sumanah is taught :) [21:47:47] marktraceur: any idea about how to debug why multimedia-metrics.wmflabs.org isn't working? [21:47:53] do you have it pointing anywhere? [21:48:24] I don't right now [21:48:32] it's not even hitting the access logs in apache [21:48:33] oh! [21:48:34] milimetric: What instance has the limn? [21:48:35] restart apache [21:48:36] duh [21:48:42] limn0 [21:48:48] http://multimedia-metrics.wmflabs.org/ [21:49:11] hm, restarting apache didn't do it either [21:49:13] hm hm [21:49:15] Coolio [21:50:20] k, if i curl it locally it gets me the inde [21:50:21] *index [21:50:30] so it's definitely an apache type of thing [21:50:32] hm hm [21:52:03] this smells a lot like a dns issue marktraceur [21:52:07] gonna go ask in labs [21:52:46] I can get it [21:53:30] boo you [21:53:39] ah! [21:53:41] I know why [21:53:42] hahaha [21:53:50] it's because I added google to my DNS [21:54:01] must get all confused [21:54:30] ok, so marktraceur: how's it look? [21:54:44] your graphs are there, etc? [21:54:52] Well [21:54:54] My datasources [21:55:00] I don't have any graphs [21:55:06] k, but those show up ok? [21:55:12] want to go over how you update once you have stuff? [22:04:14] k, I'm out for now. marktraceur just send me an email if you need help with anything [22:04:35] I think it's fine, and I'm heading out soon anyway [22:14:00] [travis-ci] develop/cf60e5b (#155 by milimetric): The build passed. http://travis-ci.org/wikimedia/limn/builds/16961218 [22:21:39] [travis-ci] master/70424ed (#159 by milimetric): The build passed. http://travis-ci.org/wikimedia/limn/builds/16961548 [23:38:07] (PS3) Jdlrobson: [WIP] Use NewEditorEdit schema for new graph [analytics/limn-mobile-data] - https://gerrit.wikimedia.org/r/105869