[07:26:00] Analytics / Tech community metrics: Allow contributors to update their own details in tech metrics directly - https://bugzilla.wikimedia.org/58585#c25 (Alvaro) Hi Tapasweni, First of all, thank you very much for your interest in this project. Sarvesh did a nice prototype of what the project is lookin... [09:20:56] (PS1) Gilles: Add new graphs to global dashboard [analytics/multimedia/config] - https://gerrit.wikimedia.org/r/160620 [09:21:23] (CR) Gilles: [C: 2] "Tested locally" [analytics/multimedia/config] - https://gerrit.wikimedia.org/r/160620 (owner: Gilles) [09:21:35] (CR) Gilles: [V: 2] Add new graphs to global dashboard [analytics/multimedia/config] - https://gerrit.wikimedia.org/r/160620 (owner: Gilles) [12:07:29] (PS1) QChris: DO NOT MERGE Hive query to implement Webstatscollector logic [analytics/refinery] - https://gerrit.wikimedia.org/r/160636 [12:11:14] (CR) Yuvipanda: "Code-Review:+2 Verified:+2" [analytics/refinery] - https://gerrit.wikimedia.org/r/160636 (owner: QChris) [12:11:26] YuviPanda: :-P [12:11:32] qchris: :D [12:25:21] (PS2) QChris: DO NOT MERGE Hive query to implement Webstatscollector logic [analytics/refinery] - https://gerrit.wikimedia.org/r/160636 [12:26:43] (CR) QChris: [C: -2] DO NOT MERGE Hive query to implement Webstatscollector logic (4 comments) [analytics/refinery] - https://gerrit.wikimedia.org/r/160636 (owner: QChris) [12:32:19] (CR) QChris: DO NOT MERGE Hive query to implement Webstatscollector logic (1 comment) [analytics/refinery] - https://gerrit.wikimedia.org/r/160636 (owner: QChris) [13:01:27] qchris: gotta love how short that hive query is [13:01:32] even if it's not ready I imagine it won't get much longer [13:04:19] It already feels like a way too long query to me :-/ [13:12:22] (PS1) Milimetric: Change metric categories to what research defined [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/160641 (https://bugzilla.wikimedia.org/70871) [13:49:08] (CR) Ottomata: "> However, we do it nonetheless, as webstatscollector is doing it too." [analytics/refinery] - https://gerrit.wikimedia.org/r/160636 (owner: QChris) [13:59:44] Analytics / Dashiki: Update metric classification in Vital Signs dashboard - https://bugzilla.wikimedia.org/70871#c2 (Dan Andreescu) The patch will ensure this happens going forward when generating the config. I changed the names manually in the generated file while we wait for that. As a side note,... [14:00:29] HANGOUT PROBLEMS [14:00:31] CMOon [14:00:46] yeah it won't let me in to the batcave and nobody's on the link from teh calendar [14:02:02] had to try 4 times [14:02:05] eventually it let me in [14:02:16] nuria_, kevinator, jgage, I'm re-inviting all of you [14:02:37] qchris: I re-invited you too [14:39:50] milimetric: is there a way in limn to tell an ordinal bar chart that the first row is a header? [14:40:03] e.g. I have a css with [14:40:12] type,value [14:40:15] foo,1 [14:40:17] bar,2 [14:40:25] and I pass this to a bar chart [14:40:47] then it will have three columns, type:0, foo:1 and bar:2 [14:42:00] I could just cut the header from the csv, but is there a nicer solution? headers are handled just fine by most other data/graph types [14:52:51] ottomata: can you help us one sec ? [14:52:58] 1 min... [14:52:59] we are on the batcave [15:40:30] Analytics / Wikimetrics: Story: EEVSUser downloads report with correct Http Cache Headers - https://bugzilla.wikimedia.org/68445 (nuria) NEW>RESO/FIX [15:40:44] tgr: sorry i was in a prod issue thing [15:40:47] lemme look [15:43:17] tgr: no, it's stupid: https://github.com/wikimedia/limn/blob/master/src/data/dataset/series-dataset.co#L53 [15:43:46] above that line we could add something like if @source.hasHeaderRow, skip a line [15:44:26] milimetric: if it's nontrivial, I can just truncate the file instead [15:44:55] it's trivial, but i have to respect some other priorities right now. If you want to submit that pull request I'll merge it and deploy [15:46:03] I'll see if I can figure it out [15:48:31] ok nuria_, am back [15:48:40] ok, committing now [15:49:40] ottomata: https://gerrit.wikimedia.org/r/160672 [15:49:45] let me test this on staging [15:50:48] nuria_ , milimetric : can I declare the “http headers” story complete at the showcase? the bug is still open [15:51:02] ok [15:51:07] kevinator, i just closed it right? [15:51:13] nevermind, I just hit refresh and it’s closed now [15:51:25] thanks [15:52:07] The deck for the showcase is here: https://docs.google.com/presentation/d/1y54uF5PkYc9Sa7VWOykKXQ4DXqh_n3VxDMAR2-CCKss/edit?usp=sharing [15:52:12] feel free to edit [15:52:16] I’m commuting to work now [15:52:22] er office [15:57:15] ottomata: changing where var/directory points to here https://gerrit.wikimedia.org/r/160672 [15:57:42] nuria_: , cool on staging? [15:57:44] i shoudl merge then? [15:57:50] no, does not work [15:57:57] oh [15:57:59] in staging there are no chnages [15:58:07] let me make sure [15:58:25] nuria, this changed, right? [15:58:26] PUBLIC_PATH : /var/lib/wikimetrics/public [15:58:29] oh [15:58:29] let me rerun puppet [15:58:31] no it didn't [15:58:32] it hsould. [15:59:40] wait, i did not pulled teh changeset , argh [16:00:19] ottomata: on staging [16:00:20] Info: Retrieving plugin [16:00:20] Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Could not find class base for wikimetrics-staging1.eqiad.wmflabs on node wikimetrics-staging1.eqiad.wmflabs [16:00:20] Warning: Not using cache on failed catalog [16:00:20] Error: Could not retrieve catalog; skipping run [16:01:17] hm, wha? [16:01:34] trying myself... [16:01:41] wait.... [16:01:46] i think things are too old [16:01:49] oph [16:01:50] oh [16:02:08] oh yeah [16:02:09] totally [16:02:16] rebase on origin/production [16:02:31] i think this is leftover from qchris and I yesterday trying to figure out why the salt puppet stuff was broker [16:02:33] broken [16:02:48] hm, actually [16:02:50] yeah, may I/ [16:02:53] i think i want to undo something... [16:02:55] nuia? [16:02:57] nuria? [16:04:18] ottomata: reruning puppet now [16:04:33] from production branch on staging [16:04:33] cool ok [16:04:51] i don't see your change on there though [16:05:04] which is updated as of sep 15th with our commit of april 28th on top [16:05:14] also, submodule is dirty, might need submodule update there [16:05:14] it's on the module [16:05:19] wha? [16:05:19] k [16:05:25] oh it is! [16:05:30] change it in role! why change in module? :) [16:05:36] argh, [16:06:13] let me re-do changing it in role. i though you mean the module [16:11:20] ottomata: the changeset: https://gerrit.wikimedia.org/r/#/c/160679/1/manifests/role/wikimetrics.pp [16:11:24] trying on staing now [16:17:37] (PS1) Gergő Tisza: Fix relative UW funnel numbers [analytics/multimedia] - https://gerrit.wikimedia.org/r/160682 [16:18:32] ottomata: Error: Could not retrieve catalog from remote server: Error 400 on SERVER: Duplicate declaration: File[/srv/wikimetrics/wikimetrics/static/public] is already declared in file /etc/puppet/modules/wikimetrics/manifests/init.pp:208; cannot redeclare at /etc/puppet/modules/wikimetrics/manifests/web/daemon.pp:18 on node wikimetrics-staging1.eqiad.wmflabs [16:18:32] Warning: Not using cache on failed catalog [16:18:32] Error: Could not retrieve catalog; skipping run [16:19:51] IIIneresting [16:20:03] daemon.pp! [16:20:07] is staging using daemon? [16:20:07] hmm [16:21:11] hm, shouldn't be [16:21:34] (PS1) Gergő Tisza: Remove useless columns from ordinal chart [analytics/multimedia] - https://gerrit.wikimedia.org/r/160684 [16:21:41] no, but puppet will read it [16:21:56] (PS2) Gergő Tisza: Fix relative UW funnel numbers [analytics/multimedia] - https://gerrit.wikimedia.org/r/160682 [16:21:58] ottomata: i think we need to change the flask setup to deploy this change [16:22:32] ottomata: making link go teh other way, correct? [16:22:33] ah, because of the ensure, it includes both [16:22:40] but ensures => absent on daemon [16:22:40] hm [16:23:24] nuria_: i think we might have to do the right thing [16:23:38] which is, not put var directoring inside of /srv/wikimetrics [16:23:48] so [16:23:54] maybe [16:23:56] we need [16:24:03] /srv/var/wikimetrics ? [16:24:21] hmm, or, maybe [16:24:25] maybe we can ask yuvi [16:24:29] if we can levae /var/lib/wikimetrics in place [16:24:39] and just re-mount that as a new logical volume too [16:24:41] YuviPanda: ? [16:24:43] can we do that? [16:26:13] i think Yuvipanda might be asleeep [16:26:17] ohk [16:29:29] ottomata: can we find out from someone that is not yuvipanda whether we can remount /var/lib/wikimetrics [16:30:08] or maybe just "/var/lib/wikimetrics" will work just fine. we will need to change links in prod [16:30:51] (PS2) Milimetric: Match colors in graph with labels [analytics/dashiki] - https://gerrit.wikimedia.org/r/160532 [16:32:01] i am looking [16:32:50] grrrr [16:32:59] annoying part is that this /srv thing took up all of the volume group [16:33:00] $size = '100%FREE', [16:33:01] hm [16:36:35] ok, nuria_, hm, i think we can do that, but we need to delete the current volume group and recreate it [16:36:36] (PS1) Nuria: [WIP] Bootstrapping from url [analytics/dashiki] - https://gerrit.wikimedia.org/r/160685 [16:36:36] yragh [16:37:27] ottomata: ay ay [16:37:37] hmm [16:37:39] maybe.. [16:37:59] if i get a say i think the better choice is /srv/var/wikimetrics [16:38:00] hmm, i might be able to srhink it. [16:38:12] as it is more obvious as to what might be going on [16:39:19] nuria_: ah i think iI can do this [16:39:22] may I try on staging? [16:39:36] yes, plis [16:39:46] Analytics / Dashiki: Story: Bookmarks. Define protocol and use it to bootstrap the dashboard - https://bugzilla.wikimedia.org/70887 (nuria) NEW p:Unprio s:normal a:None Dahboard should bootstrap from a url like: http://dashiki.com/something#project=enwiki,dewiki/metric=newlyregister,rolling... [16:40:01] Analytics / Dashiki: Story: Bookmarks. Define protocol and use it to bootstrap the dashboard - https://bugzilla.wikimedia.org/70887 (nuria) a:nuria [16:43:55] ok, nuria_, that worked, i will submit a change to allow this to be done via the lvm::srv thing yuvi uses [16:44:09] ottomata: in puppet? [16:44:11] yes [16:44:14] NICE! [16:44:29] there is manual stuff to do on these instances still, but in the future new instances will do the right thing [16:45:33] i think we'll need Yuvi to approve it though [16:45:55] (PS2) Nuria: [WIP] Bootstrapping from url [analytics/dashiki] - https://gerrit.wikimedia.org/r/160685 (https://bugzilla.wikimedia.org/70887) [16:47:14] (CR) Nuria: [C: 2] Change metric categories to what research defined [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/160641 (https://bugzilla.wikimedia.org/70871) (owner: Milimetric) [16:47:20] (Merged) jenkins-bot: Change metric categories to what research defined [analytics/wikimetrics] - https://gerrit.wikimedia.org/r/160641 (https://bugzilla.wikimedia.org/70871) (owner: Milimetric) [16:48:25] ottomata: can you send me the changeset? I will try to ping yuvi [16:48:29] https://gerrit.wikimedia.org/r/#/c/160687/ [16:49:10] wow you alredy got a acomment from harshar [16:50:08] ottomata: but isn't this change missing teh particular changes for /var/lib [16:50:59] ? [16:52:10] yes [16:52:16] will do taht in a different changeset [16:54:09] ottomata: ok, please cc-me on that change too [16:54:31] k [16:54:37] hmmm, actually, i think i'm going to do this differently... [16:56:07] ottomata: k, how so? [16:58:04] i will not use the ::srv role class [16:58:13] so I can adjust the size wihtout having to mess with other use cases [17:00:34] ottomata: aha [17:01:13] nuria: https://gerrit.wikimedia.org/r/#/c/160689/ [17:01:19] (PS1) Milimetric: Match colors from Pau's design more closely [analytics/dashiki] - https://gerrit.wikimedia.org/r/160690 [17:01:51] i'm going to try that on staging [17:02:19] ok ottomata , what will happen on vagrant [17:02:24] will taht role be available? [17:02:48] (PS2) Milimetric: Match colors from Pau's wireframes more closely [analytics/dashiki] - https://gerrit.wikimedia.org/r/160690 [17:03:58] (CR) Nuria: [C: 2] Match colors from Pau's wireframes more closely [analytics/dashiki] - https://gerrit.wikimedia.org/r/160690 (owner: Milimetric) [17:05:04] no it won't matter, vagrant doesn't use lvm for this [17:05:09] it will only matter in labs [17:05:26] gah [17:05:48] (PS1) Milimetric: Build Second Release [analytics/dashiki] - https://gerrit.wikimedia.org/r/160693 [17:06:47] ottomata: ok [17:07:08] (CR) Nuria: [C: 2] Build Second Release [analytics/dashiki] - https://gerrit.wikimedia.org/r/160693 (owner: Milimetric) [17:08:38] ottomata: looks liek some puppet linting failed, let me know if you want me to test on staging [17:09:32] Yuvi has a bug in his labs_lvm::volume define [17:09:39] it needs fixed before this will work, just submitted a patch [17:10:19] ok, nuria_, can we wait for yuvi on this one? [17:10:23] production is working as is for now, right? [17:11:14] ottomata:, we can wait until tomorrow I suppose but it should be something we tackle soon, as basically we just have changes in prod made by hand [17:12:27] ottomata: I can try to catch yuvi when i am about to go to bed as he must be getting up [17:56:06] (PS1) Gergő Tisza: Add relative timeseries [analytics/multimedia] - https://gerrit.wikimedia.org/r/160779 [18:15:12] (PS1) Gergő Tisza: Add relative timeseries [analytics/multimedia/config] - https://gerrit.wikimedia.org/r/160785 [18:50:29] milimetric, holaaa [19:26:16] Analytics / Wikistats: "Top month" and "Trend last 24 months" missing in Wikipedia columns - https://bugzilla.wikimedia.org/70900 (Tilman Bayer) NEW p:Unprio s:normal a:None The row with this information is empty in the "Wikipedia non-mobile", "Wikipedia mobile" and "Wikipedia total" column... [19:29:42] milimetric: see your cotton candy grapes and offer Sumo oranges: http://www.sumocitrus.com/ (out Feb 2015) [19:38:13] (CR) Milimetric: [C: -1] Match colors in graph with labels (2 comments) [analytics/dashiki] - https://gerrit.wikimedia.org/r/160532 (owner: Milimetric) [20:02:25] ottomata: nuria_ heya [20:02:54] yoyo [20:04:52] ottomata: what broke, exactly? [20:04:58] * YuviPanda is slightly confused [20:05:04] why do we need the other volume [20:05:05] ? [20:05:30] /var was still full [20:05:38] wikimetrics writes static files to /var/lib/wikimetrics [20:05:57] aaaaahhhhhh [20:05:58] I see [20:06:04] didn't know that [20:06:14] and we can't parameterize that? [20:06:23] it is parameterized [20:06:57] and, there is already a symlinked maintained in /srv/wikimetrics/wikimetrics/static/public to point whatever the $wikimetrics_public_dir is [20:07:02] which is $var_directory/public [20:07:08] so, $var_directory is parameterized [20:07:21] right, so why is it still writing to /var/lib? [20:07:24] but, i don't think that wikimetrics should be configured to write into its deployed checkout [20:07:24] or rather [20:07:27] so [20:07:34] we could configure the $var_directory into srv somewhere [20:07:38] then why not parameterize *that* specifically? [20:07:41] but, i think its just ugly [20:07:41] where it writes to? [20:07:51] what should it be? [20:07:54] /srv/var/wikimetrics? [20:07:57] $public_directory? [20:08:01] $var_directory [20:08:01] what is it writing? [20:08:02] output? [20:08:10] yeah static job outputs, i think [20:08:39] yah, I do the same in quarry, parameterized as $output_directory or something [20:09:21] i guess, aaaaaa, we just went through all these hoops to try to make wikimetrics respect FHS, ja know? [20:09:22] :p [20:09:28] tch tch :P [20:09:45] /srv is for server specific files :P [20:09:48] these are server specific files [20:10:35] naw [20:10:39] /var [20:10:47] This hierarchy holds state information pertaining to an application or the system. State information is data that programs modify while they run, and that pertains to one specific host. Users must never need to modify files in/var/lib to configure a package's operation. [20:11:04] output files are idempotent, I think. [20:11:10] anyway :) [20:11:19] /var/lib/ is the location that must be used for all distribution packaging support. Different distributions may use different names, of course. [20:11:35] as long as the change doesn't fuck up space on other instances where /srv is currently enabled via the role, I'm ok :) [20:12:13] so, the change won't do anything [20:12:20] and actually, i'm not using that part of it anymore [20:12:32] because I didn't want to deal with trying to set a global variable inside of the wikimetrics role class [20:12:32] so [20:12:39] we coul dremove this [20:12:40] https://gerrit.wikimedia.org/r/#/c/160687/4/manifests/role/labssrv.pp [20:12:41] if you want [20:12:50] thought i'd leave it, since I already did it, it doesn't affect anything as is [20:13:05] and could be useful if someone wants to change the amount of space /srv takes up for in instance in the wikitech interface [20:13:09] but. [20:13:13] this is necessary [20:13:14] https://gerrit.wikimedia.org/r/#/c/160687/4/modules/labs_lvm/manifests/init.pp [20:13:32] ja, if you leave labssrv untouched and just do other things that'd be simples [20:13:32] t [20:13:50] k [20:33:30] (PS3) Milimetric: Match style from Pau's wireframes more closely [analytics/dashiki] - https://gerrit.wikimedia.org/r/160690 [20:44:02] (PS4) Milimetric: Match style from Pau's wireframes more closely [analytics/dashiki] - https://gerrit.wikimedia.org/r/160690 [21:19:10] (PS3) Milimetric: Match colors in graph with labels [analytics/dashiki] - https://gerrit.wikimedia.org/r/160532 [21:19:40] (CR) Milimetric: Match colors in graph with labels (2 comments) [analytics/dashiki] - https://gerrit.wikimedia.org/r/160532 (owner: Milimetric) [21:21:54] (PS5) Milimetric: Match style from Pau's wireframes more closely [analytics/dashiki] - https://gerrit.wikimedia.org/r/160690 [22:32:09] (CR) Nuria: [C: 2] Match style from Pau's wireframes more closely [analytics/dashiki] - https://gerrit.wikimedia.org/r/160690 (owner: Milimetric)