[00:19:19] pageviews API down? I'm getting `data.params should have required property 'title'` yet I am providing a title, according to the RESTBase documentation [00:28:43] Analytics-Kanban: Group flavours of desktop Windows together in OS reports - https://phabricator.wikimedia.org/T131125#2156719 (Jdforrester-WMF) [00:30:38] Analytics-Kanban: Provide browser reports charts as %age of all pageviews - https://phabricator.wikimedia.org/T131126#2156733 (Jdforrester-WMF) [00:32:04] Analytics-Kanban: Break down "Other" a little more? - https://phabricator.wikimedia.org/T131127#2156750 (Jdforrester-WMF) [00:32:32] Analytics-Kanban: Provide browser reports charts as stacked %age of all pageviews - https://phabricator.wikimedia.org/T131126#2156733 (Jdforrester-WMF) [00:33:51] Analytics-Kanban: Browser report has odd "Not named" labels - https://phabricator.wikimedia.org/T130415#2156766 (Jdforrester-WMF) p:Triage>Normal [03:03:22] Analytics-Kanban: Add (and default to) a breakdown in percentages also for the line chart. - https://phabricator.wikimedia.org/T130406#2156947 (Nuria) Per our conversation , report updater can calculate percentages. Absolute numbers are not very useful to follow trends. [03:04:15] Analytics-Kanban: Provide browser reports charts as stacked %age of all pageviews - https://phabricator.wikimedia.org/T131126#2156948 (Nuria) duplicate of this one, right? https://phabricator.wikimedia.org/T130406 [03:07:17] MusikAnimal: I'm not sure what's going on! I wanna point out though that our team is 3 people in Europe, 2 on US East coast, and 2 on west coast - so we work pretty early hours and are probably not around after 5 PST. Apologies if your questions don't get answered immediately due to this. For the future - I recommend pinging a-team here (which pings us all) [03:07:17] so we'll definitely atleast see it later or in the morning, or filing a phab ticket and pinging there. [03:07:44] madhuvishy: no worries! I got a hold of someone in -services [03:08:02] it was due to a bad deploy. They've patched it and redeployed the fix [03:08:03] aah! cool :) did you get the query to work? [03:08:09] yup! :) [03:08:13] oh really? interesting [03:08:26] do you know if there's a related phab ticket [03:09:27] I don't, but here's the commit to reverse the error: https://github.com/wikimedia/restbase/pull/576 [03:09:46] Pchelolo is who helped me with it [04:34:12] MusikAnimal: Thanks :) [06:40:13] "Revert as a hot-fix, a proper solution, monitoring, test coverage will be done later." --> we definitely need a phab task :) [06:40:18] o/ [07:54:01] Analytics-Cluster, Analytics-Kanban, Patch-For-Review: Add automatic failover to the Hadoop's name node - https://phabricator.wikimedia.org/T129838#2157236 (elukey) Open>Resolved Documentation updated (https://wikitech.wikimedia.org/wiki/Service_restarts and https://wikitech.wikimedia.org/wiki/A... [08:43:05] Hi elukey :) [08:43:08] Waup? [08:48:14] joal: o/ [08:48:39] checking varnishkafka's memory consumption :@ [08:48:47] wow, okayyy :) [08:49:04] Had a good Easter weekend ? [08:52:08] yesss and you? [08:53:02] Very much ! A bit too much of everything (food, wine, family), very good time :) [08:59:48] ah yes same thing for me :) [10:15:05] hi team :] [10:22:28] hello! [10:27:35] (PS1) DCausse: Bump CirrusSearchRequestSet hive schema to rev 121456865906 [analytics/refinery] - https://gerrit.wikimedia.org/r/280173 [10:54:35] (PS1) Joal: Remove all-months from parameter checking for top [analytics/aqs] - https://gerrit.wikimedia.org/r/280174 [10:58:35] (CR) Mobrovac: [C: 1] Remove all-months from parameter checking for top [analytics/aqs] - https://gerrit.wikimedia.org/r/280174 (owner: Joal) [10:59:25] Hi mforns :) [10:59:47] dcausse: Do you want me to merge you CR (I can't really judge the validity of an avro schema :) [11:01:10] hi joal ! [11:11:53] (CR) Mobrovac: [C: 1] "This can and must be deployed before https://github.com/wikimedia/restbase/pull/578 is merged and deployed." [analytics/aqs] - https://gerrit.wikimedia.org/r/277784 (https://phabricator.wikimedia.org/T129518) (owner: Joal) [11:12:34] Analytics-Kanban, Operations, Patch-For-Review: nf_conntrack warnings for kafka hosts - https://phabricator.wikimedia.org/T131028#2157352 (MoritzMuehlenhoff) Connection rates are back to normal, around 115k on all kafka1* hosts. [11:46:36] joal: hi! I've tested it with an atler table, merging this one should be harmless since it's just the create table script [11:46:55] so to answer your question: yes please :) [11:47:20] dcausse: yes, merge will not harm, but I feel better having the confirmation it has been tested :) [11:47:47] I've tested the alter table in my user db and I was able to read old data [11:47:58] awesome dcause [11:48:21] dcausse: If you give me the statement, I'll apply to the prod table when you want [11:48:53] (CR) Joal: [C: 2 V: 2] "Merging as per discussed with dcausse." [analytics/refinery] - https://gerrit.wikimedia.org/r/280173 (owner: DCausse) [11:49:45] joal: thanks, the alter script is mentionned in the commit msg, it should be safe to apply anytime but let's wait for ebernhardson since he did all the work [11:51:28] ok dcausse, makes sense :) [13:00:51] (PS1) Mforns: [WIP] Remove dynamic last data point [analytics/reportupdater] - https://gerrit.wikimedia.org/r/280201 (https://phabricator.wikimedia.org/T131049) [13:22:19] the alter should be good to go, we've tested adding new fields with defaults before and things seem to work [13:25:23] joal: ^ [13:27:51] !log Update CirrusSearchRequestSet schema in hive [13:27:53] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master [13:27:58] dcausse, ebernhardson --^ [13:28:43] thanks [13:28:46] np ebernhardson [13:28:54] joal: thanks! [13:29:00] :) [14:34:37] Analytics-Kanban, Operations, Patch-For-Review: nf_conntrack warnings for kafka hosts - https://phabricator.wikimedia.org/T131028#2157677 (elukey) ``` elukey@neodymium:~$ sudo salt kafka10*.eqiad.wmnet cmd.run 'sysctl net.netfilter.nf_conntrack_count' kafka1012.eqiad.wmnet: net.netfilter.nf_conntra... [14:36:26] Analytics-Kanban, Operations, Patch-For-Review: nf_conntrack warnings for kafka hosts - https://phabricator.wikimedia.org/T131028#2157678 (Ottomata) Yeah, if it went back down, then it isn't related to the ApiAction logging in MW. That is still happening. [14:37:27] ottomata: o/ [14:37:54] I am wondering if bursts in traffic requiring API logging can trigger a lot more TCP sockets to kafka hosts [14:39:23] hiii [14:41:45] heellooooooooo [14:47:06] could be, it seems unlikely, [14:47:06] hm [14:48:02] mforns: hey you wanna chat about the tabs layout before standup? [14:48:09] milimetric, sure! [14:48:17] batcave? [14:49:43] yea omw [15:00:58] elukey: the spike in conns was almost double, ja? [15:03:07] yep [15:05:41] (PS2) Milimetric: [WIP] OPTION TWO: Header summary of graphs [analytics/dashiki] - https://gerrit.wikimedia.org/r/279441 (https://phabricator.wikimedia.org/T130144) [15:25:14] Analytics, MediaWiki-extensions-WikimediaEvents, The-Wikipedia-Library, Wikimedia-General-or-Unknown, Patch-For-Review: Implement Schema:ExternalLinkChange - https://phabricator.wikimedia.org/T115119#2157787 (Krenair) I don't think we should attempt to separate the subdomain from domain in th... [15:29:59] "Get up, stand up, stand up for your right" [15:30:43] elukey: is feeling reggae-ish [15:32:33] nuria: standuppppp [15:32:57] Analytics-Kanban: Describe threat model for sanitized pageview data - https://phabricator.wikimedia.org/T131158#2157791 (Nuria) [15:34:43] Analytics-Kanban: Describe threat model for sanitized pageview data - https://phabricator.wikimedia.org/T131158#2157805 (Nuria) Once this is done we can call our sanitization work for the quarter done. [16:01:39] https://plus.google.com/hangouts/_/wikimedia.org/a-batcave2 [16:22:23] Analytics, MediaWiki-extensions-WikimediaEvents, The-Wikipedia-Library, Wikimedia-General-or-Unknown, Patch-For-Review: Implement Schema:ExternalLinkChange - https://phabricator.wikimedia.org/T115119#2157878 (Sadads) @Krenair we would be happy to merge those items within the schema, if that w... [16:40:00] Analytics-Tech-community-metrics, DevRel-March-2016: Make GrimoireLib display *one* consistent name for one user - https://phabricator.wikimedia.org/T118169#2157953 (Lcanasdiaz) >>! In T118169#2134347, @Lcanasdiaz wrote: > @Aklapper found a new bug related to this topic. I'm working on it. > >> Going to... [16:47:48] (PS1) Milimetric: Clean up the tabs layout [analytics/dashiki] - https://gerrit.wikimedia.org/r/280253 (https://phabricator.wikimedia.org/T130144) [16:48:34] (Abandoned) Milimetric: [WIP] OPTION TWO: Header summary of graphs [analytics/dashiki] - https://gerrit.wikimedia.org/r/279441 (https://phabricator.wikimedia.org/T130144) (owner: Milimetric) [16:48:41] (Abandoned) Milimetric: [WIP] OPTION ONE: Dropdowns under tab titles [analytics/dashiki] - https://gerrit.wikimedia.org/r/279389 (https://phabricator.wikimedia.org/T130144) (owner: Milimetric) [16:48:48] (Abandoned) Milimetric: Add a legend to each graph in the tabs layout [analytics/dashiki] - https://gerrit.wikimedia.org/r/276649 (https://phabricator.wikimedia.org/T129497) (owner: Milimetric) [16:49:53] Analytics: Make legends on graphs better and more generic - https://phabricator.wikimedia.org/T129497#2158005 (Milimetric) p:High>Normal [16:55:50] milimetric: can i keep on working on jdlrobson patch or will it interfere with your recent dashiki chnages? [16:55:52] *chnages [16:55:57] *changes, argh [17:02:21] milimetric: shall we go and merge the various AQS stuff I have, and plan for a deploy tomorrow? [17:21:34] (CR) Joal: Look at filters before looking at the Pageview tagging on x-analytics (2 comments) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/279447 (https://phabricator.wikimedia.org/T128612) (owner: Nuria) [17:23:30] nuria: seems like s1-analytics-store.eqiad.wmnet is still lagging for MobileWebLanguageSwitcher_15302503. max(timestamp) still says 20160322164539, which is about a week ago. should i file a task? [17:25:28] dr0ptp4kt: there is an e-mail on analytics@ from dba recently on this regard where he recomends to query this machine and he mentions to query using s2-s7, s1 is kaput [17:25:49] * where he recomends NOT to query this machine [17:26:07] dr0ptp4kt: please take a look [17:26:18] https://www.irccloud.com/pastebin/HCYmE4WJ/ [17:33:26] (CR) Nuria: Look at filters before looking at the Pageview tagging on x-analytics (2 comments) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/279447 (https://phabricator.wikimedia.org/T128612) (owner: Nuria) [17:38:12] HaeB: can you let me know how pageviews on the blogpost are being counted? [17:45:48] (CR) Joal: Look at filters before looking at the Pageview tagging on x-analytics (2 comments) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/279447 (https://phabricator.wikimedia.org/T128612) (owner: Nuria) [17:45:53] wikimedia/mediawiki-extensions-EventLogging#546 (wmf/1.27.0-wmf.19 - 9032465 : thcipriani): The build has errored. [17:45:53] Change view : https://github.com/wikimedia/mediawiki-extensions-EventLogging/commit/903246504c32 [17:45:53] Build details : https://travis-ci.org/wikimedia/mediawiki-extensions-EventLogging/builds/119312013 [17:51:51] Analytics-Tech-community-metrics, DevRel-March-2016, Regression: top-contributors.html only displays seven entries - https://phabricator.wikimedia.org/T129837#2158312 (Lcanasdiaz) >>! In T129837#2122498, @Qgil wrote: > I see 7. @aklapper appears in the short list, and I don't. Now I am jealous. :) Don... [18:01:05] (CR) Nuria: Look at filters before looking at the Pageview tagging on x-analytics (2 comments) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/279447 (https://phabricator.wikimedia.org/T128612) (owner: Nuria) [18:17:54] a-team logging off, talk with you tomorrow!! [18:18:24] see you 'morrow :) [18:22:07] latesr! [18:26:56] nuria: OH! [19:00:20] Analytics-Kanban, Wikipedia-iOS-App-Product-Backlog: "Top Read" request rutrns url encoded titles which can exceed url length limit - https://phabricator.wikimedia.org/T128120#2064486 (Milimetric) Josh did you mean to tag #Analytics on this to put it on our radar? It doesn't seem like we can help much wit... [19:04:11] joal / nuria: sorry I missed both your pings, I stepped out to move my car and then got caught up with some UI hacking [19:04:43] nuria: that patch should be pretty independent but if you want to make sure you can rebase it on my change [19:05:18] joal: I hope you're asleep by now but I'll merge the AQS stuff now and plan on deploying tomorrow, yes [19:06:15] (CR) Milimetric: [C: 2 V: 2] "ready to deploy" [analytics/aqs] - https://gerrit.wikimedia.org/r/277784 (https://phabricator.wikimedia.org/T129518) (owner: Joal) [19:06:47] milimetric: nv i need to do plot for blog now that I am done with text, will get back to patch eventually ... i hope [19:07:08] k, lemme know if you want to chat about it [19:13:11] (PS2) Milimetric: Remove all-months from parameter checking for top [analytics/aqs] - https://gerrit.wikimedia.org/r/280174 (owner: Joal) [19:14:24] (CR) Milimetric: [C: 2 V: 2] "(just rebased not self-merging)" [analytics/aqs] - https://gerrit.wikimedia.org/r/280174 (owner: Joal) [19:19:05] Thx for the merges and message milimetric :) [19:19:11] * joal is going back to bed [19:19:23] grr, sorry for the ping [19:19:31] :D [19:19:39] Analytics-EventLogging, MediaWiki-Vagrant: EventLogging vagrant role fails to provision - https://phabricator.wikimedia.org/T131085#2155589 (Ottomata) This sounds like a gerrit permission problem. I haven't been able to reproduce this, but I did just add a read permissions for Anonymous Users in the ger... [19:19:51] not to worry, I was away, just back to double check everything ok [19:20:36] Analytics-EventLogging, scap, Scap3 (Scap3-Adoption-Phase1): Move EventLogging service to scap3 - https://phabricator.wikimedia.org/T118772#2158644 (Ottomata) I think we'll probably need an eventlogging/deploy repository for this. [19:21:13] (PS9) Milimetric: Add cassandra unique devices load job [analytics/refinery] - https://gerrit.wikimedia.org/r/277507 (https://phabricator.wikimedia.org/T129519) (owner: Joal) [19:21:20] (CR) Milimetric: [C: 2 V: 2] Add cassandra unique devices load job [analytics/refinery] - https://gerrit.wikimedia.org/r/277507 (https://phabricator.wikimedia.org/T129519) (owner: Joal) [19:23:35] (PS4) Milimetric: Add labels to the hierarchy sunburst [analytics/dashiki] - https://gerrit.wikimedia.org/r/277733 (https://phabricator.wikimedia.org/T124296) (owner: Mforns) [19:23:44] (PS3) Milimetric: Highlight greatest leaf of hierarchy by default [analytics/dashiki] - https://gerrit.wikimedia.org/r/277746 (https://phabricator.wikimedia.org/T124296) (owner: Mforns) [19:25:22] milimetric: if you got a few, would appreciate a python pip tox brainbounce [19:25:41] yes, lemme merge a couple of things, no problem [19:26:05] (CR) Milimetric: [C: 2 V: 2] "this is great" [analytics/dashiki] - https://gerrit.wikimedia.org/r/277733 (https://phabricator.wikimedia.org/T124296) (owner: Mforns) [19:26:18] (CR) Milimetric: [C: 2 V: 2] Highlight greatest leaf of hierarchy by default [analytics/dashiki] - https://gerrit.wikimedia.org/r/277746 (https://phabricator.wikimedia.org/T124296) (owner: Mforns) [19:26:32] (PS2) Milimetric: Clean up the tabs layout [analytics/dashiki] - https://gerrit.wikimedia.org/r/280253 (https://phabricator.wikimedia.org/T130144) [19:26:45] k, making some tea... [19:27:15] nuria: before I forget, did you know about flex layout? http://www.w3schools.com/cssref/css3_pr_flex.asp [19:27:20] ottomata: ready whenever [19:27:33] milimetric: nooo [19:27:57] nuria: omg, it's like I woke up from a nightmare and I found out none of it was real [19:28:08] jaja [19:28:13] it helped me to fix the dashiki vertical sizing problem [19:28:23] I'll link to the file, one sec [19:28:56] nuria: line 43 on the right hand side: https://gerrit.wikimedia.org/r/#/c/280253/2/src/layouts/tabs/styles.css [19:29:06] flex: 1 is like magic [19:29:20] and it needs 25,26,27 too [19:29:55] nice [19:30:48] this can be applied on the other layout, which I will do ... while I wait for andrew to get tea :) [19:32:23] ok milimetric batcave! [20:31:03] https://meta.wikimedia.org/w/index.php?title=Research:Page_view&action=history - analytics folks, could you log in when making edits there? so that people on meta know that these changes are official, and so that one can follow up with whoever made them if necessary [20:35:20] milimetric: yt? [20:35:36] yeah, chatting with Andrew in the cave [20:35:42] HaeB: ya, sorry, that's me [20:36:20] milimetric: whn you have a min, please take a look at small vizualization on blogpost: https://docs.google.com/document/d/1uF7a3bBp6b3C-gRX697yw2ueLAN_Zl44ax1URvCKM6s/edit# [20:36:42] milimetric: can upload to commons if it looks good [20:36:54] k, will do [20:37:04] milimetric: tried to keep it SUPER simple [20:43:26] I was like AAAH, image disappeared :) [20:43:29] but you were just moving it [20:44:21] nuria: I like it, you don't have to tell all the secrets of the data in one chart, just one secret is enough to get people excited :) [20:46:49] milimetric: ok, will add more options for coms [20:47:03] milimetric: so they have a couple to choose from [22:27:49] Analytics-Kanban, Patch-For-Review: Browser visualization in dashiki, too many items below the fold - https://phabricator.wikimedia.org/T130144#2159299 (Aklapper) [23:59:48] (PS3) Nuria: Look at filters before looking at the Pageview tagging on x-analytics [analytics/refinery/source] - https://gerrit.wikimedia.org/r/279447 (https://phabricator.wikimedia.org/T128612)