[08:30:14] !log Insert fake test data in aqs pagecounts endpoint to set monitoring back to non-alarm state [08:30:14] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [08:40:44] joal: thanks a lot :) [08:40:46] alarms cleared [08:40:58] elukey: I double checked before logging ;) [08:41:24] elukey: I'm sorry I completely forgot to remind nuria yesterday while telling her to truncate :/ [08:44:49] elukey: how is an1002 behaving? [08:45:26] oh, another quextion elukey, have you restarted daily-unique job that faioled?g [08:48:13] from what I can see an1002 seems good, but I need to check in depth (doing it in a bit) [08:48:17] didn't restart daily unique [08:48:39] weird elukey: we got an alarm email, but job is marked as successful in hue :/ [08:52:38] 10Analytics-EventLogging, 06Analytics-Kanban: Research Spike: Better support for Eventlogging data on hive - https://phabricator.wikimedia.org/T153328#3162778 (10JAllemandou) Awesome ! I'm in favor of creating a new task for prototyping. I'm also gently waving to @ottomata to provide help if needed :) [08:57:03] :/ [08:59:11] (03CR) 10Joal: [C: 031] "LGTM ! Please merge when needed." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/346802 (https://phabricator.wikimedia.org/T162157) (owner: 10Nuria) [09:24:24] restarted a couple of nodemanagers [09:24:35] to see if the heap size aligns to the avg [09:25:32] (03PS3) 10Joal: [WIP] Add wikidata json to parquet spark code [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/346726 [11:51:01] * elukey lunch! [13:04:21] 06Analytics-Kanban, 06Operations, 10netops, 15User-Elukey: Review ACLs for the Analytics VLAN - https://phabricator.wikimedia.org/T157435#3163723 (10elukey) [13:08:51] for some reason my network doesn't allow me to connect to gerrit [13:09:55] I'm going to have to send the two changesets tomorrow from a different place joal nuria [13:21:46] fdans: mmmm is ssh working fine for you? You should be able to post code reviews without any issues [13:24:25] elukey: just connected to stat1002 fine, it's gerrit that keeps timeouting [13:26:14] fdans: what is origin in .git/config of the repo? [13:27:29] elukey: ssh://fdans@gerrit.wikimedia.org:29418/analytics/refinery/source [13:27:57] I mean I had no problems with this yesterday, and now I'm in a different place so it must be the connection right? [13:30:21] weird since it should go through ssh [13:31:11] maybe some filtering for 29418 > [13:31:12] ? [13:31:15] mmm [13:31:58] fdans: can you try telnet gerrit.wikimedia.org 29418 ? [13:32:49] https://www.irccloud.com/pastebin/B1HCSQoJ/ [13:32:52] elukey: ^ [13:32:55] weird [13:33:16] mmmmm [13:33:21] so git review hangs? [13:33:34] or all the git commands? [13:36:00] elukey it was all the git commands, but it's working now since I telnet'd (??) [13:39:54] lol [13:41:32] now I just got conflicts! Thank you for your help elukey! [14:16:52] People I just restarted the namenode on analytics1002 with -Xms 4096m [14:16:57] (equal to xmx) [14:20:53] 10Analytics: Adding top counts for wiki projects (ex: WikiProject:Medicine) to pageview API - https://phabricator.wikimedia.org/T141010#3163969 (10Shizhao) 05Open>03Resolved a:03Shizhao see https://en.wikipedia.org/wiki/Wikipedia:WikiProject_Medicine/Popular_pages https://en.wikipedia.org/wiki/User:Communi... [14:35:39] 06Analytics-Kanban, 15User-Elukey: Apply Xms Java Heap settings to all the Hadoop daemons - https://phabricator.wikimedia.org/T159219#3164005 (10elukey) [14:37:01] 10Analytics-Cluster, 06Analytics-Kanban: Enable hyperthreading on analytics100[12] - https://phabricator.wikimedia.org/T159742#3164011 (10elukey) [14:37:05] 06Analytics-Kanban, 06Operations, 15User-Elukey: Reimage the Hadoop Cluster to Debian Jessie - https://phabricator.wikimedia.org/T160333#3164009 (10elukey) [14:40:31] 06Analytics-Kanban, 06Operations, 15User-Elukey: Reimage the Hadoop Cluster to Debian Jessie - https://phabricator.wikimedia.org/T160333#3164016 (10elukey) Status: * All worker nodes except analytics1030 (down for hw failures) have Debian Jessie * Some worker nodes needs to be rebooted to pick up the Linux... [14:45:44] 06Analytics-Kanban, 15User-Elukey: Apply Xms Java Heap settings to all the Hadoop daemons - https://phabricator.wikimedia.org/T159219#3164018 (10elukey) List of daemons that could benefit from Xms: * Hadoop HDFS datanode (workers) * Hadoop Yarn nodemanager (workers) * Hadoop Yarn Resource Manager (master node... [14:46:52] (03CR) 10Nuria: "Actually I need to discount meta before counts are aggreggated. Need to reload again." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/346802 (https://phabricator.wikimedia.org/T162157) (owner: 10Nuria) [14:56:11] elukey, joal: Hola! i need to reload cassandra again as i need to remove meta data BEFORE agreggation (duh!) this is an overwrite so no truncate needed like yesterday [14:56:40] sure! Jo loaded fake data so alarms are gree now [14:56:40] elukey, joal: see why here: https://analytics.wikimedia.org/dashboards/reportcard/#pagecounts-dec-2007-dec-2016 [14:56:43] *green [14:57:35] makes sense nuria, I have not thought about it :( [14:57:56] joal: ya me neither and i looked at data before i loaded , ains [15:00:55] ping milimetric joal [15:01:00] standdduppp [15:04:28] 10Analytics-Cluster, 06Analytics-Kanban, 06Operations, 15User-Elukey: Reimage the Hadoop Cluster to Debian Jessie - https://phabricator.wikimedia.org/T160333#3164054 (10Nuria) [15:31:52] 10Analytics, 10ChangeProp, 10EventBus, 06Revision-Scoring-As-A-Service, and 3 others: Create generalized "precache" endpoint for ORES - https://phabricator.wikimedia.org/T148714#3164091 (10Halfak) In T159615, @mobrovac asked about this. This change is now deployed. We should be ready to receive changepr... [15:43:49] 10Analytics, 10ChangeProp, 10EventBus, 06Revision-Scoring-As-A-Service, and 3 others: Create generalized "precache" endpoint for ORES - https://phabricator.wikimedia.org/T148714#3164103 (10mobrovac) Thanks @Halfak for the info! I've just taken a look at the PR and am wondering why you opted to consume the... [15:47:37] * milimetric lunching [15:48:54] 06Analytics-Kanban, 15User-Elukey: Apply Xms Java Heap settings to all the Hadoop daemons - https://phabricator.wikimedia.org/T159219#3164107 (10elukey) https://archive.cloudera.com/cdh5/cdh/5/hadoop/hadoop-project-dist/hadoop-common/ClusterSetup.html [16:01:47] milimetric: we cannot edit dashiki's config [16:01:51] milimetric: "You cannot edit this revision because its content model is Dashiki, which differs from the current content model of the page JsonConfig.Dashiki." [16:02:06] milimetric: sorry! did not see you were lunching [16:02:15] milimetric: we can talk about this when you rae back [16:02:19] milimetric: we can talk about this when you are back [16:03:15] 06Analytics-Kanban: cannot edit dashiki's production configuration - https://phabricator.wikimedia.org/T162465#3164133 (10Nuria) [16:03:27] 06Analytics-Kanban: cannot edit dashiki's production configuration - https://phabricator.wikimedia.org/T162465#3164145 (10Nuria) p:05Triage>03Unbreak! [16:29:05] (03PS4) 10Joal: [WIP] Add wikidata json to parquet spark code [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/346726 [16:30:43] nuria, yeah, it must be it got messed up with the last deploy, there's a Special page that helps to repair, I'll try it in a bit [16:30:57] 06Analytics-Kanban: cannot edit dashiki's production configuration - https://phabricator.wikimedia.org/T162465#3164173 (10Nuria) a:03Milimetric [16:31:23] milimetric: ok, created ticket and assigned it to you, i think it should be pretty high priority to fix [16:31:25] https://phabricator.wikimedia.org/T162465 [16:32:43] yep, first thing after I eat [16:32:43] 06Analytics-Kanban: cannot edit dashiki's production configuration - https://phabricator.wikimedia.org/T162465#3164133 (10demon) We fixed this I thought? [16:33:41] 06Analytics-Kanban: cannot edit dashiki's production configuration - https://phabricator.wikimedia.org/T162465#3164183 (10Milimetric) It must've happened again after the last bad deploy I made. I'm looking at how to fix it now. [16:38:41] 06Analytics-Kanban: cannot edit dashiki's production configuration - https://phabricator.wikimedia.org/T162465#3164185 (10Milimetric) Hm. I think it's more complicated because it seems the extension is mis-configured again. Normally when editing a page in the Config:Dashiki: namespace, you should see the speci... [16:38:55] nuria: ok, so this is involved, there's something strange going on [16:39:21] I have to take the time and test it properly on vagrant because we've already manually fixed these pages twice already [16:39:33] milimetric: on meeting , can talk in abit [16:39:49] ok, let me know what page you need to edit, and I'll delete/recreate for you so you can edit it. [17:01:53] milimetric: back [17:02:18] milimetric: boy, this json extension has eaten tons of your time [17:02:34] milimetric: I was trying to edit the reportcard: Config:Dashiki:ReportCard [17:03:40] logging off people! [17:03:50] the xms changes looks good for the moment [17:04:02] I'll keep the daemons monitored :) [17:04:05] byeeee o/ [17:04:10] have a good weeked [17:04:15] *weekend [17:04:24] fdans: have a nice holiday :) [17:04:40] graaaazie elukey !!! [17:05:10] 06Analytics-Kanban: Improve purging for analytics-slave data on Eventlogging - https://phabricator.wikimedia.org/T156933#3164214 (10Nuria) [17:05:42] 06Analytics-Kanban: Improve purging for analytics-slave data on Eventlogging - https://phabricator.wikimedia.org/T156933#2990326 (10Nuria) [17:05:55] 10Analytics: Improve purging for analytics-slave data on Eventlogging - https://phabricator.wikimedia.org/T156933#2990326 (10Nuria) [17:06:43] 10Analytics: Improve purging for analytics-slave data on Eventlogging - https://phabricator.wikimedia.org/T156933#2990326 (10Nuria) Let's re-task this in the light of @elukey being interested in doing these changes, we might be able to incorporate them to the upgraded replication script [17:07:38] milimetric: ping me when you are back [17:10:18] nuria: ok, so sadly I guess I can delete and re-create the reportcard, see if that works [17:10:30] milimetric: ok [17:11:51] nuria: ok, https://meta.wikimedia.org/wiki/Config:Dashiki:ReportCard [17:12:12] (you can edit now) [17:12:24] for now, until we figure out what the hell is going on, we have to do this delete/recreate nonsense [17:12:32] it's because nobody seems to know exactly how JsonConfig works [17:28:58] milimetric: ok, let's try to figure it out [17:32:00] milimetric: k, data looks sane now: https://analytics.wikimedia.org/dashboards/reportcard/#pagecounts-dec-2007-dec-2016 [17:32:09] milimetric: just have to see what is up with wikidata [17:32:28] cc joal [17:32:35] nuria: awesome [17:32:41] nuria: still missing wikidata ? [17:32:51] joal: will merge my latest change and document dataset [17:34:42] joal: wikidata working now too cc milimetric , ok, we are good i think will document and announce this today [17:35:03] cool [18:08:36] joal: for pageview api pagecounts (cc milimetric) i think it would make sense to document them here: https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pagecounts? or even under https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageviews?) [18:14:30] I think AQS/Pagecounts [18:15:42] nuria: I think mforns started a placeholder, and I moved it here: https://wikitech.wikimedia.org/wiki/Analytics/AQS/Legacy_Pageviews [18:15:50] Please feel free to move / update etc [18:16:46] joal: ok, let's keep the pageview api wiki for everything else, most users will search for that in wikitech and should be easily findable [18:17:36] nuria: Actually we also have AQS/Unique Devices [18:17:43] joal: k [18:17:49] And more datasets in AQS should mean mode entries :) [18:17:53] will put this one under aqs [18:18:18] nuria: if you create pagecounts, please delete Legacy Pageview :) [18:18:21] thanks ! [18:18:44] joal: will do [18:58:08] joal: ok, got ready page for 1st stab: https://wikitech.wikimedia.org/wiki/Analytics/AQS/Legacy_Pagecounts cc milimetric [18:58:32] 06Analytics-Kanban, 13Patch-For-Review: Add mobile-site to AQS legacy pagecounts metric - https://phabricator.wikimedia.org/T161494#3164524 (10Nuria) 05Open>03Resolved [18:59:21] 06Analytics-Kanban, 13Patch-For-Review: Pagecounts all sites data issues - https://phabricator.wikimedia.org/T162157#3164527 (10Nuria) [18:59:23] 06Analytics-Kanban, 13Patch-For-Review: Populate aqs with legacy page-counts - https://phabricator.wikimedia.org/T156388#3164526 (10Nuria) [19:01:22] 06Analytics-Kanban, 13Patch-For-Review: Move reportcard to dashiki and new datasources - https://phabricator.wikimedia.org/T130117#3164533 (10Nuria) Annoucement sent about migrated reportcard after ironing data issues: https://lists.wikimedia.org/pipermail/analytics/2017-April/005843.html [19:01:25] 06Analytics-Kanban, 13Patch-For-Review: Migrate the simplest limn dashboards to dashiki tabular {frog} - https://phabricator.wikimedia.org/T126358#3164537 (10Nuria) [19:01:28] 06Analytics-Kanban, 13Patch-For-Review: Move reportcard to dashiki and new datasources - https://phabricator.wikimedia.org/T130117#3164535 (10Nuria) 05Open>03Resolved [19:02:10] 06Analytics-Kanban: Document and publicize AQS legacy page counts endpoint - https://phabricator.wikimedia.org/T159959#3084517 (10Nuria) First stab at docs done here: https://wikitech.wikimedia.org/wiki/Analytics/AQS/Legacy_Pagecounts [19:02:28] good start nuria [19:02:51] well, more than a start really, I don't think I'd want any more as a consumer [19:10:28] Is there still a limitation on how far back pageview data is available in Druid/Pivot? (if yes, could that be documented in https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Druid please? ) [19:18:39] HaeB: it is documented per dataset: [19:18:57] https://usercontent.irccloud-cdn.com/file/xuIMGKl0/Screen%20Shot%202017-04-07%20at%2012.18.24%20PM.png [19:22:22] (03PS3) 10Nuria: Correcting loading of pagecounts into cassandra [analytics/refinery] - 10https://gerrit.wikimedia.org/r/346802 (https://phabricator.wikimedia.org/T162157) [19:23:42] (03PS4) 10Nuria: Correcting loading of pagecounts into cassandra [analytics/refinery] - 10https://gerrit.wikimedia.org/r/346802 (https://phabricator.wikimedia.org/T162157) [19:25:15] (03PS5) 10Nuria: Correcting loading of pagecounts into cassandra [analytics/refinery] - 10https://gerrit.wikimedia.org/r/346802 (https://phabricator.wikimedia.org/T162157) [19:26:22] (03CR) 10Nuria: "@joal: loaded this now, want to take a last look for sanity? i think data looks fine: https://analytics.wikimedia.org/dashboards/reportcar" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/346802 (https://phabricator.wikimedia.org/T162157) (owner: 10Nuria) [19:37:48] 10Analytics, 10Analytics-Dashiki: annotations should show on tab layout - https://phabricator.wikimedia.org/T162482#3164702 (10Nuria) [19:39:43] 10Analytics-Dashiki, 06Analytics-Kanban: Refactor aqs api and usage for simplicity - https://phabricator.wikimedia.org/T161933#3164715 (10Nuria) @fdans: Ping me when you get a changeset and I can continue next week, thank you. [21:40:34] nuria: thanks! I added it to that page. (For some reason I had been using Pageviews Hourly instead of Pageviews Daily, and was wondering why there was no data from before January...) [23:04:47] 10Analytics, 10Analytics-EventLogging, 06Editing-Analysis, 07Easy: Record an EventLogging event every time a new mainspace page is created - https://phabricator.wikimedia.org/T150369#3165411 (10kaldari) [23:06:47] 10Analytics, 10Analytics-EventLogging, 06Editing-Analysis, 07Easy: Record an EventLogging event every time a new mainspace page is created - https://phabricator.wikimedia.org/T150369#3165415 (10kaldari) [23:10:28] 10Analytics, 10Analytics-EventLogging, 06Editing-Analysis, 07Easy: Record an EventLogging event every time a new mainspace page is created - https://phabricator.wikimedia.org/T150369#3165418 (10kaldari) [23:12:30] 10Analytics, 10Analytics-EventLogging, 06Editing-Analysis, 10Wikimedia-Hackathon-2017, 07Easy: Record an EventLogging event every time a new mainspace page is created - https://phabricator.wikimedia.org/T150369#3165419 (10kaldari)