[05:51:33] Analytics, Reading-analysis, Research-and-Data, Research-consulting: Report on Wikimedia's industry ranking - https://phabricator.wikimedia.org/T141117#2501143 (leila) Proposal was reviewed by Dario, and now is with Toby and Wes for their feedback. [08:42:43] team I am going afk for an hour, ttl! [08:42:48] later AlexZ [08:42:56] elukey sorry [08:43:04] tab mistake (sorry AlexZ ) [09:30:21] (PS1) Addshore: dumpDownloads - use log dir from config [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301561 (https://phabricator.wikimedia.org/T119070) [09:31:55] (PS1) Addshore: Throw exceptions on non existant config keys [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301562 [09:32:24] (CR) Addshore: "Cherry picked to https://gerrit.wikimedia.org/r/#/c/301562/" [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301513 (owner: Addshore) [09:55:13] Analytics, Pageviews-API: Suddenly outrageous higher pageviews for main pages - https://phabricator.wikimedia.org/T141506#2501321 (Sjoerddebruin) [11:17:53] Analytics, Pageviews-API: Suddenly outrageous higher pageviews for main pages - https://phabricator.wikimedia.org/T141506#2501374 (Sjoerddebruin) [11:29:53] Hi mforns o/ [11:29:57] hi joal ! [11:30:05] how's it going? [11:30:16] Maaaan, crazy :) [11:30:32] I can't find the reason I'm observing what I have [11:30:42] joal, do you need a rubber duck? [11:31:00] I'm gonna loop over the problem, but can't find a reason/solution [11:31:10] mforns: if you have a minute, it'll be great to share :) [11:31:14] ok [11:31:17] batcave! [11:31:22] OMW ! [12:50:58] (PS1) Addshore: Improve output & speed [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301587 [12:51:22] (CR) jenkins-bot: [V: -1] Improve output & speed [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301587 (owner: Addshore) [12:53:00] (CR) Hashar: "recheck" [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301587 (owner: Addshore) [12:54:13] (PS2) Addshore: dumpDownloads - use log dir from config [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301512 (https://phabricator.wikimedia.org/T119070) [12:54:16] (CR) Hashar: "Not sure what is happening there but the merge fail:" [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301587 (owner: Addshore) [12:54:19] (PS2) Addshore: Throw exceptions on non existant config keys [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301513 [12:56:35] (PS2) Addshore: Improve output & speed [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301587 [12:57:04] (PS1) Addshore: Improve output & speed [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301588 [13:41:10] Analytics-Cluster: Install SparkR on the cluster - https://phabricator.wikimedia.org/T102485#1365857 (elukey) @Ottomata thoughts about this request? [13:44:24] joal: do we need to keep https://phabricator.wikimedia.org/T101792 open? [13:44:41] hi elukey :) [13:44:54] I don't think so elukey , seems done [13:45:12] Analytics-Cluster: Stop Cluster from $SOMETIMES not running oozie jobs - https://phabricator.wikimedia.org/T92819#2501781 (elukey) Open>Resolved a:elukey Very old task, resolving. [13:46:13] joal: all right, closing :) [13:46:17] I am doing some cleanups [13:46:18] :P [13:46:24] milimetric: https://phabricator.wikimedia.org/T101792 - can we close it? :) [13:46:33] thanks elukey :) [13:47:08] Analytics-Cluster, Analytics-Kanban, Datasets-General-or-Unknown: {slug} Pageview API Ops - https://phabricator.wikimedia.org/T101792#2501785 (elukey) Open>Resolved a:elukey PageViews API have been running for a while in prod, old task :) https://wikitech.wikimedia.org/wiki/Analytics/AQS [14:31:47] Hey elukey! [14:32:04] How can I give the 'analytics-wmde' user on stat1002 access to hive / hadoop? [14:32:52] woa I just woke up! [14:33:29] yeah elukey we can close old tasks like that. We said next time we all hang out we should groom the left column [14:33:33] dAN [14:33:35] I WOKE UP SO LATE TODAY [14:33:36] yes [14:33:37] so HOT [14:33:39] I know! [14:33:41] it sucks [14:34:29] i think today was the culmination of poor tossy turny heat sleep finally catching up and making me trrrd [14:34:59] yeah, must be [14:35:06] I'm so dizzy [14:35:06] addshore: mmmmm can you give us more info about the use case? [14:35:17] i feel a little out of it too, pretty weird [14:35:18] not sure if you have already discussed it with ottomata [14:35:21] ahh wait, I think I have found it! [14:35:27] good to have someone else corroborate this experience [14:35:53] addshore: i think you found it :) [14:35:56] The task is https://phabricator.wikimedia.org/T141525 (only just opened it) and I believe the user have to be added to the hieradata files for hadoop (ill make a patch) [14:36:06] ah [14:36:08] not quite [14:36:09] :) [14:36:13] its a system user [14:36:59] addshore: if we are going to do that, the group and user resources will need to be factored out [14:37:07] maybe added to role::analytics_cluster::users [14:37:16] you can include that classs in your statistics::wmde class i think [14:37:20] OR [14:38:05] you could factor them out in the stats module in their own class, and then include that class in the analytics_cluster::users [14:38:06] hmm [14:38:08] not sure which is better [14:38:37] all the yaml stuff is just for real users, not system users [14:38:53] ahh, so analytics-search-users is a real user? [14:39:14] oh wait, thats a group *ignore me* [14:40:00] so, I could add the wmde user to role::analytics_cluster::users as the search user is already there? [14:41:29] although option 2 is also nice? factor the wmd user out into a class, and then include that class in the main wmde bit and also in analytics_cluster::users? [14:47:24] hm, yeah. so ja probably a separeate class, statistics::wmde::user or something is a good idea [14:47:38] and hten include that in cluster::users [14:51:51] ottomata: something like this? https://gerrit.wikimedia.org/r/#/c/301610/ [15:03:45] addshore: hmm, right homedir is in working_path [15:03:45] hm [15:03:49] thinking [15:04:00] 1 comment is that statistics::wmde::user would be a better class [15:04:16] youd' have to make a new wmde dir and put user.pp in it: wmde/user.pp [15:04:35] addshore: i think you should parameterize homedir on the user class [15:04:36] so [15:05:04] statistics::wmde::user($home = '/srv/analytics-wmde') { ... [15:05:18] and then override it when including from the statistics::wmde class [15:05:25] to $statistics::working_path [15:06:30] that way you don't need to include other statistics class stuff from analytics_cluster::users [15:06:37] hmm, but wouldn't that be odd, as then both statistics::wmde_user and role::analytics_cluster::users could pass in different params? [15:07:03] ja but that's ok, since they can't do that on the same host, and, $statistics::working_path is variable on stat nodes anyway [15:07:06] e.g. on stat1002 it is /a [15:07:13] For renaming it to statistics::wmde::user I agree (I was just copying what was in the dir already) :) [15:07:16] aye [15:07:37] yeah, its a shame that we have to use $statistics::working_path at all :/ [15:10:34] addshore: also, add a comment in analytics_cluster::users as to what it is and why that class is being incldued there please [15:11:02] okay! [15:12:26] addshore: i think i'm missing one thing too, but it looks like it is missing for the analytics-search user too [15:12:41] puppet does create homedirs in hdfs for some users [15:12:47] i would have thought it was doing it for analytics-search too but [15:12:47] hm [15:12:51] i don't see that [15:12:54] ahh I know where that is :) [15:12:59] do I need to add this user there too? [15:13:02] where? [15:13:18] there is a script that does it for user groups [15:13:31] hieradata/eqiad/cdh/hadoop/users.yaml is where i found it [15:13:36] # Ensure that users in these posix groups have home directories in HDFS. [15:13:51] but that contains analytics-search-users not analytics-search [15:14:38] is a hdfs homedir needed to simply run select queries (no writing at all)? [15:15:04] not certain, it might be, i think you'd get at least warnings if one doesn't exist [15:16:00] hm, i suppose adding analytics-wmde to the users.yaml file would work.... [15:16:01] hmm but the analytics-search user is not in the analytics-search-users group. [15:16:07] right [15:16:14] i wonder if someone (me?) created it manually [15:16:18] the homdeir [15:16:21] :D [15:16:38] addshore: i think adding analytics-wmde (and analytics-search) to that list of groups would do it [15:16:40] analytics-search-users is in the .yaml files [15:16:45] since analytics-search user will be the only user in that group [15:16:52] and analytics-wmde will be the only user in analytics-wmde group [15:16:53] okay! [15:17:04] Shall I just do that in the same patch? [15:17:17] addshore: just do yours for now and we'll see, if it works then I'll add analytics-search in another patch [15:17:21] okay! [15:17:56] updated https://gerrit.wikimedia.org/r/#/c/301610 [15:20:02] addshore: looking good couple more comments [15:20:24] when you include a paramtered class [15:20:26] it looks like [15:20:49] class { 'class::name': [15:20:49] 'paramA' => 'valueA', [15:20:49] } [15:21:53] hm, i was going to make a comment about making the $user and $home vars consistent with statistics::user as $username and $homedir....buuuut i'm not actually sure which I like better [15:21:59] so i thikn i'll let you decide there :) [15:22:05] :D [15:24:48] ottomata: another new patch up :) [15:25:02] aha [15:25:16] addshore: class {'class::name': 'param' => 'value' } [15:25:34] note the quotes around the param name too [15:25:35] hmm, with the params in quotes? [15:25:38] yup [15:25:38] ooooh, okay [15:25:46] wait [15:25:48] no [15:25:48] i'm sorry [15:25:49] haha [15:25:51] dunno why i wrote that [15:25:56] :D [15:25:57] def not param names in quotes [15:26:00] haha [15:26:00] too hot? ;) [15:26:02] maybe so [15:26:16] values should be in quotes though [15:26:19] if you are making another patch [15:26:27] could you do it for the 'present' values in your new ::user class [15:26:27] ? [15:26:36] its not a big deal, but part of style guide [15:26:39] booleans not quoted [15:26:50] but all strings (even if special values like present) should be quoted [15:27:09] done! new patch up :) [15:28:35] Analytics, Reading-analysis, Research-and-Data, Research-consulting: Report on Wikimedia's industry ranking - https://phabricator.wikimedia.org/T141117#2502043 (Nuria) Please let us know when reviews have happened. [15:30:54] addshore: class docs for your user class? [15:31:10] {{doing}} [15:32:19] Done! [15:34:32] les do it [15:35:57] :D [15:36:27] ottomata: I also have https://gerrit.wikimedia.org/r/#/c/301511/ in puppet to be merged (I added it to puppet swat) but if you feel like giving that a +2 too then I'll take it out of swat! [15:38:13] can do [15:39:24] joal: I just forwarded to you a very interesting answer from Faidon about raid0 in the new AQS [15:39:26] running puppet [15:40:04] * joal reads [15:40:23] OH [15:40:29] addshore: i missed that you didn't change the location o the class [15:40:33] you need to put it in [15:40:38] statistics/manifests/wmde/user.pp [15:40:44] otherwise autoload won't find it [15:40:55] ahhhhh, okay! [15:40:57] wmde_user.pp no good :) [15:41:58] {{doing}} [15:43:39] epic! [15:43:50] (CR) Addshore: [C: 2] dumpDownloads - use log dir from config [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301512 (https://phabricator.wikimedia.org/T119070) (owner: Addshore) [15:43:53] (CR) Addshore: [C: 2] dumpDownloads - use log dir from config [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301561 (https://phabricator.wikimedia.org/T119070) (owner: Addshore) [15:43:56] (CR) Addshore: [C: 2] Throw exceptions on non existant config keys [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301513 (owner: Addshore) [15:43:59] (CR) Addshore: [C: 2] Throw exceptions on non existant config keys [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301562 (owner: Addshore) [15:44:02] (CR) Addshore: [C: 2] Improve output & speed [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301587 (owner: Addshore) [15:44:05] (CR) Addshore: [C: 2] Improve output & speed [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301588 (owner: Addshore) [15:44:08] (Merged) jenkins-bot: dumpDownloads - use log dir from config [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301512 (https://phabricator.wikimedia.org/T119070) (owner: Addshore) [15:44:11] (Merged) jenkins-bot: dumpDownloads - use log dir from config [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301561 (https://phabricator.wikimedia.org/T119070) (owner: Addshore) [15:44:14] (Merged) jenkins-bot: Throw exceptions on non existant config keys [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301513 (owner: Addshore) [15:44:17] (Merged) jenkins-bot: Improve output & speed [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301587 (owner: Addshore) [15:44:20] (Merged) jenkins-bot: Throw exceptions on non existant config keys [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301562 (owner: Addshore) [15:44:23] (Merged) jenkins-bot: Improve output & speed [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301588 (owner: Addshore) [15:45:25] haha [15:45:26] ADDSHORE [15:45:42] class { 'statistics::wmde::user': [15:45:42] homedir => ... [15:45:42] } [15:45:50] thought you did that, didn't check [15:45:59] so [15:46:00] ok [15:46:21] make that change and I will explain to you :) [15:46:44] addshore: ^ [15:46:53] joal: I think that we have very good things to evaluate for testing [15:47:04] like what happens if we turn down one/two instances [15:47:13] how it reflects on performances [15:47:14] etc.. [15:47:29] Analytics, Reading-analysis, Research-and-Data, Research-consulting: Report on Wikimedia's industry ranking - https://phabricator.wikimedia.org/T141117#2502104 (leila) yup, Nuria. :) [15:54:14] addshore: ah ok, so our guess about analytics-wmde in the users.yaml file doesn't work [15:54:19] it uses getent groups [15:54:29] which looks for supplementary group users, not main group [15:54:33] so nope, won't work [15:54:41] anyway, we'd have to specify the proper group perms [15:54:58] i'm going to make a comment in analytics_cluster::users that if someone adds a system user there, they have to manually create the hdfs homedir [15:55:06] okay! :D [15:55:46] hm, addshore, we should add analytics-wmde-users to that list though. [15:55:51] hmm, ok gonna make a patch to finalize all this [15:59:24] :D [16:01:09] ottomata, joal: standdup? [16:02:07] OO [16:02:07] sorry [16:13:25] Analytics-Kanban: Scale MySQL edit history reconstruction data extraction - https://phabricator.wikimedia.org/T134791#2502168 (Nuria) a:JAllemandou [16:25:35] addshore: hm [16:25:36] oh [16:25:43] you are trying to access webrequest data with this user? [16:25:46] oh. [16:27:18] yes! [16:27:27] didn't realize that. did we talk about this before/ [16:27:28] ? [16:29:05] gonna be a problem... [16:29:14] we don't haven't had a use case like this before [16:29:39] webrequest is 'private' data [16:29:48] which analytics-wmde-users does not have access to [16:29:55] nor does analytics-wmde system user [16:29:56] and [16:30:11] there is no good way to add a system user to a real user group [16:30:15] puppet doesn't like it [16:30:40] so, even if you got permission to have that group/user access private data, we don't currently have a way to implement adding system users to real user groups [16:31:07] Analytics: Compare Wikistats data with Hive data - https://phabricator.wikimedia.org/T141536#2502226 (ezachte) [16:31:13] addshore: maybe i'm remember incorrectly, but i thought when we originally talked that this user just needed to run scripts on stat1002 [16:31:21] not in hadoop... [16:31:39] yes, just run scripts, but one of the scripts runs a hive command and then does stuff with the result [16:31:51] This is something I will want to move to oozie sooner or later! [16:31:55] hive script != stat1002 :/ [16:32:09] hmm? [16:32:21] stat1002 is just a client node, when you run hive, its a hadoop job [16:32:29] which runs on Analytics cluster across lots of nodes [16:32:39] you are just using stat1002 to launch the job [16:32:50] which, on its own isn't a big deal [16:32:51] ahh yes, I guess! [16:32:59] analytics-search does like you are saying [16:33:01] but [16:33:11] analytics-search does not need access to analytics private data [16:33:16] like webrequest [16:33:20] that's an extra permission [16:33:24] aaaahhh [16:33:30] that we've never given to a system user before [16:33:38] Analytics: Compare early results of Wikistats 2.0 with Wikistats 1.0 - https://phabricator.wikimedia.org/T141536#2502241 (ezachte) [16:33:47] oooh, okay! Then i guess the easiest solution is to ooziefy the job? [16:33:53] won't matter [16:34:07] you can run this as your own user, if you are in analytics-privatedata-users grou [16:34:07] p [16:34:16] but you can't run it as analytics-wmde [16:34:20] yeh thats fine! if I ooziefy it it wont have anything to do with analytics-wmde! [16:34:45] Now that I know how oozie works I should be able to write a job to do the same thing, then remove the script from the analytics-wmde user etc. [16:34:49] well, it will if you run the oozie job as that user [16:34:57] its going to run in hadoop as somebody's user [16:35:13] ok, addshore can I revert these changes then? [16:35:19] analytics-wmde in analytics_cluster? [16:35:22] we can keep the ::user class [16:35:22] ottomata: yes! [16:35:24] that might be useful [16:35:30] but i'll remove it from analytics cluste rnodes and hdfs [16:35:42] yup! And I'll disable the script trying to do hive things [16:36:22] ok [16:38:09] Analytics: Compare early results of Wikistats 2.0 with Wikistats 1.0 - https://phabricator.wikimedia.org/T141536#2502256 (Nuria) [16:38:11] Analytics-Kanban: Wikistats 2.0. Edit Reports: Setting up a pipeline to source Historical Edit Data into hdfs {lama} - https://phabricator.wikimedia.org/T130256#2502255 (Nuria) [16:38:40] (PS1) Addshore: Stop running wikidata/specialEntityData.php in cron [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301637 (https://phabricator.wikimedia.org/T141525) [16:39:07] (PS1) Addshore: Stop running wikidata/specialEntityData.php in cron [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301638 (https://phabricator.wikimedia.org/T141525) [16:39:20] (CR) Addshore: [C: 2] Stop running wikidata/specialEntityData.php in cron [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301637 (https://phabricator.wikimedia.org/T141525) (owner: Addshore) [16:39:25] (CR) Addshore: [C: 2] Stop running wikidata/specialEntityData.php in cron [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301638 (https://phabricator.wikimedia.org/T141525) (owner: Addshore) [16:39:28] (Merged) jenkins-bot: Stop running wikidata/specialEntityData.php in cron [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301637 (https://phabricator.wikimedia.org/T141525) (owner: Addshore) [16:39:31] (Merged) jenkins-bot: Stop running wikidata/specialEntityData.php in cron [analytics/wmde/scripts] - https://gerrit.wikimedia.org/r/301638 (https://phabricator.wikimedia.org/T141525) (owner: Addshore) [16:43:27] Analytics, Pageviews-API: Suddenly outrageous higher pageviews for main pages - https://phabricator.wikimedia.org/T141506#2501321 (Nuria) Please quantify the question more? Do you suspect a bug on pageview counting? are there data discrepancies? Can you link actual examples.? [16:44:27] Analytics, Analytics-Wikistats: Design new UI for Wikistats 2.0 - https://phabricator.wikimedia.org/T140000#2502305 (Nuria) [16:44:29] Analytics-Kanban: Wikistats 2.0. Edit Reports: Setting up a pipeline to source Historical Edit Data into hdfs {lama} - https://phabricator.wikimedia.org/T130256#2502304 (Nuria) [16:48:36] Analytics-EventLogging, Analytics-Kanban, EventBus: Upgrade eventlogging kafka client used for producing - https://phabricator.wikimedia.org/T141285#2502316 (Ottomata) [16:48:48] Analytics-EventLogging, Analytics-Kanban, EventBus: Upgrade eventlogging kafka client used for producing - https://phabricator.wikimedia.org/T141285#2493093 (Ottomata) [16:48:50] Analytics-Kanban: EventBus Maintenace: Fork child processes before adding writers - https://phabricator.wikimedia.org/T141470#2502318 (Ottomata) [16:49:36] Analytics-Kanban, Patch-For-Review: Change or upgrade eventlogging kafka client used for consumption - https://phabricator.wikimedia.org/T133779#2502334 (Ottomata) [16:50:29] Analytics-EventLogging, Analytics-Kanban, EventBus: Change or upgrade eventlogging kafka client used for producing - https://phabricator.wikimedia.org/T141285#2493093 (Ottomata) [16:51:05] Analytics-Kanban: EventBus Maintenace: Fork child processes before adding writers - https://phabricator.wikimedia.org/T141470#2502356 (Ottomata) [16:54:14] Analytics-Kanban: User history: rewrite the user history script to use the new algorithm - https://phabricator.wikimedia.org/T141468#2502367 (Milimetric) [16:58:00] Analytics-Kanban, EventBus, Patch-For-Review: Upgrade kafka main clusters to 0.9 - https://phabricator.wikimedia.org/T138265#2502397 (Milimetric) [17:03:26] Analytics-Kanban: Browser dashboard blogpost - https://phabricator.wikimedia.org/T141267#2502421 (Milimetric) [17:03:28] Analytics-Kanban: Notify all schema owners that the auto-purging is about to start {tick} - https://phabricator.wikimedia.org/T135191#2502423 (Nuria) Open>Resolved [17:11:58] Analytics, Analytics-Cluster: Deploy hive-site.xml to HDFS separately from refinery - https://phabricator.wikimedia.org/T133208#2502437 (Milimetric) [17:12:13] Analytics, Analytics-Cluster: Deploy hive-site.xml to HDFS separately from refinery - https://phabricator.wikimedia.org/T133208#2225074 (Milimetric) p:Triage>Normal [17:14:46] Analytics-Kanban: Capacity projections of pageview API document on wikitech - https://phabricator.wikimedia.org/T138318#2502460 (Milimetric) [17:17:02] Analytics-Kanban: Eventbus POST event failures after kafka 0.9 upgrade - https://phabricator.wikimedia.org/T141336#2502496 (Milimetric) [17:17:04] Analytics-EventLogging, Analytics-Kanban, EventBus: Change or upgrade eventlogging kafka client used for producing - https://phabricator.wikimedia.org/T141285#2502494 (Milimetric) [17:18:33] Analytics, Analytics-Cluster, Analytics-Kanban, Patch-For-Review: https://yarn.wikimedia.org/cluster/scheduler should be behind ldap - https://phabricator.wikimedia.org/T116192#2502506 (elukey) [17:20:57] PROBLEM - Disk space on Hadoop worker on analytics1032 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [17:21:06] Analytics, Analytics-Cluster: Improve Hue user management - https://phabricator.wikimedia.org/T127850#2502514 (elukey) We already moved the user accounts to the Mysql meta instance a while back. [17:21:26] PROBLEM - Hadoop DataNode on analytics1032 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [17:21:46] PROBLEM - Hadoop NodeManager on analytics1032 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [17:22:36] PROBLEM - YARN NodeManager Node-State on analytics1032 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [17:24:44] Analytics, Analytics-Cluster: Improve Hue user management - https://phabricator.wikimedia.org/T127850#2502525 (Ottomata) We should still get Hue LDAP to work. [17:28:41] Analytics-Kanban: Research spike: load enwiki data into Druid to study lookup table performance - https://phabricator.wikimedia.org/T141472#2502546 (Milimetric) [17:33:58] Analytics-Cluster, Analytics-Kanban, Patch-For-Review: https://yarn.wikimedia.org/cluster/scheduler should be behind ldap - https://phabricator.wikimedia.org/T116192#2502569 (Milimetric) p:Low>Normal [17:34:16] Analytics-Kanban: Bot from an Azure cloud cluster is causing a false pageview spike (can we identify as bot?) - https://phabricator.wikimedia.org/T137454#2502572 (Milimetric) p:Triage>Low [17:34:24] Analytics-Kanban: Stop generating pagecounts-raw and pagecounts-all-sites - https://phabricator.wikimedia.org/T130656#2502578 (Milimetric) p:Triage>Normal [17:35:47] Analytics, Pageviews-API: Suddenly outrageous higher pageviews for main pages - https://phabricator.wikimedia.org/T141506#2501321 (MusikAnimal) I believe @Sjoerddebruin is looking for a general research investigation. Per [[ https://meta.wikimedia.org/wiki/Research:FAQ#Where_do_I_find_data_or_statistics_... [17:37:28] Analytics: Compare early results of Wikistats 2.0 with Wikistats 1.0 - https://phabricator.wikimedia.org/T141536#2502613 (Milimetric) p:Triage>Normal [17:37:32] !log powercycling analytics1032 [17:37:34] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master [17:40:26] Analytics-Kanban, Patch-For-Review: Extract edit oriented data from MySQL for simplewiki - https://phabricator.wikimedia.org/T134790#2502639 (JAllemandou) [17:40:28] Analytics-Kanban: Scale MySQL edit history reconstruction data extraction - https://phabricator.wikimedia.org/T134791#2502638 (JAllemandou) [17:41:28] Analytics-Kanban: User history: rewrite the user history script to use the new algorithm - https://phabricator.wikimedia.org/T141468#2502641 (JAllemandou) [17:41:30] Analytics-Kanban: Scale MySQL edit history reconstruction data extraction - https://phabricator.wikimedia.org/T134791#2277164 (JAllemandou) [17:41:59] Analytics-Kanban: Scale scala algorithms using graph partitioning - https://phabricator.wikimedia.org/T141548#2502642 (JAllemandou) [17:42:56] Analytics: Compare early results of Wikistats 2.0 with Wikistats 1.0 - https://phabricator.wikimedia.org/T141536#2502659 (Milimetric) Also not to forget, this comparison will be interesting too: https://stats.wikimedia.org/EN/TablesWikipediaSIMPLE.htm#editdistribution [17:43:08] Analytics-Kanban: Productionize edit history extraction for all wikis using Sqoop - https://phabricator.wikimedia.org/T141476#2502660 (JAllemandou) [17:43:46] Analytics, Pageviews-API: Suddenly outrageous higher pageviews for main pages - https://phabricator.wikimedia.org/T141506#2502663 (Nuria) Right, most likely cause are bots, as we do not identify those throughly. Pageviews tagged with "user" just mean that agent is not s self-reported bot. Removing analy... [17:44:07] Analytics, Pageviews-API: Suddenly outrageous higher pageviews for main pages - https://phabricator.wikimedia.org/T141506#2502664 (Nuria) Sorry, moving into radar for analytics. [17:45:09] ottomata: let's merge yesterday's change with child processes and writters right? [17:47:47] ottomata: going afk, do you need help with analytics1032? [17:47:59] elukey: i'm on it go ahead [17:48:06] thanks :) [17:48:11] looks like a disk failuer [17:48:14] The following VDs are missing: 04 [17:48:15] ahh ouch [17:48:47] a bit brutal, but hw failure is better than unknown software bug that triggers this [17:48:54] will see if i can get it booted or at least inspect. ja ya [17:49:02] super, thank you! [17:49:09] bye team! see you tomorrow! [17:49:58] bye! [17:52:20] ottomata: ah, sorry, let me know when you think you have time [17:58:08] Analytics-Cluster, Analytics-Kanban, Operations, ops-eqiad: analytics1032 disk failure - https://phabricator.wikimedia.org/T141550#2502716 (Ottomata) [17:59:17] Analytics-Cluster, Analytics-Kanban, Operations, ops-eqiad: analytics1032 disk failure - https://phabricator.wikimedia.org/T141550#2502732 (Ottomata) I've disconnected from the console, but I've left the box open to the PERC menu. [17:59:30] nuria_: wassuppp? [17:59:54] ottomata: should we merge yesterday's chnage about writters and child processes? [18:28:32] ja! [18:28:44] nuria_: sorry dunno how i missed that ping [18:28:48] i think we should [18:29:01] will deploy all that in beta [18:29:02] ottomata: np, I am watching ellery's presentation on metrics meeting [18:29:06] oh! [18:29:09] i will join [18:29:27] nuria_: link? [18:29:30] looking... [18:29:38] ottomata: https://www.youtube.com/watch?v=EbRLlhFS4f8 [18:29:46] danke [18:34:32] (PS1) Addshore: WIP Create WikidataSpecialEntityDataMetrics [analytics/refinery/source] - https://gerrit.wikimedia.org/r/301657 (https://phabricator.wikimedia.org/T141525) [18:37:09] (CR) jenkins-bot: [V: -1] WIP Create WikidataSpecialEntityDataMetrics [analytics/refinery/source] - https://gerrit.wikimedia.org/r/301657 (https://phabricator.wikimedia.org/T141525) (owner: Addshore) [18:41:47] (PS2) Addshore: WIP Create WikidataSpecialEntityDataMetrics [analytics/refinery/source] - https://gerrit.wikimedia.org/r/301657 (https://phabricator.wikimedia.org/T141525) [18:49:27] (PS1) Addshore: WIP Create wikidata/specialentitydata_metrics coordinator [analytics/refinery] - https://gerrit.wikimedia.org/r/301661 (https://phabricator.wikimedia.org/T141525) [18:50:15] bd808: I'm getting to understand this oozie stuff a fair bit now, might have a crack at the api actions -> graphite. Do you know the ticket for it? [18:52:19] addshore: T102079 ? or are you looking for something else? [18:52:20] T102079: Metrics about the use of the Wikimedia web APIs - https://phabricator.wikimedia.org/T102079 [18:52:58] bd808: that looks like the one! [18:53:37] I guess each of the points under "Metrics requested" should have a sub task [18:54:04] well... maybe? [18:54:21] T137321 handles most of them into hadoop [18:54:22] T137321: Run ETL for wmf_raw.ActionApi into wmf.action_* aggregate tables - https://phabricator.wikimedia.org/T137321 [18:54:42] I think "most requested actions" is the only metrics requested that overlaps with my 'stuff' however [18:55:18] ooh *reads that task* [19:02:32] ottomata: back. Should we merge change and I can look at the one with the schemas? [19:03:34] ja sure [19:03:39] that change is good to go i think [19:03:48] i'm learning more about how to handle some things with confluent kafka producer atm [19:03:52] might have to make another patch for that too [19:04:00] been testing failure scenarios [19:07:35] ottomata: ok, merging and looking into other patch [19:07:55] milimetric: is this one good to go? [19:08:13] which? [19:10:03] nuria_: which one? [19:11:17] milimetric: argh sorry: https://gerrit.wikimedia.org/r/#/c/299824/ [19:13:15] nuria_: package.json has a small indent problem, other than that it's good [19:13:55] milimetric:wait... [19:14:12] milimetric: no package.json at https://gerrit.wikimedia.org/r/#/c/299824/ [19:14:18] lol, sorry [19:14:18] haha [19:14:25] milimetric: ah ya jshint [19:14:26] it was the jshintrc [19:17:13] (PS4) Nuria: Normalize project parameter [analytics/aqs] - https://gerrit.wikimedia.org/r/299824 (https://phabricator.wikimedia.org/T136016) [19:17:36] milimetric: corrected [19:18:50] (CR) Milimetric: [C: 2 V: 2] Normalize project parameter [analytics/aqs] - https://gerrit.wikimedia.org/r/299824 (https://phabricator.wikimedia.org/T136016) (owner: Nuria) [19:19:10] sweet, let me know if/when you want to deploy nuria_ [19:19:29] hm, but maybe we should coordinate with our european friends in case they're doing something to the servers [19:20:24] milimetric: we can ask services [19:20:35] they might not know of aqs specific stuff [19:20:54] milimetric: ah like? [19:21:20] I donno, maybe they're monitoring the server compaction on one cluster vs the other and restarting would affect it? [19:21:27] (probably not because we're not messsing with cassandra) [19:21:39] milimetric: no, we are not [19:21:48] milimetric: the only thing would be the loading jobs [19:22:01] milimetric: the compaction is completely different as we chnaged the scheme [19:22:17] milimetric: i think in hue we can see if there are loading jobs no? [19:23:45] milimetric: i do not think there are any [19:23:56] milimetric: running right now: https://hue.wikimedia.org/oozie/list_oozie_workflows/ [19:24:00]