[00:00:23] Pchelolo: yes, code is on a branch: https://gerrit.wikimedia.org/r/#/q/status:merged+project:analytics/aqs+branch:new-aqs-cluster [00:00:37] Pchelolo: let's touch base on this tomorrow [00:01:51] nuria_: kk, ping me if you have troubles.. We'd need to update AQS to the newest RB development state after that [00:02:33] Pchelolo: I am sure there is a way to solve docker issues on ubuntu, i have no doubt but the root issue is that deploying in this platform is too cumbersome [00:03:16] and error prone [00:03:42] Pchelolo: but I will talk to gwicke about it next week and hopefully state of affairs can improve next quarter [00:05:50] Pchelolo: super thanks for your help and prompt responses though, i just wish [00:06:12] deploying wouldn't need help on your end [00:06:18] nuria_: hehe, np. Ye, I guess we can improve something on that front [07:11:22] Analytics, Operations: Remove cronspam from stat1002 to root@ - https://phabricator.wikimedia.org/T145606#2635815 (elukey) [08:18:56] Quarry, Discovery, Labs-project-other, Wikidata, and 2 others: Setup sparqly service at https://sparqly.wmflabs.org/ (like Quarry) - https://phabricator.wikimedia.org/T104762#2635939 (Multichill) With the current SPARQL setup it's easy to share queries either by full url or by short url. I think... [08:30:19] Analytics, Operations: Remove cronspam from stat1002 to root@ - https://phabricator.wikimedia.org/T145606#2635986 (elukey) [09:37:19] * elukey afk for 30 mins [10:49:27] Quarry, Discovery, Labs-project-other, Wikidata, and 2 others: Setup sparqly service at https://sparqly.wmflabs.org/ (like Quarry) - https://phabricator.wikimedia.org/T104762#2636433 (Base) Do I get it right that now a query cannot be longer than URL length limit? How much exactly is that number... [10:55:59] Quarry, Discovery, Labs-project-other, Wikidata, and 2 others: Setup sparqly service at https://sparqly.wmflabs.org/ (like Quarry) - https://phabricator.wikimedia.org/T104762#1426314 (jcrespo) @Base, your questions are very interesting, and you seem to have really nice suggestions, but I would s... [11:12:29] joal: you around? [11:12:33] I am [11:12:39] aloha :) [11:12:51] hey elukey :) [11:13:08] elukey: What'sup? [11:13:12] yesterday I tried to build aqs-deploy on my mac but there seems to be a problem with the current service runner that we are using [11:13:24] hm [11:13:25] so I can't build the docker image [11:13:56] the services team suggested to upgrade but we have others changes in the pipeline [11:14:03] right [11:14:38] IIRC you have all the settings working fine on your dev desk right? Do you have some time to build/deploy with me the new branch to aqs100[456] ? [11:14:53] elukey: I can ! [11:14:58] thanksssssssss [11:15:04] should be new-aqs-cluster [11:15:21] elukey: only thing I'm sure of is using non-master branch from gerrit [11:15:30] elukey: first time for me ;) [11:15:46] elukey: hopefully I'll find my path :) [11:15:53] yes I think that few people did it :D [11:16:08] maybe we can discuss what can break? [11:16:27] not sure if scap tolerates multiple branches [11:17:08] elukey: I have no idea [11:18:01] elukey: I'm gonna go step by step: first, gather the patches on a specific branch for aqs src [11:19:10] elukey: done, easy enough [11:20:07] joal: the only thing that worries me are the repo-symlinks that scap creates [11:20:24] because I am not sure if it assumes a single branch or not [11:20:25] elukey: hm [11:20:57] elukey: even before scap, I'm wondering about how docker commits to aqs-deploy repo [11:21:29] ah you mean if you can specify the target branch [11:21:31] mmmmm [11:21:33] good point [11:22:07] * elukey invokes the master of services mobrovac (if he has time :) [11:23:36] elukey: tests passed for me on the dedicated branch (just another check) [11:24:04] elukey: Will try the local-deploy-repo-commit process [11:27:39] I didn't find anything to force server.js build to a specific branch, but it might be smart and figure out which one it is tracked by the current local one? [11:27:44] too much magic? [11:28:16] elukey: agreed, currently looking [11:30:45] elukey: from what I read, automagic deploy process only happens on master [11:31:39] elukey: https://github.com/wikimedia/service-runner/blob/master/lib/docker.js#L356 [11:31:47] error = EMAGIREQUESTED [11:32:13] ah yes seems clear from the code :D [11:32:35] joal: let's wait for Nuria then and decide what to do [11:32:35] elukey: I can trick it though: auto-deploy locally, pick the commit in a branch, revert on master [11:32:56] mmmmm [11:33:01] elukey: git magic against service-runner magic ;) [11:34:28] yeah [11:34:51] thanks for checking! We'll decide later on.. now I am going to see if I can do something with pivot [11:34:54] :D [11:35:11] elukey: no prob, let me know if you want me to take actions :) [11:36:02] sure :) [11:36:09] elukey: by the way, I think you know it, but new aqs is FULL of DATAZZZZZZ :D [11:36:30] WE HAVE THE DATAZ LOAD3D!!!111!!! [11:36:33] \o/ [11:36:36] great job :) [11:37:10] * joal thanks elukey for being a fantastic colleague :d [11:37:23] :D [11:37:30] big fingers ... [11:43:05] mforns: Hellooooooooo ! [11:43:19] * joal was waiting for mforns :) [11:43:21] joal, hi!! [11:44:09] mforns: I have interesting results on data vettingh [11:44:18] joal, oh! [11:44:21] batcave? [11:44:26] mforns: meaning: no magic :) [11:44:31] which is good [11:44:38] sure mforns, OMW [12:36:56] elukey: i'm here now, what's up? [12:39:57] aloha, so we were wondering if scap is able to deploy a branch that is not master.. we have two clusters up and running atm (new and old AQS) that need different settings. So since we need to deploy to the newer one we created a new branch in aqs-deploy and aqs repos [12:40:40] but after a quick check it seems that we'd need to hack a bit to make it work [12:41:02] (both from the service runner side and scap side) [12:41:33] so I wanted to know from you if you ever had to deploy different branches from tin -> different clusters [12:43:40] hm i think it should be doable with scap deploy --rev [12:43:59] --rev is supposed to take a ref, so it could be a branch in theory [12:44:09] i've never tried/needed to deploy a branch though [12:44:41] with regards to different clusters, you can use --environment which allows you to have different settings [12:44:59] there's a task pending to make it also override scap.cfg stuff [12:45:26] and for service runner, you can specify the remote branch to pull in when building the repo [12:45:37] lemme find the docs for this case [12:47:38] https://www.mediawiki.org/wiki/ServiceTemplateNode/Deployment#Local_git [12:49:00] ah nice I didn't find the git config deploy.remote deploy_repo_remote_name before [12:49:03] thanks :) [13:20:42] hey yall [13:22:05] o/ [13:22:27] Hi milimetric [13:22:39] joal: checking out edit data in druid [13:22:45] cool milimetric :) [13:22:47] so cool :) [13:23:07] milimetric: we have a better run now, will update druid soon (currently vetting) [13:23:51] that's great [13:24:15] joal: before you update, what's "revision text bytes dim"? [13:24:40] milimetric: the dimension - (allows to sum them when grouping by) [13:25:04] but that's a metric not a dimension [13:25:10] renaming is needed to also have it as a dimensions (meant measure in the previous line sorry) [13:25:35] joal, the data is in the table. [13:25:50] mforns: saw that, vetting: wayyyyyy better :) [13:25:58] mforns: About half the nulls gone [13:26:02] ok, hm, maybe we can do "text_bytes" and "text_bytes_diff" instead? [13:26:16] joal, cool! [13:26:25] milimetric: do you have 10/20 minutes today to chat about pivot deployment? (also if you feel better) [13:26:25] milimetric: I don't get it :) [13:26:33] elukey: yes [13:26:35] hi milimetric, are you feeling better? :] [13:26:37] joal: batcave? [13:26:42] sure, OMW [13:26:45] yeah, still a little weird, but better [13:26:52] milimetric, can I join you guys? [13:26:57] no [13:26:59] not allowed [13:27:09] o.O [13:27:15] :D [13:27:39] mforns: come on in! [13:27:44] * mforns is hesitant now [13:27:49] hehe [13:46:31] milimetric: (whenever you have time) - I am a bit confused now about what should be the entry point of the pivot app [13:46:52] I have bin/pivot for the dev case (that calls build/server.js IIRC) [14:02:06] hi elukey, now's good, https://hangouts.google.com/hangouts/_/wikimedia.org/a-batcave-2 [14:02:56] Analytics, MediaWiki-API, Reading-Infrastructure-Team: Add pageview stats to the action API - https://phabricator.wikimedia.org/T144865#2636753 (dr0ptp4kt) @tgr, yes, I would recommend getting the unique devices count with the 60 day approach mentioned in the description and email thread. Regarding... [14:04:29] ottomata: where's your pivot running, we need to steal the config [14:07:29] hiiii [14:07:44] um, i suppose /home/otto/imply-pivot on stat1002 [14:07:49] unless elukey has it elsewhere now [14:09:06] yep! [14:09:22] Hi elukey ! [14:09:29] elukey: I think we broke pivot :) [14:09:36] Do you mind restarting it? [14:10:20] ahahhah [14:10:20] sure [14:11:15] done :) [14:11:58] thx elukey :) [14:12:23] joal: I tried to dump the config and it lists dimensions and measures [14:12:32] so it is something that we should specify in the config file [14:12:35] when we change them [14:12:41] atm is does auto-discovery afaics [14:12:47] elukey: hm [14:14:56] mforns: event_user_text_latest is indeed buggy [14:15:06] joal, aha [14:15:28] mforns: or at least I think [14:15:37] joal, I agree [14:16:46] also mforns, milimetric : Unlinked revisions are not coming from archive (revision_deleted_timestamp is null) [14:16:57] ok [14:17:24] yo doods have you heard of this? [14:17:24] http://zeppelin.apache.org/ [14:17:25] cause i haven't [14:17:26] :) [14:17:49] ottomata: I told you about that one a year ago, in mexico I think ;) [14:18:10] oh hahah [14:18:13] then i have heard of it! [14:18:21] ottomata: But at that time we were going toward jupyter integration [14:18:32] ottomata: and I think it makes sense [14:18:51] aye k [14:19:18] i was just browsing aroudn waiting for jenkins, (well 'waiting for zookeeper'), and was looking at apache 'big data' projects [14:19:19] :) [14:20:02] ottomata: I mean, given the discussion we had yesterday, we definitely will think about this more soon (I hope !) [14:21:25] mforns: really weird things about user text [14:21:33] joal, aha [14:21:47] mforns: back to the cave again? [14:21:54] ooook :] [14:27:53] urandom: quick gulp / servicerunner question [14:28:05] so this repo elukey and I are trying to deploy needs to be built with gulp [14:28:26] I was thinking it makes sense to call gulp from servicerunner's build (when we do server.js build) [14:30:25] milimetric: you may be asking the wrong person [14:30:30] * urandom googles gulp [14:31:11] "automate and enhance your workflow" [14:31:12] nice. [14:31:23] milimetric: have you tried mobrovac ? [14:32:01] urandom: boo me, I'm just getting back from two days sick and my brain is not declogged, sorry [14:32:04] yes, mobrovac ^ [14:32:11] milimetric: i mean, i'm happy to have this bounced off me and to help you think it through, but i seem to be staring at a disadvantage [14:32:36] no dude, totally my fault :) sorry [14:32:44] k :) [14:33:45] service-runner build can help you on the last step, when all that is left to do is install the correct ndoe modules in the deploy repo [14:34:05] milimetric: what gulp stuff do you need to do with the repo? [14:35:12] mobrovac: just "gulp", their src is in typescript so they need to compile to js [14:35:44] mobrovac: I can do it as part of dev and push it to the repo, but was thinking the build step made sense [14:36:04] (PS8) Mforns: [WIP] Join and denormalize all histories into one [analytics/refinery/source] - https://gerrit.wikimedia.org/r/307903 (owner: Milimetric) [14:36:18] ah i see [14:40:41] urandom: do you want to deploy the puppet change for cassandra today ? :) [14:40:49] sure [14:41:01] ok maybe in ~1 hr? [14:41:04] sure [14:41:16] elukey: ping me when it's convenient [14:41:20] super [14:41:29] * elukey should have answered "sure" [14:41:32] :D [14:41:33] :) [14:42:02] mobrovac: so, any ideas? [14:43:00] milimetric: access to the repo you are trying to build would greatly help [14:43:01] :D [14:43:52] mobrovac: heh, we're working off my fork until they figure out their issues: https://github.com/milimetric/pivot [14:44:12] kk, will take a look and let you know today a bit later [14:44:18] thx mobrovac [14:47:59] joal: if you want to see the pivot config that is autogenerated you can run /home/otto/imply-pivot/bin/pivot --druid druid1001.eqiad.wmnet:8082 --print-config --with-comments on stat1002 [14:50:30] k elukey [14:50:52] so we could do two things: [14:51:15] 1) use the --config option and change it everytime we add dimensions/measures/etc.. [14:51:40] 2) rely completely on auto-discovery but each time something changes we'd need to restart pivot [14:52:08] (with 1) we'll have a related systemd restart pivot) [14:52:16] elukey: if we need to change the config manully, wouldn't we need to restart pivot as well? [14:52:24] elukey: Ok ! [14:53:07] elukey: I guess having a puppet managed config file is probably better, particularly if puppet understands it needs to restart the thing after a change [15:03:01] a-team: standup [15:03:11] EEK [15:04:10] Analytics-Kanban, Continuous-Integration-Infrastructure, Differential, EventBus, Wikimedia-Stream: Run Kasocki tests in Jenkins via Differential commits - https://phabricator.wikimedia.org/T145140#2636952 (Ottomata) [15:18:09] joal: btw you're technically going to scrum of scrums today, but I can cover for you if you want [15:18:24] we never talked about that, but it's pretty late for europe [15:18:43] milimetric: I planned to go there, it'll be good for me to do it :) [15:18:50] ok, good [15:19:04] thanks for offering milimetric :) [15:26:24] elukey: quick confirmation: Are going to sevilla from madrid, or direct with plane? [15:27:31] joal: I still need to finalize my trip but on saturday and Thursday there is a very cheap ryanair direct flight [15:27:55] so it would be super great for me [15:28:32] elukey: sounds great :) [15:28:45] elukey: finalizing train, so just wsant to be sure :) [15:29:15] :) [15:29:50] (CR) Ottomata: "Hm, um, it is POC, there is not project associated with this. I was just parking in gerrit. Should I not?" [analytics/refinery/source] - https://gerrit.wikimedia.org/r/225485 (owner: Ottomata) [15:32:19] elukey: your ops summary was great, gj [15:33:08] Analytics, MediaWiki-API, Reading-Infrastructure-Team: Add pageview stats to the action API - https://phabricator.wikimedia.org/T144865#2612941 (Antigng) The patch works for me. [15:33:09] milimetric: thanks! it usually takes me a while to get things but eventually it works :D [15:40:04] a-team: puppet is failing on all the analytics hosts due to a switch of the puppet master (palladium -> puppetmaster1001) [15:40:17] firewall rules need to be upgraded :) [15:40:52] ooh [15:46:07] oh the vlan acl rules? [15:46:08] haha [15:47:25] yes :D [15:47:53] joal, milimetric, the data created with the split-by-month optimization is in denormalized2 [15:48:03] mforns: great ! [15:48:05] it took around 20 mins [15:48:06] mforns: faster? [15:48:08] right [15:48:15] half time [15:48:24] but haven't checked if it's correct [15:48:38] looking for ways to easily check if 2 tables are identical in hive [15:57:37] joal: isn't aqs loading in oozie? I was looking around https://github.com/wikimedia/analytics-refinery/tree/master/oozie [15:58:00] milimetric: it is! But it's named cassandra [15:58:09] aha [15:58:19] sorry, thx [15:58:29] np milimetric, aqs would have been better [15:58:53] there was a question on phab about if it's possible for us to emit an event when cassandra loading is done for a specific endpoint [15:59:00] and I thought yes but wanted to double check [15:59:02] looks like yes [15:59:43] milimetric: I think oozie can do that, probably depends on what event means [16:00:00] event bus event, kafka [16:00:13] just wanted to make sure we have a workflow step that would know that condition is met, joal [16:00:17] looks like we do here: https://github.com/wikimedia/analytics-refinery/blob/master/oozie/cassandra/daily/workflow.xml#L395 [16:01:04] milimetric: correct, if the loading succeeds, temp data is deleted (that means the thing has worked) [16:01:58] joal: in case you want to follow in phab: https://phabricator.wikimedia.org/T143912#2618239 [16:03:01] thanks milimetric [16:03:58] taking a break a-team, will be back before SoS [16:04:05] ok joal :] [16:07:36] !log cassandra on aqs100[123] restarted for T130861 [16:07:37] T130861: Investigate and implement possible simplification of Cassandra Logstash filtering - https://phabricator.wikimedia.org/T130861 [16:07:38] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master [16:07:43] joal: --^ [16:16:11] milimetric, do you have 5 mins to look at a query? [16:48:32] * elukey goes afk! [18:02:47] Analytics-Cluster, Operations: decom titanium - https://phabricator.wikimedia.org/T145666#2637714 (Dzahn) [18:05:13] Analytics-Cluster, Operations, Patch-For-Review: decom titanium - https://phabricator.wikimedia.org/T145666#2637734 (Dzahn) [18:12:25] Analytics-Cluster, Operations, Patch-For-Review: Migrate titanium to jessie (archiva.wikimedia.org upgrade) - https://phabricator.wikimedia.org/T123725#2637765 (Dzahn) Open>Resolved titanium has been replaced by meitnerium. This is done. The remaining decom steps (up to physically removing it... [18:12:51] Analytics-Cluster, Operations: Migrate titanium to jessie (archiva.wikimedia.org upgrade) - https://phabricator.wikimedia.org/T123725#2637771 (Dzahn) [18:13:18] mforns: sure, cave? [18:14:18] Analytics-Cluster, Operations, Patch-For-Review: decom titanium - https://phabricator.wikimedia.org/T145666#2637783 (Dzahn) a:Dzahn>None removing from pupet, preparing the decom, will wait another couple days or so before physical shutdown and removal from DNS [18:14:27] Analytics-Cluster, Operations, Patch-For-Review: decom titanium - https://phabricator.wikimedia.org/T145666#2637785 (Dzahn) a:Dzahn [18:26:23] milimetric, hi [18:26:35] cave now? [18:26:49] omw [19:05:55] Analytics-Tech-community-metrics, Possible-Tech-Projects, Epic: Allow contributors to update their own details in tech metrics directly - https://phabricator.wikimedia.org/T60585#2637927 (Sumit) This task featured in GSoC 15', but does it need further work? whats the current progress? Is this still a... [20:06:16] Analytics-Tech-community-metrics, Developer-Relations (Jul-Sep-2016): Allow AKlapper to access https://wikimedia.biterg.io/edit/ - https://phabricator.wikimedia.org/T144704#2638147 (Aklapper) [20:15:17] (PS2) Ottomata: [WIP] POC for Realtime Trending Pageviews [analytics/refinery/source] - https://gerrit.wikimedia.org/r/225485 [20:15:40] (CR) Aklapper: "Setting "[WIP]" prefix per https://www.mediawiki.org/wiki/Gerrit/Code_review/Getting_reviews#Test_your_changes" [analytics/refinery/source] - https://gerrit.wikimedia.org/r/225485 (owner: Ottomata) [20:18:55] mforns: btw, this probably doesn't affect you but I'm querying the tables I have in testing/mediawiki/tables right now [20:19:13] milimetric, ok, no I don't think there's any problem [20:19:17] (just in case you saw weird performance) [20:19:42] no, no, I'm also wrapping up for today [20:20:05] ok, cool, have a good night [20:27:35] (CR) Yurik: [C: 2 V: 2] Ensure all metrics are logged with the same time [analytics/discovery-stats] - https://gerrit.wikimedia.org/r/308816 (https://phabricator.wikimedia.org/T144652) (owner: MaxSem) [20:28:40] bye team! [23:12:39] (PS1) MaxSem: Ensure all metrics are logged with the same time [analytics/discovery-stats] - https://gerrit.wikimedia.org/r/310691 (https://phabricator.wikimedia.org/T144652) [23:12:50] (CR) MaxSem: [C: 2 V: 2] Ensure all metrics are logged with the same time [analytics/discovery-stats] - https://gerrit.wikimedia.org/r/310691 (https://phabricator.wikimedia.org/T144652) (owner: MaxSem)