[01:24:38] Analytics, Editing-Analysis, Notifications, Collab-Team-2016-Apr-Jun-Q4: Numerous Notification Tracking Graphs Stopped Working at End of 2015 - https://phabricator.wikimedia.org/T132116#2321766 (Neil_P._Quinn_WMF) @Nuria, I have tweaked the SQL in that repo and it does use reportupdater, but it o... [03:45:46] Analytics-Kanban, Datasets-General-or-Unknown, WMDE-Analytics-Engineering: Fix permissions on dumps.wm.o access logs synced to stats1002 - https://phabricator.wikimedia.org/T134776#2321868 (Nuria) Open>Resolved [07:34:49] joal: ping me when you're online and have time to debug the failing queries? [07:35:27] the fix for the first aqs firewall problem is merged since yesterday [08:01:38] moritzm: o/ [08:55:07] Analytics-Cluster, Analytics-Kanban, Patch-For-Review: Configure Spark YARN Dynamic Resource Allocation - https://phabricator.wikimedia.org/T101343#2322119 (elukey) >>! In T101343#2320978, @Ottomata wrote: > Hmmm, > > Just noticed 2 things. [[ http://spark.apache.org/docs/1.5.0/configuration.html#d... [10:49:15] * elukey lunch! [11:59:02] Analytics-Kanban, RESTBase, Services, RESTBase-API, User-mobrovac: Enable rate limiting on pageview api - https://phabricator.wikimedia.org/T135240#2322536 (mobrovac) a:Nuria>mobrovac >>! In T135240#2319092, @mobrovac wrote: > [Gerrit 290264](https://gerrit.wikimedia.org/r/#/c/290264/... [12:09:50] Analytics, Analytics-Wikistats, Internet-Archive: Total page view numbers on Wikistats do not match new page view definition - https://phabricator.wikimedia.org/T126579#2322567 (ezachte) After adding Wp zero earlier, this week I added two more categories that were missing: mobile traffic to other pro... [12:28:28] joal: you there? [12:28:42] Analytics-Wikistats: Unexpected increase in traffic for 4 languages in same region, on smaller projects - https://phabricator.wikimedia.org/T136084#2322595 (ezachte) [12:46:41] tried to execute joal's oozie load job to AQS test but [12:46:42] Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: aqs1004-a.eqiad.wmnet/10.64.0.126:9042 (com.datastax.driver.core.TransportException: [aqs1004-a.eqiad.wmnet/10.64.0.126:9042] Cannot connect)) [12:46:46] same error [12:49:59] !log stopping kafka on kafka1013 and rebooting the host for kernel upgrade [12:50:00] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master [12:50:47] hi team! [12:50:55] Hi elukey ! [12:51:35] elukey: I saw you tried my query (it sends me an email ;), great [12:52:02] ahhhh okok :D [12:52:05] hellooo joal [12:52:11] elukey, moritzm: I'm sorry to ask you some more help because of that failure [12:52:16] Hi elukey :) [12:52:31] I backloged a bit IRC, saw you had trouble with kafka a bit? [12:52:51] yeah all solved for the moment, the partitions were filling up [12:52:58] due to the migration [12:53:06] elukey: how come? [12:53:09] retention changed? [12:53:17] https://grafana.wikimedia.org/dashboard/db/kafka?panelId=35&fullscreen [12:53:48] so logsizes were increasing right after the migration, then got purged after normal retention [12:54:16] we checked on the host and there were more logs for the days of the switch [12:55:23] Yeah, I can imagine: one week of data copied on a single day (because of real retention, and new version integration) --> Leads to two actual weeks of data at the end of the first week [12:55:30] PROBLEM - Check status of defined EventLogging jobs on eventlog1001 is CRITICAL: CRITICAL: Stopped EventLogging jobs: processor/client-side-11 processor/client-side-07 processor/client-side-04 processor/client-side-03 processor/client-side-00 [12:55:51] hello EL! [12:55:54] let me restart you [12:56:10] elukey: EL problems due to kafka? [12:56:16] yep [12:56:37] !log EL restarted after kafka1013 node stop (kernel upgrades) [12:56:39] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master [12:56:53] k makes sense elukey ;0 [12:57:32] RECOVERY - Check status of defined EventLogging jobs on eventlog1001 is OK: OK: All defined EventLogging jobs are runnning. [12:57:56] yeah we need to fix that problem :( [12:58:35] elukey: have you tried spark dynamic allocation? [12:58:51] elukey: saw you merged and restarted, so I want to SEE IT ! [12:58:54] joal: nope! I was waiting you or ottomata, but it works :) [12:59:02] ergh it is set correctly [12:59:03] YAYYYY [13:08:23] joal: I mistakenly set max executors in the spark defaults initially but removed it this morning [13:08:44] elukey: meaning, maxExecutirs back to infinity? [13:08:53] yep [13:09:07] ottomata suggested not to cap them for the moment [13:09:19] k, thanks for having let me know, I would have tried to see if the restriction was working, and would have killed the lcuster ;) [13:09:22] and if we want to do it only with proper variable substitution [13:12:13] (CR) Mforns: [C: -1] "Great idea, I should have done this instead of the displayName. I -1'd it only because there's a missing semicolon." (2 comments) [analytics/dashiki] - https://gerrit.wikimedia.org/r/290306 (https://phabricator.wikimedia.org/T122533) (owner: Nuria) [13:16:14] mforns / joal: yall wanna meet about edit data since we missed yesterday? [13:16:24] milimetric: sure ! [13:16:41] we could chat a bit now and then again later if nuria and ottomata wanna join [13:16:46] I'll be in the cave [13:17:03] works for me milimetric [13:19:11] mforns, milimetric batcave now is it ? [13:19:36] I'm in the cave but we could wait for mforns instead [13:24:45] for some reason grafana is acting weird joal https://grafana.wikimedia.org/dashboard/db/kafka [13:24:54] messages in shows a hole [13:25:14] but if you put your mouse on it then it shows correct values [13:25:24] this is one of the "mmmmmmm" situations [13:26:23] grafana looks like a war zone and it was only a reboot [13:27:17] milimetric: o/ [13:27:54] hi elukey [13:29:23] milimetric, joal, I'm coming, give me 5 minutes please [13:29:33] no rush mforns we have all morning [13:30:12] elukey: dynamic allocation tested with special parameters in spark shell command [13:30:19] elukey: Works awesome :) [13:30:20] :D [13:31:19] wooooooooo [13:31:48] joal: what did you set specifically? [13:32:01] elukey: one caveat found (in spark config, need to be carefull on how executors are released), but that real great [13:32:20] elukey: --conf spark.shuffle.service.enabled=true --conf spark.dynamicAllocation.enabled=true [13:32:48] joal: spark.dynamicAllocation.enabled should be set to true by CDH theoretically, does it work even without it? [13:32:54] more lightweight CSS that we could use for some of our layouts maybe: https://github.com/picturepan2/spectre [13:33:05] elukey: not set by default for us [13:33:33] elukey: When not setting it, we get back to default number of executors (2) [13:34:04] this is weird, it should have been enabled :/ [13:35:42] milimetric: I also confirm that, in addition to being a better yarn citizen, spark is real faster in dynamic allocation mode (it can actually use more of the unused resources) [13:36:06] sweet [13:43:32] elukey: Have you tried discussing with moritzm about the new-aqs loading issue? [13:44:01] joal, milimetric, do you want to batcave now? [13:44:09] sure [13:44:12] omw [13:44:23] elukey: to me dynamic allocation is working !!! [13:45:08] elukey: we need some doc about how to manage memory and caching though, cause there is a strong link between cached data and number of live executor that will live for inifinity [13:45:18] joal: not yet because we were upgrading kafka1013 with the 4.4 kernel, but I tried to re-run the job and found the same issue.. even after the firewall changes [13:45:58] it took me a while because I didn't know where to get the info, and eventually I found the map logs :) [13:46:55] joal: yeah I added 10 as max executors to have a standard config so we might think to enforce a limit rather than relying on people :) [13:47:29] we can talk about it during the ops sync maybe? [13:52:23] elukey, joal: we can look into it now, on which host is the query failing? [13:52:37] moritzm: I was about to write :) [13:52:53] moritzm: I have not launched it yet, and we still have the issue of not knowing where it's executed :( [13:52:56] so I tried to run telnet aqs1004-a.eqiad.wmnet 9042 on analytics1052 (last one to fail) and aqs1005.eqiad.wmnet [13:53:17] on analytics1052 I see it hanging [13:53:27] meanwhile on aqs1005.eqiad.wmnet works fine (I get a session) [13:54:05] joal: there is some info the the mappers about the host that emits errors no? [13:54:19] Yes, we acn know it after, but not before :) [13:54:28] ah yeah okok :) [13:56:06] so I'd say I'll simply add logging rules to all analytics105* nodes and then a few queries should catch it, right? [13:56:29] moritzm: hm, I hope so ;) [13:56:43] if not at first try, at least at second [13:56:48] moritzm: --^ [13:57:29] k, let me make tea and then I'll add the rules [13:57:35] kk [14:05:04] k, feel free to go ahead with one or a few queries now [14:12:22] joal: started 0011493-160519124420827-oozie-oozi-C [14:12:28] (from stat1004) [14:12:40] moritzm: the job is running, should fail in a bit [14:16:44] moritzm: analytics1053.eqiad.wmnet failed [14:17:25] let me have a look [14:17:42] mmm no sorry I might be confused, lemme check better [14:18:11] ack, nothing dropped on 1053 [14:18:47] nothing on analytics105* in fact [14:19:20] so analytics1056.eqiad.wmnet is the one [14:19:37] can see in the logs Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: aqs1004-a.eqiad.wmnet/10.64.0.126:9042 (com.datastax.driver.core.TransportException: [aqs1004-a.eqiad.wmnet/10.64.0.126:9042] Cannot connect)) [14:21:31] moritzm: --^ [14:21:45] ok, so it's not a matter of the hadoop node not accepting the traffic (nothing is logged there), I added logging code to aqs1004, can you repeat the query? [14:21:54] sure! [14:22:08] 0011509-160519124420827-oozie-oozi-C started [14:26:54] did you get an error so far? [14:27:55] yep just checked [14:28:21] analytics1028,1033,1055,1052 failed in reduce step [14:30:14] then it's not something failing on the iptables level, there's no traffic dropped by iptables on either analytics1052 or aqs1004 [14:32:22] moritzm: so it is probably happening on application level [14:32:37] but it looks weird [14:32:52] I can telnet from analytics to aqs, but I can from aqs to aqs [14:32:53] mmmm [14:33:11] there's a "n't" missing somewhere above? [14:33:34] yes there is sorry, long day :) [14:33:58] so from where does telnet work and from where doesn't? [14:34:17] I can't telnet from analytics to aqs but I can from aqs to aqs (port 9042) [14:34:28] last one specifically aqs1005 to aqs1004-a [14:34:50] maybe I can retry from analytics1052 [14:35:01] where did you connect to, to the actual IP of the aqs host or to one of the IP addresses assigned to the Cassandra instances? [14:35:33] cassandra instance, aqs1004-a [14:36:03] so I just tried from analytics1052 [14:36:07] telnet aqs1004-a.eqiad.wmnet 9042 [14:36:11] telnet aqs1004.eqiad.wmnet 9042 [14:36:25] and I get Trying $IP1 and $IP2 [14:36:32] correspondent to the above domains [14:36:53] meanwhile [14:36:54] elukey@aqs1005:~$ telnet aqs1004-a.eqiad.wmnet 9042 [14:36:54] Trying 10.64.0.126... [14:36:54] Connected to aqs1004-a.eqiad.wmnet. [14:37:44] compared the IPs from the various telnets, all looks good [14:40:23] the aqs hosts are in the standard eqiad network, while the hadoop nodes are in the analytics network. you should talk to Faidon, I'm pretty sure this needs changes on the router side, probably only aqs100[1-3] are currently granted there [14:40:30] actually milimetric, while looking at our schema definition, the new_editor (+ productive and survaving) are present ... [14:40:38] once that is checked/fixed we can revisit the iptables/ferm level [14:40:41] moritzm: yes this is a very good point! [14:40:50] I forgot about it! [14:40:58] joal: present? [15:23:27] joal: cassandra-daily-coord-pageviews_per_article_flat_LCS is now running like a charm :) [15:23:38] elukey: YESSSSSS ! [15:23:53] elukey: in meeting, but I'll get back to you soo [15:36:17] Thanks a lot moritzm for having found the issue :) [15:45:16] Analytics, Editing-Analysis, Notifications, Collab-Team-2016-Apr-Jun-Q4: Numerous Notification Tracking Graphs Stopped Working at End of 2015 - https://phabricator.wikimedia.org/T132116#2323211 (Nuria) @Neil_P._Quinn_WMF Ah, I see, Thanks for looking into it, let us know if you need any help.... [15:49:51] Analytics-Cluster, Analytics-Kanban, Patch-For-Review: Configure Spark YARN Dynamic Resource Allocation - https://phabricator.wikimedia.org/T101343#2323217 (Ottomata) Need to: - parameterize and set spark.shuffle.service.enabled true Consider removing explicit executor settings for production oozie... [15:56:57] Analytics, Analytics-Wikistats, Internet-Archive: Total page view numbers on Wikistats do not match new page view definition - https://phabricator.wikimedia.org/T126579#2323297 (Nuria) >Also missing but negligible are mediawiki (6M/mon) and www.wikisource (0.5M/mon, the precursor of language-specific... [16:00:59] ottomata / elukey: standup [16:25:02] Analytics-Kanban, RESTBase, Services, RESTBase-API, User-mobrovac: Enable rate limiting on pageview api - https://phabricator.wikimedia.org/T135240#2323436 (mobrovac) >>! In T135240#2299055, @Milimetric wrote: > The PR for this change: https://github.com/wikimedia/restbase/pull/614 It's been... [16:47:18] Analytics-Kanban, RESTBase, Services, RESTBase-API, User-mobrovac: Enable rate limiting on pageview api - https://phabricator.wikimedia.org/T135240#2323504 (Nuria) What i do not understand is what is exactly enabled here: - from @mobroac's comments it looks like limits per worker are enabled... [16:48:12] elukey: here ? [16:49:42] joal: yep [16:51:43] elukey: batcave ? [16:56:39] Analytics-Kanban, RESTBase, Services, RESTBase-API, User-mobrovac: Enable rate limiting on pageview api - https://phabricator.wikimedia.org/T135240#2292830 (Pchelolo) @Nuria The DST is enabled in production right now. All the limits are per endpoint, not per worker. [16:57:26] Analytics, cassandra: AQS Cassandra cluster: Restart cassandra-metrics-collector - https://phabricator.wikimedia.org/T134513#2323530 (Eevans) Open>Resolved a:Eevans Since opening this issue, I've had Puppet updated to automatically restart cassandra-metrics-collector, and a subsequent run (on... [16:59:58] a-team: retrroooooo [17:01:07] ottomata: retrooo [17:38:05] ottomata: I'm gonna poke around to see what's up with the druid community [17:38:17] k [17:42:24] ottomata: could we merge this one? https://gerrit.wikimedia.org/r/#/c/290284/ [17:43:05] nuria_: , quotes around 'latest' and yes! [17:43:26] ottomata: will do, i saw it both ways on puppet, wasn't sure, thank you [17:45:52] (PS3) Nuria: MonthlyUniqueDevices metric should be bookmarkeable [analytics/dashiki] - https://gerrit.wikimedia.org/r/290306 (https://phabricator.wikimedia.org/T122533) [17:45:53] joal: sorry if I wasn't super chatty but today was looong, my brain is segfaulting now :D [17:46:28] (CR) Nuria: MonthlyUniqueDevices metric should be bookmarkeable (1 comment) [analytics/dashiki] - https://gerrit.wikimedia.org/r/290306 (https://phabricator.wikimedia.org/T122533) (owner: Nuria) [17:46:45] elukey: no worries, didn't even notice :) [17:51:54] a-team logging off, talk with you tomorrow :) [17:51:59] laters! [17:52:00] bye elukey ! [17:52:02] elukey, bye! [17:52:05] nite [17:52:08] bye elukey :) good night! [17:52:11] o/ [17:52:49] mforns: will go for dinner in a few minutes, is there anything I can help on rapidly? [17:53:09] nuria puppet pulled! [17:53:17] joal, I am doing a quick review of dashiki, and I will continue with scala [17:53:27] but ja, looks cached :) [17:53:42] k mforns, will come and help after diner [17:53:49] * joal is away for diner [17:53:54] so, thanks joal we'll see later, bon appetit [17:54:12] oh, nuria_ naw, i just had to hard referesh [17:54:24] link is fixed [17:54:51] ottomata: k, great! [17:56:21] (CR) Mforns: [C: 2 V: 2] "LGTM!" [analytics/dashiki] - https://gerrit.wikimedia.org/r/290306 (https://phabricator.wikimedia.org/T122533) (owner: Nuria) [18:00:02] ottomata: Do you think we should still do this? https://phabricator.wikimedia.org/T132177 [18:00:20] mforns: all right, let's deploy to labs for now. I will be removing vital-signs i think when we announce analytics.wikimedia.org [18:00:36] nuria_: you could set up a redirect [18:00:44] seems kinda abrupt to kill it [18:00:59] madhuvishy: yes, that is what i will do [18:01:04] cool [18:01:11] ah, ja madhuvishy that is probably a good one to do [18:01:13] madhuvishy: but not yet, we need erik to change links in wikistats [18:01:19] do you need that soon? [18:01:24] madhuvishy: also test domains will remain cause those are super useful [18:01:32] Analytics-Kanban: Backfill Android Apps pageviews from May 2nd hour 21 - https://phabricator.wikimedia.org/T135299#2323872 (Tbayer) Great, thanks! Will take a closer look, but at least the percentage of app views among overall pageviews is back to previous levels (1.3% for the week until May 22). What abou... [18:01:47] ottomata: if you can do that now, I can update that part of the job before getting it merged [18:02:08] but it's not necessary to finish this project [18:02:44] nuria_, I will deploy to staging, but remember that we can not see the unique devices there, until we change the config wiki for production, they share the same config page [18:02:45] nuria_: ah yeah cool - I've been wanting to ask - how are you deploying the dashboards to stat1001? [18:02:57] madhuvishy: static git clone [18:03:01] ok, i can make the user real easy, but storing the pw in the pwstore has always been a pain and i've put off doing that [18:03:03] :( [18:03:12] mforns: nah, i will deploy no worries [18:03:16] hm, oh, but it will be in puppet, ja? [18:03:17] hmmmm [18:03:18] that isn't so hard [18:03:20] in puppet private [18:03:32] madhuvishy: where will you save the login info? [18:03:43] ottomata: cool - i'll need to save it jenkins anyway [18:03:46] let me give you link [18:03:46] mforns: i will re test in the browsers i have and deploy [18:03:55] madhuvishy: where /how does jenkins save it? [18:04:09] ottomata: https://integration.wikimedia.org/ci/credential-store/ [18:04:30] ottomata: to be specific, here - https://integration.wikimedia.org/ci/credential-store/domain/_/ [18:04:48] all private keys, sensitive passwords etc are stored here [18:04:59] huh, ok [18:05:03] interesting [18:05:17] it's still labs [18:05:36] ottomata: I have the archiva creds available there now - but ideally I'd remove those and add new ones. [18:05:49] and have it available in a global maven settings file [18:05:51] aye [18:06:01] and any one can use it [18:06:01] is 'archiva-ci' a good name? you think? [18:06:21] username [18:06:28] ottomata: sure! [18:06:32] k [18:07:08] nuria_: why static clone? we should probably extend the deployment process to stat1001 if we are going to have more dashboards [18:08:16] ottomata: you can just add it here - https://integration.wikimedia.org/ci/credential-store/domain/_/newCredentials and dont have to send me the password (username with password, global scope) [18:09:45] ok [18:11:25] madhuvishy: git clone is the easiest to deploy to 1001, is it not? it's all static for all dashboards so there are not several sites, just one that is cached by varnish. [18:11:48] nuria_: but deployment should not be controlled by puppet ideally [18:11:54] puppet sets up infrastructure [18:12:02] deployments are handled by us [18:12:48] madhuvishy: done [18:12:55] ottomata: <3 thank you [18:12:58] madhuvishy: in theory sure, in reality we do not have an easier way to deploy a static site that i know of, have in mind that this could be deployed to s3, is all client side [18:13:17] madhuvishy: ja we all talked about this. deploying via scap would be fine too [18:13:20] we might do it eventually [18:13:26] for now a git clone of static files was pretty easy [18:13:31] hmmm [18:13:34] scap is overkill in my opinion [18:13:40] finishing with a hammer [18:13:58] nuria_: i mean, it's kinda more correct, but ja, in this case all scap would do is a git clone anyway [18:14:00] git pull [18:14:12] we just told puppet it to do it every time it runs [18:14:16] instead of doing it manually [18:14:28] right - but this way - once merged, its deployed [18:14:44] madhuvishy: right [18:15:31] madhuvishy: which for this site is no issue that i can see of, for a real application (server+client+storage) it would be [18:15:39] so scap i guess atleast lets you control when something is deployed. i mean, we could still use fabric but then have to deploy from local [18:16:10] madhuvishy: in an ideal world I would have a copySRC build that creates a package and a deploy process that pulls that package (no deploy from git) [18:16:37] yeah i'm not a fan of the deploy from git for this case too [18:16:58] madhuvishy: the fact that scap still deploys from git -which means i need to commit the build- makes me not think good thinks about that system [18:17:21] i would buy it if deployed not from source [18:17:23] ah nuria_ I guess scap was not built with static sites in mind [18:17:32] hmmm [18:20:42] ya [18:21:20] deploying from the depot makes me sad and also makes me think we are back in the PAST [18:22:39] bbib [18:23:53] wikimedia/mediawiki-extensions-EventLogging#558 (wmf/1.28.0-wmf.3 - 57f911c : Mukunda Modell): The build has errored. [18:23:53] Change view : https://github.com/wikimedia/mediawiki-extensions-EventLogging/commit/57f911c05ff8 [18:23:53] Build details : https://travis-ci.org/wikimedia/mediawiki-extensions-EventLogging/builds/132637611 [18:29:26] Analytics-Kanban: Create separate archiva credentials to be loaded to the Jenkins cred store {hawk} - https://phabricator.wikimedia.org/T132177#2190727 (madhuvishy) archiva-ci user has been created, and is available in the Jenkins credential store [18:29:28] Analytics-Kanban, RESTBase, Services, RESTBase-API, User-mobrovac: Enable rate limiting on pageview api - https://phabricator.wikimedia.org/T135240#2323937 (Nuria) @Pchelolo : so i should be able to see limits getting logged if i run a test using apache bench correct? [18:30:42] Analytics: Have archiva server credentials available via the Config File Builder in global maven settings.xml - https://phabricator.wikimedia.org/T132178#2323940 (madhuvishy) Done. Available in https://integration.wikimedia.org/ci/configfiles/ as ArchivaCredentialsSettings [18:30:58] Analytics-Kanban: Have archiva server credentials available via the Config File Builder in global maven settings.xml - https://phabricator.wikimedia.org/T132178#2323941 (madhuvishy) [18:34:20] Analytics-Kanban: Figure out if the Changelog file can be updated in the release process by Jenkins {hawk} - https://phabricator.wikimedia.org/T132181#2323958 (madhuvishy) Wrote a script to compute changelog from commit messages and have it automatically committed and pushed as part of the release process. W... [18:34:38] Analytics-Kanban, RESTBase, Services, RESTBase-API, User-mobrovac: Enable rate limiting on pageview api - https://phabricator.wikimedia.org/T135240#2323960 (Pchelolo) @Nuria not really, page views are varnish cached, so for `ab` only the first request will actually hit the servers. You need t... [18:35:03] Analytics: Dashiki: move available.projects.json to a better location - https://phabricator.wikimedia.org/T136120#2323964 (Nuria) [18:48:14] Analytics-Kanban, RESTBase, Services, RESTBase-API, User-mobrovac: Enable rate limiting on pageview api - https://phabricator.wikimedia.org/T135240#2324009 (Nuria) @Pchelolo : well, i was going to "replay" requests to article endpoint which is real easy to do as we have them all. Looking at... [18:53:20] Analytics-Kanban, RESTBase, Services, RESTBase-API, User-mobrovac: Enable rate limiting on pageview api - https://phabricator.wikimedia.org/T135240#2324016 (Pchelolo) >>! In T135240#2324009, @Nuria wrote: > Should I send a PR to change it here: https://github.com/wikimedia/restbase/blob/maste... [19:07:42] mforns: ok, tested a bunch found a small issue with breakdowns that it is not super easy to fix but that i think doesn't need fixing quite yet. [19:07:59] nuria_, mmmmmm buf [19:08:10] mforns: nothing to do with the new code though [19:08:16] nuria_, can you summarize? [19:08:17] mforns: no ui issue [19:08:33] mforns: yes, for small projects in which there is no mobile-app breakdown [19:08:54] the promise to request that data fails (as req returns 404) [19:08:58] nuria_, mmmmm I see [19:09:09] mforns: and thus promises.all() falls into rejected patterns [19:09:23] nuria_, well [19:09:28] mforns: it is fixable though [19:09:39] sure, I can create a task and do it this week [19:10:08] mforns: nah, i will do it. no rush [19:10:16] Analytics-Kanban: Backfill Android Apps pageviews from May 2nd hour 21 - https://phabricator.wikimedia.org/T135299#2324062 (JAllemandou) @Tbayer : I Forgot about those two (they don't depend on pageview). Currently backfilling daily, no need to do it for monthly. [19:13:06] joal: Yt? [19:13:11] yes nuria_ [19:13:56] joal: i was looking at graphana and it seems that when we get about 30/40 requests on aqs endpoint (per article) the number of 500 spikes [19:14:07] joal: does that sound right? [19:14:20] joal: ( i know issues with date ranges might be on top of this though) [19:14:27] nuria_: correct [19:14:53] nuria_: 500 begin to rise after 25, and higher after 30 [19:15:01] (in my looking) [19:17:41] Analytics-Kanban: Create separate archiva credentials to be loaded to the Jenkins cred store {hawk} - https://phabricator.wikimedia.org/T132177#2324101 (Ottomata) I need to add this to the ops pwstore, but am having trouble committing. Keep this open. [19:17:45] Analytics-Kanban, RESTBase, Services, RESTBase-API, User-mobrovac: Enable rate limiting on pageview api - https://phabricator.wikimedia.org/T135240#2324102 (Nuria) @Pchelolo : number for us at which we want to start throttling it would be ~25/30 request per whole cluster (which has three mach... [19:19:34] joal: k [19:25:59] !log deploying latest master to dashiki 08cc9a2545bcc0a183a3c00c18e81f21326a41b [19:26:00] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master [19:30:26] nuria_: just fyi, jquery 3.0 rc is out, if you work on that .all() promises bug, you can upgrade if you like their changes (they just re-worked deferred a bit, don't think it affects the bug you found) [19:30:49] for that bug, probably just adding a .always handler instead of the .then [19:31:25] milimetric: always? wait, let me see if that is on the native promises [19:31:32] milimetric: ja, taht might work [19:31:43] these are jquery promises I think, not native [19:31:53] though in jquery 3.0 it sounds like they'll be fully compatible [19:32:05] milimetric: right, the native ones do not have always [19:32:12] milimetric: k, noted, need to file bug [19:35:59] milimetric, mforns: I think "unique Devices" should probably "daily unique devices" though, for clarity [19:36:08] *should probably be [19:36:08] nuria_, makes sense [19:36:28] my thought before was that every other metric is daily, so that was like the assumed default [19:36:41] true [19:36:43] and if we changed one to "daily ..." it might be confusing if we don't change them all [19:36:57] right, very true, leaving as is then [19:36:58] buut... I donno, I agree "daily unique devices" is clearer [19:37:39] might make sense in the future to change all metrics to include the granularity or to have selectable granularity from a dropdown that knows all available choices [19:37:52] ya readers-.daily->unique devices [19:38:04] readers->daily->unique devices [19:38:19] yep, and we could easily do that in the categorizedMetrics computable, it already does this kind of thing with the categories [19:38:38] or maybe readers -> unique devices -> daily [19:39:00] milimetric: ok, will file task for taht too [19:39:02] *that [19:39:21] mforns, milimetric : deployed, https://vital-signs.wmflabs.org/#projects=eswiki/metrics=MonthlyUniqueDevices [19:39:41] mforns, milimetric : i think it looks good, am about to send e-mail about it [19:39:41] yes!!! [19:39:48] \o/ [19:39:53] woooohoooo!!! [19:39:56] really nice job mforns + nuria_ [19:40:13] + milimetric :] [19:40:30] omg this reminds me I have to change metrics-by-project to use a flex layout, so much better [19:42:08] Analytics-Kanban, RESTBase, Services, RESTBase-API, User-mobrovac: Enable rate limiting on pageview api - https://phabricator.wikimedia.org/T135240#2324160 (Pchelolo) @Nuria this is per-endpoint limit. You have 3 endpoints, so 10 would be a good number [19:46:13] milimetric: can you try IE? [19:46:23] milimetric: i tried chrome and FF [19:46:34] not yet, I have to be at home [19:46:42] but I'll def. try it when I get home [19:47:37] milimetric: will send two e-mails one external one internal [19:48:05] sweet, I'm happy to help with that if you want [19:57:19] Analytics: Dashiki, Unique Devices and Pageview data breakdown doesn't work if any of the items are not available for the project - https://phabricator.wikimedia.org/T136125#2324192 (Nuria) [20:00:26] Analytics: Dashiki: Better menu for metrics - https://phabricator.wikimedia.org/T136126#2324211 (Nuria) [20:11:11] Analytics-Kanban, RESTBase, Services, RESTBase-API, User-mobrovac: Enable rate limiting on pageview api - https://phabricator.wikimedia.org/T135240#2324260 (Nuria) https://github.com/wikimedia/restbase/pull/618 [20:13:20] Analytics: Dashiki: Breakdowns should be bookmarkeable - https://phabricator.wikimedia.org/T136127#2324269 (Nuria) [20:22:39] hi terrrydactyl :) hope all's well [20:22:53] hi! [20:33:08] Analytics, Analytics-Dashiki: Breakdowns should be bookmarkeable - https://phabricator.wikimedia.org/T136127#2324348 (Danny_B) [20:34:01] Analytics, Analytics-Dashiki: Better menu for metrics - https://phabricator.wikimedia.org/T136126#2324354 (Danny_B) [20:35:10] Analytics, Analytics-Dashiki: Dashiki, Unique Devices and Pageview data breakdown doesn't work if any of the items are not available for the project - https://phabricator.wikimedia.org/T136125#2324367 (Danny_B) [20:35:26] Analytics, Analytics-Dashiki: move available.projects.json to a better location - https://phabricator.wikimedia.org/T136120#2324369 (Danny_B) [20:37:27] Analytics, Analytics-Dashiki: Simplify readiness checking by making a ready computed - https://phabricator.wikimedia.org/T136025#2324373 (Danny_B) [20:37:57] Analytics, Analytics-Dashiki: Searching for nl... does not bring nlwikipedia only nlwikidata - https://phabricator.wikimedia.org/T133718#2324375 (Danny_B) [20:38:22] Analytics-Dashiki, Analytics-Kanban, Patch-For-Review: Add support to 'displayName' param in config - https://phabricator.wikimedia.org/T134924#2324377 (Danny_B) [20:40:04] Analytics-Dashiki, Analytics-Kanban, Patch-For-Review: Visualize unique devices data {bear} - https://phabricator.wikimedia.org/T122533#2324380 (Danny_B) [20:41:00] Analytics, Analytics-Dashiki: Add extension and category (ala Eventlogging) for DashikiConfigs - https://phabricator.wikimedia.org/T125403#2324390 (Danny_B) [20:41:25] Analytics, Analytics-Dashiki: Upgrade Dashiki to semantic-2 for all layouts - https://phabricator.wikimedia.org/T125409#2324392 (Danny_B) [20:41:42] Analytics, Analytics-Dashiki: Have dashiki read and write GET params to pass stateful versions of dashboard pages {crow} - https://phabricator.wikimedia.org/T119996#2324394 (Danny_B) [20:42:07] Analytics, Analytics-Dashiki: Allow clicking on links in annotations - https://phabricator.wikimedia.org/T110459#2324395 (Danny_B) [21:00:17] Analytics, Wikimedia-Site-requests: Need a Dashiki namespace so we can protect configs {crow} - https://phabricator.wikimedia.org/T112268#2324496 (Danny_B) [21:32:44] ottomata: still around? [21:33:03] ja hi [21:33:20] quick question on your comment on https://gerrit.wikimedia.org/r/#/c/288210/6/jsonschema/mediawiki/revision_create/1.yaml [21:34:04] ottomata: every field is named rev_* except for page and user related ones ... So I guess if we change user, we change page? [21:34:45] hmmmm [21:34:57] oh no, i mean we should just reorder them so that they are next to each other in the schema [21:35:00] no functional change [21:35:03] just aesthetic [21:35:18] OoooOOOh ! [21:35:23] Didn't get it :) [21:35:35] Ok, aesthetic change it will be :) [21:48:16] nuria: in dashiki, src/app/apis/wikimetrics.js has the comment: "This module returns an instance of an object that knows how to get reports run by WikimetricsBot on wikimetrics". [21:48:22] What's WikimetricsBot? [21:50:12] ottomata: did one big last change to the schemas (page_move one) [21:50:43] ottomata: after that we should discuss with Mark on details/technical aspects, since we seem to be globally in agreement [21:57:05] k cool [22:07:31] joal: you still up? if so, how would you feel about a druid/hadoop brainbounce real quick? [22:07:40] ottomata: let's go :) [22:08:26] ottomata: in da cave [22:26:22] bye a-team! tomorrow [22:26:27] bye mforns ! [22:26:34] bye :) [22:26:49] joal: do we have a refinery release sometime soon? [22:26:59] madhuvishy: nothing planed yet [22:27:07] joal: alright [22:31:27] Analytics-Kanban, Patch-For-Review: Translate the analytics-release-test job to YAML config in integration/config {hawk} - https://phabricator.wikimedia.org/T132182#2324797 (madhuvishy) [22:35:49] Analytics: Figure out the exact strategy for release {hawk} - https://phabricator.wikimedia.org/T132180#2324843 (madhuvishy) This will have to be: Go to https://integration.wikimedia.org/ci/job/analytics-refinery-release/m2release/. Verify the versions - Check the Specify custom SCM version checkbox - chang... [22:36:11] Analytics-Kanban: Figure out the exact strategy for release {hawk} - https://phabricator.wikimedia.org/T132180#2324844 (madhuvishy) [22:52:08] Analytics-Kanban: Backfill Android Apps pageviews from May 2nd hour 21 - https://phabricator.wikimedia.org/T135299#2324875 (JAllemandou) @ TBayer: Mobile apps uniques backfilled ! [23:28:54] off for tonight a-team, see you tomorrow ! [23:35:55] neilpquinn: still there? [23:39:59] yep, I'm here! [23:40:34] nuria ^ :) [23:40:48] or actually nuria_ :) [23:41:39] neilpquinn: to your dashiki question, did you get an answer? [23:41:58] no, I didn't. nothing urgent but I'm curious :) [23:42:16] neilpquinn: mostly is that it does not matter for dashiki purposes, we have an automated user for wikimetrics [23:42:23] which we call wikimetrics bot [23:42:49] neilpquinn: it calculates the edit metrics like rolling active editor [23:43:09] neilpquinn: but you know we are working on having edit data on hdfs and do away with all those calculations right? [23:44:32] nuria_: Yeah, I'm generally aware of that work, although I don't really know what it consists of or how it will work with our use cases specifically. Who's the best person to talk with about that? Dan? [23:45:57] nuria_: And I hear it's at least 3-6 months away so we still have to have editor metrics in the interim. [23:46:08] nuria_: Anyway, what I'm doing right now is trying to understand Dashiki so I can add things to our dashboard :) [23:46:14] neilpquinn: k [23:52:43] nuria_: so who's the best person to talk to about the editing data project? :)