[00:44:44] (03CR) 10Jforrester: [V: 032] Add frwiki, hewiki, etwiki to list of RCFilters wikis [analytics/limn-ee-data] - 10https://gerrit.wikimedia.org/r/348028 (owner: 10Catrope) [00:58:14] (03PS6) 10Nuria: Changes internal aqs api to accept a project or array of same [analytics/dashiki] - 10https://gerrit.wikimedia.org/r/347305 (owner: 10Fdans) [01:01:24] (03CR) 10Nuria: Changes internal aqs api to accept a project or array of same (031 comment) [analytics/dashiki] - 10https://gerrit.wikimedia.org/r/347305 (owner: 10Fdans) [01:33:44] 06Analytics-Kanban: Check abnormal pageviews for XHamster - https://phabricator.wikimedia.org/T158071#3192604 (10MusikAnimal) We talked on IRC, but I'll also thank you here for the thorough analysis! I am going to keep XHamster excluded in [[ https://tools.wmflabs.org/topviews | Topviews ]] even though traffic a... [07:00:55] since we're now running the Hadoop cluster on jessie, how about migrating from Java 7 to Java 8? we support both for jessie [07:01:22] background: there's a new round of Java security issues, so we need a rolling restart anyway [07:01:33] it takes a few days until a new OpenJDK release is cut, so I don't expect that we have new packages until next week. though [07:28:41] moritzm: o/ [07:29:13] I'll have a chat with the team this evening during standup, but I suppose we could upgrade one node and see how it goes before rolling it out everywhere [07:31:05] yeah, that's what I was thinking a well, it'll take at least a week until new OpenJDK 8 for the April CPU packages are ready [07:31:30] all right I'll ask and I'll let you know :) [08:49:50] Hi elukey [08:49:53] elukey: https://www.cloudera.com/documentation/enterprise/release-notes/topics/rn_consolidated_pcm.html [08:50:07] I think it's a bad idea to do a rolling move [08:55:25] 06Analytics-Kanban, 13Patch-For-Review, 15User-Elukey: Apply Xms Java Heap settings to all the Hadoop daemons - https://phabricator.wikimedia.org/T159219#3193121 (10elukey) I can see positive effects on Datanodes and Nodemanagers after a week of data-points: {F7630395} {F7630399} Last one to do is the Yar... [08:55:27] 10Analytics: upgrade druid to 0.9.2 - https://phabricator.wikimedia.org/T157977#3193122 (10JAllemandou) From Fangjin email, looks like new versions will be released soon: - Druid 0.10 - pivot 0.11 Let's wait for those :) [08:56:07] joal: o/ - sorry checking now, I was writing https://phabricator.wikimedia.org/T159219#3193121 (JVM improvements after adding Xms!) [09:00:25] joal: we'd need to test java8 somehow before moving to it though, maybe in labs? [09:00:39] elukey: labs seems a good place [09:01:05] okok, we could open a phab task and then schedule this work [09:01:15] elukey: But this move is huge in term of user-impact: cluster needs to be stopped, then upgrade, then restart with potential issues :( [09:01:53] joal: sure sure not saying that we have to do it now, it was more to schedule/prepare for it [09:01:59] if running a single hadoop node on java 8 causes problems, how is the eventual upgrade going to happen? this won't be atomic either since the rolling restart/upgrade will take at least an hour as well? [09:02:51] moritzm: I think this move needs to be atomic cluster wide (i might be wrong though) [09:03:12] moritzm: stop the cluster, upgrade, change conf in cluster, restart [09:04:21] 10Analytics-EventLogging, 06Analytics-Kanban: Implement EventLogging Hive refinement - https://phabricator.wikimedia.org/T162610#3193141 (10JAllemandou) Thanks for the report @ottomata ! About the normalization and camel cased fields, we could do it in 2 passes (again). It's not clean, but might work: # Merg... [09:04:43] ok [09:31:20] 10Analytics, 10Analytics-Cluster: Add CI job Oozie XML stylesheet validation for the analytics/refinery repository - https://phabricator.wikimedia.org/T147072#3193171 (10hashar) Dropping #continuous-integration-config . CI is configured to run maven and adding Oozie should be done via the project pom.xml. [09:35:49] Leaving for medical apointment with Melissa a-team, see you after ! [09:44:18] (03CR) 10Matthias Mullie: Query the most common UploadWizard exceptions & errors (038 comments) [analytics/limn-multimedia-data] - 10https://gerrit.wikimedia.org/r/340720 (https://phabricator.wikimedia.org/T156694) (owner: 10Matthias Mullie) [09:44:30] (03PS2) 10Matthias Mullie: Query the most common UploadWizard exceptions & errors [analytics/limn-multimedia-data] - 10https://gerrit.wikimedia.org/r/340720 (https://phabricator.wikimedia.org/T156694) [10:17:34] * elukey lunch before the switchover [10:51:09] 06Analytics-Kanban, 06DC-Ops, 06Operations, 10ops-eqiad: analytics1030 stuck in console while booting - https://phabricator.wikimedia.org/T162046#3193224 (10elukey) @Cmjohnson do we need to order replacement parts for this host or is it simply into an inconsistent state? [13:18:50] joal: you mentioned earlier this month that in order to deploy the seconds to revert we have to wait to the end of the month, but is there anything that remains to be done before that? [13:18:58] (bonjour mon ami!) [13:45:32] Hey fdans ! [13:45:42] We can move forward with the current patch as it is [13:54:20] joal: wonder what you think about flattening vs keep going with the work [13:54:36] i'm leaning towards keep going, especially if it will work better if we eventually get hive 1.2 [13:54:45] but, flattening will work and be simpler [13:54:56] hm, maybe we can talk post standup about this [13:54:59] joal: batcave and talk about that for 2 mins? [13:55:10] haven't seen you yet :'( [13:55:41] ottomata: I like the fact of not flattening for elegance and reusability, but for EL, flattening will also help users finding fields etc ... I have no real side [13:55:45] ottomata: o/ - switchover about to start :) [13:55:52] fdans: syure ! [13:56:16] wooowee ok! [13:56:21] ottomata: I have meetings all evening today, will be difficult to find time [13:56:28] ya [13:56:35] no i mean with everybody post standup, i'll see what they thikn [13:56:43] your suggestion on ticket is good [13:56:47] i think i have enough to work with [13:56:59] or, maybe i shouldn't ask because i also want non flattening to work [13:57:02] ottomata: ok cool, sounds good :) [13:57:02] so i'll just try for a bit more [13:57:04] :) [13:57:11] ok awesome :) [14:04:02] 06Analytics-Kanban, 13Patch-For-Review: Add daily unique devices dataset to pivot - https://phabricator.wikimedia.org/T159471#3193623 (10JAllemandou) [14:12:36] 06Analytics-Kanban: Add monthly unique devices dataset to pivot - https://phabricator.wikimedia.org/T163327#3193688 (10JAllemandou) [14:12:50] milimetric: hellooooo do you have plans to deploy mediawikihistory this week? cc joal [14:20:26] joal,ottomata - I'd skip the meeting due to the switchover in progress [14:20:34] sure elukey, no prob [14:20:40] ottomata: scala then? [14:23:51] elukey: want to talk to you about zk stuff and upcoming row D networking stuff and hadoop nodes an30 and an69, but we can move meeting, maybe tomorrow or friday? [14:24:00] same time? [14:24:10] joal: ya ok [14:24:17] i'm working out what I need to do, would be good to bb [14:24:20] we can do it after standup if you want! [14:24:28] elukey: ok! [14:24:38] actually tomorrow same time might be better for me [14:24:58] elukey, ottomata: I'm fully busy after standup, but please go without me [14:29:29] (03PS2) 10Mforns: Improve annotations [analytics/dashiki] - 10https://gerrit.wikimedia.org/r/348227 (https://phabricator.wikimedia.org/T162482) [14:31:03] I've not been working on the history stuff fdans, why? [14:33:52] milimetric: joal and I have a couple changes pending to be deployed and just wanted to ask, in case you had plans for it [14:34:25] oh, I can definitely do it if you need, it's my ops week so sure [14:35:13] (03CR) 10Mforns: "Hey :] I think this is ready for review and merge-if-proceeds." [analytics/dashiki] - 10https://gerrit.wikimedia.org/r/348227 (https://phabricator.wikimedia.org/T162482) (owner: 10Mforns) [14:45:05] milimetric: you sure? I was just asking if you were going to do it anyway, not to put more stuff on your plate :) [14:45:59] yeah, no problem, that's what ops duty is for, what needs deploying? Refinery / refinery-source? [14:52:52] ottomata: you wanna see a seemingly magical problem? [14:53:45] * milimetric going to the batcave, could use elukey or ottomata to brainbounce with [14:56:20] a-team: switchover done, we are in Dallas! But there are things to fix so I'll probably skip standup (and send e-scrum) [14:56:28] sure elukey [14:56:30] ottomata: I am ok to chat tomorrow same time! [15:03:09] 10Analytics-Cluster, 06Analytics-Kanban, 13Patch-For-Review: Spark + ORES dependencies in Hadoop - https://phabricator.wikimedia.org/T162706#3193887 (10Ottomata) [15:04:30] elukey: https://www.youtube.com/watch?v=dEUugovB2dI [15:16:30] 10Analytics-Tech-community-metrics, 06Developer-Relations (Apr-Jun 2017): https://wikimedia.biterg.io shows 2017 contributors who are not listed in mediawiki-identities/wikimedia-affiliations.json - https://phabricator.wikimedia.org/T161235#3125956 (10Qgil) This task committed to #devrel-apr-jun-17 needs an ow... [15:16:54] 10Analytics-Tech-community-metrics, 06Developer-Relations (Apr-Jun 2017): Updated data in mediawiki-identities DB not deployed onto wikimedia.biterg.io? - https://phabricator.wikimedia.org/T157898#3020057 (10Qgil) This task committed to #devrel-apr-jun-17 needs an owner. [15:17:13] 10Analytics-Tech-community-metrics, 06Developer-Relations (Apr-Jun 2017): On the "Git" dashboard, filtering on one organization still lists authors who are with another organization - https://phabricator.wikimedia.org/T157709#3013972 (10Qgil) This task committed to #devrel-apr-jun-17 needs an owner. [15:17:29] 10Analytics-Tech-community-metrics, 06Developer-Relations (Apr-Jun 2017): When indexing new users, identify identical email addresses and merge identities accordingly in the DB - https://phabricator.wikimedia.org/T151634#2823252 (10Qgil) This task committed to #devrel-apr-jun-17 needs an owner. [15:29:13] 06Analytics-Kanban: Investigate rise in IE views from Pakistan since 2015 - https://phabricator.wikimedia.org/T157404#3194046 (10fdans) Hi! So here are my findings so far. Looking at a sample of requests from Pakistan in February 15 2017, by far the most common user agent string is the following: `Mozilla/4.0 (... [15:30:19] 10Analytics: Getting different versions of the same file - https://phabricator.wikimedia.org/T163338#3194051 (10Milimetric) [15:32:52] milimetric: i think you pasted the wrong request [15:32:57] in the ticket there are 2 with pass [15:33:00] and they are different paths [15:33:03] frwiki and aawiki [15:33:16] https://phabricator.wikimedia.org/T163338 [15:34:17] 10Analytics: Getting different versions of the same file - https://phabricator.wikimedia.org/T163338#3194080 (10Milimetric) [15:34:27] sorry about that ottomata, updated [15:47:36] 10Analytics: Getting different versions of the same file - https://phabricator.wikimedia.org/T163338#3194051 (10Ottomata) I'm pretty baffled by this. I can't reproduce the `pass` ever, and these requests are pretty similar. We may want to go ahead and add a rule to always pass /datasets URIs, since this is pr... [15:49:06] milimetric: I can see Cache-Control: max-age=86400, public, must-revalidate in the response header, it might cause some troble no? [15:49:24] IIRC this is telling to caches and browser to re-validate after a day [15:50:39] why the browser is passing is weird though [15:54:28] ahh so if I use incognito I get the cached content, meanwhile if not (via browser) the last one [15:55:08] it is probably one header set by the browser that forces Varnish to pass [16:01:48] milimetric: I tried with a random url https://analytics.wikimedia.org/datasets/periodic/reports/metrics/browser/ [16:01:57] and I can repro the problem with/without incognito [16:02:14] the diff seems to be a cookie for anonymous users [16:04:36] sorry elukey, just got out of meeting, reading up [16:05:41] elukey: cassandra meeting deleted for switchover? [16:05:51] elukey: so if the browser accepts cookies, it gets the cached version and if it doesn't it passes and gets the latest? [16:06:04] joal: aargh joining sorry [16:06:14] elukey: urandom is not here either ;) [16:06:28] * milimetric deploying refinery and refinery-source a-team [16:06:36] k [16:08:16] milimetric: that could be an explanation [16:09:27] hm, I still get cp1058 miss, cp2012 hit/1, cp2012 hit/9 in incognito for that same frwiki.tsv file [16:10:05] milimetric: in incognito you should get the cached content right? [16:10:18] (that is the cp2012 hit/9) [16:10:26] I get the cached content in both incognito or not incognito [16:10:58] wait you showed us originally that you were able to get the last file with the browser :D [16:11:34] sorry, three things [16:12:08] dashboard/incognito: cached, dashboard/normal: cached, datasets download/incognito: cached, datasets download/normal: *NOT* cached [16:12:16] I lied, four things [16:13:27] ah good the last two are what I am seeing too :) [16:13:57] cute histogram https://phabricator.wikimedia.org/T145661#3194084 [16:15:42] (03PS1) 10Milimetric: Update changelog.md to v0.0.45 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/348957 [16:15:52] (03CR) 10Milimetric: [V: 032 C: 032] Update changelog.md to v0.0.45 [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/348957 (owner: 10Milimetric) [16:17:55] milimetric: curl https://analytics.wikimedia.org/datasets/periodic/reports/metrics/language/compact-language-links/frwiki.tsv -i --header "cookie:ajs_anonymous_id=hellodan" [16:18:31] cool, that's updated [16:18:46] is that a magic cookie or basically any cookie would work? [16:19:02] I am not sure, I just added it since it was in my Browser's request [16:19:25] checking [16:19:46] yeah, any cookie [16:19:53] but you *have* to send some cookie [16:20:52] weird... I mean I guess I could update dashiki to always send a random cookie... seems like a bad idea though :) [16:26:33] what?! [16:26:34] FATAL: your Apache Maven build is setup to use a config with id org.jenkinsci.plugins.configfiles.maven.MavenSettingsConfig.ArchivaCredentialsSettings but can not find the config [16:26:45] I hate Jenkins, I'm sorry, it's the worst [16:27:35] joal: sorry I think I need help [16:27:44] oh you're in meeting [16:27:45] a-team - Do we batcave for backend? [16:27:50] oh poop [16:27:51] joal, k [16:27:53] right [16:28:20] milimetric: why do you use maven (should be done for you by jenskin IIRC) [16:28:43] joal: nuria set up a hangout for it i think [16:29:03] we're in batcave fdans [16:29:11] omw [16:37:46] 10Analytics: Getting different versions of the same file - https://phabricator.wikimedia.org/T163338#3194051 (10elukey) So any Cookie header set would force a pass, meanwhile without it (like with curl etc..) the cached content is served. This is due to a cache::misc setting (where anaytics.w.o lives), that can'... [16:38:07] milimetric: --^ [16:58:31] going afk team!! [16:58:33] o/ [16:58:40] bye elukey :] [17:06:32] nuria, yt? [17:15:35] 06Analytics-Kanban: Investigate rise in IE views from Pakistan since 2015 - https://phabricator.wikimedia.org/T157404#3194491 (10Nuria) >And this is the top three paths taken by requests using that same UA string. So overwhelmingly English Wikipedia's root and main page This (most commonly) indicates a connectio... [17:26:35] ottomata: Heya, you here? [17:27:46] ya [17:27:47] heya [17:28:04] joal: hi [17:28:05] ottomata: we're having a problem with jenkins (seems similar to what we had the other day [17:28:17] ottomata: https://integration.wikimedia.org/ci/job/analytics-refinery-release/64/console [17:28:39] whaa [17:29:31] :( [17:31:05] 10Analytics: Finalize list of metrics, breakdowns, and filters for Wikistats 2.0 backend - https://phabricator.wikimedia.org/T163356#3194569 (10Milimetric) [17:31:14] k, doc updated [17:34:36] joal: i just made a change, that was really a no-op [17:34:38] try again? [17:40:13] Thanks a lot milimetric :) [17:40:18] ottomata: build started [17:40:37] ottomata: same error [17:42:14] ok, got anothier thing to try... [17:51:11] joal try again please [17:51:53] same again ottomata -- can not find the config [17:52:14] sigh [17:52:33] ok got another thing.. [18:00:53] nm, i dobn't know [18:00:55] errgggh [18:01:01] madhuvishy: you there? :) :///// [18:01:19] ottomata: hey [18:01:20] yes :) [18:01:55] sorry madhuvishy, everytime I try to use jenkins I break it :( [18:02:04] jenkins refinery source deploy is busted again, this time it look slike the same thing [18:02:10] i tried deleting and recreating credentials, etc. [18:02:15] ah [18:02:16] https://integration.wikimedia.org/ci/job/analytics-refinery-release/66/console [18:02:26] FATAL: your Apache Maven build is setup to use a config with id org.jenkinsci.plugins.configfiles.maven.MavenSettingsConfig.ArchivaCredentialsSettings but can not find the config [18:02:29] one q [18:02:34] https://integration.wikimedia.org/ci/configfiles/editConfig?id=org.jenkinsci.plugins.configfiles.maven.MavenSettingsConfig [18:02:41] has [18:02:44] ID org.jenkinsci.plugins.configfiles.maven.MavenSettingsConfig [18:02:45] and [18:02:48] name [18:02:49] ArchivaCredentialsSettings [18:02:53] in jenkins job config [18:02:58] maven: [18:02:58] goals: clean package [18:02:58] settings: [18:02:58] 'org.jenkinsci.plugins.configfiles.maven.MavenSettingsConfig.ArchivaCredentialsSettings' [18:03:11] does it know to just append the name to the id to find the job settings? [18:03:34] does the ID maybe hae to be the full name? [18:03:34] org.jenkinsci.plugins.configfiles.maven.MavenSettingsConfig.ArchivaCredentialsSetting [18:03:35] ? [18:05:16] i'm going to try to make a new settings config with the name org.jenkinsci.plugins.configfiles.maven.MavenSettingsConfig.ArchivaCredentialsSettings [18:05:27] or, id [18:06:40] joal: ok, try again? [18:06:41] ya [18:06:46] id has to be the full name [18:06:51] how did it change?!?! [18:06:58] i don't understand why that happens [18:07:02] yeah, gross [18:07:03] :( [18:07:05] may be some jenkins upgrades affecting it? [18:07:17] doesn't make a lot of sense though [18:12:42] joal: any luck? [18:20:25] was away for a minute ottomata [18:20:53] worked ottomata, madhuvishy !!! [18:20:56] Thanks a lot ! [18:21:04] back to diner, will be back after [18:22:28] weird, ok [18:22:28] ! [18:23:03] :/ [18:23:07] that's so weird [18:23:18] ottomata: i'd ask releng [18:23:20] ya [18:27:46] Hey folks! [18:28:10] I just sent a reminder of the research showcase to analytics-l from the wrong email address. Can someone approve it quick? [18:28:49] milimetric, I forgot to let you know specifically, but the core developer of WikiBrain (that big-ass index system I've been trying to pull over to wikimedia infra) is presenting at the showcase today. :) [18:28:54] In 1 minute D: [18:42:32] 06Analytics-Kanban, 07Easy: Story: VitalSignsUser selects Monthly Pageviews metric - https://phabricator.wikimedia.org/T75331#3195059 (10mforns) a:03mforns [18:44:30] 06Analytics-Kanban, 07Easy: Story: VitalSignsUser selects Monthly Pageviews metric - https://phabricator.wikimedia.org/T75331#762282 (10mforns) The fact that the monthly API is faster than the daily (contains less data) triggers a race condition that breaks Dashiki. Namely, the project colors are not assigned... [18:47:31] 06Analytics-Kanban, 07Easy: Story: VitalSignsUser selects Monthly Pageviews metric - https://phabricator.wikimedia.org/T75331#3195097 (10Nuria) mmm... does that still happens with my reafctoring change of api: https://gerrit.wikimedia.org/r/#/c/347305/6 Do let me know, doesn't seem like it would be the case [18:51:31] joal: still around? [18:51:44] yup, checking compile after diner [18:51:45] wassup? [18:52:01] would love brainbounce if not too late for you [18:52:16] ottomata: 10 minutes ;) [18:52:35] k :) [19:05:27] mforns: back at my desk... [19:05:39] mforns: just commented on ticket about monthly pageviews [19:05:59] nuria, hi! I was just pinging you in hopes that you could make it to the second part of the WS2.0 meeting [19:06:02] mforns: can you see if you can repro bug on top of my changes for api? https://gerrit.wikimedia.org/r/#/c/347305/6 [19:06:03] k [19:06:24] nuria, sure [19:06:37] mforns: if you are not doing anything else, that is [19:06:59] no no, I'm on this right now [19:10:11] milimetric: we managed to deploy refinery-source! [19:10:17] Thanks again madhuvishy :) [19:10:42] in deed, thank you madhuvishy [19:10:50] i didn't do anything :) [19:10:51] :( I failed my ops week [19:11:05] joal: did the deploy included the daily unique devices job? [19:11:10] but glad you got it deployed [19:11:18] nuria: it did, you merged it [19:11:29] right, yesterday [19:11:37] nuria: for the moment, only refinery-source [19:11:50] still refinery to go (with druid-uniques) [19:12:41] joal; okeis [19:13:13] milimetric: Did you see my comment here: https://gerrit.wikimedia.org/r/#/c/347305/5/src/app/apis/aqs-api.js [19:13:46] yes, doing reviews in a bit, late lunch, meetings ran late [19:14:04] milimetric: there is a patch needed before we deploy refinery [19:14:16] milimetric: if you don't mind, let's do that tomorrow morning together [19:14:22] joal: ok, sure [19:14:27] thanks ;) [19:14:36] gone for tonight folks :) [19:14:43] nite [19:15:11] milimetric: ok, you let me know. [19:20:15] nuria, yes! your patch fixed the problem [19:20:36] mforns: ok, taht makes sense cause patch is deterministic [19:20:43] cool [19:20:56] will make a patch on top of yours [19:21:14] mforns: so just add the fact that task is blocked on merging the other, I can probably take care of both today [19:21:23] after i talk to milimetric later [19:21:29] there are some changes needed in src/app/config.js [19:21:50] k [19:24:32] to add the monthly pageviews? [19:24:37] ^ mforns [19:24:47] nuria, yes [19:24:56] your patch fixes everything else [19:25:11] they are alredy on reportcard: https://analytics.wikimedia.org/dashboards/reportcard/#pageviews-july-2015-now [19:25:26] monthly? [19:25:28] mforns: yes [19:25:45] O.o [19:26:32] mforns: see config (sample, for desktop) : https://meta.wikimedia.org/wiki/Config:Dashiki:Sample/tabs [19:29:53] nuria, yes... [19:30:23] nuria, but metrics-by-project layout's config works a bit differently, in that the name in the json section is an identifier of the metric [19:30:33] and you can not define 2 metrics with same name [19:30:47] one with daily and one with monthly granularity [19:31:07] mforns: let me see the config [19:31:08] thus, the config.js file needs to store info about the metric [19:31:11] sure [19:31:31] nuria, look at unique devices and monthly unique devices [19:31:37] it's the same scenario [19:32:02] nuria, https://meta.wikimedia.org/wiki/Dashiki:CategorizedMetrics [19:32:12] mforns: right i remember now [19:32:18] mforns: ok [19:32:45] k, will submit the patch then [19:33:55] 06Analytics-Kanban, 13Patch-For-Review: Update DataLake History schema (`revision_is productive` --> `revision_time_to_identity_revert`) - https://phabricator.wikimedia.org/T157362#3002613 (10Nuria) @fdans: would you please add some points to this task? [19:34:17] mforns: on top of api change right? [19:34:22] nuria, yes [19:34:38] mforns: ok, will try to deploy those today if milimetric is ok with merge [19:36:32] 06Analytics-Kanban, 07Easy: Story: VitalSignsUser selects Monthly Pageviews metric - https://phabricator.wikimedia.org/T75331#3195283 (10mforns) Cool, that patch fixes the problems, but still Dashiki needs a couple lines changed in src/app/config.js. Will submit a patch on top of yours. [19:37:42] (03PS1) 10Mforns: Add support for monthly pageviews in metrics-by-project [analytics/dashiki] - 10https://gerrit.wikimedia.org/r/348990 (https://phabricator.wikimedia.org/T75331) [19:38:35] (03CR) 10Mforns: "The dependency change needs to be merged in order for this patch to work." [analytics/dashiki] - 10https://gerrit.wikimedia.org/r/348990 (https://phabricator.wikimedia.org/T75331) (owner: 10Mforns) [19:38:41] nuria, ^ [19:39:35] (03CR) 10Nuria: [V: 032 C: 032] Add support for monthly pageviews in metrics-by-project [analytics/dashiki] - 10https://gerrit.wikimedia.org/r/348990 (https://phabricator.wikimedia.org/T75331) (owner: 10Mforns) [19:43:13] I'm catching up on all these changes now [19:45:13] milimetric: ok, let me know [20:17:00] 10Analytics, 10DBA: Json_extract available on analytics-store.eqiad.wmnet - https://phabricator.wikimedia.org/T156681#3195441 (10Tbayer) Any updates on this? With the recent rollout of T153207, which introduced a JSON field to all EventLogging tables, the need for [[https://mariadb.com/kb/en/mariadb/json-func... [20:20:52] 10Analytics-EventLogging, 06Analytics-Kanban, 13Patch-For-Review: Change userAgent field to user_agent_map in EventCapsule - https://phabricator.wikimedia.org/T153207#3195457 (10Tbayer) >>! In T153207#2956196, @Milimetric wrote: > Reminder to also change the description of the userAgent field in the capsule:... [20:20:58] 10Analytics, 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice: Make banner impression counts available somewhere public - https://phabricator.wikimedia.org/T115042#3195458 (10DStrine) [20:23:26] nuria: thanks for being proactive about JSON in MariaDB earlier ( https://phabricator.wikimedia.org/T153207 )! ... wondering how we can keep this on the DBA agenda [20:23:36] 10Analytics-EventLogging, 06Analytics-Kanban, 13Patch-For-Review: Change userAgent field to user_agent_map in EventCapsule - https://phabricator.wikimedia.org/T153207#3195466 (10Milimetric) Hm, we could just edit the schema and then have a different revision than is used, but functionally the same, or we cou... [20:24:44] 10Analytics, 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice: Make banner impression counts available somewhere public - https://phabricator.wikimedia.org/T115042#3195499 (10DStrine) @Nuria fr-tech, @Jseddon and I would like to discuss this topic. We understand analytics would need to work on p... [20:25:35] 10Analytics-EventLogging, 06Analytics-Kanban, 13Patch-For-Review: Change userAgent field to user_agent_map in EventCapsule - https://phabricator.wikimedia.org/T153207#3195507 (10Nuria) Sorry, I though I had comented on this earlier. We do not need to change the capsule as type of field hasn't changed. it is... [20:27:54] (03CR) 10Milimetric: "Nuria, do you want to abandon this change now that the task is deprioritized?" [analytics/dashiki] - 10https://gerrit.wikimedia.org/r/306980 (https://phabricator.wikimedia.org/T143689) (owner: 10Nuria) [20:31:15] 10Analytics-EventLogging, 06Analytics-Kanban, 13Patch-For-Review: Change userAgent field to user_agent_map in EventCapsule - https://phabricator.wikimedia.org/T153207#3195524 (10Tbayer) >>! In T153207#3195507, @Nuria wrote: > Sorry, I though I had comented on this earlier. We do not need to change the capsul... [20:31:29] 10Analytics, 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice: Make banner impression counts available somewhere public - https://phabricator.wikimedia.org/T115042#3195527 (10Nuria) @Dstrine: banner counts are available on Pivot, see: http://bit.ly/2pgY9vN pivot is available through NDA so I bel... [20:35:36] 10Analytics, 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice: Make banner impression counts available somewhere public - https://phabricator.wikimedia.org/T115042#3195537 (10DStrine) @Nuria there are many CentralNotice admins who are not under NDA and most campaigns in CN are not for fundraising... [20:38:14] nuria: I'm looking at the change, a bit stuck on why http://localhost:5000/src/layouts/metrics-by-project/#projects=eswiki,itwiki,enwiki,jawiki,dewiki,ruwiki,frwiki/metrics=NewlyRegistered is not working. We removed NewlyRegistered and similar metrics out of vital-signs, but it shouldn't be broken, it's asking for a bad URL right now [20:38:22] 10Analytics-EventLogging, 06Analytics-Kanban, 13Patch-For-Review: Change userAgent field to user_agent_map in EventCapsule - https://phabricator.wikimedia.org/T153207#3195545 (10Tbayer) >>! In T153207#3195466, @Milimetric wrote: > Hm, we could just edit the schema and then have a different revision than is u... [20:40:17] 10Analytics, 10Fundraising-Backlog, 10MediaWiki-extensions-CentralNotice: Make banner impression counts available somewhere public - https://phabricator.wikimedia.org/T115042#3195557 (10Nuria) >We are interested in getting some data to them to judge the effectiveness of their campaigns. Obviously this needs... [20:42:53] 10Analytics, 10ChangeProp, 10EventBus, 06Services (later): Create JobQueue implementation that posts to EventBus - https://phabricator.wikimedia.org/T163379#3195566 (10Pchelolo) [20:43:10] nuria: (correction: i meant https://phabricator.wikimedia.org/T156681 above, regarding JSON in MariaDB) [20:45:05] 10Analytics, 10ChangeProp, 10EventBus, 06Services (later), 15User-mobrovac: Support posting to multiple JobQueues simultaneously - https://phabricator.wikimedia.org/T163380#3195595 (10Pchelolo) [22:06:52] 10Analytics, 10EventBus, 06Services (watching): Delete stale kafka topics - https://phabricator.wikimedia.org/T163392#3195930 (10Pchelolo) [22:09:42] (03CR) 10Milimetric: [C: 04-1] "small comma bug, but big style suggestion (I'm not gonna lie, I hope you like it but I'm not sure :)))" (034 comments) [analytics/dashiki] - 10https://gerrit.wikimedia.org/r/348227 (https://phabricator.wikimedia.org/T162482) (owner: 10Mforns) [22:11:57] 10Analytics, 10EventBus, 06Services (watching): Delete stale kafka topics - https://phabricator.wikimedia.org/T163392#3195947 (10Pchelolo) To enable topic deletion we need to put `delete.topic.enable=true` to the kafka `server.properties` and restart instances. [23:55:53] 10Analytics: Fix iOS vs. Android detection in mobile apps uniques tables - https://phabricator.wikimedia.org/T163403#3196212 (10Tbayer)