[01:08:36] (PS2) Joal: Improve sorted-json job [analytics/wikihadoop] - https://gerrit.wikimedia.org/r/251311 (https://phabricator.wikimedia.org/T114359) [09:35:56] Analytics-Tech-community-metrics, Developer-Relations, DevRel-November-2015: Check whether it is true that we have lost 40% of (Git) code contributors in the past 12 months - https://phabricator.wikimedia.org/T103292#1807248 (Aklapper) >>! In T103292#1800559, @Dicortazar wrote: > I've removed the comme... [10:45:28] (CR) Joal: [V: 1] "Good for me, I let Nuria merge." [analytics/refinery/source] - https://gerrit.wikimedia.org/r/253045 (https://phabricator.wikimedia.org/T118592) (owner: BryanDavis) [10:46:00] (CR) Joal: [C: 1] "Looks good to me, not tested though." [analytics/refinery/source] - https://gerrit.wikimedia.org/r/253046 (https://phabricator.wikimedia.org/T118592) (owner: BryanDavis) [10:50:31] (CR) Joal: "CRed but not actually merged --> Merging" [analytics/refinery] - https://gerrit.wikimedia.org/r/221280 (owner: Madhuvishy) [11:24:40] Analytics-Tech-community-metrics, DevRel-November-2015: Many 404s and graphs not displayed on gerrit_review_queue.html - https://phabricator.wikimedia.org/T118461#1807413 (Dicortazar) As far as I remember, you modified that HTML source code and moved the graphs section to the scr.html page. Is this still... [11:28:08] Analytics-Tech-community-metrics, DevRel-November-2015: Many 404s and graphs not displayed on gerrit_review_queue.html - https://phabricator.wikimedia.org/T118461#1807428 (Lcanasdiaz) I'm afraid we have some many 404 errors due to the implementation of the Loader.js we're using. It's not elegant at all b... [11:51:38] joal, yt? [11:52:02] Hey mforns [11:52:07] Howdy? [11:52:12] hello, how are you? [11:52:15] I'm good [11:52:38] Kind of hangover without having been drunk ... [11:52:46] mmm [11:53:01] how is your family in paris? [11:53:27] No direct loss for me [11:53:54] But friends of friends ... [11:54:13] mm [11:56:19] so terrible [11:56:26] Very sad [11:58:00] I have been struggling all weekend about sending an email to wmf-all about how I felt supported in the fact of working for the foundation [11:58:19] aha [11:58:36] I feel it's a very strong chance, in times of darkness, to be surrounded by light [11:59:05] understand [12:00:21] But then there have all the messages about strategy, and all, and I didn't wanted to add to the weekend noise [12:00:30] So I share now :) [12:00:40] Sorry, you're the one here to listen ;) [12:00:44] yes... [12:01:44] I'm not sorry :] you can maybe send to the analytics-internal [12:02:31] I'll tell our fellows when they are in (like that I'll tell 5 or 6 times, and hopefully feel better each time) [12:03:23] What about you mforns ? [12:03:53] or maybe send to wmf-all anyway, this is more important than the strategy controversy [12:03:58] I'm ok :] [12:04:21] hm mforns :) [12:04:26] Thanks for the support [12:04:38] hey I wanted to ask you about the pv api data [12:04:47] since when do we have data> [12:04:48] ? [12:05:02] I mean, when is the first day we have records? [12:05:10] you mean, how far back have we backfilled? [12:05:23] ha :) [12:05:25] yes [12:05:39] well, how far do we plan to backfill, yes [12:05:43] So, there is a test record set on 1970-01-01, but it's a test one [12:05:54] xD [12:06:11] Currently I am finishing backfilling september (only a few days of per-article left) [12:06:35] cool, are there any plans to backfill more? What should I put in the docs? [12:06:55] And if it works well, the plan is to go back to may, but I'd rather wait and see because we don't know how the system will behave with much data [12:07:17] ok [12:07:43] and one other question: we fill in the past day at what time? [12:09:14] Every daily job is launched at midnight for the day before, and the ones that take long (top and per-article) finish usually before 5am) [12:09:36] UTC right? [12:09:56] UTC indeed mforns [12:10:01] ok [12:11:25] and joal, I guess the weekly and monthly tables are also loaded the same way, starting at 00:00 UTC of the day after the week/month ends, no? [12:11:34] correct mforns [12:11:39] cool, thanks! [12:11:47] For the moment, only the per-project monthly works [12:11:54] I see [12:11:58] I struggle with top-monthly (hive doesn't cope with the job) [12:13:06] so for now TOP is only daily [12:13:41] correct [12:13:49] ok [12:14:00] The thing is ready to work for monthly, but no data yet [12:14:28] aha [12:14:56] joal, I had one idea the other day [12:15:02] regarding top [12:15:02] yup ? [12:15:25] one prequestion :] [12:15:31] what is the size of that table? [12:15:39] is it the smallest one? [12:15:40] ? [12:15:43] top [12:15:48] top-daily [12:15:53] we go for the wmf.pageview_hourly [12:16:01] no, I mean in cassandra [12:16:06] Ah [12:16:28] it's a small one (not the smallest, but not very big) - let me check [12:18:07] mforns: 2.5 month of daily top in cassandra --> 1.8Gb [12:18:19] ok [12:18:53] 2.5 month of hourly, daily and monthly per-project aggregate in cassandra --> 210 Mb [12:19:28] so the idea is, instead of storing top 1000, store top 100000 (and continue serving top1000 only) [12:19:58] and then try to approximate the monthly top aggregating the daily ones [12:20:04] Almost 2.5 month of daily per-article in cassandra (missing 8 days) --> 873 Gb [12:20:14] aha [12:20:41] Yeah, the page_title dimensions makes the explosion for sure [12:20:48] mforns: ould be feasible [12:21:33] we might want to know, what's the probability of having a close enough approximation of a montlhy top using daily aggregation [12:21:37] mforns: If the original job still doesn't work in hive, I'll try that approach in hive [12:22:04] ah, computing it in hive, makes sense too [12:22:11] cause there'll be no other way to have the data if the core computation fails [12:22:20] aha [12:22:50] My guess would be: better to have a 99.9% right thing available than a 100% right thing unavailable [12:22:58] hehehe, yes [12:22:58] :) [12:23:09] 99.9% would be awesome! [12:23:29] I'll double check with weekly [12:23:29] joal, when do we load the hourly data? [12:23:42] at the end of each hour :) [12:23:45] ok [12:24:11] or more precisely, after the data is made avalable in hive ( after refine and pageview extraction) --> end of hour + 40mins [12:24:22] oh! ok [12:24:35] so this will happen also for daily data, right> [12:24:35] ? [12:25:09] hm, not sure what you mean [12:25:32] so, it depends on data type: daily per-project - End of day + 40 mins [12:25:34] the daily data also waits for refine and extraction [12:25:41] ok [12:26:33] daily top - End of day + 2h [12:26:40] aha [12:27:58] daily per-article - end of day + 4h [12:28:05] roughly [12:28:05] ok [12:28:09] thx [12:28:17] np mforns, thank you for the docs ! [12:31:50] joal, I have to leave, they're closing the library :'( [12:31:56] see you in a while [12:32:09] np mforns, I'll be off in a few hours as well, and the nback [12:32:22] I'll follow your advice, and will send my message to wmf [13:04:56] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL: CRITICAL: 26.67% of data above the critical threshold [30.0] [13:06:47] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK: OK: Less than 25.00% above the threshold [20.0] [13:22:46] Analytics-Tech-community-metrics, JavaScript: Failed to load resource: the server responded with a status of 404 (Not Found) - https://phabricator.wikimedia.org/T65061#1807549 (Aklapper) >>! In T118461#1807413, @Dicortazar wrote: > Regarding to the 404 errors, this is a general issue of the dashboard. Thi... [13:25:47] Analytics-Tech-community-metrics, DevRel-November-2015: Many 404s and graphs not displayed on gerrit_review_queue.html - https://phabricator.wikimedia.org/T118461#1807555 (Aklapper) Open>Invalid >>! In T118461#1807413, @Dicortazar wrote: > As far as I remember, you modified that HTML source code and... [14:34:54] Analytics-Kanban, EventBus, Services, Patch-For-Review: Package EventLogging and dependencies for Jessie - https://phabricator.wikimedia.org/T118578#1807668 (Ottomata) Tornado 4.2 is available already on the cluster via jessie-backports. I think that's it for dependencies that need packaged. [14:41:26] Hi all! Can anyone direct me towards the numbers / dashboards for active editors? [14:41:49] Analytics-Backlog, Analytics-Cluster, Improving access, Research-and-Data: Hashed IP addresses in refined webrequest logs - https://phabricator.wikimedia.org/T118595#1807697 (Ottomata) [14:46:20] ls [14:46:27] heh... [14:56:46] Is one of y’all Kevin? Or do I need to wait a few hours for him? [14:57:02] hey andrewbogott [14:57:14] Kevin goes by kevinator [14:57:31] And he’s in pst? [14:57:37] He is [14:58:03] ok, will check back later on then, thanks [14:58:38] np andrewbogott :) [14:58:41] Analytics-Tech-community-metrics, DevRel-November-2015: Make GrimoireLib display *one* consistent name for one user, plus the *current* affiliation of a user - https://phabricator.wikimedia.org/T118169#1807716 (Dicortazar) Reassigning this to @Lcanasdiaz as he'll work on this task. [14:58:50] Analytics, Traffic, operations, Patch-For-Review: Replace Analytics XFF/client.ip data with X-Client-IP - https://phabricator.wikimedia.org/T118557#1807718 (Ottomata) I'd like both @JAllemandou and @leila to review this and give the go ahead to remove `ip` and `x_forwarded_for`. If they say cool,... [14:58:59] Analytics-Tech-community-metrics, DevRel-November-2015: Make GrimoireLib display *one* consistent name for one user, plus the *current* affiliation of a user - https://phabricator.wikimedia.org/T118169#1807719 (Dicortazar) a:Dicortazar>Lcanasdiaz [15:05:06] Analytics, CirrusSearch, Discovery, operations: Delete logs on stat1002 in /a/mw-log/archive that are more than 90 days old. - https://phabricator.wikimedia.org/T118527#1807738 (Dzahn) [15:05:08] Analytics-Tech-community-metrics, DevRel-November-2015: Make GrimoireLib display *one* consistent name for one user, plus the *current* affiliation of a user - https://phabricator.wikimedia.org/T118169#1807740 (Lcanasdiaz) I guess you want use to include this information on the people.html or .. do you wa... [15:05:22] Analytics, CirrusSearch, Discovery, operations, audits-data-retention: Delete logs on stat1002 in /a/mw-log/archive that are more than 90 days old. - https://phabricator.wikimedia.org/T118527#1802620 (Dzahn) [15:06:53] Analytics-Tech-community-metrics, DevRel-November-2015: Explain / sort out / fix SCR repository number mismatch on korma - https://phabricator.wikimedia.org/T116484#1807745 (Dicortazar) This seems to be an error when counting activity in empty repositories. There may exist Gerrit repositories with no acti... [15:07:31] Analytics-Tech-community-metrics, DevRel-November-2015: Explain / sort out / fix SCR repository number mismatch on korma - https://phabricator.wikimedia.org/T116484#1807747 (Dicortazar) a:Dicortazar>Lcanasdiaz [15:08:30] Analytics-Tech-community-metrics, DevRel-November-2015: Explain / sort out / fix SCM repository number mismatch on korma - https://phabricator.wikimedia.org/T116483#1807755 (Dicortazar) Similar issue as {T116484}. Re-assigning this to @Lcanasdiaz. [15:08:45] Analytics-Tech-community-metrics, DevRel-November-2015: Explain / sort out / fix SCM repository number mismatch on korma - https://phabricator.wikimedia.org/T116483#1807760 (Dicortazar) a:Dicortazar>Lcanasdiaz [15:09:18] Analytics-Tech-community-metrics, DevRel-November-2015: Fix 404s (VizGrimoireJS entirely broken) on korma's mediawiki.html - https://phabricator.wikimedia.org/T118167#1807764 (Dicortazar) a:Dicortazar>Lcanasdiaz [15:09:38] Analytics-Tech-community-metrics, DevRel-November-2015: Fix 404s (VizGrimoireJS entirely broken) on korma's mediawiki.html - https://phabricator.wikimedia.org/T118167#1793181 (Dicortazar) Reassigning this to @Lcanasdiaz. This is a clear product issue. [15:10:52] Analytics-Tech-community-metrics, DevRel-November-2015: Many profiles on profile.html do not display identity's name though data is available - https://phabricator.wikimedia.org/T117871#1807768 (Dicortazar) Re-assigning this to @Lcanasdiaz. This is a bug in the product. [15:11:08] Analytics-Tech-community-metrics, DevRel-November-2015: Many profiles on profile.html do not display identity's name though data is available - https://phabricator.wikimedia.org/T117871#1807770 (Dicortazar) a:Dicortazar>Lcanasdiaz [15:11:59] Analytics-Tech-community-metrics, DevRel-November-2015: Legend for "review time for reviewers" and other strings on repository.html - https://phabricator.wikimedia.org/T103469#1807780 (Dicortazar) I guess @Lcanasdiaz can provide a better answer than me :). [15:12:16] Analytics-Tech-community-metrics, DevRel-November-2015: Legend for "review time for reviewers" and other strings on repository.html - https://phabricator.wikimedia.org/T103469#1807785 (Dicortazar) a:Dicortazar>Lcanasdiaz [15:13:22] Analytics-Tech-community-metrics, DevRel-December-2015: Time axis on repository.html only displays two months, repeated several items - https://phabricator.wikimedia.org/T115872#1807788 (Dicortazar) a:Dicortazar>Lcanasdiaz [15:14:36] Analytics-Tech-community-metrics, DevRel-December-2015: Empty "subject" and "creator" fields for mailing list thread on mls.html - https://phabricator.wikimedia.org/T116284#1807794 (Dicortazar) This seems to be an error in unicode as you suggest. [15:16:55] Analytics-Tech-community-metrics, DevRel-November-2015: Affiliations and country of resident should be visible in Korma's user profiles - https://phabricator.wikimedia.org/T112528#1807807 (Dicortazar) a:Dicortazar>Lcanasdiaz [15:19:03] Analytics-Tech-community-metrics, DevRel-December-2015: Contributor pages which show user name but not any other data should include an explanation - https://phabricator.wikimedia.org/T58111#1807831 (Dicortazar) This weird issue is related to the fact that not all of the developers are listed, but the mos... [15:19:19] Analytics-Tech-community-metrics, DevRel-December-2015: Contributor pages which show user name but not any other data should include an explanation - https://phabricator.wikimedia.org/T58111#1807832 (Dicortazar) a:Dicortazar>Lcanasdiaz [15:36:00] morning y'all [15:36:15] good morning milimetric :] [15:37:35] hey mforns I have one more little tweak for you on the comparison thing [15:37:54] it seems that there's some sort of media query on the article auto-complete input box [15:38:08] and it resizes it to be super tiny on smaller screens, one sec I'll try to find it [15:39:44] mornin [15:41:09] mforns: ok, go tit [15:41:12] *got it [15:41:30] try opening dev tools and the reloading the page with dev tools open [15:41:36] milimetric, yes? [15:41:41] the select2 box should be tiny now [15:41:55] because its select2-container gets for some weird reason a width: 47px [15:42:06] so instead, just do this: .select2-container { [15:42:06] width: 100%!important; [15:42:06] } [15:42:10] mmm [15:42:14] let me try [15:42:41] the 100% should just always work there [15:42:43] you reproduce the error just by opening the dev tools? [15:42:47] no [15:42:53] it's on loading of the page apparently [15:43:03] because it's probably JS [15:43:10] I don't get that event when reloading [15:43:26] hm, I'll show you a screenshot (you in FF or Chrome?) [15:43:29] oh! now I get that [15:43:32] ok [15:43:40] yeah, JS is messing with the style and there must be a bug [15:43:44] also! Another bug. [15:43:46] I narrowed the screen horizontally [15:43:51] right [15:44:08] oh right, sorry, my dev tools open on the right [15:44:15] Does Dario appear in here? [15:44:19] because I often look to see what pages look like width-wise [15:44:26] Reedy: usually he's in here yea, but also -research [15:44:31] milimetric, right makes sense [15:44:59] mforns: ok, other subtle bug is, if you change the articles, it seems like the old graphs aren't being destroyed [15:45:07] so I couldn't reproduce this on my laptop [15:45:13] but on my ipad there's this crazy behavior [15:45:21] aha [15:45:24] if I select a few different pages, I sometimes get a ghost graph [15:45:34] and when I scroll and zoom it shows up and replaces the graph for a second [15:45:36] milimetric, this is a bug in select2 [15:45:40] sometimes it gets stuck on the ghost graph [15:45:51] mforns: yeah, i know, they screwed up the logic on page load [15:45:52] yes... [15:46:13] if they want to mess with the size they really have to bind to window.resize [15:46:38] this bug was worse indeed, but in the end I managed to patch it almos 100%, but I didn't test safari... [15:46:54] ah, that must be it [15:47:01] ok, then, maybe don't worry about it [15:47:12] it don't have to be perfect :) [15:47:25] you know, I like dygraphs more and more despite how awfully crude it is [15:47:30] at least it's rock solid and super fast [15:47:35] milimetric, if you look at resetArticleSelector function, that's what it does. [15:47:43] gotcha, /me checks [15:47:59] in theory the destroy() should be enough [15:48:08] but it doesn't work by itself [15:48:33] after some... googling, I found out this worked for chrome... in almost all occasions. [15:49:05] maybe you have another idea? [15:49:05] right... grr they must be setting up some temporary state somewhere on the page and it's not being wiped properly [15:49:14] seems so [15:49:17] nah, I'm totally sure I could find it but it's not worth my time :) [15:49:24] hehe [15:50:02] I swear, this is the most common bug these days, people don't clean up properly. Everyone's writing their tools just for that one-time demo and they don't productionize them properly [15:50:16] aha [15:51:39] milimetric: amen [15:52:35] psh I say "people" because of course I'm a saint and all my code is born production-ready :P [16:01:33] milimetric, I updated the gist with the width 100% !important [16:03:05] sweet, thx mforns [16:03:14] thanks for spotting the bug [16:03:23] i fixed limn1 over the weekend, you wanna deploy there and pick a name for the demo? [16:03:31] sure [16:04:54] milimetric, I think we all agree with: analytics.wmflabs.org/demo/pageview-api [16:05:18] ok, sweet. I'll set it up [16:05:39] can I do it with you? [16:10:28] Analytics-EventLogging: Many duplicate events collected from client side javascript - https://phabricator.wikimedia.org/T101867#1808066 (EBernhardson) Open>Resolved a:EBernhardson [16:24:57] sorry mforns I haven't done it yet, so yes! let's go [16:24:59] to the batcave [16:25:01] ok [16:45:29] Analytics-Tech-community-metrics, DevRel-November-2015: Make GrimoireLib display *one* consistent name for one user, plus the *current* affiliation of a user - https://phabricator.wikimedia.org/T118169#1808175 (Aklapper) @Lcanasdiaz: == Username == I expect a consistent single name displayed across kor... [16:53:55] milimetric, btw the new docs are here: https://wikitech.wikimedia.org/wiki/Analytics/Pageview_API [16:55:04] oh mforns I had moved the page you made here: https://wikitech.wikimedia.org/wiki/Analytics/AQS/Pageview_API [16:55:12] because we were saying we'd put it under AQS [16:55:19] oh! yes, I was looking for it hehe [16:55:20] ok [16:55:34] makes sense will update this one [16:55:36] sorry, I figured you'd find the redirect [16:56:49] mmm, don't know why I didn't get there.. a typo probably, it worked now for me [16:57:00] milimetric, can you please remove the page I just created? [16:57:16] sure [16:57:20] I will update the one you passed me [17:02:26] mforns: helloo, did you filed that ticket for google code in? [17:02:35] hi nuria, standup? [17:02:37] nuria, sorry forgoy [17:02:40] forgot [17:02:45] will do [17:02:45] milimetric: yesss [17:12:16] Analytics-Kanban: Create celery chain or other organization that handles validation and computation {kudu} [8 pts] - https://phabricator.wikimedia.org/T118308#1808247 (madhuvishy) a:madhuvishy [17:22:36] Analytics-Kanban: Pageview API documentation for end users {slug} - https://phabricator.wikimedia.org/T117226#1808274 (mforns) [17:22:38] Analytics-Backlog: fix 'day' description in RESTBase Pageview API - https://phabricator.wikimedia.org/T117502#1808273 (mforns) [17:22:55] Analytics-Backlog: fix 'day' description in RESTBase Pageview API - https://phabricator.wikimedia.org/T117502#1808276 (mforns) [17:22:56] Analytics-Kanban: Pageview API documentation for end users {slug} - https://phabricator.wikimedia.org/T117226#1769137 (mforns) [17:23:24] Analytics-Backlog: fix 'day' description in RESTBase Pageview API - https://phabricator.wikimedia.org/T117502#1775890 (mforns) [17:23:25] Analytics-Kanban: Pageview API documentation for end users {slug} - https://phabricator.wikimedia.org/T117226#1769137 (mforns) [17:24:12] Analytics-Kanban: Pageview API documentation for end users {slug} - https://phabricator.wikimedia.org/T117226#1769137 (mforns) [17:24:13] Analytics-Backlog: fix 'day' description in RESTBase Pageview API - https://phabricator.wikimedia.org/T117502#1808286 (mforns) Open>Resolved a:mforns Already fixed in parent task. [17:32:25] (PS3) Joal: Improve sorted-json job [analytics/wikihadoop] - https://gerrit.wikimedia.org/r/251311 (https://phabricator.wikimedia.org/T114359) [17:34:20] milimetric: did you see twitter dashboard: https://analytics.twitter.com/user/pantojacoder/home [17:38:26] (CR) DCausse: [C: -1] "I'm having issues when I try to run camus with this schema and the kafka topic used in production:" [analytics/refinery/source] - https://gerrit.wikimedia.org/r/252958 (https://phabricator.wikimedia.org/T118570) (owner: DCausse) [17:38:58] Analytics-Tech-community-metrics, Gerrit-Migration: Make MetricsGrimoire/korma support gathering Code Review statistics from Phabricator's Differential - https://phabricator.wikimedia.org/T118753#1808364 (Aklapper) NEW [17:45:27] Analytics-Kanban: Pageviews per country wikistats report. Feed data from hive {lama} [13 pts] - https://phabricator.wikimedia.org/T118323#1808405 (Milimetric) [17:47:47] Analytics-Backlog, Analytics-Cluster, Analytics-Kanban: https://yarn.wikimedia.org/cluster/scheduler should be behind ldap - https://phabricator.wikimedia.org/T116192#1808413 (Milimetric) a:Ottomata [17:48:46] Analytics-Kanban, Analytics-Wikistats, Patch-For-Review: Send email out to community notifying of change {lama} [1 pts] - https://phabricator.wikimedia.org/T115922#1808414 (madhuvishy) a:Tnegrin>Nuria [17:49:45] Analytics-Backlog, Analytics-EventLogging, Analytics-Kanban: More solid Eventlogging alarms for raw/validated {oryx} [8 pts] - https://phabricator.wikimedia.org/T116035#1808421 (Milimetric) [18:28:39] Analytics-Kanban, EventBus: Move EventLogging/server to its own repo and set up CI - https://phabricator.wikimedia.org/T118761#1808616 (Ottomata) NEW a:Ottomata [18:28:45] Analytics-EventLogging, Analytics-Kanban, EventBus: Move EventLogging/server to its own repo and set up CI - https://phabricator.wikimedia.org/T118761#1808624 (Ottomata) [18:36:14] twitter dashboard's so cool, nuria [18:36:27] one of the biggest projects that I can't believe is not getting traction at WMF is a user profile [18:36:55] like, ok everyone, let's please stop pretending we're in 1995 and start letting our users have a proper profile page that they don't have to hack myspace style [18:37:12] sorry, myspace was way advanced compared to our user pages. I meant geocities-style [18:42:40] agreed milimetric [18:42:46] user profile would really be cool [18:43:03] We have, or had, an extension that did just that! It was implemented poorly and no one liked it. [18:46:20] joal, were you able to get the JSON job restarted? [18:46:29] If not, I have a crazy idea. [18:51:12] halfak: I restarted the stuff indeed [18:51:30] halfak: gone for diner, will be back after for a few minutes [18:51:45] Cool. Thank you :) [19:05:55] milimetric: juas to profile pages [19:06:24] milimetric: i should close stuff on done column right? [19:08:20] nuria: yes [19:10:27] also, for those that don't know like me: "Juas is a form of laughter on the Internet, something that isn't extremely funny, but you don't have an emoticon to express it" [19:38:21] Hey milimetric. Remember that email you sent out with an NPR show that had a Wikipedia quiz? [19:39:57] halfak: job going [19:40:03] halfak: what was your idea? [19:40:30] Analytics-Backlog, Analytics-EventLogging: Update Eventlogging jrm tests so they include userAgent into capsule - https://phabricator.wikimedia.org/T118770#1808872 (Nuria) p:Triage>Normal [19:40:46] Do the JSON conversion in streaming with python and a basic text input format. [19:40:58] Analytics-Backlog: Traffic Breakdown Report - Browser Major Minor Version {lama} - https://phabricator.wikimedia.org/T115590#1808879 (Nuria) [19:40:59] So, I have a utility that converts a stream of XML to JSON. [19:41:14] Analytics-Kanban: Pageview API showcase App {slug} - https://phabricator.wikimedia.org/T117224#1808880 (Nuria) Open>Resolved [19:41:14] halfak: very feasible ! [19:41:20] If we force the mapper to *not* split the input files, then we can guaranteed that each utility/streamer gets a from XML doc. [19:41:28] Analytics-Backlog: Traffic Breakdown Report - Client OS Major Minor Version {lama} - https://phabricator.wikimedia.org/T115591#1808882 (Nuria) [19:41:30] Analytics-Kanban: Understand the Perl code for Client OS Major Minor Version report {lama} - https://phabricator.wikimedia.org/T117246#1808881 (Nuria) Open>Resolved [19:41:45] This would give me more power over changes to the JSON schema and stuff. But would also likely result in a performance hit. [19:42:07] Analytics-Kanban, Analytics-Wikistats, Patch-For-Review: Feed Wikistats traffic reports with aggregated hive data {lama} [21 pts] - https://phabricator.wikimedia.org/T114379#1808884 (Nuria) [19:42:09] Analytics-Kanban, Analytics-Wikistats, Patch-For-Review: Send email out to community notifying of change {lama} [1 pts] - https://phabricator.wikimedia.org/T115922#1808883 (Nuria) Open>Resolved [19:42:21] Analytics-Kanban: Bring wikimetrics staging up to date {dove} [1 pts] - https://phabricator.wikimedia.org/T118484#1808885 (Nuria) Open>Resolved [19:42:38] Analytics-Kanban: Improve loading Analytics Query Service with data {slug} [5 pts] - https://phabricator.wikimedia.org/T115351#1808887 (Nuria) Open>Resolved [19:42:48] Analytics-Kanban: Update Cassandra loading job - per-project [5 pts] {slug} - https://phabricator.wikimedia.org/T118447#1808888 (Nuria) Open>Resolved [19:43:50] Analytics-Kanban: Update cassandra monthly top job [3 pts] {slug} - https://phabricator.wikimedia.org/T118448#1808895 (Nuria) Open>Resolved [19:43:59] halfak: performace because of less splits parallel treatment, and also more because of not sorting being done [19:44:12] Analytics-Kanban, RESTBase, Services, RESTBase-API: configure RESTBase pageview proxy to Analytics' cluster {slug} [34 pts] - https://phabricator.wikimedia.org/T114830#1808901 (Nuria) Open>Resolved [19:46:40] Analytics-Kanban, RESTBase, Services, RESTBase-API: configure RESTBase pageview proxy to Analytics' cluster {slug} [34 pts] - https://phabricator.wikimedia.org/T114830#1808905 (mobrovac) Resolved>Open This hasn't been resolved yet as we haven't made any progress on domain-specific URI paths. C... [19:46:54] nuria: :P [19:48:33] joal, less splits [19:48:47] We'd still want to sort in the reduce. [19:50:08] Analytics-EventLogging, Deployment-Systems, Release-Engineering-Team, Scap3: Move EventLogging service to scap3 - https://phabricator.wikimedia.org/T118772#1808930 (demon) NEW [19:50:26] ok halfak [19:50:55] Anyway, we should let this run finish. [19:51:46] halfak: a bit of a perf hit because of splits, but not so much (177 files, currently 196 in parallel) [19:52:06] (CR) DCausse: "In fact I'm afraid that our chain does not support schema evolution." [analytics/refinery/source] - https://gerrit.wikimedia.org/r/252958 (https://phabricator.wikimedia.org/T118570) (owner: DCausse) [19:52:18] We've got that one massive page that is probably our biggest concern either way. [19:52:20] I could probably augment parallelism (less memory per map), but since the time constraint comes from the huge last mapper, not sure we'd win [19:52:29] Yeah [19:52:30] yup [19:52:32] :D [19:58:42] Analytics-Kanban: Reformat pageview API responses to allow for status reports and messages {slug} - https://phabricator.wikimedia.org/T117017#1808981 (Nuria) Open>Resolved [19:58:42] On the other hand: We know we work only full pages in mappers, and we want to sort inside pages --> We could sort inside mappersb [19:58:43] halfak: It would involve tricky code since the items surelly don't fit in memory, but would be feasible [19:58:43] Why not just do that in the reduce? [19:58:43] Will our file-sort be faster than hadoop's file-sort? [19:58:44] It would prevent us to have to write the value for the reducer, which involve spilling, therefore read xml - spill json - read back spilled json - write json for reducer - read json in reducer - write for sort - read for sort [19:58:45] halfak: --^ [19:58:45] We earn a bit on IOs [19:58:45] "read back spilled json - write json for reducer" Wat [19:58:46] Seems like that is at least one unnecessary step. [19:58:46] Isn't that the step where sorting happens? [19:58:46] It's not like the reducer would do the sorting. [19:58:47] The concern is not to sort in the reducer or in the mapper [19:58:47] It's about the IO involved in actually materialising the thing [19:58:48] (CR) Madhuvishy: "Yeah, since we don't have a schema repo that handles versions of schemas, the assumption when we wrote this was that there was no notion o" [analytics/refinery/source] - https://gerrit.wikimedia.org/r/252958 (https://phabricator.wikimedia.org/T118570) (owner: DCausse) [19:59:10] joal, yeah. I'm just trying to reason out where the loss is, but I should just take your word for it. [19:59:20] The thing I worry about is adding complexity. [19:59:31] halfak: It would, for sure [19:59:34] (CR) Nuria: ">In fact I'm afraid that our chain does not support schema evolution." [analytics/refinery/source] - https://gerrit.wikimedia.org/r/252958 (https://phabricator.wikimedia.org/T118570) (owner: DCausse) [19:59:36] When hadoop already has this fancy pantsy file sorty thingy. [19:59:44] dcausse, madhuvishy : about schema evolution [20:00:12] nuria: yeah [20:00:47] halfak: yes [20:00:50] dcausse: all data needs to validate to latest, which implies that we only support changes on schema that are backwards compatible. Hopefully this makes sense, as it is a known restriction of our system [20:00:51] halfak: if (hopefully) the job I have lauched works, I'll show you the final counters and it'll be easier to understand [20:00:52] npr show [20:01:10] joal, cool [20:01:28] milimetric, cool! So, I've been talking to Trevor about doing something like that for allstaff. [20:01:37] Want to host a Wikipedia gameshow with me? [20:01:47] nuria: in fact without a schema repository it's impossible to make any evolution even the one that "backward compatible" :( [20:02:07] halfak: ofc, that'd be fun :) [20:02:43] dcausse: mmm.. if every schema is a super set of prior with only "additional" fields it should be possible [20:02:57] Cool. I don't have anything really figured out yet, but Trevor wants to give us a 30 minute slot. [20:03:01] nuria: the change I made is backward compat (new field with default value). But when reading messages from kafka the avro decoder needs to know both [20:03:02] dcausse: sorry, "non required", rather than addituonal [20:03:05] Could do full audience or have contestants. [20:03:08] *additional [20:03:11] They might provide a prize for us too. [20:03:28] * halfak kinda wants to do price is right style contestant selection. [20:05:13] dcausse: it would if it was using schema versions, but it is not, correct? [20:05:50] dcausse: it should be able to validate data (old or new) to latest schema, am i missing something? [20:06:00] dcausse: seems like i am... [20:06:09] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [30.0] [20:07:04] nuria: I thought also, but testing in JAVA, it fails if you do not provide the writerSchema (schema used to produce the binary) and the expected schema (the new schema) [20:08:00] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK: OK: Less than 25.00% above the threshold [20.0] [20:10:02] nuria: see Avro section here: https://martin.kleppmann.com/2012/12/05/schema-evolution-in-avro-protocol-buffers-thrift.html [20:10:35] dcausse: looking, sorry, my wi fi today is terrible, on and off [20:11:18] dcausse: really? i thought you could use reader schema, and as long as it has been evolved properly, it will work with binary data from writer schema [20:11:25] since....in files the writer schema should be present [20:11:53] ottomata: in fact no :( [20:12:13] you need both: the writer schema and reader schema if they are not the same :( [20:12:31] a-team, I leave for tonight [20:12:39] laters joal [20:12:40] later joal, have a good night [20:12:45] Hopefully some more news tomorrow ! [20:12:48] dcausse: isn't the writer schema in the binary file? [20:13:03] which binary file? [20:13:19] the files stored in hdfs? [20:14:01] yes [20:14:09] so in hive it works [20:14:26] the problem is when reading message out of kafka [20:14:33] ah [20:14:34] yes [20:14:36] ah [20:14:37] dcausse: what i do not get is how the readerschema and writerschema cannot be same in our case [20:14:50] ciao joal [20:14:55] because you can't change at the same time both [20:15:33] dcausse: our decoder overrides the getSchema method right? [20:15:49] dcausse: always returning latest, as we do not pass schema version with message [20:15:53] dcausse: correct? [20:16:38] dcausse: in your change though [20:16:44] dcausse: the field is not optional [20:16:58] dacause: and thus older data that does not have it [20:17:07] dcausse: will not validate [20:17:23] I used "default" I think [20:17:35] dcausse: i *think* that doesn't make it optional [20:17:50] hmmm... let me check [20:17:52] dcausse: "By making a union with the null type (which is simply encoded as zero bytes) you can make a field optional." [20:18:11] dcausse: ya default doesn't mean what we think it should in avro [20:18:18] it's.. confusing [20:18:31] agree it is very confusing there :/ [20:18:40] dcausse, madhuvishy : super confusing, so much that i guess i forgot [20:18:50] dcausse, madhuvishy : let me re-read [20:18:55] to be really optional, you need {null, string}, default: null [20:18:57] or something like that [20:19:09] yup i think so [20:19:13] union {null, string} default: null [20:19:30] yes but this is a different schema [20:19:38] so the problem will remain the same no? [20:20:00] PROBLEM - Packetloss_Average on analytics1026 is CRITICAL: packet_loss_average CRITICAL: 8.22282191919 [20:20:24] yeah i think if you made data that does not have the optional type defined like that [20:20:35] ottomata: does that --^ means too much pressure on hadoop ? [20:20:42] joal: no. [20:20:45] that is a udp2log related thing [20:20:51] ok cool :) thx ! [20:20:53] :) [20:21:13] I am trying to be carefull with the bacjfilling and all [20:21:16] ottomata: --^ [20:21:19] cool :) [20:21:20] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [30.0] [20:21:50] hm, so dcausse i am not aware of the situation, you have binary data that was written with a field that had just type union {null, string} (or whatever), but no default [20:22:09] or, no you are talking about reading from kafka dcausse? [20:22:40] ottomata: my problem is reading from kafka and how to handle schema changes [20:22:56] schema changes are perfectly handled by hive automagically [20:22:59] RECOVERY - Packetloss_Average on analytics1026 is OK: packet_loss_average OKAY: 1.3934430303 [20:23:10] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK: OK: Less than 25.00% above the threshold [20.0] [20:23:45] ottomata: but when reading from kafka it fails if the schema used to produce is not the same as the schema used to read [20:23:58] dcausse: in your case try to make your new field truly optional (using a union) [20:24:15] dcausse: the decoder doesn't know of schema versions as it is now [20:24:23] it is trying to validate to latest, [20:25:03] nuria: reading how avro binary format is encoded I'm afraid it won't be possible :( [20:25:10] I'll test [20:25:21] dcausse: in our case, we do not use the schema id at all, see: [20:28:45] https://github.com/wikimedia/analytics-refinery-source/blob/master/refinery-camus/src/main/java/org/wikimedia/analytics/refinery/camus/coders/AvroBinaryMessageDecoder.java#L102 [20:29:53] nuria: yes [20:30:14] Analytics-Backlog: MobileWikiAppDailyStats should not count Googlebot - https://phabricator.wikimedia.org/T117631#1809057 (JKatzWMF) @tbayer, cool! thanks for sharing the additional context. [20:30:41] nuria: and line 113 we should use new GenericDatumReader(writerSchema, readerSchema) [20:32:19] dcausse: DatumReader reader = new GenericDatumReader(helper.getSchema()); [20:32:21] mforns_gym: turns out cdnjs was a bad suggestion, they track and disclose IPs and it's probably not kosher on labs. I'm going to update by downloading all the links locally and referencing them from the html on limn1. You can leave the sample the way it is [20:32:32] (leave the gist the way it is) [20:32:52] nuria: yes it's the problem I think [20:33:25] avro cannot decode if it does not have the *exactly* same schema [20:34:27] dcausse: let'see, the schema is appended as a header to every record, this is to help the decoder to retrieve the schema in question [20:34:45] no nuria, its not [20:34:59] dcausse: now, we do not use that header, and we override the decoder to set decoder [20:35:04] oh you mean it was before [20:35:05] sorry [20:35:09] dcausse: according to kafka topic [20:35:16] ottomata: right, right. [20:35:42] dcausse: if what you say is true, it shows our inexperience with avro :) [20:35:48] ottomata, dcausse in which case that avro sends schema is irrelevant as long as we know that prior and current data validate to latest schema [20:36:53] that means that the schema_id thing that linkedin and others are doing is more important than we though, it means that we are more likely to have bugs where data in kafka is un-deserializable and unreadable, because we lose specific schema version association [20:36:59] thought* [20:37:26] i also think that you *should* be able to read any binary data with a properly evolved later schema...that seems to be the point. buuuuuut [20:37:26] nuria: because when you say data validate to latest schema. In avro world it means the new schema is compatible with the old one. But does not mean you can decode binary data writter with the old schema [20:37:50] ottomata: https://martin.kleppmann.com/2012/12/05/schema-evolution-in-avro-protocol-buffers-thrift.html [20:37:56] dcausse: right, the newer schema should be able to decode old data [20:38:00] dcausse: i think you should be able to..>>>..MAYBE it is just because of your union default problem? but maybe we are wrong. [20:38:04] dcausse: but opposite is not true [20:38:35] dcausse: i think is the union, let's try to prove that with a test with avro tools [20:39:46] dcausse: can you paste some older and newer data? [20:39:47] dcausse might be right nuria, according to the method signature he posted [20:40:01] how would a deserialize call look like if you only have the latest schema? [20:40:13] nuria: I have a java unit test [20:40:23] ottomata: wait but our code doesn't use that signature... does it? [20:40:24] could you just pass the same (latest) schema for both writer and readers schema? [20:40:31] looking [20:41:15] ottomata: you'll end up with exception like EOF ... of even OutOfMemory in prod because it tries to read a length that is not a length [20:41:33] hmmm [20:41:57] dcausse: even if your writer wrote the schema properly with a union and a default? [20:42:08] ottomata: I'll try this one :) [20:42:10] wrote the data usin ga schema properly [20:42:11] ok [20:42:33] but my current data was not written with a schema with union [20:43:02] dcausse: ok, let's try that, caue without a union it could not work [20:44:29] nuria: will try tomorrow, thanks your help again! :) [20:44:49] dcausse: can you give me a piece of data so i can try out? [20:45:25] nuria: Can I send a review with a unit test? [20:46:14] dcausse: excellent! [20:47:22] kevinator: Can you please read (and, presumably, approve) Nuria’s request here? https://phabricator.wikimedia.org/T117473 [20:50:47] (PS1) DCausse: Test do not merge [analytics/refinery/source] - https://gerrit.wikimedia.org/r/253393 [20:51:21] dcausse: nice!, will look at it now [20:51:32] nuria: thanks! [20:52:45] andrewbogott, nuria I'm reading the ticket right now. [20:53:12] nuria: do you only want to be part of the group aqs-user ? [20:59:55] thanks kevinator [21:00:02] yw [21:14:10] ottomata: could you ansible-deploy for me? [21:14:18] pageview API is going live baby :) [21:27:09] getting this error (repeatedly): "ERROR 2006 (HY000): MySQL server has gone away" ... [21:27:18] ...from stat1003 (mysql:research@analytics-store.eqiad.wmnet ) [21:28:05] HaeB, which DB server [21:28:11] Oh I see. [21:28:22] I'm not having trouble with analytics-store. [21:28:34] How long are the queries running before you get that message? [21:29:19] it appears right after you enter the query [21:29:33] now it's working again (in one terminal, while the other is still hanging) [21:29:33] milimetric: checking [21:29:46] sweeet [21:29:49] * milimetric prepares email [21:29:56] -._o_.- [21:30:04] nuria: I'm thinking engineering + wikitech for one and wmfall for the other thread [21:30:09] milimetric: check good, deploying [21:30:12] !log deploying aqs [21:30:21] sweeter [21:30:35] milimetric: analytics+engineering+wikitech-l then [21:30:39] milimetric: sounds good [21:30:52] sweet [21:31:02] * milimetric dies of sugar overdose [21:31:02] milimetric: looks good! [21:31:03] :P [21:31:04] as i am sure there are people on analytics that are not on the other two lists [21:31:11] yep [21:31:55] ok, a-team: about to send this out. Any last updates / thoughts? [21:32:16] milimetric, I saw your comment, was googling about it [21:32:26] mforns: oh, I fixed that [21:32:33] ok [21:33:17] oh... hm... the spec isn't updated on the front-end restbase yet, maybe they're not done with that deploy [21:33:28] milimetric: One suggestion - mentioned to log bugs/issues on Analytics-Backlog? [21:33:37] mention* [21:33:56] yes!!! Last minute!!! :) I had my finger on the big red button [21:34:04] ha ha sorryyy [21:34:10] no, good call [21:34:21] updating. also gotta wait for rb team to finish full deploy [21:34:55] you have a 1/9 chance of seeing the new spec right now [21:35:27] we deploy to a canary node first, then wait for a while before continuing with a full rolling deploy [21:36:44] gotcha, makes sense :) [21:36:53] We'll wait till it's more like 7/9 :) [21:45:03] (PS2) Nuria: Test do not merge [analytics/refinery/source] - https://gerrit.wikimedia.org/r/253393 (owner: DCausse) [21:46:40] Where do the apache access logs end up? :) [21:46:52] dcausse: (i think you are gone for the day) but see my notes on commit message: https://gerrit.wikimedia.org/r/#/c/253393/ [21:47:00] dcausse: I also updated test a bit [21:47:46] I guess it's analytics-privatedata-users [21:48:21] madhuvishy: see if this makes sense, i think every addition needs to be specified like: https://gerrit.wikimedia.org/r/#/c/253393/2/refinery-camus/src/main/avro/CirrusSearchRequestSet3.avsc [21:48:26] cc ottomata [21:49:04] Reedy: apache logs? are you after sampled plain request logs? [21:49:19] Reedy: cause most of requests do not get to apache [21:49:27] Oh, I know that :) [21:49:43] https://phabricator.wikimedia.org/T118739 [21:51:16] Reedy: ah , ok, if you are after logs for dumps.wikimedia.org i think you need to talk to ops [21:55:44] milimetric, great email :D [21:57:06] I'm very giddy. I think this is the first time I have coder's high without coding [21:57:12] milimetric: :D [21:57:46] hehe [22:01:12] Analytics, Analytics-Kanban, Discovery, EventBus, and 8 others: EventBus MVP - https://phabricator.wikimedia.org/T114443#1809193 (Ottomata) [22:01:16] Analytics-Kanban, EventBus, Services, Patch-For-Review: Package EventLogging and dependencies for Jessie - https://phabricator.wikimedia.org/T118578#1809191 (Ottomata) Open>Resolved This was easier than I thought. I don't think there are any dependencies in Jessie we don't already have availa... [22:02:38] Analytics-EventLogging, Analytics-Kanban, EventBus: Puppetize eventlogging-service - https://phabricator.wikimedia.org/T118780#1809194 (Ottomata) NEW a:Ottomata [22:07:29] Analytics-Backlog, Wikimedia-Developer-Summit-2016: Developer summit session: Pageview API overview - https://phabricator.wikimedia.org/T112956#1809224 (Ocaasi) Something I have been dreaming of is a simple widget that computes the following: "The number of pageviews all articles I have ever created to... [22:10:05] Analytics-Tech-community-metrics, DevRel-November-2015: Automated generation of (Git) repositories for Korma - https://phabricator.wikimedia.org/T110678#1809233 (Aklapper) >>! In T110678#1800561, @Dicortazar wrote: > @Aklapper, my only concern is about the git repositories that are not part of Gerrit but... [22:34:05] Analytics-Engineering, Community-Tech: Add page view statistics to page information pages (action=info) [AOI] - https://phabricator.wikimedia.org/T110147#1809263 (Milimetric) [22:34:06] Analytics-General-or-Unknown, Possible-Tech-Projects: Pageviews for Wikiprojects and Task Forces in Languages other than English - https://phabricator.wikimedia.org/T56184#1809264 (Milimetric) [22:34:19] YES!!!! [22:34:19] :D [22:36:50] madhuvishy: did you see my notes to teh cirrus schema? [22:36:52] *the [22:37:03] milimetric: i'm doing this wikimetrics work - and i have a question - i'm not sure where the percentage valid in the cohort should be calculated [22:37:11] nuria: no not yet [22:38:48] Analytics-Backlog: Missing Pageview API data for one article - https://phabricator.wikimedia.org/T118785#1809274 (Ainali) NEW [22:40:34] Analytics-Backlog, Wikimedia-Developer-Summit-2016: Developer summit session: Pageview API overview - https://phabricator.wikimedia.org/T112956#1809281 (mforns) Hey @Ocaasi I think that's a very cool idea! I don't think this is doable right now in the current status of the API, because it only has data... [22:44:42] madhuvishy: right, good question, lemme read that code again a little [22:45:49] madhuvishy: there's this function to get the stats after validation finishes: https://github.com/wikimedia/analytics-wikimetrics/blob/master/wikimetrics/api/cohorts.py#L279 [22:46:00] milimetric: yeah i've seen that [22:46:10] you can add the percent valid in that dictionary there? [22:46:42] yeah alright [22:47:19] milimetric: and i am going to return the cohort from validate_cohort's run [22:47:22] (it's ok if there are extra fields in there for all the consumers of it, they won't care) [22:47:24] so it can be passed on [22:47:32] makes sense [22:47:38] to the RunGlobalReport thingy [22:47:52] yeah, that should also be innocent, because right now I think just nothing's happening with it [22:48:16] i tried to do [22:48:24] https://www.irccloud.com/pastebin/jdnAkOTt/ [22:48:42] from my new RunGlobalReport class in models [22:48:53] and I get this AttributeError: 'CohortStore' object has no attribute 'has_validation_info' [22:48:58] milimetric: ^ [22:49:03] that's where i got stuck [22:49:35] the cohort has to be of the right type when you create it, did you take care of that? [22:49:53] I forget how that works, but it's got to be a validateable cohort or whatever [22:49:54] one sec [22:50:23] see here: https://github.com/wikimedia/analytics-wikimetrics/tree/master/wikimetrics/models/cohorts [22:50:31] that's the cohort type hierarchy [22:51:07] and centralauth_cohort is the right type, because it's validated and will expand to all wikis [22:51:54] and I guess this is where it decides you're doing a centralauth cohort: https://github.com/wikimedia/analytics-wikimetrics/blob/master/wikimetrics/models/validate_cohort.py#L78 [22:52:53] so if you're using from_upload (which I think you can) then fake it that way, otherwise set the class_name to 'CentralAuthCohort' somehow [22:56:24] milimetric: oh, okay looking into it [23:25:17] Analytics-General-or-Unknown, The-Wikipedia-Library: Category based-pageview collection for non-Article space, via Treeviews or similar - https://phabricator.wikimedia.org/T112157#1809410 (Sadads) Hi @Magnus I am excited for the change to T44259 ! Definitely exciting. Do you have a sense of how you are p... [23:39:02] milimetric: still around? [23:39:33] (CR) Nuria: "Bryan:" [analytics/refinery/source] - https://gerrit.wikimedia.org/r/253046 (https://phabricator.wikimedia.org/T118592) (owner: BryanDavis) [23:46:46] Analytics-Backlog, Reading-Admin, Zero: Country mapping routine for proxied requests - https://phabricator.wikimedia.org/T116678#1809499 (dr0ptp4kt) Would it be worth it for us to verify whether we could rely upon the route info via the query expressed in https://developers.facebook.com/docs/sharing/web... [23:53:38] mforns: you there? [23:53:45] madhuvishy, yes [23:54:07] what's up? [23:55:40] mforns: no, i was looking at the bug someone filed [23:55:44] on the api [23:55:54] did you just fix something? [23:55:55] oh [23:56:00] because it got fixed [23:56:13] mforns: this one https://phabricator.wikimedia.org/T118785 [23:56:22] * mforns looks [23:57:27] mmm [23:57:35] i was able to reproduce the bug 5 minutes back, but i don't see the bug anymore [23:58:35] madhuvishy, I get data only for 3 days [23:58:42] mforns: yeah me too [23:58:50] he he don't know automagically fixed [23:59:14] why do you think it is fixed? [23:59:20] it shoul return more days no? [23:59:43] Analytics-Backlog: Missing Pageview API data for one article - https://phabricator.wikimedia.org/T118785#1809549 (madhuvishy) @Ainali can you check if you are still seeing this bug?