[08:57:56] ottomata: https://analytics.wikimedia.org/dashboards/standard-metrics/#simplewiki [09:24:24] mforns: joal milimetric, mw history talk went great! thanks for your help understanding it all [09:24:51] ottomata: Thanks for putting me in the lot of people that have helped while I missed the call ;) [09:25:49] ottomata: I assume ops guys like the idea of history being available in a better way than mysql :) [09:40:17] Analytics-Tech-community-metrics, Developer-Relations (Jul-Sep-2016): Identify Wikimedia's most important/used info panels in korma.wmflabs.org - https://phabricator.wikimedia.org/T132421#2676627 (Aklapper) [09:40:33] def some thoughts around dumps and privacy, and also they were excited about pivot for pageviews like things, they really liked that :) [09:42:47] ottomata: great :) [10:29:12] hi [10:29:38] hi [10:29:44] on https://analytics.wikimedia.org/dashboards/browsers/#all-sites-by-os is there a way to see a further breakdown of Linux OS's? [10:30:00] Linux Mint people are concerned that their traffic is being lumped into Ubuntu [10:31:24] legoktm: no more breakdown on the given page that what is hown [10:31:29] shown sorry [10:32:40] hmm okay. is there a way I could test a Linux Mint UA is being treated as Ubuntu or not? [10:35:08] legoktm: We use uaparser to provide a human-readable version of user agents - first would be to check there [10:35:43] thanks [10:35:53] np [10:35:58] [10:35:58] apparently they just use the stock Ubuntu Firefox UA, which explains why this is happening: https://forums.linuxmint.com/viewtopic.php?t=161065 [10:36:03] definitely not a Wikimedia issue [11:02:34] Analytics, Collaboration-Team-Triage, Community-Tech, Editing-Analysis, and 7 others: statistics about edit conflicts according to page type - https://phabricator.wikimedia.org/T139019#2676807 (Tobi_WMDE_SW) [12:05:35] Analytics-Tech-community-metrics, Developer-Relations (Jul-Sep-2016): Identify Wikimedia's most important/used info panels in korma.wmflabs.org - https://phabricator.wikimedia.org/T132421#2676914 (Aklapper) [12:40:55] Analytics-Tech-community-metrics, Regression: Git repo blacklist config not applied on wikimedia.biterg.io? - https://phabricator.wikimedia.org/T146135#2676950 (Aklapper) Listing other examples from "Top Authors" on https://wikimedia.biterg.io/app/kibana#/dashboard/Git-Demographics to allow verifying / r... [14:07:05] (PS1) Milimetric: Update for October meeting [analytics/reportcard/data] - https://gerrit.wikimedia.org/r/313403 [14:07:17] (CR) Milimetric: [C: 2 V: 2] Update for October meeting [analytics/reportcard/data] - https://gerrit.wikimedia.org/r/313403 (owner: Milimetric) [14:27:10] Analytics-Kanban: Some recent ExternalLinksChange data lost - https://phabricator.wikimedia.org/T146815#2677148 (Samwalton9) I'm not sure on the technical details, perhaps @Milimetric can explain what happened - he said that event logging wasn't working correctly because of some recently deployed schema. [14:28:50] Analytics-Kanban: Some recent ExternalLinksChange data lost - https://phabricator.wikimedia.org/T146815#2677149 (Milimetric) Nuria was the one dealing with that outage, Sam, so I guess it's not related. But, Nuria, do you see events from other schemas? If so, something broke on this schema, but we don't kn... [14:30:01] joal: o/ oozie the complainer didn't like the last 503 spike in upload :( [14:31:25] Hi elukey, done ! [14:32:35] thanks a lot! [14:33:07] hopefully avoiding vk restarts will help but I suspect that the underlying issue with cache:upload will keep biting us [14:33:35] hm elukey, we'll need to find a proper oozie solution then [14:35:06] yeah, maybe raising to 10% the tolerance but not sure if good enough.. let's see how the restart patch goes first [14:35:25] (I mean permanently raising to 10%( [14:36:57] elukey: in previous failures we were closter to 25% [14:39:49] joal: previous upload failures? [14:39:55] yup [14:40:12] depends when, but I have seen some examples [14:40:42] ok so I'll need to check the hive table [14:40:51] but 25% is really huge [14:41:21] :( [14:42:15] omg mforns, I did it, I got through all the names [14:42:25] milimetric, wow [14:42:33] (my part of the alphabet) [14:42:37] aha [14:42:43] I could not, not yey [14:42:45] when you get to the end, Greenhouse goes "BOOM. You did it" [14:42:45] yet [14:42:53] hehe [14:43:02] wanna split up the rest? [14:43:21] today's presentation went well, people were really interested in pivot and data exposed by it [14:43:28] cool, elukey [14:43:28] especially pageviews [14:43:51] yeah, that makes sense, editing stuff is more for researchers [14:44:06] elukey: Andrew said earlier that people had concerns about privacy and dumps [14:44:14] yall gonna send notes? [14:44:51] I didn't get the questions about privacy and dumps but maybe Andrew followed up with other people [14:44:52] milimetric, well if you still got air in your lungs [14:45:02] there is an etherpad, finding it [14:45:49] so there is https://etherpad.wikimedia.org/p/ops-offsite-2016-lightning-talks but it does not contain notes [14:45:54] will ask to mr ottomata :) [14:45:58] milimetric, I had from N to Z, so maybe you want to do N to S? And I do T to Z? [14:46:40] mforns: sure, I'll do N to S, you should probably close the application review if you have it open, and reopen it [14:46:53] (not sure how it works, but just in case) [14:46:55] milimetric, no, I closed it [14:49:44] https://etherpad.wikimedia.org/p/ops-offsite-2016-lightning-talks has my outline [14:49:47] === MediaWiki History Overview === [14:50:03] there weren't 'concerns' about privacy and dumps, just interest [14:50:07] mostly from jaime :) [14:50:12] :) oh, good [14:50:38] thx, yeah, let him know I want to hang out and work on that *anytime* he's available [14:50:43] I'll wake up at 4am if he wants [14:50:45] haha ok [14:54:12] elukey: I double checked: most values are between 10 and 20% in the table (only one more than 20) [14:58:07] mforns: lol, this guy wrote "NA" in their cover letter [14:58:17] cover letter, sir? [14:58:17] nah [14:58:30] nah right back atcha, sir [14:58:30] :) [15:00:00] xDDD [15:00:03] Analytics-Kanban: Some recent ExternalLinksChange data lost - https://phabricator.wikimedia.org/T146815#2677228 (Nuria) >I'm not sure on the technical details, perhaps @Milimetric can explain what happened - he said that event logging wasn't working correctly because of some recently deployed schema. Nah, so... [15:16:02] a-team, browser kicked me off, sorrt [15:24:48] Analytics, Fundraising-Analysis, Fundraising-Backlog, MediaWiki-extensions-CentralNotice: Provide performant query access to banner show/hide numbers - https://phabricator.wikimedia.org/T90649#2677273 (Nuria) Let's look at this one as another use case for streaming cc @Milimetric and @Ottomata [15:36:23] (CR) Nuria: [C: 2 V: 2] Update hive-site.xml references [analytics/refinery] - https://gerrit.wikimedia.org/r/312538 (https://phabricator.wikimedia.org/T133208) (owner: Joal) [16:24:34] Analytics: Kill limn1 - https://phabricator.wikimedia.org/T146308#2677372 (Nuria) There are two types of dashboards: - report card -> we own and maintain - other teams , the ee dashboard, this one is not used frequently but looks updated by edit team [16:24:59] Analytics: Kill limn1 - https://phabricator.wikimedia.org/T146308#2677374 (Nuria) [16:37:20] Analytics: Kill limn1 - https://phabricator.wikimedia.org/T146308#2677422 (Milimetric) Hey all pinged folks, this is what we're looking to do with dashboards from limn1.eqiad.wmnet:/var/lib/limn/ **kill:** analytics/ debugging/ edit-reportcard/ example/ gp/ (check with @Ijon) mobile-reportcard/ (easy to mi... [16:38:50] Analytics: Kill limn1 - https://phabricator.wikimedia.org/T146308#2677425 (Nuria) Let's work on killing instances (cleaning up labs proxy config and such) and migrating reportcard at the same time. Let's hopefully work with owners to move edit dashboards to dashiki. [16:40:15] Analytics: Move instances of dashiki hosted on limn1 to another server - https://phabricator.wikimedia.org/T147000#2677428 (Nuria) [16:40:29] Analytics: Move instances of dashiki hosted on limn1 to another server - https://phabricator.wikimedia.org/T147000#2677428 (Nuria) move (host on another server): edit-analysis/ extdist/ extdist-reportcard/ language-reportcard/ [16:45:09] Analytics: Move instances of dashiki hosted on limn1 to another server - https://phabricator.wikimedia.org/T147000#2677449 (Nuria) Instance is ashiki-01.dashiki.eqiad.wmflabs on dashiki project Current dashboards there: edit-analysis.wmflabs.org language-reportcard.wmflabs.org edit-analysis-... [16:46:50] Analytics: Kill limn1 - https://phabricator.wikimedia.org/T146308#2677451 (JKatzWMF) @Tbayer FYI I don't see any problem with it at this point. I haven't looked at it in at least 6 months and new PM hasn't been told about it. [16:47:33] Analytics: Kill dashboards on limn1 that are no longer used - https://phabricator.wikimedia.org/T147000#2677454 (Nuria) [16:48:10] Analytics: Kill dashboards on limn1 that are no longer used - https://phabricator.wikimedia.org/T147000#2677428 (Nuria) analytics/ debugging/ edit-reportcard/ example/ gp/ (check with @Ijon) mobile-reportcard/ (easy to migrate but used by anyone? ping @Jdlrobson and @JKatzWMF) multimedia-metrics/ (hard to mi... [16:49:35] Analytics-Tech-community-metrics, Developer-Relations (Jul-Sep-2016): Identify Wikimedia's most important/used info panels in korma.wmflabs.org - https://phabricator.wikimedia.org/T132421#2677459 (Aklapper) [16:50:26] Analytics: Kill dashboards on limn1 that are no longer used - https://phabricator.wikimedia.org/T147000#2677463 (Nuria) Work is to remove dns domains from labs . [16:51:22] Analytics-Tech-community-metrics, Developer-Relations (Oct-Dec-2016): Identify Wikimedia's most important/used info panels in korma.wmflabs.org - https://phabricator.wikimedia.org/T132421#2197987 (Aklapper) Updated the task desc after meeting with Qgil today. Next steps: 10 ✚ (non-Maniphest) items to d... [16:54:26] Analytics-Tech-community-metrics, Developer-Relations (Oct-Dec-2016): Go through default Kibana widgets; decide which ones are not relevant for us and remove them - https://phabricator.wikimedia.org/T147001#2677473 (Aklapper) [16:54:29] Analytics-Tech-community-metrics, Developer-Relations (Oct-Dec-2016): Go through default Kibana widgets; decide which ones are not relevant for us and remove them - https://phabricator.wikimedia.org/T147001#2677473 (Aklapper) [16:54:34] Analytics-Tech-community-metrics, Developer-Relations (Jul-Sep-2016): Allow AKlapper to access https://wikimedia.biterg.io/edit/ - https://phabricator.wikimedia.org/T144704#2677486 (Aklapper) [16:54:43] Analytics-Tech-community-metrics, Developer-Relations (Oct-Dec-2016): Go through default Kibana widgets; decide which ones are not relevant for us and remove them - https://phabricator.wikimedia.org/T147001#2677473 (Aklapper) p:Triage>Low [16:56:38] Analytics-Kanban: Kill dashboards on limn1 that are no longer used - https://phabricator.wikimedia.org/T147000#2677491 (Nuria) [16:57:05] Analytics: Migrate the simplest limn dashboards to dashiki tabular {frog} - https://phabricator.wikimedia.org/T126358#2677495 (Nuria) [16:57:29] Analytics: Migrate the simplest limn dashboards to dashiki tabular {frog} - https://phabricator.wikimedia.org/T126358#2012091 (Nuria) [16:58:53] Analytics: Migrate the simplest limn dashboards to dashiki tabular {frog} - https://phabricator.wikimedia.org/T126358#2012091 (Nuria) [17:01:21] Analytics: Deprecate reportcard - https://phabricator.wikimedia.org/T130117#2677521 (Nuria) [17:01:27] Analytics: Migrate the simplest limn dashboards to dashiki tabular {frog} - https://phabricator.wikimedia.org/T126358#2677520 (Nuria) [17:02:06] Analytics: Move reportcard to dashiki and reportupdater - https://phabricator.wikimedia.org/T130117#2126320 (Nuria) [17:06:07] Analytics: Move reportcard to dashiki and reportupdater - https://phabricator.wikimedia.org/T130117#2677547 (Nuria) [17:09:32] Analytics: Move reportcard to dashiki and reportupdater - https://phabricator.wikimedia.org/T130117#2677563 (Nuria) [17:10:08] Analytics: Move reportcard to dashiki and reportupdater - https://phabricator.wikimedia.org/T130117#2126320 (Nuria) [17:11:32] Analytics: Move reportcard to dashiki and reportupdater - https://phabricator.wikimedia.org/T130117#2677570 (Nuria) [17:11:45] Analytics: Move reportcard to dashiki and reportupdater - https://phabricator.wikimedia.org/T130117#2126320 (Nuria) [17:14:06] Analytics: Move reportcard to dashiki and new datasources - https://phabricator.wikimedia.org/T130117#2677580 (Nuria) [17:17:39] Analytics-Kanban: Migrate the simplest limn dashboards to dashiki tabular {frog} - https://phabricator.wikimedia.org/T126358#2677588 (Nuria) [17:23:03] Analytics-Dashiki, Analytics-Kanban, Need-volunteer, Patch-For-Review: Vital-signs layout is broken - https://phabricator.wikimedia.org/T118846#2677598 (Nuria) [] Touch events on mobile are buggy [] Picking project needs better autocomplete when it comes to style [] Data breakdown component ooc... [17:51:29] Analytics, Analytics-Dashiki: Dashiki (layout fixes) mostly mobile - https://phabricator.wikimedia.org/T147007#2677690 (Nuria) [17:53:06] Analytics, Analytics-Dashiki: Create a Universal Layout for Dashiki for staging / testing config purposes - https://phabricator.wikimedia.org/T147009#2677693 (Milimetric) [17:56:14] Analytics, Analytics-Dashiki: Dashiki (layout fixes) mostly mobile - https://phabricator.wikimedia.org/T147007#2677707 (Nuria) [17:56:20] man if someone pings me during these grooming sessions I'd never see it in this sea of pings :) [19:27:57] (PS4) Nuria: Add Oozie job to extract data for WDQS research [analytics/refinery] - https://gerrit.wikimedia.org/r/311964 (https://phabricator.wikimedia.org/T146064) (owner: Nschaaf) [19:56:18] (PS5) Nuria: Add Oozie job to extract data for WDQS research [analytics/refinery] - https://gerrit.wikimedia.org/r/311964 (https://phabricator.wikimedia.org/T146064) (owner: Nschaaf) [20:36:37] Analytics-Kanban: Some recent ExternalLinksChange data lost - https://phabricator.wikimedia.org/T146815#2678309 (Nuria) a:Nuria [20:40:45] (CR) Nuria: "@joal : updated patch with new hive-site.xml path, running on 1002" [analytics/refinery] - https://gerrit.wikimedia.org/r/311964 (https://phabricator.wikimedia.org/T146064) (owner: Nschaaf) [20:45:10] Analytics-Kanban: Some recent ExternalLinksChange data lost - https://phabricator.wikimedia.org/T146815#2678337 (Nuria) I do not see anything else for us to do here as I said I do not think those events ever made it to the system, they are not on the logs for valid or non valid events. [21:18:15] (PS1) Mforns: Add build artifacts and pivot url generator [analytics/pivot] - https://gerrit.wikimedia.org/r/313507 (https://phabricator.wikimedia.org/T146389) [21:25:30] (CR) Milimetric: [C: 2 V: 2] "nice" [analytics/pivot] - https://gerrit.wikimedia.org/r/313507 (https://phabricator.wikimedia.org/T146389) (owner: Mforns) [21:25:47] milimetric, thx [21:26:15] ty sir, nice test script [22:31:49] Analytics-Kanban, Patch-For-Review: Upload the final version of the pivot repo and load test - https://phabricator.wikimedia.org/T146389#2678579 (mforns) Here's the documentation in Wikitech https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Druid/Load_test [22:35:04] night team! cya tomorrow