[09:06:40] 06Analytics-Kanban, 06Operations, 10Traffic, 13Patch-For-Review: Add global last-access cookie for top domain (*.wikipedia.org) - https://phabricator.wikimedia.org/T138027#3021008 (10ema) The patch is ready @Nuria, please let me know when it's OK to merge! [09:16:45] 06Analytics-Kanban, 06Operations, 10netops: Review ACLs for the Analytics VLAN - https://phabricator.wikimedia.org/T157435#3021058 (10elukey) Remove elastic1001 -> 1016: ``` delete firewall family inet filter analytics-in4 term es from destination-address 10.64.0.108/32 delete firewall family inet filter an... [09:28:38] 10Analytics, 06Discovery, 06Discovery-Search, 10Elasticsearch: Analytics cluster should connect to elasticsearch over SSL - https://phabricator.wikimedia.org/T157943#3021112 (10Gehel) [09:59:16] 10Analytics, 06Discovery, 06Discovery-Search, 10Elasticsearch: Analytics cluster should connect to elasticsearch over SSL - https://phabricator.wikimedia.org/T157943#3021171 (10dcausse) ES hosts used by analytics jobs are defined in [[https://phabricator.wikimedia.org/diffusion/WDAN/browse/master/oozie/tra... [10:58:34] 10Analytics: Fix description of webrequest table - https://phabricator.wikimedia.org/T157951#3021381 (10Tbayer) [12:33:06] 10Analytics, 10Analytics-Cluster, 06Operations: Move cloudera packages to a separate archive section - https://phabricator.wikimedia.org/T155726#3021593 (10MoritzMuehlenhoff) I also have a (slight) preference towards cdh. [12:46:25] hello! I updated all the ACLs related to traffic from analytics to elastic search [12:46:32] (and alerted discovery) [12:46:38] should be good but let me know if you see any problem [12:49:00] [fdans] elukey helloooo [12:49:50] " [fdans]" ??? [12:49:58] is it your proxy to IRC? [12:49:59] :D [12:50:14] haha yeshh [12:50:27] needs some refinement though [12:51:13] better :D [12:52:37] I mean I don't think this integration will ever substitute using an irc client, but it's nice to have it [13:13:42] 06Analytics-Kanban, 06Operations, 10netops: Review ACLs for the Analytics VLAN - https://phabricator.wikimedia.org/T157435#3021684 (10elukey) Fixed elastic IPs (not added annotations to analytics-in4). Next ones: 1) Remove udplog ? ``` term udplog { from { destination-address { 233... [14:02:44] mmmmmm [14:03:01] we are restarting zookeeper for jvm upgrades [14:03:04] https://grafana.wikimedia.org/dashboard/db/zookeeper?panelId=40&fullscreen&from=now-3h&to=now&var-cluster=main-eqiad&var-zookeeper_hosts=All [14:03:16] (main-codfw and main-eqiad) [14:03:27] there are a couple of things that look weird [14:03:50] 1) we don't set -Xmx among the JVM parameters, so I guess that it auto sets some value - don't like it :D [14:04:07] 2) conf1001 was not the leader and it showed a weird memory consumption [14:04:22] 2) may be related to ChangeProp? [14:04:56] joal you there? :) [14:05:00] Hi fdans [14:05:04] Here I am :) [14:06:09] joal: one question, which is the piece of the puzzle I'm missing in configs... where are the names of the tables that aqs gets the data from? [14:07:12] fdans: those names are generated :) [14:10:40] joal and where would that generation take place? [14:11:29] fdans: looking for it now [14:12:16] fdans: https://github.com/wikimedia/restbase-mod-table-cassandra/blob/master/lib/db.js#L202 [14:12:34] ohhhh nice [14:12:46] thank you so much [14:12:46] fdans: more or less:) [14:16:48] 06Analytics-Kanban, 10Wikimedia-Extension-setup, 13Patch-For-Review: Deploy mediawiki-Dashiki extension to meta.wikimedia.org - https://phabricator.wikimedia.org/T156971#3021805 (10Milimetric) @demon I added you to https://gerrit.wikimedia.org/r/#/c/336444/, mind taking a look and helping deploy? I can bug... [14:29:43] 10Analytics, 10Analytics-Cluster, 06Operations: Zookeeper heap usage patterns - https://phabricator.wikimedia.org/T157968#3021877 (10elukey) [14:29:52] 10Analytics, 10Analytics-Cluster, 06Operations: Zookeeper heap usage patterns - https://phabricator.wikimedia.org/T157968#3021890 (10elukey) p:05Triage>03Normal [14:36:58] hi team :] [14:43:59] o/ [14:54:19] 10Analytics, 10Analytics-Cluster, 06Operations: Zookeeper heap usage patterns - https://phabricator.wikimedia.org/T157968#3021956 (10elukey) [15:07:54] 10Analytics, 10Analytics-Cluster, 06Operations: Zookeeper heap usage patterns - https://phabricator.wikimedia.org/T157968#3021963 (10elukey) [15:14:18] milimetric found another semantic sketch kit that seemed to have more components https://github.com/garystover/Semantic-UI-Kit [15:14:24] is it ok for me to use this one? [15:14:37] want to make sure i don't grab components that aren't easy to code [15:15:12] ashgrigas: I'll take a look now, but should be totally fine [15:15:19] ok cool [15:15:41] and for fonts should i just use Open Sans and Lato (the ones in the semantic file)? [15:15:54] theyre open source so should be ok on your end [15:16:10] ashgrigas: yeah, that looks like really close to the latest version, I can't guarantee pixel accuracy but we can always tweak [15:16:16] ok [15:16:28] ashgrigas: I am literally the worst person in the world to ask about fonts [15:16:40] just want to make sure technically [15:16:40] I can't tell the difference between Times New Roman and Windings 3 [15:16:50] those are ok in terms of not just being system defauly [15:16:52] default [15:17:00] using a webkit font [15:17:18] oh yeah, technically any font is fine, they're all kind of a pain but the pain is equal [15:17:21] ok [15:17:26] cool just wanted to make sure [15:17:27] thanks! [15:18:17] oh thank you, happy to brain-bounce if you wanted to think about some of the trickier components and you don't find the widgets you need in that Sketch kit [15:18:40] we can definitely build whatever you need, just a matter of saving time and $ [15:43:00] elukey: I don't know much about mariadb, and I think I need to upgrade the piwik server to use it, got a sec? [15:47:12] 06Analytics-Kanban, 06Operations, 10Traffic, 13Patch-For-Review: Add global last-access cookie for top domain (*.wikipedia.org) - https://phabricator.wikimedia.org/T138027#3022039 (10Nuria) Let's merge! thank you. [15:47:39] milimetric: sure but I don't have ton of experience with it.. what are you planning to do? [15:48:02] I think I need connection pooling on the piwik server [15:48:24] and Jaime showed me that you can enable it on MariaDB without touching the code [15:48:39] which is great, except https://github.com/wikimedia/operations-puppet/blob/9b4bc170fba7d1623032d47a42ccf878ba382750/modules/role/manifests/piwik/server.pp#L34 means it's running plain mysql... right? [15:49:19] I'd have to first figure out what that package is, where our mariadb package is, how to upgrade (seems easy without puppet), and then enable connection pooling in the config [15:49:36] everything seems simple to me except working through puppet [15:50:22] like I'd sudo apt-get install mariadb and then set "thread_handling=one-thread-per-connection" [15:51:01] elukey: ^ [15:51:13] milimetric: hola. i have the ops week this week, i can take care of those changes [15:51:39] nuria: morning [15:51:47] but why, I'm in the middle of it [15:52:04] milimetric: nevermind if you have started [15:52:36] milimetric: so there is puppet code to handle a mariadb installation, but it would be super great if we could loop in our dear dbas so it will fall under their radar.. I'll have a chat with Manuel :) [15:52:47] elukey: wait no! [15:52:48] :) [15:53:18] sooner or later it will break badly :D [15:53:33] if we rely on the mariadb puppet code and they don't know about it [15:53:33] context: piwik is my responsibility, we purposefully left the dbas out of it. Because the whole idea was that it was *not* going to be an ops burden. So if my question takes more than a few minutes of your time, then nevermind, I'll just figure it out [15:53:59] I said as much to Jaime too: https://phabricator.wikimedia.org/T154558#2982604 [15:54:12] and he just linked me to the threadpool stuff [15:55:08] also, elukey: I'm not going to reference puppet code, just use a different package, I don't need to depend on how the DBAs do anything else, I don't think [15:55:12] this is stand-alone [15:56:13] milimetric: my point is that piwiki is production now and we should start supporting is a such.. We have too many things in this weird area that is "experimental" and managed by us [15:56:24] (I can think about all the screen sessions on druid) [15:56:42] nono, this is not the same thing at all [15:57:03] I see that I should give you more context though, let's chat before stsandup? [15:58:44] milimetric: maybe we can chat post-standup, there is not a lot of time.. we'll also decide what to do with piwik, I am really willing to help but I'd need to have a clear view of what we need to do [15:58:57] yeah, that's fair [16:00:33] a-tem: standdup [16:00:36] holaaa [16:00:41] going... [16:40:19] 10Analytics: upgrade druid to 0.9 - https://phabricator.wikimedia.org/T157977#3022179 (10Nuria) [16:40:53] 10Analytics: Blog post about druid - https://phabricator.wikimedia.org/T157978#3022191 (10Nuria) [16:45:10] milimetric: the mariadb module is really nice, a lot of templates, sane defaults.. I am pretty sure that there is a quick and simple way to just include the puppet class and that's it [16:45:17] rather than pulling the debian version [16:45:26] it also have custom units, etc.. [16:45:57] if you are patient, would you let me try some experiments tomorrow before proceeding? [16:50:03] 06Analytics-Kanban, 10Analytics-Wikistats: Add "Interwicket" to the list of bots - https://phabricator.wikimedia.org/T154090#3022229 (10Nuria) [17:02:04] 10Analytics: Hive code to count global unique devices per top domain (like *.wikipedia.org) - https://phabricator.wikimedia.org/T143928#3022261 (10Nuria) * code for last access to count global counts on 13 domains * early vetting to clear out bugs * communicate with reading for 2nd round of vetting [17:02:34] 10Analytics: Serve global unique device counts externally - https://phabricator.wikimedia.org/T157981#3022262 (10Nuria) [17:07:21] 06Analytics-Kanban: Hive code to count global unique devices per top domain (like *.wikipedia.org) - https://phabricator.wikimedia.org/T143928#2583528 (10Nuria) [17:28:43] 06Analytics-Kanban, 10Wikimedia-Extension-setup, 13Patch-For-Review: Deploy mediawiki-Dashiki extension to meta.wikimedia.org - https://phabricator.wikimedia.org/T156971#3022333 (10demon) >>! In T156971#3021805, @Milimetric wrote: > @demon I added you to https://gerrit.wikimedia.org/r/#/c/336444/, mind takin... [17:35:28] 10Analytics, 10Analytics-Wikistats: Visual prototype for community feedback for Wikistats 2.0 - https://phabricator.wikimedia.org/T157827#3022359 (10Nuria) Goal of prototype: look at the bare bones data modeling with visual aids while we have designer on team. Bonus: get more pointed feedback from community.... [17:47:54] 10Analytics, 10Analytics-Wikistats: Visual prototype for community feedback for Wikistats 2.0 - https://phabricator.wikimedia.org/T157827#3022402 (10Nuria) Prior to start; * optional build osme Ui arround wireframe-like data model 1 iteration: * Get visuals from ash * map widgets to components * implement m... [17:49:00] 10Analytics, 10Analytics-Wikistats: Visual prototype for community feedback for Wikistats 2.0 - https://phabricator.wikimedia.org/T157827#3022408 (10Nuria) One iteration would be a week ~almost two people [17:49:42] 10Analytics, 10Analytics-Wikistats: Visual prototype for community feedback for Wikistats 2.0 iteration 1. - https://phabricator.wikimedia.org/T157827#3022409 (10Nuria) [17:49:51] 06Analytics-Kanban, 10Analytics-Wikistats: Visual prototype for community feedback for Wikistats 2.0 iteration 1. - https://phabricator.wikimedia.org/T157827#3017940 (10Nuria) [17:54:16] 06Analytics-Kanban, 10Analytics-Wikistats: Simple json model of the site - https://phabricator.wikimedia.org/T157986#3022441 (10Milimetric) [17:57:16] 10Analytics: Agreggate banner dataset for long term retention - https://phabricator.wikimedia.org/T157582#3022465 (10Nuria) @AndyRussG : Can we talk to the users of this data to understand the use cases of what you would like to retain long term? [18:10:42] * elukey afk! byeeee [18:10:43] o/ [18:30:21] 10Analytics: Agreggate banner dataset for long term retention - https://phabricator.wikimedia.org/T157582#3022554 (10AndyRussG) Hi! >>! In T157582#3012642, @mforns wrote: > @AndyRussG > 1) On which pages or page types are banners shown? They're only suppressed on Special pages and when a user is editing. Any... [19:35:12] 06Analytics-Kanban: Kill limn1 - https://phabricator.wikimedia.org/T146308#3022852 (10Milimetric) @Andrew pinged me in chat about this for an update. Marcel is currently working on T156388 which will get us the only data still unique to the dashboards on limn1. So technically limn1 is still useful until that t... [19:42:07] Hi all, been prodded here by -operations - does anyone have five minutes to step me through accessing some EventLogging on stat1003? I'm `statistics-users` but I have a sinking feeling that I need access to `research-client.cnf` to get at the data I need [19:50:21] milimetric, mforns question about edits [19:50:52] sure, shoot [19:51:08] milimetric: if an IP is present on an edit [19:51:16] milimetric: the editor is always anonymous right? [19:51:21] milimetric: [19:52:38] depends how you define "present on". Mediawiki stores IPs in the rev_user_text field. So if that has an IP in it, and rev_user is null, then it's an anonymous edit [19:53:10] but it's possible rev_user_text is coincidentally an IP (like a user with an IP as their name) [19:53:27] I think I even saw people named like 127.0.0.1 [19:53:48] nuria ^ [19:54:09] milimetric: i was wondering if ips+ editor names would be present on RC feed [19:54:33] milimetric, nuria, but except those extreme cases, the rule is that IPs are not stored if logged in, right? [19:54:47] mforns: k, that si what i though [19:54:49] they are on RC [19:55:13] the recent_changes table stores IPs for all edits [19:55:13] RC has IP + username???? [19:55:15] logged-in or not [19:55:23] wait... [19:55:25] yeah, right? [19:55:30] is that public? [19:55:34] verifying [19:55:44] no, i don't think so... but then how do people map edits... [19:55:49] i always forget this [19:56:12] webrequest log maybe? [19:57:26] no, webrequest log has no edit user info [19:57:30] mforns: yeah, so the recentchanges table has IP, username, and userid for every edit, anon or not [19:57:46] lemme check the feed [19:57:57] uau [19:58:33] * nuria confused [19:58:42] milimetric, no IP in the feed: [19:58:43] https://www.mediawiki.org/wiki/Manual:RCFeed [19:58:56] that makes sense [19:59:05] ok [19:59:15] nuria: what's confusing? [19:59:22] RC table stores data for how long? [19:59:26] 90 days? [19:59:27] it would have been very confusing if IP was there [19:59:32] ok, but makes sense [19:59:37] ok [19:59:52] no mention of retention on here https://www.mediawiki.org/wiki/Manual:Recentchanges_table [20:00:15] oh yeah, sorry to even suggest that was possible, that would've been crazy :) [20:00:46] oh wait, no, I never said the feed had it, just the table, you confused me [20:01:00] nuria: that answers your question? [20:01:16] milimetric: yesyes [20:01:19] thank you [20:01:30] 06Analytics-Kanban: Kill limn1 - https://phabricator.wikimedia.org/T146308#3023025 (10Andrew) Thanks for the update. It's fine with me if the instance lives on into March, as long as I know someone is on the case and y'all are aware that 3/31 is a hard deadline. [20:01:32] hehe [21:05:08] milimetric: RC stores last 30 days I think? [21:05:17] 90 is the MediaWiki default [21:05:31] oh ok, good to know [21:07:50] milimetric: https://github.com/wikimedia/operations-mediawiki-config/blob/master/wmf-config/CommonSettings.php#L311 [21:08:59] thanks tgr! I feel cool now that I kind of know how mediawiki-config works [21:17:50] (03PS1) 10Milimetric: Update banner with design consultation [analytics/wikistats] - 10https://gerrit.wikimedia.org/r/337452 [21:35:16] 06Analytics-Kanban, 10Wikimedia-Extension-setup, 13Patch-For-Review: Deploy mediawiki-Dashiki extension to meta.wikimedia.org - https://phabricator.wikimedia.org/T156971#3023250 (10Milimetric) >>! In T156971#3022333, @demon wrote: > Yeah, I dropped the ball on this last week (sorry, things came up). Let's ge... [21:36:24] a-team [21:36:27] argh [21:36:36] you ok? [21:36:40] anybody is working with webrequest-load-upload? [21:37:05] not me, anything I can look into for you since it's late there? [21:38:22] !log Restarted webrequest-load-coord-upload 19:00 - failed and Hue returning 500s [21:38:22] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [21:38:42] milimetric: tried to restart the job, let's see how it goes.. really weird, Hue returns 500s when I check the logs :/ [21:38:55] * milimetric checks [21:40:03] this one looks ok to me now https://hue.wikimedia.org/jobbrowser/jobs/job_1486634611252_12587/single_logs [21:40:11] at least, looks like it's working and reducing stuff [21:41:47] ah yes that one was started a minute ago :D [22:02:29] milimetric: seems to work now, not sure why it failed (can't find logs from the oozie cli, maybe my brain does not connect) [22:02:50] let's see if oozie the complainer will be quiet :) [22:02:53] ttl! [22:02:55] o/ [22:13:13] 10Analytics, 10MediaWiki-extensions-WikimediaEvents, 10The-Wikipedia-Library, 10Wikimedia-General-or-Unknown, 13Patch-For-Review: Implement Schema:ExternalLinksChange - https://phabricator.wikimedia.org/T115119#1942994 (10Samtar) All been tried before I imagine, but for what it's worth nothing after 30+...