[06:52:27] hello! [06:52:54] so the hdfs balancer in my tmux session still need to move 17.35 TB [07:43:24] Maaan - Thanks a lot elukey for having found that [07:43:31] itr would have bitten us badly [07:46:10] (03PS1) 10Elukey: Append '-skipTrash' to all the hdfs -rm invocations [analytics/refinery] - 10https://gerrit.wikimedia.org/r/423844 (https://phabricator.wikimedia.org/T189051) [07:46:46] joal: going to add an alarm for age of processes like the hdfs balancer, so we'll get a notification if something is tuck [07:46:50] *stuck [07:50:25] elukey: reminder - I'll be mostly off before standup - kids day :) [07:54:37] joal: thankssss! Sooner I'll remember :P [09:25:45] * elukey errand + early lunch! (ping me if needed on hangouts!) [10:30:25] 10Analytics-Kanban, 10Patch-For-Review: Refresh SWAP notebook hardware - https://phabricator.wikimedia.org/T183145#4103901 (10Tbayer) I have been using [[https://pypi.python.org/pypi/impyla |impyla]] on notebook1001 to run Hive queries, but this no longer works on notebook1003. Any ideas what might be wrong? S... [11:17:34] 10Analytics-Kanban, 10Patch-For-Review: Refresh SWAP notebook hardware - https://phabricator.wikimedia.org/T183145#3844739 (10elukey) >>! In T183145#4103901, @Tbayer wrote: > I have been using [[https://pypi.python.org/pypi/impyla |impyla]] on notebook1001 to run Hive queries, but this no longer works on noteb... [11:21:29] (03CR) 10Jonas Kress (WMDE): "How long will it take for the tagging to be in place?" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/422930 (owner: 10Jonas Kress (WMDE)) [11:25:42] 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Move away from jmxtrans in favor of prometheus jmx_exporter - https://phabricator.wikimedia.org/T175344#4104047 (10elukey) [11:25:45] 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Fix outstanding bugs preventing the use of prometheus jmx agent for Hive/Oozie - https://phabricator.wikimedia.org/T184794#4104045 (10elukey) 05Open>03Resolved Sadly after a long battle there seems not to be a good way to add prometheus for the hive/... [11:43:32] (03PS3) 10Joal: Update sqoop script to allow for parquet import [analytics/refinery] - 10https://gerrit.wikimedia.org/r/423235 [11:43:49] (03CR) 10Joal: [V: 031] Update sqoop script to allow for parquet import (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/423235 (owner: 10Joal) [12:40:00] 10Analytics-Tech-community-metrics, 10Developer-Relations: Have "Last Attracted Developers" information for Gerrit automatically updated / Integrate new demography panels in GrimoireLab product - https://phabricator.wikimedia.org/T151161#4104172 (10Aklapper) [12:40:08] 10Analytics-Tech-community-metrics, 10Developer-Relations: One account (in "gerrit_top_developers" widget) counted as two accounts (in "gerrit_main_numbers" widget) - https://phabricator.wikimedia.org/T184741#4104173 (10Aklapper) [13:00:13] (03PS1) 10Fdans: Adds/fixes touch interactions in graphs [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/423904 (https://phabricator.wikimedia.org/T188277) [13:05:44] (03CR) 10Fdans: [C: 032] Fixing formatting of dates on time selector [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/423051 (https://phabricator.wikimedia.org/T191097) (owner: 10Nuria) [13:06:07] (03CR) 10Fdans: [V: 032 C: 032] Fixing formatting of dates on time selector [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/423051 (https://phabricator.wikimedia.org/T191097) (owner: 10Nuria) [13:08:34] (03CR) 10Fdans: [V: 032 C: 032] "LGTM! Merging to add change to the upcoming deploy" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/423172 (https://phabricator.wikimedia.org/T191121) (owner: 10Sahil505) [13:19:52] 10Analytics, 10Analytics-Dashiki: Add pivot parameter to tabular layout graphs - https://phabricator.wikimedia.org/T126279#2009951 (10Amire80) [13:21:38] (03PS1) 10Fdans: Release 2.2.1 [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/423910 [13:22:21] (03CR) 10Fdans: [C: 032] Release 2.2.1 [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/423910 (owner: 10Fdans) [13:25:04] (03Merged) 10jenkins-bot: Release 2.2.1 [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/423910 (owner: 10Fdans) [13:25:43] (03PS1) 10Fdans: Release 2.2.1 [analytics/wikistats2] (release) - 10https://gerrit.wikimedia.org/r/423913 [13:28:28] o/ [13:29:19] o/ [13:29:32] (03CR) 10Fdans: [V: 032 C: 032] Release 2.2.1 [analytics/wikistats2] (release) - 10https://gerrit.wikimedia.org/r/423913 (owner: 10Fdans) [13:41:17] 10Analytics-Kanban, 10Patch-For-Review: Refresh SWAP notebook hardware - https://phabricator.wikimedia.org/T183145#4104514 (10Ottomata) Hm, in the meantime, I’ve also installed pyhive, which I think has a similar interface. https://github.com/dropbox/PyHive Try that? [13:43:48] 10Analytics, 10Contributors-Analysis: Add raw sites table to Analytics Data Lake - https://phabricator.wikimedia.org/T191412#4104521 (10Neil_P._Quinn_WMF) [13:45:53] 10Analytics, 10Services (watching), 10User-Elukey: Upgrade Kafka Burrow to 1.0 - https://phabricator.wikimedia.org/T188719#4104535 (10elukey) Tried to build burrow 1.0 using all debian dependencies (and not godeps added to the package) but this is what I get: ``` # github.com/linkedin/Burrow/core/internal/h... [13:46:27] elukey: ! !!AH [13:46:33] i did not know there was a labels thing you could add! [13:46:35] to jmx exporter [13:46:36] ah! [13:46:48] i needed that wow, that would make targeting the mirror maker instances so much easier [13:47:18] ah! cool [13:49:02] elukey: is instance_prefix just a tag? [13:49:40] e.g. https://puppet.com/docs/puppet/5.3/lang_tags.html [13:49:44] used for targeting? [13:52:24] ottomata: so instance_prefix should be the class title's prefix that we give to prometheus::jmx_exporter_instance when we instanciate it [13:52:57] hey guys, are stats machines up? I can't ssh...not sure why [13:53:05] there is a puppetdb query in prometheus::jmx_exporter_config that now should leverage that [13:53:16] dsaez: I bet you are using bast1001 :) [13:53:26] elukey: title prefix? [13:53:28] (deprecated, bast1002 is now the new bastion) [13:53:38] dsaez: you'd need to change the ssh config [13:53:55] elukey, ook! let me see [13:54:33] elukey: so like [13:54:35] profile::prometheus::jmx_exporter { "kafka_mirror_ [13:54:57] i could declare a job jmx_exporter_config [13:54:59] that used [13:55:03] instance_prefix => 'kafka_mirror_, [13:55:10] and it would collect all mirror stuff [13:55:11] cool. [13:55:13] ut [13:55:14] but [13:55:34] yes that's the idea, buuut only for the title gave to prometheus::jmx_exporter_instance (not the profile, if they are the same then ok) [13:55:45] elukey: just this line Host *.eqiad.wmnet *.wikimedia.org !bast1001.wikimedia.org? [13:56:12] dsaez: basically all the occurrences of bast1001 should become bast1002 (the quickest way) [13:56:21] elukey: ok, i would love to use this to add labels like mirror_name to only certain kafka_mirror_ instances [13:56:28] i guess i could do that because I put $mirror_name in the title [13:56:29] but [13:56:40] dsaez: if you want to check the fingerprint https://wikitech.wikimedia.org/wiki/Help:SSH_Fingerprints/bast1002.wikimedia.org [13:57:51] elukey: what if we added a labels => param to jmx_exporter_instance (passed in from profile) [13:57:53] ? [13:58:13] that would allow adding of labels to a targeted instance from the declared jmx_exporter_instance? [13:58:14] like [13:58:34] hello teaaaam [13:58:37] mforns: o/ [13:58:38] profile::prometheus::jmx_exporter { ... labels => { 'mirror_name' => "main_to_jumbo" } } [13:58:39] then [13:58:53] resources = query_resources( bla bla [13:58:57] then in the .erb template [13:59:31] 'labels' => @labels.merge({'cluster' => cluster}).merge(@resources[host][lables]), [13:59:32] ? [14:00:58] then you could apply the labels at the where you declare the profile::prometheus::jmx_exporter, rather than having to configure it specifically for the prometheus host jmx_exporter_config [14:00:59] ? [14:01:00] ottomata: I think it could be possible but I am not sure if people will be happy about this puppetdb use :D [14:01:09] wow it would be WAY better, no? [14:01:33] it would be fewer prometheus::jmx_exporter_config declarations [14:01:47] right now you have to make a new job for every 'cluster' [14:01:53] in order to target them [14:02:47] we could think about a proposal and then ask to Filippo what he thinks [14:02:59] ok, i think i will make a little patch and see [14:03:00] ok? [14:03:10] maybe it won't work! [14:03:22] sure, would you prefer that I block mine to see if yours is accepted first or shall I go? [14:03:33] yeah, beacuse it would make yours not necessariy mostly i think [14:03:38] instead of declaring new jobs for everything [14:03:39] sure [14:03:42] you just set the labels you need [14:03:50] i think..... [14:03:58] not really urgent, I can wait :) [14:04:01] but i'm basing it on yours, i didn't quite understand that labels could be applied like this [14:04:15] ya if it works it would make grafana dashboarding and alert targeting way easier [14:04:46] I started since now on grafana the zk main-eqiad/codfw clusters have as cluster "etcd" [14:04:48] btw elukey with bigger request.timeout.ms and smaller batch size...things have been stable since yesterday! [14:04:50] that is so annoying [14:04:51] :D [14:04:54] lets see what happens if more volume comes in [14:05:02] ottomata: suuuuuper happy about it [14:05:03] i might temporarily mirror the change prop topics to see if it breaks things [14:05:11] ack [14:05:30] yeah we should be able to apply whatever labels make sense [14:05:41] and you can't do it in the jmx exporter metric config [14:05:42] so ya [14:07:07] (03CR) 10Mforns: [C: 031] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/423235 (owner: 10Joal) [14:07:47] elukey: it works! thanks! [14:17:16] elukey: ok one weird thing i hadn't thought about, and maybe it makes my idea not as good, not sure: [14:17:33] the labels will be collected from all resoruces that are matched by query_resources [14:17:47] when the jmx_exporter_config.erb file iterates over the matches resources [14:17:49] it currently does [14:17:55] @resources[host].each{ |instance| [14:17:56] targets.push( ... ) [14:18:00] that's where i'd also get the labels [14:18:05] but, since i'm getting it per matched host [14:18:16] it is not impossible that different hosts might declare different labels [14:18:24] that is not the intended use [14:18:26] but it could happen [14:18:53] it would mean that if hostA declares: custom_labelA: XXX and hostB; declares custom_lableB: YYY [14:19:08] both hostA and hostB would get custom_labelA and custom_labelB [14:19:16] which is maybe fine and possibly useful? [14:19:19] but also [14:19:30] if both hosts declare the same label with different values [14:19:53] only one of them would stick, the one from the last host in the list returned by query_resources [14:20:48] so maybe not... [14:21:09] but, maybe it would be really nice to be able to do fancier stuff in the jmx_exporter_config, like relabel_config [14:21:09] ? [14:21:12] https://prometheus.io/docs/prometheus/latest/configuration/configuration/#relabel_config [14:21:44] e.g. for mirror maker, i could apply the mirror_name label from whatever is set in the client_id? [14:21:51] maybe too fancy dunno [14:23:20] yep seems really good [14:24:37] OH [14:24:39] but [14:24:40] hm [14:25:04] the file_sd_config that jmx_exporter_config.erb renders does not have to apply the same labels to all targets [14:25:05] OOO [14:25:12] ok i can do this [14:25:12] https://prometheus.io/docs/prometheus/latest/configuration/configuration/#file_sd_config [14:25:32] mforns: whenever you have time can we discuss the next steps for the EL whitelist? I don't want to slowdown you, what's the status? [14:25:54] elukey, no slowdown at all, on the contrary [14:26:22] status is, there's still a couple tasks in CR that have to be merged, but we're good to go, we can put the whitelist in its place [14:26:47] the mysql EL cleaner script, OK... [14:26:48] so [14:27:04] 1st thing we should do I guess is: [14:28:24] do a puppet change that: 1) replaces the current whitelist with the new one in the EL mysql hosts, and 2) adds the --yaml true flag to the EL cleaner script call [14:28:50] this change should be merged after we merge the cleaner script [14:28:58] change that adds the --yaml flag [14:29:07] xD, OK: [14:29:34] 1) review and merge https://gerrit.wikimedia.org/r/#/c/420685/ [14:30:03] 2) Create a puppet change that replaces the whitelist AND adds the --yaml true flag to the cleaner script call [14:30:26] 3) Check that everything is good on the next execution of the script in prod [14:30:39] This is all on the mysql side, I think [14:30:45] makes sense yes [14:30:58] on the hive side: [14:31:15] 1) review and merge https://gerrit.wikimedia.org/r/#/c/420795/ [14:31:33] 2) create a puppet patch that adds the new whitelist to analytics1003 [14:31:36] (let's put this into https://phabricator.wikimedia.org/T189692's description with [] to mark the status) [14:31:49] elukey, cool, I can do that [14:31:56] super [14:32:51] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: [EL sanitization] Modify mysql purging script to read from the new YAML whitelist - https://phabricator.wikimedia.org/T189692#4104774 (10mforns) Still TODO: [] Review and merge https://gerrit.wikimedia.org/r/#/c/420685/ [] Create a puppet change that repl... [14:32:53] done [14:33:47] and hive 3) (maybe on the same puppet patch) add a cron job that calls the EventLoggingSanitization in hive [14:34:07] 4) check results after first execution [14:35:03] elukey, do you want me to create the puppet patch? [14:35:11] for mysql [14:35:12] ? [14:36:28] mforns: lemme review the yaml patch first for the el cleaner [14:36:37] k [14:36:40] then I think we can flip it to the new whitelist [14:43:15] elukey: is there a way to run PCC to test puppet db based stuff? [14:43:45] ottomata: I am not sure, since IIRC the pcc uses its own puppetdb, so probably not super reliable [14:43:55] hm [14:43:58] i guess i'll try [14:43:59] the patch looks cool, just had a quick glance at it [14:44:16] ya, im' not sure if we want to do this, but i might allow us to have only one jmx expoter job configured [14:44:26] that collects all declared instances in the $site [14:44:42] since afaict the only reason there are seprate ones are to apply the cluster label [15:16:42] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 3 others: CirrusSearchCheckerJob should have a title - https://phabricator.wikimedia.org/T190958#4104999 (10mobrovac) [15:31:20] elukey: you got a few minutes to talk about the geoip cron job? [15:31:41] (buonasera!) [15:48:21] fdans: here I am sure [15:48:36] (was talking with Andrew before) [15:48:39] (I am in the cave) [15:48:49] elukey: going to the cava! [16:04:38] (03CR) 10Nuria: "This task actually needs design, please ping us before grabbing tasks as some of them are not ready for grabs." [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/423252 (https://phabricator.wikimedia.org/T191019) (owner: 10Amitjoki) [16:06:54] !log killed banner-impression related jvms on an1003 to finish openjdk-8 upgrades (they should be brought back via cron) [16:06:56] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:06:58] joal: --^ [16:07:16] Thanks for ping elukey [16:08:07] joal: doing rolling restarts for openjdk upgrades is becoming challenging :D [16:08:32] I can see that elukey [16:08:43] elukey: Would there be ways for us to facilitate? [16:09:05] joal: probably not, but it is good to do caos monkey and see if we can improve :D [16:14:32] 10Analytics-Kanban: Vet new geo wiki data - https://phabricator.wikimedia.org/T191343#4105308 (10Nuria) [16:15:09] 10Analytics-Kanban: Puppetize job that saves old versions of Maxmind geoIP database - https://phabricator.wikimedia.org/T136732#4105310 (10Nuria) [16:15:46] 10Analytics-Kanban: Puppetize job that saves old versions of Maxmind geoIP database - https://phabricator.wikimedia.org/T136732#2345955 (10Nuria) FYI that /home/milimetric/GeoIP-toolbox/MaxMind-database/GeoIP is a git repository with all the backups of the GeoIP dbs that we kept over the years [16:19:41] (03CR) 10Ottomata: [C: 031] Append '-skipTrash' to all the hdfs -rm invocations [analytics/refinery] - 10https://gerrit.wikimedia.org/r/423844 (https://phabricator.wikimedia.org/T189051) (owner: 10Elukey) [16:27:21] 10Analytics, 10New-Readers, 10Operations, 10Traffic, and 2 others: Opera mini IP addresses reassigned - https://phabricator.wikimedia.org/T187014#4105369 (10atgo) Partnerships has been looking for a contact at Opera. We reached out to someone yesterday who is OOO until next week. Will keep you updated. [16:35:16] 10Analytics-Tech-community-metrics, 10Developer-Relations (Apr-Jun-2018): Explain decrease in number of patchset authors for same time span when accessed 3 months later - https://phabricator.wikimedia.org/T184427#4105410 (10Aklapper) 05stalled>03Resolved On 20180404, exactly the same results as in T184427#... [16:35:46] 10Analytics-Tech-community-metrics, 10Developer-Relations (Apr-Jun-2018): Explain decrease in number of patchset authors for same time span when accessed 3 months later - https://phabricator.wikimedia.org/T184427#4105412 (10Aklapper) 05Resolved>03declined Well, changing task status to 'declined' as I canno... [16:42:16] elukey: soryr 5 more mins then bc [16:42:17] -2 [16:42:35] ottomata: we can do it tomorrow don't worry! Just doing other stuff in the meantime [16:44:49] les do it! [16:45:04] https://hangouts.google.com/hangouts/_/wikimedia.org/a-batcave-2 [16:59:53] 10Analytics, 10CirrusSearch, 10Discovery, 10EventBus, and 3 others: Exception thrown while running DataSender::sendData in cluster codfw: Data should be a Document, a Script or an array containing Documents and/or Scripts - https://phabricator.wikimedia.org/T191024#4105542 (10mobrovac) [17:00:32] (03PS10) 10Amitjoki: [WIP] Label map and top metrics with the month they belong to [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/423144 (https://phabricator.wikimedia.org/T182990) [17:02:34] 10Analytics, 10ChangeProp, 10Discovery, 10Discovery-Search, and 4 others: Port CirrusSearch update JobQueue jobs to EventBus - https://phabricator.wikimedia.org/T150283#4105563 (10mobrovac) [17:02:36] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 5 others: Migrate CirrusSearch jobs to Kafka queue - https://phabricator.wikimedia.org/T189137#4105567 (10mobrovac) [17:03:26] 10Analytics, 10CirrusSearch, 10Discovery, 10EventBus, and 3 others: Exception thrown while running DataSender::sendData in cluster codfw: Data should be a Document, a Script or an array containing Documents and/or Scripts - https://phabricator.wikimedia.org/T191024#4105582 (10mobrovac) [17:03:31] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 5 others: Migrate CirrusSearch jobs to Kafka queue - https://phabricator.wikimedia.org/T189137#4032605 (10mobrovac) [17:03:52] (03CR) 10Amitjoki: "> Uploaded patch set 10." [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/423144 (https://phabricator.wikimedia.org/T182990) (owner: 10Amitjoki) [17:04:28] 10Analytics, 10Services (watching), 10User-Elukey: Upgrade Kafka Burrow to 1.0 - https://phabricator.wikimedia.org/T188719#4105593 (10elukey) Ok mistery solved after checking with Andrew. The version of https://github.com/julienschmidt/httprouter in Debian is stuck at the 1.1 tag (from 2015), and since then... [17:12:26] (03CR) 10Amitjoki: "> This task actually needs design, please ping us before grabbing" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/423252 (https://phabricator.wikimedia.org/T191019) (owner: 10Amitjoki) [17:12:35] ottomata: https://github.com/julienschmidt/httprouter/issues/207 - I don't have a good feeling :D [17:12:51] "opened this issue on Jul 21, 2017 · 3 comments" [17:13:51] elukey: drop the wikimedia name in there, say we are tryign to use it [17:13:55] sometimes it helps :) [17:15:39] this go-debian dependency tracking is a bit of a mess, in this case I'd need to get a positive feedback from the upstream version and also from the Debian mantainer before having a workable build [17:15:43] sigh [17:17:14] yeah, its even worse for java :p [17:17:53] elukey: you could try to build a deb for http router? then you can build from a fork tag [17:18:00] and burrow could still use all deb packages [17:18:02] or [17:18:15] just give up and use binaries/committed dependencies :/ [17:19:44] yeah the latter is the best one [17:19:57] quicker and less painful [17:20:02] will do it tomorrow morning :) [17:20:21] logging off for today! [17:20:22] bye! [17:20:24] * elukey off! [17:37:22] (03Abandoned) 10Nuria: Add UI in mobile view to switch to table view [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/423252 (https://phabricator.wikimedia.org/T191019) (owner: 10Amitjoki) [17:57:05] (03CR) 10Nuria: [C: 031] "Looks good, did we tested that -skipTrash parameter actually does what you would expect?" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/423844 (https://phabricator.wikimedia.org/T189051) (owner: 10Elukey) [18:13:33] 10Analytics: Mount dumps on SWAP machines (notebook1001.eqiad.wmnet / notebook1002.eqiad.wmnet) - https://phabricator.wikimedia.org/T176091#4105943 (10madhuvishy) [18:15:25] if i want to push log messages into a kafka topic from analytics network, so ELK (logstash.wikimedia.org) picks them up and makes them available, should i use kafka-jumbo or the other cluster? [18:15:56] or is there another way stuff in analytics already gets data into ELK? [18:25:00] ebernhardson: I'm not aware of kafka logs making it to ELK, but since I'm not using ELK, I'd probably not know anyhow :) [18:25:32] ebernhardson: I also have a question for you [18:26:25] ebernhardson: would it be ok for your team if we move to spark2.3 as default for the cluster (I'm thinking of your production jobs that would probably need a patch) [18:32:14] ottomata: ping meeting? [18:35:03] OHH [18:37:12] joal: hmm, i think we can manage 2.3. I'd have to test but worst case it's very easy to refer to a different SPARK_HOME [18:37:44] joal: and indeed i dont think anything makes it through currently but i was thinking i could set something up, ELK can read from kafka just fine i just need to tell it where to look [18:38:10] kafka happens to be convenient because it's available on both sides of the firewall :) [18:53:57] ottomata, qq: who does transform EL hive userAgent field into a userAgentMap? [18:54:25] oh, sorry in meeting I see [18:55:07] ebernhardson: The reason I ask about spark2 is because since we would change the shuffler, spark1.6 might not be working anymore (not sure) [18:55:27] ebernhardson: About kafka-ELK, I don't see why it would be an issue :) [19:01:29] joal: 1.6 we won't need, we are on 2.1 atm [19:02:21] ebernhardson: great :) [19:05:01] mforns: that is eventlogging processor [19:05:07] hmm [19:05:09] wait [19:05:16] userAgentMap? [19:05:29] ottomata, yes, I mean, transform the string into a map [19:05:42] yeah, eventlogging does that [19:05:48] oh ok [19:06:07] it uses UAparser I guess [19:06:13] mforns: https://github.com/wikimedia/eventlogging/blob/master/eventlogging/utils.py#L318 [19:06:13] ya [19:06:51] ottomata, OK, because I'm trying to see if is_bot is equivalent to the agent_type being spider in pageview_hourly [19:09:23] mforns: it should be: https://github.com/wikimedia/eventlogging/blob/master/eventlogging/utils.py#L41 [19:09:27] ask fdans he worked some on this [19:09:32] the intention was to keep it the same [19:09:59] ottomata, yes I got it, looking now into Webrequest.java to see if it matches [19:12:07] yes, looks the same :] https://github.com/wikimedia/analytics-refinery-source/blob/master/refinery-core/src/main/java/org/wikimedia/analytics/refinery/core/Webrequest.java#L84 [19:12:15] thanks! [19:19:09] ottomata: doc added here https://wikitech.wikimedia.org/wiki/SWAP#Spark [19:24:39] 10Analytics-Kanban, 10MW-1.31-release-notes (WMF-deploy-2018-02-27 (1.31.0-wmf.23)), 10Patch-For-Review: Record and aggregate page previews - https://phabricator.wikimedia.org/T186728#4106322 (10mforns) Another potential issue I've seen is that VirtualPageView.event.page_title is formatted as text (with spac... [19:26:06] 10Analytics, 10EventBus, 10MediaWiki-JobQueue, 10Goal, and 2 others: FY17/18 Q4 Program 8 Services Goal: Complete the JobQueue transition to EventBus - https://phabricator.wikimedia.org/T190327#4106349 (10mobrovac) [19:26:09] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 4 others: Select candidate jobs for transferring to the new infrastucture - https://phabricator.wikimedia.org/T175210#4106350 (10mobrovac) [19:26:14] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 5 others: Migrate CirrusSearch jobs to Kafka queue - https://phabricator.wikimedia.org/T189137#4106348 (10mobrovac) [19:26:49] 10Analytics, 10EventBus, 10MediaWiki-JobQueue, 10Goal, 10Services (doing): FY17/18 Q3 Program 8 Services Goal: Migrate two high-traffic jobs over to EventBus - https://phabricator.wikimedia.org/T183744#4106352 (10mobrovac) [19:26:53] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 2 others: Services Q2 2017/18 goal: Migrate a subset of jobs to multi-DC enabled event processing infrastructure. - https://phabricator.wikimedia.org/T175212#4106353 (10mobrovac) [19:26:59] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 4 others: Select candidate jobs for transferring to the new infrastucture - https://phabricator.wikimedia.org/T175210#3586259 (10mobrovac) [19:28:02] 10Analytics, 10EventBus, 10MediaWiki-JobQueue, 10Goal, and 2 others: FY17/18 Q4 Program 8 Services Goal: Complete the JobQueue transition to EventBus - https://phabricator.wikimedia.org/T190327#4070155 (10mobrovac) [19:28:05] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 4 others: Create custom per-job metric reporters capability - https://phabricator.wikimedia.org/T182274#4106358 (10mobrovac) [19:28:10] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 5 others: Migrate htmlCacheUpdate job to Kafka - https://phabricator.wikimedia.org/T182023#4106360 (10mobrovac) [19:28:45] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 3 others: [EPIC] Develop a JobQueue backend based on EventBus - https://phabricator.wikimedia.org/T157088#4106365 (10mobrovac) [19:28:49] 10Analytics, 10EventBus, 10MediaWiki-JobQueue, 10Goal, 10Services (done): FY17/18 Q3 Program 8 Services Goal: Migrate two high-traffic jobs over to EventBus - https://phabricator.wikimedia.org/T183744#3862480 (10mobrovac) 05Open>03Resolved [19:28:59] 10Analytics-Kanban, 10MW-1.31-release-notes (WMF-deploy-2018-02-27 (1.31.0-wmf.23)), 10Patch-For-Review: Record and aggregate page previews - https://phabricator.wikimedia.org/T186728#4106366 (10Tbayer) >>! In T186728#4106322, @mforns wrote: > Another potential issue I've seen is that VirtualPageView.event.p... [19:32:00] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, 10Services (next): Enable CP4JQ support for private wikis - https://phabricator.wikimedia.org/T191464#4106369 (10mobrovac) p:05Triage>03Normal [19:32:24] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, 10Services (next): Enable CP4JQ support for private wikis - https://phabricator.wikimedia.org/T191464#4106382 (10mobrovac) [19:32:28] 10Analytics, 10EventBus, 10MediaWiki-JobQueue, 10Goal, and 2 others: FY17/18 Q4 Program 8 Services Goal: Complete the JobQueue transition to EventBus - https://phabricator.wikimedia.org/T190327#4106383 (10mobrovac) [19:33:17] 10Analytics-Kanban, 10MW-1.31-release-notes (WMF-deploy-2018-02-27 (1.31.0-wmf.23)), 10Patch-For-Review: Record and aggregate page previews - https://phabricator.wikimedia.org/T186728#4106385 (10mforns) @Tbayer Would it be possible to send the page_title (and source_title) in URL format already from the cli... [19:50:13] 10Analytics-Kanban, 10MW-1.31-release-notes (WMF-deploy-2018-02-27 (1.31.0-wmf.23)), 10Patch-For-Review: Record and aggregate page previews - https://phabricator.wikimedia.org/T186728#4106420 (10Jdlrobson) @mforns for page_title we are at mercy data in the response of REST which includes spaces as " " not "... [19:58:50] 10Analytics-Kanban, 10MW-1.31-release-notes (WMF-deploy-2018-02-27 (1.31.0-wmf.23)), 10Patch-For-Review: Record and aggregate page previews - https://phabricator.wikimedia.org/T186728#4106436 (10mforns) @Jdlrobson In https://en.wikipedia.org/api/rest_v1/page/summary/San_Francisco I can see the URL-formatted... [19:59:24] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 5 others: Migrate CirrusSearch jobs to Kafka queue - https://phabricator.wikimedia.org/T189137#4106442 (10dcausse) [20:03:42] 10Analytics, 10Analytics-EventLogging, 10Performance: Spin out a tiny EventLogging RL module for lightweight logging - https://phabricator.wikimedia.org/T187207#4106471 (10AndyRussG) [20:04:53] joal, the job just finished [20:05:01] mforns: success? [20:05:01] I'm about to copy the data [20:05:04] yep! [20:05:06] awesome :) [20:05:21] Thanks for having monitored mforns ! [20:05:42] hmmm [20:05:53] I see that the script writes the success files itself [20:06:22] but I don't want to copy it over to the output folder [20:07:03] oh! right, I'm copying only certain wikis, the success file is not copied [20:07:05] coool [20:09:47] 10Analytics, 10Analytics-EventLogging, 10Performance: Spin out a tiny EventLogging RL module for lightweight logging - https://phabricator.wikimedia.org/T187207#4106490 (10AndyRussG) Changed the title back to "RL" (from "URL")... Sorry if I wasn't clear... I meant "ResourceLoader" module. The suggestion is t... [20:37:47] ottomata: are you here by any chance? [20:38:15] joal: did you decided on names for split code? [20:38:30] nuria_: yes - We changed to "subsets" instead of splits [20:38:45] I'm currently revamping the thing [20:40:18] nuria_: I'm thinking of actually moving the ore job to spark instead of hive [20:40:46] joal: migrate the hive queries to spark you mean? [20:41:05] nuria_: the query that explode the webrequest into splits [20:41:49] joal: what would be the benefit (i know you are expert , asking for my benefit) [20:42:16] nuria_: batcave for a minute? [20:42:26] joal: omw [20:45:33] joal hyaa [20:46:28] ottomata: in the cave with nuria_, will ask in a minute [20:53:46] 10Analytics-Kanban, 10MW-1.31-release-notes (WMF-deploy-2018-02-27 (1.31.0-wmf.23)), 10Patch-For-Review: Record and aggregate page previews - https://phabricator.wikimedia.org/T186728#4106755 (10Jdlrobson) Oh you're right! Looks like we could use that. We can fix this, but we'd be at the mercy of deployment... [21:02:38] 10Analytics-Kanban, 10MW-1.31-release-notes (WMF-deploy-2018-02-27 (1.31.0-wmf.23)), 10Patch-For-Review: Record and aggregate page previews - https://phabricator.wikimedia.org/T186728#4106785 (10mforns) > Can you workaround this issue in the mean time since we are collecting data and this may take up to a we... [21:03:34] ottomata: back [21:03:42] ottomata: question on spark-refine [21:04:24] ottomata: Can we write a transform-function that alters the partition-scheme (adding a new partition)? [21:05:17] hmm [21:06:33] hm, not as is joal [21:06:42] ottomata: That's what I thought [21:06:49] would need code modifications, to somehow return something from the function which should be partitions [21:07:00] the partitioning is done by the DataFrameWriter (hive) when writing [21:07:30] ottomata: we give the partitions to the function IIRC, but we don't get them back (changed or not) [21:07:34] but, it could be done, make tranform functions do take the current hive partition [21:07:42] they could return a tuple (df, newPartition) [21:07:46] right [21:07:56] !log copied sqooped data for mediawiki history from /user/mforns over to /wmf/data/raw/mediawiki/tables/ for wikidatawiki and commonswiki, tables: revision, logging and pagelinks [21:07:56] i'm for it! [21:07:57] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [21:08:04] joal: feel free to try and make that change if you want [21:08:36] ottomata: maybe we could make a wrapper over our DFs: PDF (df: DataFrame, partitions: Seq[String]) [21:08:46] And we always pass this over [21:08:48] partitions is a HiveParititon [21:08:49] we have that [21:13:30] anyway ottomata - Will think about that :) [21:13:32] Thanks ! [21:13:37] Gone to bed - Tomorrow a-team [21:13:43] byeeee [21:23:24] (03CR) 10Mforns: "Indeed Amit, there's something weird happening with the graphModel that is local to the top articles data. Will discuss it with the team." [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/423144 (https://phabricator.wikimedia.org/T182990) (owner: 10Amitjoki) [21:37:57] 10Analytics, 10Analytics-EventLogging, 10Performance: Spin out a tiny EventLogging RL module for lightweight logging - https://phabricator.wikimedia.org/T187207#4106874 (10Nuria) @AndyRussG what is the impact on performance of EL code that you see? (cc @krinkle) Please be so kind as to provide numbers if... [21:58:54] 10Analytics, 10Analytics-EventLogging, 10Performance: Spin out a tiny EventLogging RL module for lightweight logging - https://phabricator.wikimedia.org/T187207#4106927 (10Nuria) Also, do not know much about mediawiki but seems to me that EL is loaded on every js-enabled pageview? From my brief inspection of... [22:48:39] 10Analytics-Kanban, 10Analytics-Wikistats: Broken "1-month" interval on timerange selector - https://phabricator.wikimedia.org/T191097#4107053 (10Nuria) Still more issues on time selector but this one is fixed, will open other ticket. [22:48:53] 10Analytics-Kanban, 10Analytics-Wikistats: Broken "1-month" interval on timerange selector - https://phabricator.wikimedia.org/T191097#4107054 (10Nuria) 05Open>03Resolved [22:50:55] fdans: please remember to asign points to tasks [23:10:32] 10Analytics-Kanban, 10Discovery-Analysis, 10Wikipedia-Android-App-Backlog, 10Patch-For-Review: Bug behavior of QTree[Long] for quantileBounds - https://phabricator.wikimedia.org/T184768#4107081 (10Nuria) I cannot put my finger on what it is the problem but I think the data from session job has issues, not... [23:59:08] 10Analytics, 10Operations, 10Ops-Access-Requests, 10Research, and 3 others: Restricting access for a collaboration nearing completion - https://phabricator.wikimedia.org/T189341#4107259 (10DarTar) @Ottomata if you can review the privs in the changeset and confirm they are good, that'd be awesome.