[00:44:12] 10Analytics, 10Analytics-Kanban, 10User-Elukey: Change permissions for daily traffic anomaly reports on stat1007 - https://phabricator.wikimedia.org/T219546 (10ssingh) @elukey: Is it also possible to kill `jdcc's` currently running processes as they are no longer being used? The reason I ask is because I am... [01:10:37] PROBLEM - Check the last execution of refinery-eventlogging-saltrotate on an-coord1001 is CRITICAL: CRITICAL: Status of the systemd unit refinery-eventlogging-saltrotate [05:33:58] ahhh lovely start of the morning :D [06:14:48] (03CR) 10Milimetric: Improve restart examples after deploy this week (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/508927 (owner: 10Milimetric) [06:19:39] 10Analytics, 10Analytics-EventLogging, 10DBA, 10Operations, 10ops-eqiad: db1107 (eventlogging db master) possibly memory issues - https://phabricator.wikimedia.org/T222050 (10elukey) @Cmjohnson I'd need a heads up of ~15 mins before the maintenance to shutdown the host properly, but we can do it anytime! [06:40:36] so first alert for the HDFS RPC queue len [06:40:55] ti last for 10/15 mins and effectively I can see a big jump in the RPCs [06:41:04] from ~250k to 3M [06:41:22] and a correspondent jump of +1M files (went away, so probably temporary) [06:51:35] but the oldest entry in the hdfs audit log is 2019-05-09 03:22:50 [06:51:59] it would have been useful to see what was going on [07:49:45] rolling restart hdfs master to pick up new log settings [08:12:12] done! [08:40:12] (03PS1) 10Amire80: Add need-review chart to published CX2 translation [analytics/limn-language-data] - 10https://gerrit.wikimedia.org/r/509007 [08:51:07] (03PS2) 10Amire80: Add need-review chart to published CX2 translation [analytics/limn-language-data] - 10https://gerrit.wikimedia.org/r/509007 (https://phabricator.wikimedia.org/T209868) [09:17:06] (03PS1) 10Elukey: oozie/wikidata: move user to analytics [analytics/refinery] - 10https://gerrit.wikimedia.org/r/509016 (https://phabricator.wikimedia.org/T220971) [09:17:58] !log restart oozie wikidata coordinators (3 in total) with the analytics user [09:17:59] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [09:52:50] (03PS1) 10Elukey: oozie/mediawiki/geoeditors: move coordinators to the analytics user [analytics/refinery] - 10https://gerrit.wikimedia.org/r/509026 (https://phabricator.wikimedia.org/T220971) [10:02:10] PROBLEM - Check the last execution of check_webrequest_partitions on an-coord1001 is CRITICAL: CRITICAL: Status of the systemd unit check_webrequest_partitions [10:25:30] ah yes one webrequest hour is missing [10:26:04] loss seems 5.93, barely above the threshold [11:04:00] !log kill oozie mediawiki geoeditors coords (3 in total) + chown -R analytics /wmf/data/wmf/mediawiki_private (raw data already chowned with analytics:analytics) + restart of the coords with the analytics user [11:04:01] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:45:52] * elukey lunch! [13:24:51] (03PS20) 10Elukey: Add artifacts for Debian Buster and upgrade to 0.32rc2 [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/495182 (https://phabricator.wikimedia.org/T212243) [13:27:15] (03CR) 10Nuria: [C: 03+1] "Looks good to me but i do not know much about d3 best practices so leaving merge up to @fdans. Seems like we need to remove one of the pro" (032 comments) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/504429 (https://phabricator.wikimedia.org/T218045) (owner: 10Milimetric) [14:16:24] anyone ever get a `TSocket read 0 bytes (code THRIFTTRANSPORT): TTransportException('TSocket read 0 bytes',)` error from Hue? [14:20:06] bearloga: that looks very weird, what were your trying to do? [14:20:56] elukey: just trying to run a query [14:22:44] bearloga: mmm never use Hue for that, but Hive directly.. any special use case? [14:25:48] elukey: nothing special. never encountered that error before. Hue has been real weird lately. a lot of "java vm out of heap memory" errors lately https://www.irccloud.com/pastebin/VrgZy1HO/ [14:26:31] heya team [14:26:35] oh... alarms... [14:31:10] bearloga: might be something to follow up, would you mind to open a task with the repro and the error? [14:31:22] elukey: sure! [14:39:09] thanks! [14:48:49] (03CR) 10Milimetric: [C: 04-1] Improve restart examples after deploy this week (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/508927 (owner: 10Milimetric) [14:53:01] milimetric, wanna chat 5 mins before our meeting with James? [14:53:19] mforns: I'll jump in the hangout, sure [14:53:26] (not cave, the meeting one) [14:53:30] ok :] [15:03:17] milimetric,mforns - as FYI I am attemping to re-run webrequest load for the failed hour, just to rule out any weirdness [15:05:14] 10Analytics, 10Analytics-Kanban, 10Operations, 10Traffic, 10Patch-For-Review: Add prometheus metrics for varnishkafka instances running on caching hosts - https://phabricator.wikimedia.org/T196066 (10fgiunchedi) a:05mforns→03colewhite Reporting here a chat between me and @Ottomata re: metric naming a... [15:05:57] 10Analytics: "TSocket read 0 bytes" error in Hue when querying - https://phabricator.wikimedia.org/T222895 (10mpopov) [15:21:22] 10Analytics, 10EventBus, 10serviceops, 10Core Platform Team Backlog (Watching / External), 10Services (watching): Set up LVS for eventgate-main on port 32192 - https://phabricator.wikimedia.org/T222899 (10Ottomata) [15:29:07] (03CR) 10Fdans: "Nice job on the async load of the topojson." (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/504429 (https://phabricator.wikimedia.org/T218045) (owner: 10Milimetric) [15:33:21] mforns: you're right, this should always join at least to the current hour: https://github.com/wikimedia/analytics-refinery/blob/master/hive/webrequest/check_dataloss_false_positives.sparksql#L68-L71 [15:33:30] we're obviously missing something in the SQL [15:34:04] 10Analytics, 10Analytics-Kanban, 10Operations, 10Traffic, 10Patch-For-Review: Add prometheus metrics for varnishkafka instances running on caching hosts - https://phabricator.wikimedia.org/T196066 (10Ottomata) I think there are a few more branches: - producer, consumer - global, per broker, per topic, p... [15:39:24] (03PS5) 10Milimetric: Change projection of world map to eckert 3 [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/504429 (https://phabricator.wikimedia.org/T218045) [15:39:42] (03CR) 10Milimetric: Change projection of world map to eckert 3 (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/504429 (https://phabricator.wikimedia.org/T218045) (owner: 10Milimetric) [15:41:26] 10Analytics, 10EventBus, 10Core Platform Team (Modern Event Platform (TEC2)), 10Core Platform Team Kanban (Doing), 10Services (doing): Make EventBus extension support configurable per-event/stream EventServiceName - https://phabricator.wikimedia.org/T222822 (10Pchelolo) [15:44:03] (03CR) 10Milimetric: [C: 03+1] "cool with me, just waiting on the tests then" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/499968 (https://phabricator.wikimedia.org/T219112) (owner: 10Fdans) [15:47:10] (03PS21) 10Elukey: Add artifacts for Debian Buster and upgrade to 0.32rc2 [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/495182 (https://phabricator.wikimedia.org/T212243) [15:57:45] a-team better use of dat ameeting is today [15:57:47] will miss standup [15:57:53] but will be at grooming [15:58:06] mforns: I will do the refine puppet change after meetings [15:58:59] k [15:59:30] ooo elukey "MirrorMaker 2.0 is in the Kafka 2.3.0 release plan. Please take a look so [15:59:30] we can get this merged." [16:00:29] ping fdans joal [16:02:53] hi all [16:03:04] i haven't been able to run queries with Hue for the last couple days [16:03:17] yesterday i was getting GC errors and Java heap space errors [16:03:27] today, I get: TSocket read 0 bytes (code THRIFTTRANSPORT): TTransportException('TSocket read 0 bytes',) [16:03:41] does anyone else experience this? [16:07:24] PROBLEM - HDFS Namenode RPC 8020 call queue length on an-master1001 is CRITICAL: 285 ge 20 https://grafana.wikimedia.org/dashboard/db/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen [16:08:09] 10Analytics, 10Analytics-Kanban, 10Operations, 10Traffic, 10Patch-For-Review: Add prometheus metrics for varnishkafka instances running on caching hosts - https://phabricator.wikimedia.org/T196066 (10Ottomata) BTW for reference here's what EvenGate is currently exporting: https://gist.github.com/ottomat... [16:10:18] Marshall_: yep! bearloga opened https://phabricator.wikimedia.org/T222895, can you add your case in there? [16:10:25] I guess you are using Hue right? [16:12:19] one job is breaking the cluster [16:13:08] !log killed application_1555511316215_77583 from Yarn CLI [16:13:09] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:14:18] RECOVERY - HDFS Namenode RPC 8020 call queue length on an-master1001 is OK: (C)20 ge (W)10 ge 0 https://grafana.wikimedia.org/dashboard/db/hadoop?var-hadoop_cluster=analytics-hadoop&orgId=1&panelId=54&fullscreen [16:28:36] (03CR) 10Ladsgroup: [C: 03+1] "Given that it's not private data, analytics seems fine. Do we have a wmde-analytics user? If not, should we?" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/509016 (https://phabricator.wikimedia.org/T220971) (owner: 10Elukey) [16:30:24] 10Analytics, 10Analytics-Kanban, 10EventBus, 10serviceops, and 3 others: Set up LVS for eventgate-main on port 32192 - https://phabricator.wikimedia.org/T222899 (10fdans) p:05Triage→03High [16:30:41] 10Analytics, 10Analytics-Kanban, 10EventBus, 10serviceops, and 3 others: Set up LVS for eventgate-main on port 32192 - https://phabricator.wikimedia.org/T222899 (10fdans) [16:31:02] @elukey -- yes, I am using Hue and seeing the same things as @bearloga. I'll add to the Phab task. [16:31:15] thanks :) [16:31:49] 10Analytics: "TSocket read 0 bytes" error in Hue when querying - https://phabricator.wikimedia.org/T222895 (10fdans) p:05Triage→03High [16:31:54] 10Analytics: "TSocket read 0 bytes" error in Hue when querying - https://phabricator.wikimedia.org/T222895 (10MMiller_WMF) I am experiencing the exact same things -- both the TSocket error and the heap space error. [16:33:04] 10Analytics, 10EventBus, 10Core Platform Team (Modern Event Platform (TEC2)), 10Core Platform Team Kanban (Doing), and 2 others: Make EventBus extension support configurable per-event/stream EventServiceName - https://phabricator.wikimedia.org/T222822 (10fdans) p:05Triage→03High [16:33:05] Marshall_: I am wondering one thing - can you retry now? [16:33:26] we just had an issue with the HDFS masters being overwhelmeed [16:33:59] Marshall_: yeah, it might just be an unfortunate coincidence, we happened to deploy yesterday and restart a bunch of big jobs [16:34:00] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review: Make Refine use JSONSchemas of event data to support Map types and proper types for integers vs decimals - https://phabricator.wikimedia.org/T215442 (10fdans) [16:34:05] 10Analytics, 10Analytics-Kanban, 10Discovery-Analysis, 10EventBus, and 2 others: Fix active EventLogging schemas that added backwards incompatable required fields. - https://phabricator.wikimedia.org/T222656 (10fdans) 05Open→03Declined Given the amount of work required to fix this and with eventgate th... [16:34:10] just tried again and got TSocket [16:35:05] Marshall_: send the query (or point me to a file somewhere if it has private details) [16:35:44] 10Analytics: Tracking failures in your Matomo Analytics - https://phabricator.wikimedia.org/T222651 (10fdans) p:05Triage→03Low [16:37:35] ah Hive is OOMing [16:37:36] sigh [16:38:15] !log restart hive-server2 on an-coord1001 due to OOMs [16:38:16] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:38:54] Marshall_: mind to try again? [16:39:50] it worked! [16:40:00] super weird, not sure what happened [16:41:54] thank you [16:44:12] thanks for the report! [16:44:55] 10Analytics: "TSocket read 0 bytes" error in Hue when querying - https://phabricator.wikimedia.org/T222895 (10elukey) Hive server 2 was OOMing, still not sure why. After a restart it seems working fine again.. Will need to dig a bit deeper in the logs! [16:56:29] elukey: see my email? [16:56:54] groceryheist: o/ [16:57:02] thanks a lot, sorry for the kill :( [16:57:17] yeah the main issue is that we don't have a good way to throttle at the HDFS master level [16:57:28] all those temporary files were generating a RPC call each to the HDFS master [16:57:36] "hey create this temp file please" etc.. [16:57:44] and it was building a huge backlog [16:58:44] huh, any ideas what I can do differently? [17:00:11] maybe i'm partitioning too granularly [17:01:25] yeah that is usually the issue [17:01:32] since you were creating million of files [17:02:13] ah yeah that's probably it [17:02:43] also, the spark max number of workers should be tuned to something like 128 to avoid saturating the cluster (if possible) [17:02:54] because other people might come in and need powa :) [17:03:26] Joseph wrote some docs in [17:03:27] https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Spark#Spark_tuning_for_big_jobs [17:04:00] but in your case, I think that 600 is a bit too much + the partitioning is super granula [17:04:03] *granular [17:04:31] oooo that's good [17:04:34] thanks [17:04:52] Joseph is out today but he'll probably follow up in the email when he'll read it [17:05:11] ok i don't need that many workers [17:05:21] actually i restarted the job and looks like it's almost finished already [17:05:24] :) [17:06:15] I found an optimization that made the many workers unneccesary. [17:07:51] sorry and thank you! [17:07:57] nice! Thanks a lot for the patience [17:08:26] groceryheist: if you can follow up in the email with the fix it would be awesome (so all will know what happened etc..) [17:14:02] elukey: done [17:17:52] saw it, thanks! [18:02:24] mforns: if you are still there: https://gerrit.wikimedia.org/r/c/operations/puppet/+/508863 [18:15:39] (merged) [18:15:43] will watch the next run [18:29:50] ottomata, ok [18:40:14] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review: Make Refine use JSONSchemas of event data to support Map types and proper types for integers vs decimals - https://phabricator.wikimedia.org/T215442 (10Ottomata) yessss! ` 19/05/09 18:36:56 INFO Refine: Successfully refined 1 of 1 dataset pa... [18:40:35] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review: Make Refine use JSONSchemas of event data to support Map types and proper types for integers vs decimals - https://phabricator.wikimedia.org/T215442 (10Ottomata) [18:41:49] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review: Make Refine use JSONSchemas of event data to support Map types and proper types for integers vs decimals - https://phabricator.wikimedia.org/T215442 (10Ottomata) I'd say this task is done. We still need to migrate all of the EventBus ones to... [19:11:12] ottomata, can you help me create a dir in an-coord1001? I need to create the local folder to store salts [19:11:30] the place I initially imagined is /etc/refinery/eventlogging_sanitization_salts [19:12:09] and as saltrotate is run by the user hdfs, I thought the dir should also belong to the hdfs user [19:12:37] however, /etc/refinery belongs to root, so I can not create the folder [19:12:40] no permits [19:15:17] mforns: seems like the puppet code should do it , right? [19:15:30] mforns: so if we swap machines the directory creation is automated [19:15:53] nuria, yes, but I need to put the salt file in there first, or else saltrotate will create another one... [19:16:41] well... theoretically, if we merge the puppet patch now and wait for puppet agent to run, it will create the directory, and I still will have time to copy the salt over... [19:16:53] before midnight [19:17:01] hm... [19:17:10] mforns: seems like it should be puppet the one that copies the salt no? [19:17:24] mforns: we can mege and run puppet no? [19:18:37] wait [19:18:57] puppet will create the local salt directory in an-coord1001 [19:19:05] mforns: aham [19:19:40] then we should add there the current salt for this quarter with correct naming [19:20:03] so that when saltrotate runs this midnight, it doesn't create another one [19:20:30] also, when we merge that puppet patch, the sanitization will start running with the new refinery-source code, [19:20:40] so salts should be in hdfs already in the correct format [19:20:51] OK, I think the plan is: [19:21:16] 1) merge puppet directory creation 2) run puppet 3) move salt by hand [19:21:17] 1) create the salts directory in HDFS with current salt correctly named [19:21:43] 2) merge puppet change and wait for puppet agent to run [19:22:01] mforns: we can run puppet right after [19:22:09] 3) move salt by hand to local copy in an-coord1001 (will be in HDFS already) [19:22:12] right [19:22:32] ok, but before we merge, I have to create the salt structure in HDFS [19:23:00] mforns: but the current sanitization has teh salt , just the one right? [19:23:19] mforns: not the one from last quarter as well, correct? [19:24:27] nuria, yes, but in another format [19:25:15] the new way of storing salts is within a directory, and the salts are named: "__" [19:25:42] now, there's only 1 salt named: eventlogging_sanitization_salt.txt [19:26:23] mforns: k, let's do it [19:26:31] ping ottomata for the puppet merge [19:26:46] yep, doing salts in HDFS [19:26:55] mforns: it really does not matter if puppet is merged first right? cause dir will be empty [19:27:08] mforns: you want to bc? (fine to say no) [19:27:18] mforns: is the salt generation documented anywhere? [19:28:41] nuria, it does matter, because puppet also changes the jar version that will be executed [19:28:48] yes bc! [19:29:06] k [19:29:43] mforns: on bc [19:30:50] mforns: sorry missed you rping! [19:31:14] ottomata, np, still working on the salts [19:31:25] not to be merged yet [19:32:21] mforns: ok let me know what you need, i'm here! [19:39:10] k thanks! [19:44:18] ok ottomata, I created the necessary salts in HDFS for sanitization to run, I think the puppet patch is ready to be merged [19:44:25] ok [19:44:34] patch? [19:44:44] https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/485063/ [19:44:59] please check that it looks good! [19:52:37] looking [19:53:10] mforns: will we every use this salt idea for things other than 'eventlogging'? [19:53:26] dunno maybe [19:53:31] maybe better direcotry: [19:53:41] but each system that uses it will have its own directory [19:53:48] $config_dir/salts/eventlogging_sanitization [19:53:49] ? [19:54:04] oh, yea [19:54:06] makes sense to me [19:54:26] will change [19:54:31] k [19:56:01] hm ,and i tihnk you don't need the if defined(File... stuff in this case mforns [19:56:09] this is in a class that will only be included once on this node [19:56:15] once per node [19:56:42] the if !defined is only needed if you are making a reusable define instead of a class [19:56:57] or if there was another class out there that defined the file [19:57:00] ah! [19:57:31] so just file { ... } ? [20:00:06] yup [20:00:22] if we do add more salt stuff users later [20:00:28] we can factor this out into another class [20:00:29] its fine here for now [20:00:38] you can even make in one go like: [20:01:11] file { "${...config_dir}/salts", "${...config_dir}/salts/eventlogging_sanitization": ... } [20:01:15] oh sorry [20:01:18] in array [20:01:35] file { ["/file/one", "/file/one/two" ]: ... } [20:01:41] mforns: here if you want to bc again [20:06:16] ottomata, great thanks [20:06:53] nuria, depends on you, wanna go over the code or something? [20:07:28] mforns: no, if andrew has cr-ed it , i can be tour rubber duck (if you need one) for the enabling steps [20:09:15] nuria, I think we're good: 1) set up salt directory in HDFS [DONE] 2) apply minor changes Andrew suggested to puppet patch 3) merge puppet patch and run puppet agent 4) copy the current salt over to an-coord1001 local salt directory [20:12:26] mforns: k! [20:15:06] ottomata, can I do: $config_dir = ::profile::analytics::refinery::config_dir ? [20:15:23] or do I need to: $config_dir = "${::profile::analytics::refinery::config_dir}" ? [20:15:32] couldn't find any other example in the repo [20:15:51] ya you can do that mforns [20:15:55] k [20:16:18] (03CR) 10Fdans: [C: 03+2] Change projection of world map to eckert 3 [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/504429 (https://phabricator.wikimedia.org/T218045) (owner: 10Milimetric) [20:16:18] the quotes are only needed if you are e.g. appending to the string [20:16:24] $config_dir = $::profile::... [20:21:55] (03PS12) 10Fdans: Replace time range selector [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/499968 (https://phabricator.wikimedia.org/T219112) [20:23:26] ottomata, yea I was unsure if an expression starting with :: was like a regular variable or had some special conditions [20:23:39] ruby [20:26:42] (03CR) 10jerkins-bot: [V: 04-1] Replace time range selector [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/499968 (https://phabricator.wikimedia.org/T219112) (owner: 10Fdans) [20:27:50] mforns: it is just fully qualified like that [20:27:54] scoping is really weird in puppet [20:28:01] aha [20:28:02] so it is safest to alwasy fully qualify starting with :: [20:28:19] e.g. if there was a globally defined $config_dir [20:28:24] and a locally defined $config_dir [20:28:31] $::config_dir would reference the global one [20:28:38] if there was no locally defined $config_dir [20:28:41] so would $config_dir [20:28:59] it gets even weirder with class inheritance, which we avoid using altogether now [20:29:23] I see [20:29:27] makes sense [20:34:14] ottomata, I think I fixed it: https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/485063/ [20:36:37] ok mforns let's go! [20:36:57] k! [20:36:58] oh weiird a really old draft comment just got posted? [20:37:00] weird. [20:37:11] ? [20:37:18] in gerrit apparently i had a draft comment [20:37:25] and i Just replied with the PCC output link [20:37:29] and my old comment got posted allong with it [20:37:30] i guess [20:37:33] not relevatn anymore [20:37:45] k [20:37:47] mforns: i should merge yes? [20:38:00] yes :] [20:43:56] ok merged and puppet run [20:44:11] mforns: your dir is there [20:44:27] :D [20:44:34] will add the salt file no [20:44:36] now [20:47:04] ok, done. this should be ready for next run of ELsanitization, and also next run of saltrotate [20:47:05] (03CR) 10Milimetric: [C: 04-1] "oops, the project reactivity is still broken on the dashboard, errors when changing projects from the wiki selector" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/499968 (https://phabricator.wikimedia.org/T219112) (owner: 10Fdans) [20:47:09] crossing fingers [20:47:12] thanks, ottomata [20:50:06] yup! [21:05:52] (03PS13) 10Fdans: Replace time range selector [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/499968 (https://phabricator.wikimedia.org/T219112) [21:06:22] (03CR) 10Fdans: "milimetric: yea I was silly and wrote project instead of this.project" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/499968 (https://phabricator.wikimedia.org/T219112) (owner: 10Fdans) [21:08:17] (03CR) 10jerkins-bot: [V: 04-1] Replace time range selector [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/499968 (https://phabricator.wikimedia.org/T219112) (owner: 10Fdans) [21:10:15] (03PS14) 10Fdans: Replace time range selector [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/499968 (https://phabricator.wikimedia.org/T219112) [21:13:05] (03CR) 10jerkins-bot: [V: 04-1] Replace time range selector [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/499968 (https://phabricator.wikimedia.org/T219112) (owner: 10Fdans) [21:40:25] (03PS15) 10Fdans: Replace time range selector [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/499968 (https://phabricator.wikimedia.org/T219112) [21:43:21] (03CR) 10jerkins-bot: [V: 04-1] Replace time range selector [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/499968 (https://phabricator.wikimedia.org/T219112) (owner: 10Fdans) [21:59:05] (03PS16) 10Fdans: Replace time range selector [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/499968 (https://phabricator.wikimedia.org/T219112) [22:02:13] 10Analytics, 10Analytics-Kanban, 10Operations, 10Traffic, 10Patch-For-Review: Add prometheus metrics for varnishkafka instances running on caching hosts - https://phabricator.wikimedia.org/T196066 (10colewhite) I agree with dropping the prefix in favor of "rdkafka". I see the type branch (producer and c... [22:05:02] so tests are now V+2, which totally makes sense because my only change compared to the previous PS was removing two spaces [22:09:15] of course they are V+2 in the first place because I removed the router tests, which now I have to put back in there [22:10:10] (03PS17) 10Fdans: Replace time range selector [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/499968 (https://phabricator.wikimedia.org/T219112)