[00:06:58] 10Analytics, 10Analytics-EventLogging, 10Growth-Team, 10MediaWiki-extensions-GuidedTour, 10User-DannyS712: GuidedTour fails if EventLogging isn't installed, but doesn't explicitly require it - https://phabricator.wikimedia.org/T264145 (10DannyS712) [02:08:20] 10Analytics, 10Analytics-EventLogging, 10Growth-Team, 10MediaWiki-extensions-GuidedTour, 10User-DannyS712: GuidedTour fails if EventLogging isn't installed, but doesn't explicitly require it - https://phabricator.wikimedia.org/T264145 (10Reedy) From README >* Install the EventLogging extension, which is... [02:24:55] 10Analytics, 10Analytics-EventLogging, 10Growth-Team, 10MediaWiki-extensions-GuidedTour, 10User-DannyS712: GuidedTour fails if EventLogging isn't installed, but doesn't explicitly require it - https://phabricator.wikimedia.org/T264145 (10DannyS712) >>! In T264145#6503825, @Reedy wrote: > From README > >... [03:10:49] mforns: finally i got the staff with the entropy that i wanted to get going going so i will hopefully join apacheconf tomorrow! [04:31:19] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Epic: Add data quality alarm for mobile-app data - https://phabricator.wikimedia.org/T257692 (10Nuria) This is the daily timeseries for entrophy of os_family on the 'mobile-app' access_method {F32368403} And this is the hourly entropy of os_versions... [05:44:50] good morning [05:47:13] !log "PURGE BINARY LOGS BEFORE '2020-09-22 00:00:00';" on an-coord1001's mariadb - T264081 [05:47:15] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [05:47:16] T264081: Increase in usage of /var/lib/mysql on an-coord1001 after Sept 21st - https://phabricator.wikimedia.org/T264081 [05:49:28] 10Analytics: Increase in usage of /var/lib/mysql on an-coord1001 after Sept 21st - https://phabricator.wikimedia.org/T264081 (10elukey) I had a chat with Francisco and the bulk load should in theory last another 3/4 days (there is currently an issue under investigation though, so bulk loading is stopped atm). I... [06:11:59] 10Analytics: Fix Maxmind geoip database archive - https://phabricator.wikimedia.org/T264152 (10elukey) [06:22:50] 10Analytics-Radar, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker11[02-17] - https://phabricator.wikimedia.org/T259071 (10elukey) Executed the cookbook also up to an-worker1110, all looks good! [06:24:13] just executed the init workers cookbook on an-worker1103-1110, all 12 partitions set up correctly in a couple of minutes [06:24:48] 10Analytics-Radar, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker11[02-17] - https://phabricator.wikimedia.org/T259071 (10elukey) 05Stalled→03Open [06:25:54] klausman: o/ good morning - when you are online let's chat about https://phabricator.wikimedia.org/T255138 [06:58:24] (03CR) 10Elukey: [V: 03+2 C: 03+2] Update to Superset 0.37.2 [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/627738 (https://phabricator.wikimedia.org/T262162) (owner: 10Elukey) [06:58:39] upgrading superset! [07:04:38] !log superset upgraded to 0.37.2 on analytics-tool1004 - T262162 [07:04:41] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [07:04:41] T262162: Upgrade to Superset 0.37.x - https://phabricator.wikimedia.org/T262162 [07:29:53] !log execute "alter table superset_production.alerts drop key ix_alerts_active;" on db1108's analytics-meta instance to fix replication after Superset upgrade - T262162 [07:29:56] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [07:29:57] T262162: Upgrade to Superset 0.37.x - https://phabricator.wikimedia.org/T262162 [07:41:18] * elukey taking a little break [08:13:32] 10Analytics-Clusters: Co-locate Presto with Hadoop worker nodes - https://phabricator.wikimedia.org/T256108 (10elukey) This work is paused until we test https://www.alluxio.io/ (included in bigtop). [08:14:36] 10Analytics-Clusters, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker10[18-40] - https://phabricator.wikimedia.org/T260445 (10elukey) >>! In T260445#6431168, @Cmjohnson wrote: > @elukey Are you trying to re-use hostnames? We should be using an-worker1118+ Sorry didn't see... [08:25:54] 10Analytics-Clusters, 10Analytics-Kanban, 10Patch-For-Review: Create a cookbook to automate the bootstrap of new Hadoop workers - https://phabricator.wikimedia.org/T262189 (10elukey) Tested it multiple times, works fine! [08:27:09] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Set up environment for Product Analytics system user - https://phabricator.wikimedia.org/T258970 (10elukey) @mpopov is there any action item left to do from me? I lost track of what it is needed, if so please tell me and I'll do :) [08:30:51] 10Analytics-Clusters, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker10[18-40] - https://phabricator.wikimedia.org/T260445 (10elukey) @Cmjohnson I checked the items listed in the package slip but I don't see the quantity, only the fact that a 480G disk is listed (T258727#64... [08:32:26] 10Analytics-Clusters, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker10[18-41] - https://phabricator.wikimedia.org/T260445 (10elukey) [08:32:34] 10Analytics-Clusters, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker10[18-41] - https://phabricator.wikimedia.org/T260445 (10elukey) [08:36:05] 10Analytics-Clusters, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Repurpose db1108 as generic Analytics db replica - https://phabricator.wikimedia.org/T234826 (10elukey) The db1108 host has been running fine for the past couple of months, replicating correctly analytics-meta and piwik data. Ther... [08:37:48] 10Analytics-Kanban, 10Better Use Of Data, 10Product-Analytics: Superset Updates - https://phabricator.wikimedia.org/T211706 (10elukey) [09:11:18] 10Analytics-Radar, 10Operations, 10Patch-For-Review: Move Hue to a Buster VM - https://phabricator.wikimedia.org/T258768 (10MoritzMuehlenhoff) >>! In T258768#6497946, @elukey wrote: > Currently two outstanding UI issues: > In theory those are not blocking the migration of Hue, so this task could probably be... [09:23:47] elukey: I am awake and my brain is actually working now :) [09:24:41] :) [09:28:25] so about https://phabricator.wikimedia.org/T255138 - if you have time it could be an interesting one to do when not busy with ats [09:30:45] I haven't made much progress with the ats switch due to CTS (mostly been watching talks and reading docs) :-S [09:33:23] yep yep not a problem, I can take care of the above task, I asked if you were interested since it is about gpu deployments on hadoop workers etc.. [09:34:45] I can help with it, as long as it's not too much typing :) [09:35:47] klausman: nono sorry I completely forgot about it, I'll add you to code reviews so we can do it together [09:36:00] sounds good [10:23:37] rack patch looks good [10:24:34] Same for the rocm33 on Stretch patch [10:25:40] thanks! [10:26:29] Be aware of the hang-on-install for the rocm driver. A reboot (needing a full powercycle) usually fixes it. [10:33:05] yeah :( [10:33:11] I am going afk for lunch! ttl :) [10:41:17] same [10:56:25] 10Analytics-Clusters: Upgrade to Superset 0.37.x - https://phabricator.wikimedia.org/T262162 (10MoritzMuehlenhoff) >>! In T262162#6497439, @elukey wrote: > New version is out: 0.37.2 > > The new code is currently deployed on an-tool1005 (staging instance) for further testing. As soon as the Analytics team is do... [12:18:55] (03CR) 10Mforns: [C: 03+1] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/630682 (owner: 10Joal) [14:13:14] (03PS6) 10Mforns: Improve path discovery in drop-older-than [analytics/refinery] - 10https://gerrit.wikimedia.org/r/628933 (https://phabricator.wikimedia.org/T263495) [15:36:00] klausman: I just found that /opt/rocm/bin/rocminfo on stat1005 hangs badly (goes into D+ state) [15:36:10] it is a util that shows basic info about the GPU [15:37:03] on an-worker1097 it works [15:37:03] elukey@an-worker1097:~$ sudo /opt/rocm/bin/rocminfo [15:37:03] ROCk module is loaded [15:37:05] etc.. [15:37:10] very strange [15:38:00] 10Analytics-Radar, 10Better Use Of Data, 10Product-Analytics, 10Product-Infrastructure-Data, and 4 others: Session Length Metric. Web implementation - https://phabricator.wikimedia.org/T248987 (10jlinehan) The instrument is now merged, thanks to @Krinkle @Gilles @Nuria @mforns @Mholloway for providing such... [15:39:15] ah snap I am also trying a tensorflow script, it hangs [15:41:29] elukey: is rocminfo rcm33? And what base distro are the two hosts? [15:41:58] klausman: it used to work fine, I remember that I tested it on stat1005 after the upgrade [15:42:17] but the fact that also tf doesn't work smells like gpu in a weird state [15:43:06] I can try https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/AMD_GPU#Reset_the_GPU_state [15:43:21] it didn't work before your work with dkms, mayyybee it does now [15:43:22] dmesg shows oom kills. Could that have left the GPU in a bad state? [15:43:27] no idea.. [15:43:57] I'd give the reset a try. At worst, we'll have to power cycle the machine :-/ [15:44:02] ok done [15:44:06] [Wed Sep 30 15:44:02 2020] amdgpu 0000:06:00.0: GPU reset(1) succeeded! [15:44:24] still processes stuck in D state [15:44:44] what's odd is that two are in D, one is in S [15:44:51] oh nvm, that's just the sudp [15:47:07] I suspect we'll need to rebot.. [15:47:10] *reboot [15:48:04] on 1008 works fine [15:48:32] Still, not very confidence-inspiring [15:49:21] yes I agree [15:49:51] also, to add more fun, my tf2 test script on an-worker1097 (stretch but with the same xenial amd rocm packages) doesn't find any gpu [15:50:00] BUT all the cli tools etc.. show that there is one [15:50:28] weird. sounds a bit like a permission issue [15:53:19] I thought the same, but still haven't figured it out [15:53:35] I manually apt-get installed the packages on that host, maybe something is missing [15:53:56] anyway, we'll see, does it seem ok to schedule a reboot? [16:05:45] There is a bunch of stuff running, but I dunno how much of a nuisance a locked GPU is [16:06:19] Maybe with a 30m or 1h pre-warning? I figure most people in EMEA and NA should be awake at this time [16:07:12] ah yes yes with "schedule" I meant tomorrow [16:07:16] or even in two days [16:07:30] to respect the 48h warning that we agreed [16:08:14] I also may have found the problem with stretch [16:08:15] elukey@an-worker1097:~$ ls -l /dev/kfd [16:08:15] crw------- 1 root root 242, 0 Sep 30 14:38 /dev/kfd [16:08:21] elukey@stat1008:~$ ls -l /dev/kfd [16:08:22] crw-rw---- 1 root render 240, 0 Sep 16 12:08 /dev/kfd [16:11:24] 10Analytics, 10Product-Analytics, 10Structured Data Engineering, 10Patch-For-Review, and 2 others: Develop a new schema for MediaSearch analytics or adapt an existing one - https://phabricator.wikimedia.org/T263875 (10egardner) [16:11:54] 10Analytics, 10Analytics-EventLogging, 10Growth-Team, 10MediaWiki-extensions-GuidedTour, 10User-DannyS712: GuidedTour fails if EventLogging isn't installed, but doesn't explicitly require it - https://phabricator.wikimedia.org/T264145 (10Reedy) Aye, I'm not disagreeing :) [16:12:07] on buster we add every user in privatedata to the 'render' group, that it is not on stretch [16:14:29] 10Analytics, 10Product-Analytics, 10Structured Data Engineering, 10Patch-For-Review, and 2 others: Develop a new schema for MediaSearch analytics or adapt an existing one - https://phabricator.wikimedia.org/T263875 (10egardner) [16:16:42] I wonder if the gid there is a kernel module option [16:18:19] mforns: have you managed to join? [16:18:41] joal: yes, but don't find the analytics chan [16:18:50] mforns: wikimedia :) [16:18:59] oh, ok [16:20:58] klausman: we are all following apachecon so let's skip the ops sync [16:27:46] Roger [16:54:48] 10Analytics-Clusters, 10Analytics-Kanban, 10Patch-For-Review: Put 6 GPU-based Hadoop worker in service - https://phabricator.wikimedia.org/T255138 (10elukey) I manually installed the rocm packages on an-worker1097, rebooted and tested with a simple tensorflow script, but the GPU is not recognized. I think th... [16:58:00] 10Analytics-Clusters, 10Analytics-Kanban: Upgrade to Superset 0.37.x - https://phabricator.wikimedia.org/T262162 (10elukey) [17:23:27] 10Analytics, 10Operations, 10Traffic: varnishkafka 1.1.0 CPU usage increase - https://phabricator.wikimedia.org/T264074 (10herron) p:05Triage→03High [17:34:37] 10Analytics-Clusters, 10Analytics-Kanban, 10Patch-For-Review: Put 6 GPU-based Hadoop worker in service - https://phabricator.wikimedia.org/T255138 (10elukey) @MoritzMuehlenhoff for this use case I'd try to add `component/systemd241` that should bring in the `render` group (since we also have automation to ad... [17:44:15] 10Analytics-Radar, 10Better Use Of Data, 10Product-Analytics, 10Product-Infrastructure-Data, and 3 others: [EPIC] Count unique iOS & Android users precisely and in a privacy conscious manner that does not require opt in to send data - https://phabricator.wikimedia.org/T202664 (10kzimmerman) [17:49:40] 10Analytics-Clusters, 10Analytics-Kanban, 10Patch-For-Review: Put 6 GPU-based Hadoop worker in service - https://phabricator.wikimedia.org/T255138 (10elukey) Tried with: ` elukey@an-worker1097:~$ cat /etc/udev/rules.d/70-kfd.rules SUBSYSTEM=="kfd", KERNEL=="kfd", TAG+="uaccess", GROUP="video" ` And it work... [18:06:38] 10Analytics-Radar, 10Better Use Of Data, 10Product-Analytics, 10Product-Infrastructure-Data: session_tick stream configs - https://phabricator.wikimedia.org/T256311 (10mpopov) p:05Medium→03High Raising the priority for this since we want to start using the new session tracking instrument as soon as pos... [19:00:08] mforns: do we test /run the drop script with python3 or 2.7? [19:00:23] nuria: hmmm [19:00:40] nuria: python3 [19:16:01] 10Analytics-Radar, 10Better Use Of Data, 10Product-Analytics, 10Product-Infrastructure-Data, and 4 others: Session Length Metric. Web implementation - https://phabricator.wikimedia.org/T248987 (10Krinkle) Be sure to enable it on Beta Cluste first as well so that it remains in sync with prod, and to allow f... [20:31:47] 10Analytics, 10Analytics-Kanban: Fix Maxmind geoip database archive - https://phabricator.wikimedia.org/T264152 (10Nuria) a:03razzi [20:32:56] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Debianize Python's pid library to be able to use it from reportupdater - https://phabricator.wikimedia.org/T262574 (10Nuria) 05Open→03Resolved [20:33:00] 10Analytics, 10Patch-For-Review, 10good first task: Reportupdater: do not write execution control files in source directories - https://phabricator.wikimedia.org/T173604 (10Nuria) [20:33:05] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Debianize Python's pid library to be able to use it from reportupdater - https://phabricator.wikimedia.org/T262574 (10Nuria) Can @mforns confirm that reportupdater can use this package as its? [20:34:33] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10good first task: Reportupdater: do not write execution control files in source directories - https://phabricator.wikimedia.org/T173604 (10Nuria) given that the missing pid package is now debianized let's deploy this code? [20:34:44] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10good first task: Reportupdater: do not write execution control files in source directories - https://phabricator.wikimedia.org/T173604 (10Nuria) Assigning to @mforns for final CR. [20:35:01] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10good first task: Reportupdater: do not write execution control files in source directories - https://phabricator.wikimedia.org/T173604 (10Nuria) a:05paulkernfeld→03mforns [20:36:13] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats, 10I18n, 10good first task: Add link to translatewiki.net in wikistats footer - https://phabricator.wikimedia.org/T261502 (10Nuria) 05Open→03Resolved [20:38:54] 10Analytics-Clusters, 10Analytics-Kanban, 10Patch-For-Review: Create a cookbook to automate the bootstrap of new Hadoop workers - https://phabricator.wikimedia.org/T262189 (10Nuria) 05Open→03Resolved [20:38:56] 10Analytics, 10Analytics-Kanban: Hadoop Hardware Orders FY2019-2020 - https://phabricator.wikimedia.org/T243521 (10Nuria) [20:40:56] 10Analytics, 10Analytics-Kanban: Fix Maxmind geoip database archive - https://phabricator.wikimedia.org/T264152 (10fdans) Thanks for filing @elukey. I like option 3 the best, but if I remember correctly, the logic for the backup to be present on the host is to make it easier to use in case past snapshots need... [20:58:24] fdans: in what repo is the max mind archive? [20:58:35] fdans: sorry, the script [20:58:47] fdans: not the archivals themselves [20:58:49] nuria: puppet [21:01:21] fdans: i see, this one : ./modules/profile/manifests/analytics/geoip/archive.pp [21:10:21] 10Analytics-Radar, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker11[02-17] - https://phabricator.wikimedia.org/T259071 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by robh on cumin1001.eqiad.wmnet for hosts: ` an-worker1111.eqiad.wmnet ` The log can be found... [21:32:19] 10Analytics-Radar, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker11[02-17] - https://phabricator.wikimedia.org/T259071 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['an-worker1111.eqiad.wmnet'] ` Of which those **FAILED**: ` ['an-worker1111.eqiad.wmnet'] ` [21:57:12] 10Analytics-Radar, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker11[02-17] - https://phabricator.wikimedia.org/T259071 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by robh on cumin1001.eqiad.wmnet for hosts: ` ['an-worker1114.eqiad.wmnet', 'an-worker1115.eqi... [22:04:58] 10Analytics-Radar, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker11[02-17] - https://phabricator.wikimedia.org/T259071 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by robh on cumin1001.eqiad.wmnet for hosts: ` an-worker1114.eqiad.wmnet ` The log can be found... [22:11:21] 10Analytics-Radar, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker11[02-17] - https://phabricator.wikimedia.org/T259071 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['an-worker1114.eqiad.wmnet'] ` Of which those **FAILED**: ` ['an-worker1114.eqiad.wmnet'] ` [22:12:44] 10Analytics-Radar, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker11[02-17] - https://phabricator.wikimedia.org/T259071 (10RobH) [22:35:29] 10Analytics-Radar, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker11[02-17] - https://phabricator.wikimedia.org/T259071 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['an-worker1115.eqiad.wmnet'] ` Of which those **FAILED**: ` ['an-worker1114.eqiad.wmnet'] ` [22:37:35] 10Analytics-Radar, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker11[02-17] - https://phabricator.wikimedia.org/T259071 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by robh on cumin1001.eqiad.wmnet for hosts: ` an-worker1116.eqiad.wmnet ` The log can be found... [22:37:58] 10Analytics-Radar, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker11[02-17] - https://phabricator.wikimedia.org/T259071 (10ops-monitoring-bot) Script wmf-auto-reimage was launched by robh on cumin1001.eqiad.wmnet for hosts: ` an-worker1117.eqiad.wmnet ` The log can be found... [22:40:10] 10Analytics-Radar, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker11[02-17] - https://phabricator.wikimedia.org/T259071 (10RobH) [22:42:41] (03PS2) 10Razzi: Test using mocked file tree in refinery-drop-older-than [analytics/refinery] - 10https://gerrit.wikimedia.org/r/630680 (https://phabricator.wikimedia.org/T263495) [22:44:59] (03CR) 10Razzi: Test using mocked file tree in refinery-drop-older-than (035 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/630680 (https://phabricator.wikimedia.org/T263495) (owner: 10Razzi) [22:48:52] 10Analytics-Radar, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker11[02-17] - https://phabricator.wikimedia.org/T259071 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['an-worker1117.eqiad.wmnet'] ` Of which those **FAILED**: ` ['an-worker1117.eqiad.wmnet'] ` [23:04:17] 10Analytics, 10Analytics-Kanban: Fix Maxmind geoip database archive - https://phabricator.wikimedia.org/T264152 (10Nuria) I think it will be fine to archive to hdfs alone. We use those files but sparingly so I do not think there is an issue with them being available just in hdfs. @razzi, see archive.sh in... [23:05:47] 10Analytics-Radar, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker11[02-17] - https://phabricator.wikimedia.org/T259071 (10ops-monitoring-bot) Completed auto-reimage of hosts: ` ['an-worker1116.eqiad.wmnet'] ` and were **ALL** successful. [23:07:26] 10Analytics-Radar, 10DC-Ops, 10Operations, 10ops-eqiad: (Need By: TBD) rack/setup/install an-worker11[02-17] - https://phabricator.wikimedia.org/T259071 (10RobH) [23:16:04] (03CR) 10Nuria: [C: 03+1] "Looks good to me" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/628933 (https://phabricator.wikimedia.org/T263495) (owner: 10Mforns) [23:27:03] (03CR) 10Nuria: [C: 03+1] "These tests are a nice addition, thank you." (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/630680 (https://phabricator.wikimedia.org/T263495) (owner: 10Razzi) [23:28:42] 10Analytics-Kanban, 10Better Use Of Data, 10Product-Analytics: Superset Updates - https://phabricator.wikimedia.org/T211706 (10Nuria) [23:29:23] 10Analytics-Clusters, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Upgrade the Hadoop test cluster to BigTop - https://phabricator.wikimedia.org/T244499 (10Nuria) 05Open→03Resolved [23:29:25] 10Analytics-Clusters, 10User-Elukey: Update to CDH 6 or other up-to-date Hadoop distribution - https://phabricator.wikimedia.org/T203693 (10Nuria) [23:30:15] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: Combine filters and splits on wikistats UI - https://phabricator.wikimedia.org/T249758 (10Nuria) ta-ta-channn!!! Let's close when @fdans sends announcement e-mail [23:31:02] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Patch-For-Review: Add dimensions to editors_daily dataset - https://phabricator.wikimedia.org/T256050 (10Nuria) Next quarter as in Q2 [23:40:25] 10Quarry, 10cloud-services-team (Kanban): Prepare Quarry for multiinstance wiki replicas - https://phabricator.wikimedia.org/T264254 (10Bstorm) [23:41:05] 10Quarry, 10cloud-services-team (Kanban): Prepare Quarry for multiinstance wiki replicas - https://phabricator.wikimedia.org/T264254 (10Bstorm) p:05Triage→03Medium [23:43:04] 10Quarry, 10cloud-services-team (Kanban): Prepare Quarry for multiinstance wiki replicas - https://phabricator.wikimedia.org/T264254 (10Bstorm) So far, I've discovered that the local vagrant dev environment has different DB encoding than the live DB, which is interesting. I believe I need to add a column to th... [23:55:09] 10Analytics, 10Security: Review request for data export - https://phabricator.wikimedia.org/T264255 (10leila)