[00:03:47] 10Analytics, 10Analytics-Kanban, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019): Develop a tool or integrate feature in existing one to visualize WMCS edits data - https://phabricator.wikimedia.org/T226663 (10srishakatux) @Milimetric @JAllemandou Hello! Thank you both for your help. This project is... [00:17:50] 10Analytics, 10Analytics-Kanban, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019): Develop a tool or integrate feature in existing one to visualize WMCS edits data - https://phabricator.wikimedia.org/T226663 (10bd808) a:05JAllemandou→03srishakatux > Milimetric reassigned this task from srishakatux... [00:20:41] 10Analytics, 10Pageviews-API: Track page views by page ID rather than title (handles moved pages) - https://phabricator.wikimedia.org/T159046 (10Pine) This lack of the logs accounting for page moves surprised me. Some discussion at https://en.wikipedia.org/wiki/Talk:2I/Borisov#Page_view_log_for_this_article. [04:49:27] 10Analytics, 10Analytics-Kanban, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019): Develop a tool or integrate feature in existing one to visualize WMCS edits data - https://phabricator.wikimedia.org/T226663 (10Nuria) ah , sorry about changing ownership I mean to assign to Analytics the subtask {T2335... [04:49:35] 10Analytics, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019): Develop a tool or integrate feature in existing one to visualize WMCS edits data - https://phabricator.wikimedia.org/T226663 (10Nuria) [05:02:12] 10Analytics, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019): Develop a tool or integrate feature in existing one to visualize WMCS edits data - https://phabricator.wikimedia.org/T226663 (10Nuria) @srishakatux you can use reportupdater to pull data out of hive , there is a change in place that is migra... [07:23:21] 10Analytics, 10Analytics-Kanban: Eventlogging to druid daily timers not executing? - https://phabricator.wikimedia.org/T234494 (10elukey) ` elukey@an-coord1001:~$ sudo systemctl list-timers NEXT LEFT LAST PASSED UNIT... [07:27:38] 10Analytics, 10Analytics-Kanban: Eventlogging to druid daily timers not executing? - https://phabricator.wikimedia.org/T234494 (10elukey) Ok now it makes sense: ` elukey@an-coord1001:~$ sudo systemctl cat eventlogging_to_druid_navigationtiming_daily # /lib/systemd/system/eventlogging_to_druid_navigationtiming... [07:47:54] !log restart superset to test if a stale status might cause data not to be shown [07:47:56] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [07:48:42] !log restart druid-broker on druid1003 (used by superset) [07:48:44] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [07:49:47] 10Analytics, 10Analytics-Kanban: superset not showing data after 09/16 for some datasources - https://phabricator.wikimedia.org/T234471 (10elukey) >>! In T234471#5542517, @Nuria wrote: > I see errors like: > > io.druid.java.util.common.RE: Failure getting results for query[7e7c183d-4d16-430e-8e53-ed88fd072125... [07:49:53] ah it worked [07:50:04] now I can see data in superset after the Sept 16th [07:50:07] oh my [07:56:34] Hi team [07:56:39] What a mess :( [08:15:16] (03CR) 10Joal: Add oozie job to load top mediarequests data (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/538880 (https://phabricator.wikimedia.org/T233717) (owner: 10Fdans) [08:22:47] 10Analytics, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019): Develop a tool or integrate feature in existing one to visualize WMCS edits data - https://phabricator.wikimedia.org/T226663 (10JAllemandou) >>! In T226663#5542860, @srishakatux wrote: > @Milimetric @JAllemandou Hello! Thank you both for you... [08:27:04] (03CR) 10Joal: [V: 03+2] "Tested on cluster" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/540124 (https://phabricator.wikimedia.org/T234333) (owner: 10Joal) [08:29:11] (03PS11) 10Joal: Add spark code for wikidata json dumps parsing [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/346726 [08:31:53] 10Analytics, 10Analytics-Kanban: superset not showing data after 09/16 for some datasources - https://phabricator.wikimedia.org/T234471 (10elukey) Found some old occurrences (~1 month ago) of the same error on druid1001's broker, but not on druid1002. Interestingly, 1001 and 1003 are the only ones that UIs con... [08:31:56] joal: --^ [08:32:00] does it make sense? [08:33:29] https://superset.wikimedia.org/superset/dashboard/search-query-suggestions/ works now [08:33:33] ebernhardson: --^ [08:34:53] elukey: I understand your finding, but I have no clue of how we got in that situation [08:35:29] elukey: It feels it could come when a broker is overwhelmed with requests it can't handle, but it's just an assumption [08:35:55] I am seeing reports like https://github.com/apache/incubator-druid/issues/6043 [08:36:09] but they are very generic [08:36:39] elukey: I also support widely nuria's point of reindexing datasources with small segments [08:37:56] sure [08:43:38] !log apply 5% threshold to the HDFS balancer - T231828 [08:43:41] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [08:43:41] T231828: Change HDFS balancer threshold - https://phabricator.wikimedia.org/T231828 [08:58:54] 10Analytics, 10Patch-For-Review, 10Performance-Team (Radar): Eventlogging processors are frequently failing heartbeats causing consumer group rebalances - https://phabricator.wikimedia.org/T222941 (10elukey) It seems that 1.4.7 is out https://github.com/dpkp/kafka-python/issues/1418#issuecomment-536756917 A... [08:58:58] 10Analytics, 10MinervaNeue, 10Readers-Web-Backlog (Kanbanana-2019-20-Q2): MinervaClientError sends malformed events - https://phabricator.wikimedia.org/T234344 (10pmiazga) a:03pmiazga [09:02:39] hi joal if you want we can talk about the mediawiki history stuff [09:02:48] hi fdans - sure [09:03:27] as side note, except from my mess yesterday with pageviews we should be running druid_loader.py with python3 now \o/ [09:03:41] well done elukey :) Thanks for that! [09:04:00] joal: i'm on bc [09:48:56] !log ran apt-get autoremove -y on all Hadoop workers to remove old Python 2 deps [09:49:04] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [09:57:09] 10Analytics, 10Research-Backlog, 10Wikidata: Copy Wikidata dumps to HDFs - https://phabricator.wikimedia.org/T209655 (10JAllemandou) this is done @GoranSMilovanovic. Raw data is here `/user/joal/wmf/data/raw/mediawiki/wikidata/all_jsondumps/20190902` and parquet data is here `/user/joal/wmf/data/wmf/mediawik... [10:01:19] hey teamm :] [10:01:34] o/ [10:04:39] mforns: was the rsync stopped yesterday? [10:12:52] elukey, yes: https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/540442/ [10:14:28] elukey, do you see any difference in stat1007's load? [10:15:04] not really, a ton of sockets opened [10:15:13] :C [10:16:06] ahhh I just noticed something [10:16:16] in ps a lot of rsyncs are listed in "D" state [10:16:23] that should be uninterruptible sleep [10:16:39] the might be hanging on the fuse mount point [10:16:44] * joal is after such an "uninterruptible sleep" [10:18:11] elukey@stat1007:~$ ls /mnt/hdfs/ [10:18:11] ls: cannot open directory '/mnt/hdfs/': Transport endpoint is not connected [10:22:09] ok I had to stop rsync.service and now analytics-mysql works fine [10:22:10] super quick [10:22:17] so it was definitely the rsync [10:22:25] but now the fuse mountpoint is in a weird state [10:25:32] mforns, fdans I see you logged in on stat1007, I may need to reboot it [10:25:54] elukey: no prob! [10:26:03] elukey, good to know, no problemo as well [10:26:11] ah no fixed [10:36:40] wow elukey hive is so much faster now [10:37:21] on stat1007 right? [10:39:33] yes [10:40:25] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Move the Analytics Refinery to Python 3 - https://phabricator.wikimedia.org/T204735 (10elukey) it seems to work, I just ran `apt-get autoremove` on all hadoop workers and no error surfaced. [10:42:03] super :) [10:42:13] the fuse mountpoint is really horrible [10:43:45] I am wondering if https://github.com/EDS-APHP/py-hdfs-mount could be better [11:11:44] (03PS1) 10Elukey: download-project-namespace-map: handle binary enconding for python3 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/540581 [11:13:15] (03PS2) 10Elukey: download-project-namespace-map: handle binary enconding for python3 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/540581 (https://phabricator.wikimedia.org/T204735) [11:23:27] (03PS3) 10Elukey: download-project-namespace-map: handle binary enconding for python3 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/540581 (https://phabricator.wikimedia.org/T204735) [11:24:56] plan to start a rolling restart of hadoop-yarn-nodemanager shortly. https://wikitech.wikimedia.org/wiki/Service_restarts#Hadoop_workers suggests they can just be done one at a time, anything elses to consider? [11:25:13] elukey: ^^ [11:25:30] jbond42: may I ask why? :) [11:26:16] openssl update [11:27:35] I can take care of it, but if you want you can do in batches yes [11:27:47] it will not disrupt anything [11:28:26] ok ill leave it to you thanks [11:32:37] (03PS4) 10Elukey: download-project-namespace-map: handle binary enconding for python3 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/540581 (https://phabricator.wikimedia.org/T204735) [11:35:16] (03PS3) 10Mforns: Migrate reports from MySQL EventLogging to Hive [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/540159 (https://phabricator.wikimedia.org/T223414) [11:37:29] (03PS4) 10Mforns: Migrate reports from MySQL EventLogging to Hive [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/540159 (https://phabricator.wikimedia.org/T223414) [11:39:13] (03CR) 10Fdans: [C: 03+1] download-project-namespace-map: handle binary enconding for python3 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/540581 (https://phabricator.wikimedia.org/T204735) (owner: 10Elukey) [11:43:02] (03PS5) 10Mforns: Migrate reports from MySQL EventLogging to Hive [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/540159 (https://phabricator.wikimedia.org/T223414) [11:44:27] fdans: can I restart the current (broken) namespace timer to see if it logs properly? [11:44:40] I think so but I don't want to interfere with anything you are doing now [11:44:55] elukey: go ahead! but we might want to delete the empty directory on hdfs [11:45:14] yes right after that for sure [11:45:30] yes not it is logging properly [11:45:47] elukey: [11:45:50] hdfs dfs -rm -r /wmf/data/raw/mediawiki/project_namespace_map/snapshot=2019-0 [11:45:54] sorry [11:45:58] hdfs dfs -rm -r /wmf/data/raw/mediawiki/project_namespace_map/snapshot=2019-09 [11:46:30] Oct 3 11:46:03 an-coord1001 refinery-download-project-namespace-map[195330]: File "/srv/deployment/analytics/refinery/bin/download-project-namespace-map", line 172, in [11:46:34] Oct 3 11:46:03 an-coord1001 refinery-download-project-namespace-map[195330]: spamwriter.writerow([unicode(s).encode("utf-8") for s in row]) [11:46:37] Oct 3 11:46:03 an-coord1001 refinery-download-project-namespace-map[195330]: NameError: name 'unicode' is not defined [11:46:40] so yes the puppet change fixed the logging problem as well [11:46:42] :( [11:47:02] mforns: o/ - can you check https://gerrit.wikimedia.org/r/540581 when you have a minute? [11:47:14] python3 migration hit mw history :( [11:49:03] (03CR) 10Mforns: [C: 03+2] "LGTM!" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/540581 (https://phabricator.wikimedia.org/T204735) (owner: 10Elukey) [11:50:03] elukey, python3 migration hit WMH where? in this change ^ ? [11:50:42] mforns: yeah.. basically Fran discovered that the issue with mw check denormalize is due to the mappings, that were not generated [11:50:46] due to the unicode erorr [11:50:48] *error [11:50:55] so we'll likely have to restart the whole thing again [11:51:00] after fixing the namespace map :( [11:52:12] going to eat something and I'll be back shortly [11:52:36] ok makes sense [12:01:02] (03PS6) 10Mforns: Migrate reports from MySQL EventLogging to Hive [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/540159 (https://phabricator.wikimedia.org/T223414) [12:01:45] (03CR) 10Fdans: [V: 03+2 C: 03+1] download-project-namespace-map: handle binary enconding for python3 [analytics/refinery] - 10https://gerrit.wikimedia.org/r/540581 (https://phabricator.wikimedia.org/T204735) (owner: 10Elukey) [12:02:57] mforns: sending a postmortem now :) [12:03:36] thank you fdans :] [12:46:55] 10Analytics, 10Research-Backlog, 10Wikidata: Copy Wikidata dumps to HDFs - https://phabricator.wikimedia.org/T209655 (10GoranSMilovanovic) @JAllemandou Awesome, thank you! [13:04:49] o/ joal [13:04:54] will join up in a minute. [13:05:01] I forgot we were meeting this morning! [13:10:34] AH [13:10:35] ME TOO! [13:10:41] I never see meetings that start at 9 [13:10:42] whaaa [13:17:37] fdans: if you haven't re-created the namespace map, I'd do it from an-coord1001 with the "patched" version of the service [13:17:47] so we test that the new code works properly etc.. [13:17:50] how does it sound? [13:18:50] elukey: if you can do that it would be great [13:19:04] I have to go afk for like an hour for a doctor appointment [13:19:12] fdans: we can do it now if you have a minute [13:19:20] ah sorry you have to go now [13:19:24] okok I'll do it :) [13:19:50] elukey: thank you, it can also wait [13:23:24] elukey, I deleted all our ops meetings and will create new ones before and after my ops week [13:25:54] ack! [13:26:14] !log re-run refinery-download-project-namespace-map (modified with recent fixes for encoding and python3) [13:26:16] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:29:36] the new script seems to work fine [13:29:42] the new mappings are in HDFS for you guys to check [13:53:56] ottomata: I had q auick read over your code for backfilling ores-events [13:54:26] ottomata: There is a strong perf improvement in not using partitionBy, but using the fact that data is prepartitionned [13:54:50] when using partitionBy, all data is shuffled then repartitioned - It's a shame when it's already partitionned :) [13:55:05] Anyway - was just a comment passing by [13:55:09] Gone for kids [13:59:27] (03PS7) 10Mforns: Migrate reports from MySQL EventLogging to Hive [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/540159 (https://phabricator.wikimedia.org/T223414) [14:17:13] !log stop the Hadoop test cluster to migrate it to the new kerberos cluster [14:17:15] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:20:24] joal: interesting! i didn't need partition by when writing to a hive table [14:20:35] but i couldn't seem to insert into a hihve table [14:20:39] only saveAsTable [14:20:50] because of some issues with nullable fields and parquet? not sure [14:20:56] but i was able to just write.parquet [14:21:25] and I assumed that I needed the partitionBy with .write.parquet, since i doubted the partition info was carried over from the input dataset [14:21:28] was it? [14:40:51] milimetric: brain bounce with me about MEP stuff todayyYyyYY [14:52:18] ottomata: definitely, give me a sec to bike to this cafe [14:52:28] k! [14:52:56] 10Analytics, 10Analytics-EventLogging, 10EventBus, 10CPT Initiatives (Modern Event Platform (TEC2)), and 2 others: Modern Event Platform: Stream Configuration: Implementation - https://phabricator.wikimedia.org/T233634 (10Ottomata) Current idea: Create a new 'ConfigExporter' MW extension that can be confi... [15:10:58] ok ottomata I'll go to cave, only got 20 min till hangout with lex [15:11:16] i got 1:1 then too [15:15:09] (03PS8) 10Mforns: Migrate reports from MySQL EventLogging to Hive [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/540159 (https://phabricator.wikimedia.org/T223414) [15:15:36] (03CR) 10Mforns: [V: 03+2] Migrate reports from MySQL EventLogging to Hive [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/540159 (https://phabricator.wikimedia.org/T223414) (owner: 10Mforns) [15:20:57] (03CR) 10Mforns: [V: 03+2] "Finally, this works." [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/540159 (https://phabricator.wikimedia.org/T223414) (owner: 10Mforns) [15:24:00] lost ya milimetric l! [15:25:53] yeah, wifi is weaker than I thought :( [15:41:19] 10Analytics, 10Analytics-Kanban, 10User-Elukey: Make the Kerberos infrastructure production ready - https://phabricator.wikimedia.org/T226089 (10elukey) ` elukey@re0.cr1-eqiad# show | compare [edit firewall family inet filter analytics-in4 term kerberos from destination-address] 10.64.0.182/32 { ...... [15:47:37] ottomata: o/ [15:47:54] there is one report that kafka-python 1.4.7 seems to have fixed the rebalancing issue [15:48:00] we could package/deploy/test it [15:54:08] 10Analytics, 10Analytics-EventLogging, 10EventBus, 10CPT Initiatives (Modern Event Platform (TEC2)), and 2 others: Modern Event Platform: Stream Configuration: Implementation - https://phabricator.wikimedia.org/T233634 (10jlinehan) Making this generic will be cool. What if we wanted to serve a configuratio... [15:57:12] 10Analytics: Request for a large request data set for caching research and tuning - https://phabricator.wikimedia.org/T225538 (10Danielsberger) Hi @Nuria and @fdans , I just wanted to politely check in about this quarter's schedule. I'm hoping some people are back and someone has a few cycles to create and exp... [15:57:36] a-team: staff or standup? [15:57:43] was about to sak that [15:57:56] elukey: i vote for staff given yesterday's data issues [15:58:07] elukey: SORRY standup [15:58:13] 10Analytics, 10Analytics-EventLogging, 10EventBus, 10CPT Initiatives (Modern Event Platform (TEC2)), and 2 others: Modern Event Platform: Stream Configuration: Implementation - https://phabricator.wikimedia.org/T233634 (10Ottomata) > Should the client always have to know what streams it is interested in? N... [15:58:39] ok for me :) [16:00:58] ping ottomata standup? [16:01:40] elukey: standup? [16:04:02] 10Analytics, 10Analytics-Kanban: superset not showing data after 09/16 for some datasources - https://phabricator.wikimedia.org/T234471 (10Nuria) a:03Nuria [16:04:40] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Scoring-platform-team, 10Patch-For-Review: Change event.mediawiki_revision_score schema to use map types - https://phabricator.wikimedia.org/T225211 (10Nuria) a:03Ottomata [16:07:11] oh elukey aye, ya we could... :) [16:13:56] so i'm going to be a continuous source of errors it seems....my druid loading jobs has been running great for months, but todays version now wants to emit 'ImportError: No module named requests' from druid_loader.py used by refinery/oozie/util/druid/load/workflow.xml. Seems odd that requests wouldn't be available (i've run the job three times now, so re-running doesn't seem to help) [16:14:35] ebernhardson: we're in standup, but it feels related to elukey change for python3 [16:20:44] ebernhardson: yes it is me this time! druid_loader.py has a new shebang now, so you'll have to restart the oozie coordinator to use an up to date version of refinery on HDFS [16:21:20] (there are only python3-* deps on hadoop workers nodes now) [16:21:23] sorry :( [16:22:16] shabang/shebang sorry, I never got it right [16:23:22] (03CR) 10Milimetric: "let me know if you want to pair to make sure the stitching is ok" [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/540159 (https://phabricator.wikimedia.org/T223414) (owner: 10Mforns) [16:23:42] milimetric, sure :] [16:30:25] elukey: ok, will do [16:51:55] redeploy seems to have worked, thanks! [16:52:13] \o/ ebernhardson [16:53:02] 10Analytics, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019): Develop a tool or integrate feature in existing one to visualize WMCS edits data - https://phabricator.wikimedia.org/T226663 (10bd808) >>! In T226663#5543449, @JAllemandou wrote: > One question on the interaction with @milimetric about neede... [16:53:08] nice thanks! [16:53:43] python2 is really difficult to remove [17:21:45] 10Analytics: Upgrade eventlogging to Python 3 - https://phabricator.wikimedia.org/T233231 (10elukey) @Ottomata do you remember why the `reload(sys)` stuff was done in the first place? I tried to dig a bit into git blame/log but didn't find anything useful.. If you don't remember I'll dig into the code :) [17:26:56] (03CR) 10Milimetric: [WIP] Publish monthly geoeditor numbers (032 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/530878 (https://phabricator.wikimedia.org/T131280) (owner: 10Milimetric) [17:28:03] * elukey off! [17:36:46] 10Analytics: Add anomaly detection alarm to detect traffic blockage per country - https://phabricator.wikimedia.org/T234484 (10mforns) I think the query that defines this quality metric should just forward the absolute value of pageviews per country. I think normal+stddev is not needed there, because the anomal... [17:46:05] 10Analytics, 10Cloud-Services, 10Developer-Advocacy (Oct-Dec 2019): Develop a tool or integrate feature in existing one to visualize WMCS edits data - https://phabricator.wikimedia.org/T226663 (10JAllemandou) Understood @bd808 - Thanks for the answer :) [18:35:05] 10Analytics: Upgrade eventlogging to Python 3 - https://phabricator.wikimedia.org/T233231 (10Ottomata) No idea why: https://github.com/wikimedia/eventlogging/commit/375ae076ef70d87d7c3b7b9676d0433cd5c139c4 It looks like it has something to do with python 2 utf-8, which I'd assume isn't necessary in python 3. I... [18:39:31] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10EventBus, and 2 others: Figure out how to $ref common schema across schema repositories - https://phabricator.wikimedia.org/T233432 (10Ottomata) Talked with @Milimetric today, we concluded that a new 'common' (to-be-named) repository that can be... [18:40:21] PROBLEM - Check the last execution of reportupdater-flow on stat1006 is CRITICAL: NRPE: Command check_check_reportupdater-flow_status not defined https://wikitech.wikimedia.org/wiki/Analytics/Systems/Managing_systemd_timers [18:41:14] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Better Use Of Data, and 6 others: Modern Event Platform: Schema Guidelines and Conventions - https://phabricator.wikimedia.org/T214093 (10Ottomata) Talked with @Milimetric today and we think this approach makes sense. We don't need to move forwa... [18:42:27] 10Analytics, 10Analytics-Kanban: superset not showing data after 09/16 for some datasources - https://phabricator.wikimedia.org/T234471 (10Nuria) @elukey I was soooo happyy when i saw this comment this morning cause i had totally run out of ideas. [18:42:32] 10Analytics, 10Analytics-Kanban: superset not showing data after 09/16 for some datasources - https://phabricator.wikimedia.org/T234471 (10Nuria) 05Open→03Resolved [18:46:09] 10Analytics: Add anomaly detection alarm to detect traffic blockage per country - https://phabricator.wikimedia.org/T234484 (10Nuria) @mforns agreed [18:49:02] (03CR) 10Nuria: "Let's please document on train etherpad the gotachas on how to deploy this:" [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/540159 (https://phabricator.wikimedia.org/T223414) (owner: 10Mforns) [18:49:15] ok [18:50:50] (03CR) 10Nuria: [C: 03+2] Add site-info dump type to importer [analytics/refinery] - 10https://gerrit.wikimedia.org/r/540124 (https://phabricator.wikimedia.org/T234333) (owner: 10Joal) [19:16:02] milimetric: still around? [19:25:06] hey ottomata yep, I was going to talk to connie in a few minutes, you want to bb about the annotations more? [19:25:18] naw on to next things [19:25:23] :) [19:25:34] but right now am poking around with resourceloader and wa slooking for help too [19:26:00] oh ok, will you be around after 4? If you prefer to talk now I can swap [19:29:48] ya will be around til 5 [19:39:35] nuria, do you know who I should ping for pingback reports? so they know the data will be moved to stat1007? [19:57:46] ok ottomata, free whenever you want to chat [19:58:29] k gimmn 6 mins actually i'm onto something... :) [19:58:59] mforns: CindyCicaleseWMF wrote the pingback reports so we can ping her and tell her the data will be moved to stat1007. But the dashboard should keep working fine since the files will be synced to the same public place [19:59:24] milimetric, thanks! yes, I'm mentioning that [19:59:49] milimetric: mforns: thanks for the heads up [20:00:46] CindyCicaleseWMF, no problem :], I'll ping you in the related Phabricator task as well for more info, thanks! [20:01:22] 👍 [20:03:37] ok milimetric bc [20:04:47] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Patch-For-Review: Move reportupdater reports that pull data from eventlogging mysql to pull data from hadoop - https://phabricator.wikimedia.org/T223414 (10mforns) Hi @CCicalese_WMF and @santhosh We Analytics are migrating some reportupdater rep... [23:01:41] mforns: yes, cindy on corey's team [23:01:44] mforns: they have a dasboard