[00:33:01] 10Analytics, 10Analytics-EventLogging, 10Community-Tech, 10Contributors-Analysis, 10EventBus: Add index to mediawiki_page_create_1 table - https://phabricator.wikimedia.org/T170990#3568676 (10DannyH) [01:40:43] 10Analytics, 10Analytics-EventLogging, 10Performance-Team, 10Patch-For-Review: Make webperf eventlogging consumers use eventlogging on Kafka - https://phabricator.wikimedia.org/T110903#3568747 (10Krinkle) >>! In T110903#3568625, @thcipriani wrote: > For clarification, can this task be removed as a blocker... [01:42:10] 10Analytics, 10Analytics-EventLogging, 10Performance-Team, 10Patch-For-Review: Make webperf eventlogging consumers use eventlogging on Kafka - https://phabricator.wikimedia.org/T110903#3568748 (10Krinkle) [02:08:25] 10Analytics, 10Operations: Invalid "wikimedia" family in unique devices data due to misplaced WMF-Last-Access-Global cookie - https://phabricator.wikimedia.org/T174640#3568769 (10Tbayer) [02:12:13] 10Analytics, 10Operations: Invalid "wikimedia" family in unique devices data due to misplaced WMF-Last-Access-Global cookie - https://phabricator.wikimedia.org/T174640#3568792 (10Tbayer) Regarding prioritization: While this is a clear bug, it does not affect the (from the Readers team's perspective) most impor... [03:04:54] 10Analytics-Kanban, 10Reading-analysis: Final Vetting of Family Wide unique devices data - https://phabricator.wikimedia.org/T169550#3568823 (10Tbayer) I had been looking into this from various angles before Wikimania, including reading through the intricate investigations at T143928 (and the bugs that were un... [08:05:09] o/ [08:05:32] I am working on the eventlogging_cleaner.py script, if you need me just ping :) [09:44:44] * elukey early lunch + errand (available on hangouts/phone) [11:14:28] back :) [12:41:01] (03PS1) 10Joal: Update mediawiki history oozie denormalize job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/374987 (https://phabricator.wikimedia.org/T174484) [12:42:21] * elukey waves to joal [12:42:34] * joal waves back [12:42:53] How are you elukey ? [12:44:28] goood! The new version of eventlogging_cleaner seems good, and hopefully partman is sorted [12:44:51] Wow, great ! [12:51:32] On my end, finishing testing the update to mediawiki-history code (T174484) [12:51:35] T174484: Add redirect and pagelinks tables for partition repair in sqoop job for mediawiki history - https://phabricator.wikimedia.org/T174484 [12:51:52] elukey: Will take my break soon I think, will be back for standup (except if you need me ) [12:53:11] nono super fine! [12:53:34] ok cool [12:53:40] Later then ;) [12:56:19] hellooooo [12:57:33] (03CR) 10Joal: "Tested, seems ok." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/374987 (https://phabricator.wikimedia.org/T174484) (owner: 10Joal) [13:02:43] mforns: o/ [13:02:51] hello elukey :] [13:03:03] I'm reading the changes and comments to EL script [13:03:28] mforns: I don't remeber one thing.. when we do the check of the batch size with SELECT timestamp where etc..., why don't we just do max(timestamp)? [13:03:52] elukey, because then the limit does not work [13:04:10] you will get the max timestamp for the start_ts self.end interval [13:04:12] ah right [13:04:32] Riccardo added some comments, I'll let you review them :) [13:04:33] is that what volans meant? [13:04:36] ok [13:04:51] nono I think he was trying to suggest a way not to select all the elements [13:04:54] and then just pick the last [13:09:44] aha [13:19:21] (03CR) 10Ottomata: [C: 031] "Only did a passing review" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/374987 (https://phabricator.wikimedia.org/T174484) (owner: 10Joal) [13:41:55] mforns: any idea about eventlogging? [13:42:30] if you want we can work together in the cave [13:44:12] elukey, sure! omw I'm reviewing and commenting [13:44:48] i'm in [13:50:06] 10Analytics-Kanban, 10Contributors-Analysis, 10Patch-For-Review: Provide cumulative edit count in Data Lake edit data - https://phabricator.wikimedia.org/T161147#3569880 (10Nuria) @Neil_P._Quinn_WMF given that user's latest edit count is cumulative the latest count for user will always be the maximum, right?... [13:58:48] 10Analytics, 10Analytics-Kanban: Create phabricator space for tickets with legal restrictions - https://phabricator.wikimedia.org/T174675#3569915 (10Nuria) [13:59:45] 10Analytics: Create phabricator space for tickets with legal restrictions - https://phabricator.wikimedia.org/T174675#3569929 (10Nuria) a:03Aklapper [14:29:38] elukey, I pushed the changes [14:30:44] mforns: super [14:37:42] 10Analytics, 10Analytics-Cluster, 10Operations, 10ops-eqiad, and 2 others: kafka-jumbo.cfg partman recipe creation/troubleshooting - https://phabricator.wikimedia.org/T174457#3570102 (10elukey) The last code review removes the need for the 'placeholder' logical volume and removes a unused/not-necessary par... [15:03:56] 10Analytics-Kanban, 10Patch-For-Review: Add redirect and pagelinks tables for partition repair in sqoop job for mediawiki history - https://phabricator.wikimedia.org/T174484#3563442 (10JAllemandou) [15:04:17] 10Analytics-Kanban, 10Patch-For-Review: Add redirect and pagelinks tables for partition repair in sqoop job for mediawiki history - https://phabricator.wikimedia.org/T174484#3563442 (10JAllemandou) a:03JAllemandou [15:30:33] a-team: some trouble with zookeeper in codfw sorry, checking it [15:30:42] np elukey [15:32:25] 10Analytics: Make Spark 2.1 easily available on new CDH5.10 cluster - https://phabricator.wikimedia.org/T158334#3570247 (10JAllemandou) Discussed in standup 2017-08-31: Let's use scap to deploy spark-2.1.1 release folder (with small changes in config for logging and hadoop-conf setting) on stat100[345] and anal... [16:04:08] 10Analytics: Request for python package csvsort on stat1005.eqiad.wmnet - https://phabricator.wikimedia.org/T174577#3566528 (10Ottomata) I'd have to create a python debian package for this. Q: can you used Hadoop? You wouldn't have memory problems then! :) [16:04:48] 10Analytics: Request for python package csvsort on stat1005.eqiad.wmnet - https://phabricator.wikimedia.org/T174577#3566528 (10Nuria) +1 @Ottomata hadoop seems best choice [16:06:15] 10Analytics-Kanban, 10Wikimedia-Stream, 10Wikimedia-Incident: Alerts for common/important EventStreams topic volume - https://phabricator.wikimedia.org/T174493#3570395 (10Nuria) p:05Triage>03High [16:06:41] 10Analytics-Kanban, 10Wikimedia-Stream, 10Wikimedia-Incident: Alerts for common/important EventStreams topic volume - https://phabricator.wikimedia.org/T174493#3563830 (10Nuria) It is probably easiest to have alerts for volume starting with RCStream [16:11:44] 10Analytics-Cluster, 10Analytics-Kanban, 10Operations, 10ops-eqiad, and 2 others: kafka-jumbo.cfg partman recipe creation/troubleshooting - https://phabricator.wikimedia.org/T174457#3570463 (10Nuria) [16:12:09] 10Analytics-Cluster, 10Analytics-Kanban, 10Operations, 10ops-eqiad, and 2 others: kafka-jumbo.cfg partman recipe creation/troubleshooting - https://phabricator.wikimedia.org/T174457#3562647 (10Nuria) [16:13:03] 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review: Make tranquility work with Spark - https://phabricator.wikimedia.org/T168550#3570468 (10JAllemandou) [16:13:12] 10Analytics, 10Wikimedia-Stream: Stop tracking EventStreams client lag in graphite - https://phabricator.wikimedia.org/T174435#3561871 (10Nuria) [16:13:36] 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: Add edits endpoint to AQS using druid as a backend - https://phabricator.wikimedia.org/T174174#3570473 (10JAllemandou) [16:13:46] 10Analytics-Kanban, 10Patch-For-Review: Add redirect and pagelinks tables for partition repair in sqoop job for mediawiki history - https://phabricator.wikimedia.org/T174484#3570475 (10JAllemandou) [16:15:11] 10Analytics-Kanban, 10Patch-For-Review: Monthly Mediawiki Sqoop job failed - https://phabricator.wikimedia.org/T172426#3570478 (10Nuria) [16:15:23] 10Analytics-Kanban, 10Patch-For-Review: Troubleshoot Wikimetrics "magic button" - https://phabricator.wikimedia.org/T173585#3570479 (10mforns) [16:16:10] 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review: Make tranquility work with Spark - https://phabricator.wikimedia.org/T168550#3570495 (10JAllemandou) [16:16:12] 10Analytics-Kanban: Make banner realtime jobs more resilient - https://phabricator.wikimedia.org/T169101#3570497 (10JAllemandou) [16:18:14] 10Analytics, 10cloud-services-team: Remove logging from labs for schema https://meta.wikimedia.org/wiki/Schema:CommandInvocation - https://phabricator.wikimedia.org/T166712#3570499 (10Nuria) @Andrew : you guys own the script that logs this data (i think it was created by Yuvi a while back) . It doesn't seem li... [16:20:09] 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review: Hadoop: Add a lower priority queue: nice queue - https://phabricator.wikimedia.org/T156841#3570530 (10Nuria) 05Open>03Resolved [16:30:31] 10Analytics-Kanban, 10Patch-For-Review: Add tagging to webrequest refine process - https://phabricator.wikimedia.org/T171760#3570551 (10Nuria) [16:31:03] 10Analytics-Kanban: Webrequest tagging and distribution. Measuring non-pageview requests - https://phabricator.wikimedia.org/T164019#3570556 (10Nuria) [16:31:05] 10Analytics-Kanban, 10Patch-For-Review: Add tagging to webrequest refine process - https://phabricator.wikimedia.org/T171760#3475367 (10Nuria) 05Open>03Resolved [16:58:08] ottomata: hey, fyi for T174577, they can use notebook and pip install whatever [16:58:09] T174577: Request for python package csvsort on stat1005.eqiad.wmnet - https://phabricator.wikimedia.org/T174577 [16:58:19] in their own venv [16:58:53] Don't want to step on your hadoop recommendation, but might be worth pointing out :) [17:05:36] ottomata: zk fixed in both main clusters [17:06:03] a-team: sorry if I skipped the meetings but zk in codfw was down, I'll write an incident report tomorrow [17:30:23] great! thanks luca [17:30:36] just sent an email to ops@, will write an incident report tomorrow [17:30:38] sigh [17:30:55] ottomata, qq: the current reportupdater instance that is sync'ing report files to analytics datasets is in stat1003 or stat1006? [17:31:38] mforns: I'd need to go now sorry, if you do anything today with eventlogging_cleaner lemme know and I'll keep going tomorrow [17:31:41] have a nice vacation :) [17:31:50] thanks elukey [17:31:51] ! [17:31:55] see you soon [17:32:09] I don't think I will touch EL script, looks good to me! [17:32:19] \o/ [17:32:25] * elukey off! [17:32:25] stat1006 [17:32:25] mforns: [17:32:26] should be [17:33:06] ottomata, if 1006, I think there might be a problem, because reportupdater can not find the creds files that are pointing to /a/.my.cnf.research [17:33:23] ah! because /a doesn't exist! [17:33:35] i dont' remember, why is it looking in /a anyway [17:33:40] ok, should I change the references to it in the code? [17:34:05] it's config code in several reportupdater-queries projects [17:34:20] yes [17:34:22] shoudl be /etc/mysql/conf.d/stats-research-client.cnf [17:34:27] that is the puppetized location [17:34:44] i can't remember why we were using the /a.my.cnf.research file, its just a symlink to that one [17:34:58] ottomata, https://github.com/wikimedia/analytics-limn-ee-data/blob/master/ee/config.yaml#L5 [17:35:05] that's an example [17:35:18] ya, but why are we using that file? maybe it was for backwards compatibility [17:35:26] when we started making the cnf files in /etc/mysql? [17:35:33] those reports were using it since the start [17:35:40] aye [17:35:50] can we change them to /etc/mysql/conf.d/stats-research-client.cnf ? [17:36:14] we shoudl also probably add a comment in puppet where it creates the .cnf files that they are referenced by external reportupdate config [17:36:15] sure, I can create patches for all active jobs [17:36:28] k i'll do the puppet comment part [17:36:31] k [17:50:27] 10Analytics-Kanban: Fix path to mysql credentials file in reportupdater query repositories - https://phabricator.wikimedia.org/T174706#3570837 (10mforns) [17:51:03] (03PS1) 10Mforns: Correct path to credentials file [analytics/limn-flow-data] - 10https://gerrit.wikimedia.org/r/375035 (https://phabricator.wikimedia.org/T174706) [17:54:40] (03PS1) 10Mforns: Correct path to credentials file [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/375037 [17:55:42] 10Analytics, 10cloud-services-team: Remove logging from labs for schema https://meta.wikimedia.org/wiki/Schema:CommandInvocation - https://phabricator.wikimedia.org/T166712#3570865 (10Krenair) modules/toollabs/files/log-command-invocation in puppet [17:55:52] (03PS2) 10Mforns: Correct path to credentials file [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/375037 (https://phabricator.wikimedia.org/T174706) [17:58:14] (03PS1) 10Mforns: Correct path to credentials file [analytics/limn-language-data] - 10https://gerrit.wikimedia.org/r/375040 (https://phabricator.wikimedia.org/T174706) [17:59:08] 10Analytics-Kanban, 10Patch-For-Review: Fix path to mysql credentials file in reportupdater query repositories - https://phabricator.wikimedia.org/T174706#3570874 (10Ottomata) Mforns, +1 from me on all of these changes. [18:01:43] (03PS1) 10Mforns: Correct path to credentials file [analytics/limn-ee-data] - 10https://gerrit.wikimedia.org/r/375041 (https://phabricator.wikimedia.org/T174706) [18:04:55] (03PS1) 10Mforns: Correct path to credentials file [analytics/limn-multimedia-data] - 10https://gerrit.wikimedia.org/r/375042 (https://phabricator.wikimedia.org/T174706) [18:10:07] (03PS1) 10Mforns: Correct path to credentials file [analytics/discovery-stats] - 10https://gerrit.wikimedia.org/r/375043 (https://phabricator.wikimedia.org/T174706) [18:13:04] (03PS1) 10Mforns: Correct path to credentials file [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/375044 (https://phabricator.wikimedia.org/T174706) [18:14:50] 10Analytics-Kanban, 10Patch-For-Review: Fix path to mysql credentials file in reportupdater query repositories - https://phabricator.wikimedia.org/T174706#3570979 (10mforns) @Ottomata OK will merge them, to unbreak the reports. [18:15:33] (03CR) 10Mforns: [V: 032 C: 032] "Self-merging to unbreak reports." [analytics/limn-flow-data] - 10https://gerrit.wikimedia.org/r/375035 (https://phabricator.wikimedia.org/T174706) (owner: 10Mforns) [18:15:42] (03CR) 10Mforns: [V: 032 C: 032] "Self-merging to unbreak reports." [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/375037 (https://phabricator.wikimedia.org/T174706) (owner: 10Mforns) [18:15:56] (03CR) 10Mforns: [V: 032 C: 032] "Self-merging to unbreak reports." [analytics/limn-language-data] - 10https://gerrit.wikimedia.org/r/375040 (https://phabricator.wikimedia.org/T174706) (owner: 10Mforns) [18:16:06] (03CR) 10Mforns: [V: 032 C: 032] "Self-merging to unbreak reports." [analytics/limn-ee-data] - 10https://gerrit.wikimedia.org/r/375041 (https://phabricator.wikimedia.org/T174706) (owner: 10Mforns) [18:16:13] (03CR) 10Mforns: [V: 032 C: 032] "Self-merging to unbreak reports." [analytics/limn-multimedia-data] - 10https://gerrit.wikimedia.org/r/375042 (https://phabricator.wikimedia.org/T174706) (owner: 10Mforns) [18:17:17] (03CR) 10Mforns: [C: 031] "This fix needs to be merged to unbreak production reports." [analytics/discovery-stats] - 10https://gerrit.wikimedia.org/r/375043 (https://phabricator.wikimedia.org/T174706) (owner: 10Mforns) [18:17:42] (03CR) 10Mforns: [V: 032 C: 032] "Self-merging to unbreak reports." [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/375044 (https://phabricator.wikimedia.org/T174706) (owner: 10Mforns) [18:18:11] ottomata, do you have +2 in this one? https://gerrit.wikimedia.org/r/#/c/375043/ I can not merge [18:18:42] (03CR) 10Ottomata: [C: 032] Correct path to credentials file [analytics/discovery-stats] - 10https://gerrit.wikimedia.org/r/375043 (https://phabricator.wikimedia.org/T174706) (owner: 10Mforns) [18:18:45] (03CR) 10Ottomata: [V: 032 C: 032] Correct path to credentials file [analytics/discovery-stats] - 10https://gerrit.wikimedia.org/r/375043 (https://phabricator.wikimedia.org/T174706) (owner: 10Mforns) [18:18:47] done [18:18:51] thanks! [19:12:42] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 3 others: Visualize page create events for all wikis - https://phabricator.wikimedia.org/T170850#3571190 (10mforns) Hi @Nettrom I found some problem with the path to the configuration file. It was outdated since migration fro... [19:25:21] 10Analytics-Kanban, 10Patch-For-Review: Create purging script for mediawiki-history data - https://phabricator.wikimedia.org/T162034#3571223 (10mforns) @Nuria I applied the changes that joseph suggested and also the ones to make the logging compatible with our cronjob scheme. Also tested that in Hadoop, and e... [19:26:34] 10Analytics-Kanban, 10Analytics-Wikistats: Cleanup Routing code - https://phabricator.wikimedia.org/T170459#3571228 (10mforns) Hi @fdans ! I was finally able to rebase the code and push. I tested this thoroughly and I think it can be deployed. Please, review and merge if OK. Cheers! [19:27:47] 10Analytics-Kanban, 10Patch-For-Review: Fix path to mysql credentials file in reportupdater query repositories - https://phabricator.wikimedia.org/T174706#3570837 (10mforns) I checked in stat1006, and now all reports seem to be getting updated. Will move to done. [19:27:56] 10Analytics-Kanban, 10Patch-For-Review: Fix path to mysql credentials file in reportupdater query repositories - https://phabricator.wikimedia.org/T174706#3571233 (10mforns) [19:29:32] 10Analytics-Kanban, 10Discovery, 10Discovery-Analysis, 10Patch-For-Review: Add purge info for Kartographer schema - https://phabricator.wikimedia.org/T171622#3571234 (10mforns) Only merging is missing. [19:29:48] 10Analytics-Kanban, 10Research, 10Patch-For-Review: Add QuickSurvey schemas to EventLogging white-list - https://phabricator.wikimedia.org/T172112#3571235 (10mforns) Only merging is missing. [20:38:37] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 3 others: Visualize page create events for all wikis - https://phabricator.wikimedia.org/T170850#3571424 (10Nettrom) Hi @mforns Ah, I remember being confused by the configuration file path in the examples I looked at, but for... [22:29:54] bye a-team! see you in one week :] [22:51:58] anyone around who knows anything about deployment-kafka01? [22:57:42] 10Analytics, 10Beta-Cluster-Infrastructure: deployment-kafka01 - disk is full - https://phabricator.wikimedia.org/T174742#3571849 (10Krenair) [22:59:04] 10Analytics, 10Beta-Cluster-Infrastructure: deployment-kafka01 - disk is full - https://phabricator.wikimedia.org/T174742#3571866 (10Krenair)