[00:16:57] (03PS3) 10Ottomata: DataFrameToHive - Use DataFrame .write.parquet instead of .insertInto [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425578 (https://phabricator.wikimedia.org/T159962) [00:20:04] (03PS2) 10Ottomata: RefineTarget - Use Hadoop FS to infer input format rather than Spark [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425597 (https://phabricator.wikimedia.org/T159962) [00:21:59] (03CR) 10Ottomata: RefineTarget - Use Hadoop FS to infer input format rather than Spark (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425597 (https://phabricator.wikimedia.org/T159962) (owner: 10Ottomata) [00:22:14] (03PS3) 10Ottomata: RefineTarget - Use Hadoop FS to infer input format rather than Spark [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425597 (https://phabricator.wikimedia.org/T159962) [00:23:56] (03CR) 10Ottomata: [C: 032] DataFrameToHive - Use DataFrame .write.parquet instead of .insertInto [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425578 (https://phabricator.wikimedia.org/T159962) (owner: 10Ottomata) [00:24:56] (03PS4) 10Ottomata: RefineTarget - Use Hadoop FS to infer input format rather than Spark [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425597 (https://phabricator.wikimedia.org/T159962) [00:25:46] (03CR) 10Ottomata: [V: 032 C: 032] RefineTarget - Use Hadoop FS to infer input format rather than Spark [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425597 (https://phabricator.wikimedia.org/T159962) (owner: 10Ottomata) [01:06:21] 10Analytics-Kanban, 10Patch-For-Review: Refresh SWAP notebook hardware - https://phabricator.wikimedia.org/T183145#4125683 (10Neil_P._Quinn_WMF) Hmm, I'm having the same problem as @Tbayer, but that workaround isn't working for me. ``` > !pip show impyla Name: impyla Version: 0.13.8 > !pip show thrift Name: th... [04:27:27] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Add Ecosia and Startpage to list of search engines - https://phabricator.wikimedia.org/T191714#4125852 (10Nuria) a:03Nuria [05:02:36] (03PS4) 10Nuria: Add new search engines [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/423336 (https://phabricator.wikimedia.org/T191714) (owner: 10Jonas Kress (WMDE)) [05:04:05] (03PS5) 10Nuria: Add new search engines [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/423336 (https://phabricator.wikimedia.org/T191714) (owner: 10Jonas Kress (WMDE)) [05:05:00] (03PS6) 10Nuria: [WIP] Add new search engines [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/423336 (https://phabricator.wikimedia.org/T191714) (owner: 10Jonas Kress (WMDE)) [05:07:44] (03CR) 10VolkerE: [C: 04-1] Darken subdued text and alter footer area for WCAG AA compliance (032 comments) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/419958 (https://phabricator.wikimedia.org/T185533) (owner: 10Sturmkrahe) [05:07:56] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Add new search engines [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/423336 (https://phabricator.wikimedia.org/T191714) (owner: 10Jonas Kress (WMDE)) [05:22:47] (03PS1) 10Sahil505: Changed 'NaN' to 0 in metrics % change over the selected time range [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425747 (https://phabricator.wikimedia.org/T192028) [05:30:51] (03CR) 10Sahil505: "> Uploaded patch set 1." (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425747 (https://phabricator.wikimedia.org/T192028) (owner: 10Sahil505) [06:17:14] hello everybody [06:17:36] so the stat100[45] boxes were overloaded from yesterday and /mnt/hdfs seemed not accessible [06:17:43] I had to kill fuse_dfs [06:18:02] Timing seems to match (more or less) with me restarting the Namenodes [06:18:54] the check that I added on stat1005 to do 'timeout 5 ls -l /mnt/hdfs' was hanging when executed [06:18:57] sigh [06:19:11] (so the alert was reported as UNKNOWN in icinga, not CRITICAL, so not firing) [07:42:57] (03PS2) 10Joal: Correct Mediawiki-history denormalize job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/425569 [07:43:20] joal: o/ [07:43:38] Hi elukey [07:43:51] Good morning :) [07:43:53] if you have time this morning we could triple check the trash thing and then send an email to analytics@ research@ (and maybe engineering?) [07:43:57] morninggg [07:45:56] elukey: sure ! [07:47:35] elukey: Just deleted and retrieved some files in hadoop - Looks the trash works :) [07:47:42] \o/ [07:49:13] this is what I find now in my trash (after yesterday's deletion) [07:49:14] -rw-r--r-- 3 elukey elukey 15 2018-04-11 16:51 /user/elukey/.Trash/180412000000/user/elukey/test [07:49:30] elukey: Nice :) [07:49:37] so that is the checkpoint for 12/04/2018 (done probably at midnight UTC) [07:50:15] or maybe better, up to 12/04/2018 at midnight [07:52:00] so now the last step is to add some documentation on wikitech [07:52:23] so when people will freak out there will be a step by step guide about getting back their data :D [07:52:38] :D [07:52:53] good morning everyone [07:52:58] hola! [07:53:06] bon giorno. [07:53:11] Bonjour [07:53:33] (clearly french is the most elegant one) [07:53:38] true [07:53:43] hm - not sure :) [07:54:38] joal: do you have a minute? I want to run the full process to produce a parquet dump using your scripts. I'm trying to understand gerrit :) [07:56:27] dsaez: not sure how I can help :) [07:58:00] joal: just be around, specific questions will come up ;) [08:03:34] joal: what do you think about adding a note in https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster#FAQ about the trash? [08:03:42] elukey: Sounds good :) [08:03:47] ack! [08:04:32] elukey: Let's make sure it shows up in the content-table :) [08:06:28] joal: of the page you mean? [08:06:46] or somewhere else? [08:06:46] yes [08:06:53] no no on he page elukey :) [08:06:53] ah yes yes :) [08:24:07] joal: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster#recover_files_deleted_by_mistake_using_the_hdfs_CLI_rm_command? [08:25:05] Awesome elukey :) [08:25:17] elukey: I had forgotten we decided to keep data for a month :) [08:25:41] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Add trash folder to hadoop - https://phabricator.wikimedia.org/T189051#4126188 (10elukey) Added documentation to https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster#recover_files_deleted_by_mistake_using_the_hdfs_CLI_rm_command?,... [08:27:03] joal: we can tune that as we wish, IIRC we decided that one month was a good compromise since probably people will not delete huge things [08:27:20] yup - I even thought it was 1 week - But 1 month is better ) [08:27:49] yeah I'd feel better with a month :D [08:28:10] all right sending the email! [08:28:14] thanks for the review :) [08:28:47] elukey: OMG i finally managed to test my manifest locally [08:28:50] * fdans dances [08:34:41] 10Analytics, 10TCB-Team, 10Two-Column-Edit-Conflict-Merge, 10WMDE-Analytics-Engineering, and 5 others: How often are new editors involved in edit conflicts - https://phabricator.wikimedia.org/T182008#4126212 (10GoranSMilovanovic) 05Open>03Resolved [08:35:14] fdans: nice! [08:38:29] there you go, email sent to analytics@ and engineering@ [08:38:38] and forwarded to research-internal [08:38:42] done done done [08:40:01] Many thanks elukey for setting the HDP-TRSH :) [09:16:06] 10Analytics, 10Contributors-Analysis: Make an Analytics Data Lake table to provide meta info about wikis - https://phabricator.wikimedia.org/T184576#4126300 (10JAllemandou) Quick note: Knowing the domain of any project, it's relatively easy to extract the project-family and the language (if any). [11:22:52] 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Refresh zookeeper nodes in eqiad - https://phabricator.wikimedia.org/T182924#4126533 (10elukey) Re-tested today the swap of one node in labs (analytics project) to verify again logs and things that might break. Some details about the procedure: starting... [11:24:26] 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Refresh zookeeper nodes in eqiad - https://phabricator.wikimedia.org/T182924#4126534 (10elukey) From the consumers point of view (two kafka clusters and one hadoop cluster) I have observed only some non critical logs in one of the hadoop masters when zoo... [11:24:55] so I've re-tested the zk upgrade in --^ [11:25:12] and there seems to be a moment in which the zookeper cluster might have a brief unavailability [11:25:30] afaics no real damage on kafka or hadoop is done [11:26:05] so overall good, I know what to expect when migrating and hopefully it should not cause any issue. [11:26:23] worst case is a quick rollback to the previous cluster ensemble [12:00:06] * elukey afk for a bit! [12:52:58] yeahhhh no overnight refine emails woohoo! [12:53:50] awesome ottomata :) [12:53:55] I waited for you to deploy [12:54:00] Shall I do that tonight? [12:54:49] yaa! oh i wanted to add one thing...to see if i can put the yarn application id in the email report on failure [12:54:57] i had trouble finding the logs for failed jobs sometimes! [12:55:01] but ya [12:55:13] i also didn't do the head vs count thing, we could try that...but maybe not worth it? [12:57:44] ottomata: small improvement, but samll + small + small :) [12:58:09] ya [12:58:25] well i'm poking around in there, might as well try for a sec? [12:58:40] ottomata: I'm still fighting a bit myself with MWH, so yeah, let's continue :) [12:58:48] We'll deploy tomorrow or monday [13:01:04] (03PS1) 10Joal: Fix MediawikiHistory-denormalize [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425798 [13:03:21] milimetric: o/ [13:05:04] (03PS1) 10Ottomata: Include applicationId in Refine email failure report [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425799 [13:05:17] elukey: o/ [13:05:31] in jobqueue meeting yesterday, they wanted to stop even replicating job queue (and change prop) topics between main clusters for a while [13:05:48] they want to add job queue to private wikis, and we don't want to do cross DC non encrypted private stuff soooo [13:05:54] until we upgrade main and get TLS for mirrormaker [13:05:58] we'll just take that stuff out [13:07:30] joal everything i'm reading online says that count is sloser than head or take [13:08:00] which makes sense, beacuse if it is not empty, it has to count everything [13:08:15] if it is not empty, head should only look at the first element of the first partition, right? [13:10:29] ottomata: ack! [13:15:16] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 3 others: [EPIC] Develop a JobQueue backend based on EventBus - https://phabricator.wikimedia.org/T157088#4126775 (10Pchelolo) [13:15:21] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 3 others: Investigate use-cases for delayed job executions - https://phabricator.wikimedia.org/T172832#4126772 (10Pchelolo) 05Open>03Resolved We've already found all the current use-cases and implemented support for delayed jobs, so resol... [13:18:32] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, 10Services (blocked): Disable MirrorMaker for job queue events - https://phabricator.wikimedia.org/T192005#4126778 (10Ottomata) This goes for change-prop too, right? So blacklist `.+\.(job|change-prop)\..+`? [13:19:28] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, 10Services (blocked): Disable MirrorMaker for job queue events - https://phabricator.wikimedia.org/T192005#4126779 (10Pchelolo) @Ottomata I think we can disable anything that have `/mediawiki\.job/` in there. [13:19:41] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 2 others: Enable CP4JQ support for private wikis - https://phabricator.wikimedia.org/T191464#4126780 (10Ottomata) +1 :) To clarify: we haven't officially scheduled any MirrorMaker TLS work, but after we upgrade main Kafka clusters, it should... [13:21:21] elukey: the change is ready for review! the only thing is that I'm still getting a violation for using class { '::geoip::data::archive': } [13:21:38] but otherwise the cron works as expected [13:22:20] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, 10Services (blocked): Disable MirrorMaker for job queue events - https://phabricator.wikimedia.org/T192005#4126797 (10Ottomata) Ok, but we should keep (other?) change-prop topics mirrored? [13:23:14] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, 10Services (blocked): Disable MirrorMaker for job queue events - https://phabricator.wikimedia.org/T192005#4126800 (10Pchelolo) @Ottomata yes. We're not enabling normal mediawiki events for private wikis because RESTBase or EventStreams are not... [13:26:08] fdans: what kind of violation? [13:26:26] elukey: wmf-style: class 'statistics::compute' includes geoip::data::archive from another module [13:31:16] yeah so statistics::compute is probably not where you want to put the class [13:31:19] better in a profile [13:31:38] classes should be self contained [13:31:39] the statistics:: classes are all v old and crufty and hard to make conform to new profile rules [13:35:03] AH HOLD ON [13:35:30] elukey: sorry, my bad, I left that by accident in statistics:compute [13:35:32] removing [13:39:55] (03PS1) 10Ottomata: DataFrameToHive - Use df.take(1).isEmpty rather than exception [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425808 [13:40:06] fdans: you probably didn't like the part that we discussed about not adding backup for the moment etc.. right ? :D [13:40:37] elukey: I mean it's commented out! [13:40:44] * fdans removes [13:41:14] ah no I was reading rev6, didn't notice that you had another one out [13:41:56] yeah, the thing in hiera, didn't realise I had commited that [13:43:39] (03PS1) 10Ottomata: Rename SparkSQLHiveExtensions to just HiveExtensions [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425809 [13:45:18] (03PS2) 10Ottomata: Rename SparkSQLHiveExtensions to just HiveExtensions [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425809 [13:50:37] (03CR) 10Ottomata: "Lots of leftover commented out code? Did you intend to leave that in?" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425798 (owner: 10Joal) [13:52:28] (03CR) 10Ottomata: [C: 032] DataFrameToHive - Use df.take(1).isEmpty rather than exception [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425808 (owner: 10Ottomata) [13:52:35] (03CR) 10Ottomata: [C: 032] Include applicationId in Refine email failure report [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425799 (owner: 10Ottomata) [13:52:57] (03CR) 10Ottomata: [C: 032] Rename SparkSQLHiveExtensions to just HiveExtensions [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425809 (owner: 10Ottomata) [13:55:22] joal: i've merged and tested my little tweaks [13:55:30] happy to talk more about the .head vs .count thing [13:55:40] but i just changed it to do take(1).isEmpty [13:55:51] but, i'm ready to go for release [13:56:13] heyaaa team [13:57:58] mforns: hola Marcelo, if you want we can check the el sanitization yaml results :) [14:06:13] whoa was there a 9am meeting i missed milimetric elukey? [14:06:34] argh I missed it too elukey so sorry [14:06:44] sneaky 9am meeting [14:06:46] i'd never notice those [14:06:56] i usually look to see if I have meetings today about this time! [14:06:56] haha [14:07:08] ugh I was the one that set it up, I'm such a butt [14:07:15] haha [14:07:18] i could do now? [14:07:30] yeah, me too if elukey is ok [14:07:50] sure, gimme 2 mins! [14:08:00] ok, sweet, I'll be in the cave [14:08:18] 2 mins as well.. [14:16:09] ottomata: coming? [14:20:02] AHK [14:20:03] GRAGH [14:20:05] i forgot [14:20:06] haha [14:20:12] was waiting for somehting and got caught in code review [14:20:33] (03CR) 10Mforns: Changed 'NaN' to 0 in metrics % change over the selected time range (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425747 (https://phabricator.wikimedia.org/T192028) (owner: 10Sahil505) [14:22:31] elukey, sure! [14:23:55] I'll do that [14:24:29] (03CR) 10Nuria: Fix MediawikiHistory-denormalize (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425798 (owner: 10Joal) [14:27:50] (03CR) 10Nuria: Changed 'NaN' to 0 in metrics % change over the selected time range (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425747 (https://phabricator.wikimedia.org/T192028) (owner: 10Sahil505) [14:56:34] 10Analytics, 10Analytics-Wikistats, 10Patch-For-Review: Display of radio buttons in Wikistats 2 is somewhat confusing - https://phabricator.wikimedia.org/T183185#4127154 (10Amitjoki) @Nuria sure. I'll remove all the other changes and will just implement that switch from green to blue for now :) [15:16:29] elukey, I checked mysql purging and tl;dr looks great. Checked that 1) default fields are whitelisted 2) non-whitelisted capsule fields are purged 3) whitelisted capsule fields are kept 4) non-whitelisted event fields are purged 5) whitelisted event fields are kept 5) userAgent fields with partial spec actually keep the whole field 6) tables not in the whitelist are completely purged [15:17:32] nice! [15:17:36] I compared events from 2018-01-11 (until 11h) purged by yesterday's execution, vs events from 2018-01-12 (until 11h) purged by today [15:17:39] 's execution [15:17:40] (03CR) 10Amitjoki: ">" (033 comments) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425541 (https://phabricator.wikimedia.org/T191824) (owner: 10Amitjoki) [15:17:55] mforns: if you want I can scp the log of the cleaner to a stat host so you can check it [15:17:58] and events from 2018-01-13 not yet purged [15:18:12] elukey, ok, sure! [15:19:35] mforns: stat1005, your home, eventlogging_cleaner.log [15:20:59] mforns: another thing - while working on some shared puppet code for the whitelist (and after some suggestions by ottomata) I thought to put it under /etc/analytics/sanitization/whitelist.yaml [15:21:06] on an1003 and the db hosts [15:21:13] would it be fine for you? [15:21:18] seems generic and self explanatory [15:21:33] maybe /etc/analytics/sanitization/eventlogging_purging_whitelist.yaml [15:21:38] yeah better [15:22:51] ottomata: wdyt? [15:27:07] (03PS2) 10Amitjoki: Add Vue Filters to make the code clean and use them as necessary for casing [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425541 (https://phabricator.wikimedia.org/T191824) [15:27:09] (03PS5) 10Amitjoki: Fix the confusing display of radio buttons [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/422407 (https://phabricator.wikimedia.org/T183185) [15:28:39] elukey: ya fine with me, i don't really like putting this in /etc/_anything_, seems a little strange, but i hope it'll be temporary til it is moved to refinery [15:29:27] ottomata: yeah I am trying to use something that kinda explains what those files are meant for, ideally refinery would be better :( [15:34:44] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: [EL sanitization] Ensure presence of EL YAML whitelist in analytics1003 - https://phabricator.wikimedia.org/T189691#4127240 (10elukey) a:03elukey [15:42:13] (03CR) 10Milimetric: [C: 04-1] "adding a -1 to draw attention to the review comments in case you didn't see them. I didn't look at the code yet because the thoughts are " [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/423904 (https://phabricator.wikimedia.org/T188277) (owner: 10Fdans) [15:55:36] (03PS3) 10Amitjoki: Add Vue Filters to make the code clean and use them as necessary for casing [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425541 (https://phabricator.wikimedia.org/T191824) [15:56:51] (03Abandoned) 10Amitjoki: Add Vue Filters to make the code clean and use them as necessary for casing [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425541 (https://phabricator.wikimedia.org/T191824) (owner: 10Amitjoki) [15:57:21] (03Abandoned) 10Amitjoki: Fix the confusing display of radio buttons [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/422407 (https://phabricator.wikimedia.org/T183185) (owner: 10Amitjoki) [15:59:31] (03CR) 10Nuria: "Changes look fine, please just add them to a patch that sits on top of master" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425541 (https://phabricator.wikimedia.org/T191824) (owner: 10Amitjoki) [16:00:02] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Add Ecosia and Startpage to list of search engines - https://phabricator.wikimedia.org/T191714#4114658 (10Nuria) [16:00:16] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: Updating Wikistats 2.0 github Readme - https://phabricator.wikimedia.org/T191567#4127299 (10Nuria) [16:00:20] (03CR) 10Amitjoki: "> Changes look fine, please just add them to a patch that sits on top" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425541 (https://phabricator.wikimedia.org/T191824) (owner: 10Amitjoki) [16:00:32] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: Updating Wikistats 2.0 github Readme - https://phabricator.wikimedia.org/T191567#4110264 (10Nuria) 05Open>03Resolved [16:08:45] (03PS1) 10Amitjoki: Add Vue Filters to make the code clean and use them as necessary for casing [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425845 (https://phabricator.wikimedia.org/T191824) [16:09:26] 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: Limit pan in Wikistats2 maps - https://phabricator.wikimedia.org/T189195#4127342 (10Nuria) [16:20:40] (03PS1) 10Amitjoki: Fix the confusing color alteration when toggling the radio button [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425847 (https://phabricator.wikimedia.org/T183185) [16:29:41] nuria_: Notice: /Stage[main]/Profile::Analytics::Refinery::Job::Sqoop_mediawiki/Cron[refinery-sqoop-mediawiki]/monthday: monthday changed '2' to '5' [16:29:44] :) [16:47:36] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: Change 'NaN' to something more helpful in metrics % change over the selected time range - https://phabricator.wikimedia.org/T192028#4124963 (10fdans) p:05Triage>03Normal [16:48:14] 10Analytics, 10Analytics-Kanban, 10ChangeProp, 10EventBus, and 3 others: Disable MirrorMaker for job queue events - https://phabricator.wikimedia.org/T192005#4127541 (10fdans) [16:49:47] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: The popup in the line chart is obstructive - https://phabricator.wikimedia.org/T191985#4127547 (10fdans) a:05Amitjoki>03fdans [16:50:26] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: The popup in the line chart is obstructive - https://phabricator.wikimedia.org/T191985#4123443 (10fdans) Hey @sahil505 I'm already working on this as part of T188277. Thanks! [16:50:39] elukey: thnaks [16:51:38] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: The popup in the line chart is obstructive - https://phabricator.wikimedia.org/T191985#4123443 (10fdans) p:05Triage>03High [16:52:00] 10Analytics: Clickstream dataset for Persian Wikipedia only includes external values - https://phabricator.wikimedia.org/T191964#4122737 (10fdans) p:05Triage>03High [16:54:38] 10Analytics, 10User-Elukey: Report updater setting log ownership incorrectly (leading to cronspam) - https://phabricator.wikimedia.org/T191871#4127576 (10fdans) p:05Triage>03Normal [16:55:44] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Add Ecosia and Startpage to list of search engines - https://phabricator.wikimedia.org/T191714#4127583 (10fdans) p:05Triage>03Normal [16:57:51] Hi nuria_ ! Do you have time for a question? [16:58:03] chelsyx: on meeting, we can talk in 1 hr? [16:58:11] Sure! [16:59:30] 10Analytics, 10Analytics-EventLogging, 10MediaWiki-General-or-Unknown, 10WMF-Legal, 10Privacy: Collect IPs for pingback - https://phabricator.wikimedia.org/T191691#4127600 (10fdans) 05Open>03Resolved a:03fdans The IPs are already stored in hdfs for 90 days, which you can access via hive. Please reo... [16:59:54] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Upgrading Wikistats 2.0 footer UI/design - https://phabricator.wikimedia.org/T191672#4127605 (10fdans) [17:03:35] 10Analytics: Research how hard it would be to do oauth with wkimediafoundation.org as an option for access to superset - https://phabricator.wikimedia.org/T191563#4127625 (10fdans) 05Open>03declined This site will move to a 3rd party soon. [17:06:32] (03CR) 10Nuria: [C: 032] "Could not agree more!" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425799 (owner: 10Ottomata) [17:06:34] 10Analytics: stats.wikimedia.org home page should link to wikistats 2 - https://phabricator.wikimedia.org/T191555#4127630 (10fdans) [17:06:36] 10Analytics-Kanban, 10Analytics-Wikistats: Wikistats Beta - https://phabricator.wikimedia.org/T186120#4127629 (10fdans) [17:07:03] 10Analytics: stats.wikimedia.org home page should link to wikistats 2 - https://phabricator.wikimedia.org/T191555#4109929 (10fdans) p:05Triage>03Normal [17:09:59] 10Analytics, 10Analytics-Wikistats: Add edit/upload distinction to mediawiki history pipeline - https://phabricator.wikimedia.org/T178017#4127656 (10fdans) p:05Triage>03Low [17:10:50] 10Analytics, 10Analytics-Kanban: Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#4127659 (10fdans) [17:14:13] 10Analytics, 10Analytics-EventLogging, 10Performance: Spin out a tiny EventLogging RL module for lightweight logging - https://phabricator.wikimedia.org/T187207#4127672 (10fdans) p:05Triage>03High [17:15:32] 10Analytics, 10Analytics-Wikistats: Wikistats Bug - differences in view stats for smaller wikipedias - https://phabricator.wikimedia.org/T188613#4013530 (10fdans) p:05Triage>03High [17:21:55] 10Analytics: Whitelist analytics.wikimedia.org and stats.wikimedia.org in ad blockers - https://phabricator.wikimedia.org/T182816#4127689 (10fdans) p:05Triage>03Low [17:22:21] 10Analytics: Remove request for font.googleapis.com from analytics.wikimedia.org - https://phabricator.wikimedia.org/T182804#4127690 (10fdans) p:05Triage>03Low [17:23:35] 10Analytics, 10Analytics-EventLogging, 10Easy: Archive and drop the MobileOptionsTracking EventLogging MySQL table - https://phabricator.wikimedia.org/T185339#4127692 (10fdans) p:05Triage>03Low [17:25:13] 10Analytics: More granular permits for HDFS user when it comes to data access - https://phabricator.wikimedia.org/T189135#4127696 (10fdans) 05Open>03declined Not needed since trash [17:25:47] 10Analytics, 10User-Elukey: Expand the Hadoop Journal nodes from 3 to 5 to improve resiliency - https://phabricator.wikimedia.org/T189105#4127698 (10fdans) p:05Triage>03Low [17:26:41] 10Analytics, 10Analytics-Kanban, 10Services (watching), 10User-Elukey: Upgrade Kafka Burrow to 1.0 - https://phabricator.wikimedia.org/T188719#4127699 (10fdans) 05Open>03Resolved [17:27:58] 10Analytics: AQS edits API should not allow queries without time bounds - https://phabricator.wikimedia.org/T189623#4047895 (10fdans) p:05Triage>03Normal [17:29:05] 10Analytics, 10Performance-Team (Radar): Possible statsv corruption? - https://phabricator.wikimedia.org/T189530#4127705 (10fdans) p:05Triage>03Low [17:44:38] (03PS1) 10Ladsgroup: Track number of constraint definition snaks over time [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/425872 (https://phabricator.wikimedia.org/T191891) [17:44:44] (03CR) 10jerkins-bot: [V: 04-1] Track number of constraint definition snaks over time [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/425872 (https://phabricator.wikimedia.org/T191891) (owner: 10Ladsgroup) [17:45:56] (03CR) 10Ladsgroup: "recheck" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/425872 (https://phabricator.wikimedia.org/T191891) (owner: 10Ladsgroup) [17:55:18] * elukey off! [18:19:31] chelsyx: i have time now [18:19:44] chelsyx: let me know if you want to met [18:22:03] nuria_: I have a question about our discussion yesterday: Can we white list the user history table? If not, with our 90-day purging policy, some user state info would be lost even if that is the user's most current state. [18:22:26] chelsyx: mmm.. no, cause it is purged by date [18:22:34] chelsyx: only if >90 days old [18:22:53] chelsyx: makes sense? [18:23:08] nuria_: yes, but the timestamp would be the time that user last update this property value, right? [18:23:45] chelsyx: right, so updates older than 90 days will be lost if fields are to be purged [18:24:00] nuria_: for example, user add a new list 90 days ago, so in the user history table, I will have record like [user1, readinglist_listcount, 3] [18:24:46] chelsyx: and that would be purged according to privacy policy if fields are sensitive [18:24:49] nuria_: if the listcount don't change in 90 days, then this record would be deleted, and I won't be able to know the listcount for this user after 90days [18:24:57] chelsyx: i am not certain all are however [18:29:10] nuria_: We will have appInstallID as the PK. Is it considered sensitive? [18:29:34] chelsyx: that is being decided by legal now, still on the works so maybe yes [18:30:07] chelsyx: not definitive though [18:30:11] nuria_: hmmmmm [18:32:34] chelsyx: MAYBE not [18:32:39] chelsyx: hard to say [18:33:04] 10Analytics-Kanban, 10Patch-For-Review: Refresh SWAP notebook hardware - https://phabricator.wikimedia.org/T183145#4127859 (10Ottomata) > But in any case I can't get pyhive to work either right now Hm, pyhive seems to work just fine for me: ``` from pyhive import hive cursor = hive.connect('analytics1003.eqi... [18:43:17] (03PS2) 10Joal: Update MediawikiHistory-denormalize [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425798 [18:46:16] (03CR) 10Nuria: "Let's please remember to update wiki: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Data_Lake/Edits/Pipeline/Page_and_user_history" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425798 (owner: 10Joal) [18:47:02] (03CR) 10Nuria: [C: 032] Rename SparkSQLHiveExtensions to just HiveExtensions [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/425809 (owner: 10Ottomata) [18:47:22] nuria_: I actually read those wiki pages and the mf-files doc in the code, and no mention of the by-year split was made ! [18:47:57] nuria_, milimetric: And also, I have looked deeper in the 2GB file problem --> It's not yet solve [18:48:31] However, spark2 provides a "better" compression scheme for RDDs with a lot (>2000) partitions [18:48:56] joal: right, i was thking that probably page needs an update since september 2017, right? [18:49:01] 10Analytics-Kanban, 10Patch-For-Review: Spark 2 as cluster default (working with oozie) - https://phabricator.wikimedia.org/T159962#4127907 (10Ottomata) [18:49:04] joal: more details? [18:49:59] interesting [18:52:07] 10Analytics-Kanban, 10Patch-For-Review: Refresh SWAP notebook hardware - https://phabricator.wikimedia.org/T183145#4127943 (10Ottomata) [19:04:45] chelsyx: not considering what is PII or not, (and also knowing nothign about your convo with nuria), it sounds like you could just maintain your own aggregate per user table/dataset somewhere [19:05:44] evmit events like (user_id, list_count_current (or list_count_change)) [19:05:54] then each hour, launch a job that updates a table [19:06:06] (if you need realtime, that is possible, but nobody really does it) [19:06:19] i assume list_counts can go up and down per use? [19:07:51] elukey: still there? did we finish the jumbo broker bounce? [19:08:02] i'm going to move the java.security task to done [19:09:44] ottomata: Do you have time for a quick chat? I can explain to you [19:11:24] ottomata: her use case does not require table updates i think, it is just simple events [19:11:34] ottomata: like "add to list" or "remove from list" [19:11:55] ottomata: actually the timeseries is useful, without it there is no way to stablish how frequently feature is used [19:11:57] cc chelsyx [19:12:06] to correct me if needed [19:13:43] nuria_: I also have a event table to track when a user add to list, etc. So I don't need the time series in user history [19:14:25] nuria_: at least for now, we are not interested in a single user's time series [19:15:15] chelsyx: ok,m i see, in any case , seems like updates are uneeded as last record per appinstallid is the one you are interested on [19:15:31] 10Analytics, 10Analytics-Kanban, 10EventBus, 10Patch-For-Review, 10Services (watching): Use --new.consumer for main codfw <-> eqiad Kafka MirrorMaker - https://phabricator.wikimedia.org/T190940#4128010 (10Ottomata) We had planned to pause this until after the main Kafka upgrade in T167039, but from https... [19:15:47] chelsyx: ya i do [19:16:11] nuria_: exactly [19:16:55] (03CR) 10Mforns: Changed 'NaN' to 0 in metrics % change over the selected time range (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425747 (https://phabricator.wikimedia.org/T192028) (owner: 10Sahil505) [19:17:05] ottomata: my laptop is giving me some trouble. I have to restart. Will join the batcave in a minute [19:17:57] k [19:34:43] 10Analytics, 10Analytics-Kanban: Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#4128048 (10Ottomata) [19:44:51] (03CR) 10Mforns: [V: 032 C: 032] "LGTM!" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425847 (https://phabricator.wikimedia.org/T183185) (owner: 10Amitjoki) [19:53:47] 10Analytics, 10Analytics-Kanban: Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#4128097 (10Ottomata) [20:04:19] ottomata: have you looked into the pagecounts-ez problem more? [20:04:31] this thing says it should run daily: https://github.com/wikimedia/analytics-wikistats/blob/master/dammit.lt/read.me [20:04:41] the scripts in section B2 I think are the relevant ones [20:04:59] and they should be running under ezachte's username, on cron [20:12:43] 10Analytics, 10Analytics-Kanban: Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#4128145 (10Ottomata) [20:13:32] 10Analytics, 10Analytics-Kanban: Upgrade Kafka on main cluster with security features - https://phabricator.wikimedia.org/T167039#3315367 (10Ottomata) New producer in MirrorMaker 1.x is not compatible with old 0.9 broker. So once this upgrade happens, we can no longer Mirror to Old Kafka analytics cluster.... [20:14:44] milimetric: i'm pretty sure those files aren't showing up because of https://phabricator.wikimedia.org/T189283 [20:14:56] ezachte was supposed to have changed his jobs to write to the new /srv/dumps directory on stat1005 [20:15:02] where the new labstore hosts are pulling from [20:15:11] so his cron is probably running, but not writing to the proper place [20:16:46] monday is a holiday? [20:16:53] ottomata: gotcha, so maybe we can merge the change and deploy? Because it seems Erik hasn't answered for a bit [20:17:01] the change? [20:17:08] to his scripts [20:17:15] if you know how, go right ahead! :) [20:18:40] I can try to figure out how, but the problem is if he has something else locally that's not in source [20:18:48] then doing this will break things [20:18:56] yeah [20:19:01] I can check his directory maybe [20:19:09] do you know what folder this all runs out of? [20:19:17] on stat1005? [20:19:38] i think in his home somewhere [20:20:01] can you check his cron for the script dammit_compact_daily.sh? [20:21:05] milimetric: /home/ezachte/wikistats/dammit.lt/bash/dammit_compact_daily.sh [20:22:01] oh, i see what's happening milimetric the simplest thing you can do is search that for dataset1001 [20:22:05] ok, good, and it used to be writing to dataset1001.wikimedia.org::pagecounts-ez/merged/ before, and now it has to write to /srv/dumps/ but what path? pagecounts-ez/merged? or other/pagecounts-ez-merged? [20:22:47] checking his folder now for local changes [20:23:37] milimetric: hm /srv/dumps/pagecounts-ez/merged [20:23:43] https://gerrit.wikimedia.org/r/#/c/423539/2/modules/dumps/manifests/web/fetches/stat_dumps.pp [20:23:46] we'll have to merge ^ too [20:23:47] oooof... a LOT of changes [20:23:48] but we can do that i think [20:24:08] madhuvishy: i think dan is going to fix ezachtes stuff, you ok if I merge ^^^ after ? [20:25:00] Uh yeah go for it [20:25:03] ottomata: I think you have to fix it 'cause it would conflict with local changes he has to all those files [20:25:09] (03CR) 10Mforns: [V: 032 C: 032] "LGTM! Thanks for working on this :]" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425845 (https://phabricator.wikimedia.org/T191824) (owner: 10Amitjoki) [20:25:12] Labstore1006&7 [20:25:13] so the only way would be to fix it locally [20:25:21] wow [20:25:24] I'd check the compiler once [20:25:24] and I don't have permits [20:25:26] wow ok [20:25:29] geez [20:25:46] I'm not sure if it's worth it but I do know that Andrew West and other important consumers need this data [20:26:25] so milimetric all references to dataset1001 in dammit.lt/bash/* right? [20:26:56] ottomata: that would fix all the pageview stuff, but not sure if there's anything else in other folders, checking now [20:28:00] ottomata: I think this means we'd have to copy the existing stuff from dataset1001.wikimedia.org:: to /srv/dumps though, because some of these scripts just work off of existing data and they may go crazy if it's not there [20:29:25] ? [20:29:33] ottomata: ok, I think it's all references to dataset1001 everywhere, perl and other scripts too: https://github.com/wikimedia/analytics-wikistats/search?utf8=%E2%9C%93&q=dataset1001&type= [20:29:33] but they can't read from dataset1001 [20:32:07] milimetric: oof this is too much [20:32:12] can we just to dammit.lt/bash for now? [20:32:15] and get erik to do the rest? [20:32:44] ottomata: just checked everything more closely, yeah, you can do that [20:32:46] ok [20:32:47] they're all talking about output [20:32:57] none of them are reading from there [20:33:07] i've made a copy of dammit.lt/bash to my home dir [20:33:09] i'm about to run [20:33:10] the perl script has it commented out [20:33:51] for f in $(sudo grep -l dataset1001.wikimedia.org *); do sed -i 's@dataset1001.wikimedia.org::@/srv/dumps/@g' $f; done [20:34:52] !log replacing references to dataset1001.wikimedia.org:: with /srv/dumps in stat1005:~ezachte/wikistats/dammit.lt/bash: for f in $(sudo grep -l dataset1001.wikimedia.org *); do sudo sed -i 's@dataset1001.wikimedia.org::@/srv/dumps/@g' $f; done T189283 [20:34:53] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [20:34:54] T189283: Replace cron jobs from EZachte's home directory on stat1005 with rsync fetches - https://phabricator.wikimedia.org/T189283 [20:36:03] ottomata: looks good to me [20:36:14] hopefully nothing explodes, thanks [20:37:07] merged madhu's patch, running puppet [20:37:12] then we...wait? [20:37:19] (03CR) 10Sahil505: "@Mforns - I'll make corresponding changes based on if change cannot be calculated then an appropriate message is displayed that justifies " [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425747 (https://phabricator.wikimedia.org/T192028) (owner: 10Sahil505) [20:39:18] thabnks for rememebering about pagecounts-ez milimetric [20:39:45] ottomata: yeah, depends on how his crons are set up, we can check tomorrow [20:40:09] nuria_: yea, saw the email and saw that it was stuck on phab, but a lot of people use this data [20:40:32] milimetric: ya, totally, i wanted to follow up but been spending some time on page previews [20:40:41] and forgot completely [20:42:20] his crons run overnight [20:44:32] k, great, then we should check tomorrow morning. I'll make a separate task so I don't forget [20:44:47] k [20:47:03] 10Analytics-Kanban: Check that pagecounts-ez cron is ok - https://phabricator.wikimedia.org/T192110#4128266 (10Milimetric) [20:55:36] 10Analytics, 10EventBus, 10MediaWiki-JobQueue, 10Goal, and 2 others: FY17/18 Q4 Program 8 Services Goal: Complete the JobQueue transition to EventBus - https://phabricator.wikimedia.org/T190327#4128298 (10mobrovac) [21:01:39] Thanks ottomata and milimetric. I'll check in tomorrow too [21:37:12] ottomata: question if you may [21:37:19] or milimetric if you rae arround [21:37:22] *are [21:43:34] (03PS7) 10Nuria: [WIP] Add new search engines [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/423336 (https://phabricator.wikimedia.org/T191714) (owner: 10Jonas Kress (WMDE)) [21:55:52] nuria_: ya here [21:56:06] ottomata: nerverminddd i figured it out yeah! [21:56:14] ok! [21:56:15] :) [21:56:24] nuria_: did you know monday is a holiday? [21:56:32] ottomata: waht? [21:56:45] i just heard in services...is it erally? [21:56:45] ottomata: no [21:56:46] can't tell [21:56:55] ottomata: yes it is on calendar! [21:56:56] wo [21:57:02] ottomata: i will work anyways [21:57:08] ottomata: as i had planned to do [21:57:14] what is the holiday? [21:57:18] ottomata: but please take the day off [21:57:23] ottomata: happy april? [21:57:26] ottomata: no clue [21:57:26] hah i guess! [21:57:42] i'm not working tomorrow or next friday (worked last friday in order to take tomorrow off) [21:57:44] hadn't planned monday [21:57:46] might work it, might take it off [21:57:53] k [21:57:58] or maybe will save it and have a chill thurs and fri next week? [21:58:40] (03PS8) 10Nuria: Identify new search engines as such [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/423336 (https://phabricator.wikimedia.org/T191714) (owner: 10Jonas Kress (WMDE)) [21:58:47] ottomata: that works too [21:58:56] ottomata: just checkout with luca [21:59:33] (03PS9) 10Nuria: Identify new search engines as such [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/423336 (https://phabricator.wikimedia.org/T191714) (owner: 10Jonas Kress (WMDE)) [22:00:50] (03PS10) 10Nuria: Identify new search engines as such [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/423336 (https://phabricator.wikimedia.org/T191714) (owner: 10Jonas Kress (WMDE)) [22:01:30] (03PS11) 10Nuria: Identify new search engines as such [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/423336 (https://phabricator.wikimedia.org/T191714) (owner: 10Jonas Kress (WMDE)) [22:04:47] ottomata: reday for CR: https://gerrit.wikimedia.org/r/#/c/423336/ cleaned up the referrer code i think [22:07:28] 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Move away from jmxtrans in favor of prometheus jmx_exporter - https://phabricator.wikimedia.org/T175344#4128480 (10Nuria) [22:07:30] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Add the prometheus jmx exporter to all the Zookeeper daemons - https://phabricator.wikimedia.org/T177460#4128479 (10Nuria) 05Open>03Resolved [22:07:52] 10Analytics, 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Fix Mirror Maker erratic behavior when replicating from main-eqiad to jumbo - https://phabricator.wikimedia.org/T189464#4128482 (10Nuria) [22:07:54] 10Analytics, 10Analytics-Cluster, 10Analytics-Kanban: Spike: Consider alternatives to MirrorMaker: uReplicator, Confluent Replicator - https://phabricator.wikimedia.org/T190049#4128481 (10Nuria) 05Open>03Resolved [22:08:12] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10SEO: Make Google API Python Client Library available on stat* machines - https://phabricator.wikimedia.org/T190767#4128484 (10Nuria) 05Open>03Resolved [22:08:31] 10Analytics-Kanban, 10Patch-For-Review: Sqoop cu_changes table for geowiki - https://phabricator.wikimedia.org/T184759#4128487 (10Nuria) [22:08:33] 10Analytics-Kanban, 10Patch-For-Review: Oozie job to compute geowiki on top of sqooped data - https://phabricator.wikimedia.org/T188113#4128486 (10Nuria) 05Open>03Resolved [22:08:35] 10Analytics, 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review: Alert for Kafka MirrorMaker lag - https://phabricator.wikimedia.org/T189611#4128488 (10Nuria) 05Open>03Resolved [22:08:37] 10Analytics, 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Fix Mirror Maker erratic behavior when replicating from main-eqiad to jumbo - https://phabricator.wikimedia.org/T189464#4042527 (10Nuria) [22:09:02] 10Analytics-Kanban, 10Patch-For-Review: Remove sensitive fields from whitelist for QuickSurvey schemas (end of Q2) - https://phabricator.wikimedia.org/T174386#4128490 (10Nuria) 05Open>03Resolved [22:09:30] 10Analytics, 10Analytics-Kanban, 10ORES, 10Scoring-platform-team, 10Patch-For-Review: Enable ores::base on stat1006 - https://phabricator.wikimedia.org/T181646#4128501 (10Nuria) 05Open>03Resolved [22:10:00] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats, 10Patch-For-Review: Scroll should not bounce horizontally when swiping left/right - https://phabricator.wikimedia.org/T190959#4128502 (10Nuria) 05Open>03Resolved [22:10:17] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Mount dumps on SWAP machines (notebook1003.eqiad.wmnet / notebook1004.eqiad.wmnet) - https://phabricator.wikimedia.org/T176091#4128505 (10Nuria) 05Open>03Resolved [22:11:51] 10Analytics, 10Collaboration-Team-Triage, 10Contributors-Analysis, 10Contributors-Team, and 7 others: statistics about edit conflicts according to page type - https://phabricator.wikimedia.org/T139019#4128517 (10Neil_P._Quinn_WMF) p:05Normal>03Lowest [22:11:58] 10Analytics, 10Collaboration-Team-Triage, 10Contributors-Analysis, 10Contributors-Team, and 7 others: statistics about edit conflicts according to page type - https://phabricator.wikimedia.org/T139019#2416933 (10Neil_P._Quinn_WMF) p:05Lowest>03Triage [22:12:06] 10Analytics, 10Collaboration-Team-Triage, 10Contributors-Analysis, 10Contributors-Team, and 7 others: statistics about edit conflicts according to page type - https://phabricator.wikimedia.org/T139019#2416933 (10Neil_P._Quinn_WMF) p:05Triage>03Normal [22:15:36] (03PS12) 10Nuria: Identify new search engines as such [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/423336 (https://phabricator.wikimedia.org/T191714) (owner: 10Jonas Kress (WMDE)) [22:40:42] 10Analytics, 10Analytics-Wikistats, 10Patch-For-Review: Some metrics don't work in the topic selector - https://phabricator.wikimedia.org/T188268#4128555 (10sahil505) [22:54:06] 10Analytics-EventLogging, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Verify duplicate entry warnings logged by the m4 mysql consumer - https://phabricator.wikimedia.org/T185291#3912027 (10Nuria) Sorry it took me soooo looong to look (and understand) this code [22:54:20] 10Analytics-EventLogging, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Verify duplicate entry warnings logged by the m4 mysql consumer - https://phabricator.wikimedia.org/T185291#4128585 (10Nuria) 05Open>03Resolved [22:55:37] 10Analytics-Kanban, 10Operations, 10Patch-For-Review: Eventlogging mysql consumers inserted rows on the analytics slave (db1108) for two hours - https://phabricator.wikimedia.org/T188991#4128588 (10Nuria) 05Open>03Resolved [23:14:47] 10Analytics-Dashiki, 10Analytics-Kanban, 10Patch-For-Review: Show most recent data in table-timeseries - https://phabricator.wikimedia.org/T191273#4128642 (10Nuria) 05Open>03Resolved [23:14:56] 10Analytics-Dashiki, 10Analytics-Kanban, 10Patch-For-Review: Show most recent data in table-timeseries - https://phabricator.wikimedia.org/T191273#4099340 (10Nuria) [23:21:45] (03PS1) 10Sturmkrahe: Fix accessibility/markup issues [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/425925 (https://phabricator.wikimedia.org/T185533) [23:35:44] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats: Upgrading Wikistats 2.0 footer UI/design - https://phabricator.wikimedia.org/T191672#4128681 (10sahil505) Here is the updated mock-up for the footer. Have paid more attention to CSS in this one. Improvements/corrections made : - Wikimedia logo is correc... [23:45:17] 10Analytics, 10Analytics-Kanban, 10Analytics-Wikistats, 10Google-Summer-of-Code (2018): GSoC Proposal 2018 : [Analytics] Improvements to Wikistats2 front-end - https://phabricator.wikimedia.org/T189964#4128700 (10sahil505)