[00:23:34] <Krinkle>	 I'm trying to test the eventstreams in Beta, but not getting any results
[00:23:38] <Krinkle>	 I thought it was enabled there recently
[00:23:47] <Krinkle>	 Using curl https://eventstreams-beta.wmflabs.org/v2/stream/recentchange
[00:23:56] <Krinkle>	 Also tried on a modified version of http://codepen.io/ottomata/pen/VKNyEw?editors=0011
[00:24:15] <Krinkle>	 deleted various pages and created various pages, but no results.
[00:24:22] <Krinkle>	 the revision-create channel does work
[00:31:41] <wikibugs>	 10Analytics, 10Analytics-EventLogging: Remove ad-hoc UA logging from existing schemas - https://phabricator.wikimedia.org/T61832#2984125 (10Krinkle)
[00:32:43] <wikibugs>	 10Analytics, 10Analytics-EventLogging: Remove ad-hoc UA logging from existing schemas - https://phabricator.wikimedia.org/T61832#625670 (10Krinkle)
[01:43:06] <wikibugs>	 10Analytics, 10EventBus, 10Reading-Web-Trending-Service, 13Patch-For-Review, and 2 others: Compute the trending articles over a period of 24h rather than 1h - https://phabricator.wikimedia.org/T156411#2984614 (10Fjalapeno) It sounds like we need to decouple the amount of data that we are ingesting from the...
[07:28:35] <wikibugs>	 10Analytics, 10DBA: Json_extract available on analytics-store.eqiad.wmnet - https://phabricator.wikimedia.org/T156681#2983518 (10Marostegui) `research` user doesn't appear to have EXECUTE privileges. Was that working before or is it the first time you've tried to play with that function?  Thanks!
[07:46:18] <icinga-wm>	 RECOVERY - cassandra-b CQL 10.64.0.237:9042 on aqs1007 is OK: TCP OK - 0.000 second response time on 10.64.0.237 port 9042
[07:46:46] <elukey>	 \o/
[08:19:05] <wikibugs>	 10Analytics-EventLogging, 06Analytics-Kanban, 13Patch-For-Review: Add user_agent_map field to EventCapsule - https://phabricator.wikimedia.org/T153207#2984935 (10Nemo_bis) > Replacing the raw UA entirely is something we could discuss too, but this would need to be preceded by a consultation  As Nuria said, t...
[08:21:33] <wikibugs>	 10Analytics-General-or-Unknown, 06Analytics-Kanban, 13Patch-For-Review, 07Privacy: analytics.wikimedia.org loads resources from third parties - https://phabricator.wikimedia.org/T156347#2984950 (10Nemo_bis)
[08:31:52] <elukey>	 !log started Cassandra nodetool cleanup for aqs1004-a
[08:31:53] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[08:41:34] <elukey>	 cleanup is running fine, no impact to the cluster
[08:42:34] <elukey>	 will do it for aqs1004-b and aqs1007-a too today/tomorrow
[08:43:00] <elukey>	 and I think we are ready to put aqs1007 active in the load balancer to serve trafifc!
[08:43:03] <elukey>	 *traffic!
[08:45:05] <moritzm>	 I need to install an nginx security update on analytics1027, interruption should be less than a second during nginx's reload, ok to proceed or rather a diffrent time?
[08:45:56] <elukey>	 moritzm: all good, it is not a critical system
[08:46:30] <moritzm>	 and done
[08:46:52] <elukey>	 thanks!
[09:25:12] <elukey>	 so aqs1007 is serving HTTP traffic
[09:25:25] <elukey>	 and aqs1007-b and aqs1004-a are compacting 
[09:25:32] <elukey>	 latencies are good
[09:25:57] <elukey>	 theoretically I could start compacting on aqs1004-b too
[09:29:11] <wikibugs>	 10Analytics, 10DBA: Drop m3 from dbstore servers - https://phabricator.wikimedia.org/T156758#2985275 (10Marostegui)
[09:49:57] <joal>	 elukey: awesome job :)
[10:10:32] <wikibugs>	 (03PS14) 10Joal: Add mediawiki history spark jobs to refinery-job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/325312 (https://phabricator.wikimedia.org/T141548)
[11:19:16] <mforns>	 hey team :]
[11:20:49] <elukey>	 o/
[11:35:28] <wikibugs>	 (03PS1) 10Nschaaf: (in progress) Store anonymized and purge sensitive data for WDQS [analytics/refinery] - 10https://gerrit.wikimedia.org/r/335211 (https://phabricator.wikimedia.org/T146915)
[12:03:46] <joal>	 Hi schana, do you want us to spend some time now?
[12:03:57] <schana>	 works for me
[12:04:40] <joal>	 schana: k - https://hangouts.google.com/hangouts/_/wikimedia.org/a-batcave
[12:22:02] <wikibugs>	 (03CR) 10Nschaaf: [C: 04-1] "Needs to be reviewed in terms of privacy policy for what data gets kept along with the implementation/naming." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/335211 (https://phabricator.wikimedia.org/T146915) (owner: 10Nschaaf)
[13:06:05] <wikibugs>	 10Analytics-EventLogging, 06Analytics-Kanban, 13Patch-For-Review: Add user_agent_map field to EventCapsule - https://phabricator.wikimedia.org/T153207#2986055 (10Tbayer) >>! In T153207#2984935, @Nemo_bis wrote: >> Replacing the raw UA entirely is something we could discuss too, but this would need to be prec...
[13:07:36] * elukey commuting to the office! afk for a bit
[13:12:53] <fdans>	 Lunch!
[13:50:54] <wikibugs>	 10Analytics-EventLogging, 06Analytics-Kanban, 10DBA, 13Patch-For-Review: Add autoincrement id to EventLogging MySQL tables. {oryx} - https://phabricator.wikimedia.org/T125135#1979211 (10Marostegui) Hi  Is this ticket a consequence of: https://phabricator.wikimedia.org/T87661?  So removing the autoincrement...
[13:59:14] <elukey>	 just rebooted analytics1039
[14:00:05] <joal>	 k elukey, as a test for the rest of the cluster?
[14:00:57] <elukey>	 joal: yes I am testing my hacky fstab refactor script :)
[14:01:03] <joal>	 :)
[14:01:27] <elukey>	 it wasn't running any app master afaics
[14:03:59] <wikibugs>	 10Analytics, 10Analytics-Cluster, 15User-Elukey: Audit fstabs on Kafka and Hadoop nodes to use UUIDs instead of /dev paths - https://phabricator.wikimedia.org/T147879#2986275 (10elukey) Results look good!  ``` elukey@analytics1039:~$ cat /proc/mounts rootfs / rootfs rw 0 0 sysfs /sys sysfs rw,nosuid,nodev,no...
[14:04:08] <elukey>	 all good, an1039 up and running
[14:07:36] <wikibugs>	 10Analytics-Tech-community-metrics, 06Labs: http://korma.wmflabs.org/ got erased - https://phabricator.wikimedia.org/T156253#2986278 (10Lcanasdiaz) @Aklapper if we want to recover the legacy Bitergia dashboard what we have to do is: * deploying a virtual machine with a web server like apache/nginx * clone this...
[14:07:38] <wikibugs>	 10Analytics-Cluster, 06Analytics-Kanban, 15User-Elukey: Audit fstabs on Kafka and Hadoop nodes to use UUIDs instead of /dev paths - https://phabricator.wikimedia.org/T147879#2986279 (10elukey)
[14:08:05] <wikibugs>	 10Analytics-Cluster, 06Analytics-Kanban, 15User-Elukey: Audit fstabs on Kafka and Hadoop nodes to use UUIDs instead of /dev paths - https://phabricator.wikimedia.org/T147879#2706774 (10elukey) @Ottomata - sanity check before applying it to other nodes? :)
[14:09:14] <wikibugs>	 (03PS10) 10Mforns: Add banner activity oozie jobs [analytics/refinery] - 10https://gerrit.wikimedia.org/r/331794 (https://phabricator.wikimedia.org/T155141)
[14:11:22] <wikibugs>	 (03CR) 10Mforns: "I added the field 'country_matches_geocode', which is a boolean that is true when the country field (passed in the URI by the banner clien" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/331794 (https://phabricator.wikimedia.org/T155141) (owner: 10Mforns)
[14:16:32] <elukey>	 urandom: ping (if you have 5 mins)
[14:20:00] <wikibugs>	 10Analytics, 06Developer-Relations, 10MediaWiki-API, 06Reading-Admin, and 3 others: Is User-Agent data PII when associated with Action API requests? - https://phabricator.wikimedia.org/T154912#2986296 (10Nemo_bis)
[14:23:05] <wikibugs>	 06Analytics-Kanban, 10Fundraising-Backlog, 13Patch-For-Review: Productionize banner impressions druid/pivot dataset - https://phabricator.wikimedia.org/T155141#2986311 (10mforns) @AndyRussG @JAllemandou  I have added the 'country_matches_geocode' field to the banner_activity jobs, as per your suggestion. See...
[14:25:34] <wikibugs>	 06Analytics-Kanban, 10Fundraising-Backlog, 13Patch-For-Review: Productionize banner impressions druid/pivot dataset - https://phabricator.wikimedia.org/T155141#2986319 (10JAllemandou) Works for me @mforns, it seems indeed the best choice you made (as usual ;)
[14:26:14] <wikibugs>	 (03CR) 10Joal: [C: 031] "LGTM again, let's wait for @andyrussg before merging :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/331794 (https://phabricator.wikimedia.org/T155141) (owner: 10Mforns)
[14:27:57] <wikibugs>	 06Analytics-Kanban, 10Fundraising-Backlog, 13Patch-For-Review: Productionize banner impressions druid/pivot dataset - https://phabricator.wikimedia.org/T155141#2934689 (10Ottomata) BTW, we can pause deletion jobs for a bit without violating privacy policy, if that helps!
[14:37:42] <wikibugs>	 06Analytics-Kanban, 10Fundraising-Backlog, 13Patch-For-Review: Productionize banner impressions druid/pivot dataset - https://phabricator.wikimedia.org/T155141#2986358 (10mforns) @Ottomata Good idea!
[14:52:01] <wikibugs>	 06Analytics-Kanban, 10Pageviews-API, 06Reading-analysis: Skewed pageviews for Azerbaijani and Bulgarian Wikipedias, September, October and November 2016 - https://phabricator.wikimedia.org/T153699#2986387 (10JAllemandou) More investigation: Looking at December 2016 22nd:  - Almost every 'xss' pageview is fro...
[15:01:18] <mforns>	 joal, yt?
[15:15:51] <joal>	 Hey mforns 
[15:15:57] <mforns>	 hey joal!
[15:15:59] <joal>	 sorry, missed your ping
[15:16:13] <mforns>	 np! do you have 2 mins to batcave?
[15:16:19] <joal>	 sure mforns !
[15:16:23] <mforns>	 ok, omw
[15:22:35] <wikibugs>	 06Analytics-Kanban, 10Fundraising-Backlog, 13Patch-For-Review: Productionize banner impressions druid/pivot dataset - https://phabricator.wikimedia.org/T155141#2986529 (10Ottomata) Ok, done in https://gerrit.wikimedia.org/r/#/c/335237/  I've extended the refined webrequest retention to 90 days.  We need to s...
[15:24:01] <wikibugs>	 10Analytics, 10ChangeProp, 10EventBus, 06Revision-Scoring-As-A-Service, and 2 others: Rewrite ORES precaching change propagation configuration as a code module - https://phabricator.wikimedia.org/T148714#2986549 (10Ottomata) Hmmm, ideas....not really.  A couple of comments maybe:  > Writing your own Kafka...
[15:30:09] <wikibugs>	 (03PS15) 10Joal: Add mediawiki history spark jobs to refinery-job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/325312 (https://phabricator.wikimedia.org/T141548)
[15:30:25] <mforns>	 ottomata, yt? :]
[15:31:34] <ottomata>	 yaaa
[15:31:54] <ottomata>	 mforns:  hi	
[15:32:44] <mforns>	 hey, I'm looking for a way to put pagecounts-raw or pagecounts-ez since 2008 in hdfs, Joseph suggested to mount in on top of stat1002:/mnt/data, but I don't know where those files live
[15:32:48] <mforns>	 ottomata, ^
[15:33:40] <mforns>	 I could curl them from dumps.wikimedia.org (+- 10 files, 100MB each), but probably there's a better way
[15:35:22] <wikibugs>	 10Analytics-EventLogging, 06Analytics-Kanban, 13Patch-For-Review: Add user_agent_map field to EventCapsule - https://phabricator.wikimedia.org/T153207#2986597 (10Nemo_bis) I'm not saying the consultation was perfect, only that it already happened. It's always good to store less private data; a new consultati...
[15:36:54] <ottomata>	 mforns:  on stat1002,  /mnt/data/xmldatadumps/public/other
[15:37:02] <ottomata>	 pagecounts-ez, pagecounts-raw
[15:37:11] <ottomata>	 that's an NFS mount from the dumps.wm.org server
[15:37:19] <ottomata>	 so you can hdfs dfs -put those 
[15:37:22] <ottomata>	 from stat1002
[15:37:28] <mforns>	 ottomata, oh cool, thanks!
[15:38:01] <joal>	 oh, didn't know about those ! Awesome ! Thanks ottomata :)
[15:38:37] <elukey>	 ottomata: do we have an NFS mount now or was it already there?
[15:41:16] <ottomata>	 it's been there forever elukey
[15:41:26] <ottomata>	 ezachte uses it to make wikistats every month
[15:41:44] <ashgrigas>	 nuria mforns milimetric fdans updated the feedback wiki https://www.mediawiki.org/wiki/Wikistats_2.0_Design_Project/RequestforFeedback/Round1
[15:41:50] <ashgrigas>	 try to review before we meet if you have time
[15:41:59] <mforns>	 ashgrigas, OK
[15:42:03] <ashgrigas>	 havent filled in key questions as i thought we could do it together
[15:42:17] <elukey>	 ahhaah I will never get how those hosts are configured
[15:42:23] <fdans>	 ashgrigas: will do, thank you!
[15:42:39] <milimetric>	 thanks ashgrigas, will do
[15:43:20] <wikibugs>	 10Analytics-EventLogging, 06Analytics-Kanban, 10DBA, 13Patch-For-Review: Add autoincrement id to EventLogging MySQL tables. {oryx} - https://phabricator.wikimedia.org/T125135#2986648 (10Ottomata) Hah, hilarious.  Ok, IF all of the current tables that eventlogging is writing to have auto-increment IDs, then...
[15:45:46] <wikibugs>	 10Analytics-EventLogging, 06Analytics-Kanban, 10DBA, 13Patch-For-Review: Add autoincrement id to EventLogging MySQL tables. {oryx} - https://phabricator.wikimedia.org/T125135#2986659 (10jcrespo) > and use normal MySQL replication  Which I'd love, but this application refuses to be compatible with, and I go...
[15:46:55] <wikibugs>	 10Analytics-EventLogging, 06Analytics-Kanban, 10DBA, 13Patch-For-Review: Add autoincrement id to EventLogging MySQL tables. {oryx} - https://phabricator.wikimedia.org/T125135#2986686 (10Nuria) @Marostegui : +1 to what @Ottomata said  The prior incarnation of the system (which has been heavily upgraded now)...
[15:48:06] <wikibugs>	 10Analytics-EventLogging, 06Analytics-Kanban, 10DBA, 13Patch-For-Review: Add autoincrement id to EventLogging MySQL tables. {oryx} - https://phabricator.wikimedia.org/T125135#2986694 (10Marostegui) >>! In T125135#2986648, @Ottomata wrote: > Hah, hilarious.  Ok, IF all of the current tables that eventloggin...
[15:56:09] <wikibugs>	 (03PS3) 10Joal: Add spark streaming job for banner impressions [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/333655
[15:57:20] <wikibugs>	 10Analytics-EventLogging, 06Analytics-Kanban, 10DBA, 13Patch-For-Review: Add autoincrement id to EventLogging MySQL tables. {oryx} - https://phabricator.wikimedia.org/T125135#2986724 (10jcrespo) > This is key, is that somehow doable from the application side?  Note analytics has refused to acknowledge/decl...
[15:57:28] <wikibugs>	 10Analytics, 10Analytics-Wikistats, 10Android-app-Bugs, 06Wikipedia-Android-App-Backlog: Gõychi Konknni's English Wikistats translation is incorrect - https://phabricator.wikimedia.org/T156814#2986727 (10Niedzielski)
[16:00:16] <nuria>	 a-team: standuppp
[16:03:36] <wikibugs>	 10Analytics, 10Analytics-Wikistats, 10Android-app-Bugs, 06Wikipedia-Android-App-Backlog: Gõychi Konknni's English Wikistats translation is incorrect - https://phabricator.wikimedia.org/T156814#2986749 (10The_Discoverer)
[16:04:01] <elukey>	 !log started Cassandra nodetool cleanup for aqs1004-b
[16:04:02] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[16:05:34] <joal>	 elukey: 1004-a finished?
[16:06:01] <wikibugs>	 10Analytics-EventLogging, 06Analytics-Kanban, 13Patch-For-Review: Add user_agent_map field to EventCapsule - https://phabricator.wikimedia.org/T153207#2986756 (10Nuria) a:05fdans>03Nuria
[16:09:17] <elukey>	 joal: yep
[16:09:26] <joal>	 heard it at standup :)
[16:09:35] <elukey>	 but theoretically we could have ran them in parallel
[16:09:35] <joal>	 cool elukey, it's fast !
[16:09:40] <joal>	 yes, know that
[16:09:43] <elukey>	 BUT
[16:09:53] <elukey>	 I am always on the fence with cassandra :P
[16:09:56] <joal>	 I think you actually can do that for 1007-a in parallel from 1004-b
[16:09:58] <elukey>	 maybe I can start aqs1007-a
[16:09:59] <joal>	 :D
[16:10:06] <elukey>	 ahhaha ok! 
[16:10:07] <joal>	 ok, same page here :)
[16:10:17] <elukey>	 aqs1007-a should be quick
[16:10:39] <joal>	 I actually also think it's be good to do it on aqs1007-b even if it shouldn't do anything
[16:10:58] <joal>	 elukey: --^ Like clean rack everywhere :)
[16:11:25] <elukey>	 !log started Cassandra nodetool cleanup for aqs1007-a
[16:11:26] <stashbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log
[16:11:43] <elukey>	 sure!
[16:12:48] <wikibugs>	 10Analytics-EventLogging, 06Analytics-Kanban, 13Patch-For-Review: Add user_agent_map field to EventCapsule - https://phabricator.wikimedia.org/T153207#2986793 (10Nuria) a:05Nuria>03fdans
[16:14:02] <joal>	 ottomata, elukey: CDH 5.10 is out ;)
[16:14:25] <joal>	 BUT, there's nothing game changing for us in there
[16:15:10] <elukey>	 buuuuu
[16:15:41] <joal>	 yeh, I'm willing to get spark 2.0 (or even 2.1)
[16:17:35] <ottomata>	 whaaa no spark 2.0?!
[16:19:38] <joal>	 nope :(
[16:20:38] <joal>	 a guy at Apache conf last nov told me it would move with next major version change
[16:25:42] <elukey>	 we might want to think about upgrading before spark 2.0
[16:25:47] <wikibugs>	 10Analytics-EventLogging, 06Analytics-Kanban, 10DBA, 13Patch-For-Review: Add autoincrement id to EventLogging MySQL tables. {oryx} - https://phabricator.wikimedia.org/T125135#2986872 (10Ottomata) > Note analytics has refused to acknowledge/decline this problem several times Not sure this is an accurate rep...
[16:25:50] <joal>	 elukey: agreed
[16:26:33] <wikibugs>	 06Analytics-Kanban: Debian package for ua parser latest version - https://phabricator.wikimedia.org/T156821#2986875 (10Nuria)
[16:26:43] <joal>	 elukey: spark 2.0 is usable with current version even if not main version
[16:27:50] <elukey>	 goooood
[16:28:06] * elukey wants Debian Jessie on all the hosts
[16:31:38] <fdans>	 @ottomata are you guys still doing civ6 online games?
[16:33:06] <milimetric>	 mforns: if the transfer of those files takes like 3 months, let's see if we can work around it somehow, like physically moving a drive or something
[16:33:52] <mforns>	 milimetric, 3 months!!! why? It should be like 2.5 GB no?
[16:34:31] <mforns>	 milimetric, oh no! that were projectcounts files... you're right
[16:36:04] <ottomata>	 fdans:  ha, we did civ V
[16:36:14] <fdans>	 ah right
[16:36:27] <ottomata>	 but not often!  we did a giant 2 day run with 7 human players on quick mode, but still only got halfway through the game
[16:36:44] <mforns>	 milimetric, oh man... it's like 8 TB
[16:36:45] <nuria>	 fdans,mforns: retrooooo
[16:38:24] <ottomata>	 oh ya elukey sorry, looking into that cron now
[16:38:40] <elukey>	 ottomata: \o/
[16:38:56] <elukey>	 I can see that the dir is not there anymore but didn't have the time to review what you guys are doing :(
[16:43:22] <ottomata>	 yeah, elukey ok, i could fix now, but i actually want to revert some work I did after talkign to dan on thursday... will do after meetings
[16:43:41] <elukey>	 no hurry, I was just alerting you :)
[17:03:54] <ottomata>	 oh elukey if/when you add that jmx port
[17:03:55] <ottomata>	 https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Ports#JMX
[17:04:49] <mforns>	 milimetric, I guess you're going to have lunch right?
[17:04:58] <milimetric>	 I was debating... need to review that wiki.
[17:05:10] <milimetric>	 I'm gonna hang out and read it, but eat lunch at the same time
[17:05:19] <milimetric>	 I'll be in the cave
[17:06:01] <mforns>	 ok, will grab sth to eat, to be solidary :]
[17:08:37] <elukey>	 ottomata: ack!
[17:09:06] <wikibugs>	 06Analytics-Kanban: Debian package for ua parser latest version - https://phabricator.wikimedia.org/T156821#2987057 (10Nuria) a:03elukey
[17:09:32] <ottomata>	 elukey:  I should be able to do ^ if you are busy
[17:11:58] <ottomata>	 gotta make a grocery lunch run, back shortly...
[17:16:52] <elukey>	 ottomata|afk: sure! but let me know how you do it since I am curious :)
[17:20:21] <milimetric>	 fdans: we're hanging out looking at the design consultation btw, if you wanna join
[17:20:43] <milimetric>	 ashgrigas: you're obviously also welcome, we're in the batcave eating lunch and thinking about the page
[17:21:17] <milimetric>	 (but for now we're just trying to come up with feedback for you)
[17:35:31] <wikibugs>	 (03CR) 10Nuria: (in progress) Store anonymized and purge sensitive data for WDQS (035 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/335211 (https://phabricator.wikimedia.org/T146915) (owner: 10Nschaaf)
[17:53:52] <ashgrigas>	 milimetric sorry just stepped out too to get lunch
[18:00:24] <milimetric>	 no prob, we're still here :)
[18:01:02] <milimetric>	 ashgrigas / nuria: want to join us in the batcave?
[18:01:09] <nuria>	 omw
[18:01:49] <wikibugs>	 (03CR) 10AndyRussG: "The added field looks good!! :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/331794 (https://phabricator.wikimedia.org/T155141) (owner: 10Mforns)
[18:02:01] <mforns>	 thx AndyRussG!
[18:02:17] <AndyRussG>	 mforns: igualmente :)
[18:05:43] <wikibugs>	 06Analytics-Kanban: Debian package for ua parser latest version - https://phabricator.wikimedia.org/T156821#2987237 (10Ottomata) a:05elukey>03Ottomata
[18:07:58] * elukey going afk!
[18:15:17] <joal>	 mforns: about banners data, do you want me to merge, or later?
[18:15:30] <mforns>	 joal, I'd like to test the monthly job
[18:15:39] <mforns>	 will do that today :]
[18:15:59] <joal>	 k
[18:16:32] <joal>	 please drop me an email after, so that I could merge tomorrow before Luca deploys
[18:17:19] <joal>	 mforns: --^
[18:17:38] <mforns>	 joal, OK, depends on the vetting of the data...
[18:17:50] <mforns>	 mmmm
[18:17:56] <joal>	 ok mforns, nevermind, we'll deploy without :)
[18:18:29] <mforns>	 joal, maybe this time it is faster (without huge queries in hadoop, no?)
[18:18:42] <mforns>	 so, maybe it will be ready tomorrow first hour for me to vet
[18:19:00] <joal>	 mforns: hm, if you're regenrating data it'll be long
[18:19:48] <mforns>	 joal, yes, but last time it took 16 hours, maybe today it takes 10, and I can look at the data tomorrow first hour (europe morning)
[18:19:53] <joal>	 Also mforns, quick fact on data transfers from stat1002 dump mount point: I copied 2T of xml dumps files the other day - It took about 2 days
[18:20:02] <joal>	 don't bother mforns :)
[18:20:14] <joal>	 And, why 10 hours this time if 16 last time?
[18:20:24] <mforns>	 joal, OK, after a quick calculation, I think pagecounts-raw data is ~8TB
[18:20:36] <joal>	 mforns: we'll deploy next week, so it'll come at that point :)
[18:20:48] <mforns>	 joal, 10h because no huge hive queries in the background
[18:21:17] <joal>	 mforns: Oh, yes, it was while there was the huge hive things :) I recall
[18:21:37] <joal>	 Maybe :) But don't bother waking up early especially for that :)
[18:21:46] <joal>	 As said, it'll wait next week without problems
[18:22:45] <mforns>	 joal, OK I'll let you know early enough if the data is finished and vetted so that the patch can be merged
[18:23:12] <joal>	 awesome mforns, I'm sorry to put you under pressure :( It was not my initial idea :(
[18:23:35] <mforns>	 joal, no no xD no problem :]
[18:24:46] <joal>	 By the way a-team, I really thing it could be usefull for one of us to be part of the cross-org anti-harassment group (see wmf-all email)
[18:25:05] <joal>	 Anyone interested? or should I do it?
[18:25:32] <mforns>	 joal, Nuria, Dan, Francisco and I are in a meeting with Ash, so I guess they will respond later ;]
[18:25:49] <nuria>	 joal: you go ahead
[18:26:00] <nuria>	 please
[18:33:40] <joal>	 mforns: about moving data physically, if milimetric or ottomata think it;s easier, might be considered :)
[18:33:56] <mforns>	 joal, I see
[18:34:09] <ottomata>	 physically?
[18:34:16] <joal>	 moving a hard drive?
[18:36:07] <fdans>	 joal: if there's a lot on your plate, I wouldn't mind doing it! (re: anti-harassment group)
[18:36:27] <joal>	 fdans: I think there is space for both of us :)
[18:36:38] <joal>	 fdans: please join, I'll be the cover
[18:37:56] <fdans>	 joal: 👌🏼
[18:38:22] <joal>	 fdans: emoji :(
[18:38:54] <joal>	 Yes ! Found it on the internetz fdans :)
[18:44:16] <ottomata>	 joal:  what hd?
[18:47:33] <ottomata>	 YES!!!!
[18:47:34] <ottomata>	 http://codepen.io/ottomata/pen/VKNyEw?editors=0011
[18:51:56] <joal>	 Wow ottomata, that is great :)
[18:51:59] * joal love that !
[18:52:20] <joal>	 ottomata: Was thinking of moving the 8T of pagecounts-raw to HDFS
[18:52:22] <ottomata>	 credit goes to Krinkle for that, i just forked his thing and made it use EventStreams
[18:52:35] <ottomata>	 joal:  ya
[18:52:45] <ottomata>	 but, to do it in HDFS properly, we don't want to move just a HD
[18:52:54] <ottomata>	 we want it replicated throught hdfs in blocks, right?
[18:53:12] <joal>	 ottomata: Correct, but we could prevent having to go through nfs etc maybe?
[18:53:48] <joal>	 ottomata: like plugging a new drive to an anlytics machine and move from there ? Maybe it;s not even faster
[18:54:11] <ottomata>	 hm, it would probably be a little faster, but not by much.  the best reason to do that would be to have pity on the dumps machine hardware
[18:54:21] <ottomata>	 we could look into installing an hdfs client on dumps
[18:54:25] <ottomata>	 and just hdfs -put from there
[18:54:31] <ottomata>	 but, if we only have to do this one time
[18:54:37] <ottomata>	 i'd say we just load it from the NFS mount (if we can)
[18:54:42] <ottomata>	 and just wait...
[18:54:44] <joal>	 ottomata: I think we can
[18:55:05] <joal>	 ottomata: I did it for xml dumps (2T), gently, one file at a time, it took time but worked
[18:55:43] <ottomata>	 xmldumps are smaller than historical pagecounts?!
[18:56:02] <ottomata>	 you coudl also fetch them from http...
[18:56:07] <ottomata>	 in a distributed job :)
[18:56:14] <joal>	 ottomata: all projects, history dumps, 1 date
[18:56:14] <ottomata>	 heheh
[18:56:50] <joal>	 ottomata: I think concurrent http downloads are 2 per clients max
[18:57:31] <ottomata>	 aye ok, scratch that then :)
[18:57:39] <joal>	 hehe
[18:57:55] <joal>	 ottomata: single loo over file will do - It'll take 2 weeks
[18:59:14] <ottomata>	 ok :/
[18:59:22] <ottomata>	 it'll be good to have that stuff in there anyway
[18:59:48] <joal>	 correct ottomata
[19:03:26] <wikibugs>	 10Analytics: Add user queue in hadoop to execute (possibly long and resource intensive) low priority jobs - https://phabricator.wikimedia.org/T156841#2987445 (10Nuria)
[19:03:39] <wikibugs>	 10Analytics, 10Analytics-Cluster: Add user queue in hadoop to execute (possibly long and resource intensive) low priority jobs - https://phabricator.wikimedia.org/T156841#2987457 (10Nuria)
[19:03:46] <joal>	 thanks for that nuria --^
[19:03:57] <joal>	 had to do ti and forgot :/
[19:04:36] <wikibugs>	 10Analytics, 10Analytics-Cluster: Add user queue in hadoop to execute (possibly long and resource intensive) low priority jobs - https://phabricator.wikimedia.org/T156841#2987445 (10Nuria) {F5432224}
[19:04:54] <wikibugs>	 10Analytics, 10Analytics-Cluster: Add user queue in hadoop to execute (possibly long and resource intensive) low priority jobs - https://phabricator.wikimedia.org/T156841#2987469 (10Nuria) See screenshot of cluster usage for some user queries run as of recent.
[19:05:16] <mforns>	 ok ottomata and joal, thanks for the thoughts, I will start then with hdfs dfs -put from nfs mount into hdfs
[19:05:59] <nuria>	 msg ottomata "We'd love to make all this better." jajajaja
[19:06:08] <nuria>	 love it
[19:10:47] <joal>	 interesting ottomata: #.h1dska2r7
[19:10:55] <joal>	 oops: https://medium.com/airbnb-engineering/streamalert-real-time-data-analysis-and-alerting-e8619e3e5043
[19:15:49] <milimetric>	 ottomata: cool codepen, nice
[19:18:13] <travis-ci>	 wikimedia/mediawiki-extensions-EventLogging#632 (wmf/1.29.0-wmf.10 - 04e3fe4 : Translation updater bot): The build has errored.
[19:18:13] <travis-ci>	 Change view : https://github.com/wikimedia/mediawiki-extensions-EventLogging/compare/wmf/1.29.0-wmf.10
[19:18:13] <travis-ci>	 Build details : https://travis-ci.org/wikimedia/mediawiki-extensions-EventLogging/builds/197062130
[19:19:11] <mforns>	 joal, launched the monthly job: 0095487-161121120201437-oozie-oozi-C
[19:38:08] <wikibugs>	 10Analytics-EventLogging, 06Analytics-Kanban, 10DBA, 13Patch-For-Review: Add autoincrement id to EventLogging MySQL tables. {oryx} - https://phabricator.wikimedia.org/T125135#2987577 (10Ottomata) > This is key, is that somehow doable from the application side? It would help to reduce this snowflake we have...
[19:39:23] <wikibugs>	 10Analytics, 10DBA, 06Operations: Improve eventlogging replication procedure - https://phabricator.wikimedia.org/T124307#2987580 (10Ottomata)
[19:39:26] <wikibugs>	 10Analytics-EventLogging, 06Analytics-Kanban, 10DBA, 13Patch-For-Review: Add autoincrement id to EventLogging MySQL tables. {oryx} - https://phabricator.wikimedia.org/T125135#2987579 (10Ottomata)
[19:40:42] <wikibugs>	 10Analytics-EventLogging, 06Analytics-Kanban, 10DBA, 13Patch-For-Review: Add autoincrement id to EventLogging MySQL tables. {oryx} - https://phabricator.wikimedia.org/T125135#1979211 (10Ottomata) >> Ok, IF all of the current tables that eventlogging is writing to have auto-increment IDs, then yeah, no acti...
[19:44:31] <wikibugs>	 10Analytics, 10DBA, 06Operations: Improve eventlogging replication procedure - https://phabricator.wikimedia.org/T124307#2987587 (10Ottomata) @Marostegui ok!  So the T125135 auto-increment thing is a very small piece of this larger issue.  Let's see if we can hammer out a way to use regular MySQL replication...
[19:54:57] <icinga-wm>	 PROBLEM - YARN NodeManager Node-State on analytics1039 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds.
[19:55:47] <icinga-wm>	 RECOVERY - YARN NodeManager Node-State on analytics1039 is OK: OK: YARN NodeManager analytics1039.eqiad.wmnet:8041 Node-State: RUNNING
[20:01:17] <wikibugs>	 06Analytics-Kanban: Debian package for ua parser latest version - https://phabricator.wikimedia.org/T156821#2987617 (10Nuria) Newest code for eventloggimg will parse incoming user agent thus ua-praser is needed as part of EL install
[20:01:22] <wikibugs>	 10Analytics, 10DBA, 06Operations: Prep to decommission old dbstore hosts (db1046, db1047) - https://phabricator.wikimedia.org/T156844#2987618 (10Ottomata)
[20:02:20] <wikibugs>	 10Analytics, 10DBA, 06Operations: Prep to decommission old dbstore hosts (db1046, db1047) - https://phabricator.wikimedia.org/T156844#2987634 (10Ottomata)
[20:02:41] <wikibugs>	 10Analytics, 10DBA, 06Operations: Prep to decommission old dbstore hosts (db1046, db1047) - https://phabricator.wikimedia.org/T156844#2987618 (10Ottomata) a:05jcrespo>03None
[20:30:09] <joal>	 Gone for now a-team, tomorrow !
[20:31:07] <ottomata>	 laters!
[20:33:29] <Krinkle>	 ottomata: Cool. I was testingn the stream/recentchange yesterday on beta but couldn't get it to work. It responsed okay but no results ever came from making edits etc.
[20:33:39] <Krinkle>	 the prod one is working now though, that's awesome
[20:40:36] <wikibugs>	 (03CR) 10Nuria: "Let's run oncluster using dry run. Please have in mind this needs companion puppet changes." (032 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/335158 (owner: 10EBernhardson)
[20:47:08] <ottomata>	 hm, Krinkle the other day beta worked for me ok
[20:47:14] <ottomata>	 how were you testing?
[20:48:02] <Krinkle>	 ottomata: Modified version of your codepen fork that showed beta/revision-create
[20:48:09] <Krinkle>	 changed it to show recentchange instead
[20:48:11] <Krinkle>	 also tried curl
[20:48:16] <Krinkle>	 I'll try again
[20:48:17] <Krinkle>	 h/o
[20:49:26] <ottomata>	 Krenair: eventstreams-beta.wmflabs.org/v2/stream/recentchange
[20:49:27] <ottomata>	 ?
[20:49:30] <ottomata>	 oops
[20:49:32] <ottomata>	 sorry, wrong ping
[20:49:34] <ottomata>	 Krinkle: ^
[20:49:40] <Krinkle>	 Yes
[20:49:45] <Krinkle>	 curl https://eventstreams-beta.wmflabs.org/v2/stream/recentchange
[20:50:01] <Krinkle>	 I'm gonan delete https://en.wikipedia.beta.wmflabs.org/wiki/Selenium_Echo_mention_test_0.764725462207031 now 
[20:50:19] <Krinkle>	 And... not yet seeing a message in the curl stream
[20:53:06] <wikibugs>	 (03CR) 10EBernhardson: "Ran on stat1002 as dry run, reports that it will remove partitions up to  2016/11/02 for ApiAction and CirrusSearchRequestSet. This seems " [analytics/refinery] - 10https://gerrit.wikimedia.org/r/335158 (owner: 10EBernhardson)
[20:54:30] <wikibugs>	 (03PS2) 10EBernhardson: Script to drop mediawiki log partitions in HDFS [analytics/refinery] - 10https://gerrit.wikimedia.org/r/335158
[20:56:17] <Krinkle>	 ottomata: Hm.. connection was just closed.
[20:56:22] <Krinkle>	 Maybe I'm doing it wrong
[20:57:55] <ottomata>	 hm
[20:58:00] <ottomata>	 trying too
[21:00:30] <wikibugs>	 10Analytics, 06Editing-Department, 10MediaWiki-Authentication-and-authorization, 07Browser-Support-Firefox, and 2 others: Frequent loss of session data (since around 2016-11-28) - https://phabricator.wikimedia.org/T151770#2987725 (10Aklapper)
[21:00:33] <ottomata>	 hm, i think the labs web proxy is being weird
[21:00:34] <ottomata>	 gonna recreate
[21:01:02] <wikibugs>	 10Analytics, 06Editing-Department, 10MediaWiki-Authentication-and-authorization, 07Browser-Support-Firefox, and 2 others: Frequent loss of session data (since around 2016-11-28) - https://phabricator.wikimedia.org/T151770#2899131 (10Aklapper)
[21:01:41] <Krinkle>	 ottomata: I'm getting ":ok" at some point after a few minutes. About a minute before the socket gets closed 
[21:01:55] <ottomata>	 oh you are?
[21:01:58] <ottomata>	 huh
[21:02:04] <ottomata>	 it works locally inside deployment-prep fine
[21:02:39] <ottomata>	 directly to the host i mean
[21:02:45] <ottomata>	  curl -v http://deployment-sca01.eqiad.wmflabs:8092/v2/stream/recentchange
[21:03:01] <ottomata>	 hmm, Krinkle mabye the https is weird
[21:03:11] <ottomata>	 Krinkle:  btw, i just deleted eventstreams-beta proxy
[21:03:17] <ottomata>	 recreated it as stream-beta.wmflabs.org
[21:03:21] <ottomata>	 to better match prod hostname
[21:03:43] <ottomata>	 yeah
[21:03:49] <ottomata>	 Krinkle:  i think https problem
[21:03:54] <ottomata>	 doing it at http seems fine
[21:04:07] <Krinkle>	 k, will use http
[21:09:40] <Krinkle>	 ottomata: Tried the full wmflabs url, still getting :ok, but no messages
[21:11:39] <wikibugs>	 10Analytics, 10Pageviews-API, 06Reading-analysis: Skewed pageviews for Azerbaijani and Bulgarian Wikipedias, September, October and November 2016 - https://phabricator.wikimedia.org/T153699#2987759 (10Nuria)
[21:11:48] <wikibugs>	 10Analytics, 10Pageviews-API, 06Reading-analysis: Skewed pageviews for Azerbaijani and Bulgarian Wikipedias, September, October and November 2016 - https://phabricator.wikimedia.org/T153699#2887617 (10Nuria) Untagging kanban. Adding as subtask of bot task
[21:12:01] <ottomata>	 hm Krinkle I do
[21:12:20] <ottomata>	 curl http://stream-beta.wmflabs.org/v2/stream/recentchange
[21:12:23] <ottomata>	 then
[21:12:23] <ottomata>	 edit
[21:12:26] <ottomata>	 https://deployment.wikimedia.beta.wmflabs.org/wiki/Test1
[21:12:27] <ottomata>	 and i see a change
[21:12:36] <wikibugs>	 10Analytics, 10Pageviews-API, 06Reading-analysis: Skewed pageviews for Azerbaijani and Bulgarian Wikipedias, September, October and November 2016 - https://phabricator.wikimedia.org/T153699#2987765 (10Nuria)
[21:12:38] <wikibugs>	 10Analytics, 06Research-and-Data-Backlog: Improve bot identification at scale - https://phabricator.wikimedia.org/T138207#2987764 (10Nuria)
[21:12:54] <Krinkle>	 ottomata: Yep, I saw yours just now
[21:13:01] <ottomata>	 cool!
[21:13:02] <Krinkle>	 But I'm not getting my own continuous page creation/page deletion
[21:13:05] <Krinkle>	 been doing dozeens
[21:13:07] <Krinkle>	 none of gthem show up
[21:13:08] <ottomata>	 hmmmm
[21:13:16] <ottomata>	 trying creating
[21:13:18] <wikibugs>	 10Analytics, 10Pageviews-API: Pageview API: Better filtering of bot traffic on top enpoints - https://phabricator.wikimedia.org/T123442#2987771 (10Nuria)
[21:13:21] <wikibugs>	 10Analytics, 06Research-and-Data-Backlog: Improve bot identification at scale - https://phabricator.wikimedia.org/T138207#2393202 (10Nuria)
[21:13:33] <Krinkle>	 ottomata: It's forwarding all rc events, no filtering, right?
[21:13:34] <ottomata>	 hm!
[21:13:37] <ottomata>	 it should be! yeah
[21:13:41] <Krinkle>	 als no other log events
[21:13:51] <ottomata>	 do you seem them on the prod endpoing?
[21:14:40] <Krinkle>	 ottomata: https://gist.github.com/Krinkle/c50d5a8f171e8dd62dec55b9cc332c80
[21:14:42] <Krinkle>	 ottomata: will try
[21:15:00] <Krinkle>	 ottomata: Is there a filter parameter by wiki?
[21:15:08] <ottomata>	 Krinkle:  no
[21:15:11] <Krinkle>	 okay
[21:15:15] <Krinkle>	 I'll use grep then :)
[21:15:25] <ottomata>	 Krinkle:  https://phabricator.wikimedia.org/T152731
[21:15:27] <ottomata>	 tbd still
[21:17:01] <wikibugs>	 (03CR) 10Nuria: [V: 031 C: 031] "Looks good to me, waiting for otto as there might be gotchas (other than companion change in puppet) that i am not aware of" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/335158 (owner: 10EBernhardson)
[21:17:44] <Krinkle>	 ottomata: I'd say the minimal filter is that for a wiki. If anything, to allow for reducing bandwidth and processing. It's a lot.
[21:18:08] <Krinkle>	 but maybe it's not that much?
[21:18:26] <Krinkle>	 I'm imagining mobile devices using the stream to show recent changes and micro contributions for patrolling etc.
[21:18:49] <ottomata>	 if you want anything but enwiki or wikidata, its probably worth it for folks
[21:18:53] <Krinkle>	 This isn't just  for server-to-server. Various gadgets poll the API and/or proxy irc, and/or use rcstream now.
[21:19:07] <ottomata>	 eeee that could be cool, brandon was pretty clear that this shouldn't be used for client side features
[21:19:40] <ottomata>	 at least not ones that we deploy widely.  tools and stuff, cool.
[21:20:00] <ottomata>	 he doesn't want to open direct pipes to backend servers through the caches for lots and lots of clients
[21:20:04] <Krinkle>	 ottomata: Eh, I'm confused.
[21:21:47] <Krinkle>	 Well, it's unlikely a reader would be interseted in these tools. It's gonna be a subset of editors.
[21:21:58] <Krinkle>	 But if we can't use this client-side, then I have to cancel 80% of the expected uses.
[21:22:04] <Krinkle>	 That is highly unexpected.
[21:22:07] <ottomata>	 Krinkle:  about?  client side stuff?  i think we have to take that up with brandon and other ops people.  i think he just wants to limit the number of stream connections
[21:22:20] <Krinkle>	 how is this different from rcstream
[21:22:34] <ottomata>	 i think if the usage is about the same as rcstream, it'll be no problem
[21:22:50] <ottomata>	 he just doesn't want every browser with a wikipedia page open to open a stream connection
[21:23:22] <Krinkle>	 Well, if we expect to survive as an organisation, we need to focus more on edit review. That means a 10-200X increase in reviewers is neccecary. This was a strategic goal for 5 years unfortutely mostly left unattended. That's changing now, finally.
[21:23:51] <Krinkle>	 Sure, it's an engaging feature for editors/reviewers.
[21:24:04] <Krinkle>	 We already bypass the varnish for all of the thousands of reviewers eveyr day
[21:24:12] <Krinkle>	 mostly by polling hte API with uncacheable queries every 5 seconds
[21:25:18] <Krinkle>	 It requiring to be connectable from a client was one of the foundational design goals, and why it had to be WebSockets or SSE
[21:25:22] <ottomata>	 halfak:  has a lot of opinions on this.  we had a meeting last week with review stream folks.  not sure if you saw this: https://phabricator.wikimedia.org/T143743#2966929
[21:25:34] <halfak>	 o/
[21:25:36] <ottomata>	 ok, lemme rephrase
[21:25:46] <Krinkle>	 Yeah, I know and agree with halfak that streams isn't a design requirement for edit review.
[21:25:55] <Krinkle>	 It can do polling instead to get MVP out sooner.
[21:26:03] <halfak>	 Am knee deep in a discussion about labsDB.  Will be back shortly to read scrollback and participate. 
[21:26:06] <Krinkle>	 But it is a design requirement for the stream itself.
[21:26:18] <Krinkle>	 We've been waiting for this for years.
[21:26:20] <ottomata>	 i should have said: brandon was pretty clear that this shouldn't be used for widely deployed readership based client side features
[21:26:30] <Krinkle>	 Sure, that's fine :)
[21:26:52] <ottomata>	 he really just doesn't want to start streaming everything everywhere
[21:27:05] <Krinkle>	 and once our review base grow further, thigns will scale one way or another and we'll deal with that *success* at the time.
[21:27:16] <ottomata>	 if there are specific tools that say an editor has to log in, or choose to use explicitly (like, huggle, whatever), then i think he's fine with t hat
[21:27:27] <ottomata>	 i didn't mean 'client side feature' as a blanket "can't use this in a browser"
[21:27:34] <ottomata>	 heheh yeah
[21:28:25] <Krinkle>	 I don't think it's desirable or realistic to require a hard check that the user is logged-in but it's a fairly sure given that this would never reachable for a reader. Other than techn savvy readers that follow our development and open up demos like yours or mine.
[21:28:45] <Krinkle>	 they're all gadgets and tools.
[21:28:50] <Krinkle>	 and unpopular ones at that
[21:28:51] <ottomata>	 aye
[21:28:59] <Krinkle>	 Anyhow :)
[21:33:46] <Krinkle>	 ottomata: Not getting page creation/deletion in prod eithier
[21:33:52] <Krinkle>	 just tried on mediawiki.org
[21:33:56] <Krinkle>	 did get the edit though
[21:34:30] <ottomata>	 o!
[21:34:33] <ottomata>	 ok that's a problem
[21:35:07] <wikibugs>	 06Analytics-Kanban, 10EventBus, 10Wikimedia-Stream, 13Patch-For-Review, and 2 others: RecentChanges in Kafka - https://phabricator.wikimedia.org/T152030#2987811 (10Ottomata) @Krinkle reports that page creation and deletion isn't making it through.  Need to investigate...
[21:36:45] <Krinkle>	 ottomata: thanks
[21:49:54] <wikibugs>	 06Analytics-Kanban, 10EventBus, 10Wikimedia-Stream, 13Patch-For-Review, and 2 others: RecentChanges in Kafka - https://phabricator.wikimedia.org/T152030#2836021 (10Pchelolo) There's also a bunch of 400 errors in the evenlogging-service logs, mostly something like ``` (MainThread) Failed processing event: F...
[21:52:11] <wikibugs>	 06Analytics-Kanban, 10EventBus, 10Wikimedia-Stream, 13Patch-For-Review, and 2 others: RecentChanges in Kafka - https://phabricator.wikimedia.org/T152030#2987929 (10Ottomata) Makes sense, likely the schema we made isn't comprehensive enough.  Will have a few minutes soon..
[22:02:57] <wikibugs>	 10Analytics, 06Discovery, 06Discovery-Analysis (Current work), 03Interactive-Sprint, 13Patch-For-Review: Add Maps tile usage counts as a Data Cube in Pivot - https://phabricator.wikimedia.org/T151832#2987971 (10mpopov) >>! In T151832#2982365, @Nuria wrote: > Moving to "radar", do ping us if you need more...
[22:03:56] <ottomata>	 Krinkle:  still there?
[22:04:28] <Krinkle>	 ottomata: I am
[22:04:39] <ottomata>	 log_params
[22:04:43] <ottomata>	 should be an array or object?
[22:05:00] <Krinkle>	 ottomata: Probably either
[22:05:12] <Krinkle>	 Given that they are arrays in PHP, but in JSON may become either
[22:05:16] <Krinkle>	 it can also be a string
[22:05:21] <Krinkle>	 for certain older log events
[22:05:25] <Krinkle>	 separated by \n
[22:05:46] <ottomata>	 hmmmm
[22:05:49] <ottomata>	 nasty, ok
[22:05:50] <ottomata>	 :)
[22:05:58] <Krinkle>	 it's documented as mixed
[22:11:05] <ottomata>	 Krinkle: Pchelolo: https://gerrit.wikimedia.org/r/#/c/335365/
[22:11:41] <ottomata>	 hm, guessi can't do it that way...
[22:12:51] <Krinkle>	 ottomata: From a quick look at RecentChange.php, rc_log_type can also be null 
[22:13:11] <Krinkle>	 since they are optional, but some of the "optional" fields are created by default as null 
[22:13:14] <Krinkle>	 to reflect the DB
[22:13:29] <Krinkle>	 the optional fields that are computed are typically omitted if not applicable though
[22:13:37] <Krinkle>	 https://github.com/wikimedia/mediawiki/blob/master/includes/changes/RecentChange.php
[22:14:21] <Krinkle>	 and https://github.com/wikimedia/mediawiki/blob/master/includes/rcfeed/MachineReadableRCFeedFormatter.php
[22:14:30] <ottomata>	 ok
[22:14:41] <Krinkle>	 That should cover all the default fields
[22:14:49] <Krinkle>	 additionalProperties:true covers the rest
[22:15:09] <ottomata>	 updated changeset
[22:15:17] <ottomata>	 should pass schema test now
[22:15:19] <ottomata>	 had to quote "null"
[22:15:21] <Krinkle>	 looks like the string log_params becomes an array
[22:17:27] <Krinkle>	 ottomata: log_action_commnet is also string|null
[22:17:36] <Krinkle>	 comment*
[22:18:00] <ottomata>	 ok
[22:18:18] <ottomata>	 updated.
[22:21:34] <wikibugs>	 06Analytics-Kanban, 13Patch-For-Review: Clean up datasets.wikimedia.org - https://phabricator.wikimedia.org/T125854#2988063 (10Ottomata) Yeehaw!  Ok, so, that cron thing should be fixed.  I reverted some datasets.wm.org stuff back to how it was before.  Also!  {/a,/srv}/published-datasets now exists on stat100...
[22:21:49] <wikibugs>	 06Analytics-Kanban, 13Patch-For-Review: Move datasets.wikimedia.org to analytics.wikimedia.org/datasets - https://phabricator.wikimedia.org/T132594#2988066 (10Ottomata)
[22:21:52] <wikibugs>	 06Analytics-Kanban, 13Patch-For-Review: Clean up datasets.wikimedia.org - https://phabricator.wikimedia.org/T125854#2988069 (10Ottomata)
[22:22:09] <wikibugs>	 06Analytics-Kanban, 13Patch-For-Review: Clean up datasets.wikimedia.org - https://phabricator.wikimedia.org/T125854#1998898 (10Ottomata) I've merged T132594 as a duplicate, since really this whole cleanup now involves both sites.
[22:22:17] <ottomata>	 milimetric: https://phabricator.wikimedia.org/T125854#2988063
[22:23:01] <ottomata>	 Krinkle:  merging schema change
[22:24:58] <ottomata>	 looks like id needs to be nullable too
[22:27:31] <ottomata>	 looking better
[22:30:32] <wikibugs>	 06Analytics-Kanban, 13Patch-For-Review: Clean up datasets.wikimedia.org - https://phabricator.wikimedia.org/T125854#2988107 (10Milimetric) Ok, have a few other priorities first, but will get back to this on Friday.  Will first move all our reportupdater reports and update their configured location on meta.  Th...