[01:54:39] <wikibugs>	 10Analytics, 10Analytics-Wikimetrics, 10Patch-For-Review, 10WorkType-Maintenance: flake8 errors on wikimetrics - https://phabricator.wikimedia.org/T210320 (10Milimetric) @rafidaslam, I'll leave this up to you, do whatever you like: ignore 504, 503, or even ignore both.  It's a matter of style that I don't...
[01:55:57] <wikibugs>	 (03CR) 10Milimetric: "generally, I'd +2 any approach that fixes the tox checks, whether it be fixing 503 and ignoring 504 or even just ignoring both.  I'll chec" [analytics/wikimetrics] - 10https://gerrit.wikimedia.org/r/475591 (https://phabricator.wikimedia.org/T210320) (owner: 10Rafidaslam)
[02:07:30] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10MW-1.33-notes (1.33.0-wmf.6; 2018-11-27), 10Patch-For-Review: Deprecation Information for EventLogging ResourceLoader modules - https://phabricator.wikimedia.org/T205744 (10Milimetric)
[02:17:05] <wikibugs>	 10Analytics, 10Analytics-Dashiki, 10Analytics-Kanban: Dashiki should filter out empty newlines - https://phabricator.wikimedia.org/T210570 (10Milimetric)
[02:49:00] <wikibugs>	 (03PS1) 10Milimetric: Filter out empty lines when parsing text data [analytics/dashiki] - 10https://gerrit.wikimedia.org/r/476195 (https://phabricator.wikimedia.org/T210570)
[02:49:17] <wikibugs>	 (03CR) 10Milimetric: [V: 032 C: 032] Filter out empty lines when parsing text data [analytics/dashiki] - 10https://gerrit.wikimedia.org/r/476195 (https://phabricator.wikimedia.org/T210570) (owner: 10Milimetric)
[04:19:35] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Patch-For-Review: Resurrect eventlogging_EventError logging to in logstash - https://phabricator.wikimedia.org/T205437 (10Tgr) Awesome, thank you for working on this!  Is there some kind of rate limiting? I don't think Logstash could deal with one or more errors per p...
[06:33:30] <wikibugs>	 10Analytics, 10Dumps-Generation, 10ORES, 10Scoring-platform-team, and 3 others: [Epic] Make ORES scores available in Hadoop and as a dump - https://phabricator.wikimedia.org/T209611 (10Ladsgroup) p:05Triage>03High
[06:33:46] <wikibugs>	 10Analytics, 10Dumps-Generation, 10ORES, 10Scoring-platform-team, and 3 others: [Epic] Make ORES scores available in Hadoop and as a dump - https://phabricator.wikimedia.org/T209611 (10Ladsgroup) p:05High>03Triage
[06:34:10] <wikibugs>	 10Analytics, 10Dumps-Generation, 10ORES, 10Scoring-platform-team, and 3 others: [Epic] Make ORES scores available in Hadoop and as a dump - https://phabricator.wikimedia.org/T209611 (10Ladsgroup) p:05Triage>03High
[06:35:23] <wikibugs>	 10Analytics, 10ORES, 10Scoring-platform-team: Include feature values in ORES changeprop stream - https://phabricator.wikimedia.org/T209734 (10Ladsgroup) p:05Triage>03Normal
[06:35:29] <wikibugs>	 10Analytics, 10ORES, 10Scoring-platform-team: Backfill ORES Hadoop scores with historical data - https://phabricator.wikimedia.org/T209737 (10Ladsgroup) p:05Triage>03Normal
[06:35:38] <wikibugs>	 10Analytics, 10Dumps-Generation, 10ORES, 10Scoring-platform-team: Produce dump files for ORES scores - https://phabricator.wikimedia.org/T209739 (10Ladsgroup) p:05Triage>03High
[06:35:45] <wikibugs>	 10Analytics, 10ORES, 10Scoring-platform-team: Purge ORES scores from Hadoop and begin backfill when model version changes - https://phabricator.wikimedia.org/T209742 (10Ladsgroup) p:05Triage>03Normal
[06:36:16] <wikibugs>	 10Analytics, 10Dumps-Generation, 10ORES, 10Scoring-platform-team: Produce dump files for ORES scores - https://phabricator.wikimedia.org/T209739 (10Ladsgroup) p:05High>03Triage
[06:36:20] <wikibugs>	 10Analytics, 10ORES, 10Scoring-platform-team: Include feature values in ORES changeprop stream - https://phabricator.wikimedia.org/T209734 (10Ladsgroup) p:05Normal>03Triage
[06:36:38] <wikibugs>	 10Analytics, 10ORES, 10Scoring-platform-team: Backfill ORES Hadoop scores with historical data - https://phabricator.wikimedia.org/T209737 (10Ladsgroup) p:05Normal>03Triage
[06:36:43] <wikibugs>	 10Analytics, 10ORES, 10Scoring-platform-team: Purge ORES scores from Hadoop and begin backfill when model version changes - https://phabricator.wikimedia.org/T209742 (10Ladsgroup) p:05Normal>03Triage
[06:37:16] <wikibugs>	 (03PS2) 10Rafidaslam: tox.ini: Ignore flake8 W504 rule [analytics/wikimetrics] - 10https://gerrit.wikimedia.org/r/475591 (https://phabricator.wikimedia.org/T210320)
[06:38:33] <wikibugs>	 10Analytics, 10Analytics-Wikimetrics, 10Patch-For-Review, 10WorkType-Maintenance: flake8 errors on wikimetrics - https://phabricator.wikimedia.org/T210320 (10rafidaslam) Alright, I'll choose to ignore `W504` as `wikimetrics`'s codebase already used "line break after binary operator" style.
[08:36:42] <fdans>	 morniiing joal, let's merge this so I can deploy? https://gerrit.wikimedia.org/r/#/c/analytics/aqs/+/476033/
[08:51:29] <elukey>	 I am here if needed :)
[09:32:43] <wikibugs>	 (03PS1) 10Fdans: Add project families to uniques loading job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/476220 (https://phabricator.wikimedia.org/T167539)
[09:50:00] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Patch-For-Review: Resurrect eventlogging_EventError logging to in logstash - https://phabricator.wikimedia.org/T205437 (10phuedx) >>! In T205437#4780015, @Tgr wrote: > Is there some kind of rate limiting?  No AFAIK though the Logstash Kafka input will only consume mes...
[11:35:05] <aharoni>	 hi mforns_ 
[11:35:27] <aharoni>	 at https://analytics.wikimedia.org/datasets/periodic/reports/metrics/cx/ I see the file with the useless data
[11:35:37] <aharoni>	 What needs to be done to get it fixed?
[11:40:04] * elukey lunch!
[12:37:18] <aharoni>	 mforns_: around?
[12:48:21] <wikibugs>	 (03CR) 10Lucas Werkmeister (WMDE): Update metric's items and properties automatically (032 comments) [analytics/wmde/toolkit-analyzer] - 10https://gerrit.wikimedia.org/r/475807 (https://phabricator.wikimedia.org/T209399) (owner: 10Michael Große)
[14:17:29] <wikibugs>	 (03CR) 10Milimetric: [C: 032] tox.ini: Ignore flake8 W504 rule [analytics/wikimetrics] - 10https://gerrit.wikimedia.org/r/475591 (https://phabricator.wikimedia.org/T210320) (owner: 10Rafidaslam)
[14:18:47] <wikibugs>	 10Analytics, 10Analytics-Wikimetrics, 10Patch-For-Review, 10WorkType-Maintenance: flake8 errors on wikimetrics - https://phabricator.wikimedia.org/T210320 (10rafidaslam) 05Open>03Resolved Thanks! I think this can be marked as resolved now
[14:19:09] <travis-ci>	 wikimedia/analytics-wikimetrics#3 (master - 5b3caf3 : Rafid Aslam): The build has errored.
[14:19:09] <travis-ci>	 Change view : https://github.com/wikimedia/analytics-wikimetrics/compare/3fb8810a6a5a...5b3caf385604
[14:19:09] <travis-ci>	 Build details : https://travis-ci.com/wikimedia/analytics-wikimetrics/builds/92854810
[14:19:52] <wikibugs>	 (03CR) 10Milimetric: "I'll rebase this now and see if that flask upgrade causes problems." [analytics/wikimetrics] - 10https://gerrit.wikimedia.org/r/475142 (https://phabricator.wikimedia.org/T209945) (owner: 10MarcoAurelio)
[14:19:58] <wikibugs>	 (03PS3) 10Milimetric: Bump dependencies in requirements.txt to their latest versions [analytics/wikimetrics] - 10https://gerrit.wikimedia.org/r/475142 (https://phabricator.wikimedia.org/T209945) (owner: 10MarcoAurelio)
[14:29:27] <wikibugs>	 10Analytics, 10EventBus, 10Wikimedia-production-error: EventBus::createEvent Call to a member function format() on a non-object (boolean) - https://phabricator.wikimedia.org/T210608 (10hashar)
[14:29:50] <wikibugs>	 10Analytics, 10EventBus, 10Wikimedia-production-error: EventBus::createEvent Call to a member function format() on a non-object (boolean) - https://phabricator.wikimedia.org/T210608 (10hashar)
[14:38:13] * elukey just discovered the hive server 2 ui - ssh -L 10002:an-coord1001.eqiad.wmnet:10002 an-coord1001.eqiad.wmnet
[14:38:36] <fdans>	 milimetric: wanna talk 2 min?
[14:39:09] <milimetric>	 omw
[14:39:10] <milimetric>	 fdans: 
[14:40:38] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Patch-For-Review: Resurrect eventlogging_EventError logging to in logstash - https://phabricator.wikimedia.org/T205437 (10Ottomata) > Would this transport method be suitable for JS error logging as well?   > Yes. If you can get JS errors into a Kafka topic (presumably...
[14:46:09] <elukey>	 ottomata: o/ - shall we move the ops sync half an hour earlier? it clashes with another meeting :)
[14:49:49] <ottomata>	 elukey:  anytime!
[14:50:03] <ottomata>	 now is good too!
[14:56:37] <fdans>	 milimetric: mind taking a quick look at this so that I can deploy? https://gerrit.wikimedia.org/r/#/c/analytics/aqs/+/476033/
[14:57:41] <wikibugs>	 (03CR) 10Milimetric: [V: 032 C: 032] Expose offset and underestimate numbers in uniques [analytics/aqs] - 10https://gerrit.wikimedia.org/r/476033 (https://phabricator.wikimedia.org/T164201) (owner: 10Fdans)
[14:58:03] <elukey>	 ottomata: in half an hour is ok as well?
[14:58:12] <elukey>	 I am fighting with Hive
[14:58:24] <elukey>	 it looks like I am going to loose badly but I'd like to try once more
[14:58:34] <ottomata>	 sure
[14:58:43] <ottomata>	 we all fight battles! YAR!
[14:58:47] <ottomata>	 lemme know if i can help
[15:03:34] <mforns>	 hi teammm :]
[15:07:36] <ottomata>	 o/
[15:14:33] <milimetric>	 fdans: have you noticed that the tests hang now?  https://travis-ci.org/wikimedia/analytics-aqs/jobs/460813428 if you look at the details, the job fails because it times out after all tests pass
[15:14:45] <milimetric>	 I'm spinning too many things, mind taking a look?
[15:15:11] <wikibugs>	 (03PS2) 10Ottomata: [WIP] HiveExtensions StructType .merge supports Arrays and Maps with complex types [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476179 (https://phabricator.wikimedia.org/T210465)
[15:15:17] <fdans>	 milimetric: I already got a patch coming for that, as part of getting CI to AQS
[15:15:22] <wikibugs>	 (03PS3) 10Ottomata: HiveExtensions StructType .merge supports Arrays and Maps with complex types [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476179 (https://phabricator.wikimedia.org/T210465)
[15:15:30] <fdans>	 tests have been hanging for a while now on AQS
[15:23:13] <elukey>	 ottomata: if you have time we can do it now
[15:23:16] <elukey>	 in bc
[15:24:10] <wikibugs>	 (03CR) 10Milimetric: "hm, lots of errors on tests and some upgrade incompatibilities.  I'm working through fixing them as I'm already in the middle of it.  I'll" [analytics/wikimetrics] - 10https://gerrit.wikimedia.org/r/475142 (https://phabricator.wikimedia.org/T209945) (owner: 10MarcoAurelio)
[15:26:28] <ottomata>	 elukey:  ok!
[15:26:59] <ottomata>	 actually give me 4 mins to finish what i'm looking at...
[15:43:24] <wikibugs>	 (03PS1) 10Mforns: Upgrade profig lib to 2.3.3 after bug fix upstream [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476290 (https://phabricator.wikimedia.org/T202429)
[15:44:37] <mforns>	 ottomata, I created https://gerrit.wikimedia.org/r/#/c/analytics/refinery/source/+/476290/ is there anything else to be done, like loading the lib to archiva or sth?
[15:46:10] <wikibugs>	 (03CR) 10jerkins-bot: [V: 04-1] Upgrade profig lib to 2.3.3 after bug fix upstream [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476290 (https://phabricator.wikimedia.org/T202429) (owner: 10Mforns)
[15:46:21] <mforns>	 uoo
[15:52:07] <wikibugs>	 (03CR) 10Milimetric: [C: 04-1] "Unfortunately it seems both of these upgrades introduce lots of errors that I couldn't solve with just simple updates to the new syntax.  " [analytics/wikimetrics] - 10https://gerrit.wikimedia.org/r/475142 (https://phabricator.wikimedia.org/T209945) (owner: 10MarcoAurelio)
[15:54:20] <wikibugs>	 (03PS1) 10Fdans: Add offset and underestimate to uniques v1 spec [analytics/aqs] - 10https://gerrit.wikimedia.org/r/476297 (https://phabricator.wikimedia.org/T167539)
[15:54:37] <fdans>	 milimetric:  sorryyyy one more
[15:54:47] <fdans>	 ---^
[15:55:30] <wikibugs>	 (03CR) 10Milimetric: [V: 032 C: 032] Add offset and underestimate to uniques v1 spec [analytics/aqs] - 10https://gerrit.wikimedia.org/r/476297 (https://phabricator.wikimedia.org/T167539) (owner: 10Fdans)
[15:55:31] <nuria>	 fdans: hola! 
[15:55:48] <nuria>	 fdans: let's fix test issues before commiting any new code to aqs right? 
[15:55:51] <nuria>	 cc milimetric 
[15:55:55] <milimetric>	 fdans: make sure that's all tested, I'm just checking the code makes sense
[15:56:03] <milimetric>	 test issues?
[15:56:07] <milimetric>	 oh, the timeouts
[15:56:11] <milimetric>	 yeah, that'd be good to do
[15:56:28] <fdans>	 nuria milimetric there's no test issues, all test pass, but I'm fixing that in a separate change
[15:57:05] <fdans>	 the test command doesn't exit because the service never gets killed, but there's not test errors
[15:57:11] <nuria>	 fdans: right, i understand  but is no les simportant to have tess passing than to deploy that code
[15:57:26] <nuria>	 fdans: right, what we talked about before
[15:58:00] <wikibugs>	 10Analytics, 10EventBus, 10Wikimedia-production-error: EventBus::createEvent Call to a member function format() on a non-object (boolean) - https://phabricator.wikimedia.org/T210608 (10zeljkofilipin) p:05Triage>03Unbreak! Train blockers are UBN.
[15:58:00] <fdans>	 ok will post change to fix tests
[15:58:02] <nuria>	 fdans: let's fix that cause such an issue will get in the way if test were to fail due to a bad patch, we might not notice
[15:58:48] <nuria>	 fdans: will be on meetings for the next hour, we can talk after that
[15:59:18] <fdans>	 nuria: hmm, I don't competely agree, but i'll submit the patch to fix tests
[16:00:49] <fdans>	 btw a-team I got an eye  doctor appointment now, so I won't be in standup but I'll be here later
[16:23:12] <ottomata>	 joal:  i had an idea last night
[16:23:31] <ottomata>	 why not use kafka connect for revision-score? if it works?
[16:23:32] <ottomata>	 trying it...
[16:23:54] <ottomata>	 mforns:  i think that should be it
[16:24:36] <ottomata>	 mforns:  it looks like there are compile errors though
[16:24:46] <ottomata>	 looks like some things have changed upstream maybe?
[16:35:06] <chasemp>	 milimetric: I saw a ping in another channel you may want (summary tables used on labsdb but not puppetized) https://phabricator.wikimedia.org/T59617
[16:35:09] <chasemp>	 just a heads up :)
[16:35:21] <chasemp>	 milimetric: https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/375349/
[16:36:06] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Patch-For-Review: Resurrect eventlogging_EventError logging to in logstash - https://phabricator.wikimedia.org/T205437 (10phuedx) >>! In T205437#4780562, @phuedx wrote: > Neither of [[ https://kafka.apache.org/documentation/#design_quotas | Kafka's quota mechanisms ]]...
[16:40:06] <milimetric>	 chasemp: but there's no real activity here since 2017, and it makes me sad because we could generate this kind of summary table very easily and expose it, that's basically what we do all day every day
[16:40:28] <milimetric>	 ok, going to think more on this as we work together on the pipeline
[16:40:50] <milimetric>	 clearly I think it needs some new tooling, custom invented here scripts seem to make everyone very very sad
[16:59:09] <chasemp>	 milimetric: yep, I thiink the synergies here are pretty exciting man
[16:59:52] <milimetric>	 indeed
[17:00:20] * elukey is scared by security people popping up in the channel
[17:00:29] * elukey especially Chase
[17:00:32] <elukey>	 :P
[17:00:54] <nuria>	 ping elukey  standup
[17:48:55] <wikibugs>	 10Analytics, 10EventBus, 10Core Platform Team Kanban (Doing), 10Patch-For-Review, and 2 others: EventBus::createEvent Call to a member function format() on a non-object (boolean) - https://phabricator.wikimedia.org/T210608 (10mobrovac) a:03mobrovac
[17:51:29] <wikibugs>	 10Analytics, 10Operations, 10ops-eqiad, 10User-Elukey: rack/setup/install dbstore100[3-5].eqiad.wmnet - https://phabricator.wikimedia.org/T209620 (10RobH)
[17:52:03] <wikibugs>	 10Analytics, 10Operations, 10User-Elukey: rack/setup/install dbstore100[3-5].eqiad.wmnet - https://phabricator.wikimedia.org/T209620 (10RobH) a:05Cmjohnson>03elukey These are now ready for @elukey to take over.  Assigning this task to him.  You can resolve this as you see fit!
[17:52:33] <wikibugs>	 10Analytics, 10Operations, 10User-Elukey: rack/setup/install dbstore100[3-5].eqiad.wmnet - https://phabricator.wikimedia.org/T209620 (10elukey) 05Open>03Resolved Thanks a lot! We are going to follow up in https://phabricator.wikimedia.org/T210478
[17:52:47] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10DBA: Migrate dbstore1002 to a multi instance setup on dbstore100[3-5] - https://phabricator.wikimedia.org/T210478 (10elukey)
[18:04:24] <wikibugs>	 (03CR) 10Milimetric: [C: 04-1] "I did some more digging, and there are just all kinds of problems with this codebase now, that it hasn't been looked after for so long.  T" [analytics/wikimetrics] - 10https://gerrit.wikimedia.org/r/475142 (https://phabricator.wikimedia.org/T209945) (owner: 10MarcoAurelio)
[18:09:10] <wikibugs>	 10Analytics, 10Patch-For-Review: Refinery Spark HiveExtensions schema merge should support merging of arrays with struct elements - https://phabricator.wikimedia.org/T210465 (10Ottomata) @Pchelolo, we discussed this more in standup today, and we thought juuust a bit more about the scores model.  We think that...
[18:09:59] <ottomata>	 joal:  lemme know what you think of refine code when you get a chance
[18:10:31] <joal>	 sure ottomata - after meeting
[18:13:01] <milimetric>	 joal: I'm gonna take some lunch and continue on the spark job afterwards, hopefully will finish by end of day.
[18:13:18] <joal>	 great milimetric 
[18:32:20] * elukey off!
[18:34:34] <joal>	 ottomata: need to go for diner, will check your code after
[18:36:02] <ottomata>	 k laters!
[18:36:03] <ottomata>	 no worries!
[18:54:11] <mforns>	 ottomata, have you had before this problem with scala implicits colliding?
[18:54:35] <mforns>	 I'm looking at it, and I think one way to solve it should be not import the implicit from profig
[18:55:09] <mforns>	 but it doesn't seem to work
[18:55:48] <ottomata>	 hm, mforns i haven't encountered that before
[18:55:51] <mforns>	 another way is removing our implicit converter... however, ours is better, because it allows for yes, no, 1 and 0
[18:55:57] <ottomata>	 did maybe profig add extra implicits that we also implemented?
[18:56:06] <mforns>	 no, that was the first one
[18:56:11] <ottomata>	 oh is it just the boolean one?
[18:56:12] <mforns>	 only for boolean
[18:56:14] <mforns>	 yea
[18:56:22] <ottomata>	 oh, i think we can remove ours, we don't use yes, no, 1 or 0
[18:56:26] <ottomata>	 anywhere afaik
[18:56:30] <ottomata>	 it was just cool, but not really needed
[18:56:32] <ottomata>	 true/false is fine
[18:56:43] <mforns>	 right, there are some tests that will need modification, but OK
[18:56:49] <mforns>	 if you say so, I will fix!
[19:13:28] <ottomata>	 thanks mforns!
[19:32:29] <nuria>	 joal: added comments to your design document
[19:43:51] <joal>	 Thanks nuria - Will update the doc accordingly
[19:44:15] <nuria>	 joal: we can go over it together if you want, you let me know
[19:44:40] <joal>	 nuria: I'm going to make a first pass over it, then we'll see :)
[19:44:58] <nuria>	 joal: ok
[19:45:22] <joal>	 ottomata: I have a question/comment on the recursive-merge patch
[19:45:33] <joal>	 let me know when you have time
[19:50:35] <wikibugs>	 (03CR) 10Nuria: [C: 04-1] "Let's please discuss how this data will surface in the API ,agent_type splits are not possible on "per family" data." (033 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/476220 (https://phabricator.wikimedia.org/T167539) (owner: 10Fdans)
[19:51:07] <wikibugs>	 (03CR) 10Nuria: [C: 04-1] "Ping on this ..." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/471038 (https://phabricator.wikimedia.org/T208332) (owner: 10Nettrom)
[19:51:18] <ottomata>	 joal:  oh hi
[19:51:22] <ottomata>	 ya ask me plz!
[19:52:12] <joal>	 ottomata: I wonder about the explicit fail in the merge function
[19:53:52] <ottomata>	 the isComplexType one?
[19:54:07] <joal>	 ottomata: right
[19:54:48] <joal>	 ottomata: if we remove that case, we endup in the next one, that return the "original" type - Isn't that the expected behavior?
[19:54:56] <ottomata>	 joal, can you remind me how to resolve Required field 'client_protocol' is unset! Struct:TOpenSessionReq(client_protocol:null, configuration:{use:database=default})) ?
[19:55:02] <ottomata>	 i thought it was just with the proper hive jars...but...
[19:55:26] <ottomata>	 hm
[19:55:59] <joal>	 ottomata: related to hive client/server version discrepency - Can have multiple root causes
[19:56:07] <joal>	 ottomata: The jar you use is not the correct one
[19:56:27] <joal>	 ottomata: There are other jars in the classpath loaded before yours (for spark, can very much be the case)
[19:56:51] <ottomata>	 hm
[19:57:00] <ottomata>	 joal, re expception
[19:57:00] <ottomata>	 hm
[19:57:02] <ottomata>	 you are right
[19:57:19] <ottomata>	 but i wonder if it is better to fail early like this in this case?
[19:57:21] <ottomata>	 not sure
[19:57:52] <joal>	 ottomata: I'm gonna try to cast an array<string> to string, check if it works
[19:57:55] <joal>	 if so, well ....
[19:58:03] <joal>	 If it doesn't we can early fail
[20:00:00] <ottomata>	 AHH yes i see why i'm geting the hive error; yarn vs local mode classpath stuff
[20:00:07] <ottomata>	 joal sonds good
[20:01:09] <joal>	 ottomata: array<int> can be cast to string: [1, 2]
[20:01:21] <joal>	 somehow unexpected, but nice :)
[20:01:33] <ottomata>	 uh
[20:01:35] <ottomata>	 ok then let's remove it
[20:01:47] <joal>	 ottomata: testing maps
[20:02:43] <joal>	 same
[20:02:59] <ottomata>	 oh?
[20:03:08] <ottomata>	 some toString method i guess eh?
[20:06:18] <joal>	 probably ottomata
[20:07:10] <ottomata>	 ok joal, just tested the change, it does what is expected
[20:07:18] <ottomata>	 table gets altered, but still fails because of missing error during cast
[20:07:28] <joal>	 makes sense ottomata 
[20:07:33] <ottomata>	 SQL stuff seems to work just as well too, 
[20:07:39] <ottomata>	 wanted to verify that after using dataType.sql
[20:07:39] <joal>	 ?
[20:07:44] <ottomata>	 instead of my manual backquoting
[20:07:45] <joal>	 Ah yes
[20:07:48] <joal>	 Sonds good
[20:08:07] <ottomata>	 am running on a couple of hours of non revision score stuff
[20:08:10] <ottomata>	 will check on EL data too
[20:09:23] <joal>	 ottomata: except from that little thing, the rest looks good :)
[20:09:39] <joal>	 ottomata: Will +1 and leave a comment about the chat here
[20:10:33] <ottomata>	 k
[20:11:06] <wikibugs>	 (03CR) 10Joal: [C: 031] "LGTM except for the fail-fast of complex/primitive type merge. This has been discussed on IRC and a patch is on its way :)" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476179 (https://phabricator.wikimedia.org/T210465) (owner: 10Ottomata)
[20:11:10] <ottomata>	 btw joal, i ran into a very tricky issue with kafka connect stuff, not sure yet....
[20:11:21] <ottomata>	 there seems to be some problem with the way it adds fields to structs in arrays
[20:11:35] <joal>	 heha?
[20:11:36] <ottomata>	 since it doesn' thave a field name for the struct (it is just an array element), i think there is a key collision error somewhere
[20:11:41] <ottomata>	 it uses some default field name
[20:12:06] <ottomata>	 which later causes conversion from each array element to be inserted .put into the same struct?  not 100% sure
[20:12:07] <ottomata>	 its weird
[20:12:28] <ottomata>	 some reference by name in the underlying connect data structure collision
[20:12:36] <joal>	 hum - I think I don't it completely
[20:12:40] <ottomata>	 yeah, me neither
[20:12:42] <joal>	 :)
[20:12:54] <ottomata>	 but, if i have two struct elements in an array
[20:13:07] <ottomata>	 it seems they all get converted to the last value
[20:13:30] <joal>	 ottomata: then, no double reading, but schema-merging from schema-repo instead of json-read schema?
[20:13:41] <joal>	 ottomata: Wow, that is worth than what we do :)
[20:13:45] <joal>	 worse sorry
[20:13:59] <ottomata>	 haha
[20:14:18] <ottomata>	 well, that would at least allow the json data to be read with the proper schema!
[20:14:25] <ottomata>	 and hopefully fill in nulls?  dunno
[20:14:40] <ottomata>	 what would happen if we tried to read the array elem without error
[20:14:45] <ottomata>	 into a spark schema that has error?
[20:14:45] <ottomata>	 DUNNO!
[20:14:48] <joal>	 ottomata: I think it works correctly
[20:15:02] <ottomata>	 but joal, i think the answer rigiht now is my patch + fixing the data model as nuria suggested
[20:15:05] <joal>	 it would fill nulls (IIRC from the previous stuff we did)
[20:15:18] <joal>	 correct ottomata !
[20:15:30] <nuria>	 ottomata: i was just editing teh schema and trying to find  a good example
[20:15:35] <joal>	 nuria's point on schema structure is so true!
[20:16:29] <ottomata>	 ya i think that will be better for sure
[20:16:32] <ottomata>	 will make querying easier too
[20:17:24] <joal>	 indeed
[20:20:25] <wikibugs>	 (03CR) 10Fdans: Add project families to uniques loading job (033 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/476220 (https://phabricator.wikimedia.org/T167539) (owner: 10Fdans)
[20:20:31] <ottomata>	 joal q:
[20:20:33] <joal>	 a-team: I plan on deploying refinery tomorrow morning -- nothing to deploy in refinery-source, and mostly cron/timer rlated patches in refinery (sqoop from compat, correction in refinery-druid-drop-public-snapshots, offset and underestimate for uniques per domain job (to be restarted on the 1st of december through bundle), and pageview whitelist update)
[20:20:40] <ottomata>	 it will be ok if the top level errors array is not present, right?
[20:20:42] <joal>	 sorry for the loog message
[20:20:47] <ottomata>	 e.g if no models encountered an error?
[20:21:09] <ottomata>	 joal maybe somethign for refinery source for ya... ! :)
[20:21:16] <joal>	 ottomata: it should, yes - It's in a struct, so merge and cast should work
[20:21:21] <ottomata>	 ok right
[20:21:22] <mforns>	 joal, I'm working now on a fix for profig library upgrade
[20:21:24] <joal>	 ottomata: ahaha :)
[20:21:28] <joal>	 k
[20:21:33] <ottomata>	 that means that scores itself won't be a required field anymore
[20:21:41] <ottomata>	 since maybe all models encountered errors
[20:21:42] <joal>	 no prob mforns - can you drop me an email if you wish me to deploy it?
[20:21:49] <mforns>	 joal, if you could deploy that tomorrow, that would be great
[20:21:55] <mforns>	 otherwise, I can deploy also
[20:22:13] <mforns>	 joal, and I think there's also the ELSanitization fix to be deployed in refinery-source
[20:22:46] <mforns>	 joal, depending on the tim you want to do that, I can be there as well
[20:23:32] <joal>	 mforns: I was planning to do it early, first thing in morning - but it can wait :)
[20:23:38] <joal>	 mforns: I'll wait for you to be there
[20:23:57] <mforns>	 how you prefer, I will join around 12h
[20:24:11] <joal>	 Should be ok mforns 
[20:24:15] <mforns>	 ok
[20:24:50] <joal>	 mforns: I actually have patch to be merged as well - it;s just that it has been merged long emough for me to have forgotten it had not been deployed :)
[20:25:00] <joal>	 ok, so change: refinery-source then refinery :)
[20:25:06] <mforns>	 yea ok
[20:27:48] <wikibugs>	 (03PS4) 10Ottomata: HiveExtensions StructType .merge supports Arrays and Maps with complex types [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476179 (https://phabricator.wikimedia.org/T210465)
[20:28:45] <ottomata>	 joal, nuria: https://gerrit.wikimedia.org/r/#/c/mediawiki/event-schemas/+/476381/1/jsonschema/mediawiki/revision/score/1.yaml
[20:31:22] <joal>	 ottomata: I think you;re good --^
[20:31:38] <joal>	 ottomata: I think it should
[20:31:47] <joal>	 oops sorry mispassed
[20:35:51] <ottomata>	 joal:  gonna merge https://gerrit.wikimedia.org/r/#/c/analytics/refinery/source/+/476179/ ok?
[20:36:06] <ottomata>	 i'll also make a patch that removes the findIncompatible stuff...
[20:36:35] <joal>	 +2 ottomata :)
[20:36:47] <nuria>	 ottomata: i see, and the removal of "scores" from required means that  scores might be absent when errors are present 
[20:37:23] <ottomata>	 right, if all models caused errors
[20:38:19] <nuria>	 ottomata: tried to describe issue in schema guidelines: https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging/Schema_Guidelines#Schema_set_up
[20:38:31] <nuria>	 and i mean *TRIED*
[20:39:55] <ottomata>	 hmmm
[20:40:34] <ottomata>	 nuria:  i think i get it, in your exampel thouh {'blocked'} is not valid
[20:40:35] <nuria>	 ottomata: ya, please improve my very imperfect docs
[20:40:47] <ottomata>	 did you mean 
[20:40:52] <nuria>	 ottomata:  {status:blocked}
[20:40:54] <ottomata>	 aye
[20:40:58] <nuria>	 ottomata: k
[20:40:58] <ottomata>	 hm
[20:41:14] <wikibugs>	 (03PS1) 10Ottomata: Remove unused HiveExtensions findIncompatibleFields method [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476383
[20:41:31] <ottomata>	 MERGED
[20:41:31] <ottomata>	 ?
[20:41:37] <ottomata>	 https://gerrit.wikimedia.org/r/#/c/476381/
[20:41:38] <ottomata>	 ????
[20:41:42] <ottomata>	 NOOO Nuria!
[20:42:09] <nuria>	 ottomata: wait, isn't there a submit after?
[20:42:12] <nuria>	 ottomata: ahahahahah
[20:42:16] <ottomata>	 reverted
[20:42:16] <nuria>	 ottomata: reverting
[20:42:18] <ottomata>	 https://gerrit.wikimedia.org/r/#/c/mediawiki/event-schemas/+/476385/
[20:42:42] <nuria>	 ottomata: sorry, i am lossing track what repos require additional SUBMIT step!!!!!!!
[20:42:44] <ottomata>	 nuria that would cause a lot of revision-score change prop and eventbus validation erros!!!
[20:42:58] <nuria>	 ottomata: and we sure do not want that....
[20:43:10] * nuria puts 10$ on teh jar
[20:43:13] <nuria>	 *the 
[20:44:00] <nuria>	 sorry  ottomata 
[20:44:30] <ottomata>	 :)
[20:44:35] <ottomata>	 i think we caught it fast enough
[20:45:27] <icinga-wm>	 PROBLEM - https://grafana.wikimedia.org/dashboard/db/eventbus grafana alert on icinga1001 is CRITICAL: CRITICAL: https://grafana.wikimedia.org/dashboard/db/eventbus is alerting: EventBus POST Response Status alert.
[20:45:50] <ottomata>	 haha maybe not!
[20:46:18] <ottomata>	 running puppet...
[20:48:37] <nuria>	 ottomata: sorry again what a mistake!
[20:49:06] <nuria>	 ottomata: on the positive side, good job peter with the laerts
[20:49:09] <ottomata>	 yeah!
[20:49:09] <nuria>	 *alerts
[20:49:12] <ottomata>	 you were just testing!
[20:49:27] <ottomata>	 no harm really, nothing is using these events yet so its ok
[20:49:34] <nuria>	 ottomata: thanks
[20:49:46] <wikibugs>	 (03CR) 10Ottomata: [C: 032] Remove unused HiveExtensions findIncompatibleFields method [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476383 (owner: 10Ottomata)
[20:49:51] <wikibugs>	 (03CR) 10Ottomata: [C: 032] HiveExtensions StructType .merge supports Arrays and Maps with complex types [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476179 (https://phabricator.wikimedia.org/T210465) (owner: 10Ottomata)
[20:51:35] <wikibugs>	 (03PS1) 10Ottomata: Update README for HiveExtensions changes [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476387
[20:51:42] <wikibugs>	 (03CR) 10Ottomata: [V: 032 C: 032] Update README for HiveExtensions changes [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476387 (owner: 10Ottomata)
[20:52:35] <icinga-wm>	 RECOVERY - https://grafana.wikimedia.org/dashboard/db/eventbus grafana alert on icinga1001 is OK: OK: https://grafana.wikimedia.org/dashboard/db/eventbus is not alerting.
[21:00:20] <wikibugs>	 (03Merged) 10jenkins-bot: HiveExtensions StructType .merge supports Arrays and Maps with complex types [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476179 (https://phabricator.wikimedia.org/T210465) (owner: 10Ottomata)
[21:05:17] <wikibugs>	 (03Merged) 10jenkins-bot: Remove unused HiveExtensions findIncompatibleFields method [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476383 (owner: 10Ottomata)
[21:12:39] <wikibugs>	 (03PS2) 10Mforns: Upgrade profig lib to 2.3.3 after bug fix upstream [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476290 (https://phabricator.wikimedia.org/T202429)
[21:14:48] <wikibugs>	 10Analytics, 10Operations, 10WMF-Legal, 10Software-Licensing: Can exfat be used in WMF production? - https://phabricator.wikimedia.org/T210667 (10chasemp)
[21:15:50] <wikibugs>	 (03PS3) 10Mforns: Upgrade profig lib to 2.3.3 after bug fix upstream [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476290 (https://phabricator.wikimedia.org/T202429)
[21:18:41] <wikibugs>	 10Analytics, 10Operations, 10WMF-Legal, 10Software-Licensing: Can exfat be used in WMF production? - https://phabricator.wikimedia.org/T210667 (10chasemp)
[21:21:41] <wikibugs>	 10Analytics, 10Operations, 10WMF-Legal, 10Software-Licensing: Can exfat be used in WMF production? - https://phabricator.wikimedia.org/T210667 (10chasemp) @fgiunchedi I need to sync up with you here for other reasons, but if you could take a look at this that would be great
[21:26:14] <wikibugs>	 10Analytics, 10Operations, 10WMF-Legal, 10Software-Licensing: Can exfat be used in WMF production? - https://phabricator.wikimedia.org/T210667 (10MoritzMuehlenhoff) exfat-fuse itself is free software (GPL) and part of Debian main. Debian's approach on patents is written up at https://www.debian.org/reports...
[21:26:42] <wikibugs>	 10Analytics, 10Operations, 10Security-Team, 10WMF-Legal, 10Software-Licensing: Can exfat be used in WMF production? - https://phabricator.wikimedia.org/T210667 (10chasemp)
[21:27:27] <wikibugs>	 (03CR) 10Nuria: [C: 031] Add project families to uniques loading job (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/476220 (https://phabricator.wikimedia.org/T167539) (owner: 10Fdans)
[21:35:36] <wikibugs>	 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review, 10User-Elukey: rack/setup/install cloudvirtan100[1-5].eqiad.wmnet - https://phabricator.wikimedia.org/T207194 (10Cmjohnson)
[21:37:05] <wikibugs>	 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review, 10User-Elukey: rack/setup/install cloudvirtan100[1-5].eqiad.wmnet - https://phabricator.wikimedia.org/T207194 (10Cmjohnson) @robh these are ready for installs, I changed the primary nic to boot from the 10G NIC.  The raid was set up exactly like...
[21:37:54] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10EventBus, 10Services (watching): Modern Event Platform: Stream Intake Service: Implementation - https://phabricator.wikimedia.org/T206785 (10Ottomata) Simply EventIntake?  class EventIntake EventIntake service EventIntake instance  ... Hm.  Maybe it's not bad after...
[22:07:57] <wikibugs>	 (03CR) 10Nuria: "Nice, thanks for cleaning up" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/476383 (owner: 10Ottomata)
[22:16:03] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10EventBus, 10Services (watching): Modern Event Platform: Stream Intake Service: Implementation - https://phabricator.wikimedia.org/T206785 (10mobrovac) I put down some proposals which are more literary (//JozefK//, //DerProcess//, etc) but going with the obvious Even...
[22:29:19] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10EventBus, 10Services (watching): Modern Event Platform: Stream Intake Service: Implementation - https://phabricator.wikimedia.org/T206785 (10Nuria) "EventGate" as in : "the place where events enter our galaxy"...
[22:51:18] <wikibugs>	 10Analytics, 10Operations, 10Security-Team, 10WMF-Legal, 10Software-Licensing: Can exfat be used in WMF production? - https://phabricator.wikimedia.org/T210667 (10Legoktm) >>! In T210667#4783289, @MoritzMuehlenhoff wrote: > exfat-fuse itself is free software (GPL) and part of Debian main. Debian's approa...