[00:33:50] Analytics-EventLogging, operations, Graphite: Statsv down since 2015-09-20 07:53 - https://phabricator.wikimedia.org/T113315#1661778 (ori) There were two gmond processes running, each saturating a CPU core. As a temporary measure, I stopped `ganglia-monitor` and disabled Puppet to prevent it from getti... [00:59:58] Analytics-EventLogging, operations, Graphite: Statsv down since 2015-09-20 07:53 - https://phabricator.wikimedia.org/T113315#1661805 (ori) The statsv error log is filled with: ``` Traceback (most recent call last): File "/srv/deployment/statsv/statsv/statsv.py", line 51, in data = json.... [01:23:44] !log Logging from this channel is picked up by stashbot now [01:33:50] Analytics-Kanban: Requesting access to stat1002 for Dan Foy - https://phabricator.wikimedia.org/T113324#1661834 (kevinator) NEW [01:34:15] Analytics-Kanban: Requesting access to stat1002 for Dan Foy - https://phabricator.wikimedia.org/T113324#1661834 (kevinator) [01:38:08] Analytics-EventLogging, operations, Graphite: Statsv down since 2015-09-20 07:53 - https://phabricator.wikimedia.org/T113315#1661854 (Krinkle) p:Unbreak!>High [01:39:29] :) [02:52:51] Analytics-EventLogging, operations, Graphite: Statsv down since 2015-09-20 07:53 - https://phabricator.wikimedia.org/T113315#1661927 (Ottomata) Hm, not sure why this would happen all of the sudden at this time. The only change that happened today was a restart of eventlogging to change a setting on th... [05:27:09] Krinkle: thanks :) [05:39:55] Analytics-Kanban, Patch-For-Review: Resurrect logbot so we can leave a breadcrumb trails for incident reports, teammates, etc. [5 pts] - https://phabricator.wikimedia.org/T111393#1662016 (madhuvishy) Messages logged with !log in #wikimedia-analytics now show up here: https://tools.wmflabs.org/sal/analytics... [06:30:26] Analytics-Wikistats: Adding Odia (Oriya) Wikisource to Stats Wiki - https://phabricator.wikimedia.org/T108012#1662053 (Jnanaranjan_sahu) Thannk @ezachte . [09:02:25] Analytics-Tech-community-metrics, DevRel-September-2015: Provide open changeset snapshot data on Sep 22 and Sep 24 (for Gerrit Cleanup Day) - https://phabricator.wikimedia.org/T110947#1662354 (Dicortazar) Some numbers to have some context. * Total number of changesets still open: 3029. * Total number of... [10:13:37] Analytics-Backlog, Wikimedia-Developer-Summit-2016: Developer summit session: Pageview API overview - https://phabricator.wikimedia.org/T112956#1662455 (Aklapper) [10:26:31] Analytics-Backlog: Re-baselining checkpoints periodically - https://phabricator.wikimedia.org/T112009#1662471 (Aklapper) [10:37:15] Analytics-Tech-community-metrics, DevRel-September-2015: Provide open changeset snapshot data on Sep 22 and Sep 24 (for Gerrit Cleanup Day) - https://phabricator.wikimedia.org/T110947#1662499 (Dicortazar) Hi again, some improved numbers after some minor filtering process: * Total number of changesets wai... [10:38:39] Analytics-Tech-community-metrics, DevRel-September-2015: Provide open changeset snapshot data on Sep 22 and Sep 24 (for Gerrit Cleanup Day) - https://phabricator.wikimedia.org/T110947#1662501 (Dicortazar) [10:50:12] Analytics-Tech-community-metrics, DevRel-September-2015: Provide open changeset snapshot data on Sep 22 and Sep 24 (for Gerrit Cleanup Day) - https://phabricator.wikimedia.org/T110947#1662510 (Qgil) Nemo is clearly independent. Legoktm and Alex Monk (Krenair) are WMF employees, but they are cases of devel... [11:13:10] Analytics-Tech-community-metrics, DevRel-September-2015: Tech metrics should talk about "Affiliation" or "Organization" instead of companies - https://phabricator.wikimedia.org/T62091#1662529 (Dicortazar) Ok, this has been updated from "companies" to "organizations". Closing this task. Kudos to quan @ bit... [11:13:21] Analytics-Tech-community-metrics, DevRel-September-2015: Tech metrics should talk about "Affiliation" or "Organization" instead of companies - https://phabricator.wikimedia.org/T62091#1662531 (Dicortazar) Open>Resolved [13:05:29] o/ joal [13:11:10] Hey halfak :) [13:11:29] :S [13:24:47] Analytics-Cluster, Analytics-Kanban: Send email when load jobs fails {hawk] [8 pts] - https://phabricator.wikimedia.org/T113253#1662687 (JAllemandou) a:JAllemandou [13:34:36] Analytics-Backlog, Analytics-Cluster: webrequest_sequence_stats_hourly fails/continues Load job {hawk} [13 pts] - https://phabricator.wikimedia.org/T113252#1662698 (Ottomata) Hey, I was thinking about this more after talking with Madhu about the Gobblin vs. Camus spike yesterday, and I think I'd like to c... [13:51:13] milimetric: this is more ready now, would love a review! https://gerrit.wikimedia.org/r/#/c/238854/ [13:51:40] will do after trying to fix the restbase mistakes i made yesterday [13:51:44] k :) [14:13:56] Analytics-Wikistats: Percentage pageviews from Russia is too low in recent geographical breakdowns in Wikistats - https://phabricator.wikimedia.org/T109582#1662808 (ezachte) Wikistats traffic scripts that parse squid logs can't handle secure messages. Up till early 2015 only 1 or 2 percent of message was sec... [14:20:00] git log [14:20:05] mwarf ... [14:20:33] (PS1) Joal: Add oozie email sending subworkflow wrapper [analytics/refinery] - https://gerrit.wikimedia.org/r/240094 [14:20:35] (PS1) Joal: Add email sending on error in webrequest-load [analytics/refinery] - https://gerrit.wikimedia.org/r/240095 (https://phabricator.wikimedia.org/T113253) [14:26:41] milimetric: does restbase use gerrit? [14:26:49] no [14:51:38] milimetric: one (probably dumb) sql question [14:51:47] shoot [14:53:41] nuria: ^ [14:54:06] milimetric: to do a group by .. don't we need to do an order by before [14:54:22] so we make sure "groups" do not appear more than once ? [14:54:40] nuria: no, that's only in weird unix land :) [14:54:55] group by just keeps a running set of buckets and puts whatever it finds in the right bucket [14:55:30] milimetric: ahhh, so it used to be like that in ancient unix sql , which is all i know, ok , good [14:55:39] order by is a post-select operator. It applies on the set of rows after they're ready [15:07:02] joal: do you remember how to run restbase locally? I lost all my history since it's been a while and I can't re-trace [15:08:07] milimetric: I think it's "node server.js -c config-file.yaml" [15:08:14] IIRC [15:08:20] wasn't there a port? [15:08:28] 7xxx or something? [15:08:29] Don't think so [15:08:44] k, maybe my brain made it up :) [15:08:46] thx much [15:08:56] np :) [15:27:17] halfak: altiscale meeting or not :) [15:28:38] I'm planning to given them my feedback about how I don't see hadoop as a viable medium for ad-hoc work until they can figure out how to make it easier to iterate. [15:28:49] k [15:28:56] need me in any way ? [15:29:02] Na. I don't think so. [15:29:15] oh key ! [15:29:21] Thanks :) [15:32:08] ottomata: standup? [15:32:13] AH [15:34:15] ottomata: statsv is collected from reqs using kafka and also rebroadcasted using forwarded that way https://wikitech.wikimedia.org/wiki/Graphite#statsv [15:35:46] (CR) Joal: "Actually I think we can ottomata, I'll try to have that setup." [analytics/refinery] - https://gerrit.wikimedia.org/r/240095 (https://phabricator.wikimedia.org/T113253) (owner: Joal) [15:38:57] Analytics-EventLogging, operations, Graphite: Statsv down since 2015-09-20 07:53 - https://phabricator.wikimedia.org/T113315#1662979 (Krinkle) >>! In T113315#1661927, @Ottomata wrote: > Also, this is statsv, right? statsv != eventlogging, so I'm not sure what Kafka code you are referring to. See http... [15:41:24] Analytics-EventLogging, operations, Graphite: Statsv down since 2015-09-20 07:53 - https://phabricator.wikimedia.org/T113315#1662994 (Ottomata) Aye, I don't know what encoding varnishkafka is sending, but it is C and I wouldn't be surprised if it was just ascii. But as far as I know, nothing has chang... [15:42:18] Analytics-Cluster, Analytics-Kanban, operations, Patch-For-Review: Fix llama user id {hawk} [5 pts] - https://phabricator.wikimedia.org/T100678#1663000 (Dzahn) moved to "Done" means it's resolved, right? [15:45:12] Analytics-Cluster, Analytics-Kanban, operations, Patch-For-Review: Fix llama user id {hawk} [5 pts] - https://phabricator.wikimedia.org/T100678#1663010 (Dzahn) shouldn't that user have "system=> true" though if the point is to create it as a system user? [15:50:38] Analytics-Tech-community-metrics, DevRel-September-2015: Provide open changeset snapshot data on Sep 22 and Sep 24 (for Gerrit Cleanup Day) - https://phabricator.wikimedia.org/T110947#1663029 (Aklapper) Thanks for providing these numbers (and seeing the problems of medians that I didn't think about)! Mos... [15:53:23] Analytics-EventLogging, MediaWiki-extensions-MultimediaViewer: 60% of MultimediaViewerNetworkPerformance dropped (exceeds maxUrlSize) - https://phabricator.wikimedia.org/T113364#1663032 (Krinkle) NEW [15:53:31] Analytics-EventLogging, MediaWiki-extensions-MultimediaViewer: 60% of MultimediaViewerNetworkPerformance events dropped (exceeds maxUrlSize) - https://phabricator.wikimedia.org/T113364#1663042 (Krinkle) [15:56:31] You guys need +c [15:56:49] Or...-c [15:56:52] I hate IRC modes [15:58:07] Analytics-Kanban, RESTBase-API: create RESTBase endpoints [21 pts] {slug} - https://phabricator.wikimedia.org/T107053#1663063 (madhuvishy) a:madhuvishy>Milimetric [16:06:40] milimetric: if you need second set of eyes to debug the restbase things I'm here to help! [16:08:11] Analytics-Backlog, Analytics-Cluster: {epic} Implement better Webrequest load monitoring {hawk} - https://phabricator.wikimedia.org/T109192#1663089 (kevinator) [16:08:13] Analytics-Cluster, Analytics-Kanban: Spike replacing Camus with Gobblin {hawk} [13 pts] - https://phabricator.wikimedia.org/T111409#1663087 (kevinator) Open>Resolved We're going to keep camus for now, but if we need more features, we'll switch to Goblin in the future. [16:09:10] Analytics-EventLogging, Multimedia, UploadWizard: Half the time, 100% of UploadWizardExceptionFlowEvent events are dropped - https://phabricator.wikimedia.org/T113366#1663098 (Krinkle) NEW [16:09:51] Analytics-Kanban, Patch-For-Review: Resurrect logbot so we can leave a breadcrumb trails for incident reports, teammates, etc. [5 pts] - https://phabricator.wikimedia.org/T111393#1663118 (kevinator) Open>Resolved [16:10:43] Analytics-Cluster, Analytics-Kanban, operations, Patch-For-Review: Fix llama user id {hawk} [5 pts] - https://phabricator.wikimedia.org/T100678#1663123 (kevinator) Open>Resolved [16:11:00] Analytics-Kanban, Patch-For-Review: enforce group-writeable in stat1002:/a/aggregate-datasets/, stat1003:/a/public-datasets/, and stat1003:/a/limn-public-data {hawk} [3 pts] - https://phabricator.wikimedia.org/T111956#1663125 (kevinator) Open>Resolved [16:16:41] Analytics-Backlog, Analytics-Cluster: Investigate using camus offset files to start hive load job {hawk} [5 pts] - https://phabricator.wikimedia.org/T113251#1663139 (JAllemandou) [16:17:13] Analytics-Backlog, Analytics-Cluster: Investigate using camus offset files to start hive load job {hawk} [5 pts] - https://phabricator.wikimedia.org/T113251#1659868 (JAllemandou) a:Ottomata>JAllemandou [16:17:24] Analytics-Cluster, Analytics-Kanban: Investigate using camus offset files to start hive load job {hawk} [5 pts] - https://phabricator.wikimedia.org/T113251#1659868 (JAllemandou) [16:21:54] Analytics-Cluster, Analytics-Kanban: Create Kafka deployment checklist on wikitech {hawk} [5 pts] - https://phabricator.wikimedia.org/T111408#1663162 (Ottomata) Added checklist here: https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Kafka/Administration#Upgrade_Checklist [16:22:36] Analytics-Cluster, Analytics-Kanban: Create Kafka deployment checklist on wikitech {hawk} [5 pts] - https://phabricator.wikimedia.org/T111408#1603220 (Ottomata) @joal, would you look that over and let me know what you think or would change? [16:25:56] Analytics-Cluster, Analytics-Kanban: Create Kafka deployment checklist on wikitech {hawk} [5 pts] - https://phabricator.wikimedia.org/T111408#1663182 (JAllemandou) @ottomata A nic addition would be the grafana dashboard URL, but except from that it looks good :) [16:25:58] madhuvishy: I might take you up on that after my meetings today [16:26:18] thx [16:28:15] Hey kevinator, quick question for you [16:28:21] yes [16:28:29] We are wondering if you want to receive emails from failing oozie jobs :) [16:28:42] ottomata thinks not, but I prefer to ask :) [16:29:01] probably not... I wouldn't know what to do with them except pass it on to the team [16:29:08] ok ;0 [16:29:48] The rest of the team (ottomata, milimetric, mforns , madhuvishy, nuria and me) are on the list [16:31:11] (CR) Joal: Add oozie email sending subworkflow wrapper (4 comments) [analytics/refinery] - https://gerrit.wikimedia.org/r/240094 (owner: Joal) [16:34:16] (PS2) Joal: Add oozie email sending subworkflow wrapper [analytics/refinery] - https://gerrit.wikimedia.org/r/240094 (https://phabricator.wikimedia.org/T113253) [16:41:59] (CR) Ottomata: [C: 1] Add oozie email sending subworkflow wrapper [analytics/refinery] - https://gerrit.wikimedia.org/r/240094 (https://phabricator.wikimedia.org/T113253) (owner: Joal) [16:48:50] (PS2) Joal: Add email sending on error in webrequest-load [analytics/refinery] - https://gerrit.wikimedia.org/r/240095 (https://phabricator.wikimedia.org/T113253) [17:00:04] (PS2) Joal: Add pageview quality check to pageview_hourly [analytics/refinery] - https://gerrit.wikimedia.org/r/240099 (https://phabricator.wikimedia.org/T109739) [17:06:20] milimetric: sure [17:09:13] kevinator, aren't we having the preparation meeting right now? [17:16:11] mforns: yes [17:16:23] you can join the hangout and hangout... [17:16:43] A/V is setting up the room and the video [17:23:32] joal: should I put "bot" in the list of possible user agents for the restbase endpoints? [17:23:57] kevinator, I'm in the hangout, but nobody is there [17:24:05] ok [17:25:25] mforns: are you in the hangout? there is no one on our end [17:26:51] kevinator, yes, I am [17:27:07] kevinator, https://plus.google.com/hangouts/_/wikimedia.org/setup-lighting [17:27:32] ok, the av folks want to practice switching from Moriel's presentation to yours [17:27:43] ok [17:28:24] ah, there's another link. [17:28:36] the av guy will join your hangout now [17:28:42] kevinator, still no one in the hangout, there's a typo in the hangout name, maybe this is the problem? [17:28:42] his name is brendan [17:28:45] ok [17:33:10] joal: are we sure we wnat to restrict our search to mobile and text partitions? [17:33:47] joal: don't we want images too like pngs and such? (this is regarding PI data) [17:38:41] nuria: You can try with full partitions, but pageview_hourly is extracted only from text adn mobile [17:38:56] milimetric: you should indeed ! [17:39:20] For the moment we have no result, but hopefully after having communicated the convention, it might fill-in [17:39:34] joal: ok, i saw it included png files so i imagine those come via "text" (i thought that was bits) [17:39:51] bits is dead :) [17:42:07] milimetric, do you remember when that parallel processing switchoff patch went out? [17:42:20] (I'm trying to rely on an EL schema that straddles that fix and want to work out what evens to trust) [17:42:39] Ironholds: the data should've started getting better I think around 16:00 last Monday [17:42:48] poifect! Thanks :D [17:42:51] but I'm totally guessing, I commented in the relevant thread with the exact timestamp I thought [17:42:56] ta [17:42:56] Ironholds: ^ [17:43:12] it's "around" that time though [17:43:14] yep, got it [18:23:07] not super urgent, but perhaps someone has a quick idea how to correct that ganglia link https://wikitech.wikimedia.org/wiki/Talk:Analytics/Cluster/Hadoop/Load (i think qchris put it in earlier) [18:31:25] milimetric: nice work by the vega guy eh? [18:34:41] yeah, def. [18:38:34] madhuvishy: the latest version of my PR seems to work ok: https://github.com/wikimedia/restbase/pull/350 [18:40:14] milimetric: this naming conflict for analytics_v1_pageviews is endless :P [18:42:54] yes. I'm going to call it angry_chicken if they don't all agree soon :) [18:43:13] :P [18:46:06] (CR) Ottomata: [C: 2] "COOOL" [analytics/refinery] - https://gerrit.wikimedia.org/r/240095 (https://phabricator.wikimedia.org/T113253) (owner: Joal) [18:53:33] Bye a-team ! [18:53:38] See you tomorrow :) [18:53:59] Good luck milimetric with restbase :S [18:54:12] nite joal, restbase is all set i think [18:54:18] You rock :) [18:54:55] we still need to deploy it, but I'll work with andrew after lightning talks [18:55:49] Thx Mate [18:57:12] good night joal! [19:28:39] mforns: nice presentation [19:28:58] mforns: super clear [19:29:07] cool! thx [19:29:21] nuria, ^ [19:30:38] milimetric: i am yours! [20:27:59] HaeB: Just saw your ping from 2 hours earlier, and tried to find a new url for you, but ganglia's kafka view does not [20:28:42] show any numbers for me, and drilling down on a single kafka server (e.g.: kafka1012) gives me 1900 metrics, which totally grinds [20:28:50] qchris: whatcha looking for? [20:28:51] my browser to a halt. [20:28:56] http://grafana.wikimedia.org/#/dashboard/db/kafka [20:29:04] I was just suggesting to ping you :-D [20:29:18] Aaaaah :-) [20:29:45] Haeb: See the above link by ottomata. [20:30:14] http://grafana.wikimedia.org/#/dashboard/db/kafka?panelId=19&fullscreen [20:30:21] ^ that should be the one you're after. [20:30:38] cool, thanks - so should we replace the link at https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hadoop/Load#Detecting_a_stalling_cluster_using_only_a_browser with this one? [20:30:42] But better ask ottomata ... he might have even better graphs. [20:31:17] HaeB: Might be true ... but I am not sure if that's still the suggested way to detect if the cluster is overloaded. [20:31:24] ottomata ... is it? [20:31:31] HaeB: good call, probably so.but this way of detecting stalled import might not work in several months [20:31:41] especially if we start using kafka in different ways, e.g. eventbus [20:32:11] HaeB: if you are looking for stalled webrequest related imports in Hadoop [20:32:21] a good thing to check is Hue oozie coordiantors [20:32:28] HaeB: do you have a Hue login? [20:32:48] yes [20:32:52] k [20:32:52] then [20:33:06] if you go here: [20:33:07] https://hue.wikimedia.org/oozie/list_oozie_bundles/ [20:33:11] you can see the running bundles [20:33:19] if interest are probably load an refine webrequest [20:33:32] here's upload [20:33:32] https://hue.wikimedia.org/oozie/list_oozie_coordinator/0122646-150605005438095-oozie-oozi-C/?bundle_job_id=0122641-150605005438095-oozie-oozi-B [20:33:37] what are bundles? [20:33:38] its got some nasty read workflows! [20:33:46] red* [20:33:49] could be ok [20:33:56] but if there are a lot of them in a row then it is probalby not ok [20:34:20] but, stalled would usually mean that there is a backlog of orange (missing data dependencies) jobs [20:35:24] HaeB: https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Oozie#Some_terms [20:35:35] (i should say that i came to that page via https://wikitech.wikimedia.org/wiki/Analytics/Cluster/Hive/Queries#My_query_does_not_progress_.2F_finish ...i.e. mainly interested in checking whether hive/hadoop is overloaded or not) [20:35:45] ah [20:35:56] HaeB: I think that is easier then to just look at the yarn job list [20:36:00] https://yarn.wikimedia.org/cluster/scheduler [20:36:40] i don't see any job there from you at the moment, but you can see the list of all jobs currently running in Hadoop [20:36:42] including Hive jobs [20:37:36] yes, that was last night (when i left that talk page comment). i did check the job list as suggested on that page and my query finished fine eventually... [20:38:08] ...just thought that that link, or also the instructions in that section, could be fixed [20:38:33] ahh ok, thanks [20:38:38] anyway, as i said, not terribly urgent - i'll check the job list first in the future [20:41:45] ottomata: Those grafana dashboards are slick. Awesome! [20:42:07] qchris: check out the eventlogging one! [20:42:12] http://grafana.wikimedia.org/#/dashboard/db/eventlogging [20:42:14] Did that already :-) [20:42:17] :) [20:42:22] They're great. [20:42:39] And they are so responsive. [20:42:50] (If only Kibana would be like that :-) ) [20:54:52] ottomata, qchris https://wikitech.wikimedia.org/w/index.php?title=Analytics/Cluster/Hadoop/Load&diff=184219&oldid=153499 [20:55:32] and (yarn) so the hapoop queries are all public? (kind of, partially, temporary) [20:56:19] didn't know that. i mean i don't have too many secrets to hide in that respect, but perhaps users should be aware... [20:56:20] HaeB: Thanks. LGTM :-) [20:57:34] thanks HaeB [20:57:38] ottomata: I was starving had to eats [20:57:52] you still around for a bit? [20:59:05] AH was about to leave [20:59:12] can consult for a few mins [20:59:30] helpful? batcave? [21:01:30] milimetric: ^^ [21:01:38] yes, cave [21:11:39] ottomata: IRC's fine :) [21:11:45] ah ok [21:11:47] so [21:11:50] so you're saying basically factor out the separate cassandra and restbase roles [21:11:51] ja [21:11:54] something like [21:11:57] into a single aqs.pp [21:12:18] role::analytics::aqs::cassandra { .. } [21:12:18] yes [21:12:18] under a top level 'aqs' class [21:12:20] then the [21:12:43] role::analytics::aqs { [21:12:43] include ...::aqs::cassandra [21:12:43] include ...::aqs::restbase [21:12:43] } [21:12:47] if you like. [21:12:52] then the nodes in site.pp are just [21:12:55] not just include ...::aqs ? [21:12:57] role analytics::aqs [21:13:30] right wouldn't it be better to include role::analytics::aqs rather than ::cassandra and ::restbase specifically? [21:13:38] that way if we add stuff to aqs it's also included? [21:15:33] milimetric: https://gist.github.com/ottomata/7373448b8488feeafff9 [21:15:57] class role::analytics::aqs is optional, up to yall if you think you want that, i'm not picky [21:16:03] oh :) [21:16:06] i really just don't want the role/analytics/cassandra.pp file [21:16:14] yeah, agreed [21:16:21] as you are configuring AQS here, not some global cassandra instance for the analtyics cluster [21:16:27] ottomata: but then it's different from the other restbase instance btw [21:16:55] (the current one that's running has different restbase and cassandra .pp files) [21:17:43] hm, milimetric i'm looking at the other cassanrda role [21:17:49] do you really need to copy paste this at all? [21:17:59] i think you can just include that and set hiera vars appropriately, no? [21:18:33] hmm, not sure, if you can't then this is either already poorly named or poorly designed [21:18:50] i would except a role/cassandra.pp file to be generic enough for any cassanrda instance. [21:18:51] um, I'm not sure either [21:18:52] otherwise it shouldn't be a role. [21:19:11] I didn't do that of my own accord, I think I made those files after some patches and comments [21:19:29] milimetric: i think you can [21:19:36] the only reall difference I see is include ::passwords::cassandra [21:19:50] otherwise its a straight copy paste [21:19:57] yeah, pretty much [21:20:07] not very DRY! :p [21:20:38] Analytics-Kanban, Research consulting, Research-and-Data: Validate Uniques using Last Access cookie {bear} - https://phabricator.wikimedia.org/T101465#1339575 (madhuvishy) Work done on this is documented here: https://wikitech.wikimedia.org/wiki/Analytics/Unique_clients/Last_access_solution/Validation [21:20:54] if restbase is supposed to be a generic API service, then it should not have a top level role class that isn't useable by others [21:21:21] that sounds like it makes sense. Do you think we can just go forward like this since everyone agreed and refactor laer? [21:21:24] *later [21:21:25] yeah [21:21:29] no, i'm going to comment [21:21:32] this is dumb [21:21:38] you don't need restbase.pp or cassandra.pp classes at all [21:21:46] all you need is an aqs.pp class [21:21:48] that includes these ones. [21:21:52] or not even that [21:21:54] ok :) feel free to take over if you want [21:21:56] this actually looks well done. [21:22:10] I MAY DO SO! [21:22:13] Commenting. [21:22:18] I like salty ottomata [21:22:27] :) [21:23:28] * milimetric takes off his puppet programmer Halloween costume and stops being scared of his own reflection [21:24:42] hehe [21:24:44] puppet isnt' so much of a language as it is an incantation phrase book powered by souls devoured thousands of years ago and given form by the heartache of opsen everywhere [21:25:16] that's dark man [21:25:48] ...and then there was cake [21:25:50] ok milimetric gotta run, seeayYYay [21:25:55] nite ottomata [21:54:26] Analytics-Backlog: Enable use of Python 3 in Spark - https://phabricator.wikimedia.org/T113419#1664530 (ellery) NEW a:Ottomata [22:02:51] Analytics-Backlog: Update passport-mediawiki module URLs and documentation - https://phabricator.wikimedia.org/T113234#1664580 (Milimetric) Also, as mentioned here: https://phabricator.wikimedia.org/T112705#1662183 a 403 error would have been useful in debugging. [23:14:46] (PS1) Ori.livneh: Dead-simple parallelization [analytics/statsv] - https://gerrit.wikimedia.org/r/240290 [23:19:44] (CR) Legoktm: [C: 1] Dead-simple parallelization [analytics/statsv] - https://gerrit.wikimedia.org/r/240290 (owner: Ori.livneh) [23:20:07] (CR) Ori.livneh: [C: 2] Dead-simple parallelization [analytics/statsv] - https://gerrit.wikimedia.org/r/240290 (owner: Ori.livneh)