[00:05:24] oh looks like it needs to be 'mediawiki' instead of 'www.mediawiki' [00:15:59] yes, kaldari, sorry for the crap error message, we're working on that [00:16:14] NP [00:16:33] the ones that have www. as a prefix work with just the main hostname, like wikidata, mediawiki, etc. [00:16:42] then commons.wikimedia is weird [00:16:44] got it [00:16:54] but I thought I merged the change that stripped the www. hm.... [00:16:59] Maybe that's not deployed yet [01:08:36] milimetric: doesn't seem to be :) [01:09:21] yep, I'll check up on it tomorrow [04:34:56] Analytics, Wikipedia-iOS-App-Product-Backlog, iOS-app-feature-Analytics, iOS-app-Bugs, iOS-app-v5.0.3-Disco: Invalid pageview data for iOS app - https://phabricator.wikimedia.org/T131824#2185971 (Tbayer) @Mhurd: Awesome, thanks for resolving the mystery! The cleanest solution would be to updat... [08:15:52] Analytics-Cluster, Analytics-Kanban: Story: Community has periodic browser stats report generated from Hadoop data - https://phabricator.wikimedia.org/T69053#734731 (Addshore) @Nuria please see T130102. Would it be possible to split off wikidata data so that it can be looked at separately? [08:33:46] Hi elukey :) [08:35:01] o/ [08:35:32] node upgrade elukey, or not yet ? [08:35:56] joal: I wanted to ask the same thing :) [08:36:02] huhu :) [08:36:22] I'd say let's go for it (particularly knowing we don't have to stop cassandra :) [08:36:24] I believe that we could proceed simply de-pooling from LVS and stopping restbase, without touching our dear friend cassandra [08:36:30] :) [08:36:32] indeed [08:36:48] any luck in finding why the loading job doesn't work? [08:36:49] elukey: following on the monitoring [08:36:54] I did :) [08:36:59] wooooooooooo [08:37:03] 22:48:24 * joal has won against hadoop-cassandra [08:37:03] 22:48:24 * joal has won against hadoop-cassandra [08:37:16] :) [08:37:40] Problem was in loader: race condition in the multi-threading stuff [08:38:11] was it a known bug or you just patched it by hand? [08:38:18] I still need to find the exact root cause to patch, but the easiest way was to make a simple output format without multithreading [08:38:39] * elukey thinks joal ROCK [08:38:52] verrryyy niceeee [08:38:58] elukey: The thing is, even if advertising the opposite, open source code to have hadoop and cassandra talk together is very lame [08:39:52] mmm what do you mean? Because of HBase? Or did I just missed completely the point? [08:40:08] elukey: When using the paying stuff from datastax, you get super quality hadoop-cassandra links [08:40:08] * elukey is ignorant [08:40:25] elukey: but when using the open source part of it, well, it's buggy :) [08:40:35] ahhhhhh [08:40:43] Thanks for sharing excitement elukey :) [08:40:57] I was happy yesterday when I found that (and I still am actually) [08:41:56] so there might be some upstream patch?? [08:42:05] to the opensource buggy stuff :P [08:42:31] elukey: I offered a patch a few month ago, but didn't follow if it had been merged or not [08:42:45] elukey: I should do that (follow and offer a new patch) [08:46:35] yeppppp! [08:46:44] it would be super great for you and for the analytics team :) [08:46:54] all right so I am going to finish a thing then I'll start the node migration [08:47:18] elukey: awesome, waiting for the log message ;) [08:59:25] !log removed aqs1001.eqiad.wmnet from LVS pool via confd for nodejs upgrade [08:59:27] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master [09:09:57] elukey: ok with depooling so far? [09:10:43] yep yep I saw three dependencies to be removed (libc-ares-dev libv8-3.14-dev libv8-3.14.5) and I am double checking that it is the right thing to do :) [09:11:03] elukey: no rush, was just trying to follow :P [09:11:10] ah no it doesn't remove them, just with 'apt-get autoremove' [09:11:12] goooooooo [09:12:44] all right we have nodejs4.3 [09:12:53] on aqs1001? [09:13:03] yep, and it is answering health checks [09:13:17] aqs started? [09:13:35] yep yep [09:13:58] dan told me to check https://github.com/wikimedia/analytics-aqs-deploy/blob/master/scripts/insert_monitoring_fake_data.cql to see if everything is really working, but if you have other stuff we can test it [09:14:15] ah no sorry this is cassandra [09:14:19] scratch that :D [09:14:21] elukey: that's not the one ;) [09:14:29] I believe we just need to issue some curls [09:14:46] elukey: I think you're thinking /home/milimetric/test.sh [09:15:00] That's the one I use ;) [09:15:36] niceee [09:15:49] everything works fine, do you want to make other tests? [09:15:55] otherwise I'll re-pool it [09:15:57] /srv/deployment/analytics/aqs/deploy/test/test_local_aqs_urls.sh [09:16:14] is the same, but I forgot to give the file 755 [09:16:31] elukey: REPOOOOOOOOL ! [09:17:05] oook [09:18:03] !log re-added aqs1001.eqiad.wmnet to LVS pool via confctl [09:18:05] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master [09:18:51] requests are flowing fine, just watching them [09:19:03] elukey: awesome :) [09:19:37] elukey: currently playing with tools.wmflabs.org/pageviews, works really fine (awesome tool by way) [09:20:08] didn't know it! :) [09:20:26] all right I'll wait 10 minutes and then I'll proceed with 1002/1003 [09:20:45] cool [09:24:34] also another thing that I forgot to ask ottomata was if we have plans to migrate analytics* to Debian [09:24:59] elukey: I have no idea ... I assume that would be good [09:30:07] !log aqs1002.eqiad.wmnet de-pooled via confctl. Nodejs upgrade will follow. [09:30:09] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master [09:36:15] aqs1002 re-pooled [09:43:29] !log aqs1002.eqiad.wmnet re-pooled, aqs1003.eqiad.wmnet de-pooled/re-pooled too (nodejs upgrade) [09:43:30] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master [09:43:37] joal: all good :) [09:44:45] elukey: Yaya ! [09:55:24] Analytics-Kanban, Operations: Upgrade aqs* to nodejs 4.3 - https://phabricator.wikimedia.org/T123629#2186361 (elukey) >>! In T123629#2184014, @Eevans wrote: > Do we need the Cassandra restart here? Nope! Just upgraded node only stopping restbase: ``` aqs1003.eqiad.wmnet: ii nodejs... [10:12:41] elukey: aqs1003? [10:13:16] elukey: sorry, didn't notice the message with 2 nodes :) [10:13:39] Analytics-Kanban, Operations, Traffic: varnishkafka logrotate cronspam - https://phabricator.wikimedia.org/T129344#2186374 (elukey) @faidon do you prefer to upgrade the vk package anyway or the above solution would suffice for the moment? [10:16:17] joal: there is also nodejs-legacy that needs to be upgraded, I didn't notice it until Moritz pinged me [10:16:33] it should be safe to upgrade it without doing pool/depool [10:16:35] elukey: I guess you're good for another round [10:17:40] all right I'll start with aqs1001 [10:21:21] !log nodejs-legacy upgraded too on all aqs nodes [10:21:22] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master [10:22:14] Analytics-Kanban, Operations, Traffic: varnishkafka logrotate cronspam - https://phabricator.wikimedia.org/T129344#2186378 (faidon) The Varnishkafka package shipping an rsyslog config is a bug in itself, so we should queue a fix for it. As you've already handled the issue at hand via puppet, it's okay... [10:22:55] Analytics-Kanban, Operations: Upgrade aqs* to nodejs 4.3 - https://phabricator.wikimedia.org/T123629#2186379 (elukey) I probably needed some coffee, I upgraded nodejs-legacy too (thanks @MoritzMuehlenhoff for the hint): ``` aqs1002.eqiad.wmnet: ii nodejs 4.3.0~dfsg-2~wmf1... [11:21:37] Analytics-Kanban, Patch-For-Review: Fix limn-mobile-data mobile-options-last-3-months report after RU changes {lama} - https://phabricator.wikimedia.org/T131849#2186425 (mforns) Deployment plan for this: 1) Merge the limn-mobile-data changes [ALREADY DONE] 2) Merge both RU changes 3) Wait for puppet to ap... [11:24:56] Analytics-Kanban, Patch-For-Review: Add (and default to) a breakdown in percentages also for the line chart. - https://phabricator.wikimedia.org/T130406#2186427 (mforns) Points 1. to 4. in the deployment plan are DONE. Before we can proceed with 5. and 6., we have to: 1) Make the reports list rates (0.5)... [11:38:08] brb lunch! [11:57:16] I'm away foe a while a-team, will be back for standup [12:26:51] back! [15:37:38] (PS1) Milimetric: Split up utils [analytics/dashiki] - https://gerrit.wikimedia.org/r/282183 [15:38:04] Analytics, Analytics-Cluster: cdh::hadoop::directory (and other hdfs puppet command?) should quickly check if namenode is active before executing - https://phabricator.wikimedia.org/T130832#2187011 (elukey) a:elukey [15:39:41] (CR) Mforns: [C: 2 V: 2] "LGTM and seems to work :]" [analytics/limn-ee-data] - https://gerrit.wikimedia.org/r/280371 (https://phabricator.wikimedia.org/T131206) (owner: Neil P. Quinn-WMF) [15:41:04] ottomata: can we close https://phabricator.wikimedia.org/T90640 ? [15:41:51] oh ja! [15:42:13] elukey: you just made me think about organizing the analytics-cluster task into at least one more column [15:42:23] so we can at least more easily look at ones we want to do sooner rather than later [15:43:08] yep definitely! [15:43:22] so I'll be able to pick up stuff when I am free [15:43:22] Analytics-Cluster, Operations: Complete installation of analytics1017.eqiad.wmnet - https://phabricator.wikimedia.org/T125055#2187062 (Ottomata) Open>declined Yuvi is taking over this server [15:43:52] (PS1) Mforns: Make browser reports show rates instead of percentages [analytics/reportupdater-queries] - https://gerrit.wikimedia.org/r/282185 (https://phabricator.wikimedia.org/T130406) [15:44:42] ottomata: (if you have time) - https://gerrit.wikimedia.org/r/#/c/282161/1 - to complete the task [15:47:47] Analytics-Cluster, Operations: setup stat1004/WMF4721 for hadoop client usage - https://phabricator.wikimedia.org/T131877#2187108 (Ottomata) [15:48:17] Analytics, Analytics-Cluster, Operations, Traffic: Upgrade analytics-eqiad Kafka cluster to Kafka 0.9 (or 0.10?) - https://phabricator.wikimedia.org/T121562#2187115 (Ottomata) [15:49:15] Hmm, elukey maybe we should leave the logrotate for the varnishkafka.stats.json stuff [15:49:20] Analytics-Kanban: Remove frequency and timeboxed options from limn-x-data repos config files {lama} - https://phabricator.wikimedia.org/T132062#2187148 (mforns) [15:49:31] hmm, checking default config [15:50:03] ottomata: I checked and we only have the per instance one (that is rotated) [15:50:03] (PS1) Mforns: Remove frequency and timeboxed config options [analytics/limn-flow-data] - https://gerrit.wikimedia.org/r/282187 (https://phabricator.wikimedia.org/T132062) [15:50:11] yeah, elukey the config shipped with the .deb package though [15:50:14] does log to that file by default [15:50:23] but, it doesn't have anything to do with rsyslog [15:50:52] yep but Faidon asked to remove logrotate stuff too :) [15:51:03] so, if we don't ship a logrotate with the package, by the default varnishkafka instance will log there periodically, but never rotate, which has the potential to fill up disk [15:51:10] hm, maybe he just meant for rsyslog? [15:51:11] checking [15:51:38] ottomata: well shouldn't be added by puppet/other that config? [15:51:54] hmm, if we make the default config not write to a stats file [15:51:57] then i'd be ok with it [15:52:08] we ship a custom default config with the package [15:52:16] see debian/varnishkafka.conf line 195 [15:52:33] we could set log.statistics.interval = 0 [15:52:39] and comment out the log.statsistics.file [15:52:41] that would disable it [15:53:10] ahhh makes sense [15:53:15] I can amend it easily [15:56:16] ok [15:56:25] yeah i'm fine with that [15:56:31] make people configure vk to have it on if they want it [15:57:37] a-team: stuck in some traffic. Gonna be a little late to standup. [15:58:29] (PS1) Mforns: Remove frequency and timeboxed config options [analytics/limn-edit-data] - https://gerrit.wikimedia.org/r/282189 (https://phabricator.wikimedia.org/T132062) [15:59:45] currently fighting with chrome, joining in a minute! :@ [16:01:23] elukey: k [16:04:28] elukey: https://phabricator.wikimedia.org/project/board/655/ [16:06:34] (PS1) Amire80: Add ady to language list [analytics/limn-language-data] - https://gerrit.wikimedia.org/r/282190 [16:06:49] (PS2) Amire80: Show the top user name along with the top article count [analytics/limn-language-data] - https://gerrit.wikimedia.org/r/280827 [16:08:44] (PS7) Nuria: Allow filtering of data breakdowns [analytics/dashiki] - https://gerrit.wikimedia.org/r/278395 (https://phabricator.wikimedia.org/T131547) (owner: Jdlrobson) [16:32:00] joal: you solved the loading issue? [16:32:37] I did urandom, found that it comes from the code used to connect cassandra from hadoop [16:33:06] 6There is a weird interaction between that code and the datastax driver [16:33:51] Leading to an infinite loop of new connection creation, but driver says that it can't be because already brokeny [16:33:54] urandom: --^ [16:34:08] joal: wow, nice work tracking that down! [16:34:14] urandom: I might come and ask you for some ideas around datastax driver [16:34:41] urandom: I should have got into that track faster, but I didn't want to dive :) [16:34:52] joal: sure, whatever i can do to help [16:35:04] joal: any progress on using bulk-load? [16:35:20] urandom: We have not worked into that no [16:35:51] joal: are you going to be at wikimania? [16:36:03] attending the hackathon by any chance? [16:36:09] urandom: we'll discuss with the team later on today about what to do next on the issue, I might suggest we ivest in the new loading [16:36:52] urandom: none of them, we have a off-site at Berlin in June (13 to 17) [16:37:00] urandom: :( [16:37:14] joal: oh, what a shame, i'll just miss you [16:37:25] you'll be in Berlin ? [16:37:27] yes [16:37:31] Arrrf ! [16:37:39] from June 4-11 [16:38:20] Will arrive at Berlin on the 12th ... What a shame [16:38:59] urandom: when I'm happy with my understanding of the issue, and the team agrees on what to do next, I'll ping you if I need help :) [16:39:08] joal: anyway, happy to help with that if i can; bulk-loading seems like a very promising approach for this [16:39:11] urandom: Thanks a lot for the support ! [16:39:15] sure! [16:39:45] joal: oh, before i forget; you mentioned being on-board with an upgrade to 2.2.5 first, yes? [16:39:54] yessir ! [16:40:10] I need to have team agreement, but would be good for me [16:40:13] urandom: --^ [16:40:18] cool, i'm going to start working on that soon, testing and puppetization [16:40:23] sure [16:40:23] awesome :) [16:40:42] urandom: I think elukey would be interested to follow you on that one as well [16:40:45] do you have a test/staging env somewhere? [16:41:00] that would be great [16:41:09] * urandom prefers not to work in a vacuum [16:41:36] urandom: I think there is a deployement project, but I don't use it, so can't help really [16:41:52] kk [16:45:37] (PS1) Milimetric: Make number format configurable [analytics/dashiki] - https://gerrit.wikimedia.org/r/282200 (https://phabricator.wikimedia.org/T131965) [16:54:10] joal, urandom: 2.2.5 is a cassandra update? [16:54:53] anyhow, glad to work on restbase/cassandra : [16:54:55] :) [16:57:08] elukey: the idea would be test upgrading cassandra to 2.2.5 on aqs first [16:57:13] urandom: --^ [16:57:38] make sense, count me in! [16:59:48] nuria_: I made a mistake, I thought this was built-in to foreach, but we need an extra binding to make tables fast: https://github.com/mbest/knockout-table [17:00:01] I'll add you and mforns to the reviews and I'm working on this last one, should be relatively easy [17:22:33] Analytics, Wikipedia-iOS-App-Product-Backlog, iOS-app-feature-Analytics, iOS-app-Bugs, iOS-app-v5.0.3-Disco: Invalid pageview data for iOS app - https://phabricator.wikimedia.org/T131824#2179692 (Fjalapeno) @Tbayer additionally you may be able to search the user agent string for "iOS" and filte... [17:55:07] https://www.igvita.com/2012/02/06/sstable-and-log-structured-storage-leveldb/ - this helped my A TON to understand SSTables [17:55:15] amazing [17:57:31] joal --^ (even if you know these things really well :) [17:57:42] going offline team!!! byyyeeeeeeee o/ [17:59:41] Analytics, WMDE-Analytics-Engineering, Wikidata: [Task] dashboard showing browser usage distribution for Wikidata - https://phabricator.wikimedia.org/T130102#2187689 (Nuria) [18:00:39] Analytics-Cluster, Analytics-Kanban: Story: Community has periodic browser stats report generated from Hadoop data - https://phabricator.wikimedia.org/T69053#2187691 (Nuria) Updated related ticket, closing. this one. [18:00:50] Analytics-Cluster, Analytics-Kanban: Story: Community has periodic browser stats report generated from Hadoop data - https://phabricator.wikimedia.org/T69053#2187692 (Nuria) Open>Resolved [18:00:52] Analytics-Kanban, Analytics-Wikistats, Reading-Admin: {lama} Wikistats traffic reports 2.0 - https://phabricator.wikimedia.org/T107175#2187693 (Nuria) [18:01:15] bye elukey! [18:03:13] (PS1) Mforns: Remove frequency and timeboxed config options [analytics/limn-language-data] - https://gerrit.wikimedia.org/r/282213 (https://phabricator.wikimedia.org/T132062) [18:04:43] (PS1) Mforns: Remove frequency and timeboxed config options [analytics/limn-extdist-data] - https://gerrit.wikimedia.org/r/282214 (https://phabricator.wikimedia.org/T132062) [18:06:51] (PS1) Mforns: Remove frequency and timeboxed config options [analytics/limn-ee-data] - https://gerrit.wikimedia.org/r/282215 (https://phabricator.wikimedia.org/T132062) [18:09:08] (PS1) Mforns: Remove frequency and timeboxed config options [analytics/limn-multimedia-data] - https://gerrit.wikimedia.org/r/282218 (https://phabricator.wikimedia.org/T132062) [18:10:39] Analytics-Kanban, Patch-For-Review: Remove frequency and timeboxed options from limn-x-data repos config files {lama} - https://phabricator.wikimedia.org/T132062#2187730 (mforns) Those are the changes needed to totally remove the frequency and timeboxed config options. Note limn-mobile-data config was alr... [18:15:46] (CR) Nuria: "Nice job splitting config but I think is still being eagerly loading all of it. Let me verify." [analytics/dashiki] - https://gerrit.wikimedia.org/r/282183 (owner: Milimetric) [18:46:43] milimetric: yt? [18:47:27] hey nuria_, yes but I'm finishing up a meeting [18:47:30] 15 min? [18:47:33] sure [18:48:33] Analytics, WMDE-Analytics-Engineering, Wikidata: [Task] dashboard showing browser usage distribution for Wikidata - https://phabricator.wikimedia.org/T130102#2187851 (Lydia_Pintscher) I don't have any data to look at - either for Wikidata or any of the other projects. So I have no idea. [18:56:44] ottomata: Hey ! [18:57:25] (CR) Nuria: "Ya, this makes a lot of sense. For example, the "vital-signs" layout only needs utils/arrays which will be eagerly loaded inside the scrip" [analytics/dashiki] - https://gerrit.wikimedia.org/r/282183 (owner: Milimetric) [18:57:33] ottomata: I'm tired tonight, can we take time tomorrow? [18:58:50] (CR) Nuria: Split up utils (1 comment) [analytics/dashiki] - https://gerrit.wikimedia.org/r/282183 (owner: Milimetric) [18:59:17] urandom: I'm gonna head off, I'll try to have some more info to talk tomorrow [19:01:08] joal: hey! [19:01:09] sure thang! [19:01:18] in my morn? will ping ya [19:01:22] cool [19:01:37] ottomata: I've managed to explain the thing to nuria_ and elukey, [19:01:48] ottomata: I'll sleep on it, then explain again :) [19:02:21] nuria_: ok, back [19:03:22] k :) [19:03:30] Analytics, WMDE-Analytics-Engineering, Wikidata: [Task] dashboard showing browser usage distribution for Wikidata - https://phabricator.wikimedia.org/T130102#2187873 (Nuria) @Lydia_Pintscher: Ah! Sorry, I should have included this: The data is all on wedrequest table of wmf db on hive for all projec... [19:03:42] have a good end of day a-team [19:03:46] nite! [19:03:52] laters joal! [19:04:04] milimetric: looked at your utils change , nice, makes loads of sense [19:04:10] milimetric: made 1 comment [19:04:33] k, /me checks [19:04:49] milimetric: didn't +1, but change looks good :) [19:05:05] nuria_: did you see the formatting one too? https://gerrit.wikimedia.org/r/#/c/282200/ [19:05:39] I had not understodd at first why not reuse fakeHour to insert 00, but then understand that they can be and are actually disjointed [19:05:49] nuria_: did you bower install? I added one dependency [19:05:50] milimetric: -^ [19:05:57] which is one of the reasons I split it up, 'cause there were too many deps [19:05:58] milimetric: i did [19:06:02] weird [19:06:06] tests pass for me [19:06:14] what's up with this damn thing, wanna batcave and look at these tests [19:06:34] joal: yeah, the unique thing uses that but has a YYYYMMDD format in cassandra [19:06:43] true milimetric [19:06:45] it's a little confusing, not gonna lie :) [19:06:59] milimetric: yes, that's true as well, but does the job :) [19:07:00] but it's such a simple codebase that I don't think it matters [19:07:07] we can refactor when it grows up past tiny [19:07:16] sounds good :) [19:07:24] I'm gonna +1 [19:07:28] thx [19:08:12] milimetric: let me run tests again but i suspect node dependency joy [19:08:14] (CR) Joal: [C: 1] "LGTM, merge when needed." [analytics/aqs] - https://gerrit.wikimedia.org/r/281982 (https://phabricator.wikimedia.org/T131641) (owner: Milimetric) [19:11:19] milimetric: ah no, this is happen before its because getTime returns different values in your system and mine [19:11:31] argh [19:11:33] sorry [19:11:33] joal: sounds good. [19:11:38] milimetric:i fixed this before [19:11:41] I should've just utc() it [19:11:44] cool urandom :) [19:11:46] milimetric: you just have to give it a tz [19:11:46] stupid test, I'll fix [19:11:51] * joal is gone for real ! [19:12:00] milimetric: i can do it [19:12:06] uh, sure [19:12:09] thx [19:12:15] then you're welcome to merge [19:12:32] milimetric: k will do [19:12:35] nuria_: but you don't have to give it a timezone, you can just do .utc() after the moment(date) thing [19:12:42] k [19:18:46] milimetric: ah no, getTime() returns UTC, let me see: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Date/getTime [19:19:46] (PS1) Amire80: Add sorted errors [analytics/limn-language-data] - https://gerrit.wikimedia.org/r/282228 [19:27:53] Analytics, Wikipedia-iOS-App-Product-Backlog, iOS-app-feature-Analytics, iOS-app-Bugs, iOS-app-v5.0.3-Disco: Invalid pageview data for iOS app - https://phabricator.wikimedia.org/T131824#2187907 (JMinor) @JAllemandou @madhuvishy The fix on the client will take a bit to develop and release. Is... [19:34:52] !log restarting eventlogging so it runs out of the scap deploy in eventlogging/analytics [19:34:54] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master [19:37:10] (PS2) Amire80: Add sorted errors [analytics/limn-language-data] - https://gerrit.wikimedia.org/r/282228 [19:43:35] (PS2) Nuria: Split up utils [analytics/dashiki] - https://gerrit.wikimedia.org/r/282183 (owner: Milimetric) [19:44:09] Analytics-EventLogging, Analytics-Kanban, Scap3 (Scap3-Adoption-Phase1): Stop using global eventlogging install on hafnium (and any other eventlogging lib user) - https://phabricator.wikimedia.org/T131977#2187945 (Ottomata) [19:44:11] Analytics-EventLogging, Analytics-Kanban, scap, Patch-For-Review, Scap3 (Scap3-Adoption-Phase1): Use scap3 to deploy eventlogging/eventlogging - https://phabricator.wikimedia.org/T118772#2187944 (Ottomata) [19:44:18] milimetric: please take a look to make sure this is what you meant: https://gerrit.wikimedia.org/r/282183 [19:44:24] k [19:44:38] (PS2) Nuria: Make number format configurable [analytics/dashiki] - https://gerrit.wikimedia.org/r/282200 (https://phabricator.wikimedia.org/T131965) (owner: Milimetric) [19:45:02] yeah, that's a better test, thanks nuria_ [19:45:19] milimetric: k merging [19:45:32] Analytics-EventLogging, Analytics-Kanban, scap, Patch-For-Review, Scap3 (Scap3-Adoption-Phase1): Use scap3 to deploy eventlogging/eventlogging - https://phabricator.wikimedia.org/T118772#1808930 (Ottomata) DONE! (Well mostly) The Analytics eventlogging role is now deployed to eventlog1001 us... [19:45:38] (CR) Nuria: [C: 2 V: 2] Split up utils [analytics/dashiki] - https://gerrit.wikimedia.org/r/282183 (owner: Milimetric) [19:46:36] almost done with this table improvement [19:48:43] (CR) Nuria: Make number format configurable (1 comment) [analytics/dashiki] - https://gerrit.wikimedia.org/r/282200 (https://phabricator.wikimedia.org/T131965) (owner: Milimetric) [19:49:31] yeehaw a-team! our eventlogging instance is deployed using scap3! [19:49:35] docs have been updated [19:49:36] :] [19:50:35] nuria_, are you reviewing Dan's other change? [19:50:49] mforns: the formatting one? yes, i just did [19:50:55] ok ok [19:51:01] ottomata: thank you [19:53:20] nuria_: about that nit, kmb is kind of annoying to me too, I hate abbreviations, but the long version is sooo long :) "thousandsMillionsBillions" [19:54:14] jaja [19:55:21] If we do not want to change name let's at least add a comment as to what is expected...? [19:55:46] changing name would be best but that is along one [20:07:11] (PS3) Milimetric: Make number format configurable [analytics/dashiki] - https://gerrit.wikimedia.org/r/282200 (https://phabricator.wikimedia.org/T131965) [20:07:57] (PS1) Milimetric: Improve rendering speed of table-timeseries [analytics/dashiki] - https://gerrit.wikimedia.org/r/282232 [20:08:20] ok, nuria_, added comments and I'll add you to that last one which is the faster tables [20:08:23] and we should be good to go [20:08:30] i'll go merge marcel's patches now [20:08:40] :] [20:09:58] (CR) Nuria: [C: 2 V: 2] "Test pass. Merging." [analytics/dashiki] - https://gerrit.wikimedia.org/r/282200 (https://phabricator.wikimedia.org/T131965) (owner: Milimetric) [20:11:22] (CR) Milimetric: [C: 2 V: 2] Add 'execute' option to RU config [analytics/reportupdater] - https://gerrit.wikimedia.org/r/281682 (https://phabricator.wikimedia.org/T131849) (owner: Mforns) [20:17:07] heh, mforns I like how you have the same superstition as me that date logic will fail around year/month boundaries :) [20:17:42] milimetric, mmm yes, why do you say that sherlock? :] [20:18:00] oh, the tests? [20:18:02] https://gerrit.wikimedia.org/r/#/c/281815/1/test/writer_test.py [20:18:03] yea [20:18:05] hehe [20:19:06] joal: frontend RESTBase is deployed: https://wikimedia.org/api/rest_v1/?doc#!/Unique_devices_data/get_metrics_unique_devices_project_access_site_granularity_start_end [20:20:56] (CR) Milimetric: [C: 2 V: 2] Add max_data_points option (1 comment) [analytics/reportupdater] - https://gerrit.wikimedia.org/r/281815 (https://phabricator.wikimedia.org/T131849) (owner: Mforns) [20:21:58] thanks milimetric! [20:23:19] sorry for kind of surrendering on the more substantial part of that [20:23:39] since the tests are happy I'm happy for now, and I'll think about it more carefully next time I have to make an update [20:23:51] milimetric, the update_results part took a long time for me to understand and as we changed that part a couple times in the last months, there were some lines that didn't make sense any more, I think it was because of my last change before this [20:24:12] so, I almos rewrote it [20:24:16] np [20:25:37] yeah, I agreed with the new structure, just didn't have the brainpower to think it through right now [20:28:53] milimetric the problem I think was that when I wrote the 'allow removing columns' change this method was getting too spaghetti.. plus the max-data-points change, I thought it needed some organizing [20:34:24] Analytics-EventLogging, MediaWiki-Vagrant: EventLogging vagrant role fails to provision - https://phabricator.wikimedia.org/T131085#2188143 (AdHuikeshoven) @Ottomata thanks for looking it to this. I have good news. The good news is that my machine is now provisioned with eventlogging without errors. I s... [21:26:10] nuria_: wanna look at https://gerrit.wikimedia.org/r/#/c/282232/ and then I can deploy the new changes? [21:27:23] (CR) Milimetric: [C: 2 V: 2] Make browser reports show rates instead of percentages [analytics/reportupdater-queries] - https://gerrit.wikimedia.org/r/282185 (https://phabricator.wikimedia.org/T130406) (owner: Mforns) [21:29:41] Analytics-EventLogging, Analytics-Kanban, scap, Patch-For-Review, Scap3 (Scap3-Adoption-Phase1): Use scap3 to deploy eventlogging/eventlogging - https://phabricator.wikimedia.org/T118772#1808930 (greg) >>! In T118772#2187944, @Ottomata wrote: > There are misc users of the eventlogging codebase... [21:59:09] Analytics, Operations, hardware-requests, Patch-For-Review: eqiad: (3) AQS replacement nodes - https://phabricator.wikimedia.org/T124947#1970805 (RobH) [22:11:35] (CR) Milimetric: "after merging this I changed all the percent-outputting reports by dividing the last column by 100. In case I screwed something up, I lef" [analytics/reportupdater-queries] - https://gerrit.wikimedia.org/r/282185 (https://phabricator.wikimedia.org/T130406) (owner: Mforns) [22:11:50] (CR) Milimetric: [C: 2 V: 2] Remove frequency and timeboxed config options [analytics/limn-flow-data] - https://gerrit.wikimedia.org/r/282187 (https://phabricator.wikimedia.org/T132062) (owner: Mforns) [22:12:11] (CR) Milimetric: [C: 2 V: 2] Remove frequency and timeboxed config options [analytics/limn-edit-data] - https://gerrit.wikimedia.org/r/282189 (https://phabricator.wikimedia.org/T132062) (owner: Mforns) [22:12:26] (CR) Milimetric: [C: 2 V: 2] Remove frequency and timeboxed config options [analytics/limn-language-data] - https://gerrit.wikimedia.org/r/282213 (https://phabricator.wikimedia.org/T132062) (owner: Mforns) [22:12:47] (CR) Milimetric: [C: 2 V: 2] Remove frequency and timeboxed config options [analytics/limn-multimedia-data] - https://gerrit.wikimedia.org/r/282218 (https://phabricator.wikimedia.org/T132062) (owner: Mforns) [22:12:59] (CR) Milimetric: [C: 2 V: 2] Remove frequency and timeboxed config options [analytics/limn-ee-data] - https://gerrit.wikimedia.org/r/282215 (https://phabricator.wikimedia.org/T132062) (owner: Mforns) [22:13:12] (CR) Milimetric: [C: 2 V: 2] Remove frequency and timeboxed config options [analytics/limn-extdist-data] - https://gerrit.wikimedia.org/r/282214 (https://phabricator.wikimedia.org/T132062) (owner: Mforns) [22:13:14] thank you! [22:16:36] (CR) Milimetric: [C: 2 V: 2] "merging for hopefully deploy soon" [analytics/aqs] - https://gerrit.wikimedia.org/r/281982 (https://phabricator.wikimedia.org/T131641) (owner: Milimetric) [22:16:46] (CR) Milimetric: [C: 2 V: 2] Fix double-decode issue in pageview article route [analytics/aqs] - https://gerrit.wikimedia.org/r/280934 (https://phabricator.wikimedia.org/T131369) (owner: Milimetric) [22:21:39] (PS3) Milimetric: Strip out www. in front of project names [analytics/aqs] - https://gerrit.wikimedia.org/r/275681 (https://phabricator.wikimedia.org/T127030) [22:22:11] (CR) Milimetric: [C: 2 V: 2] Strip out www. in front of project names [analytics/aqs] - https://gerrit.wikimedia.org/r/275681 (https://phabricator.wikimedia.org/T127030) (owner: Milimetric) [22:31:47] Analytics-Cluster, Operations: setup stat1004/WMF4721 for hadoop client usage - https://phabricator.wikimedia.org/T131877#2188546 (RobH) [22:34:17] (CR) Yurik: Strip out www. in front of project names (1 comment) [analytics/aqs] - https://gerrit.wikimedia.org/r/275681 (https://phabricator.wikimedia.org/T127030) (owner: Milimetric) [22:36:42] (CR) Milimetric: Strip out www. in front of project names (1 comment) [analytics/aqs] - https://gerrit.wikimedia.org/r/275681 (https://phabricator.wikimedia.org/T127030) (owner: Milimetric) [22:37:16] milimetric: looking [22:38:12] milimetric: yt? [22:38:43] partly, whatsup [22:40:35] nuria_: yea, what's up [22:40:56] milimetric: I think you do not need to add the table code wrapped [22:41:04] milimetric: you can add a shim to requirejs config [22:41:13] milimetric: and install package via bower [22:41:14] i tried it unwrapped and it didn't work with shims [22:41:18] milimetric: like this: [22:41:57] there's no bower package for this one [22:42:05] so that complicates things [22:42:08] you have to ref the git [22:42:27] milimetric: wait i saw one on github.. maybe was a different table? [22:43:03] yeah, there's a different knockout-table in bower [22:44:21] milimetric: right, that was another one [22:44:59] Analytics-Cluster, Operations: setup stat1004/WMF4721 for hadoop client usage - https://phabricator.wikimedia.org/T131877#2188623 (RobH) a:RobH>Ottomata [22:45:47] Analytics-Cluster, Operations: setup stat1004/WMF4721 for hadoop client usage - https://phabricator.wikimedia.org/T131877#2181451 (RobH) puppet/salt keys signed and ready for service implementation. I've assigned to @Ottomata since he filed the initial #hardware-requests . [22:46:49] k, i gotta run, nite! [22:48:54] milimetric: ciao [22:53:22] (CR) Nuria: [C: 2 V: 2] "Looks like there is no bower component for this table so I think wrapping on require js is the only clean option" [analytics/dashiki] - https://gerrit.wikimedia.org/r/282232 (owner: Milimetric) [22:58:33] !log deployed browser-reports master branch to labs [22:58:35] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master [22:59:45] milimetric: tested, merged & deployed [23:00:12] Analytics, Wikipedia-iOS-App-Product-Backlog, iOS-app-feature-Analytics, iOS-app-Bugs, iOS-app-v5.0.3-Disco: Invalid pageview data for iOS app - https://phabricator.wikimedia.org/T131824#2188743 (Mhurd) @madhuvishy Sound ok if I open a pull request modifying this string? https://github.com/w... [23:11:02] yay, thanks nuria, I'll test on mobile