[11:23:26] (PS1) Mforns: Productionize failure types by user type [analytics/limn-edit-data] - https://gerrit.wikimedia.org/r/212276 (https://phabricator.wikimedia.org/T91123) [13:05:56] Good morning ottomata [13:06:23] mooornin [13:07:15] How is the update of our biutifull cluster going ? [13:08:25] i only did the one node yesterday, it was getting later and I didn't want to keep doing [13:08:30] going to do 1030 today [13:08:37] worked more on reqstats [13:09:05] oki :) [13:12:33] oof joal, but i don't really want to [13:12:37] we have 4 unhealthy nodes right now [13:12:40] due to disk space [13:12:43] balancer is running [13:12:47] i think we need to delete some data [13:12:54] ok [13:13:03] What do you want us to go for first ? [13:13:07] raw, probably ? [13:13:30] I said last time, there is some user data that is big as well [13:15:45] yeah let's check that first [13:15:52] am doing that now [13:15:53] also on phone... [13:16:10] What first, raw or user ? [13:16:21] user [13:16:34] ok [13:16:39] my dir is 3T soooo [13:16:40] :) [13:17:38] 1T usefull [13:17:48] I think we should ask halfack a [13:17:56] and bob west [13:18:00] to clean a bit [13:18:09] They represent the big ones [13:18:36] yes [13:19:24] I'll send and email to them with you in copy [13:19:36] Should I add analytics internal ? [13:20:34] sure, i can do it too though, just on the phone at the moment :) [13:20:41] np, doing now [13:22:54] k [13:23:26] done [13:23:36] We should still go for raw webrequests, no ? [13:23:45] well, let's see how much we can clear out [13:23:47] lemme poke around [13:25:51] hm -> automatic deletion of raw data older than 1 month, right ottomata ? [13:25:57] yes [13:26:00] that happens now [13:26:03] k [13:26:06] it should! anyway! [13:26:10] if you see it isn't that would be a problem! :) [13:26:33] So it only leaves us refined webrequest [13:26:43] well, we could delete more raw if we wanted to [13:26:47] as long as we refine fast enough [13:27:46] Well, so far, we refine on the flow [13:28:53] Now still the same question from me: the refinement should-it really be the same for every webrquest type ? [13:28:57] hrmf ... [13:28:59] ha joal, you have 2T :) [13:29:05] I do :) [13:29:12] I'll remove them ;-P [13:29:35] ja but, whoa, halfak and bob are big offenders [13:30:03] they are together taking up > 100 T! [13:30:12] :) [13:30:26] That's why I emailed them specifically [13:30:31] cool, thank you! [13:30:31] :) [13:30:34] np [13:30:51] i don't really want to take another node down while we have 4 unlhealthy and one dead (1028) [13:30:53] we are down 5 nodes atm. [13:30:58] gonna wait. [13:31:05] sounds a good idea [13:32:33] I'm back to 10g :) [13:32:56] great ;) [13:32:57] :) [13:33:19] If you don't restart machines, would you have a minute to discuss camus config ? [13:33:31] sure, gimme just a few, still deleting my cra [13:33:32] crap [13:33:39] np [13:33:43] ping me when ready [13:35:52] k [13:35:54] joal: ready [13:36:16] k, to the batcave ? [13:37:44] k [13:59:06] Analytics-Tech-community-metrics: Community Bonding evaluation for "Allowing contributors to update their own details in tech metrics" - https://phabricator.wikimedia.org/T98045#1298949 (Dicortazar) @Sarvesh.onlyme, please remember that you need to work on the report bonding period. The deadline for this is... [14:12:11] (CR) Milimetric: [C: 2 V: 2] Productionize failure types by user type (1 comment) [analytics/limn-edit-data] - https://gerrit.wikimedia.org/r/212276 (https://phabricator.wikimedia.org/T91123) (owner: Mforns) [15:02:41] (Abandoned) Milimetric: [WIP] Add tabular layout with generic router [analytics/dashiki] - https://gerrit.wikimedia.org/r/196230 (owner: Milimetric) [15:12:29] hey milimetric, would like a python brainbounce now :) [15:12:31] if you got a sec [15:12:37] let's do it! [15:12:38] batcave [15:18:28] ottomata: you were right, camus only supports timestamp partitioning :( [15:24:57] joal: naw [15:24:59] i mean, by default [15:25:05] but you can implement your own partitioner [15:25:13] hmmmm [15:25:19] Need to look into that :) [15:34:38] halfak: Heya ! [15:34:44] halfak: Tha [15:34:51] halfak: Thanks for cleaning :) [15:35:02] No prob. [15:35:15] :) [15:35:16] Will you have a minute after my standup ? [15:35:28] Negative. In conference mode. I could do email though. [15:35:36] np, later then :) [15:37:06] joal, you are coming to the hackathon in Lyon, right? [15:38:25] negative. Couldn't manage to get my wife agree on both wikimania and lyon hackathon :) [15:38:37] halfak: --^ [15:39:15] Boo. OK. Understood. [15:39:55] Hoever if you want to come and spend some time n britanny, I welcome you with pleasure ! [15:40:52] joal, if I didn't already plan to be away from my wife too long, I'd make the side-trip. But then again, I'll be back in Europe in August. [15:41:17] So, I might still take you up on that this summer. [15:41:28] For sure ! [15:41:34] Would be very nicve :) [17:02:23] milimetric, I already had reviewed the changes prior to the last Dashiki merge [17:02:38] milimetric, I'm having another look right now [17:03:08] oh, good [17:03:20] yeah, lemme know if you have questions. gonna run out and get a burger [17:20:00] joal: I want to test the updated spark job. I see I can do it with spark-submit, but one of the arguments is the path to refinery jar. how do i build the jar? do you have a process for doing this? [17:20:51] step 1: find nice, ground glass [17:21:09] yuvipanda: sshh [17:21:22] * yuvipanda autocorrects madhuvishy to ssh [17:22:02] * madhuvishy shoos yuvipanda [17:22:22] * T9AutoCorrect autocorrects madhuvishy to 'shoes' [17:22:51] T9AutoCorrect: how's new york [17:23:05] haven't seen much of it - my friend got sick so I'm mostly taking care of her [17:23:35] T9AutoCorrect: ouch. [17:24:05] madhuvishy: I was looking at doing hackerschool but apparently that's fuckall if you're on a H1B :) [17:24:28] T9AutoCorrect: you mean we can't do it? [17:24:34] basically [17:24:35] yes [17:24:40] T9AutoCorrect: why [17:25:14] madhuvishy: http://www.nolo.com/legal-encyclopedia/how-taking-approved-leave-affects-your-h-1b-employment.html and plenty of other things on the web [17:25:43] T9AutoCorrect: Let's get a green card. [17:25:51] madhuvishy: going to take a while :) [17:26:05] of course, if one doesn't care about the H1B so much it's not a big deal :P [17:26:14] but let's see. NY is nice, and I want to spend at least a month here [17:26:20] I can totally do that without going to HS [17:27:11] T9AutoCorrect: Yup! [17:29:23] madhuvishy: anyway, off to brooklyn bridge - that's workspace today :D [17:29:41] * madhuvishy is jealous [17:30:00] madhuvishy: at worst I'll be back here in October for 2 weeks :) [17:30:14] yuvipanda: get pizza at Julianna's. It's in Dumbo by the park. On the brooklyn side of the bridge. [17:30:41] ok! [17:33:56] hey madhuvishy [17:34:03] hey joal [17:34:29] So, you are in the process of testing :) [17:34:34] That's awesome :) [17:34:39] Analytics-EventLogging, Analytics-Kanban: ContentTranslationError event logging table is not receiving new events - https://phabricator.wikimedia.org/T98842#1299444 (Milimetric) Open>Resolved The patch to improve batch inserts was deployed Monday, so this issue should be fixed. Events will still no... [17:34:46] joal: ha ha i also have to oozify it [17:34:54] yup [17:34:58] oozify is a terrible name [17:35:18] it is, but reality of the thing is even worse ! [17:35:21] :D [17:35:24] joal: of course [17:35:33] So [17:35:38] How may I help ? [17:36:23] joal: yeah, so this spark-submit thing wants a jar path. i dont know how to build that jar [17:36:51] Right [17:36:55] joal: also, I put scopt into the pom.xml file as a dependency, and expected it to automatically resolve - but that din't happen [17:37:13] So, few tricks here and there:) [17:37:27] joal: yes please [17:38:06] To be able to get jars in your local environment, a little comment in the main pom file is needed [17:38:43] https://gist.github.com/jobar/b5297bad4681b8489b1f [17:39:02] The part I commented is to prevent maven to use the common repos [17:39:29] joal: aah, and this should go in the pom for refinery or refinery-job (i notice there are poms per module and a global one) [17:39:33] Now, doing that also makes an error easy to come: not asking ottomata to add the jar in archiva before deploy [17:39:42] global one [17:40:06] the per module poms don't have repositories settings [17:40:07] joal: cool. but the dependency can be in the local one? [17:40:09] (in our conf) [17:40:33] Yes, module pom uses the main pom as root [17:40:50] pom pom pom pom as would Bethoven sing [17:40:52] milimetric, http://vega.github.io/ !!!!!!!!!!!!!!!!!!!!!!! [17:41:21] \o/ yurik :) [17:41:26] joal: ha ha okay let me try this. [17:41:37] yurik: woa [17:41:38] :) [17:41:45] gonna mention this in SoS right now [17:41:48] i just spoke with their main guy [17:41:53] lots of things in the pipeline [17:41:59] very very excited [17:42:06] check out vegalite [17:42:17] and check out the last example of the vega2 - interactive [17:42:28] madhuvishy: Something to test before deploying is to build the jar from stat1002 [17:43:13] like that, if there is a dependency error, it shows up [17:44:04] joal: deploying is so far off :P but yeah, alright. how do i build the refinery-job jar though? [17:45:24] dependency issue solved ? [17:45:39] Analytics-Kanban: Quarterly tracked animals - https://phabricator.wikimedia.org/T99108#1299478 (kevinator) p:Triage>Normal [17:47:16] madhuvishy: --^ [17:47:18] ? [17:47:55] madhu, I need to get for dinner [17:48:02] ottomata: do you know if the ulsfo problems are causing problems in our pipeline? [17:48:50] Building the jar is either for testing purpose, it can be one using intellij (or your prefered ide), or using maven: mvn clean package in the source folder [17:49:04] Will be back soon [17:50:47] joal: sorry was away for a sec. sure [17:50:49] milimetric: probably [17:51:00] if you mean, are we missing some data from ulsfo [17:51:06] last i heard though, ulsfo was drained of traffic [17:51:15] yeah, missing data or maybe the "OK" flag file thing that oozie writes [17:51:25] drained only for US/Europe I heard [17:51:34] It's still serving Asia, in a flaky way [17:51:38] ah, yeah [17:51:53] so, probalby yes, we are missing data from ulsfo [17:51:56] shoudln't affect oozie though [17:52:23] yurik: took a look at everything, yes very exciting but still early days. Some of their graphs are really bad on performance. We should exercise caution with Vega 2. [17:52:58] milimetric, i think we should move ahead on vegalite [17:53:04] that's the one i'm mostly excited about [17:53:11] well i was thinking we should check to make sure the duplicate / missing detection is working ok and not spooked by the lack of data [17:53:28] yurik: it's a bit limited but I think that's totally ok [17:53:37] I agree, good thing to have out there, would be much friendlier [17:53:42] milimetric, exactly! [17:53:46] perfect for most uses [17:53:53] it even has a "shorthand" [17:57:06] yurik: I was thinking if we do that we might want to do yaml and just translate to json on our end [17:57:30] sure, sounds good, they are one-to-one translatable [18:01:15] joal: dependency issue solved. [18:01:31] milimetric, there's some commented code in https://gerrit.wikimedia.org/r/#/c/198169/21/src/components/a-b-compare/compare-timeseries.js, are you keeping it for later use? [18:02:23] mforns: yes and no [18:02:32] the way the data flows from the compare-layout down is bad [18:02:52] I ran into a problem that Knockout 3.4 is going to solve supposedly [18:03:00] namely collisions between dependency triggers [18:03:14] and there's no great elegant solution, so the code I ended up with is really hack [18:03:32] this code commented out here would be a little cleaner (when finished, it's in-progress as commented here) [18:03:41] but it wouldn't work until the new Knockout comes out [18:03:49] milimetric, ok [18:04:16] I think I'd leave it there and have a clean-house once 3.4 is released [18:04:30] cool [18:05:53] milimetric, must watch - after 10:00 https://www.youtube.com/watch?v=GdoDLuPe-Wg [18:05:56] (10th min) [18:27:36] joal: i'm going to delete raw march data [18:27:37] s'ok? [18:29:27] ottomata, in hadoop? don't until tomorrow [18:30:55] (CR) Mforns: [C: 2 V: 2] "LGTM" [analytics/dashiki] - https://gerrit.wikimedia.org/r/198169 (https://phabricator.wikimedia.org/T94424) (owner: Milimetric) [18:31:11] yurik: raw data [18:31:13] not refined [18:31:26] this data is already in the wmf.webreuest table [18:31:29] ottomata, ok [18:31:31] i'm deleting it from wmf_raw.webrequset [18:31:31] nvm :) [18:31:35] oki [18:31:39] we are running out of space on Hadoop, hoping some users can clear up some space! [18:31:42] waiting...:) [18:31:47] buying some time [18:32:01] ottomata, i did a few days ago - half a TB [18:32:50] thank you [18:32:57] bob is using like 60T :) [18:36:02] !log dropping wmf_raw webrequest data from march to save some HDFS space [18:37:58] (CR) Mforns: [C: 2 V: 2] Add rickshaw timeseries graph [analytics/dashiki] - https://gerrit.wikimedia.org/r/197590 (owner: Milimetric) [18:41:23] phew, got 21 nodes now :) [18:41:55] going to upgrade 1030 [18:41:59] well, reboot :) [18:42:15] no burning controllers! :P [18:42:42] (CR) Mforns: [C: 2 V: 2] Add Sunburst Visualizer [analytics/dashiki] - https://gerrit.wikimedia.org/r/197234 (owner: Milimetric) [18:44:53] (CR) Mforns: [C: 2 V: 2] [Review but DO NOT MERGE] Begin funnel layout [analytics/dashiki] - https://gerrit.wikimedia.org/r/196489 (owner: Milimetric) [18:45:01] ottomata, is there any way to get the *compressed* download size, as opposed to the raw size that varnish shows [18:45:12] oops, should've changed that commit msg [18:45:20] yurik: hm, not sure [18:45:57] not that i know of off the top of my head [18:46:31] ottomata, reason i'm asking is because all partners asking this - that's their first question - how much actual bandwidth will it be [18:46:53] so we can do per country graphs, but it won't have the true bandwidth [18:49:42] yurik: isn't that what response_size is? [18:49:51] %b [18:49:51] Size of response in bytes, excluding HTTP headers. In CLF format, i.e. a '-' rather than a 0 when no bytes are sent. [18:49:55] ottomata, isn't response_size non-compressed? [18:49:58] dunno [18:50:09] i don't know much about this part of varnish. is varnish doing compression? [18:51:01] ottomata, yes and no, it keeps data as compressed blobs, but temp uncompresses them to look at the headers etc [18:51:11] or could do more elaborate stuff with ESI (broken) [18:54:11] yurik: you might want to ask bblack [18:55:02] ottomata, will do. Do you know how we get response_size from varnish? I could ask on the varnish channel [18:56:12] we get it from the shared mem logs [18:56:13] %b [18:56:23] https://www.varnish-cache.org/docs/trunk/reference/varnishncsa.html [18:56:55] milimetric: another python fu q for you [18:56:56] yt? [18:57:01] yea [18:57:18] py-fu q 4 u :) [18:57:25] batcave? [18:57:42] k [18:57:43] ottomata: ^ [18:57:45] k [19:00:07] Analytics-Tech-community-metrics, Engineering-Community, Wikimedia-Hackathon-2015: A new events/meet-ups extension - https://phabricator.wikimedia.org/T99809#1299687 (Qgil) [19:11:39] thanks milimetric :) [19:11:41] GENERATOR [19:11:43] forgot about those [19:11:44] was doing list comps [19:12:29] Ahh python [19:13:58] milimetric: another quick q [19:14:01] (am googling) [19:14:10] best way to use unittest in a single file? [19:14:16] i'm not coding a module here, just a simple script [19:14:33] can I just make unittest classes at the bottom of my file? [19:14:52] ottomata: yeah, you can inherit from unittest.TestCase anywhere [19:15:00] (or doctests) [19:15:07] or doctests, yeah, those work [19:15:19] doctests...>>>: looking [19:15:20] ottomata: you might like doctests if you like docops [19:15:23] *docopts [19:15:43] heh, funny how he already used >>> :) [19:15:51] ottomata: are you in NYC? [19:16:15] hahahah [19:16:21] yes! [19:16:25] ottomata: haha, me too :) [19:16:28] oh! [19:16:32] where? whatcha doin? [19:16:35] * yuvipanda is sitting under the brooklyn bridge [19:16:44] my friend is graduating from NYU tomorrow [19:16:46] so here for that [19:16:56] cool! [19:28:11] milimetric: how do I run a unit test taht is in a singel file? [19:28:13] i'm trying [19:28:17] python -m unittest file.py [19:28:38] are you using doctest? [19:28:57] no [19:29:04] class VarnishstatsTest(unittest.TestCase): [19:29:05] def test_count_requests(self): [19:29:05] ... [19:29:46] gotcha, i'm looking it up, I forget [19:30:35] i'm googline bunches, not finding it [19:31:52] I used to use nose test runner [19:32:16] ottomata: yeah, i remember this being annoying [19:32:26] how about python -m unittest file [19:32:29] (without the py) [19:32:37] (trying too now) [19:33:02] Ran 0 tests in 0.000s [19:33:04] tried that [19:33:08] no errors, just no tests run [19:33:25] one sec, i'm trying too [19:34:58] ottomata: that worked for me [19:35:02] so, a file called "test.py" [19:35:13] with this in it: [19:35:14] what if your file is not called test.py [19:35:14] ? [19:35:20] import unittest [19:35:20] [19:35:20] [19:35:20] >>class TestOne(unittest.TestCase): [19:35:20] >> def test_one(self): [19:35:21] assert False [19:35:33] I ran it with "python -m unittest test" [19:35:37] trying [19:35:46] I think both the file and the method and the class have to have "test" in it [19:35:53] otherwise you need to override the way it "finds" tests [19:36:05] hm, but i don't want more than one file [19:36:25] yes, there are other params you can pass to override that, reading [19:36:33] python -m unittest -h [19:36:59] hm, this ran my test [19:36:59] python -m unittest afile [19:37:13] afile.py had your test class [19:38:05] ottomata: maybe they have to start with "Test" [19:38:08] the class name [19:38:15] AH [19:38:17] my test class [19:38:17] yes [19:38:22] I wasn't able to use the -p (pattern) option [19:38:26] it just seems to... not work :) [19:38:53] yes that does it [19:38:54] got it [19:38:55] thank you! [19:53:53] madhuvishy: I am finally HERE ! [19:54:03] Took longer than expected, but made it back :) [19:54:03] joal: yay [19:54:11] no problem. i went for lunch too [19:54:27] k [19:54:32] joal: fyi, i am looping through and restarting nodes now [19:54:34] So, what's up ? [19:54:48] cool ottomata b [19:54:53] Working fine ? [19:54:55] joal: i tried to do mvn clean package on the cli - and it's failing trying to download - Downloading: http://archiva.wikimedia.org/repository/mirrored/org/apache/maven/maven-plugin-api/2.0.6/maven-plugin-api-2.0.6.jar [19:55:09] huhu [19:55:13] if i do build project in intellij, worse things happen [19:55:24] wow, that's bad :) [19:55:33] I do not have trouble building with intellij [19:55:39] But maven cli should work [19:55:46] hmmm may be i dint set it up right [19:55:54] ottomata: any idea aboutr this one from archive --^ [19:55:58] or i have some version of java [19:56:29] joal: i see that this jar is not there - but i'm not sure where this dependency is [19:57:35] hm, madhuvishy, i suppose hm [19:57:36] https://archiva.wikimedia.org/#artifact-details-download-content/org.apache.maven/maven-plugin-api/2.0.6 [19:57:38] no jar, but pom? [19:57:39] wierd. [19:57:40] um [19:57:49] one sec. [19:57:55] ottomata: yeah. it's there for a higher version. 3.0.2 or sth [19:58:15] madhuvishy: You could do the same trick as for the packages, but for the plagins [19:58:23] hm, where is that dep coming from? [19:58:26] i don't have that jar either [19:58:28] but i can compile [19:58:40] ottomata: I don't need it either it seems [19:59:34] ottomata: that's what i'm not able to figure ou [20:00:10] madhuvishy: iunno [20:00:11] but [20:00:20] go ahead and blast that from your maven repo now [20:00:23] and try building again [20:01:44] ottomata: ummm blast from where? [20:02:42] !log tools tested new /etc/hosts on tools-bastion-01, puppet run produced no diffs, all good [20:02:44] err [20:02:45] sorry [20:03:35] madhuvishy: dunno, are you on mac? [20:03:38] if so [20:03:46] ~/.m2/repo [20:03:48] rm -rf ~/.m2/repository/org/apache/maven/maven-plugin-api/2.0.6 [20:04:35] ottomata: aah. okay doing [20:07:55] ottomata: joal now it fails the same way that intellij does. [20:07:59] https://www.irccloud.com/pastebin/5m70bdj7 [20:08:48] madhuvishy: First thing you should use jdk 1.7 [20:08:57] joal: aah. okay [20:08:58] We don't use 1.8 yet :)h [20:42:07] joal: downgraded to 1.7 - and switched scala-lang version from 2.10.0 to 2.10.4 - then it compiled [20:42:17]