[09:37:08] (PS4) Joal: Add webstatcollector projectview transformation [analytics/refinery] - https://gerrit.wikimedia.org/r/220426 (https://phabricator.wikimedia.org/T101118) [10:06:25] Quarry: Show all published queries in profile - https://phabricator.wikimedia.org/T77948#1404110 (Edgars2007) Temporary solution. In the "Recent queries" page add `?limit=5000` to page URL, so you (currently) get all queries. Then you can search for your username. Yes, it isn't a simple way, but at least it... [11:40:54] Analytics-Tech-community-metrics, Engineering-Community, ECT-July-2015: Check whether it is true that we have lost 40% of code contributors in the past 12 months - https://phabricator.wikimedia.org/T103292#1404495 (Aklapper) >>! In T103292#1399462, @Qgil wrote: > How does Metrics Grimoire scan Git/Gerr... [11:47:16] Analytics-Tech-community-metrics: Exclude pulled upstream code repositories from metrics - https://phabricator.wikimedia.org/T103984#1404499 (Aklapper) NEW [11:48:19] Analytics-Tech-community-metrics, Engineering-Community, ECT-July-2015: Check whether it is true that we have lost 40% of code contributors in the past 12 months - https://phabricator.wikimedia.org/T103292#1386502 (Aklapper) >>! In T103292#1387470, @Qgil wrote: > We should probably take the repository... [12:57:57] Analytics-Kanban: Vet data in intermediate aggregate {wren} [8 pts] - https://phabricator.wikimedia.org/T102161#1404685 (JAllemandou) Analysis done one one hour of data: 2015-06-24T00:00:00, using newly generated projectview and legacy projectcounts. It is to be noted that new projectview files don't contain... [13:26:22] Analytics-Tech-community-metrics, Engineering-Community, ECT-July-2015: Check whether it is true that we have lost 40% of code contributors in the past 12 months - https://phabricator.wikimedia.org/T103292#1404729 (Aklapper) > According to the [[ http://korma.wmflabs.org/browser/scm.html | "Authors" gr... [13:27:33] (CR) Ottomata: Add webstatcollector projectview transformation (1 comment) [analytics/refinery] - https://gerrit.wikimedia.org/r/220426 (https://phabricator.wikimedia.org/T101118) (owner: Joal) [14:24:37] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL 40.00% of data above the critical threshold [30.0] [14:24:51] Hey ottomata ! [14:24:53] heya! [14:24:57] that is probably me! [14:24:59] that el alarm [14:25:00] Thx foe the reviews :) [14:25:03] yup [14:25:07] should be ok in a sec. [14:25:10] Yeah, that would have been my next question ;) [14:27:36] (PS5) Joal: Add webstatcollector projectview transformation [analytics/refinery] - https://gerrit.wikimedia.org/r/220426 (https://phabricator.wikimedia.org/T101118) [14:27:46] Hey ottomata, hopefully the last one :) [14:27:54] Thx or spotting inconsistencies ! [14:29:30] haha, joal i see one more! can we make the actual names of the hql files match too? :D [14:29:40] hive_script_transform [14:29:43] archive_webstatcollector.hql [14:29:44] ottomata: :S sorry sir ;) [14:30:06] maybe this is fine? [14:30:06] hive_script_aggregate [14:30:06] javascript:; [14:30:06] javascript:; [14:30:06] [14:30:07] [14:30:07] javascript:; [14:30:08] javascript:; [14:30:09] projectview_hourly.hql [14:30:09] projectview_hourly.hql [14:30:10] eek [14:30:10] anyway ja [14:30:16] ? [14:30:29] first one: aggregate_projectview.hql [14:30:42] second: transform_projectview.hql [14:30:47] ottomata: --^ [14:30:49] ? [14:31:27] joal: maybe transform_projectcount? [14:31:40] it is generating the projectcount dataset, right? [14:31:45] yup [14:31:48] ok [14:31:58] Then transform_projectview_projectcounts [14:32:05] ? [14:32:06] hehe [14:32:24] transform projectcounts.hql is fine :) [14:32:53] pagview_aggregator_projectview [14:32:53] projectview_transform_projectcount [14:32:53] ? [14:32:58] aggregate* [14:33:39] we can make them even more explicit joal, if you like [14:33:48] transform_projectcount_to_projectview.hql [14:34:02] soryr, other way around:) [14:34:11] in pagecounts, 2 coords, hql files are: insert_pagecounts_hourly.hql and archive_projectcounts.hql [14:34:21] transform_projectview_to_projectcount.hql [14:34:21] aggregate_pageview_to_projectview.hql [14:34:26] yeah [14:34:33] I like explicit, so let's have aggregate_pageview_to_projectview.hql [14:34:47] transform_projectview_to_projectcounts.hql [14:35:04] ok [14:35:17] ok not have hourly in the title, right ? [14:38:34] (PS6) Joal: Add webstatcollector projectview transformation [analytics/refinery] - https://gerrit.wikimedia.org/r/220426 (https://phabricator.wikimedia.org/T101118) [14:38:38] ottomata: hopefully good this time :) [14:44:37] PROBLEM - Check status of defined EventLogging jobs on analytics1010 is CRITICAL Stopped EventLogging jobs: reporter/statsd [14:45:07] that's ok! [14:45:08] PROBLEM - Eventlogging /srv disk space on analytics1010 is CRITICAL: DISK CRITICAL - free space: / 14928 MB (84% inode=93%) [14:45:11] weird that that happens [14:45:12] huh! [14:45:13] cool! [14:46:47] ottomata: cool ? [14:47:23] sorry joal, am fixing some EL puppet stuff, with you shortly [14:47:31] np :) [14:59:18] (CR) Ottomata: [C: 2 V: 2] Add webstatcollector projectview transformation [analytics/refinery] - https://gerrit.wikimedia.org/r/220426 (https://phabricator.wikimedia.org/T101118) (owner: Joal) [14:59:27] Thx andrew ;) [14:59:37] Tell me, what was cool about hte EL alarm ? [15:01:49] PROBLEM - Check status of defined EventLogging jobs on graphite consumer on hafnium is CRITICAL Stopped EventLogging jobs: reporter/statsd [15:03:38] which one? :) [15:03:53] the first one that triggered today was def a real problem, but very short [15:04:11] i am changing hosts in the URIs [15:04:14] using IP address [15:04:23] had already tested that in labs [15:04:28] but made a mistake when testing in prod [15:04:33] so it broker for a little bit [15:04:35] broke* [15:04:42] the other ones, like analytics1010, are just dumb [15:04:51] consequence of monitoring classes not being very smart about where they are applied [15:04:58] makes sense [15:05:18] PROBLEM - Check status of defined EventLogging jobs on hafnium is CRITICAL Stopped EventLogging jobs: reporter/statsd [15:05:48] PROBLEM - Eventlogging /srv disk space on hafnium is CRITICAL: DISK CRITICAL - free space: / 1129 MB (12% inode=75%) [15:07:22] ok that is more interseting, checking on that [15:07:28] i'm sure that is not real, but it shoudlnt' fire [15:07:55] So tell me, just so that I follow: you are at stage 1 (multiple outputs for processors)? [15:08:27] haven't deployed that yet [15:08:31] that is running in beta [15:08:32] but not prod [15:08:32] ok [15:08:37] want to deploy that today [15:08:49] So how come changing hosts in URI ? [15:08:49] RECOVERY - Check status of defined EventLogging jobs on graphite consumer on hafnium is OK All defined EventLogging jobs are runnning. [15:08:58] RECOVERY - Check status of defined EventLogging jobs on hafnium is OK All defined EventLogging jobs are runnning. [15:09:01] to get 0mq rfom everywhere ? [15:11:04] ottomata: ottomata you sure you wanna deploy on Friday ? [15:11:08] maybe monday ?g [15:11:12] yeah i do! [15:11:13] heheh [15:11:16] it is early enough [15:11:17] ;) [15:11:28] it'll deploy code to eventlog1001, but wont' deploy and functional changes there [15:11:36] On my side, I'll wait monday for the project stuff [15:11:43] haha [15:11:43] k [15:12:58] joal: yes pretty much, zmq everywhere. also it has to do with how the variables are set in puppet. zmq doesn't like hostnames it seems, and 0.0.0.0 is too generic. so i'm having it just bind and use the main ipaddress everywhere by default [15:14:15] ottomata: how awefull :( [15:15:51] awful!?, its ok. :) [15:15:58] huhu :) [15:15:59] it is a facter variable in puppet [15:16:12] i also refactored the puppet stuff to make it easier to put different services on different nodes [15:16:17] * joal doesn't like static IPs in conf, [15:16:19] RECOVERY - Check status of defined EventLogging jobs on analytics1010 is OK All defined EventLogging jobs are runnning. [15:16:30] :D [15:16:38] ottomata: That's cool :) [15:17:36] what happened here, unicode? https://github.com/wikimedia/operations-puppet/commit/54fa58df5bb526f3e9ec15fd7080f58f52f25e0d [15:18:20] YOU WILL NEVER FIND OUT! HAH! [15:18:39] haha [15:19:25] milimetric: it's an ops secret. we both see the same diffs, BUT WE SEE MORE THAN YOU DO! [15:19:42] milimetric: i've been climbing a lot, and that is making my pinky get beefier, which may or may not have caused it to weigh down on a certain meta key while typing a space [15:20:41] :) [15:20:45] you crazy kids [15:22:09] A meta-space is not acceptable, that's for sure ! [15:30:56] ottomata: how do you run the processor with input from a file:// and output to a file:// ? I fail: [15:30:56] time ./eventlogging-processor --sid client-side events %q file:///home/milimetric/load.test.30k file:///home/milimetric/out.load.30k [15:36:00] you don't need an sid if you are not using tcp:// but that is not your problem [15:36:44] Analytics-Kanban, Reading-Web: Cron on stat1003 for mobile data is causing an avalanche of queries on dbstore1002 - https://phabricator.wikimedia.org/T103798#1405130 (ggellerman) p:Triage>Normal [15:38:20] milimetric: what's your error (it doesn't work for me either :) ) [15:38:43] you could do [15:38:46] cat ... | stdin:// [15:40:38] handler = handlers[parts.scheme] [15:40:38] KeyError: u'file'? [15:40:53] oh, milimetric! there is no file reader [15:40:59] you'll have to use stdin [15:41:11] or make a file reader :) [15:42:00] makes sense, doh [15:44:36] o/ ottomata [15:44:46] it looks like http://datasets.wikimedia.org/ is timing out when I try to download a dataset [15:45:09] Not sure what's up. I had someone else test too. [15:45:33] halfak: what data? [15:46:29] http://datasets.wikimedia.org/public-datasets/enwiki/etc/session_revisions.20131105.tsv.gz [15:46:44] Hmm... I just got another dataset to download. [15:46:52] Or start downloading rather. [15:50:35] joal: the pulling of the aggregator-data repository is done automatically: [15:50:36] https://github.com/wikimedia/operations-puppet/blob/acacf97e2df962fef83487a461f3559fa07e4d6f/manifests/role/wikimetrics.pp#L314 [15:51:05] so as long as you're using the same repo, it will pull. However, we should change the symlink. I'll make a task explaining [15:51:11] Yeah. It looks like that one link ottomata. [15:51:19] For some reason I can't even get the download to start. [15:51:26] It's been a few minutes. [15:51:44] hmm, 9G [15:51:53] FWIW, that's one of the larger files [15:51:56] yeah. [15:52:06] But you'd expect the bits to start transfering right away. [15:52:51] hm, yeah it is weird for sure [15:52:54] i can't even HEAD it [15:53:14] joal: https://gerrit.wikimedia.org/r/#/c/220952/ [15:54:43] Seems like it is a normal file: https://gist.github.com/halfak/5d8036ce5a5609563e71 [15:54:46] ottomata, ^ [15:54:59] Analytics-Backlog: Link to new projectcounts data and serve via wikimetrics - https://phabricator.wikimedia.org/T104003#1405205 (Milimetric) NEW [15:55:05] joal: ^ [15:55:28] i think varnish is attempting to cache these large files [15:55:38] Oh! Maybe that's why it takes so long? [15:55:44] Also. AHHHH! [15:57:31] SELECT month, [15:57:31] day, [15:57:31] COUNT(DISTINCT COALESCE(x_analytics_map['wmfuuid'], [15:57:31] parse_url(concat('http://bla.org/woo/', uri_query), 'QUERY', 'appInstallID'))) AS app_uniques [15:57:31] FROM wmf.webrequest [15:57:31] WHERE user_agent LIKE('WikipediaApp%') [15:57:31] AND parse_url(concat('http://bla.org/woo/', uri_query), 'QUERY', 'action') = 'mobileview' [15:57:32] AND COALESCE(x_analytics_map['wmfuuid'], [15:57:32] parse_url(concat('http://bla.org/woo/', uri_query), 'QUERY', 'appInstallID')) IS NOT NULL [15:57:33] AND webrequest_source IN ('mobile','text') [15:57:33] AND year=2015 [15:57:34] AND month=5 [15:57:55] pastebin, madhuvishy :P [15:58:00] oops. sorry, irccloud was supposed to tell me to paste it via pastebin [15:58:02] IRCCloud even asks you! :P [15:58:08] it dint! [15:58:17] yeah, blame the computer! ;) [15:58:20] madhuvishy, why are you using concat and parse_url? [15:58:53] why not just LIKE or RLIKE on uri_query? Presumably it should be somewhat faster ;p [15:58:55] Ironholds: gah, this is not my query [15:58:58] ahhh [15:59:25] Ironholds: but no, i din't know that [15:59:39] "bla.org/woo/" [15:59:52] ಠ_ಠ [15:59:54] halfak, my friend Kara's last name is woo. She has a personal R package. All the calls are extra-fun [15:59:56] woo::merge() [16:00:03] ha [16:00:08] ottomata: still weird... [16:00:10] milimetric@analytics1004:~/EventLogging/server/bin$ ./eventlogging-processor "%q" stdin:// stdout:// [16:00:11] /usr/bin/env: python -OO: No such file or directory [16:00:45] Ironholds: halfak this query is from - https://phabricator.wikimedia.org/diffusion/ANRE/browse/master/oozie/mobile_apps/uniques/daily/generate_uniques_daily.hql [16:00:48] ha, halfak, misc varnishes only have 8G memory allocated to them [16:00:51] that file is 9G [16:01:05] milimetric: [16:01:09] python ./eventlogging-processor [16:01:23] ottomata, can we not have a varnish between datasets.wikimedia and the world? [16:01:28] madhuvishy, aha [16:01:38] no, stat1001 no longer has a public IP, and i think that is the right hting [16:01:40] we should proxy to it [16:01:44] varnish or not [16:01:47] We have lots of files bigger than 9GB and it seems like varnish isn't helping anyone [16:01:58] but, i think we should tell varnish not to cache datsets maybe? [16:02:00] somehow? [16:02:05] Oh... sure. [16:02:08] That'd work too. [16:02:17] not sure how to do that, am pining bblack [16:02:22] No varnish == varnish not doing its thing [16:02:23] and poking around [16:02:26] Thanks ottomata. [16:02:32] Should I start a phab task? [16:03:44] Analytics-Backlog: Link to new projectcounts data and serve via wikimetrics {Musk} - https://phabricator.wikimedia.org/T104003#1405234 (ggellerman) [16:04:29] Analytics-Backlog: Link to new projectcounts data and serve via wikimetrics {Musk} - https://phabricator.wikimedia.org/T104003#1405205 (ggellerman) p:Triage>Normal [16:05:03] halfak: ja [16:05:12] i think i got something, if you create a task I can link it and ask bblack [16:05:29] OK will go [16:05:31] *do [16:05:34] will go do [16:05:36] :D [16:08:42] Analytics-Cluster, operations: Can't download large datasets from datasets.wikimedia.org - https://phabricator.wikimedia.org/T104004#1405240 (Halfak) NEW [16:08:46] ottomata, https://phabricator.wikimedia.org/T104004 [16:08:47] Analytics-Backlog, Labs, Labs-Infrastructure: Report page views for labs instances - https://phabricator.wikimedia.org/T103726#1405247 (ggellerman) p:Triage>Low [16:08:47] gone did [16:10:43] halfak: danke [16:10:43] https://gerrit.wikimedia.org/r/#/c/221139/ [16:10:48] will ping bblack about that whne I see him around [16:11:03] Thanks! :) [16:13:58] Analytics-Backlog, MediaWiki-extensions-ExtensionDistributor: Set up graphs and dumps for ExtensionDistributor download statistics - https://phabricator.wikimedia.org/T101194#1405272 (ggellerman) @Legoktm - could you answer Milimetric's question? Thanks! [16:18:21] has datasets.wikimedia.org screwed someone over again? ;p [16:28:28] madhuvishy: The query you past, is that the one you are trying to run ? [16:28:48] Because it looks exactly the same as the prod one (no filter for specific domain) [16:29:01] milimetric: yo [16:29:27] cause i'm about to do it, should the topic name for the union eventlogging topic be [16:29:29] eventlogging-union? [16:29:39] is union a good name? [16:29:47] joal: hmmm, checking. [16:30:03] Analytics-Backlog, Labs, Labs-Infrastructure: Report page views for labs instances - https://phabricator.wikimedia.org/T103726#1405313 (Milimetric) Dear @Spage: we can't commit to supporting a production or labs instance of piwik which would help with this. Using Event Logging from labs might be an o... [16:30:22] combined? [16:30:37] oiy, I fear names like that [16:30:48] like whta? [16:30:52] because say we push labs events through, would that include labs events? [16:31:01] names like "all" are hard [16:31:07] i don't want all [16:31:12] because we are going to blacklist some [16:31:25] but, it will have more than one schema in it [16:31:26] so this is all - blacklist schemas [16:31:29] yes [16:31:31] joal: http://pastebin.com/wJQa2KHe [16:31:39] I'm filtering for uri_host [16:31:40] multischema [16:31:41] ew [16:31:44] uh... [16:31:57] polyschematic [16:32:12] mixed [16:32:16] yes! [16:32:17] good [16:32:39] well, better than polyschematic anyway [16:32:40] ok cool :) [16:32:41] haha misc [16:32:42] hehehe [16:33:04] heterogeneous [16:33:07] heh [16:33:21] I think mixed will make sense when listing schemas and seeing that others are schema specific [16:33:28] yeah mixed might be good [16:33:30] but union might imply "all" [16:33:32] joal: opine? [16:33:46] we need a topic name that includes by default all schemas, minus any thing that is blacklisted [16:34:02] eventlogging-mixed [16:34:02] ? [16:34:02] * joal is thinkng [16:34:26] ottomata: is ImportError: No module named eventlogging solvable? Wouldn't I have to pip install to get that? [16:35:03] export PYTHONPATH=/home/milimetric/EventLogging/server [16:35:12] ottomata: Since there a blacklist, it means that this topic is purposedly reduced, and therefore could have a more function-oriented name ? [16:35:21] cool :) [16:35:37] like what? [16:35:40] eventlogging-mysql? [16:35:48] I was wondering that [16:35:53] naw, def not that. [16:35:55] But that's not vey good either [16:36:05] ottomata: in meeting, will be back [16:36:15] ok, joal, if you dont' mind, i think we might go with mixed [16:36:25] ok please do [16:36:29] k [16:37:02] grrr, milimetric, although, I already have a topic in prod kafka called eventlogging-all [16:37:02] only concern ottomata : differenciating with schema based ones [16:37:09] we migiht want to reuse that one [16:37:50] oh you can't delete topics? [16:37:52] no [16:37:59] :) ok but... [16:38:09] 12:31:08 i don't want all [16:38:11] yeah, but the unused topic doesn't really hurt anytihng but my eyes [16:38:22] yeah [16:38:25] wait... you can't ever never ever?! [16:38:29] that's crazy [16:38:33] unless t hey add that feature, basically no. [16:38:34] you *can* [16:38:36] i have done it [16:38:40] but you have to take the whole system down [16:38:42] delte files [16:38:45] and delete zk references [16:38:47] wow, awesome [16:38:53] ok, i mean we can use -all that's fine [16:38:58] we'll know what it means [16:39:11] ja, we can change it later if we need to. [16:39:11] and maybe once we're done with the whole mysql consumer we'll repurpose it again and it'll really be all [16:39:18] ja maybe so. ok [16:39:18] yeah [16:39:21] will resuse for now [16:39:24] and add big ol comment in puppet [16:39:25] k [16:40:01] comment: # should be named eventlogging-small-schemas-that-do-not-break-mysql but, you know, java [16:40:25] mforns: I marked all the people I reached out to and waiting for response with yellow. you can pick a color and do the same for people you reached out to too if you want :) [16:41:05] madhuvishy, sure! [16:41:38] milimetric: do you know of any topic we can blacklist now? just for testing? [16:41:52] that is, are all active schemas currently used in mysql? [16:42:41] oh, the one mforns was talking about, Jared and Juliusz's schema [16:42:56] but i would just test in beta [16:43:11] ja i will test in beta first, i'm making the puppet change and will cherry pick it there first [16:43:16] * mforns reading [16:43:25] mforns: what's the name of that schema? [16:43:36] and leave the blacklist in prod alone for now, we'd only blacklist if search turns up their sampling and they're ok with hadoop for analysis [16:43:44] milimetric, PersonalBar [16:43:48] dawww ok, i wanted to blacklist in prod! :) [16:44:19] we were going to disable the logging for that schema anyway so I mention it [16:44:29] ok, nm its ok [16:44:34] we can test the blacklist stuff in prod later [16:44:38] i've laready tested that in beta [16:45:22] cool, you can def. use that schema if you want, it's not needed [17:01:37] Analytics, Traffic, operations: Provide summary of MediaWiki downloads - https://phabricator.wikimedia.org/T104010#1405400 (Krenair) So you need to get statistics on downloads from Gerrit, Gitblit, Github (not in our infrastructure...), and releases.wikimedia.org? [17:02:46] ottomata: what does this error imply? [17:02:50] https://www.irccloud.com/pastebin/7VxPVInx/ [17:03:35] madhuvishy: has your query finished ? [17:06:07] iiinteresting [17:06:39] madhuvishy: that happens when the namenode you are pointing at is in standby state, instead of active [17:06:53] but, analytics1001 is active [17:12:13] hmmm [17:28:32] mforns: our sheet's so pretty :D [17:28:37] Analytics-Kanban: Spike: gather requirements to implement unique tokens {bull} - https://phabricator.wikimedia.org/T101784#1405491 (kevinator) a:kevinator [17:31:39] Analytics-Backlog: Change mediawiki-storage api queries to adapt to the api changes [5 pts] {crow} - https://phabricator.wikimedia.org/T101539#1405515 (mforns) a:mforns [17:32:03] Analytics-Kanban: Change mediawiki-storage api queries to adapt to the api changes [5 pts] {crow} - https://phabricator.wikimedia.org/T101539#1342023 (mforns) [17:33:27] Analytics-Kanban: Gather information on all the schemas {tick} [13 pts] - https://phabricator.wikimedia.org/T102515#1366348 (mforns) Blocked waiting for Aaron's response. He will work on it on Monday Jun 29. [17:33:35] madhuvishy, hehehe [17:38:51] madhuvishy, ottomata : prod spark job seems to have been launched correctly [17:39:21] madhuvishy: I let you modify your parameter name (spark_driver_memory) and commit yout change, then I'll merge and deploy [17:43:58] joal: yeah i did that. let me push [17:44:07] madhuvishy: Great :) [17:46:01] (PS2) Madhuvishy: Add driver memory as a configurable property to Spark job [analytics/refinery] - https://gerrit.wikimedia.org/r/220952 (https://phabricator.wikimedia.org/T97876) [17:46:07] joal: ^ done [17:46:53] (CR) Joal: [C: 2 V: 2] Add driver memory as a configurable property to Spark job [analytics/refinery] - https://gerrit.wikimedia.org/r/220952 (https://phabricator.wikimedia.org/T97876) (owner: Madhuvishy) [17:47:12] madhuvishy, ottomata : deplying refinery [17:47:39] joal: was it just a resource allocation issue? were you able to launch it with 2G? [17:47:45] cool [17:47:50] Worked fine for me with 2G [17:48:08] So, I don't know more really :S [17:48:12] joal: you're magic [17:48:14] milimetric: https://gerrit.wikimedia.org/r/#/c/221155/ [17:48:17] i need lunch and power [17:48:21] be back in a bit [17:50:14] madhuvishy: would love to be more magic than th [17:50:16] at [17:50:19] , but thanx :) [17:51:08] madhuvishy, and team, I'll leave in 10m to the gym and be back in a while [17:51:16] Enjoy mforns ! [17:51:20] :] [17:52:03] mforns: okay :) [17:52:08] madhuvishy, regarding Tick, I'm blocked now waiting for the owners responses, and Aaron's [17:52:18] mforns: yeah same here [17:52:37] madhuvishy, however, I'll write to Sean Pringle to setup a meeting on possible solutions for the auto-purging [17:53:00] madhuvishy, I guess you want to be there too right? [17:53:09] mforns: yup :) [17:53:19] cool, I'll cc you :] [17:53:41] madhuvishy: Thought of something [17:53:52] mforns: thanks :) [17:53:56] joal: yes? [17:54:00] 'till later people :] [17:54:12] madhuvishy: I'll restart the job after deploy starting the 4th of may :) [17:54:23] Analytics-Tech-community-metrics, Engineering-Community, ECT-July-2015: Check whether it is true that we have lost 40% of code contributors in the past 12 months - https://phabricator.wikimedia.org/T103292#1405612 (Qgil) Is there a way to get a list of newly created repos in Git/Gerrit.wikimedia.org? [17:54:28] joal: why 4th? [17:54:29] I didn't realise that, since it is weekly, it's better to have it start on monday [17:54:36] joal: aah [17:54:36] on maybe sunday [17:54:41] I don't mind [17:55:00] joal: hmmm, yeah Sunday sounds good [17:55:03] What do you think would be best ? [17:55:10] ok, let's go for 3rd then :) [17:55:20] I'll delete already computed data and restart the thing [17:55:33] joal: great! [17:58:16] Analytics-Backlog, Analytics-Cluster: Add Pageview aggregation to Python {musk} [13 pts] - https://phabricator.wikimedia.org/T95339#1405619 (kevinator) [18:02:15] joal: did it launch? [18:02:26] Analytics-Backlog, Analytics-Cluster: Add Pageview aggregation to Python {musk} [13 pts] - https://phabricator.wikimedia.org/T95339#1405631 (kevinator) [18:02:26] still deploying [18:02:28]