[08:14:11] Hi a-tem [08:14:15] Hi a-team sorry [08:14:40] heeey joal! [08:14:51] What's up fdans? [08:15:57] not much! autumn is full on in Madrid and I've got a cold because of it :) [08:16:10] right - Kinda the same here [08:16:37] We've not even past the date, and temperatures and rain are really there [08:17:02] o/ [08:17:57] Hi elukey [08:19:38] hellooooo luca! [09:15:43] 10Analytics, 10ChangeProp, 10EventBus, 10MediaWiki-JobQueue, and 4 others: Select candidate jobs for transferring to the new infrastucture - https://phabricator.wikimedia.org/T175210#3620607 (10mobrovac) >>! In T175210#3618572, @GWicke wrote: > I honestly don't have a strong preference between the other "h... [11:50:21] joal: whenever you have time can you give me an example of a HTTP request to druid? [11:52:16] Hi elukey [11:52:23] elukey: I can surely do that now :) [11:52:57] elukey: https://gerrit.wikimedia.org/r/#/c/379000/1/oozie/mediawiki/history/reduced/mediawiki-stats-druid.md (end of file) [11:53:06] elukey: There's more than one :) [11:55:32] joal: thanks! So we always post to port 8082 with URI /druid/v2/ [11:55:40] correct elukey [11:56:09] elukey: maybe if at some point druid upgrades, it'll be /druid/v3, but for now we are at v2 :) [11:58:10] super fine, I am writing down the nginx proxy config [11:58:21] Thanks a lot for that elukey :) [11:58:31] elukey: I saw yesterday that we might have TLS issues? [11:59:09] joal: maybe I am missing something, where did yo usee it? [11:59:26] on the chan, nuria_ mentionned something around that [12:00:43] elukey: yesterday, about 21:30, nuria and andrew [12:02:26] doing http auth without TLS is completely pointless [12:02:49] elukey: I do know that ... [12:03:20] and we already have automation to create certs etc. [12:03:23] elukey: I think it would be usefull to discuss that at ops-sync and standup [12:03:26] we do it in a lot of places [12:05:12] if we want to not use TLS let's just create a LVS in front of druid and that's it [12:05:33] elukey: I'm not he one to decide on this :) [12:05:45] elukey: I think it's a matter of iming [12:05:50] but as I can see know, it should be a couple of days work to set up the tls proxy [12:05:52] *timnig sorry [12:06:03] right [12:06:23] well you guys can proceed anyway, when the LVS+proxy is ready we'll use the endpoint [12:06:32] the main question is if RB supports TLS + basic auth [12:10:11] ok I am going to stop working on the puppet code until we have an agreement [12:10:14] :D [12:10:16] :D [12:11:04] elukey: Just read a bit on basic auth - I means we'll have authentication check for every callto druid - Man - What an overhead [12:12:41] joal: sure, but we are not talking about mining bitcoins :D [12:12:54] true :) [12:13:17] there will also be TLS auth overhead, that is not that much but not negligible [12:13:24] and the proxy nginx <-> druid [12:13:49] right [12:14:32] we can measure performances and see, but usually http basic auth checks are pretty fast [12:14:55] I expect nginx/apache to have the auth file in memory, and then it is only a matter of checking headers [12:15:04] so I wouldn't worry about it [12:15:08] k [12:15:57] going to have a quick lunch! [12:16:22] laer [13:22:30] (03PS1) 10Joal: [WIP] Add mediawiki-history-metrics endpoints [analytics/aqs] - 10https://gerrit.wikimedia.org/r/379227 (https://phabricator.wikimedia.org/T175805) [13:23:03] milimetric, mforns, fdans --^ This patch contains all of the timeseries metrics defined in the .md doc [13:23:38] I take a break now, and will continue my go-fast towards metrics-city after [13:24:24] Thinks to look at particularly: endpoints URLs for names [13:37:18] joal: i dotn' see a .md file there! [13:41:16] ottomata: o/ [13:41:48] hii [13:41:50] later on if you have time let's chat about the proxy on druid [13:42:13] I didn't do anything today because it wasn't clear from yesterday if RB supports TLS, basic auth, etc.. [13:42:56] elukey: they said it should [13:43:13] the q is how are we going to do TLS? just a simple local self signed cert for the proxy http server? [13:43:20] that would be easiest for now [13:43:22] i think [13:43:48] nope, we use the puppet CA for this [13:44:07] there is a script from Giuseppe that creates the cert on puppet private and public [13:44:21] so it is basically a matter of deploying the tlsproxy puppet code [13:44:34] (plus the basic auth credentials file) [13:44:46] ok cool [14:03:05] 10Analytics, 10Analytics-Cluster, 10Patch-For-Review: can't compile numpy on stat1004 - https://phabricator.wikimedia.org/T163177#3621193 (10Ottomata) Can we close this then? [14:04:54] morning! [14:04:57] looking at patches [14:12:35] 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review: rsync-published-datasets cron should not launch multiple rsync processes - https://phabricator.wikimedia.org/T174756#3621221 (10Ottomata) [14:12:51] 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review: rsync-published-datasets cron should not launch multiple rsync processes - https://phabricator.wikimedia.org/T174756#3572217 (10Ottomata) a:03Ottomata [14:14:16] 10Analytics, 10Analytics-EventLogging: Alarm on errors on /var/log/upstart/eventlogging* files - https://phabricator.wikimedia.org/T170620#3437262 (10Ottomata) Agree, we shouldn't do weird logster stuff for this, but instead instrument eventlogging to emit errors somewhere nicely. Parsing the logs sounds a li... [14:14:43] 10Analytics, 10Analytics-EventLogging: Alarm on errors on /var/log/upstart/eventlogging* files - https://phabricator.wikimedia.org/T170620#3621247 (10Ottomata) For process flapping: we might want to just wait until (if?) we upgrade eventlogging to run on stretch and use systemd. [14:23:12] 10Analytics-Kanban, 10Wikimedia-Stream, 10Patch-For-Review: Stop tracking EventStreams client lag in graphite - https://phabricator.wikimedia.org/T174435#3621286 (10Ottomata) [14:34:08] elukey: hiii [14:34:31] (03CR) 10Milimetric: "small things" (0310 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/379000 (https://phabricator.wikimedia.org/T174174) (owner: 10Joal) [14:35:18] ottomata: coming sorry! [14:59:12] I think I found out how partman calculates the size of the partitions [14:59:21] from https://www.bishnet.net/tim/blog/2015/01/29/understanding-partman-autoexpert_recipe/ [14:59:32] and my soul is now damaged [15:00:02] elukey: There are things a man should never even think of doing [15:00:38] ping elukey [15:00:39] (03CR) 10Joal: [C: 04-1] "Thanks for the comments @milimetric, pushing a new patch now." (0310 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/379000 (https://phabricator.wikimedia.org/T174174) (owner: 10Joal) [15:01:57] (03PS2) 10Joal: Update mediawiki-history-reduced oozie job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/379000 (https://phabricator.wikimedia.org/T174174) [15:02:05] (03CR) 10Joal: Update mediawiki-history-reduced oozie job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/379000 (https://phabricator.wikimedia.org/T174174) (owner: 10Joal) [15:03:31] urandom: Hi, so that you know we have a special meeting this week, so elukey and myself won't be present at cassandra standup :( [15:05:32] (03CR) 10Milimetric: Update mediawiki-history-reduced oozie job (035 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/379000 (https://phabricator.wikimedia.org/T174174) (owner: 10Joal) [15:10:13] (03CR) 10Joal: Update mediawiki-history-reduced oozie job (034 comments) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/379000 (https://phabricator.wikimedia.org/T174174) (owner: 10Joal) [15:11:05] (03PS3) 10Joal: Update mediawiki-history-reduced oozie job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/379000 (https://phabricator.wikimedia.org/T174174) [15:28:07] joal: yeah, i think we'll cancel it [15:44:33] (03PS4) 10Joal: Update mediawiki-history-reduced oozie job [analytics/refinery] - 10https://gerrit.wikimedia.org/r/379000 (https://phabricator.wikimedia.org/T174174) [15:52:48] 10Analytics, 10Analytics-Cluster, 10Patch-For-Review: can't compile numpy on stat1004 - https://phabricator.wikimedia.org/T163177#3621578 (10EBernhardson) 05Open>03declined yup, this is unnecessary now [16:04:38] I’m having some struggles exporting some data from the Data Lake, getting a “GC overhead limit exceeded error”. Am considering trying to turn off the GC overhead limit and/or using the concurrent garbage collector, as described here: https://hvivani.com.ar/2014/12/06/hive-dealing-with-out-of-memory-and-garbage-collector-errors/ Would that be unadvisable? Other options? [16:25:38] 10Analytics, 10Operations, 10Traffic: A/B Testing solid framework - https://phabricator.wikimedia.org/T135762#3621689 (10phuedx) Deterministic bucketing is also available in MediaWiki core via [[ https://github.com/wikimedia/mediawiki/blob/00c769eb8d7746dfddff525ccc813f276046dea8/resources/src/mediawiki/medi... [16:28:02] going to eat with familly, will be b ack after [16:33:49] Nettrom: hello! Where/how are you exporting data? [16:34:36] Nettrom: as a user you do not have ability to change GC settings on hadoop but i am sure we can help you in some other way [16:35:13] 10Analytics, 10Operations, 10monitoring, 10Patch-For-Review: Eventstreams graphite disk usage - https://phabricator.wikimedia.org/T160644#3621710 (10Zoranzoki21) [16:35:15] 10Analytics-Kanban, 10Wikimedia-Stream: Stop tracking EventStreams client lag in graphite - https://phabricator.wikimedia.org/T174435#3621707 (10Zoranzoki21) 05Open>03Resolved [16:36:18] nuria_: not sure if it is the best thing to do or not, but Nettrom would be able to set the mapred.child.java.ops in hive like that, i think, as it would apply to JVM mapred proceses that his hive job spawns [16:37:10] nuria_: Hi! :) I’ve got a Python script that runs “beeline -e ~/project/actrial/export_creations.hql > enwiki_non-autopatrolled_creation.tsv”, and then crashes with the GC error [16:37:18] on stat1005 [16:37:34] ottomata: i doubt that turning off the gc limit (if doable) would help here [16:38:13] I tried to sqoop the table over to MySQL, but that crashes, apparently a Java version problem (I’m working on making a phab task for that) [16:38:19] *MariaDB [16:38:30] Nettrom: fyi sqoop does not work well from stat1005 because of java versions [16:38:32] it works from stat1004 though [16:38:40] Nettrom: right, two thins [16:38:43] *things [16:38:46] ottomata: ah, you already know about that then :) [16:39:07] Nettrom: yes, there is info on wikitech on sqoop [16:39:13] Nettrom: let me find link [16:39:32] Nettrom: it is worth thinking whether data would fit on destination though [16:39:39] Nettrom: it being your homedir or db [16:40:34] hmm, weird that you are getting GC issues [16:40:39] your creation_data table is not large [16:40:43] and your query is pretty simple [16:40:58] nuria_: I worry about that constantly, it’s a couple of million rows, probably around 150-200MB, which I compress afterwards [16:40:59] oh but i guess you are selecting it out and > .tsv [16:41:23] Nettrom: might be better to let it write the tsv in hive/hadoop, rather than redirecting stdout [16:41:36] if you are just trying to make a tsv from a hive query, i think there are better ways... [16:42:05] ottomata: I’m happy to learn about those :) [16:42:17] Nettrom: ok, sqoop info we have is mysql->hdfs but here it is: https://wikitech.wikimedia.org/wiki/Analytics/Systems/EventLogging/Administration#Sqoop [16:44:46] Nettrom: a better way to do what you want might be to 1) use hive to move data of interest to your db in hadoop [16:44:58] Nettrom: and after use jupyter notebooks to analyze it [16:45:19] looking [16:45:23] googling and trying myself [16:45:23] :) [16:45:33] nuria_: Nettrom has done that [16:45:37] Nettrom: that is an alternative, you can really do away with teh tsv step [16:45:40] created a table [16:45:43] with less data [16:45:54] he's just trying to get query output convert it to local tsv [16:46:10] although true, Nettrom why do you want tsv? [16:46:20] Ah Nettrom sorry, then using it from jupyter might be a fast way to analyze with python w/o having to move GiGs arround [16:49:09] Nettrom: https://gist.github.com/ottomata/cca4a8c7a6fe6d94491f5edb2bd0db65 [16:49:37] ottomata & nuria_: Yeah, I’m starting to think that Jupyter notebooks might be a better way to do this. I’m working on a (relatively) complex dataset of article creations, reviews, and deletions. It’ll require some further processing, but if I can input/output data through the notebook, that’ll be fine. [16:50:15] Nettrom: have you tried spark? there is a python repl [16:50:59] Nettrom: ya, i think having a 200G dataset to read on command line from python or event mysql will be not very optimal, with indexes and all might occupy quite a bit more on mysql and to be honest it might not even fit on the machines now until we finishing purging data [16:51:50] ottomata: not yet, I can read up on that. Is there some intro to that on wikitech, or should I just google? [16:52:07] Nettrom: we have an experiemntal jupyter notebook install [16:52:43] Nettrom: ssh -N notebook1001.eqiad.wmnet -L 8000:127.0.0.1:8000 [16:52:58] Nettrom: and after http://localhost:8000 [16:53:00] nuria_: cool, thanks! :) [16:53:10] mostly google, but i got a quick example for you, one sec [16:53:12] this [16:53:13] https://spark.apache.org/docs/1.6.1/sql-programming-guide.html#hive-tables [16:53:24] Nettrom: not sure about ssh permits but ottomata can help you with those [16:55:09] let me get an ssh configuration set up [16:57:16] looks like ottomata doesn’t have to help me with any ssh permits at all :) [16:57:31] * elukey off! [16:57:32] o/ [16:58:22] Nettrom: https://gist.github.com/ottomata/cca4a8c7a6fe6d94491f5edb2bd0db65#file-querying-with-pyspark [16:58:56] Nettrom: some stuff on wikitech [16:58:56] https://wikitech.wikimedia.org/wiki/Analytics/Systems/Cluster/Spark [16:59:26] ottomata: awesome, thanks! what username/password do I use to sign in to the notebook, my wikitech one? [16:59:39] that I don't know but i'd assume so [16:59:46] https://wikitech.wikimedia.org/w/index.php?title=SWAP [16:59:54] Then open http://localhost:8000/ in your browser and log in with your LDAP (wikitech) credentials. [17:03:34] hmm, that doesn’t seem to work [17:03:38] 10Analytics-Kanban, 10Wikimedia-Stream: Stop tracking EventStreams client lag in graphite - https://phabricator.wikimedia.org/T174435#3621913 (10Ottomata) 05Resolved>03Open Hey @Zoranoki21, thanks for the diligence, but we track our tasks on the [[ https://phabricator.wikimedia.org/tag/analytics-kanban/ |... [17:03:41] 10Analytics, 10Operations, 10monitoring, 10Patch-For-Review: Eventstreams graphite disk usage - https://phabricator.wikimedia.org/T160644#3621915 (10Ottomata) [17:03:52] Nettrom: i am not sure wait joal knows [17:03:57] Nettrom: is it admin admin [17:05:33] 10Analytics-Kanban, 10Wikimedia-Stream: Stop tracking EventStreams client lag in graphite - https://phabricator.wikimedia.org/T174435#3621921 (10Zoranzoki21) >>! In T174435#3621913, @Ottomata wrote: > Hey @Zoranzoki21, thanks for the diligence, but we track our tasks on the [[ https://phabricator.wikimedia.org... [17:10:30] nuria_: nah, that didn’t work either. Not sure if I’m in the right LDAP groups or not. I can wait until joal’s back to figure this out. [17:34:39] Heya folks [17:34:42] Hi Nettrom [17:35:03] Nettrom: Do you mind giving some more insights on wha you're trying do, data sizes etc? [17:35:33] About Jupyter notebooks, just tried it - It works for me with LDAP password [17:36:06] nuria_, mforns: Is now a good moment to talk about AQS endpoints and code? [17:36:22] joal: yes [17:36:32] joal: can you give me 10 mins? [17:36:38] I surely can nuria_ :) [17:36:54] joal: ok, joining cave in 10 mins [17:37:05] let's synchro here before caving [17:37:11] joal: Not at all! We want to get an understanding of article creation, reviews, and survival, and subsequently user survival (mainly for new accounts), in English Wikipedia. Over say the past 5 years. [17:38:13] so I’d like to have a dataset that has article creations by non-autopatrolled accounts (this I have), deletions (proposed and actual), and reviews. [17:39:20] Nettrom: Article creations can be extracted from mediawiki-history dataset (i guess you know) - Deletions, we have some info in that dataset as well, but maybe not enough (only actual, not proposed)? [17:39:34] As for reviews, I actually don't know what they are :) [17:40:02] joal: reviews are also called “patrol actions”, logged in the logging table, so they’re straightforward to extract [17:40:20] Nettrom: ok for reviews [17:40:24] joal, yep! [17:40:34] Nettrom: on deletes, what's your plan? [17:40:36] whenever nu-ria is ready [17:40:43] aqesome mforns, thanks :) [17:40:58] I’m assuming that proposed deletions can be identified through edit comments, so there won’t be any content parsing to be done [17:41:36] joal: I was thinking that using the page history table could give me creation/deletion events that I can then flatten (since I’m interested in pairs of these) [17:41:36] oh my... I forgot SoS [17:41:55] and when I mean “page history” I mean the mediawiki-history , sorry [17:42:32] Nettrom: You'll find actual deletions in page table, yes, but not proposed ones [17:43:24] joal: no, I’m hoping that logged patrol actions and related revision comments will expose that. Twinkle, for example, has standard edit comments when a speedy deletion is proposed. [17:44:09] Nettrom: you'll probably some/a lot of manual annotation to go by ... [17:45:24] joal: maybe :) it’s not critical that I have them, so if they cannot be easily identified I’ll definitely skip that part [17:45:49] Nettrom: ok no prob - Thank you explaining me :) [17:46:00] joal: btw, I’m unable to log in to the Jupyter notebook, but can ssh into notebook1001 just fine, and I can log in to Wikitech just fine as well. [17:46:02] Nettrom: Do you have ideas of how big the dataset you want are? [17:46:12] weird Nettrom [17:47:25] Nettrom: I assume you've tunneled to notebooks, got the login page, and login doesn't work, right? [17:47:31] joal: correct [17:47:53] mwarf - madhuvishy, our Paws angel, would you be hearing us? [17:48:58] Nettrom: try using username all small case [17:49:35] Hi madhuvishy :) Thanks for hear-dropping and helping :) [17:49:41] joal: as for dataset size, there’s ~3 million article creations that I’m interested in (from 2009 onwards). Most of what I’m storing is numbers, so maybe 750MB to 1G in total? [17:50:03] madhuvishy: thanks, lowercase also errors [17:50:37] Also you'd login with your shell uid not wiki tech name [17:50:48] If those are different [17:51:07] madhuvishy: shell username is “nettrom” as well [17:51:47] Hmmm, one sec let me check your access [17:53:52] Nettrom: I'm not sure wha means you're storing mostly numbers :) [17:54:05] you guys let me know when it is a good time [17:54:08] ottomata: yt? [17:54:17] joal: not storing things like page titles and edit comments and such that might take up space :) [17:54:24] I'm ready nuria_, but I think mforns is gone to SoS [17:54:32] joal, I'm back now [17:54:35] Ah ok :) [17:54:37] it's done [17:54:46] nuria_, mforns - Batcave? [17:54:52] y [17:55:47] 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Use Prometheus for Kafka JMX metrics instead of jmxtrans - https://phabricator.wikimedia.org/T175922#3622156 (10Ottomata) HM, why are we making an 'analytics' prometheus instance for this? kafka-jumbo is not in the Analytics VLAN,... [17:55:54] Nettrom: I think easiest first step is to filter out the events amd fields you need from mediawiki-history into another hive table (or another file on hadoop) [17:56:13] Nettrom: This way, no size issue, data is stored on the cluster, and you can check for size etc [17:57:22] (03PS2) 10Joal: [WIP] Add mediawiki-history-metrics endpoints [analytics/aqs] - 10https://gerrit.wikimedia.org/r/379227 (https://phabricator.wikimedia.org/T175805) [17:59:02] joal: yeah, now that I’ve learned I can keep the data in Hadoop/Hive that’ll be the plan [17:59:40] Nettrom: If datasize is small, why not outside he cluster, but checking onto the cluster first is safer :) [18:00:47] * Nettrom meeting, back later [18:02:45] nuria_: ya am here [18:09:17] 10Analytics-Kanban, 10Beta-Cluster-Infrastructure, 10Wikimedia-Stream, 10Patch-For-Review: Decom RCStream in Beta Cluster - https://phabricator.wikimedia.org/T172356#3495960 (10Krenair) >>! In T172356#3496434, @hashar wrote: > and a DNS entry `instance-deployment-stream.deployment-prep.wmflabs.org.` see h... [18:16:29] 10Analytics-Cluster, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Use Prometheus for Kafka JMX metrics instead of jmxtrans - https://phabricator.wikimedia.org/T175922#3622230 (10Ottomata) Dunno if I'm stepping on yall's toes with this, but I couldn't understand why I didn't see any metrics in pro... [18:40:29] madhuvishy: back from a meeting, regarding the Jupyter notebook access, did you find anything interesting in my LDAP access rights? [18:49:57] ping Deskana hola? [19:09:03] (03Abandoned) 10Mforns: Add pagelinks and redirect to refinery-drop-mediawiki-snapshots [analytics/refinery] - 10https://gerrit.wikimedia.org/r/374623 (https://phabricator.wikimedia.org/T174484) (owner: 10Mforns) [19:09:08] (03Abandoned) 10Joal: Add mediawiki-history edited-articles endpoint [analytics/aqs] - 10https://gerrit.wikimedia.org/r/377749 (https://phabricator.wikimedia.org/T175805) (owner: 10Joal) [19:09:18] (03Abandoned) 10Joal: Add mediawiki-history new-articles metric endpoint [analytics/aqs] - 10https://gerrit.wikimedia.org/r/377726 (https://phabricator.wikimedia.org/T175805) (owner: 10Joal) [19:09:51] (03Abandoned) 10Joal: Add mediawiki history edits metrics endpoint [analytics/aqs] - 10https://gerrit.wikimedia.org/r/373961 (https://phabricator.wikimedia.org/T174174) (owner: 10Joal) [19:13:25] Nettrom: sorry caught up in some other stuff, i checked puppet and it looks fine, gotta go look at logs in the server [19:15:14] madhuvishy: no problem, I’ve got other things to work on as well, thanks for helping out with this! :) [19:19:07] Nettrom: Sep 20 17:03:27 notebook1001 jupyterhub[11737]: [W 2017-09-20 17:03:27.753 JupyterHub jupyterhub_config:117] User Nettrom not in allowed groups (researchers, statistics-privatedata-users, analytics-privatedata-users, ops) [19:19:35] can you try now with username 'nettrom' [19:19:40] i'm watching logs [19:19:55] madhuvishy: yes, done [19:21:05] you are tunneling to notebook1001, correct? [19:21:17] i dont see your login attempt [19:21:48] madhuvishy: do you want me to log in to notebook1001 again as well? [19:22:32] uhh, no, what is your command for tunneling? [19:23:07] Nettrom: ^ [19:23:33] madhuvishy: let me get a gist of my setup, one min [19:25:02] madhuvishy: https://gist.github.com/nettrom/02cda69c0f3dfcd0ca5c51d192e81196 [19:25:42] hmmm, not sure why i dont see your login attempt here [19:27:38] what do you see on your end? [19:27:48] when you try to login [19:28:05] the notebook login says “Invalid username or password” [19:31:10] Off for today folks - See you tomorrowe [19:31:17] So weird [19:32:33] madhuvishy: does it pop up if I try to log in with a capitalized username? [19:32:50] yeah i see one just now [19:33:00] I still get the same “invalid username or password” error [19:33:52] (just in case that was helpful) [19:33:55] Deskana: hola? [19:41:35] Nettrom: are you a member of the wmf ldap group? [19:48:46] Nettrom: I think that's our mystery piece. Please make a ticket and tag Operations to be added to the wmf LDAP group [19:52:33] wikimedia/mediawiki-extensions-EventLogging#695 (REL1_30 - 98a95b3 : jenkins-bot): The build has errored. [19:52:33] Change view : https://github.com/wikimedia/mediawiki-extensions-EventLogging/compare/REL1_30 [19:52:33] Build details : https://travis-ci.org/wikimedia/mediawiki-extensions-EventLogging/builds/277920044 [20:00:37] nuria_: Hi! :-) [20:00:58] nuria_: What's up? [20:08:00] 10Analytics-Cluster, 10Analytics-Kanban: Port statsv to kafka-jumbo - https://phabricator.wikimedia.org/T176352#3622558 (10Ottomata) [20:12:56] Deskana: do you have a few minutes to talk goals? [20:13:41] nuria_: Sure! Editing's goals are on the Q2 page: https://www.mediawiki.org/wiki/Wikimedia_Audiences/2017-18_Q2_Goals#Contributors [20:14:07] hangout? [20:14:15] Deskana: hangout? [20:14:25] (03PS3) 10MarcoAurelio: Add hi.wikivoyage to stats [analytics/refinery] - 10https://gerrit.wikimedia.org/r/371100 (https://phabricator.wikimedia.org/T173013) [20:15:24] nuria_: It's pretty late here. Can we schedule something for tomorrow? [20:15:35] Deskana: i though you were on sf, np [20:15:42] Deskana: ya or next week [20:15:55] nuria_: I moved to Dublin! :-D [20:16:01] Deskana: ooohh [20:16:03] got it [20:16:14] nuria_: You're on West Coast time, right? [20:16:28] Deskana: yes [20:17:33] nuria_: Alright, I've scheduled something. :-) [20:20:15] (03Abandoned) 10MarcoAurelio: Add hi.wikivoyage to stats [analytics/refinery] - 10https://gerrit.wikimedia.org/r/371100 (https://phabricator.wikimedia.org/T173013) (owner: 10MarcoAurelio) [20:24:04] (03Restored) 10MarcoAurelio: Add hi.wikivoyage to stats [analytics/refinery] - 10https://gerrit.wikimedia.org/r/371100 (https://phabricator.wikimedia.org/T173013) (owner: 10MarcoAurelio) [20:24:48] (03CR) 10Nuria: [V: 032 C: 032] Add hi.wikivoyage to stats [analytics/refinery] - 10https://gerrit.wikimedia.org/r/371100 (https://phabricator.wikimedia.org/T173013) (owner: 10MarcoAurelio) [20:30:02] ottomata: yt? [20:36:29] madhuvishy: Thanks for helping me out, I’ll get a phab task filed! :) [20:49:24] ya hey nuria_ [20:54:25] back from the doctor [20:54:29] aaah, so late [20:54:33] gonna work late tonight [20:59:54] hey nuria_ and milimetric, do you want to look at the AQS endpoint paths that we discussed with Joseph and Nuria (Nuria, we added another option)? [21:00:04] https://etherpad.wikimedia.org/p/analytics-design [21:00:19] yeah, definitely [21:00:29] batcave or async and we talk tomorrow? [21:01:02] also - why not /contributing and /content ? [21:01:15] milimetric, batcavE? [21:01:19] omw [21:20:51] mforns, milimetric do we keep track of reverted revissions on data lake? [21:21:14] yes [21:21:15] nuria_, yes, what's the concrete use cas? [21:21:18] reverted and reverting [21:22:57] nuria_: quick batcave? [21:23:09] milimetric: sure [21:49:18] milimetric: am I right that we donot have revision history on mw reconstruction so you can get revisions reverted? [21:49:54] nuria, no, we answered above, it's there [21:50:07] batcave? I can show you the fields in the schema [21:50:53] yes [22:54:15] (03CR) 10Nuria: [WIP] Add mediawiki-history-metrics endpoints (031 comment) [analytics/aqs] - 10https://gerrit.wikimedia.org/r/379227 (https://phabricator.wikimedia.org/T175805) (owner: 10Joal) [23:18:08] 10Analytics, 10Analytics-Cluster, 10Operations, 10Research-management: GPU upgrade for stats machine - https://phabricator.wikimedia.org/T148843#3623169 (10dr0ptp4kt) @Ottomata do you know if it ws an s9150, or was it a W9100? I'm seeing the `04:00.0 Display controller: Advanced Micro Devices, Inc. [AMD/AT... [23:24:52] 10Analytics, 10Analytics-Dashiki, 10Community-Tech: Add draft namespace creations to page creation dashboard - https://phabricator.wikimedia.org/T176375#3623179 (10Nettrom) [23:25:54] I can review that if you submit Nettrom [23:26:03] (maybe not tonight but ping me) [23:26:21] milimetric: awesome, thanks! [23:26:43] and reviewing it tomorrow or Friday is fine, this isn’t time-critical [23:37:33] (03PS1) 10Nettrom: Add draft creation config and query [analytics/reportupdater-queries] - 10https://gerrit.wikimedia.org/r/379441 (https://phabricator.wikimedia.org/T176375)