[08:33:50] o/ [10:07:32] 10Analytics-Tech-community-metrics: Deployment of IRC panel - https://phabricator.wikimedia.org/T138004#2846406 (10Lcanasdiaz) Update: @Aklapper we're reviewing the IRC data we have with the new platform. We have half of the messages we had with the legacy version .. In any case, feel free to play with the pan... [10:10:29] joal: any news on that deployment? :) [10:10:42] Hey addshore ! [10:10:50] I'm sorry I forgot you :( [10:11:01] hehe, no worries :) [10:11:06] Code has been deployed, your jobs can be started :) [10:11:16] Or re-started :) [10:12:54] addshore: But your jobs needs to be updated first to link to the correct jar version [10:17:01] oooh, okay [10:17:05] How do I do that? :D [10:17:17] I also didn't start them the first time round (if I remember) [10:19:20] addshore: I'll restart them, no bother [10:19:41] would it be possible to get them to run for the the says in the past month too? :) [10:19:56] addshore: But, you still need to update the properties file of the jobs for which java (scala) code has changes for the interested version [10:20:52] addshore: https://github.com/wikimedia/analytics-refinery-source/blob/master/changelog.md [10:22:41] addshore: so, you need to update the jar version to 0.0.38 :) [10:22:51] okay! [10:23:06] addshore: makes sense? [10:23:12] yup! [10:25:11] (03PS1) 10Addshore: Update oozie articleplaceholder_metrics to refinery-job-0.0.38.jar [analytics/refinery] - 10https://gerrit.wikimedia.org/r/325272 [10:25:12] joal: ^^ [10:26:02] (03CR) 10Joal: [C: 032 V: 032] "LGTM !" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/325272 (owner: 10Addshore) [10:26:06] :D [10:26:21] addshore: deploying now and then restarting the job [10:26:29] again joal, thanks for all of the help :) [10:27:10] addshore: You're welcome, when somebody does the job of understanding our systems and processes, it's the minimum we can do to try to help a bit :) [10:35:46] !log deploying refinery [10:35:47] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [10:43:40] !log Deploy refinery onto hdfs [10:43:41] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [10:47:05] (03PS41) 10Joal: [WIP] Join and denormalize all histories into one [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/307903 (owner: 10Milimetric) [10:58:59] addshore: Shall I restart yout job now? [10:59:05] yes please! :) [10:59:17] addshore: nothin special but restart? [10:59:31] oh, restart but can it start from the 1st of November? [10:59:47] I think we still have data from then..? [11:00:20] addshore: I can do that -- Only thing I'm not sure of is, if graphite data gets overriden or added [11:00:34] overriden :0 [11:00:35] :) [11:00:44] Okey, then no ossue I can see [11:00:54] I'll restart starting nov1st [11:01:03] Brilliant! [11:02:12] !log Killing wikidata-articleplaceholder_metrics job and restarting it starting Nov. 1st for code update [11:02:13] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:03:24] addshore: https://hue.wikimedia.org/oozie/list_oozie_coordinator/0017203-161121120201437-oozie-oozi-C/ [11:23:05] (03PS2) 10Addshore: ExactValueQuantityProcessor [analytics/wmde/toolkit-analyzer] - 10https://gerrit.wikimedia.org/r/324906 [12:12:31] 10Analytics-Tech-community-metrics: Deployment of IRC panel - https://phabricator.wikimedia.org/T138004#2846605 (10Aklapper) Interesting, thank you! (Note to myself: The activity data in the week of 2015-06-08 looks suspiciously low, wondering what happened there...) [12:32:46] * elukey lunch! [12:35:13] hi team :] [12:36:21] Hey mforns [12:36:27] hello [13:45:38] (03PS1) 10Joal: Update version number in pom files [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/325299 [13:46:28] milimetric: if you have a minute, please review that patch and have a look to the change I just made here (https://wikitech.wikimedia.org/w/index.php?title=Analytics%2FCluster%2FDeploy%2FRefinery-source&type=revision&diff=1077957&oldid=1053811) [14:03:19] (03PS6) 10Joal: [WIP] Update edit history to GraphFrames [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/322260 [14:03:21] (03PS47) 10Joal: [WIP] Refactor Mediawiki History scala code [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/301837 (https://phabricator.wikimedia.org/T141548) [14:03:23] (03PS42) 10Joal: [WIP] Join and denormalize all histories into one [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/307903 (owner: 10Milimetric) [14:05:50] (03CR) 10jenkins-bot: [V: 04-1] [WIP] Join and denormalize all histories into one [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/307903 (owner: 10Milimetric) [14:06:43] 10Quarry, 06Labs, 10Tool-Labs: Clarify Tool Labs' rules to see if Quarry and PAWS are allowed to be hosted there - https://phabricator.wikimedia.org/T152212#2841828 (10chasemp) >>! In T152212#2845634, @valhallasw wrote: > * Exceptions to this rule can be made on a case-by-case basis. Please contact us with y... [14:11:43] (03CR) 10Milimetric: [C: 032] Update version number in pom files [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/325299 (owner: 10Joal) [14:12:14] joal: sorry about that, I asked Nuria and she said the _snapshot version didn't need to change [14:12:17] thanks for the fix [14:22:35] elukey: just to be sure, we're talking about stat1002:/home/milimetric/ right? [14:23:28] milimetric: yes this is what I gathered [14:23:56] I was only suggesting to leave you from the burden of maintaining your home dir [14:23:59] :) [14:24:29] ottomata: hellooooo [14:25:31] whenever you have time can you tell me if https://gerrit.wikimedia.org/r/#/c/324883/ looks good? [14:25:34] hiiii [14:25:36] yeah, we should talk after standup if there's a better way to do the kind of work that currently goes in our home dirs, I think it's a pretty common pattern [14:41:00] TIL I got what "data tank" means as mount point on stat1002 :P [14:42:43] haha yeah [14:43:19] elukey: maybe we should consider putting /home and /srv on the same huge partitions for the future stat box reinstalls [14:43:27] AND /tmp [14:43:39] it might be an option yes [14:45:26] hm, eventbus sync is at the same time as standup? [14:45:27] how did that happen?! [14:45:36] daylight savings hmmm?? [14:49:59] (03PS1) 10Addshore: Add ExactValueQuantityProcessor [analytics/wmde/toolkit-analyzer-build] - 10https://gerrit.wikimedia.org/r/325308 [14:50:08] (03PS3) 10Addshore: Add ExactValueQuantityProcessor [analytics/wmde/toolkit-analyzer] - 10https://gerrit.wikimedia.org/r/324906 [14:58:16] (03PS2) 10Addshore: Add ExactValueQuantityProcessor [analytics/wmde/toolkit-analyzer-build] - 10https://gerrit.wikimedia.org/r/325308 [14:59:00] ottomata: I have it at the same time of backlog grooming [14:59:14] ah calendar updated [14:59:15] ok :) [15:02:06] yeah [15:02:13] but i'm ok with occasionally missing grooming [15:03:58] (03PS3) 10Addshore: Add ExactValueQuantityProcessor [analytics/wmde/toolkit-analyzer-build] - 10https://gerrit.wikimedia.org/r/325308 [15:05:40] ottomata: updated https://grafana.wikimedia.org/dashboard/db/varnishkafka, new metrics are flowing! [15:15:22] 10Analytics, 10Analytics-Cluster, 06Operations, 06Research-and-Data, and 2 others: GPU upgrade for stats machine - https://phabricator.wikimedia.org/T148843#2847119 (10Ottomata) Hm, you know...stat1002 and stat1003 are out of warranty. We could add this card to stat1004, that would be totally fine. If we... [15:15:39] NIiice instance name drop down :) [15:16:57] :) [15:17:17] ottomata: sorry for the amount of pings, but do you know why update-alternatives --display java is in manual mode on stat1002? [15:17:23] auto on stat100[34] [15:18:26] 06Analytics-Kanban, 06Discovery, 06Operations, 06Discovery-Analysis (Current work), and 2 others: Can't install R package Boom (& bsts) on stat1002 (but can on stat1003) - https://phabricator.wikimedia.org/T147682#2847127 (10Ottomata) YES! Glad I could help! :D [15:18:52] elukey: manual mode? [15:19:38] yeah, it is set to java 7 [15:19:56] that's what it should be set to, no? [15:20:19] yes yes but as Filippo mentioned on jessie auto mode defaults to java 7 [15:20:24] ahhh but stat1002 is not jessie [15:20:29] ah! :) [15:20:55] elukey: it looks the same on a jessie node to though [15:21:14] i thought filippo was just saying that the alternatives default is java 7 [15:21:17] anyway [15:21:21] so installing 8 won't hurt anything [15:21:25] sure sure [15:21:40] Info: /Stage[main]/Mongodb/Service[mongodb]: Unscheduling refresh on Service[mongodb] [15:21:43] WHAT [15:21:46] stat1003? [15:22:01] yeah it had mongo db on it a loong time ago, can't remember why, someone wanted it for something [15:22:04] not sure if it is still used [15:22:29] cruncher.pp [15:22:30] # Aaron Halfaker (halfak) wants MongoDB for his project. [15:22:42] elukey@stat1003:~$ sudo service mongodb status [15:22:42] mongodb stop/waiting [15:22:52] ¯\_(ツ)_/¯ [15:22:55] hehehh [15:22:56] ahhahahahaha [15:22:59] we should ask aaron if he still needs it [15:23:07] yeah [15:27:30] 10Analytics, 06Operations, 13Patch-For-Review: Install java 8 to stat1002 - https://phabricator.wikimedia.org/T151896#2847172 (10elukey) 05Open>03Resolved a:03elukey @EBernhardson: ``` elukey@stat1002:~$ update-alternatives --display java java - manual mode link currently points to /usr/lib/jvm/java-... [15:32:30] milimetric, mforns: I'm strgulling with maintaining 3 patches for our MWhist thing -- I know we said we were not going to group them now, bit in fact it would make my life easier -- Do you guys minD? [15:32:56] joal: I don't mind, but aren't some of them in different repos? [15:33:05] milimetric: all on same repo [15:33:18] I'll wait for mforns approval as well [15:33:20] ok, so only the refinery-source ones then [15:33:36] k, but yeah, we haven't gotten too far at all with the code review so group away in my opinion [15:33:42] yessir, the ones about user/page, denormalization, and denormalization for graphframes [15:33:48] joal: how'd the job go over the weekend? [15:33:56] milimetric: SUCCESSSSSSSS ! [15:33:59] wow [15:34:03] not even a task error [15:34:09] * milimetric kneels in silent prayer [15:34:22] it made weeks I was targetting that, and finally got their (through pain, but loearning) [15:34:41] got there, not their ;) [15:35:13] Also managed to solve the logging thing -- We now have high level logging that is readable [15:35:25] ok, then I'll finish up the one-off metric computation queries, run them, put dashboards on top of it, and then start working on a tech talk, hopefully before the end of the year [15:35:33] (unless people are on vacation) [15:36:06] milimetric: new year for full 2016 data, or not really important? [15:36:35] yeah, we'll do it after the new year too, but this is our quarterly goal so it should be done before [15:36:43] at least as a test [15:36:51] Ok milimetric, works for me :) [15:36:59] and we can present the full year at metrics meeting in Jan [15:37:12] I'll send an email about that, now that I know everything's basically set [15:37:31] I started a new full run this afternoon with my last changes (logging), to be sure it was not just luck this weekend :0 [15:37:49] if it takes the same amount of time than this weekend, it should be in 13h :) [15:38:27] 10Analytics, 06Operations, 13Patch-For-Review: Install java 8 to stat1002 - https://phabricator.wikimedia.org/T151896#2847201 (10EBernhardson) Thanks! [15:39:07] joal, sure, we can merge them [15:39:19] for me it is also confusing [15:39:36] ok cool -- I'm gonna abandon the 3 old patches and submit a new one [15:49:53] 06Analytics-Kanban, 06Operations, 10Traffic, 13Patch-For-Review: Ganglia varnishkafka python module crashing repeatedly - https://phabricator.wikimedia.org/T152093#2847242 (10elukey) 05Open>03Resolved Merged my changes for varnishkafka statsd monitoring, and Ema cleaned up via salt all the varnishkafka... [15:51:33] 06Analytics-Kanban, 06Operations, 10Traffic, 13Patch-For-Review: Ganglia varnishkafka python module crashing repeatedly - https://phabricator.wikimedia.org/T152093#2847250 (10elukey) 05Resolved>03Open (needs to be moved to the right column of analytics kanban) [15:56:09] (03PS1) 10Joal: Add mediawiki history spark jobs to refinery-job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/325312 (https://phabricator.wikimedia.org/T141548) [15:56:48] (03Abandoned) 10Joal: [WIP] Refactor Mediawiki History scala code [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/301837 (https://phabricator.wikimedia.org/T141548) (owner: 10Joal) [15:57:05] (03Abandoned) 10Joal: [WIP] Join and denormalize all histories into one [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/307903 (owner: 10Milimetric) [15:57:20] (03Abandoned) 10Joal: [WIP] Update edit history to GraphFrames [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/322260 (owner: 10Joal) [15:57:32] Back to clealiness guys :) [16:00:19] elukey: standup :) [16:00:26] elukey: holaaaa [16:30:24] 10Analytics, 10Analytics-Cluster, 06Operations, 06Research-and-Data, and 2 others: GPU upgrade for stats machine - https://phabricator.wikimedia.org/T148843#2847414 (10Nuria) Let's please not add the card to our current 1002 or 1003 as we plan to replace those machines shortly. [16:33:13] (03CR) 10Nuria: "Thanks for spotting this." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/325299 (owner: 10Joal) [16:34:03] (03CR) 10Nuria: "You might want latest, which is 0.39" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/325272 (owner: 10Addshore) [16:34:35] nuria: no no ! [16:34:59] nuria: latest deployed is 0.0.38 -- the one changed in code is for development [16:35:08] Normally maven takes care of this [16:35:11] joal: ah sorry [16:35:16] np nuria :) [16:35:39] When there is -SNAPSHOT at the end, it means development [16:35:39] (03CR) 10Nuria: "Nevermind, this 0.38 is correct." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/325272 (owner: 10Addshore) [16:35:57] mforns, milimetric : coming ? [16:36:05] nuria, oh gosh [16:37:57] 06Analytics-Kanban, 06Discovery, 06Discovery-Analysis (Current work), 03Interactive-Sprint: Add Maps tile usage counts as a Data Cube in Pivot - https://phabricator.wikimedia.org/T151832#2829353 (10Nuria) [16:43:04] 10Analytics: Navigation timing spotty kafka metric - https://phabricator.wikimedia.org/T151907#2847466 (10Nuria) is this related to jmx trans issues? [16:45:53] 10Analytics, 10Dumps-Generation, 13Patch-For-Review, 05Security: Omit private data from being generated during dump runs - https://phabricator.wikimedia.org/T152021#2835669 (10Nuria) We have no knowledge as to what those files are used for. Untagging analytics [16:47:00] 06Analytics-Kanban, 10Analytics-Wikistats: Visual Language for http://stats.wikimedia.org replacement - https://phabricator.wikimedia.org/T152033#2836084 (10Nuria) [16:47:35] 10Analytics: Create SLA alarams for cassandra loding jobs and others - https://phabricator.wikimedia.org/T152109#2847485 (10Nuria) [16:47:56] 10Analytics: Create SLA alarams - https://phabricator.wikimedia.org/T152109#2838676 (10Nuria) [16:49:11] 10Analytics: Create SLA alarms for pageview_hourly jobs - https://phabricator.wikimedia.org/T152109#2838676 (10Nuria) [16:49:45] 06Analytics-Kanban: Create SLA alarms for pageview_hourly jobs - https://phabricator.wikimedia.org/T152109#2847491 (10Nuria) [16:55:09] 10Analytics, 06Security-Team: Statistics on Captcha success/failure rate - https://phabricator.wikimedia.org/T152219#2842060 (10Nuria) FYI that analytics team doesn't have this data. We do not agreggate application data coming from mediawiki for the most part, other than edits. There is some of this data in e... [16:57:35] 10Analytics, 10MediaWiki-General-or-Unknown: Make aggregated MediaWiki Pingback data publicly available - https://phabricator.wikimedia.org/T152222#2842133 (10Nuria) Please take a look at reportupdater, https://wikitech.wikimedia.org/wiki/Analytics/Reportupdater With this tool you can aggregate the data and p... [17:00:52] 06Analytics-Kanban: Redact data so it can be public - https://phabricator.wikimedia.org/T145091#2847562 (10Nuria) [17:04:12] 10Analytics, 10EventBus, 13Patch-For-Review, 06Services (watching): Check eventbus Kafka cluster settings for reliability - https://phabricator.wikimedia.org/T144637#2847577 (10Ottomata) Qs: - With `acks=all`, if leader gets a producer request but the replicas don't ack, does the message still get written... [17:07:40] 10Analytics, 10Datasets-General-or-Unknown: Inspect Pageview API queries (after launch ) {slug} - https://phabricator.wikimedia.org/T117242#2847603 (10Nuria) 05Open>03Resolved [17:11:10] mforns, milimetric : one minute in the cave for me? [17:11:20] joal, sure [17:20:09] 10Analytics: Investigate requests flagged as pageview in analytics header coming from bots - https://phabricator.wikimedia.org/T135251#2847683 (10Nuria) [17:20:11] 10Analytics, 06Research-and-Data-Backlog: Improve bot identification at scale - https://phabricator.wikimedia.org/T138207#2847682 (10Nuria) [17:32:05] ashgrigas: meeting? [17:32:11] (03PS2) 10Joal: Add mediawiki history spark jobs to refinery-job [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/325312 (https://phabricator.wikimedia.org/T141548) [17:32:12] ashgrigas: wikistats tour? [17:32:37] coming [17:32:51] ooh, IRC :) [17:32:57] hi ash [17:33:58] Hi ashgrigas! [17:34:10] Hi ashgrigas :) [17:36:41] Hi!! [17:38:03] 10Analytics, 10Analytics-Wikistats: Design new UI for Wikistats 2.0 - https://phabricator.wikimedia.org/T140000#2449417 (10MelodyKramer) What NPR's internal analytics dashboard looks like: https://s3-eu-west-1.amazonaws.com/uploads-eu.hipchat.com/46822/1125030/qOMDr4LcRhfSNSV/2014-09-24_12-29-49.png [18:01:53] team the new vk alarms are in place, nothing has fired during the past hour :) [18:02:01] I am going afk but will check later on [18:02:05] byyyyeeee o/ [18:03:13] 06Analytics-Kanban, 10Analytics-Wikistats: Visual Language for http://stats.wikimedia.org replacement - https://phabricator.wikimedia.org/T152033#2847879 (10Nuria) https://docs.google.com/presentation/d/1raCaUKujxXDXOjfP5HjRNaGx0AHWh3_c7C2wYyVqHuQ/edit#slide=id.g19efbce7dd_0_0 [18:04:08] cafe time, bbib [18:09:50] 06Analytics-Kanban, 10Continuous-Integration-Config, 10EventBus, 10Wikimedia-Stream: Improve tests for KafkaSSE - https://phabricator.wikimedia.org/T150436#2847916 (10hashar) [18:27:11] 10Analytics, 10EventBus, 10Wikimedia-Stream, 06Services (watching): RecentChanges in Kafka - https://phabricator.wikimedia.org/T152030#2847999 (10Ottomata) > what happened to EventRelayerKafka Guess it hasn't been totally deployed? https://phabricator.wikimedia.org/T134535 [18:33:31] (03PS1) 10Addshore: Fix examples in README [analytics/wmde/toolkit-analyzer] - 10https://gerrit.wikimedia.org/r/325332 [18:33:39] (03CR) 10Addshore: [C: 032] Fix examples in README [analytics/wmde/toolkit-analyzer] - 10https://gerrit.wikimedia.org/r/325332 (owner: 10Addshore) [18:33:43] (03CR) 10Addshore: [V: 032] Fix examples in README [analytics/wmde/toolkit-analyzer] - 10https://gerrit.wikimedia.org/r/325332 (owner: 10Addshore) [18:45:40] 10Analytics, 10EventBus, 10Wikimedia-Stream, 06Services (watching): RecentChanges in Kafka - https://phabricator.wikimedia.org/T152030#2848084 (10Ottomata) > HttpRCFeedEngine would post to the EventBus proxy. Just looked into this a bit. One problem is that we need to augment the event before it is posted... [18:46:50] 10Analytics, 10EventBus, 10Wikimedia-Stream, 06Services (watching): RecentChanges in Kafka - https://phabricator.wikimedia.org/T152030#2848089 (10Ottomata) Oh, I suppose I could make a special formatter for use with the EventBusFeedEngine that would do the parsing and augmentation of the object before it i... [18:54:30] 06Analytics-Kanban, 10Analytics-Wikistats: Navigation model concept development - https://phabricator.wikimedia.org/T152438#2848132 (10ashgrigas) [19:11:46] 10Analytics, 10EventBus, 10Wikimedia-Stream, 06Services (watching): RecentChanges in Kafka - https://phabricator.wikimedia.org/T152030#2836021 (10GWicke) Is there any information on how recentchanges information relates to the events that are already available in eventbus? At the very least, there seems to... [19:14:19] Oh, editing activity metrics disappeared from https://analytics.wikimedia.org/dashboards/vital-signs/ ? [19:19:30] 10Analytics, 10EventBus, 10Wikimedia-Stream, 06Services (watching): RecentChanges in Kafka - https://phabricator.wikimedia.org/T152030#2848278 (10Ottomata) We need recentchanges in Kafka for easy backwards compatibility for Public EventStreams. We plan to deprecate and turn off RCStream in a quarter or tw... [19:26:02] Nemo_bis: yes, they will be appearing again at the end of the quarter [19:26:47] Nemo_bis: we used to calculate those using data from labs but it is not doable. labs db cannot sustain that volume of requests and most days reports did not run for big wikis [19:27:41] Nemo_bis: we are recalculating those from our edit history reconstruction project, so they will come from our .. ahem ('data lake') on hadoop [19:28:41] Nemo_bis: by EOQ we will have edit metrics calculated up to now for all wikis (but they will not update for a while) and next quarter we will productionize updates [19:30:20] milimetric: quick Q about reportupdater -- can report IDs have /'s in them? because we're gonna have MANY queries/scripts and corresponding datasets, and it'd be nice to have them in different subdirectories rather than dozens in just one [19:31:47] bearloga: in a meeting, would you mind responding to the phab / email ping about the raw logs? email subject: raw logs audit [19:32:07] milimetric: ah! yes! will do [19:32:10] sorry [19:37:01] replied [19:38:23] thanks very much bearloga, I'll be with you in a couple hours when this meeting is over (long one) [19:38:44] nuria: I think it's important that I stay in this meeting, so I'll miss the one with Ash on audience prioritization [19:39:07] milimetric: what meeting? [19:39:17] milimetric: are you attending now? [19:39:22] nuria: I'm in the post-election one [19:39:48] milimetric: ah i see, let's reschedule as i cannot make it either cc ashgrigas [19:40:10] ok, I can ping ashgrigas after the meeting, it's over at 16:00 [19:40:10] ashgrigas: can we reschedule audience meeting for tomorrow? [19:40:28] not sure mforns is around that late [19:40:33] nuria sure [19:40:38] milimetric, I am :] [19:40:45] anytime not work tomorrow for everyone? [19:40:53] ok, then we can chat informally at 16:00 EST [19:40:56] and reschedule a time then [19:41:17] ashgrigas: tomorrow at 10:30 (13:30 est) pst should work. [19:42:15] ashgrigas: let's schedule half an hour and if we need more time we can have a follow up? [19:42:20] cc milimetric mforns [19:42:43] 10Analytics, 10EventBus, 10Wikimedia-Stream, 06Services (watching): RecentChanges in Kafka - https://phabricator.wikimedia.org/T152030#2848394 (10GWicke) > We already have the primary source of this stream and the ability to generate it in MediaWiki. I don't really see the benefit in cobbling together a ne... [19:43:17] nuria, ashgrigas, OK [19:43:30] nuria i actually have a conflict and can only do 15:00 et [19:43:32] or the morning [19:44:35] nuria do you need to join? if not, dan, marcel and i can meet today at 16h00 assuming it works for mforns [19:46:12] yeah, we can do both, meet today and follow up tomorrow morning to see what's left to do [19:47:15] ok [19:48:23] 10Analytics, 10EventBus, 10Wikimedia-Stream, 06Services (watching): RecentChanges in Kafka - https://phabricator.wikimedia.org/T152030#2848406 (10Ottomata) ORES probably should be converted to use `revision-create`, and the we could use change prop to generate an augmented `revision-create` stream with ORE... [19:48:53] ashgrigas: sounds good [19:51:20] ashgrigas, sure [19:51:48] ok 16h00 it is [19:52:17] also question about terminology - on the monthly report card it says projects and shows english french etc [19:52:23] are those projects or wikis? [19:52:43] i thought wikis were the individual language sites [19:52:50] ashgrigas, wmf uses both terms interchangably [19:52:54] projects were like wikipedia, wiktionary etc [19:52:58] I also think it is confusing [19:53:03] oh [19:53:05] hmm [19:53:06] yeah [19:53:08] very [19:54:54] ashgrigas, I guess maybe in paper projects are wikipedia, wiktionary, etc. But in practice we say 'en.wikipedia.org' is a project [19:55:02] so there could be 28 (languages) + 18 (project names) total projects? [19:55:35] yeah so we can't rely distinguish a group of top-level project and then wikis underneath that [19:55:43] if people except something different [19:57:01] ashgrigas, VitalSigns dashboard shows 'projects' as Wikipedia, Wiktionary, etc. So, maybe that is actually the correct thing, and the rest is abuse of language [19:57:22] ok [20:00:04] ashgrigas, regarding languages, Wikistats is translated to 28 languages but Wikimedia projects (<- correctly using this word now :]) there are almost 300 languages [20:10:26] ok so of the projects though we only surface the 28 languages in wikistats even though 300 exist right [20:10:28] 10Analytics, 10EventBus, 10Wikimedia-Stream, 06Services (watching): RecentChanges in Kafka - https://phabricator.wikimedia.org/T152030#2848509 (10GWicke) @Ottomata, ORES pre-generation [already uses revision-create](https://github.com/wikimedia/change-propagation/blob/2192620fbd761567930c2477f9722fd69c5dda... [20:18:37] 10Analytics, 10EventBus, 10Wikimedia-Stream, 06Services (watching): RecentChanges in Kafka - https://phabricator.wikimedia.org/T152030#2848542 (10Ottomata) > The returned information is not yet integrated into the RC feed though. Why would you integrate it into the RC feed, if ORES only cares about revisio... [20:28:51] ashgrigas, sorry I didn't see your message [20:30:34] ashgrigas, no, Wikistats shows stats for all languages, but its "narrative" (intro text, menus, labels, etc.) is available in 28 languages [20:31:57] 10Quarry, 06Labs, 10Tool-Labs: Clarify Tool Labs' rules to see if Quarry and PAWS are allowed to be hosted there - https://phabricator.wikimedia.org/T152212#2841828 (10bd808) I don't think either of these projects qualifies as "unauthenticated" in that in my understanding OAuth is used to authenticated the r... [20:31:59] you mean the site content itself is only available [20:32:02] @mforns [20:32:11] in the 28 [20:32:15] ashgrigas, yes [20:32:15] but the data is for all 300 [20:32:19] exactly [20:33:19] got it [20:34:09] ottomata: can you delete /a/squid/archive/zero ? Files left in there are owned by you and root [20:45:52] oh! [20:45:52] k [20:46:02] totally delete it milimetric [20:46:09] type the name of your cat to confirm [20:46:15] ? [20:46:25] 10Quarry, 06Labs, 10Tool-Labs: Clarify Tool Labs' rules to see if Quarry and PAWS are allowed to be hosted there - https://phabricator.wikimedia.org/T152212#2848664 (10chasemp) I take the term `unauthenticated` to mean end users are not directly authenticating to the resources under consumption, i.e. obfusca... [20:46:42] Are you sure you want to delete /a/squid/archive/zero? ( / no): [20:48:44] xD [20:56:26] 10Quarry, 06Labs, 10Tool-Labs: Clarify Tool Labs' rules to see if Quarry and PAWS are allowed to be hosted there - https://phabricator.wikimedia.org/T152212#2848690 (10bd808) The [[https://wikitech.wikimedia.org/w/index.php?title=Nova_Resource:Tools/Rules&diff=prev&oldid=120104|existing rule was introduced]]... [21:00:03] ashgrigas, mforns: batcave? [21:00:10] milimetric, ok [21:00:54] ok [21:01:44] ottomata: lol, tiger lily [21:01:57] trying to join [21:02:04] ashgrigas: https://hangouts.google.com/hangouts/_/wikimedia.org/a-batcave [21:02:12] I'll invite you again, sometimes it breaks [21:11:58] ottomata: !? [21:12:09] ottomata: is spinlister like an airbnb for bikes? [21:12:28] ya! [21:12:37] that's... awesome [21:12:57] do people buy bikes for this purpose, or are they renting their own? [21:15:01] seems like some of them will install pedals on request, that seems like the former [21:21:10] 06Analytics-Kanban, 10Continuous-Integration-Config, 10EventBus, 06Release-Engineering-Team, 10Wikimedia-Stream: Improve tests for KafkaSSE - https://phabricator.wikimedia.org/T150436#2848823 (10hashar) I am merely brain dumping my understanding for this task: node-rdkafka are nodejs bindings for librdk... [21:28:04] urandom: dunno, i haven't actually used it, but i think milimetric has, and a buddy of mine has too [21:29:39] milimetric: done [21:55:39] logging off team, have a nice end of day [22:17:24] urandom: it was a pleasant experience, I only did it once [22:17:47] milimetric: cool [22:17:51] if you know bikes you'll be able to tell pretty well how the ride will be from the description and short conversation [22:17:52] it looks too good to be true! [22:18:04] I did get this person's lights stolen [22:18:10] but she was ok if I just gave her cash for them [22:18:19] yikes [22:18:43] I'd be careful with SF - it's nuts when it comes to theft, urandom [22:19:03] we've had multiple car windows smashed and bike stuff stolen and we only visit! [22:19:48] i'll wait and see if there are other interested, and what they want, but my preference would be to head out on quieter roads, and away from SF [22:19:50] so I'd bring a hefty lock with you or make sure the spinlister person has one [22:20:31] and i see myself only getting off the bike to do my business, and get some lunch [22:20:32] there are definitely cool places to go. I'd say the mountain biking is the best around SF, maybe an hour or so away there's amazing single track and other types [22:20:40] :) good, yeah, then you'll be fine [22:21:03] the paradise loop route sounds pretty good to me so far [22:21:34] yeah, Andrew knows the best routes :)