[06:33:35] Analytics-Kanban, Analytics-Wikimetrics: Usernames with commas not supported {dove} - https://phabricator.wikimedia.org/T129422#2154307 (mforns) a:mforns [06:58:44] Analytics-Kanban, Analytics-Wikimetrics: Usernames with commas not supported {dove} - https://phabricator.wikimedia.org/T129422#2154314 (mforns) @NickK Thanks for creating this task. I checked the code and there are currently 2 ways you can create cohorts containing usernames with commas: 1. Enclose usern... [07:06:50] Analytics-Kanban: Add (and default to) a breakdown in percentages also for the line chart. - https://phabricator.wikimedia.org/T130406#2154316 (mforns) a:mforns [07:20:58] Analytics-Kanban: Add (and default to) a breakdown in percentages also for the line chart. - https://phabricator.wikimedia.org/T130406#2154320 (mforns) @Nuria @Milimetric Do we want to have a separate report that calculates percentages? And have two separate charts? Or to change the line chart to be able to... [08:34:51] Analytics, Operations, Patch-For-Review: nf_conntrack warnings for kafka hosts - https://phabricator.wikimedia.org/T131028#2154356 (elukey) ```ipv4 2 tcp 6 73 TIME_WAIT src=10.64.16.127 dst=10.64.53.12``` TIME_WAIT is a state for the TCP actor that does the active close, in this case the src... [12:09:40] Analytics-Tech-community-metrics, Developer-Relations, DevRel-March-2016, Patch-For-Review, developer-notice: Check whether it is true that we have lost 40% of (Git) code contributors in the past 12 months - https://phabricator.wikimedia.org/T103292#2154443 (Lcanasdiaz) >>! In T103292#2147983, @... [12:19:36] Analytics-Tech-community-metrics, Developer-Relations, DevRel-March-2016, Patch-For-Review, developer-notice: Check whether it is true that we have lost 40% of (Git) code contributors in the past 12 months - https://phabricator.wikimedia.org/T103292#2154458 (Lcanasdiaz) This ticket is blocked by... [12:22:02] Analytics-Tech-community-metrics, Developer-Relations, DevRel-March-2016, Patch-For-Review, developer-notice: Check whether it is true that we have lost 40% of (Git) code contributors in the past 12 months - https://phabricator.wikimedia.org/T103292#2154489 (Lcanasdiaz) [12:28:54] Analytics-Kanban: Fix 1-week lag in executing browser reports {lama} - https://phabricator.wikimedia.org/T131049#2154493 (mforns) [13:08:09] Analytics-Tech-community-metrics, Labs, DevRel-March-2016: Port 22 of korma.wmflabs.org not reachable - https://phabricator.wikimedia.org/T131046#2154538 (Dereckson) [13:12:49] Analytics-Kanban: Fix 1-week lag in executing browser reports {lama} - https://phabricator.wikimedia.org/T131049#2154541 (mforns) [13:15:10] Analytics-Tech-community-metrics, Labs, DevRel-March-2016: External service can't login through SSH to korma.wmflabs.org - https://phabricator.wikimedia.org/T131046#2154542 (Dereckson) [13:15:54] Analytics-Tech-community-metrics, Labs, DevRel-March-2016: External service can't login through SSH to korma.wmflabs.org - https://phabricator.wikimedia.org/T131046#2154444 (Nemo_bis) Just in case: docs on how to access wmflabs.org instances are at https://wikitech.wikimedia.org/wiki/Help:Access [13:16:52] Analytics-Tech-community-metrics, Labs, DevRel-March-2016: External service can't login through SSH to korma.wmflabs.org - https://phabricator.wikimedia.org/T131046#2154545 (Dereckson) The server korma.wmflabs.org listens publicly to 22 and accept incoming SSH connections. Could you try from the remot... [13:20:20] Analytics-Kanban: Fix 1-week lag in executing browser reports {lama} - https://phabricator.wikimedia.org/T131049#2154547 (mforns) I edited the task's description and removed the 'solution' paragraph, that was: ``` Solution: Apply the `lag` offset in a different moment in the pipeline. Not in `get_interval_rep... [13:45:41] mforns: merged! sorry that took so long, I meant to ask you about it and then forgot [13:46:56] ottomata, thank you! no problem. Was it OK for you? there were almost no logs before. This is going to generate one line for each report executed and a couple lines <10 for each execution of reportupdater [13:47:45] shouldn't be very verbose [13:50:23] puppet was fine [13:50:26] the cron job was updated [13:50:29] haven't checked logs on stat1002 [13:52:17] ok [14:02:30] Analytics, Operations, Patch-For-Review: nf_conntrack warnings for kafka hosts - https://phabricator.wikimedia.org/T131028#2154609 (Ottomata) Last week, ApiAction Kafka logging was turned on from MW. I'm not sure how much this would have increased the number of connections from MW. Is it possible th... [14:16:29] Analytics-Tech-community-metrics, Labs, DevRel-March-2016: External service can't login through SSH to korma.wmflabs.org - https://phabricator.wikimedia.org/T131046#2154444 (Krenair) ```alex@alex-laptop:~$ ssh korma.wmflabs.org Linux novaproxy-01 3.16.0-4-amd64 #1 SMP Debian 3.16.7-ckt20-1+deb8u4 (201... [14:50:40] Analytics-Kanban, EventBus, Patch-For-Review: Prefix eventbus Kafka topics with datacenter - https://phabricator.wikimedia.org/T130562#2154656 (Ottomata) [14:51:32] Analytics-Kanban, EventBus, Patch-For-Review: Prefix eventbus Kafka topics with datacenter - https://phabricator.wikimedia.org/T130562#2139475 (Ottomata) Doh! I just realized this morning that I don't need to do any code changes to make this work! The eventlogging kafka writer already has a topic_fo... [15:16:35] ottomata: also edits on User pages are not counted either [15:16:42] ah ok [15:16:50] ottomata: bot page is a user page: https://en.wikipedia.org/wiki/User:Cyberbot_I [15:16:57] aye [15:24:08] Analytics, Operations, hardware-requests, Patch-For-Review: eqiad: (3) AQS replacement nodes - https://phabricator.wikimedia.org/T124947#2154710 (Ottomata) Bump [15:44:35] Friends, I'm looking for some reassurance that kafkatee is still our glue of choice for simple Kafka stream processing jobs. [15:45:50] So: Vital Signs [15:46:01] The search box on the left to add projects does not seem to be responding to my input. [15:46:26] Oh, now it works. [15:46:48] awight: hmmm, glue of choice... [15:46:49] maybe [15:46:51] maybe not! [15:47:00] in meetings now but can kinda chat [15:47:03] what are you trying to do? [15:47:17] Hehe. Are there alternatives I should be looking at? [15:47:26] No rush to figure it all out now. [15:48:10] I'm trying to evaluate our choices for replacing ActiveMQ as the buffering service for Fundraising. [15:48:50] what are you buffering? how are you consuming? what are you doing when you consume? [15:48:58] i assume you'd be doing this in frack? [15:49:14] Kafka looks like the most promising, and Foundation-compatible, technology, although I worry I was sucked on my the marketing hype. [15:49:26] kafka probably is :), but kafkatee might not be [15:49:27] depends [15:50:16] This is currently several queues, traffic of only ten messages per second or so, running on frack because it contains donor PII. [15:50:32] awight: wanna do quick google hangout later to brainbounce? [15:51:14] Tmi: https://docs.google.com/a/wikimedia.org/spreadsheets/d/1lkle5RJNG33z6c3u1czQS_0scnyLX-X57Sh4W3dyCl4/edit?usp=docslist_api [15:51:38] ottomata: thanks, I'd love to. [15:51:57] Have to turn the cell off for court, out on a few hours. [15:52:28] Should I just grab a slot on your calendar? [15:56:30] ja sounds good [15:56:33] i'm on east coast time [15:56:43] sometime around 16:00 east coast today would be good [15:56:46] I like that my job could be completely Kafka agnostic, but my biggest hesitation about kafkatee is that I need to be transactional, and leave the consumer offset uncommitted sometimes. [15:56:48] i have an appointment at14:00 [15:57:01] Awesome, will do. Thanks! [16:00:37] Analytics-Kanban, Patch-For-Review: Add legends to every graph and let them filter by date - https://phabricator.wikimedia.org/T129497#2154824 (Nuria) [16:03:54] Analytics-Kanban, Patch-For-Review: Load Avro schemas from configurable external path - https://phabricator.wikimedia.org/T126501#2154836 (Nuria) [16:07:43] Analytics-Kanban, Operations, Patch-For-Review: nf_conntrack warnings for kafka hosts - https://phabricator.wikimedia.org/T131028#2154847 (Nuria) [16:12:18] Analytics: Make webrequest load and refine jobs a single bundle - https://phabricator.wikimedia.org/T130731#2144802 (Nuria) Is this different from: https://phabricator.wikimedia.org/T130734? [16:13:08] Analytics: Check if we can deprecate legacy TSVs production (same time as pagecounts?) - https://phabricator.wikimedia.org/T130729#2144772 (Nuria) Operational excellence: work for next quarter. [16:13:22] Analytics: Check if we can deprecate legacy TSVs production (same time as pagecounts?) - https://phabricator.wikimedia.org/T130729#2154869 (Nuria) p:Triage>Normal [16:14:27] Analytics: Evaluate deprecation of udp-like files produced on oozie (legacy tsvs) - https://phabricator.wikimedia.org/T130725#2154871 (Nuria) Closing as duplicate of https://phabricator.wikimedia.org/T130729 [16:14:34] Analytics: Evaluate deprecation of udp-like files produced on oozie (legacy tsvs) - https://phabricator.wikimedia.org/T130725#2154872 (Nuria) Open>declined [16:15:04] Analytics: Stop generating pagecounts-raw and pagecounts-all-sites - https://phabricator.wikimedia.org/T130656#2142207 (Nuria) Operational excellence next quarter. [16:18:29] Analytics-Cluster, Analytics-Kanban: Story: Community has periodic browser stats report generated from Hadoop data - https://phabricator.wikimedia.org/T69053#2154892 (Nuria) [16:19:45] Analytics-Kanban: Browser reports improvements (parent task) - https://phabricator.wikimedia.org/T130405#2154898 (Nuria) [16:29:28] Analytics, Operations, hardware-requests, Patch-For-Review: eqiad: (3) AQS replacement nodes - https://phabricator.wikimedia.org/T124947#2154934 (RobH) a:RobH>mark [16:31:06] Analytics, Operations, hardware-requests, Patch-For-Review: eqiad: (3) AQS replacement nodes - https://phabricator.wikimedia.org/T124947#1970805 (RobH) I accidentally assigned in another person, and didn't assign the task to mark. Not sure how I did that, but it didn't reveal any private info. E... [17:00:36] (PS1) Nuria: Making link to new browser reports more prominent [analytics/wikistats] - https://gerrit.wikimedia.org/r/279965 (https://phabricator.wikimedia.org/T129101) [17:01:34] Analytics-Kanban, Patch-For-Review: Integrate new browser visualization into wikistats - https://phabricator.wikimedia.org/T129101#2155051 (Nuria) I think we should modify the text on reports to make more prominent the location of new report plus we should remove the text that indicates that migration of... [17:10:44] bye a-team, see you tomorrow! [17:27:12] Analytics, Hovercards: Capture hovercards fetches as previews in analytics - https://phabricator.wikimedia.org/T129425#2155146 (Nuria) >Would you please elaborate? The Android app uses "preview" for link previews, which behave analogously. Indeed. You are correct and I am mistaken. I though android was d... [17:32:23] Analytics, Hovercards: Capture hovercards fetches as previews in analytics - https://phabricator.wikimedia.org/T129425#2155163 (Tbayer) >>! In T129425#2155146, @Nuria wrote: >>Would you please elaborate? The Android app uses "preview" for link previews, which behave analogously. > Indeed. You are correct... [17:35:13] Analytics-Kanban: Communicate the WikimediaBot convention {hawk} - https://phabricator.wikimedia.org/T108599#2155185 (Nuria) Open>Resolved [17:35:29] Analytics-Kanban: Parse User-Agent strings with OS like "Windows 7" correctly into the user agent map {hawk} - https://phabricator.wikimedia.org/T127324#2155187 (Nuria) [17:35:31] Analytics-Kanban, Patch-For-Review: Update UA-Parser version to 1.3.0-wmf3 in refinery-source - https://phabricator.wikimedia.org/T130399#2155186 (Nuria) Open>Resolved [17:38:15] dr0ptp4kt: replication of events doesn't happen right away [17:38:22] dr0ptp4kt: see now: [17:38:25] https://www.irccloud.com/pastebin/7a2lj9l6/ [17:40:34] Analytics, Wikipedia-iOS-App-Product-Backlog, iOS-app-feature-ANALYTICS, iOS-app-v5.0.2-Rubiks: Fix iOS uniques in mobile_apps_uniques_daily after 5.0 launch - https://phabricator.wikimedia.org/T130432#2135848 (JMinor) a:JMinor [17:43:49] nuria: phew! [17:43:56] nuria: thanks :) [17:44:25] nuria: would you recommend querying elsewhere in this case where the replag is really high? [17:45:50] dr0ptp4kt: you can tail kafka [17:46:28] from stat1002: kafkacat -b kafka1012:9092 -t eventlogging_MobileWebLanguageSwitcher_15302503 [17:46:45] you can also add -o -10 to see last messages from last 10 offsets [17:49:05] madhuvishy: right, good point. i see that's on the wiki page, forgot about that [17:49:20] madhuvishy: although in this case [17:49:45] madhuvishy: i'm getting the following error: [17:49:48] madhuvishy: ERROR: Topic eventlogging_MobileWebLanguageSwitcher_15302503 error: Broker: Leader not available [17:49:53] aah [17:50:03] may be kafka1012 is taking a break [17:50:13] you can use a list of broker strings instead [17:50:31] let me get you the full string [17:53:58] dr0ptp4kt: kafka1012.eqiad.wmnet:9092,kafka1013.eqiad.wmnet:9092,kafka1014.eqiad.wmnet:9092,kafka1018.eqiad.wmnet:9092,kafka1020.eqiad.wmnet:9092,kafka1022.eqiad.wmnet:9092 [17:54:54] madhuvishy: where does this go? [17:55:03] madhuvishy: that is, in a config file or command line or something? [17:55:04] in the -b option [17:55:12] instead of kafka1012:9092 [17:56:19] dr0ptp4kt: the command will now be kafkacat -b kafka1012.eqiad.wmnet:9092,kafka1013.eqiad.wmnet:9092,kafka1014.eqiad.wmnet:9092,kafka1018.eqiad.wmnet:9092,kafka1020.eqiad.wmnet:9092,kafka1022.eqiad.wmnet:9092 -t eventlogging_MobileWebLanguageSwitcher [17:56:35] also I see I put the revision number in the command i gave above [17:56:38] that was wrong [17:56:54] the topic names don't have revisions in them [17:57:30] oh, ok, i vaguely remember that now that you say it about no revision number [17:57:51] madhuvishy: look at that lovely stream of events. joy [17:58:01] :D [17:59:15] dr0ptp4kt: if you are testing in beta you can do the same too - the kafka broker is at deployment-kafka02:9092 [18:58:00] Analytics-EventLogging, MediaWiki-Vagrant: EventLogging vagrant role fails to provision - https://phabricator.wikimedia.org/T131085#2155603 (Jdforrester-WMF) [19:08:00] Analytics-Tech-community-metrics, Labs, DevRel-March-2016: External service can't login through SSH to korma.wmflabs.org - https://phabricator.wikimedia.org/T131046#2155632 (Lcanasdiaz) >>! In T131046#2154530, @Dereckson wrote: > @Lcanasdiaz As Korma is still operational in read only mode, Unbreak now!... [19:21:10] Analytics-Tech-community-metrics, Developer-Relations, DevRel-March-2016, Patch-For-Review, developer-notice: Check whether it is true that we have lost 40% of (Git) code contributors in the past 12 months - https://phabricator.wikimedia.org/T103292#2155666 (Lcanasdiaz) [19:21:12] Analytics-Tech-community-metrics, Labs, DevRel-March-2016: External service can't login through SSH to korma.wmflabs.org - https://phabricator.wikimedia.org/T131046#2155663 (Lcanasdiaz) Open>Resolved a:Lcanasdiaz It is working using the IP 208.80.155.168. Thank you guys! :) [19:35:17] madhuvishy: thx! [20:03:09] (CR) Erik Zachte: [V: 1] "I played around with wording and layout a bit." [analytics/wikistats] - https://gerrit.wikimedia.org/r/279965 (https://phabricator.wikimedia.org/T129101) (owner: Nuria) [20:24:55] bmansurov: ^ go up up up a bit for my exchange with madhuvishy. it looks like s1-analytics-slave is lagging (still) behind the primary master db. if you run a query on analytics-store instead you'll see newer records. nonetheless, i appreciate the fine attention to the event logging :) [20:25:23] ok, reading [20:27:21] makes sense [20:39:48] (CR) Nuria: [C: 1] "Looks good, thank you! I will let you merge." [analytics/wikistats] - https://gerrit.wikimedia.org/r/279965 (https://phabricator.wikimedia.org/T129101) (owner: Nuria) [21:03:25] bd808: hiiii is it working? [23:02:53] Analytics-Kanban: [Spike] Figure out how to automate releases with jenkins {hawk} - https://phabricator.wikimedia.org/T130576#2156462 (madhuvishy) @hashar Hi! I could use some help on this task - I've been trying to use the Maven release Jenkins plugin - https://wiki.jenkins-ci.org/display/JENKINS/M2+Release+...