[05:57:56] <wikibugs>	 Analytics-EventLogging, Analytics-Kanban, Patch-For-Review: Add the schema name to the EL EventError topic [8 pts] - https://phabricator.wikimedia.org/T115121#1720663 (madhuvishy)
[06:44:15] <wikibugs>	 Analytics-Tech-community-metrics, Possible-Tech-Projects: Improving MediaWikiAnalysis - https://phabricator.wikimedia.org/T89135#1720707 (ashitaprasad) @jgbarah @Dicortazar  Hi,  I am Ashita Prasad and I am interested in pursuing this project for @Outreachy-Round-11. I am an avid Pythonista and have expe...
[08:42:36] <joal>	 Hi a-team
[08:42:45] <joal>	 cassandra load for the weekend is a fail :(
[08:43:30] <joal>	 Some of it has worked, but most of the important one (by article) hasn't, with wrong state (no error in oozie job while the job has failed)
[08:43:39] <joal>	 I'll spend the beginning of the week on that
[08:49:18] <wikibugs>	 Analytics-Tech-community-metrics, Possible-Tech-Projects: Improving MediaWikiAnalysis - https://phabricator.wikimedia.org/T89135#1720801 (Aklapper) >>! In T89135#1720707, @ashitaprasad wrote: > Can you please guide me to gain some traction and contribute to this project.  Hi @ashitaprasad. Thanks for you...
[10:15:32] <wikibugs>	 Analytics-Kanban, Database: Delete obsolete schemas {tick} [5 pts] - https://phabricator.wikimedia.org/T108857#1720969 (mforns)
[14:08:51] <wikibugs>	 Analytics-Backlog, Research-and-Data: Historical analysis of edit productivity for English Wikipedia - https://phabricator.wikimedia.org/T99172#1721452 (JAllemandou)
[14:26:49] <halfak>	 o/ joal
[14:27:03] <joal>	 Hi halfak !
[14:27:08] <joal>	 How are you Sir ?
[14:27:32] <halfak>	 Not bad.  Was thinking that we could use the altiscale call to talk about the memory usage issues.
[14:27:51] <joal>	 feasible :)
[14:29:19] <halfak>	 cool
[14:40:44] <joal>	 halfak: can you send an invite for me not to forget ?
[14:42:18] <halfak>	 joal, looks like it is on your calendar to me.
[14:42:27] * halfak uninvites and reinvites joal
[14:42:45] <joal>	 ooooh halfak wait
[14:43:18] <joal>	 It is on my calendar indeed, but I didn't notice it because it overlaps with the new time for our standup
[14:43:29] <joal>	 halfak: --^
[14:43:39] <halfak>	 Boo!
[14:43:42] <halfak>	 hmm
[14:43:53] <joal>	 halfak: Can we move it half an hour earlier this time (since only us) ?
[14:44:05] <halfak>	 I'll check with the altiscale people.
[14:44:14] <halfak>	 They're in PDT, so they probably won't like it.
[14:44:20] <joal>	 ok, thanks and sorry for the noise :(
[14:44:23] <halfak>	 Hmm... 30 minutes later would probably be easier.
[14:44:42] <joal>	 today not feasible for me, tasking (but usually on monday)
[14:45:07] <mforns>	 joal, hi!
[14:45:11] <joal>	 Hey mforns
[14:45:15] <joal>	 how are you ?
[14:45:25] <halfak>	 OK.  I think we shouldn't worry about this week's meeting.
[14:45:28] <mforns>	 hey do you have a sec for questions on https://phabricator.wikimedia.org/T113255?
[14:45:40] <mforns>	 joal, good, thanks you?
[14:45:50] <joal>	 yeah good :)
[14:45:58] <joal>	 no prob halfak
[14:46:10] <joal>	 please mforns
[14:46:14] <joal>	 shoot :)
[14:46:16] <mforns>	 batcave?
[14:46:29] <joal>	 sure
[14:46:33] <halfak>	 joal, one thing.
[14:46:37] <joal>	 yes ?
[14:46:52] <halfak>	 Maybe we switch the day of the week to Weds and move back an hour.
[14:46:55] <halfak>	 Would that work?
[14:47:01] <halfak>	 So, it would be an hour later
[14:47:17] <joal>	 halfak: works for me :)
[14:48:07] <halfak>	 OK cool.
[14:50:46] <nuria>	 holaaa
[14:50:55] <nuria>	 joal: do we have a ticklet for teh cassandra loads?
[14:50:59] <nuria>	 *the
[14:51:15] <joal>	 hey nuria
[14:51:28] <wikibugs>	 Analytics-EventLogging, Multimedia, UploadWizard: Half the time, 100% of UploadWizardExceptionFlowEvent events are dropped - https://phabricator.wikimedia.org/T113366#1721681 (Jdforrester-WMF)
[14:51:39] <wikibugs>	 Analytics-EventLogging, Multimedia, UploadWizard: Half the time, 100% of UploadWizardExceptionFlowEvent events are dropped - https://phabricator.wikimedia.org/T113366#1721682 (Jdforrester-WMF) Open>Resolved a:Jdforrester-WMF
[14:52:07] <mforns>	 hey ottomata yt?
[14:52:21] <joal>	 nuria: I was planning on reusing the same cassandra load jobs ticket
[14:52:37] <joal>	 nuria: probably better to clse this current one and reopen a new one
[14:52:47] <nuria>	 ajam, I think is fine to reuse
[14:52:56] <joal>	 ok cool nuria
[14:53:06] <nuria>	 but we should have two tickets , one for loading code, other for the actual loading
[14:53:15] <joal>	 nuria: do we take a minute about your comments on task ?
[14:53:24] <joal>	 nuria: good idea
[14:53:30] <nuria>	 joal:yessisr
[14:53:34] <nuria>	 batcave?
[14:53:37] <nuria>	 or here?
[14:53:38] <joal>	 omw
[14:54:09] <joal>	 nuria: cave ?
[14:54:25] <nuria>	 there .. almost...
[15:16:14] <ottomata>	 mforns:  sorta not really :)
[15:16:21] <ottomata>	 am in meeting with ops folks
[15:16:31] <mforns>	 ottomata, no problemo
[15:41:01] <nuria>	 joal: should i grab this ticket: https://phabricator.wikimedia.org/T110061
[15:41:11] <nuria>	 joal: or are you working on it?
[15:41:21] <joal>	 Please go for it nuria
[15:42:11] <joal>	 code is waiting for you here: https://phabricator.wikimedia.org/T109739 (example of format, plus the fields to load)
[15:42:16] <joal>	 nuria: --^
[15:42:24] <joal>	 let me know if question nuria
[15:49:37] <nuria>	 milimetric: yt?
[15:53:51] <wikibugs>	 Analytics-Kanban, Analytics-Wikistats: Wikistats report subtask  (placeholder)  - https://phabricator.wikimedia.org/T115344#1721955 (Nuria) NEW a:Milimetric
[16:00:19] <nuria>	 milimetric: standuppp?
[16:00:23] <nuria>	 mforns: holaaa?
[16:01:46] <milimetric>	 I was grabbing lunch, nuria, I probably won't be around before standup from now on
[16:01:52] <milimetric>	 (i mean like 30 min. before)
[16:01:55] <nuria>	 milimetric: k
[16:02:07] <krrrit-wm>	 (PS1) Mforns: Add percent loss to refinery-dump-status script [analytics/refinery] - https://gerrit.wikimedia.org/r/245921 (https://phabricator.wikimedia.org/T113255)
[16:04:52] <wikibugs>	 Analytics-Kanban: Write in-depth dashiki documentation {crow} [3 pts} - https://phabricator.wikimedia.org/T112685#1721989 (ggellerman)
[16:05:05] <wikibugs>	 Analytics-Kanban: Write in-depth dashiki documentation {crow} [3 pts] - https://phabricator.wikimedia.org/T112685#1642517 (ggellerman)
[16:09:21] <wikibugs>	 Analytics-Cluster, Analytics-Kanban, Patch-For-Review: Setup pipeline for search logs to travel through kafka and camus into hadoop {hawk} [55 pts] - https://phabricator.wikimedia.org/T113521#1722010 (ggellerman)
[16:12:06] <krrrit-wm>	 (CR) Mforns: Add percent loss to refinery-dump-status script (1 comment) [analytics/refinery] - https://gerrit.wikimedia.org/r/245921 (https://phabricator.wikimedia.org/T113255) (owner: Mforns)
[16:30:06] <milimetric>	 getting an error joining the hangout again
[16:34:58] <wikibugs>	 Analytics-Backlog, Analytics-Cluster: Update last access jobs to account for nocookie header - https://phabricator.wikimedia.org/T115350#1722076 (Nuria) NEW
[16:40:09] <wikibugs>	 Analytics-Backlog: Improve loading Analytics Query Service with data {slug} [ pts] - https://phabricator.wikimedia.org/T115351#1722097 (Milimetric) NEW
[16:42:20] <wikibugs>	 Analytics-Engineering, Community-Tech: [AOI] Add page view statistics to page information pages (action=info) - https://phabricator.wikimedia.org/T110147#1722108 (DannyH) Is it possible to represent that data as weekly numbers instead of daily? There's a pattern of use over the course of a week -- I think...
[16:48:35] <wikibugs>	 Analytics-Engineering, Community-Tech: [AOI] Add page view statistics to page information pages (action=info) - https://phabricator.wikimedia.org/T110147#1722117 (kaldari) @DannyH: Yes, it would definitely be possible, although it would be more difficult since the API doesn't provide any roll-ups larger th...
[16:52:24] <wikibugs>	 Analytics-Backlog: Improve loading Analytics Query Service with data {slug} [ pts] - https://phabricator.wikimedia.org/T115351#1722133 (Nuria)
[16:52:50] <wikibugs>	 Analytics-Backlog:  improve timeuuid writing  - https://phabricator.wikimedia.org/T115353#1722135 (Nuria) NEW
[16:56:22] <wikibugs>	 Analytics-Backlog:  run job using oozie <mapreduce> rather than <java> action get proper error feedback from jobs and how they run - https://phabricator.wikimedia.org/T115355#1722168 (Nuria) NEW
[16:56:35] <wikibugs>	 Analytics-Backlog: Improve loading Analytics Query Service with data {slug} [ pts] - https://phabricator.wikimedia.org/T115351#1722174 (Nuria)
[16:56:56] <wikibugs>	 Analytics-Backlog: special character stripping on cassandra loading (like tabs) - https://phabricator.wikimedia.org/T115356#1722180 (Nuria) NEW
[16:57:43] <wikibugs>	 Analytics-Backlog:  improve timeuuid writing {slug} [5 pts]  - https://phabricator.wikimedia.org/T115353#1722186 (kevinator) p:Triage>Normal
[16:58:14] <wikibugs>	 Analytics-Backlog: special character stripping on cassandra loading (tabs) - https://phabricator.wikimedia.org/T115356#1722180 (Nuria)
[17:01:17] <wikibugs>	 Analytics-Backlog: special character stripping on cassandra loading (tabs) {slug} [5 pts] - https://phabricator.wikimedia.org/T115356#1722222 (kevinator) p:Triage>High
[17:01:54] <wikibugs>	 Analytics-Backlog: put less pressure on Cassandra - it currently fails {slug} [ pts] - https://phabricator.wikimedia.org/T115359#1722226 (Milimetric) NEW
[17:03:23] <wikibugs>	 Analytics-Backlog:  run job using oozie <mapreduce> rather than <java> action {slug} [13 pts] - https://phabricator.wikimedia.org/T115355#1722234 (Milimetric)
[17:03:34] <wikibugs>	 Analytics-Backlog:  run job using oozie <mapreduce> {slug} [13 pts] - https://phabricator.wikimedia.org/T115355#1722239 (kevinator) p:Triage>High
[17:03:41] <wikibugs>	 Analytics-Backlog: Improve loading Analytics Query Service with data {slug} - https://phabricator.wikimedia.org/T115351#1722242 (Milimetric)
[17:04:58] <wikibugs>	 Analytics-Backlog: Improve loading Analytics Query Service with data {slug} - https://phabricator.wikimedia.org/T115351#1722097 (Milimetric)
[17:04:59] <wikibugs>	 Analytics-Backlog: put less pressure on Cassandra - it currently fails {slug} [ pts] - https://phabricator.wikimedia.org/T115359#1722254 (Milimetric) Open>Invalid a:Milimetric
[17:05:36] <wikibugs>	 Analytics-Backlog: run job using oozie <mapreduce> {slug} [13 pts] - https://phabricator.wikimedia.org/T115355#1722257 (Milimetric)
[17:06:44] <wikibugs>	 Analytics-Backlog: Improve loading Analytics Query Service with data {slug} [subtasked] - https://phabricator.wikimedia.org/T115351#1722269 (Milimetric) p:Triage>High
[17:09:49] <wikibugs>	 Analytics-Backlog: cassandra backfill monitoring {slug] - https://phabricator.wikimedia.org/T115360#1722283 (kevinator) NEW
[17:10:20] <wikibugs>	 Analytics-Kanban, Traffic, operations: Flag in x-analytics in varnish any request that comes with no cookies whatsoever - https://phabricator.wikimedia.org/T114370#1722291 (Milimetric)
[17:10:44] <wikibugs>	 Analytics-Kanban, Traffic, operations: Flag in x-analytics in varnish any request that comes with no cookies whatsoever - https://phabricator.wikimedia.org/T114370#1722297 (Nuria) https://gerrit.wikimedia.org/r/#/c/244626/
[17:11:55] <wikibugs>	 Analytics-Backlog: optimize Analytics Query Service {slug} - https://phabricator.wikimedia.org/T115361#1722299 (kevinator) NEW
[17:12:10] <wikibugs>	 Analytics-Kanban, Traffic, operations: Flag in x-analytics in varnish any request that comes with no cookies whatsoever [5 pts] - https://phabricator.wikimedia.org/T114370#1722305 (Milimetric)
[17:12:29] <wikibugs>	 Analytics-Backlog: optimize Analytics Query Service {slug} - https://phabricator.wikimedia.org/T115361#1722299 (kevinator) p:High>Normal
[17:13:44] <wikibugs>	 Analytics-Kanban, Traffic, operations: Flag in x-analytics in varnish any request that comes with no cookies whatsoever {bear} [5 pts] - https://phabricator.wikimedia.org/T114370#1722320 (kevinator)
[17:17:50] <wikibugs>	 Analytics-Backlog, Analytics-Cluster: Update last access jobs to account for nocookie header - https://phabricator.wikimedia.org/T115350#1722342 (Nuria) Reserach whether nocookie header makes sense. Run numbers on a day of data, if sensical, run them on a month of data.
[17:18:20] <wikibugs>	 Analytics-Backlog, Analytics-Cluster: Update last access jobs to account for nocookie header {bear} [13 pts] - https://phabricator.wikimedia.org/T115350#1722347 (Milimetric)
[17:18:35] <wikibugs>	 Analytics-Backlog, Analytics-Cluster: Update last access jobs to account for nocookie header {bear} [13 pts] - https://phabricator.wikimedia.org/T115350#1722076 (Milimetric) p:Triage>High
[17:19:50] <wikibugs>	 Analytics-Backlog, Analytics-Cluster: Research whether no cookie header numbers improve Last access uniques {bear} [13 pts] - https://phabricator.wikimedia.org/T115350#1722356 (madhuvishy)
[17:21:00] <wikibugs>	 Analytics-Kanban, Analytics-Wikistats: Feed Wikistats traffic reports with aggregated hive data {lama} [8 pts] - https://phabricator.wikimedia.org/T114379#1722371 (Milimetric)
[17:25:33] <wikibugs>	 Analytics-Kanban, Analytics-Wikistats: Publish new pageview dataset on dumps.wikimedia.org with very clear documentation - https://phabricator.wikimedia.org/T115344#1722393 (Milimetric)
[17:31:36] <wikibugs>	 Analytics-Kanban, Analytics-Wikistats: Publish new pageview dataset with clear documentation {lama} [8 pts] - https://phabricator.wikimedia.org/T115344#1722422 (Milimetric)
[17:48:18] <travis-ci>	 wikimedia/mediawiki-extensions-EventLogging#493 (wmf/1.27.0-wmf.3 - d36fbe7 : Mukunda Modell): The build has errored.
[17:48:18] <travis-ci>	 Change view : https://github.com/wikimedia/mediawiki-extensions-EventLogging/commit/d36fbe780386
[17:48:19] <travis-ci>	 Build details : https://travis-ci.org/wikimedia/mediawiki-extensions-EventLogging/builds/85170231
[17:49:06] <nuria>	 milimetric: no more tasking?
[17:49:53] <milimetric>	 nope, we decided there was enough work for the week, nuria
[17:50:11] <nuria>	 k
[18:08:57] <icinga-wm>	 PROBLEM - Packetloss_Average on analytics1026 is CRITICAL: packet_loss_average CRITICAL: 8.86470346939
[18:11:37] <icinga-wm>	 RECOVERY - Packetloss_Average on analytics1026 is OK: packet_loss_average OKAY:0.0129243877551
[18:15:20] <wikibugs>	 Analytics-Kanban: Create white list for pageview data {hawk} [8 pts] - https://phabricator.wikimedia.org/T110061#1568100 (Nuria) Holder tables for authorized and non authorized values are here: https://gerrit.wikimedia.org/r/#/c/240099/5
[18:16:43] <nuria>	 joal: for the pageview_hourly whitelist...
[18:17:07] <nuria>	 joal: do i create a test file with "authorized" values and we load them into the table?
[18:30:55] <wikibugs>	 Analytics-Cluster, Database: Replicate Echo databases to analytics-store - https://phabricator.wikimedia.org/T115275#1722695 (Neil_P._Quinn_WMF) This is similar to {T75047}. Last word on that was in August—Ops said it should wait on some rearchitecting they're doing.
[18:45:17] <icinga-wm>	 PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL: CRITICAL: 26.67% of data above the critical threshold [30.0]
[18:48:48] <icinga-wm>	 RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK: OK: Less than 25.00% above the threshold [20.0]
[18:52:30] <wikibugs>	 Analytics-Cluster, Database: Replicate Echo tables to analytics-store - https://phabricator.wikimedia.org/T115275#1722721 (Neil_P._Quinn_WMF)
[19:21:16] <wikibugs>	 Analytics-Cluster, Analytics-Kanban, Easy: PM sees reports on browsers (Weekly or Daily) [8 pts] - https://phabricator.wikimedia.org/T88504#1722834 (mforns) a:mforns
[19:22:35] <joal>	 hey nuria
[19:24:37] <krrrit-wm>	 (CR) Ottomata: "Cool, one comment." (2 comments) [analytics/refinery] - https://gerrit.wikimedia.org/r/245921 (https://phabricator.wikimedia.org/T113255) (owner: Mforns)
[19:26:15] <joal>	 About pageviews, why not to create a test file.
[19:26:40] <joal>	 nuria --^
[19:26:58] <joal>	 nuria: I did it with a subset of fields, you can have it with the full set !
[19:39:02] <joal>	 Hey a-team, I off for tonight
[19:39:04] <wikibugs>	 Analytics-Engineering, Community-Tech: [AOI] Add page view statistics to page information pages (action=info) - https://phabricator.wikimedia.org/T110147#1722901 (Milimetric) We support dashiki too, which has for example rolling averages (type 7 in the box on the lower left: https://vital-signs.wmflabs.org...
[19:39:37] <milimetric>	 nite jo
[19:39:46] <joal>	 See you tomorrow folks
[19:54:39] <mforns>	 joal, good night!
[20:00:00] <madhuvishy>	 good night joal!
[20:10:57] <wikibugs>	 Analytics-Backlog, Discovery: Display automata and humans separately on zero results rate graph - https://phabricator.wikimedia.org/T112846#1723017 (Ironholds)
[20:11:27] <wikibugs>	 Analytics-Backlog, Discovery: Display automata and humans separately on zero results rate graph - https://phabricator.wikimedia.org/T112846#1647865 (Ironholds) Pulling this out of the sprint because it's not possible until the infrastructure exists.
[20:43:43] <krrrit-wm>	 (PS2) Mforns: Add percent loss to refinery-dump-status script [analytics/refinery] - https://gerrit.wikimedia.org/r/245921 (https://phabricator.wikimedia.org/T113255)
[20:44:10] <krrrit-wm>	 (CR) Nuria: Add pageview quality check to pageview_hourly (2 comments) [analytics/refinery] - https://gerrit.wikimedia.org/r/240099 (https://phabricator.wikimedia.org/T109739) (owner: Joal)
[20:46:21] <krrrit-wm>	 (CR) Mforns: Add percent loss to refinery-dump-status script (2 comments) [analytics/refinery] - https://gerrit.wikimedia.org/r/245921 (https://phabricator.wikimedia.org/T113255) (owner: Mforns)
[21:11:42] <krrrit-wm>	 (PS6) Nuria: Add pageview quality check to pageview_hourly [analytics/refinery] - https://gerrit.wikimedia.org/r/240099 (https://phabricator.wikimedia.org/T109739) (owner: Joal)
[21:23:22] <wikibugs>	 Analytics-Kanban, RESTBase, Services, Patch-For-Review: configure RESTBase pageview proxy to Analytics' cluster {slug} [3 pts] - https://phabricator.wikimedia.org/T114830#1723426 (Milimetric) I hesitate to disagree here, because it will ultimately cost me time.  And a few people are blocked on this...
[22:33:58] <krrrit-wm>	 (PS7) Nuria: Add pageview quality check to pageview_hourly [analytics/refinery] - https://gerrit.wikimedia.org/r/240099 (https://phabricator.wikimedia.org/T109739) (owner: Joal)
[22:44:20] <krrrit-wm>	 (PS1) Nuria: [WIP] Pageview Hourly quality check whitelist [analytics/refinery] - https://gerrit.wikimedia.org/r/246118 (https://phabricator.wikimedia.org/T110061)
[22:45:34] <krrrit-wm>	 (CR) Nuria: "@joal: let's talk about this change, did not added country info on purpose cause I am not sure as to the use case for it." [analytics/refinery] - https://gerrit.wikimedia.org/r/246118 (https://phabricator.wikimedia.org/T110061) (owner: Nuria)
[22:49:11] <wikibugs>	 Analytics-Cluster, Analytics-Kanban, Easy: PM sees reports on browsers (Weekly or Daily) [8 pts] - https://phabricator.wikimedia.org/T88504#1723784 (Nuria) Let's talk about this cause although item was filed initially for mobile team we probably want a desktop report too. Both should be calculated on p...
[22:51:07] <krrrit-wm>	 (CR) Madhuvishy: Add cassandra load job for pageview API (1 comment) [analytics/refinery] - https://gerrit.wikimedia.org/r/236224 (https://phabricator.wikimedia.org/T108174) (owner: Joal)
[22:58:48] <wikibugs>	 Analytics-Backlog, Analytics-Wikistats, DevRel-October-2015: Clean the code review queue of analytics/wikistats - https://phabricator.wikimedia.org/T113695#1723826 (Tgr) There is a proposed WikiDev16 session about improving code review, especially for volunteers: {T114419}. You are welcome to comment t...
[23:38:33] <wikibugs>	 Analytics-Backlog, Research-and-Data: Historical analysis of edit productivity for English Wikipedia - https://phabricator.wikimedia.org/T99172#1723972 (Halfak) I talked to the Altiscale folk about the issue.  They were perplexed and have accepted my Job IDs and a description of the issue.  They'll review...