[00:01:14] <wikibugs>	 Analytics: Restore MobileWebSectionUsage_14321266 and MobileWebSectionUsage_15038458 - https://phabricator.wikimedia.org/T123595#1950564 (Tbayer) ...but actually it seems that the newer of the two tables (`MobileWebSectionUsage_15038458`) has already been fully restored? That would be awesome news.   ``` mys...
[00:16:08] <nuria>	 boy 1002 is unusable ...
[00:21:43] <icinga-wm>	 PROBLEM - Overall insertion rate from MySQL consumer on graphite1001 is CRITICAL: CRITICAL: 20.00% of data under the critical threshold [10.0]
[00:28:04] <icinga-wm>	 PROBLEM - Overall insertion rate from MySQL consumer on graphite1001 is CRITICAL: CRITICAL: 20.00% of data under the critical threshold [10.0]
[00:28:21] <madhuvishy>	 nuria: what happened?
[00:29:00] <nuria>	 madhuvishy: i think there must be some processing that was running cause it was like realllll slowwww
[00:29:13] <madhuvishy>	 ah
[01:11:09] <Ironholds>	 My last night in San Francisco and I have 0 things to do. This'll be fun?
[01:20:05] * YuviPanda makes Ironholds pay rent
[01:20:10] <YuviPanda>	 funnest thing to do in SF
[01:20:16] <Ironholds>	 YuviPanda, yeah, no
[01:20:52] <YuviPanda>	 :P
[01:21:19] * YuviPanda would invite Ironholds places but is still dealing with the linux kernel vulnerability :|
[01:21:59] <Ironholds>	 linux kernel vuln?
[01:28:54] <wikibugs>	 Analytics-Tech-community-metrics, DevRel-January-2016: Make GrimoireLib display *one* consistent name for one user, plus the *current* affiliation of a user - https://phabricator.wikimedia.org/T118169#1950925 (Aklapper) @Lcanasdiaz: This looks definitely way better (I checked with the "legoktm" example in...
[01:31:33] <YuviPanda>	 Ironholds: yeah
[01:31:34] <YuviPanda>	 Ironholds: http://www.ubuntu.com/usn/usn-2870-1/
[01:31:42] <YuviPanda>	 Ironholds: allows people with shell access to obtain root
[01:31:50] <Ironholds>	 ..
[01:31:53] <Ironholds>	 all code is shit
[01:32:16] <YuviPanda>	 :D
[01:32:18] <YuviPanda>	 ultimately caused
[01:32:22] <YuviPanda>	 by a integer owerflow
[01:40:35] <ori>	 heh, wow
[01:40:43] <ori>	 totally missed that
[04:02:02] <wikibugs>	 Analytics, Fundraising-Backlog, Wikimedia-Fundraising: Public dashboards for CentralNotice and Fundraising - https://phabricator.wikimedia.org/T88744#1951155 (AndyRussG)
[04:51:34] <icinga-wm>	 PROBLEM - Overall insertion rate from MySQL consumer on graphite1001 is CRITICAL: CRITICAL: 20.00% of data under the critical threshold [10.0]
[05:42:34] <icinga-wm>	 PROBLEM - Overall insertion rate from MySQL consumer on graphite1001 is CRITICAL: CRITICAL: 20.00% of data under the critical threshold [10.0]
[07:47:56] <elukey>	 o/
[07:48:33] <elukey>	 I don't even know where to start to figure out where these icinga messages are coming from :(
[07:50:44] <wikibugs>	 Analytics-Kanban, Patch-For-Review: Burrow should be restarted automatically when config changes - https://phabricator.wikimedia.org/T123942#1951267 (elukey) Merged in Prod by Andrew, all good :)
[07:51:06] <wikibugs>	 Analytics-Kanban, Patch-For-Review: Burrow should be restarted automatically when config changes - https://phabricator.wikimedia.org/T123942#1951268 (elukey) Open>Resolved
[07:52:14] <elukey>	 ah just seen a phab ticket from ottomata, now I have things to read :)
[09:29:50] <wikibugs>	 Analytics, Analytics-Cluster, Traffic, operations: Upgrade analytics-eqiad Kafka cluster to Kafka 0.9 - https://phabricator.wikimedia.org/T121562#1951353 (elukey)
[09:29:52] <wikibugs>	 Analytics, Analytics-Cluster, Patch-For-Review: Single Kafka partition replica periodically lags - https://phabricator.wikimedia.org/T121407#1951352 (elukey)
[09:30:55] <wikibugs>	 Analytics, Analytics-Cluster, Patch-For-Review: Single Kafka partition replica periodically lags - https://phabricator.wikimedia.org/T121407#1877784 (elukey) Marking this ticket as blocked by Kafka 0.9 migration for the moment (since the bug has been fixed in that release) but I'll complete my investi...
[09:51:25] <elukey>	 mforns, joal: anybody with ~30 mins of time to review Event Logging's overall config with me?
[09:51:45] <joal>	 Hi elukey, let's do that :)
[09:52:01] <joal>	 batcave ?
[09:52:30] <elukey>	 sure!
[09:52:34] <elukey>	 joinin in a min
[11:34:01] <grrrit-wm>	 (CR) Joal: "Please wait before merging this code." [analytics/refinery/source] - https://gerrit.wikimedia.org/r/255105 (owner: DCausse)
[11:53:09] <mforns>	 hi a-team!
[11:53:23] <joal>	 Hi mforns :)(
[11:53:34] <mforns>	 hey joal!
[11:56:32] <elukey>	 hello mforns :)
[11:56:41] <mforns>	 hi elukey :]
[12:49:09] <wikibugs>	 Analytics-Cluster, EventBus, Services, operations: Investigate proper set up for using Kafka MirrorMaker with new main Kafka clusters. - https://phabricator.wikimedia.org/T123954#1951634 (Joe) I do think that we DEFINITELY want to rely events to active listeners in both datacenters. What we //don't...
[12:54:59] <joal>	 a-team, I'm away for some time, should be back for standup :)
[12:55:07] <mforns>	 ok joal see ya
[13:06:09] <icinga-wm>	 PROBLEM - Overall insertion rate from MySQL consumer on graphite1001 is CRITICAL: CRITICAL: 20.00% of data under the critical threshold [10.0]
[13:31:08] <wikibugs>	 Analytics-Kanban, Research-and-Data: Research Spike: Article Title normalization contains weird chars [8 pts] {hawk} - https://phabricator.wikimedia.org/T108867#1951665 (mforns) a:mforns
[13:37:33] <elukey>	 Am I right in saying that https://wikitech.wikimedia.org/wiki/Grafana.wikimedia.org/eventlogging-schema.json is the configuration for https://grafana.wikimedia.org/dashboard/db/eventlogging ?
[13:39:47] <elukey>	 no probably not
[13:39:51] <elukey>	 :D
[13:48:10] <elukey>	 ah added movingAvg to https://grafana.wikimedia.org/dashboard/db/eventlogging
[14:02:21] <wikibugs>	 Analytics-Kanban: Tune eventlogging.overall.inserted.rate alert to use a movingAverage transformation - https://phabricator.wikimedia.org/T124204#1951687 (elukey) For the moment I modified https://grafana.wikimedia.org/dashboard/db/eventlogging to have a separate panel for the Insertion rate, right above the...
[14:02:32] <wikibugs>	 Analytics-Kanban: Tune eventlogging.overall.inserted.rate alert to use a movingAverage transformation - https://phabricator.wikimedia.org/T124204#1951688 (elukey) p:Triage>Normal
[14:07:51] <icinga-wm>	 RECOVERY - Overall insertion rate from MySQL consumer on graphite1001 is OK: OK: Less than 20.00% under the threshold [100.0]
[14:57:27] <MarkTraceur>	 milimetric: Good morning, your friendly neighbourhood nag here...looks like there is neither a new crop of data, nor any new log entry
[14:57:37] <MarkTraceur>	 None of the reports seem to have updated
[14:58:16] <milimetric>	 K, I'll troubleshoot momentarily
[14:58:31] <MarkTraceur>	 Thanks!
[15:01:38] <MarkTraceur>	 I don't see any reports running presently, I think
[15:53:24] <elukey>	 ottomata: I changed https://grafana.wikimedia.org/dashboard/db/eventlogging to include the movingAvg creating a new panel for the insertion rate only
[15:53:46] <elukey>	 I wanted to have all the metrics.. I'll remove it once we'll proceed
[15:53:51] <ottomata>	 nice!
[15:54:21] <ottomata>	 that discussion about changing batch sizes was based on this
[15:54:27] <ottomata>	 but i think we should update the metric and alert anyway
[15:54:31] <ottomata>	 even if we change batch sizes
[15:54:43] <ottomata>	 any batching will make this metric spikey
[15:54:45] <ottomata>	 so smoothing is good
[15:55:15] <elukey>	 yep I agree.. just wanted to make sure that we alarm when we need
[15:58:22] <elukey>	 also this "batching" issue is related only to the events that gets pushed to MySQL.. what about camus and its SLA? Is there any chance to drop messages with it?
[15:59:37] <ottomata>	 !log stopping eventlogging mysql consumers for https://phabricator.wikimedia.org/T123546
[15:59:40] <analytics-logbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master
[16:01:44] <elukey>	 mmmm so each time that we stop eventlogging's consumers we risk to loose the data in the last batch right?
[16:01:47] <ottomata>	 elukey:  ja, just for eventlogging mysql consumer
[16:01:49] <ottomata>	 no
[16:01:58] <ottomata>	 not if they are stopped gracefully
[16:02:19] <ottomata>	 anything in the unsinserted batch will be inserted before stopping
[16:02:20] <elukey>	 ah ok I thought that it was a feature not implemented yet reading from the email thread
[16:02:30] <ottomata>	 ungraceful stop will lose messages
[16:02:36] <ottomata>	 camus is 100% cool
[16:02:42] <ottomata>	 no lost messages there
[16:02:48] <ottomata>	 unless its offsets get wonky
[16:02:55] <ottomata>	 (which did happen to us once, but that's because we broke kafka)
[16:03:10] <elukey>	 :D
[16:04:07] <elukey>	 other curiosity: do we have any data about data "lost" during a certain period of time for event logging's consumers issues?
[16:04:48] <elukey>	 I mean, do we have death and tears for consumers not logged/monitored?
[16:05:15] <elukey>	 I am learning and I have 10k questions, this is why I am hammering :)
[16:06:09] <ottomata>	 1. no
[16:06:11] <ottomata>	 2. no
[16:06:12] <ottomata>	 :)
[16:06:38] <ottomata>	 el analytics data is slowly gaining reliability
[16:06:47] <ottomata>	 we're on kafka as of 6 months ago, previously udp + zmq
[16:07:03] <ottomata>	 but, it isn't crucial data.
[16:07:05] <elukey>	 all right I'll explore monitors and alarms, thanks :)
[16:07:09] <ottomata>	 cool :)
[16:07:25] <ottomata>	 that is relevant to the burrow monitoring stuff we were talking about yesterday
[16:07:35] <ottomata>	 would be cool to have graphs of consumer lag
[16:08:44] <wikibugs>	 Analytics, MediaWiki-API, Reading-Infrastructure-Team, Research-and-Data, and 3 others: Publish detailed Action API request information to Hadoop - https://phabricator.wikimedia.org/T108618#1952263 (bd808)
[16:09:12] <wikibugs>	 Analytics-Kanban, Reading-Admin: Tabular layout on dashiki [8 pts] {lama} - https://phabricator.wikimedia.org/T118329#1952266 (Milimetric)
[16:10:34] <elukey>	 atm Burrow sends only emails right? So it should be a matter of pushing stuff to statsd => graphite and then build a dashboard with Grafana
[16:10:54] <wikibugs>	 Analytics, MediaWiki-API, Reading-Infrastructure-Team, Research-and-Data, and 3 others: Publish detailed Action API request information to Hadoop - https://phabricator.wikimedia.org/T108618#1952268 (bd808) I updated the data table in the description to match guidance given during Avro schema code r...
[16:11:01] <wikibugs>	 Analytics-Kanban: Dashiki visualization that shows a hierarchy [13 pts] {lama} - https://phabricator.wikimedia.org/T124296#1952269 (Milimetric) NEW a:Milimetric
[16:12:00] <wikibugs>	 Analytics-Kanban: Dashiki textual visualization [5 pts] {lama} - https://phabricator.wikimedia.org/T124297#1952277 (Milimetric) NEW a:Milimetric
[16:13:01] <wikibugs>	 Analytics-Kanban: Bookmark-able graphs in Dashiki tabular layout [8 pts] {lama} - https://phabricator.wikimedia.org/T124298#1952284 (Milimetric) NEW a:Milimetric
[16:14:40] <wikibugs>	 Analytics: Restore MobileWebSectionUsage_14321266 and MobileWebSectionUsage_15038458 - https://phabricator.wikimedia.org/T123595#1952295 (Cmjohnson)
[16:14:42] <wikibugs>	 Analytics, operations, ops-eqiad: Possible bad mem chip or slot on dbproxy1004 - https://phabricator.wikimedia.org/T123546#1952293 (Cmjohnson) Open>Resolved Replaced the bad DIMM at slot A3
[16:20:39] <ottomata>	 !log started eventlogging mysql consumers
[16:20:41] <analytics-logbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master
[16:22:29] <milimetric>	 MarkTraceur: I'm working on it but halfak's revscoring stuff is maxing out all 16 cores on that box :)
[16:22:35] <MarkTraceur>	 Ah shoot
[16:22:42] <milimetric>	 (it's ok, halfak, it hasn't been running for too long yet)
[16:22:42] <MarkTraceur>	 Curse you halfaaaaaak
[16:22:47] <milimetric>	 lol
[16:23:26] <milimetric>	 but MarkTraceur do you need these by any particular time?  I'll check on them until they get done and let you know what the problem was at the end, but just making sure you're not up against a QR deadline or something
[16:25:00] <MarkTraceur>	 I mean, not really a deadline, I just don't want to be spinning my wheels and lose momentum
[16:25:10] <halfak>	 milimetric, sorry about that.  Should be done shortly!
[16:25:38] <halfak>	 milimetric, I'm down to 8 cores.
[16:29:24] <grrrit-wm>	 (PS11) Nuria: [WIP] Daily and monthly uniques oozie jobs [analytics/refinery] - https://gerrit.wikimedia.org/r/216341 (https://phabricator.wikimedia.org/T92977) (owner: Madhuvishy)
[16:35:00] <ottomata>	 !log stopped eventlogging mysql consumers for long downtime: https://phabricator.wikimedia.org/T120187
[16:35:03] <analytics-logbot>	 Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master
[16:36:35] <wikibugs>	 Analytics-Kanban, DBA, Patch-For-Review: Pending maintenance on the eventlogging databases (db1046, db1047, dbstore1002, other dbstores) - https://phabricator.wikimedia.org/T120187#1952380 (Ottomata) EventLogging MySQL processes are stopped, downtime is scheduled, folks have been notified.  Proceed! :)
[16:36:58] <milimetric>	 MarkTraceur: hm... right now it's looking like the query is taking a really long time, lemme check on that
[16:37:27] <MarkTraceur>	 milimetric: I wouldn't be surprised, mysql crashed once while I ran the commonswiki first time uploaders
[16:38:38] <milimetric>	 MarkTraceur: that's this one? https://github.com/wikimedia/analytics-limn-multimedia-data/blob/master/multimedia/new-uploaders.sql
[16:38:43] <MarkTraceur>	 Yeah
[16:39:31] <milimetric>	 oh :) yeah... that'll not work :)
[16:41:34] <MarkTraceur>	 Er, why?
[16:41:37] <MarkTraceur>	 Did I mess it up?
[16:42:11] <milimetric>	 no, it's just going to query the whole table and group it by user, then join and discard a huge amount of data
[16:42:19] <milimetric>	 needle in the haystack kind of thing
[16:42:21] <MarkTraceur>	 Well, yes, that's the idea
[16:42:24] <milimetric>	 but there should be a better way
[16:42:31] <MarkTraceur>	 Should, but I don't think there is
[16:42:32] <milimetric>	 i'm trying some alternatives
[16:42:44] <MarkTraceur>	 I believe I talked with someone in here about this query
[16:43:20] <milimetric>	 yeah, I get why it's tricky
[16:46:41] <ottomata>	 a-team, unless someone thinks i need to be there, i'd like to skip backlog grooming today
[16:46:52] <mforns>	 np for me
[16:46:56] <milimetric>	 np
[16:47:23] <wikibugs>	 Analytics-Kanban, DBA, Patch-For-Review: Pending maintenance on the eventlogging databases (db1046, db1047, dbstore1002, other dbstores) - https://phabricator.wikimedia.org/T120187#1952430 (jcrespo) ``` my log < /srv/tmp/convert_innodb_to_tokudb.sql sleep(20) 0 sleep(20) 0 sleep(20) 0 sleep(20) 0 [deta...
[16:47:29] <joal>	 np ottomata
[16:50:57] <grrrit-wm>	 (PS12) Nuria: [WIP] Daily and monthly uniques oozie jobs [analytics/refinery] - https://gerrit.wikimedia.org/r/216341 (https://phabricator.wikimedia.org/T92977) (owner: Madhuvishy)
[16:55:13] <nuria>	 ottomata: do you know where "check_eventlogging_lag.sh" script is located? (not in 1002)
[16:57:55] <madhuvishy>	 nuria: probably 1003
[16:58:42] <nuria>	 madhuvishy: no, not either, just tried:
[16:59:23] <nuria>	 madhuvishy: or , not in the $PATH
[16:59:28] <nuria>	 madhuvishy: let me ask jynus
[17:00:34] <ottomata>	 nuria:  i don't, maybe t hat is just a jaime custom script
[17:00:43] <nuria>	 madhuvishy, ottomata standdduppp
[17:03:49] <wikibugs>	 Analytics-Kanban: Put quaterly review deck together [5 pts] - https://phabricator.wikimedia.org/T122334#1952457 (Nuria) Open>Resolved
[17:03:51] <wikibugs>	 Analytics-Kanban: Quaterly review 2016/01/22 (slides due on 19th) - https://phabricator.wikimedia.org/T120844#1952458 (Nuria)
[17:03:58] <wikibugs>	 Analytics-Kanban: Create Piwik cron to optimize dashboarding [3 pts] - https://phabricator.wikimedia.org/T124187#1952459 (Nuria) Open>Resolved
[17:04:11] <wikibugs>	 Analytics-Kanban: Implement a simple public API to calculate global metrics {kudu} [0 pts] - https://phabricator.wikimedia.org/T117285#1952461 (Nuria)
[17:04:13] <wikibugs>	 Analytics-Kanban, Patch-For-Review: Allow metrics to roll up results by user across projects {kudu} [5 pts] - https://phabricator.wikimedia.org/T117287#1952460 (Nuria) Open>Resolved
[17:04:26] <wikibugs>	 Analytics-Kanban: Implement a simple public API to calculate global metrics {kudu} [0 pts] - https://phabricator.wikimedia.org/T117285#1770422 (Nuria)
[17:04:28] <wikibugs>	 Analytics-Kanban: Create a set of celery tasks that can handle the global metric API input {kudu} [0 pts] - https://phabricator.wikimedia.org/T117288#1952462 (Nuria) Open>Resolved
[17:07:09] <nuria>	 madhuvishy: holaaa
[17:07:27] <nuria>	 madhuvishy: ah sorry, i see you are OOTO
[17:07:39] <nuria>	 don't check IRC!
[17:09:48] <madhuvishy>	 nuria: np I sent it too late
[17:12:07] <aude>	 can someone restart https://vital-signs.wmflabs.org/ ?
[17:12:15] <aude>	 idk who maintains it or where it lives
[17:12:41] <aude>	 ok, there is general labs maintenance going on now :/
[17:12:46] <aude>	 so we need to wait
[17:20:04] <wikibugs>	 Analytics-Kanban: Polish  script that checks eventlogging lag to use it for alarming  - https://phabricator.wikimedia.org/T124306#1952497 (Nuria) NEW
[17:20:51] <wikibugs>	 Analytics-Kanban: Polish  script that checks eventlogging lag to use it for alarming - https://phabricator.wikimedia.org/T124306#1952505 (Nuria) https://gerrit.wikimedia.org/r/#/c/265509/
[17:23:48] <wikibugs>	 Analytics-Kanban: Polish  script that checks eventlogging lag to use it for alarming - https://phabricator.wikimedia.org/T124306#1952509 (Nuria) Open>Resolved
[17:23:59] <milimetric>	 MarkTraceur: I got two faster versions, but I'm still waiting for my first faster version to finish (it's over 20 minutes now).  My fastest fastest version:
[17:24:04] <milimetric>	 https://www.irccloud.com/pastebin/6CY6jEbR/
[17:24:22] <MarkTraceur>	 Hmm
[17:24:31] <MarkTraceur>	 I guess that works, yeah
[17:24:42] <MarkTraceur>	 Will that be reportgeneratorable?
[17:28:55] <halfak>	 milimetric, MarkTraceur, I started up a new job on stat1003, but I nice'd it so you can have the CPU if you need it.
[17:29:09] <halfak>	 Please don't hesitate to run something.  I'm not in a rush to get this job done.
[17:29:15] <milimetric>	 no worries halfak, it's all good
[17:29:20] <MarkTraceur>	 Cool beans
[17:29:41] <milimetric>	 Mark it'll run with reportupdater but it'll take many days to catch up back to 2004 like you have it set
[17:29:43] <milimetric>	 not sure...
[17:30:08] <milimetric>	 ok! it finished - 32 minutes
[17:30:09] <milimetric>	 ouch
[17:30:09] <MarkTraceur>	 milimetric: I believe it only ever took a few hours running it as a simple mysql script
[17:30:29] <milimetric>	 that means 3 days to backfill that report... not horrible
[17:30:39] <milimetric>	 all the way back to 2004?
[17:30:43] <milimetric>	 interesting, maybe there's caching
[17:30:48] <MarkTraceur>	 Yeah, I ran it on the whole thing
[17:30:54] <MarkTraceur>	 I don't remember it taking days ever
[17:31:08] <milimetric>	 oh! yeah, that makes sense, the problem here is that we'll be running it separately for each month
[17:31:16] <milimetric>	 which is nice for maintainability but not great for performance
[17:31:18] <MarkTraceur>	 Oh, right, that's...yeah
[17:31:52] <milimetric>	 hang in there, I'm in meetings but I'll run the other one
[17:32:06] <MarkTraceur>	 No problemo
[17:32:09] <wikibugs>	 Analytics, DBA, operations: Improve eventlogging replication procedure - https://phabricator.wikimedia.org/T124307#1952524 (jcrespo) NEW
[17:33:34] <wikibugs>	 Analytics, DBA, operations: Improve eventlogging replication procedure - https://phabricator.wikimedia.org/T124307#1952534 (jcrespo)
[17:33:36] <wikibugs>	 Analytics-Kanban: Polish  script that checks eventlogging lag to use it for alarming - https://phabricator.wikimedia.org/T124306#1952533 (jcrespo)
[17:35:12] <ottomata>	 iiinteresting
[17:35:15] <ottomata>	 "ip":"24.191.206.189","http_status":"301","uri_path":"/wiki/barack_obama",
[17:35:15] <ottomata>	 "ip":"24.191.206.189","http_status":"304","uri_path":"/wiki/Barack_obama",
[17:35:22] <ottomata>	 (that's my IP, don't worry!)
[17:37:08] <ottomata>	 huh, why 304 on a redirect, weeiirrd, i guess it caches the fact that the previous response was a redirect??
[17:37:09] <ottomata>	 idunnonoo
[17:49:23] <wikibugs>	 Analytics-Kanban, DBA, Patch-For-Review: Pending maintenance on the eventlogging databases (db1046, db1047, dbstore1002, other dbstores) - https://phabricator.wikimedia.org/T120187#1952585 (jcrespo) If my previous cryptic message is not understood, it means that maintenance has started successfully, I...
[17:58:12] <nuria>	 304 is conditional get meaning that doc comes from cache
[17:59:07] <grrrit-wm>	 (PS1) Milimetric: Improve speed of new uploaders a bit [analytics/limn-multimedia-data] - https://gerrit.wikimedia.org/r/265516
[17:59:37] <milimetric>	 MarkTraceur: ok, so take a look at that ^ I think it's the fastest we're gonna get without a temp table
[18:00:06] <milimetric>	 which is sad, 'cause expensive inner queries like that really should be materialized and just re-used
[18:00:22] <milimetric>	 but if you're ok, I'll merge this and I think you should be able to backfill in a couple of days
[18:01:04] <grrrit-wm>	 (CR) MarkTraceur: [C: 1] Improve speed of new uploaders a bit [analytics/limn-multimedia-data] - https://gerrit.wikimedia.org/r/265516 (owner: Milimetric)
[18:01:09] <MarkTraceur>	 Sweet
[18:01:18] <MarkTraceur>	 A couple of days is better than weeks
[18:01:24] <grrrit-wm>	 (CR) Milimetric: [C: 2 V: 2] Improve speed of new uploaders a bit [analytics/limn-multimedia-data] - https://gerrit.wikimedia.org/r/265516 (owner: Milimetric)
[18:01:37] <MarkTraceur>	 milimetric: Do we need to kill the old query somehow?
[18:02:16] <milimetric>	 i'll make sure it's not running
[18:03:17] <mforns>	 joal, we are in the batcave
[18:04:50] <wikibugs>	 Analytics-Kanban: Foundation-only Geowiki stopped updating [3 pts] - https://phabricator.wikimedia.org/T106229#1952617 (Milimetric)
[18:05:27] <wikibugs>	 Analytics-Kanban, Patch-For-Review: Remove queue of batches from EventLogging [8 pts] {oryx} - https://phabricator.wikimedia.org/T121151#1952618 (mforns)
[18:10:44] <wikibugs>	 Analytics-Kanban: Projections of cost and scaling for pageview API. {hawk} [8 pts] - https://phabricator.wikimedia.org/T116097#1952629 (Milimetric) a:JAllemandou
[18:10:50] <wikibugs>	 Analytics-Kanban: Polish  script that checks eventlogging lag to use it for alarming - https://phabricator.wikimedia.org/T124306#1952630 (Nuria) Resolved>Open
[18:14:34] <wikibugs>	 Analytics-Kanban: Polish  script that checks eventlogging lag to use it for alarming - https://phabricator.wikimedia.org/T124306#1952667 (Milimetric) a:elukey
[18:16:31] <wikibugs>	 Analytics-Kanban, RESTBase: Update AQS config to new config format - https://phabricator.wikimedia.org/T122249#1952675 (Milimetric) a:Pchelolo>Milimetric
[18:16:41] <wikibugs>	 Analytics: Create fake data for beta AQS deployment - https://phabricator.wikimedia.org/T120841#1952679 (Milimetric)
[18:17:01] <wikibugs>	 Analytics: Create fake data for beta AQS deployment - https://phabricator.wikimedia.org/T120841#1952685 (Milimetric) a:Milimetric>None
[18:19:10] <wikibugs>	 Analytics-Kanban, Services: Improve pageview API response time with cache headers [8 pts] - https://phabricator.wikimedia.org/T119886#1952686 (Milimetric)
[18:19:59] <wikibugs>	 Analytics-Tech-community-metrics, DevRel-January-2016: Many profiles on profile.html do not display identity's name though data is available - https://phabricator.wikimedia.org/T117871#1952691 (Aklapper)
[18:20:02] <wikibugs>	 Analytics-Tech-community-metrics, Developer-Relations, DevRel-February-2016: Who are the top 50 independent contributors and what do they need from the WMF? - https://phabricator.wikimedia.org/T85600#1952690 (Aklapper)
[18:20:08] <wikibugs>	 Analytics: Better response times Pageview API  - https://phabricator.wikimedia.org/T124314#1952692 (Nuria) NEW
[18:21:40] <wikibugs>	 Analytics: Cassandra Backfill July [5 pts] {melc} - https://phabricator.wikimedia.org/T119863#1952710 (JAllemandou)
[18:22:06] <wikibugs>	 Analytics-Kanban: Projections of cost and scaling for pageview API. {hawk} [8 pts] - https://phabricator.wikimedia.org/T116097#1952716 (JAllemandou)
[18:22:08] <wikibugs>	 Analytics: Cassandra Backfill July [5 pts] {melc} - https://phabricator.wikimedia.org/T119863#1837784 (JAllemandou)
[18:23:29] <wikibugs>	 Analytics: RUBICON - https://phabricator.wikimedia.org/T105515#1952718 (Milimetric) Open>Resolved a:Milimetric
[18:25:23] <wikibugs>	 Analytics-Kanban: Productionize last access jobs for daily and monthly calculations {bear} - https://phabricator.wikimedia.org/T122514#1952722 (Nuria)
[18:25:44] <wikibugs>	 Analytics, DBA, operations: Improve eventlogging replication procedure - https://phabricator.wikimedia.org/T124307#1952723 (Milimetric) p:Triage>Normal
[18:35:43] <milimetric>	 mforns: I'm confused
[18:35:52] <milimetric>	 I'm running sudo -u stats python /srv/limn-mobile-data/generate.py /srv/limn-multimedia-data/multimedia/ >> /var/log/limn-data/limn-multimedia-data.log 2>&1 and it's just quitting right away with no messages
[18:36:01] <milimetric>	 wanna chat real quick or you busy?
[18:36:13] <mforns>	 milimetric, lets chat, but can I have 2 mins?
[18:36:19] <milimetric>	 'course
[18:36:21] <milimetric>	 ping me whenever
[18:38:34] <mforns>	 milimetric, batcave?
[18:39:16] <milimetric>	 to the batcave!
[18:42:23] <wikibugs>	 Analytics, Continuous-Integration-Infrastructure: Add json linting test for schemas in mediawiki/event-schemas - https://phabricator.wikimedia.org/T124319#1952779 (bd808) NEW
[18:43:36] <wikibugs>	 Analytics, Reading-Web, Wikipedia-iOS-App-Product-Backlog: As an end-user I shouldn't see non-articles in the list of trending articles - https://phabricator.wikimedia.org/T124082#1952795 (BGerstle-WMF) > we need to access page table data from mediawiki databases in real-time. And right now the data...
[18:49:07] <mforns>	 milimetric, are you ok with moving the Wikimedia bot meeting to next week? Joseph, Luca and Nuria have other engagements.
[18:49:20] <milimetric>	 mforns: sure
[18:49:30] <mforns>	 ok, will do
[18:49:36] <joal>	 thx mforns !
[18:49:58] <mforns>	 np, thx for the heads up, I forgot I was in europe :P
[18:56:59] <joal>	 :P
[18:59:20] <joal>	 elukey: hey
[18:59:22] <ottomata>	 elukey:  did we invite you to this meeting?
[18:59:23] <ottomata>	 you shoudl come!
[18:59:32] <ottomata>	 https://plus.google.com/hangouts/_/wikimedia.org/a-batcave
[19:01:21] <grrrit-wm>	 (PS1) Ottomata: Revert "Remove mobile webrequest_source merging it in text" [analytics/refinery] - https://gerrit.wikimedia.org/r/265538
[19:02:10] <ottomata>	 joal:  cool!
[19:02:11] <ottomata>	 ^
[19:02:17] <joal>	 Yeah !
[20:04:33] <milimetric>	 MarkTraceur: ok, so it's running now, it generated /a/limn-public-data/metrics/multimedia-health/uploaders/commonswiki.tsv
[20:04:38] <milimetric>	 i think it's probably on the new-uploaders
[20:04:51] <milimetric>	 and that'll take a *while* :)
[20:04:53] <milimetric>	 but it's chugging
[20:06:13] <wikibugs>	 Analytics-Wikistats: LIMN input file wikilytics_in_pageviews.csv no longer updated - https://phabricator.wikimedia.org/T124340#1953146 (ezachte) NEW a:ezachte
[20:06:30] <milimetric>	 I guess ideally if you already have the other results we could format them and start from what's already there
[20:15:59] <ottomata>	 heading home joal,
[20:16:13] <ottomata>	 i guess I should make a commit to revert the refinery-source change, eh?
[20:16:15] <ottomata>	 I'll do that
[20:16:18] <ottomata>	 see yaaa