[05:26:17] (CR) Madhuvishy: Update CirrusSearchRequestSet avro schema (1 comment) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/246990 (https://phabricator.wikimedia.org/T115715) (owner: EBernhardson) [05:27:15] Analytics-Kanban, Discovery, Discovery-Cirrus-Sprint, Patch-For-Review, WorkType-NewFunctionality: Update CirrusSearchRequestSet schema to have a timestamp field - https://phabricator.wikimedia.org/T115715#1733799 (madhuvishy) [05:32:17] PROBLEM - Analytics Cassanda CQL query interface on aqs1001 is CRITICAL: Connection refused [05:33:57] RECOVERY - Analytics Cassanda CQL query interface on aqs1001 is OK: TCP OK - 0.002 second response time on port 9042 [09:12:43] Analytics, ContentTranslation-Analytics, MediaWiki-extensions-ContentTranslation: find how much are the articles that are created using ContentTranslation read - https://phabricator.wikimedia.org/T105194#1438288 (Amire80) [09:19:37] PROBLEM - Analytics Cassanda CQL query interface on aqs1002 is CRITICAL: Connection timed out [09:21:08] RECOVERY - Analytics Cassanda CQL query interface on aqs1002 is OK: TCP OK - 0.011 second response time on port 9042 [09:26:27] PROBLEM - Analytics Cassanda CQL query interface on aqs1002 is CRITICAL: Connection timed out [09:44:58] RECOVERY - Analytics Cassanda CQL query interface on aqs1002 is OK: TCP OK - 0.014 second response time on port 9042 [09:53:38] Analytics-Tech-community-metrics, Possible-Tech-Projects, Outreachy-Round-11: Improving MediaWikiAnalysis - https://phabricator.wikimedia.org/T89135#1734305 (01tonythomas) I am shifting this to #Outreachy-Round-11 as the project description has two mentors, micro-tasks and looks ready for the 11th edi... [09:55:01] Analytics-Tech-community-metrics, Possible-Tech-Projects, Epic, Outreachy-Round-11: Allow contributors to update their own details in tech metrics directly - https://phabricator.wikimedia.org/T60585#1734314 (01tonythomas) I am shifting this to #Outreachy-Round-11 as the project description has atle... [10:16:01] hi, I'm testing the new mediawiki monolog -> kafka, are topics name defined somewhere (on consumers side)? I've seen mediawiki_CirrusSearchRequestSet but also monolog_CirrusSearchRequestSet so I would like to check if the name is consistent in all files that define this topic name. [10:29:28] (PS2) DCausse: Update CirrusSearchRequestSet avro schema [analytics/refinery/source] - https://gerrit.wikimedia.org/r/246990 (https://phabricator.wikimedia.org/T115715) (owner: EBernhardson) [10:30:01] (CR) DCausse: Update CirrusSearchRequestSet avro schema (1 comment) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/246990 (https://phabricator.wikimedia.org/T115715) (owner: EBernhardson) [10:30:13] (CR) DCausse: [C: 1] Update CirrusSearchRequestSet avro schema [analytics/refinery/source] - https://gerrit.wikimedia.org/r/246990 (https://phabricator.wikimedia.org/T115715) (owner: EBernhardson) [10:54:08] PROBLEM - Analytics Cassanda CQL query interface on aqs1003 is CRITICAL: Connection refused [10:55:57] RECOVERY - Analytics Cassanda CQL query interface on aqs1003 is OK: TCP OK - 0.002 second response time on port 9042 [11:00:45] Analytics, MediaWiki-extensions-ContentTranslation: Limn language dashboard: eswiki graph is wrong/stuck - https://phabricator.wikimedia.org/T99074#1734448 (Amire80) Open>declined a:Amire80 Not really used anymore. [12:23:37] Analytics-Tech-community-metrics, DevRel-October-2015: Correct affiliation for code review contributors of the past 30 days - https://phabricator.wikimedia.org/T112527#1734720 (Aklapper) Quick status update: I'm waiting for @Lcanasdiaz to provide me access. Things potentially going slow because Bitergia... [12:26:15] Analytics-Tech-community-metrics, Possible-Tech-Projects, Epic, Outreachy-Round-11: Allow contributors to update their own details in tech metrics directly - https://phabricator.wikimedia.org/T60585#1734723 (Luiscanasdiaz) Hi @aklapper, we've just included some changes in our affiliation process.... [12:28:01] Analytics-Tech-community-metrics, DevRel-October-2015: Correct affiliation for code review contributors of the past 30 days - https://phabricator.wikimedia.org/T112527#1734727 (Luiscanasdiaz) Oupss, I made a mistake with the ticket. See my comment at https://phabricator.wikimedia.org/T60585#1734723 To su... [12:35:01] Analytics-Tech-community-metrics, DevRel-October-2015: Correct affiliation for code review contributors of the past 30 days - https://phabricator.wikimedia.org/T112527#1734730 (Aklapper) >>! In T112527#1734727, @Luiscanasdiaz wrote: > To sum up, we're ready, \o/ > I need your github user. @Luiscanasdi... [12:45:05] Analytics-Tech-community-metrics, DevRel-October-2015: Correct affiliation for code review contributors of the past 30 days - https://phabricator.wikimedia.org/T112527#1734733 (Luiscanasdiaz) >> I need your github user. > > @Luiscanasdiaz: https://github.com/aklapper access granted ;) [12:50:21] Analytics-Tech-community-metrics, DevRel-October-2015: Patches with Verified -1 should not be counted as open in our code review metrics - https://phabricator.wikimedia.org/T108507#1734737 (Dicortazar) According to comments by @jgbarah and @Acs, this ticket is not a bug. Closing this. Please, let us know... [12:50:31] Analytics-Tech-community-metrics, DevRel-October-2015: Patches with Verified -1 should not be counted as open in our code review metrics - https://phabricator.wikimedia.org/T108507#1734739 (Dicortazar) Open>Resolved [12:51:48] Analytics-Tech-community-metrics, DevRel-October-2015: Automated generation of (Git) repositories for Korma - https://phabricator.wikimedia.org/T110678#1734742 (Dicortazar) @Aklapper, how can we automatically retrieve the list of Git repositories available from some place in the mediawiki infrastructure? [13:54:33] Analytics-Tech-community-metrics, DevRel-October-2015: Checking code review metrics for a specific repository may take dozens of clicks - https://phabricator.wikimedia.org/T110520#1734794 (Dicortazar) The three charts are now found in each of the repositories. Closing the task!. [13:54:42] Analytics-Tech-community-metrics, DevRel-October-2015: Checking code review metrics for a specific repository may take dozens of clicks - https://phabricator.wikimedia.org/T110520#1734795 (Dicortazar) Open>Resolved [13:57:47] Analytics-Tech-community-metrics, DevRel-October-2015: "Oldest open Gerrit changesets without code review" panel should filter WIP etc - https://phabricator.wikimedia.org/T112661#1734805 (Dicortazar) As mentioned by @Qgil, there's an error with the WIP's. Those were not filtered. @Aklapper, the filtering... [13:59:47] Analytics-Tech-community-metrics, DevRel-October-2015: "Age of open changesets by Affiliation" has some "NaN" values - https://phabricator.wikimedia.org/T110875#1734813 (Dicortazar) I confirm this issue. In the case of Google, this company is not used anymore. And this still happens in the case of WikiWo... [14:11:49] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [30.0] [14:13:38] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK: OK: Less than 25.00% above the threshold [20.0] [14:20:19] PROBLEM - Analytics Cassanda CQL query interface on aqs1002 is CRITICAL: Connection refused [14:20:55] have yall noticed that those alerts have been happening at about the same time each day? ^ [14:20:56] ?!? [14:21:01] i'm looking at the EL one now [14:21:06] and I don't see anything yet... [14:25:37] RECOVERY - Analytics Cassanda CQL query interface on aqs1002 is OK: TCP OK - 0.001 second response time on port 9042 [14:25:53] I didn't notice it was the same time... weird [14:27:57] i mean, maybe just a coincidence [14:29:01] hm, i went back a week or so and they seem to happen at all different times [14:29:17] but always between 10:00 am and 17:00 [14:30:08] ok, el who knows, am looking, but why aqs? do you know? [14:30:19] if its not prod ready yet, we should disable the alerts [14:30:25] getting lots of pages [14:30:27] ottomata: aqs is due to loading [14:30:29] i was assuming those were because joal's loading the data and cassandra freaks out [14:30:31] right [14:30:34] oh [14:30:35] huh. [14:30:36] ok [14:30:52] I am into reducing that load now [14:30:54] i forget, can you guys access icinga? [14:30:59] yes [14:31:03] hm, maybe ? [14:31:08] can you schedule downtime for services? [14:31:14] https://icinga.wikimedia.org/icinga/ [14:31:14] that way they won't ping while you are loading [14:35:20] dunno if you guys get text messages for that, but I do! [14:35:39] i got around 50+ texts this weekend :p [14:38:36] (CR) Ottomata: Update CirrusSearchRequestSet avro schema (1 comment) [analytics/refinery/source] - https://gerrit.wikimedia.org/r/246990 (https://phabricator.wikimedia.org/T115715) (owner: EBernhardson) [14:43:16] (PS3) EBernhardson: Update CirrusSearchRequestSet avro schema [analytics/refinery/source] - https://gerrit.wikimedia.org/r/246990 (https://phabricator.wikimedia.org/T115715) [14:45:42] Analytics-Tech-community-metrics, Patch-For-Review: Tech metrics missing IRC channels - https://phabricator.wikimedia.org/T56230#1734947 (Dicortazar) I've commented on the GitHub repo. As a summary: I'll close that pull request since the place to add/remove repositories takes place at https://github.com/... [14:50:42] Analytics-Tech-community-metrics, DevRel-October-2015: "Oldest open Gerrit changesets without code review" panel should filter WIP etc - https://phabricator.wikimedia.org/T112661#1734961 (Dicortazar) Data are updated and smoothly running. My doubt here is why WIPs were there. Some kind of previous decisi... [14:50:47] Analytics-Tech-community-metrics, DevRel-October-2015: "Oldest open Gerrit changesets without code review" panel should filter WIP etc - https://phabricator.wikimedia.org/T112661#1734962 (Dicortazar) Open>Resolved [14:58:25] Analytics-Tech-community-metrics, DevRel-October-2015: Patches with Verified -1 should not be counted as open in our code review metrics - https://phabricator.wikimedia.org/T108507#1734973 (Aklapper) Ah, thanks! Looks to me like you're right from a quick test: I tried with "Gadgets" on http://korma.wmfla... [15:04:13] (CR) Ottomata: "Are we sure we want to call this format 'webstatscollector format' format? For many of the 'udp2log format' TSV files that we create, we " [analytics/refinery] - https://gerrit.wikimedia.org/r/246149 (https://phabricator.wikimedia.org/T114379) (owner: Milimetric) [15:06:40] ohhhh ottomata :( [15:06:45] oh? [15:06:55] I have no text messages, didn't realized you have :( [15:07:01] haha, s'ok! [15:07:12] it just makes me ignore icinga texts, which could be bad if i ignore the wrong ones :) [15:07:32] It's a good point for me when icinga freaks (too much load), but if it sends texts, I'll schedule downtime for sure ! [15:07:47] Doing that new :) [15:08:06] yeah, cool thanks. it makes ops folks look too, and you really only want them to look if something is really wrong [15:08:17] indeed [15:08:21] Didn't realized [15:08:33] I'll post a note (log) onto ops channel [15:11:32] Analytics-Tech-community-metrics: NaN values for certain "Time from last patchset" values - https://phabricator.wikimedia.org/T115871#1734998 (Aklapper) NEW [15:12:08] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL: CRITICAL: 26.67% of data above the critical threshold [30.0] [15:13:00] joal: just fyi, i don't think ops folks are SMS paged by those alerts, but they will see them in the ops channel [15:13:18] yup, godog told me [15:13:19] i get texts because i am configured to get paged, and also am in the analytics_admins group [15:13:20] like you [15:13:34] k ottomata [15:13:58] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK: OK: Less than 25.00% above the threshold [20.0] [15:14:11] Analytics-Tech-community-metrics: Time axis on repository.html only displays two months, repeated several items - https://phabricator.wikimedia.org/T115872#1735005 (Aklapper) NEW [15:17:24] Analytics-Tech-community-metrics, Patch-For-Review: Tech metrics missing IRC channels - https://phabricator.wikimedia.org/T56230#1735025 (Dicortazar) And regarding to the rest of IRC channels. Do we have logs for all of them? The following ones were the ones added in the GitHub change: * "mediawiki": TR... [15:17:33] ottomata: do you know if we have scala libs somewhere on either stat1002 or an1027 ? [15:18:10] yes i thikn so... [15:18:26] hm, couldn't find them, will look more closely [15:18:31] looking too [15:18:44] /usr/share/java/scala-library-2.9.2.jar [15:18:45] ? [15:18:47] etc. [15:19:18] mwarf, looked in /usr/lib, forgot /usr/share [15:19:21] * joal is ashamed [15:19:31] Analytics-Tech-community-metrics: NaN values for certain "Time from last patchset" values - https://phabricator.wikimedia.org/T115871#1735028 (Aklapper) [15:19:35] thx ottomata [15:21:38] Analytics-Tech-community-metrics, Research consulting, Research-and-Data: Quantifying the "sum of all contributors" - https://phabricator.wikimedia.org/T113406#1735030 (ezachte) Some notes on what seems an unanswerable question. CODE I expect nearly all people who contributed code to our projects, di... [15:22:21] (CR) Milimetric: "I wanted to be a little clearer what they are. Webstatscollector format is not technically correct, it should be "Domasz Mituzas format"." [analytics/refinery] - https://gerrit.wikimedia.org/r/246149 (https://phabricator.wikimedia.org/T114379) (owner: Milimetric) [15:23:41] Analytics, Discovery, EventBus, MediaWiki-General-or-Unknown, and 7 others: EventBus MVP - https://phabricator.wikimedia.org/T114443#1735035 (GWicke) A PR adding remote schema support to the nodejs frontend is now available at https://github.com/wikimedia/restevent/pull/1. This means that we can n... [15:26:22] ottomata: can we install scala2.10.4 (version we use with spark) on an1027 ? It's for the camus partition checker [15:26:33] Analytics-Tech-community-metrics, DevRel-October-2015: Checking code review metrics for a specific repository may take dozens of clicks - https://phabricator.wikimedia.org/T110520#1735041 (Aklapper) >>! In T110520#1734794, @Dicortazar wrote: > The three charts are now found in each of the repositories. T... [15:26:43] ottomata: I can use the spark jar (containing scala) if you prefer, but it's a bit hacky [15:26:56] hm, can you include those deps in the jar you are buildilng? [15:27:20] ottomata: trying to prevent building fat jar ... It's the job one [15:27:20] the ones in /usr/share/java are installed via deb packages [15:27:26] as far as I can tell we don't have a newere version than that [15:27:57] the one on stat1002 is 2.9.2 [15:29:47] aye [15:29:59] joal: unless we built or backported our own deb or something [15:30:03] no. or. [15:30:12] we could deploy the scala .jars as artifacts with refinery, if youlike [15:30:31] hm [15:30:47] Would be feasible, have a lib folder in refinery [15:30:55] But not really cool [15:31:11] aye, i wouldn't event do a lib folder, just add them in mvn repo format like we do for other jars [15:31:14] deb packages for scala newer version exists, but are not in apt from what I can see [15:31:25] right ottomata [15:31:32] Analytics-Tech-community-metrics, DevRel-October-2015: "Age of open changesets by Affiliation" has some "NaN" values - https://phabricator.wikimedia.org/T110875#1735068 (Aklapper) FYI, another NaN issue is in T115871 [15:31:47] oh? [15:31:49] joal, where? [15:31:50] Will use spark jar for testing, we'll discuss that when deploy time arise [15:32:22] oh this? [15:32:22] http://www.scala-lang.org/download/2.10.2.html [15:32:23] hm. [15:32:29] yes that migth be easy to do then [15:33:05] downlaod and untar in /usr/share/java ? [15:33:16] But I guess we wnat that to be pupettized [15:33:18] naw, we'd add to our apt [15:33:21] we can do that [15:33:28] k makes more sense :) [15:33:40] Good task for Luca when he comes in I guess :) [15:34:00] oh, is it going to be that long before you want to deploy ? [15:34:16] no ottomata, but I can cheat using spark jar for the moment [15:34:23] As you wish :) [15:35:51] ha, i think, make a task, and we'll make it happen [15:36:00] already ahving a .deb makes it way easier. [15:38:56] Assinged to you ottomata ? [15:39:54] joal, do you have 5 minutes for oozie questions before standup? [15:40:00] sure mforns [15:40:02] cave [15:40:04] ok! [15:45:52] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL: CRITICAL: 40.00% of data above the critical threshold [30.0] [15:47:06] joal, problems? [15:47:42] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK: OK: Less than 25.00% above the threshold [20.0] [15:51:02] mforns: sorry, got disconnected [15:51:06] joal, np [16:04:04] madhuvishy: ut? [16:04:14] Yeah [16:04:18] standup [16:07:46] Analytics-Kanban: special character stripping on cassandra loading (tabs) {slug} [5 pts] - https://phabricator.wikimedia.org/T115356#1735129 (JAllemandou) a:JAllemandou [16:07:59] Analytics-Kanban: improve timeuuid writing {slug} [5 pts] - https://phabricator.wikimedia.org/T115353#1735130 (JAllemandou) a:JAllemandou [16:16:27] ottomata, Trey314159's hive access doesn't let him run long queries because of a lack of write permissions ;p [16:16:48] is there any documentation for non-you Opsen of "here's how to actually handle Hive access requests"? Because this is the 3rd or 4th time I've seen this happen. [16:16:56] Ironholds: in standup, but would like to know more :) [16:17:06] kk! [16:17:34] Thanks, Ironholds! [16:30:48] Ironholds: what is unclear about this? [16:30:49] https://wikitech.wikimedia.org/wiki/Analytics/Data_access#Access_Groups [16:31:08] (I will add 'hadoop/hive' into the names) [16:31:13] *descriptions [16:31:51] ottomata, nothing, but either people don't know it's there or the docs are ambiguous [16:31:57] (I think adding hive/hadoop will help) [16:32:18] hanged to [16:32:19] analytics-privatedata-users [16:32:19] Access to stat1002 to connect to the https://wikitech.wikimedia.org/wiki/Analytics/Cluster (Hadoop/Hive) and to query private data hosted there, including webrequest logs. If you want Hadoop access, you probably want this. [16:35:08] Analytics-Tech-community-metrics, Patch-For-Review: Tech metrics missing IRC channels - https://phabricator.wikimedia.org/T56230#1735259 (Aklapper) a:Aklapper>None >>! In T56230#1734947, @Dicortazar wrote: > I'll close that pull request since the place to add/remove repositories takes place at https... [16:36:06] ottomata, 'probably'? :p [16:37:24] lol kevinator [16:40:40] Ironholds: you definitely want that group :) [16:40:45] ottomata: thanks for the pointer to the documentation. ironholds: thanks for the pointer to ottomata (and for jumping ahead and following it up for me, too!) [16:41:32] a-team, ops duty meeting? I'm ready [16:42:00] madhuvishy, dan scheduled it in 19 minutes [16:42:08] mforns: aaah [16:42:09] but if everyone is ok, I'm ready too [16:42:10] din't see that [16:42:11] thanks [16:42:14] ok [16:42:17] see you then :] [16:42:21] cya [16:42:29] oh, i just dragged it up one slot [16:42:52] but we can do it then, I want to wrap up these tests maybe [16:43:13] milimetric, it appears to me starting in 17 mins [16:43:22] no? [16:43:51] i'm in standup [16:43:54] i mean [16:43:56] batcave [16:45:47] joal: we're all there, but we're just cooking [16:45:54] :D [16:45:59] ok, joining ! [17:01:48] Analytics-Tech-community-metrics, DevRel-October-2015: "Oldest open Gerrit changesets without code review" panel should filter WIP etc - https://phabricator.wikimedia.org/T112661#1735368 (Aklapper) Thank you! For the records, that change was: https://github.com/VizGrimoire/GrimoireLib/commit/c917248796b... [17:41:40] poked through the docs but not finding anything, does hive have a way to print 1 column per line,something like mysql cli \G? eg in mysql to get an idea ofa table you might `select * from foo\G` as in https://phabricator.wikimedia.org/P2209 Not a big deal, just curious [17:42:07] when something has lots of columns the 1 line per row is just very hard to parse in my head :) [17:47:17] ebernhardson: I don't think so [17:47:37] the hive cli we have is deprecated though, and they moved to this thing called beeline [17:47:42] which I think has it [17:48:07] ottomata: and I talked a couple times to make it easier to use beeline, we have a task in our backlog for that [17:51:06] ahh, well good to know :) [17:51:20] a-team: https://wikitech.wikimedia.org/wiki/Analytics/Oncall [17:52:58] thx milimetric [17:53:59] ebernhardson: i agree with you, those super long rows are nonsensical. I usually just do describe table and select only the columns I'm interested in seeing. If there are columns that aren't obvious, let us know, we can add more to their descriptions maybe [18:11:37] ebernhardson: [18:11:40] here you go [18:11:43] https://www.irccloud.com/pastebin/N5NuIxT0/ [18:12:12] ottomata: around? [18:13:42] madhuvishy: ja [18:13:52] i figured out how to use beeline ^^ [18:14:14] COOL! [18:14:36] madhuvishy: is there a way we can make that jdbc connection url default [18:14:41] in some beeline config file or something? [18:14:43] ottomata: :) i'll look into it [18:15:00] not gotten past first few lines in docs [18:15:39] ottomata: i wanted to talk to you about burrow also, we wanna set it up right? i read about it - it's pretty cool and all - only it's built in go [18:15:49] and we'll have to install Go [18:16:12] madhuvishy: we'll likley have to build a .deb package for it [18:17:15] ottomata: yeah [18:17:57] https://github.com/niemeyer/godeb [18:19:55] ottomata: I was thinking of setting up some config and playing with it in labs may be, and then filing tickets for deb packages. [18:20:39] aye madhuvishy cool! go ahead, I will be jealous because I want to do it too :) [18:20:56] ottomata: :D issok you have lots to do and i have nothing [18:21:03] haha, do it! [18:21:09] cool [18:28:38] (PS1) Milimetric: Parametrize the input filenames format [analytics/aggregator] - https://gerrit.wikimedia.org/r/247323 (https://phabricator.wikimedia.org/T114379) [18:29:07] joal - that's the change, I just added you as a reviewer ^ [18:29:10] (the python one) [18:29:17] onto puppet, but maybe lunch first [18:33:02] madhuvishy: wow, thanks! [18:35:26] !log restarting eventlogging with change to parse schema names out of errored events [18:35:29] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log, Master [18:40:57] ebernhardson: np! [18:44:59] Analytics-Tech-community-metrics, Patch-For-Review: Tech metrics missing IRC channels - https://phabricator.wikimedia.org/T56230#1735896 (Dicortazar) a:Dicortazar [18:54:26] Analytics-Tech-community-metrics, DevRel-October-2015, Patch-For-Review: Tech metrics missing IRC channels - https://phabricator.wikimedia.org/T56230#1735930 (Dicortazar) [18:54:51] joal: do we keep hive-site.xml in puppet? [18:55:23] oh, no we have it in oozie/util [18:57:57] madhuvishy: it is in puppet [18:58:09] refinery has a committed symlink to /etc/hive/conf/hive-site.xml [18:58:21] so that it is automatically copied to hdfs when refinery is depoyed there [18:58:23] ottomata: yeah, saw that - can't find it in puppet though [18:58:41] it is in the cdh module [18:59:04] https://github.com/wikimedia/puppet-cdh/blob/master/templates/hive/hive-site.xml.erb [18:59:22] aah [18:59:36] why do we have this as a submodule? [18:59:58] and not just a folder in ops puppet [19:01:33] so it can be used in places other than production and labs [19:01:37] e.g. vagrant [19:01:39] or elsewhere [19:01:50] other folks have submitted patches [19:01:57] other than wmf [19:02:04] they wouldn't have if it was buried in ops puppet [19:02:13] ottomata: hmmm [19:02:58] we can't use it in vagrant if it's ops puppet? [19:04:44] well, not in mediawiki vagrant, [19:04:51] i hear that bryan is making a new vagrant that uses ops puppet [19:05:03] but, i'm still pro git submodules for generic puppet modules [19:05:08] encourages open source contributions [19:05:18] although, I am a loaner in my opinion [19:05:21] most ops folks don't like them [19:06:08] ottomata: :) [19:06:10] figures [19:06:18] everything else is not submodule [19:06:32] right, i don't think everything should be a submodule [19:06:41] but, i do think the submoduleness of it forces you to make better design decisions [19:06:52] because then you don't introduce wmf specific stuff [19:06:58] some modules its ok to do that in. [19:07:00] depends on the module. [19:07:04] I haven't played with ops/puppet vagrant for quite a while -- https://gerrit.wikimedia.org/r/#/c/212294/ [19:07:09] right [19:07:13] if i were the guy in charge, i would have different types of modules. [19:07:16] generic and wmf specific [19:07:21] maybe even multiple module paths [19:08:05] ottomata: how would that work? [19:08:18] (multiple module paths) [19:09:16] at the most extreme, it looks like https://github.com/rodjek/librarian-puppet [19:10:42] madhuvishy: in my less extreme visions, you can configure puppet to autoload modules from multiple paths [19:10:48] so i'd just make 2 [19:11:04] maybe modules/wmf modules/generic, whatever [19:11:05] i dunno [19:11:12] mm hmm [19:11:14] Analytics-Tech-community-metrics, DevRel-October-2015, Patch-For-Review: Tech metrics missing IRC channels - https://phabricator.wikimedia.org/T56230#1735971 (Dicortazar) Based on info provided by @Aklapper, I've added all of them, but mediawiki that was already in the list. So, the list is as follows... [19:11:14] everything else would work the same [19:11:18] but, we aren't gonna do that [19:12:23] right [19:14:56] Analytics, Discovery, EventBus, MediaWiki-General-or-Unknown, and 7 others: EventBus MVP - https://phabricator.wikimedia.org/T114443#1735988 (Ottomata) Hey yalls, I've had requests that we postpone the RFC for this one more week, until Oct 28th. I'd like for one opsen and @ori to be able to att... [19:46:21] Analytics, Analytics-Cluster, Fundraising Tech Backlog, Fundraising-Backlog, operations: Verify kafkatee use for fundraising logs on erbium - https://phabricator.wikimedia.org/T97676#1736061 (Jgreen) >>! In T97676#1729945, @ellery wrote: > @awight Thanks for double checking, I understand the... [19:57:42] Analytics-Tech-community-metrics, DevRel-October-2015, Patch-For-Review: Tech metrics missing IRC channels - https://phabricator.wikimedia.org/T56230#1736096 (Aklapper) >>! In T56230#1735971, @Dicortazar wrote: > Based on info provided by @Aklapper, I've added all of them Thank you. I think we're coo... [20:02:08] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL: CRITICAL: 46.67% of data above the critical threshold [30.0] [20:02:32] hey ottomata [20:02:55] OH hye [20:02:56] coming [20:03:00] thanks [20:03:49] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK: OK: Less than 25.00% above the threshold [20.0] [20:12:27] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL: CRITICAL: 33.33% of data above the critical threshold [30.0] [20:14:07] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK: OK: Less than 25.00% above the threshold [20.0] [20:22:30] Analytics-Tech-community-metrics, DevRel-October-2015, Patch-For-Review: Tech metrics missing IRC channels - https://phabricator.wikimedia.org/T56230#1736157 (Dicortazar) @Aklapper, what I've done so far is to manually add them to the infrastructure (this is indeed a script deleting, removing, adding U... [20:27:06] Analytics-Tech-community-metrics, DevRel-October-2015: Backlogs of open changesets by affiliation - https://phabricator.wikimedia.org/T113719#1736160 (Dicortazar) @Qgil, instead of having there the backlog list, what do you think about adding a new entry to the Gerrit Backlog panel at http://korma.wmflabs... [20:35:01] (Abandoned) OliverKeyes: Add non-API searches to search UDFs [analytics/refinery/source] - https://gerrit.wikimedia.org/r/236860 (owner: OliverKeyes) [20:37:57] mforns: schema and revision going to EventError, woohoo! [20:38:05] (PS2) Milimetric: Parametrize the input filenames format [analytics/aggregator] - https://gerrit.wikimedia.org/r/247323 (https://phabricator.wikimedia.org/T114379) [20:44:50] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL: CRITICAL: 46.67% of data above the critical threshold [30.0] [20:46:37] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK: OK: Less than 25.00% above the threshold [20.0] [20:48:31] (PS6) Milimetric: Archive hourly pageviews by article in wsc format [analytics/refinery] - https://gerrit.wikimedia.org/r/246149 (https://phabricator.wikimedia.org/T114379) [20:50:22] Analytics-Tech-community-metrics, DevRel-October-2015: Correct affiliation for code review contributors of the past 30 days - https://phabricator.wikimedia.org/T112527#1736241 (Aklapper) I've got access to the JSON file (thanks!). I git cloned sortinghat, `./setup.py build` and `./setup.py install` and in... [20:54:04] ottomata, added them to the Kibana dashboard: https://logstash.wikimedia.org/#/dashboard/elasticsearch/eventlogging-errors [20:54:55] thanks :] [20:55:23] Analytics-Kanban, Analytics-Wikistats, Patch-For-Review: Send email out to community notifying of change - https://phabricator.wikimedia.org/T115922#1736253 (Tnegrin) NEW a:Tnegrin [20:55:38] COOL [20:56:01] nice! [20:56:52] :] [20:56:56] yea looks good [20:57:04] echo is having problem right now [20:59:59] Analytics, Discovery, EventBus, MediaWiki-General-or-Unknown, and 7 others: EventBus MVP - https://phabricator.wikimedia.org/T114443#1736288 (Ottomata) So, we need to be really careful here. This MVP as of yet has zero buy in from anyone in ops. In addition, both @ori and @eevans point out that... [21:02:55] mforns: yeah nice to see that [21:03:01] is that related to the flapping alerts? [21:04:54] Analytics-Tech-community-metrics, Possible-Tech-Projects, Outreachy-Round-11: Improving MediaWikiAnalysis - https://phabricator.wikimedia.org/T89135#1736292 (jgbarah) >>! In T89135#1734305, @01tonythomas wrote: > I am shifting this to #Outreachy-Round-11 as the project description has two mentors, mic... [21:16:05] ottomata, I don't think so, the alerts seem unrelated, and following the same 2-minute pattern they were having the last time [21:16:24] no idea why they're back... [21:17:01] Analytics-Tech-community-metrics, Possible-Tech-Projects, Epic, Outreachy-Round-11: Allow contributors to update their own details in tech metrics directly - https://phabricator.wikimedia.org/T60585#1736326 (jgbarah) >>! In T60585#1734314, @01tonythomas wrote: > I am shifting this to #Outreachy-Rou... [21:26:48] Analytics-Tech-community-metrics, Possible-Tech-Projects, Outreachy-Round-11: Improving MediaWikiAnalysis - https://phabricator.wikimedia.org/T89135#1736343 (jgbarah) [21:42:14] Analytics-Tech-community-metrics, DevRel-October-2015: Correct affiliation for code review contributors of the past 30 days - https://phabricator.wikimedia.org/T112527#1736373 (Aklapper) Looking at the JSON file (`85ec9724c524618f02d593d5180c29d9e166ece9` and `1ac2657a87de642b39f5e17344ea42eeba842e64`), t... [21:54:30] ottomata: what are the zookeeper hosts for labs kafka? [21:54:44] https://wikitech.wikimedia.org/wiki/Hiera:Analytics I thought these were the ones [21:54:54] but doesn't seem to be right [21:54:57] in beta or in analytics? [21:55:02] analytics [21:55:06] oh [21:55:30] is this for beta? [21:55:45] thought this was analytics [21:56:01] what? [21:56:36] ummm, we have these kafka servers in analytics labs project - kafka201, 203 etc [21:56:49] i want the zookeeper hosts that these talk to [21:57:43] madhuvishy: those were set up by me for testing the kafka migration [21:57:53] they aren't configured with puppet, since i was manually installing packages to test things [21:58:11] ottomata: okay, do you know the zk host names, or is there a way i can find them? [21:58:30] madhuvishy: i don't maintain kafka/zk in analytics project [21:58:39] anything that has been there are one offs [21:59:05] but ah [21:59:11] you want to just use these to try burrow? [21:59:11] ottomata: but but [21:59:12] hmm [21:59:15] yes [21:59:31] zookeeper.connect=kafka-jessie01.eqiad.wmflabs/k2 [21:59:38] cool [21:59:39] that's with kafka200 [21:59:40] at least [21:59:42] thanks [21:59:46] ya that should do it [22:05:17] PROBLEM - Difference between raw and validated EventLogging overall message rates on graphite1001 is CRITICAL: CRITICAL: 26.67% of data above the critical threshold [30.0] [22:12:08] RECOVERY - Difference between raw and validated EventLogging overall message rates on graphite1001 is OK: OK: Less than 25.00% above the threshold [20.0] [22:12:34] Analytics-Backlog, Analytics-Kanban, Wikimedia-Developer-Summit-2016: Developer summit session: Pageview API overview - https://phabricator.wikimedia.org/T112956#1736532 (Milimetric) @Qgil, thanks for pointing out that link. We do indeed want to make it a discussion and not a presentation. The stuf... [22:52:59] PROBLEM - puppet last run on erbium is CRITICAL: CRITICAL: Puppet has 1 failures [23:00:45] Analytics-Cluster, Analytics-Kanban: Use Burrow for Kafka Consumer offset lag monitoring - https://phabricator.wikimedia.org/T115669#1736736 (madhuvishy) p:Triage>High a:Ottomata>madhuvishy [23:04:16] Analytics-Engineering, operations: erbium (logging) - useradd: group '30001' does not exist - https://phabricator.wikimedia.org/T115943#1736743 (Dzahn) NEW [23:04:43] Analytics-Engineering, operations: erbium (logging) - useradd: group '30001' does not exist - https://phabricator.wikimedia.org/T115943#1736753 (Dzahn) [23:05:08] Analytics-Engineering, operations: erbium (logging) - useradd: group '30001' does not exist - https://phabricator.wikimedia.org/T115943#1736743 (Dzahn) @ottomata do you know anything about this? [23:05:57] ACKNOWLEDGEMENT - puppet last run on erbium is CRITICAL: CRITICAL: Puppet has 1 failures daniel_zahn https://phabricator.wikimedia.org/T115943 [23:06:24] Analytics-Engineering, operations: erbium (logging) - useradd: group '30001' does not exist - https://phabricator.wikimedia.org/T115943#1736765 (Dzahn) a:Ottomata [23:07:23] Analytics-Backlog, operations: erbium (logging) - useradd: group '30001' does not exist - https://phabricator.wikimedia.org/T115943#1736766 (madhuvishy) [23:09:19] Analytics, Monitoring, Security-General: status.wikimedia.org should not load Google Analytics - https://phabricator.wikimedia.org/T115945#1736775 (MZMcBride) NEW [23:10:10] Analytics-Backlog, Analytics-EventLogging: Convert EventLogging to use extension registration - https://phabricator.wikimedia.org/T87912#1736784 (madhuvishy) [23:10:53] Analytics-EventLogging, Editing-Department, Improving access, Reading Web Planning, discovery-system: Schema changes - https://phabricator.wikimedia.org/T114164#1736786 (atgo) [23:11:08] Analytics-EventLogging, Editing-Department, Improving access, Reading Web Planning, discovery-system: Schema changes - https://phabricator.wikimedia.org/T114164#1686646 (atgo) [23:11:32] Analytics, Monitoring, Privacy: status.wikimedia.org should not load Google Analytics - https://phabricator.wikimedia.org/T115945#1736790 (Krenair) [23:36:22] halfak: hey, quick question -- in https://www.mediawiki.org/wiki/Analytics/Metric_definitions#Active_editor , "in mainspace" means "in NS_MAIN", right?