[00:55:33] Analytics-Tech-community-metrics, Developer-Relations, Community-Tech-Sprint: Investigation: Can we find a new search API for CorenSearchBot and Copyvio Detector tool? - https://phabricator.wikimedia.org/T125459#2258430 (kaldari) FYI, we have another meeting with Microsoft tomorrow. [01:26:19] Quarry: Quarry should preserve the protocol on redirects - https://phabricator.wikimedia.org/T92600#2258455 (Huji) Open>declined Somehow in the last two weeks this is working correctly. Not sure if Yuvi_Panda has applied a patch that fixed this, or if its dependencies are updated. At any rate, I will... [01:50:36] Quarry: Quarry should preserve the protocol on redirects - https://phabricator.wikimedia.org/T92600#1115720 (Legoktm) Probably was fixed by {T107627}? [04:15:33] Analytics: Wikipedia Clickstream dataset. Programatic Access - https://phabricator.wikimedia.org/T134231#2258747 (Nuria) [04:52:00] (PS1) Amire80: Add jam to language list [analytics/limn-language-data] - https://gerrit.wikimedia.org/r/286582 [05:01:05] (CR) KartikMistry: [C: 2] Add jam to language list [analytics/limn-language-data] - https://gerrit.wikimedia.org/r/286582 (owner: Amire80) [05:01:23] (Merged) jenkins-bot: Add jam to language list [analytics/limn-language-data] - https://gerrit.wikimedia.org/r/286582 (owner: Amire80) [05:09:46] Analytics-Tech-community-metrics, Developer-Relations, Community-Tech-Sprint: Investigation: Can we find a new search API for CorenSearchBot and Copyvio Detector tool? - https://phabricator.wikimedia.org/T125459#2258785 (Tnegrin) @kaldari Are we still considering google? Was there a budget problem wi... [07:32:16] Hi a-team, reviews morning ! [07:32:28] Available if needed :) [07:39:22] me too :) [07:51:45] (CR) Nikerabbit: [C: -1] Add sorted errors (1 comment) [analytics/limn-language-data] - https://gerrit.wikimedia.org/r/282228 (owner: Amire80) [08:02:36] (PS3) Joal: Add script allowing rerunning oozie jobs [analytics/refinery] - https://gerrit.wikimedia.org/r/286471 [08:03:16] (CR) Joal: "Added some examples in the doc as well as the suggested changes. Doc still needed to be updated." (2 comments) [analytics/refinery] - https://gerrit.wikimedia.org/r/286471 (owner: Joal) [08:03:50] Analytics, DBA: Set up auto-purging after 90 days {tick} - https://phabricator.wikimedia.org/T108850#2258922 (jcrespo) a:jcrespo>None @mforns Please send an RC to operations/puppet with a plan-text list of tables, or any other info that will be needed to create a cron job to perform the purge. I... [08:03:52] Analytics, DBA: Set up auto-purging after 90 days {tick} - https://phabricator.wikimedia.org/T108850#2258924 (jcrespo) [08:08:33] Analytics-Kanban, Operations, ops-codfw, Patch-For-Review: rack/setup/deploy conf200[123] - https://phabricator.wikimedia.org/T131959#2258935 (elukey) Open>Resolved [08:09:15] Analytics-Kanban, Operations, Patch-For-Review: Upgrade stat1001 to Debian Jessie - https://phabricator.wikimedia.org/T76348#2258937 (elukey) [08:14:07] Analytics-Cluster, Operations, ops-eqiad: Analytics hosts showed high temperature alarms - https://phabricator.wikimedia.org/T132256#2258939 (elukey) Updated list (excluding empty results): ``` elukey@neodymium:~$ sudo -i salt -t 120 analytics10* cmd.run 'grep "Hardware event" /var/log/mcelog | uniq... [08:15:36] Analytics, Analytics-Cluster: Configure Spark YARN Dynamic Resource Allocation - https://phabricator.wikimedia.org/T101343#2258940 (elukey) a:Ottomata>elukey [08:17:24] Analytics, Analytics-Cluster, Patch-For-Review: https://yarn.wikimedia.org/cluster/scheduler should be behind ldap - https://phabricator.wikimedia.org/T116192#2258941 (elukey) a:Ottomata>elukey [08:26:16] ah snap I havent' started the review yet [08:26:36] * elukey blames himself for the "let's check phabricator 5 minutes" [08:26:47] elukey: it took me almost half an hour to get started zas well :) [08:27:10] joal: I guess that everything is more appealing than reviews, even tasks :P [08:27:20] elukey: for sure it is [08:27:32] elukey: however it's still interesting to reflect on the past year [09:58:35] Analytics, Analytics-Cluster, Patch-For-Review: https://yarn.wikimedia.org/cluster/scheduler should be behind ldap - https://phabricator.wikimedia.org/T116192#2259328 (elukey) p:Normal>Low [10:12:43] Analytics-Tech-community-metrics, Developer-Relations: Identify Wikimedia's most important/used info panels in korma.wmflabs.org - https://phabricator.wikimedia.org/T132421#2259418 (Qgil) [10:14:47] Analytics-Tech-community-metrics, Developer-Relations: Create basic/high-level Kibana (dashboard) documentation - https://phabricator.wikimedia.org/T132323#2259465 (Qgil) [10:16:01] Analytics-Tech-community-metrics, Developer-Relations: Mismatch between numbers for code merges per organization - https://phabricator.wikimedia.org/T129910#2259491 (Qgil) [10:16:05] Analytics-Tech-community-metrics, Developer-Relations: Play with Bitergia's Kabana UI (which might potential replace our current UI on korma.wmflabs.org) - https://phabricator.wikimedia.org/T127078#2259493 (Qgil) [10:16:26] Analytics-Tech-community-metrics, Developer-Relations, Differential: Make MetricsGrimoire/korma support gathering Code Review statistics from Phabricator's Differential - https://phabricator.wikimedia.org/T118753#2259495 (Qgil) [11:00:07] Analytics-Kanban, DC-Ops, EventBus, MediaWiki-Cache, and 5 others: setup kafka2001 & kafka2002 - https://phabricator.wikimedia.org/T121558#2259592 (elukey) Double checked that health checks work fine: ``` elukey@kafka1001:~$ curl http://kafka2002.codfw.wmnet:8085/v1/topics {"change-prop.retry.ch... [11:05:59] * elukey lunch! [12:36:59] Analytics-Kanban, DC-Ops, EventBus, MediaWiki-Cache, and 5 others: setup kafka2001 & kafka2002 - https://phabricator.wikimedia.org/T121558#2259830 (mobrovac) Note that the topics' names ought to be prefixed with `codfw.`. @elukey I guess you created them by running `./bin/ensure-kafka-topics-exis... [12:38:58] mobrovac: about --^ nope, I didn't execute anything [12:39:08] kk [12:40:54] mobrovac: I am adding LVS config for codfw event bus, anything against it? [12:41:18] no no, go ahead elukey [12:42:11] super :) [12:42:20] now I need to understand how to avoid fire on the LBs [12:45:19] elukey: i will need your help with kafka topics for the main eqiad cluster (aka the eventbus kafka instance) [12:47:45] mobrovac: sure, let me know when : [12:47:46] :) [12:49:04] elukey: i need https://gerrit.wikimedia.org/r/#/c/286246/ applied [12:49:29] elukey: i will address otto's comment and then we'll need to apply it because without it the change-prop service won't even start [12:49:32] (no such topics) [12:50:11] okok [13:02:18] halfak: meeting? [13:02:31] halfak: o/ [13:02:39] halfak: Excuse my rudeness :) [13:04:57] elukey: ok, https://gerrit.wikimedia.org/r/#/c/286246/ is good to go [13:05:58] elukey: i can self-merge and then you can run it on kafka1001? [13:07:08] mobrovac: sure [13:09:55] o/ [13:09:56] elukey: kk, you now have to go to tin into /srv/deployment/eventlogging/eventbus and do a deploy [13:09:58] Sorry slept in [13:10:04] ho prob halfak :) [13:10:12] halfak: take your time for coffee ;) [13:12:24] mobrovac: ack [13:15:22] mobrovac: done [13:15:24] all good [13:15:38] elukey: cool [13:15:50] elukey: now, need to run it on kafka1001 [13:16:57] bin/ensure-kafka-topics-exist --topic-config /srv/eventschemas/config/eventbus-topics.yaml --prefixes eqiad [13:17:15] elukey: ^ you might check if /srv/eventschemas is the right path [13:17:54] yeah was doing it [13:18:40] cool [13:18:50] done [13:18:57] grazie! [13:19:54] mobrovac: https://dpaste.de/0uKj/raw [13:20:13] yup :) [13:20:20] checked it with kafka-tee from bastion [13:20:27] elukey: thnx a lot! [13:21:28] good morning! [13:22:19] there's ottomata! [13:22:42] elukey: hm, now that the eventbus is present in codfw too, you'd need to run the command there as well [13:22:49] but with --prefixes codfw [13:24:14] ottomata: mobrovac is forcing me to execute random commands on eventbus, just FYI :P :P [13:24:24] hahahaha [13:24:37] haha [13:24:47] oh mobrovac you merged that prefixes thing eh? cool! [13:24:59] mobrovac: i actually don't fully understand the need for that [13:25:01] oh [13:25:01] yup, needed to ottomata [13:25:02] wait [13:25:03] yes i do [13:25:09] but addressed your comment first [13:25:15] cool [13:25:26] if we had auto topic create enabled, then i think we wouldn't [13:25:32] buuut, i guess it is off? checking... [13:25:49] we would [13:25:49] hmm, no its on [13:25:50] auto.create.topics.enable=true [13:26:12] change-prop subscribes to the retry queues and stuff on start-up [13:26:18] pykafka.exceptions.UnknownError: Failed to auto-create topic 'codfw.change-prop.retry.change-prop.backlinks.continue' [13:26:18] so it's a chicken-and-egg problem [13:26:21] buuuu [13:26:28] this on kafka2001.codfw [13:26:31] why? [13:26:41] ja but it should create the topics [13:26:46] if auto create topics is on [13:26:53] maybe not for consumers in 0.8.2? [13:27:02] pretty sure this should work for consumers [13:27:07] or, a good consumer :p [13:27:12] that kafka-node one is so bad [13:27:17] auto-create works only for producers producing to non-existent topics, not consumers subscribing to them [13:27:47] nope [13:27:49] i just did it with kafkacat [13:27:50] as a consumer [13:27:52] it created a topic [13:28:01] it has to do with the client [13:28:18] f*cker [13:28:21] the client needs to catch the topic not exist or whatever exception [13:28:25] and then just create the topic [13:28:30] if kafkacat does it [13:28:37] ottomata: regarding the "subscribe to all partitions" idea [13:28:43] then so will that kafka-native package thing I sent ya [13:28:46] since it uses librdkafka [13:28:55] ottomata: https://gerrit.wikimedia.org/r/#/c/286637/ [13:29:15] i don't think we want to do that in our case [13:29:57] we want to be able the consumer to be agnostic to the number of partitions, but still subscribe to only one and do so in such a manner that two different processes subscribe to different partitions [13:31:14] ja mobrovac your case is weird, since you have the same consumer that you want to subscribe to many topics [13:31:37] what about that high volume topic petr was talking about, for template expansion? [13:31:41] how high volume will that be? [13:31:45] you really only want one partition? [13:31:49] no, we want to have a stateless service and have load-balancing [13:31:56] if you ask me, that's a kafka fail honestly [13:32:42] no ottomata, we might need more than one, but still one process per partition and that has to be decided dynamically [13:33:29] mobrovac: kafka does load balancing fine within a single topic [13:33:33] with multiple partitions [13:33:55] but, if you want to balance between multiple topics, then ja, more difficult [13:34:11] no [13:34:21] it does no such thing as load-balancing, sorry [13:34:23] no? [13:34:29] no [13:34:32] how so? [13:34:50] if there are 10 partitions, (and you are using a good consumer client), if you start up 1 process [13:34:52] it consumes from 10 [13:34:54] if you start up 2 [13:34:56] they will notice [13:34:57] and rebalance [13:34:59] but please let's not get into a discussion we can't effectively do nothing about [13:34:59] and each consume 5 [13:35:15] haha, sure, just not sure I understand [13:35:42] that's not a kafka feature what you are describing ottomata [13:35:45] ? [13:35:46] that's all client-code [13:35:47] its a kafka client feature [13:35:49] indeed [13:36:07] actually, its built into the newer protocol, so that the group balancing is done via kafka [13:36:09] as far as kafka is concerned, the first to subscribe wins [13:36:14] instead of zookeeper [13:36:39] still the concept of load-balancing inside a consumer group is non-existent [13:36:45] which is a design flaw imho [13:38:36] elukey: ottomata: why no topic exists in codfw? [13:39:16] mobravac, latest librdkafka has consumer auto balancing built in, when using kafka 0.9 [13:39:29] because it uses the kafka protocol to do the balancing [13:39:32] not zookeeper [13:40:16] ottomata: but it's still offloaded to the client, it's not part of the server logic [13:40:27] how would it be part of the server logic? [13:40:39] ja, the client has to respond to a rebalance notification [13:40:39] have you ever used a msg-passing system? [13:40:59] some but not a lot! I have to admit! [13:41:07] not for high throughput anything where i thought about balancing [13:41:24] anywayyYyyYyyy, mobrovac i'm not gonna stop you, especially since you've already done tons of work on that kafka-node client, but i think it will be pretty annoying to update it for newer kafka versions going forward [13:41:39] since there is a librdkafka backed one that works, you might want to look into it eventually [13:41:50] i'd like to eventually update it for newer librdkafkas [13:41:50] the point is that kafka's limitation of "you cannot have more processes than you have partitions" is pretty bad [13:41:56] ah [13:42:04] i see [13:42:15] i think they made that decision for performance reasons, buuut anyway [13:42:20] ok q is, why no topics in codfw!? [13:42:21] hm [13:42:27] folks WAIT A MINUTE [13:42:31] WAITING [13:42:32] haha [13:42:32] hahaha [13:42:43] you guys write too much things per minute :P [13:43:01] I wrote a couple of minutes ago (like 500 sentences in the backlog) [13:43:01] kafka.admin.AdminOperationException: replication factor: 2 larger than available brokers: 1 [13:43:03] uh oh [13:43:05] pykafka.exceptions.UnknownError: Failed to auto-create topic 'codfw.change-prop.retry.change-prop.backlinks.continue' [13:43:21] 1 broker? [13:43:25] don't we have 2 nodes there? [13:43:30] should have [13:43:31] should be maybe it doesn't know about the other? [13:44:11] i only see 2002 registered in zookeeper [13:44:11] hm [13:45:09] kafka2002.codfw.wmnet:9092/bootstrap: Failed to connect to broker at kafka2002.codfw.wmnet:9092: Connection refused [13:45:33] just restarted the brokers JUST IN CASE [13:46:22] ahh kafka topic --describe doesn't work on 2001 [13:46:23] weird [13:46:29] RECOVERY - eventlogging-service-eventbus endpoints health on kafka2002 is OK: All endpoints are healthy [13:47:23] ok, i see both in zk now [13:47:28] yeppa [13:47:58] mobrovac: topics created [13:48:10] merci beaucoup! [13:48:27] hehe, mobrovac if you are hacking on the kakfa-node client... you should make it auto create topics :p [13:48:42] thanks elukey! [13:48:55] elukey: want to do a kafka upgrade in labs with me today? [13:49:10] ottomata: suure! [13:49:27] ok, gimme a few to finsih checking my email, then batcave? [13:49:43] ottomata: yeppa! I also have https://gerrit.wikimedia.org/r/#/c/286621/2, need only to update LVS [13:50:06] oh ja [13:50:08] but https://wikitech.wikimedia.org/wiki/LVS#Deploy_a_change_to_an_existing_service is a bit confusing [13:50:11] yeah i saw those reviews, will do those first too [13:51:03] cool elukey i don't know that system that well either, but that looks good to me and joe +1ed it [13:51:42] ottomata: all right I'll ask to Giuseppe later on, don't want to set everything on fire :) [13:51:54] ah naw, i don't think it will set anything on fire [13:52:14] did you already talk to him about it, or he just saw your patch? [13:53:07] yep yep he knows, but I wanted to do background reading first. Not sure if confd will manage pybal restarts or not for active/standby lbs, I'll ask him [13:53:55] ok [13:53:57] ok cool [13:54:16] i had known the answer once...because I did this for eventbus eqiad, but that is the only time I did it, and I also had guidance :) [13:54:28] thanks for taking care of this one too elukey! I'm sure mobrovac appreciates it too :D [13:54:39] yup yup [13:55:25] it has been super fun, last step will be to add dashboards for zookeeper [13:59:05] Analytics-Kanban, DC-Ops, EventBus, MediaWiki-Cache, and 5 others: setup kafka2001 & kafka2002 - https://phabricator.wikimedia.org/T121558#2260066 (elukey) Created topics with mobrovac, all good. Last step is to enable LVS. [14:02:36] hm mobrovac for now, we could make the eventschemas puppet module notify an exec that runs ensure-topic-exists or something [14:02:48] if you want to get the topics auto created once the topic config is modified [14:03:13] also ottomata do you think that I can merge https://gerrit.wikimedia.org/r/#/c/286637/1/manifests/role/logging.pp ? [14:04:46] yes sorry elukey! proceed [14:04:51] lots of emails and chats :) [14:04:57] i totally missed that. [14:05:00] too bad that isn't automatic [14:05:08] i wonder if it could be now... [14:05:11] yep :( [14:05:16] probably would require a change to kafkatee [14:05:21] but i'm pretty sure librdkafka supports it [14:05:43] * elukey is not going to touch kafkatee [14:05:56] * elukey is still scared by varnishkafka's codebase [14:06:42] haha [14:16:42] Analytics-Kanban, DBA: Set up auto-purging after 90 days {tick} - https://phabricator.wikimedia.org/T108850#2260130 (mforns) [14:23:56] (CR) Ottomata: [C: 1] Normalize oozie job names (bundles, coords, wfs) [analytics/refinery] - https://gerrit.wikimedia.org/r/285400 (https://phabricator.wikimedia.org/T130732) (owner: Joal) [14:24:39] (CR) Ottomata: [C: 1] Add script allowing rerunning oozie jobs [analytics/refinery] - https://gerrit.wikimedia.org/r/286471 (owner: Joal) [14:29:28] Analytics, Research-and-Data-Backlog: Wikipedia Clickstream dataset. Programmatic Access - https://phabricator.wikimedia.org/T134231#2260159 (DarTar) [14:29:40] Analytics, Data-release, Research-and-Data-Backlog: Wikipedia Clickstream dataset. Programmatic Access - https://phabricator.wikimedia.org/T134231#2258736 (DarTar) [14:29:52] Analytics, Data-release, Research-and-Data-Backlog: Wikipedia Clickstream dataset. Programmatic Access - https://phabricator.wikimedia.org/T134231#2258736 (DarTar) [14:30:49] elukey: i'm prepping some puppet role stuff so we can try it, trying to prep for the same process we'd do in prod, with some testing [14:32:06] ottomata: let me know when we can batcave! [14:33:00] ok, prop 15 mins [14:33:02] prob [14:41:50] ottomata: yeah, i was thinking we should do something like that [14:42:43] Analytics-Kanban, Operations, Patch-For-Review: Upgrade stat1001 to Debian Jessie - https://phabricator.wikimedia.org/T76348#2260210 (ezachte) This task is set to done, but my home dir hasn't been restored yet, and https://stats.wikimedia.org/cgi-bin/search_portal.pl?search=views produces an access e... [14:46:32] elukey: batcave?! :) [14:49:27] ottomata: skipping, just read an update about stat1001, will work on it :( [14:50:19] oh? what's up? [14:55:17] ottomata: nothing big, I think my rsync command didn't copy all the homes and some perl script is not working [14:55:52] Analytics-Kanban, Operations, Patch-For-Review: Upgrade stat1001 to Debian Jessie - https://phabricator.wikimedia.org/T76348#2260239 (elukey) Hi @ezachte, my bad, working on it now. [15:01:24] ottomata: heya! [15:03:24] HI! [15:03:32] Hmmm ok [15:03:49] ottomata: do we take some time to talk about schemas? [15:05:42] sure! now is fine i suppose [15:05:47] i'm in batcave :) [15:05:51] joining ! [15:33:29] joal, ottomata I am trying to figure out why stat1001 cgi dir is not working, I am grabbing some more minutes, letme know if you need me [15:33:52] elukey: we are in batcave, taking schemas -> no rush [15:38:19] mobrovac: when you have some time, joal and I have some qs for you, can probably just ask the easy ones via IRC [15:39:20] we can have a quick h-o in 30 mins or so if you guys want [15:40:55] mobrovac: in 50 minutes ok? we ahve standup in 10 [15:41:12] sorry, standup in 20 [15:41:42] that's going to be too late i'm afraid [15:41:57] joal: we can have a h-o tomorrow morning eu time for this [15:41:59] ? [15:42:15] mobrovac: sure ! [15:42:20] deal! [15:43:17] invite sent [15:45:48] joal: can i ask you a question? [15:45:52] Analytics-Kanban, Operations, Patch-For-Review: Upgrade stat1001 to Debian Jessie - https://phabricator.wikimedia.org/T76348#2260441 (elukey) [15:47:22] nuria_: sure, shoot ! [15:47:32] joal: I am looking at https://hue.wikimedia.org/oozie/list_oozie_workflow/0021498-160420145651441-oozie-oozi-W/ [15:47:44] which is the job (i think) that was sending oozie warnings [15:47:59] joal: but in the log tag i do not see anything about the jamaican wikipedia [15:48:07] joal: is there other way to see that? [15:49:13] nuria: hive, wmf.pageview_unexpected_values table :) [15:49:29] joal argh! [15:49:34] I have asked this before [15:49:54] nuria_: no problemo :) [15:51:06] mobrovac: I might be a bit late tomorrow morning depending on my son's schedule :) [15:51:30] mobrovac: I won't be very late though if I am [15:51:39] joal: euh, would 12h30 be a better time? [15:51:51] mobrovac: it would [15:51:53] (at 11h15 i have another engagement) [15:51:54] kk [15:51:56] will move it [15:52:00] awesome, thanks [16:04:11] Analytics-Kanban: Ease restarting and backfilling of jobs in cluster {hawk} - https://phabricator.wikimedia.org/T115985#2260555 (JAllemandou) a:JAllemandou [16:04:22] Analytics-Kanban: Ease restarting and backfilling of jobs in cluster {hawk} - https://phabricator.wikimedia.org/T115985#1737628 (JAllemandou) [16:05:35] Analytics-Cluster, Analytics-Kanban, Patch-For-Review: Fix hive-metastore vs libmysql-jar race condition when provisioning new hive metastore server - https://phabricator.wikimedia.org/T133198#2260564 (Ottomata) [16:14:58] (PS1) Nuria: Adding jam.wikipedia to domains for which we count pageviews [analytics/refinery] - https://gerrit.wikimedia.org/r/286672 [16:16:20] Analytics: Count jam.wikipedia pageviews - https://phabricator.wikimedia.org/T134279#2260635 (Nuria) [16:16:58] (PS2) Nuria: Adding jam.wikipedia to domains for which we count pageviews [analytics/refinery] - https://gerrit.wikimedia.org/r/286672 (https://phabricator.wikimedia.org/T134279) [16:18:24] (PS3) Nuria: Adding jam.wikipedia to domains for which we count pageviews [analytics/refinery] - https://gerrit.wikimedia.org/r/286672 (https://phabricator.wikimedia.org/T134279) [16:19:02] Analytics-Kanban, Patch-For-Review: Count jam.wikipedia pageviews - https://phabricator.wikimedia.org/T134279#2260653 (Nuria) [16:20:24] joal: mobrovac, quick hangout? [16:20:34] gotta run, now, sorry [16:20:38] ah ok [16:20:41] k later then! [16:22:18] hey elukey your rsync module seems fine [16:22:33] from stat1001 to pull [16:22:35] i think you just need to run [16:22:39] sudo rsync -av stat1004.eqiad.wmnet::home/home/ /home/ [16:22:42] try it with --dry-run [16:22:45] to see [16:24:51] ottomata: Iddiiidd but didn't work, will retry thanks :) [16:27:15] hm, dry run looks good to me [16:27:19] let me know if it doesn't work [16:27:23] i haven't tried it wihtout dry run [16:30:29] probably I used /srv/stat1001/home/home, will try now [16:30:41] I was playing with perl [16:31:33] ah ja [16:31:40] for rsync, you use the module name in rsyncd.conf [16:31:41] in your case [16:31:59] ottomata: you are always the best, problem solved [16:32:00] thanks! [16:32:02] [ home ] [16:32:02] path = /srv/stat1001/home [16:32:06] so you address a module like [16:32:12] :: [16:32:14] with the double colon [16:32:23] ahhhhhh [16:32:28] :: will point to whatever you have path set to [16:32:35] in rsyncd.conf [16:32:46] okok so definitely I was typing the wrong thing [16:32:49] now I got it [16:32:54] I was using :/ [16:32:55] stupid me [16:45:55] nuria_: still in meeting? [16:46:04] joal: just got out [16:46:26] * elukey is fighting with mod_cgi [16:46:28] nuria_: was looking at kanban, and looks like some little cleaning could be done [16:46:46] joal: let me look [16:46:54] nuria_: batcave? [16:47:00] joal: sure [16:47:02] joal: omw [16:47:27] Analytics-Kanban: Upgrade scripts to facilitate wiki data loading / treatment on hadoop - https://phabricator.wikimedia.org/T132590#2260756 (Nuria) Open>Resolved [16:50:05] Analytics: Wikistats 2.0. Edit Reports: Setting up a pipeline to source Historical Edit Data into hdfs {lama} - https://phabricator.wikimedia.org/T130256#2260763 (Nuria) [16:50:07] Analytics-Kanban, Analytics-Wikistats, Reading-Admin: {lama} Wikistats traffic reports 2.0 - https://phabricator.wikimedia.org/T107175#2260762 (Nuria) [16:50:48] Analytics-Kanban, Analytics-Wikistats, Reading-Admin: {lama} Wikistats traffic reports 2.0 - https://phabricator.wikimedia.org/T107175#1488807 (Nuria) [16:52:02] Analytics-Wikistats: figure out process for adding new wikis to wikistats - https://phabricator.wikimedia.org/T64739#658873 (Nuria) Wikistats is being phased out and new wikis pageview counts (not edits) are automatically added after reviewed by analytics engineering and ops. Closing. [16:52:13] Analytics-Kanban, Analytics-Wikistats, Reading-Admin: {lama} Wikistats traffic reports 2.0 - https://phabricator.wikimedia.org/T107175#2260782 (Nuria) [16:52:15] Analytics-Wikistats: figure out process for adding new wikis to wikistats - https://phabricator.wikimedia.org/T64739#2260781 (Nuria) Open>declined [16:52:33] Analytics-Kanban, Analytics-Wikistats, Reading-Admin: {lama} Wikistats traffic reports 2.0 - https://phabricator.wikimedia.org/T107175#1488807 (Nuria) Open>Resolved [16:53:01] Analytics-Kanban: Unique Devices Endpoint: Write documentation - https://phabricator.wikimedia.org/T132794#2260784 (JAllemandou) [16:53:18] Analytics, Analytics-Cluster: Deploy oozie reporting of last-access counts {bear} - https://phabricator.wikimedia.org/T103376#2260786 (Nuria) Open>Resolved [16:53:20] Analytics-Cluster, Analytics-Kanban, Epic: {bear} Last Access Counts - https://phabricator.wikimedia.org/T88647#2260787 (Nuria) [16:53:40] Analytics-EventLogging, Analytics-Kanban, Analytics-Visualization: {lion} VE Support - https://phabricator.wikimedia.org/T89924#2260788 (JAllemandou) [16:53:54] Analytics-Cluster, Analytics-Kanban, Epic: {bear} Last Access Counts - https://phabricator.wikimedia.org/T88647#1016919 (Nuria) Open>Resolved [16:54:18] Analytics-Kanban, Operations, Patch-For-Review: Upgrade stat1001 to Debian Jessie - https://phabricator.wikimedia.org/T76348#2260792 (Nuria) [16:54:21] Analytics-Kanban, Patch-For-Review: Out of service banner in dashiki - https://phabricator.wikimedia.org/T133736#2260791 (Nuria) Open>Resolved [16:54:22] Analytics-Cluster, Analytics-Kanban: Story: Community has periodic browser stats report generated from Hadoop data - https://phabricator.wikimedia.org/T69053#2260793 (JAllemandou) [16:54:41] Analytics-Tech-community-metrics, Developer-Relations, Community-Tech-Sprint: Investigation: Can we find a new search API for CorenSearchBot and Copyvio Detector tool? - https://phabricator.wikimedia.org/T125459#2155977 (Dodger67) AFC reviewers are now being criticised for missing copyvios. Earwig's... [16:54:57] Analytics-Cluster, Analytics-Kanban, Datasets-General-or-Unknown: {slug} Pageview API Ops - https://phabricator.wikimedia.org/T101792#2260796 (JAllemandou) duplicate>Open [16:55:02] Analytics-Kanban, Patch-For-Review: Client values inbound in X-analytics header (pageview and preview) are reflected in outbound X-Analytics on varnish - https://phabricator.wikimedia.org/T133204#2260798 (Nuria) [16:55:13] Analytics-Kanban, Analytics-Wikistats, Reading-Admin: {lama} Wikistats traffic reports 2.0 - https://phabricator.wikimedia.org/T107175#2260799 (JAllemandou) [16:55:16] Analytics-Cluster, Analytics-Kanban, Epic: {bear} Last Access Counts - https://phabricator.wikimedia.org/T88647#2260800 (JAllemandou) [16:55:26] Analytics-Cluster, Analytics-Kanban: {mule} Hadoop Cluster Expansion - https://phabricator.wikimedia.org/T99952#2260802 (JAllemandou) [16:55:28] Analytics-Cluster, Analytics-Kanban: {wren} PV Aggregates - https://phabricator.wikimedia.org/T96314#2260803 (JAllemandou) [16:55:35] Analytics-Kanban: {bull} Unique Tokens - https://phabricator.wikimedia.org/T102221#2260804 (JAllemandou) [16:55:39] Analytics-EventLogging, Analytics-Kanban: {stag} EventLogging on Kafka - https://phabricator.wikimedia.org/T102225#2260805 (JAllemandou) [16:55:45] Analytics-Cluster, Analytics-Kanban: {musk} Pageviews in Vital Signs - https://phabricator.wikimedia.org/T101120#2260806 (JAllemandou) [16:55:55] Analytics-Kanban, Analytics-Visualization: {Epic} Community reads pageviews per project in Vital Signs {crow} - https://phabricator.wikimedia.org/T95336#2260807 (JAllemandou) [16:56:01] Analytics-Kanban: {pika} Proactive Pageview Definition - https://phabricator.wikimedia.org/T109745#2260808 (JAllemandou) [16:56:13] Analytics-Kanban, Operations: Out of memory errors causing Yarn nodemanager to shutdown on analytics hosts - https://phabricator.wikimedia.org/T132559#2260809 (JAllemandou) [16:56:24] Analytics-Kanban: Investigate why bignbot appears on browser list when we are restricting pageviews to "user" agent type - https://phabricator.wikimedia.org/T131512#2260810 (JAllemandou) [16:56:42] Analytics-Kanban: Bug in pageviews - https://phabricator.wikimedia.org/T131386#2260811 (JAllemandou) [16:56:55] Analytics-Kanban, Operations, Ops-Access-Requests, Patch-For-Review: All members of analytics team need to have sudo -u hdfs on cluster {hawk} [2 pts] - https://phabricator.wikimedia.org/T126752#2260812 (JAllemandou) [16:57:08] Analytics-Kanban: Move vital signs to its own instance {crow} - https://phabricator.wikimedia.org/T123944#2260813 (JAllemandou) [16:57:11] Analytics-Kanban: annotations not visible on dashiki - https://phabricator.wikimedia.org/T128494#2260814 (JAllemandou) [16:57:31] Analytics-Kanban: Unique Devices Endpoint: Write documentation - https://phabricator.wikimedia.org/T132794#2260815 (JAllemandou) [16:57:33] Analytics-Kanban: Make legends on graphs better and more generic - https://phabricator.wikimedia.org/T129497#2260816 (JAllemandou) [16:57:42] Analytics-Kanban: Build a javascript client for the unique devices API - https://phabricator.wikimedia.org/T133159#2260817 (JAllemandou) [16:57:46] Analytics-Kanban: Add unique devices to vital signs - https://phabricator.wikimedia.org/T133771#2260818 (JAllemandou) [16:58:00] Analytics-Kanban, Operations, Patch-For-Review: nf_conntrack warnings for kafka hosts - https://phabricator.wikimedia.org/T131028#2260819 (JAllemandou) [16:58:05] Analytics-Cluster, Analytics-Kanban, Patch-For-Review: Add automatic failover to the Hadoop's name node - https://phabricator.wikimedia.org/T129838#2260820 (JAllemandou) [16:58:11] Analytics-EventLogging, Analytics-Kanban, Patch-For-Review: Add IP field only to schemas that need it. Remove it from EL capsule and do not collect it by default {mole} - https://phabricator.wikimedia.org/T126366#2260821 (JAllemandou) [16:58:14] Analytics-Kanban: Puppet on stat1003 keeps failing for git errors - https://phabricator.wikimedia.org/T132445#2260822 (JAllemandou) [16:58:18] Analytics-Kanban, Patch-For-Review: Corrext camus partition checker to not fail globally on one topic error - https://phabricator.wikimedia.org/T127909#2260823 (JAllemandou) [16:58:24] Analytics-Kanban, Operations, Traffic, Patch-For-Review: varnishkafka integration with Varnish 4 for analytics - https://phabricator.wikimedia.org/T124278#2260824 (JAllemandou) [16:58:31] Analytics-Kanban: Provide browser reports charts as stacked %age of all pageviews - https://phabricator.wikimedia.org/T131126#2260825 (JAllemandou) [16:58:36] Analytics-Kanban: Group flavours of desktop Windows together in OS reports - https://phabricator.wikimedia.org/T131125#2260826 (JAllemandou) [16:58:41] Analytics-Kanban, MediaWiki-API, Reading-Infrastructure-Team: Create wmf_raw.ApiAction table - https://phabricator.wikimedia.org/T129886#2260827 (JAllemandou) [16:58:47] Analytics-EventLogging, Analytics-Kanban, QuickSurveys: Add magic word for hashed client IP event log field for Quick Surveys - https://phabricator.wikimedia.org/T127502#2260828 (JAllemandou) [16:59:15] Analytics-Kanban, Patch-For-Review: Count jam.wikipedia pageviews - https://phabricator.wikimedia.org/T134279#2260830 (Nuria) [17:00:08] Analytics-Kanban, Operations, Patch-For-Review: Upgrade stat1001 to Debian Jessie - https://phabricator.wikimedia.org/T76348#2260834 (elukey) Update: thanks to @ottomata I was able to restore correctly the home directories, now everything should be fine. I am in the process of adding mod-cgi to stat... [17:01:10] i am in batcave! [17:01:14] oho i'm not [17:01:14] ! [17:01:16] now i am [17:01:34] google just returned me a 502 /o\ [17:02:38] mforns: staffff? [17:06:05] !log Manually add jam.wikipedia to the accepted project list for pageview [17:48:34] ggellerman: can you also remove Beauvais and Lovelace for staff meetings and have them at chambers may be? [17:50:21] madhuvishy: staff meeting is not on Analytics team calendar as far as I can tell. nuria_: can you help? [17:51:33] ggellerman: oh [17:58:02] now https://stats.wikimedia.org/cgi-bin/search_portal.pl?search=views works \o/ [17:59:12] yeehaw nice elukey [17:59:29] elukey: i'm in batcave, eatin a sammy [18:00:17] Analytics-Kanban, Operations, Patch-For-Review: Upgrade stat1001 to Debian Jessie - https://phabricator.wikimedia.org/T76348#2261058 (elukey) And https://stats.wikimedia.org/cgi-bin/search_portal.pl?search=views now works fine! @ezachte: would you mind to do another sanity check to verify that every... [18:05:17] I'm off for today a-team, will see you tomorrow [18:05:25] laters [18:05:40] byez! [18:15:34] (PS4) Nuria: Adding jam.wikipedia to domains for which we count pageviews [analytics/refinery] - https://gerrit.wikimedia.org/r/286672 (https://phabricator.wikimedia.org/T134279) [18:16:22] joal: now they are tabs so ...ahem, not sure what is up with diff [18:16:23] https://gerrit.wikimedia.org/r/#/c/286672/4/static_data/pageview/whitelist/whitelist.tsv [18:34:53] Hallo [18:35:19] There's a thing about which I wondered many times, but never quite got an answer. [18:35:38] Interlanguage links: Does anybody measure how frequently are they clicked, in any way? [18:36:51] [ Just as I asked an important question , DarTar, who may know the answer, came in, so I'll ask again ] [18:36:58] Interlanguage links: Does anybody measure how frequently are they clicked, in any way? [18:37:09] hey aharoni [18:38:24] I don’t think we have a quick answer / data point available, but the clickstream data Ellery prepared should allow you to easily answer that question (as it preserves interlang link referrals) [18:38:43] you may want to ask HaeB and the Reading team too [18:38:54] I think I came across some instrumentation proposal some time ago [18:39:08] Is any of them around here? I'm not on this channel frequently. [18:40:14] a-team logging off! we have HTTP/2 atm in cache::misc, https://stats.wikimedia.org/ is served also with h2 \o/ [18:40:24] bye elukey! [18:42:39] aharoni: best come over to #wikimedia-mobile (the channel of the reading team) or post to mobile-l [18:42:50] but yes, there have been various instrumentations, e.g. https://meta.wikimedia.org/wiki/Schema:MobileWebLanguageSwitcher recently [18:43:14] there we go [18:43:40] also you may remember https://en.wikipedia.org/wiki/Wikipedia:Wikipedia_Signpost/2010-06-07/News_and_notes [18:43:42] "we measured the click behavior for two groups of English Wikipedia users, Monobook and Vector (Vector users are primarily those who participated in the beta). Of Monobook and Vector users, 0.95% and 0.28% clicked on the language links (out of 126,180 and 180,873 total clicks), respectively." [18:44:26] I also believe the app team is looking at language switch data, right? [18:45:08] there are two app teams, and yes, they both developed their own language switchers and IIRC intrumented them too [18:45:50] HaeB: Heh-heh-heh, I _started_ the 2010 Interwiki debate!!! [18:45:53] overall i think there are now five different implementations, if you consider the beta feature on desktop too that the language engineering team built a while ago [18:46:00] Long before I was staff. [18:46:10] (which shows you a list of preferred language on top, but never got into production)) [18:46:31] So that's exactly why I'm raising the question: We want to make that feature out of beta finally. [18:46:46] But we want to measure whether it actually affects user behavior. [18:46:51] well maybe then you should talk to the reading team anyway ;) [18:47:30] My hypothesis is that people will click these links more, which would be a good thing, but I wonder what's the best way to measure "before and after". [19:33:09] ggellerman: staff meeting disappeared? [19:33:39] nuria: didn't see it when I was on Analytics Team calendar [19:34:57] ggellerman: maybe not on analytics calendar but it i scertainly scheduled [19:35:05] as i see it on my calendar [19:36:27] nuria_: I think that it is on your calendar, so I can't change it. Could you update the room per request from madhuvishy: ? [19:36:45] ggellerman: I will try, waht room should it be at? [19:36:48] *what [19:37:04] nuria_: chambers [19:37:13] nuria_: R37 [19:38:22] ggellerman: done, let me know if you see updates on your end [19:39:49] nuria_: only see it for May 3 instance [19:41:00] ggellerman: i see it for all..? mforns question: can you see update i just sent about analytics staff meeting room? [19:41:37] mforns: does it affect all instances of meeting? [19:41:42] nuria_: ah, probably because you only added me for this week which is actually fine [19:41:54] ggellerman: ah ok, want to be added for all? [19:42:00] ggellerman: can do that now [19:42:18] nuria: case by case is fine, or add me to all and I'll only come when needed [19:42:56] ggellerman: just did [19:46:09] nuria_: thanks! [19:52:49] mforns: fixed the pattern bug, let me see that tests run clean [19:54:30] nuria_, really? cool! [19:54:41] I was trying also with no success [20:02:43] Analytics-Kanban, Operations, ops-eqiad: rack/setup/deploy aqs100[456] - https://phabricator.wikimedia.org/T133785#2261516 (Cmjohnson) [20:26:37] Analytics-Kanban, Patch-For-Review: Fix Dashiki's metrics-by-project breakdown - https://phabricator.wikimedia.org/T133944#2261630 (mforns) [20:32:22] mforns: let me retest and will push in abit [20:32:29] nuria_, OK [20:51:28] (PS1) Nuria: Fix issues in metrics-by-project breakdown patterns [analytics/dashiki] - https://gerrit.wikimedia.org/r/286755 (https://phabricator.wikimedia.org/T133944) [20:52:46] mforns: test patch and let me know, will see your comments later [20:52:55] nuria_, sure [21:52:30] (CR) Mforns: [C: -1] "See comment :]" (1 comment) [analytics/dashiki] - https://gerrit.wikimedia.org/r/286755 (https://phabricator.wikimedia.org/T133944) (owner: Nuria) [22:01:05] Quarry, AutoWikiBrowser, WorkType-Maintenance: Quarry run result in AWB make list - https://phabricator.wikimedia.org/T134141#2261977 (Reedy) Yeah, dealing with UTF-8 isn't an issue whatsoever [22:48:05] mforns: yt? [22:48:11] nuria_, yes [22:48:14] mforns: can you tell me how to repro the bug? [22:48:20] mforns: I couldn't anymore [22:48:27] sure, nuria_ batcave? [22:48:35] I can show you in my dev tools [22:48:35] mforns: ya, give me a sec [22:49:12] k [22:49:13] omw