[01:34:49] 10Analytics, 10Beta-Cluster-Infrastructure, 10WikimediaPageViewInfo, 13Patch-For-Review: Deploy WikimediaPageViewInfo extension to beta cluster - https://phabricator.wikimedia.org/T129602#2778514 (10Reedy) [02:53:07] 06Analytics-Kanban, 06Wikipedia-iOS-App-Backlog, 10iOS-app-feature-Analytics: Drop in iOS app pageviews since version 5.2.0 - https://phabricator.wikimedia.org/T148663#2778609 (10Tbayer) >>! In T148663#2764417, @JAllemandou wrote: > `iPhone` string is currently necessary in the user_agent for webrequest bein... [08:09:23] rebooting stat1001! :) [08:29:15] (03PS1) 10Elukey: Add ec.wikimedia.org to the pageview whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/320351 [10:06:56] 06Analytics-Kanban, 06Operations, 10Traffic: Varnishlog with Start timestamp but no Resp one causing data consistency check alarms - https://phabricator.wikimedia.org/T148412#2778941 (10ema) The cache_text waitinglist issue should be solved by https://gerrit.wikimedia.org/r/#/c/320259/ [10:13:50] also --^ should have solved the text issue \o/ [10:18:41] elukey: YAY ! [10:19:31] I haven't seen any alarms during the past hours, and more or less it coincide with Brandon's merge [10:19:58] Awesome [10:52:57] 06Analytics-Kanban, 10EventBus, 13Patch-For-Review, 06Services (watching), and 3 others: Empty body in EventBus request - https://phabricator.wikimedia.org/T148251#2779034 (10mobrovac) >>! In T148251#2777557, @Ottomata wrote: > Since we know the fields that might have binary data, can we just check for it... [11:09:10] 06Analytics-Kanban, 06Operations, 10Traffic: Varnishlog with Start timestamp but no Resp one causing data consistency check alarms - https://phabricator.wikimedia.org/T148412#2722084 (10BBlack) The real problem here is a misunderstanding between varnish shm log's semantics and how we're interpreting that in... [11:09:21] 10Analytics, 10ChangeProp, 10Citoid, 10ContentTranslation-CXserver, and 11 others: Node 6 upgrade planning - https://phabricator.wikimedia.org/T149331#2779114 (10KartikMistry) Looks good after testing on a different machine(s). [11:10:02] 10Analytics, 10ChangeProp, 10Citoid, 10ContentTranslation-CXserver, and 11 others: Node 6 upgrade planning - https://phabricator.wikimedia.org/T149331#2779115 (10KartikMistry) [11:10:10] 10Analytics-Dashiki, 06Analytics-Kanban, 13Patch-For-Review: Migrate from bower to yarn and clean up folder hierarchy - https://phabricator.wikimedia.org/T147884#2706880 (10TheDJ) Related discussion on frontend library management: {T107561} [11:10:45] (03PS28) 10Joal: [WIP] Join and denormalize all histories into one [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/307903 (owner: 10Milimetric) [11:46:01] 10Analytics: Document "-" page title being a special value for "no title found" in pageview API. - https://phabricator.wikimedia.org/T150249#2779238 (10JAllemandou) [11:46:24] 10Analytics, 10ChangeProp, 10Citoid, 10ContentTranslation-CXserver, and 11 others: Node 6 upgrade planning - https://phabricator.wikimedia.org/T149331#2779250 (10mobrovac) The only ones left are the Maps services. @Yurik @Gehel could you test them with Node 6? [11:46:39] 10Analytics, 10ChangeProp, 10Citoid, 10ContentTranslation-CXserver, and 11 others: Node 6 upgrade planning - https://phabricator.wikimedia.org/T149331#2779251 (10mobrovac) p:05Triage>03Normal [11:51:44] 10Analytics, 10ChangeProp, 10Citoid, 10ContentTranslation-CXserver, and 11 others: Node 6 upgrade planning - https://phabricator.wikimedia.org/T149331#2779264 (10Gehel) @mobrovac sure, we'll have a look asap... [12:19:26] Hi milimetric :) Let me know when you want to spend some time on metrics and vetting :) [12:21:08] joal: I'm gonna go save America, brb [12:21:32] milimetric: please do that, as many time they let you ;) [12:25:11] (03PS1) 10Joal: Add ec.wikimedia to pageview whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/320376 [12:25:15] elukey: --^ [12:25:29] elukey: please ;) [12:26:42] * elukey feels ignored by joal [12:26:57] (PS1) Elukey: Add ec.wikimedia.org to the pageview whitelist [analytics/refinery] - https://gerrit.wikimedia.org/r/320351 [12:27:00] :P [12:27:20] oooooooh, I had not seen !!!! I'll kill mine ;) [12:27:23] elukey: --^ [12:27:32] hahahah no no don't worry let's merge yours [12:27:38] I was kidding [12:27:45] * joal brings a coffee to elukey and begs pardon :) [12:27:52] I replied to the email without adding reviewers :D [12:28:02] ahhahhaah I was kiddingggg [12:28:08] ;) [12:28:14] (03Abandoned) 10Elukey: Add ec.wikimedia.org to the pageview whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/320351 (owner: 10Elukey) [12:28:36] (03CR) 10Elukey: [C: 032] Add ec.wikimedia to pageview whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/320376 (owner: 10Joal) [12:28:47] joal: you are free to merge! [12:28:50] lgtm [12:28:56] elukey: ok, doing that, and dpelying [12:29:02] elukey: It'll stop the alarmn [12:29:36] elukey: Thanks mate :) Sorry for not having noticed the gerrit patch in the email -- I should have been more carefull [12:30:05] (03CR) 10Joal: [V: 032] Add ec.wikimedia to pageview whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/320376 (owner: 10Joal) [12:30:56] joal: no problem! I am happy that I got it right [12:33:22] !log Deploying refinery for patching pageview whitelist [12:33:23] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:50:46] PROBLEM - YARN NodeManager Node-State on analytics1039 is CRITICAL: CHECK_NRPE: Socket timeout after 10 seconds. [13:51:46] RECOVERY - YARN NodeManager Node-State on analytics1039 is OK: OK: YARN NodeManager analytics1039.eqiad.wmnet:8041 Node-State: RUNNING [14:16:45] joal: I've got a plumber coming by soon, so we can talk after [14:16:58] but quick question: anything special about creating a hive table on top of your latest output? [14:17:01] sure milimetric [14:17:36] i saw they're parquet files and I got the schema from _common_metadata, but I couldn't make the table read the files properly [14:17:40] milimetric: not really, more on who does what [14:17:49] the columns lined up but i got jibberish [14:17:49] :( [14:17:58] mwarf [14:18:21] i could read it in spark of course, but i was wondering why hive was not cooperating [14:18:33] hm [14:18:41] I might have a nidea [14:18:48] I'll double check [14:19:14] we had a network outage almost for Row D (so a subset of hosts), but a lot of services were impacted like eventbus [14:19:28] (and probably the an1039 alarm was for it) [14:19:41] just as FYI if you see weird errors :) [14:19:44] k [14:19:46] Thanks elukey ! [14:23:17] 06Analytics-Kanban, 10EventBus, 13Patch-For-Review, 06Services (watching), and 3 others: Empty body in EventBus request - https://phabricator.wikimedia.org/T148251#2779769 (10Ottomata) Not a bad idea either! +1 [14:39:04] joal: could it be my mapping of types is wrong? I used "bigint" where I saw "long" and "int" where I saw "integer" [14:39:25] milimetric: That should work [14:40:12] and it's not avro, right, so the create table doesn't need all that row format stuff? [14:40:21] anything special for snappy? [14:40:24] nope [14:40:36] FORMAT parquet should work straight [14:43:13] milimetric: WEhere have you created the table? [14:43:29] joal: milimetric.edit_history [14:43:36] I'm just seeing a lot of nulls, looking more [14:43:40] also milimetric: Having another run after adding user boolean fields for event_user [14:43:49] milimetric: That's expected I guess [14:44:00] oh that explains why I don't have resources to query [14:44:08] no I mean I'm getting all nulls, like every column [14:44:20] That's not expected, right :) [14:45:56] Milimetric: Have you used FORMAT parquet in your table creation ? [14:46:22] I saw lots of different syntax depending on hive version, etc. [14:46:42] I tried ROW FORMAT and now I'm trying STORED AS [14:46:46] https://www.irccloud.com/pastebin/qhUeJPCj/ [14:47:15] oh yay, that seems to have worked [14:47:15] milimetric: Shouldn't be necessary: FORMAT parquet LOCATION /... [14:47:23] that threw an exception [14:47:31] mwarf [14:47:38] STORED AS parquet you're right [14:47:39] sorry [14:48:07] Yay :) [14:48:10] sok, there's no good reference for this [14:48:11] Worked for me ;) [14:48:22] Now data seems correct [14:48:25] yes [14:48:48] one of my biggest complaints about open source in general is that there are like 1000 different places to look for information, all of them outdated and half-wrong [14:49:06] * milimetric misses msdn documentation [15:34:33] 10Analytics-Dashiki, 06Analytics-Kanban, 13Patch-For-Review: Migrate from bower to yarn and clean up folder hierarchy - https://phabricator.wikimedia.org/T147884#2779888 (10Nuria) @TheDJ : FYI that the code chnaged on this ticket is not related to mediawiki stack. [15:38:32] milimetric: do you have some time now ? Branbounce on revision metrics and redirects ? [16:01:23] joal: standdduppppp [16:01:59] 10Analytics-Cluster, 06Analytics-Kanban: Update Kafka MirrorMaker jmxtrans attributes and make MirrorMaker grafana dashboard - https://phabricator.wikimedia.org/T143320#2779936 (10Ottomata) [16:22:08] 06Analytics-Kanban, 10EventBus, 13Patch-For-Review, 06Services (watching), and 3 others: Empty body in EventBus request - https://phabricator.wikimedia.org/T148251#2780000 (10Pchelolo) Maybe with some little prefix to indicate that's base64? like `base64:axssdcasdcascsc`? [16:43:37] 10Analytics-Cluster, 06Analytics-Kanban: Update Kafka MirrorMaker jmxtrans attributes and make MirrorMaker grafana dashboard - https://phabricator.wikimedia.org/T143320#2780068 (10Ottomata) Note: We'll have to deal with underscore separated `typeNames` in the metrics for now. That is, for jmxtrans queries wi... [17:16:36] 10Analytics-EventLogging, 06Performance-Team, 07Performance: EventLogging schema modules take >1s to build (max: 22s) - https://phabricator.wikimedia.org/T150269#2780126 (10Krinkle) [17:16:47] 10Analytics-EventLogging, 06Performance-Team, 07Performance, 07Regression: EventLogging schema modules take >1s to build (max: 22s) - https://phabricator.wikimedia.org/T150269#2780138 (10Krinkle) [17:19:56] going afk team! [17:19:58] o/ [17:25:38] laters! [17:30:12] 10Analytics-EventLogging, 06Performance-Team, 07Performance, 07Regression: EventLogging schema modules take >1s to build (max: 22s) - https://phabricator.wikimedia.org/T150269#2780156 (10Krinkle) [18:13:11] Disconnecting for today a-team - Tomorrow ! [18:13:19] bye joal ! [18:13:32] nite [18:34:11] (03PS1) 10Mforns: Add template for migration to reportupdater+dashiki [analytics/limn-ee-data] - 10https://gerrit.wikimedia.org/r/320433 (https://phabricator.wikimedia.org/T126358) [18:46:03] 10Analytics-Cluster: Update wikitech Kafka documentation - https://phabricator.wikimedia.org/T150277#2780405 (10Ottomata) [18:56:24] 06Analytics-Kanban, 10EventBus, 13Patch-For-Review, 06Services (watching), and 3 others: Empty body in EventBus request - https://phabricator.wikimedia.org/T148251#2780458 (10mobrovac) Or an extra meta field indicating if and which fields have been encoded? [19:00:37] 10Analytics, 10ChangeProp, 10Citoid, 10ContentTranslation-CXserver, and 11 others: Node 6 upgrade planning - https://phabricator.wikimedia.org/T149331#2780484 (10GWicke) Thank you, @KartikMistry! [19:01:41] 10Analytics, 10ChangeProp, 10Citoid, 10ContentTranslation-CXserver, and 12 others: Node 6 upgrade planning - https://phabricator.wikimedia.org/T149331#2780487 (10Gehel) [19:05:05] 10Analytics, 10ChangeProp, 10Citoid, 10ContentTranslation-CXserver, and 12 others: Node 6 upgrade planning - https://phabricator.wikimedia.org/T149331#2780491 (10GWicke) @MoritzMuehlenhoff, it looks like we'll be ready fairly soon. I know that you are planning to be out of the office soon-ish. *Iff* you... [19:32:10] 06Analytics-Kanban, 10EventBus, 13Patch-For-Review, 06Services (watching), and 3 others: Empty body in EventBus request - https://phabricator.wikimedia.org/T148251#2780558 (10Pchelolo) @mobrovac eh... Adding this generic extra field sounds less nice then following the `base64:` prefixing path. Prefixing is... [20:12:23] 06Analytics-Kanban, 10EventBus, 13Patch-For-Review, 06Services (watching), and 3 others: Empty body in EventBus request - https://phabricator.wikimedia.org/T148251#2780751 (10mobrovac) Indeed. Let's just go with the prefix for now. We can implement other strategies at a later point easily enough. [20:34:25] 06Analytics-Kanban, 10EventBus, 13Patch-For-Review, 06Services (watching), and 3 others: Empty body in EventBus request - https://phabricator.wikimedia.org/T148251#2780886 (10Pchelolo) I've update the https://gerrit.wikimedia.org/r/320264 to send a normal data URI instead of the binary data. [21:04:04] 10Analytics, 10MediaWiki-API, 10PageViewInfo, 06Reading-Infrastructure-Team, 13Patch-For-Review: Add pageview stats to the action API - https://phabricator.wikimedia.org/T144865#2781005 (10Tgr) [21:29:27] wikimedia/mediawiki-extensions-EventLogging#617 (wmf/1.29.0-wmf.2 - 41f8b94 : Chad Horohoe): The build has errored. [21:29:27] Change view : https://github.com/wikimedia/mediawiki-extensions-EventLogging/compare/wmf/1.29.0-wmf.2 [21:29:27] Build details : https://travis-ci.org/wikimedia/mediawiki-extensions-EventLogging/builds/174323224 [21:52:54] 10Analytics-Cluster, 06Analytics-Kanban, 13Patch-For-Review: Update Kafka MirrorMaker jmxtrans attributes and make MirrorMaker grafana dashboard - https://phabricator.wikimedia.org/T143320#2781234 (10Ottomata) yes! https://grafana.wikimedia.org/dashboard/db/kafka-mirrormaker [22:19:28] latesr all!