[03:57:11] 10Analytics-Kanban: Use native timestamp types in Data Lake edit data - https://phabricator.wikimedia.org/T161150#3505028 (10Nuria) >That's a shame! Do you have a general sense of when we might upgrade to 1.2? 3 months? A year? Never? Cloudera needs to provide a distro that includes this version of hive, I think... [04:01:42] 10Analytics-Kanban, 10Analytics-Wikistats, 10Continuous-Integration-Infrastructure, 10Release-Engineering-Team (Kanban): Fix Wikistats build in Jenkins - https://phabricator.wikimedia.org/T171599#3505034 (10Nuria) [04:07:06] (03CR) 10Nuria: "+1" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/370320 (owner: 10Milimetric) [04:22:58] 10Analytics-Kanban, 10Analytics-Wikistats: Add piwik to wikistats 2.0 site - https://phabricator.wikimedia.org/T171642#3505042 (10Nuria) It can be added just like it is so on analytics.wikimedia.org. See: https://github.com/wikimedia/analytics.wikimedia.org/blob/master/index.html#L11 Snippet for tracking code... [07:13:45] 10Analytics, 10Contributors-Analysis, 10DBA, 10Chinese-Sites, 10Patch-For-Review: Data Lake edit data missing for many wikis - https://phabricator.wikimedia.org/T165233#3505101 (10Marostegui) Thanks for confirming @Milimetric - I have now fixed all the wikis listed at: T165233#3498662. Please try again a... [07:29:30] 10Analytics, 10Operations, 10ops-eqiad, 10User-Elukey: Analytics1034 eth0 negotiated speed to 100Mb/s instead of 1000Mb/s - https://phabricator.wikimedia.org/T172633#3505104 (10elukey) Tried this: * ifdown eth0 * modprobe -r tg3 * modprobe tg3 * ifup eth0 ``` [Mon Aug 7 07:28:13 2017] pps_core: LinuxPPS... [10:14:45] 10Analytics-Kanban, 10Operations, 10User-Elukey: Analytics Kafka cluster causing timeouts to Varnishkafka since July 28th - https://phabricator.wikimedia.org/T172681#3505470 (10elukey) [10:18:58] 10Analytics-Kanban, 10Operations, 10User-Elukey: Analytics Kafka cluster causing timeouts to Varnishkafka since July 28th - https://phabricator.wikimedia.org/T172681#3505486 (10elukey) [10:21:54] 10Analytics, 10Community-Liaisons, 10User-Johan: Collect information about how we collect user statistics in one place - https://phabricator.wikimedia.org/T132405#3505489 (10Qgil) Is there any team or anyone asking CLs to work on this? [10:23:03] 10Analytics-Kanban, 10Operations, 10User-Elukey: Analytics Kafka cluster causing timeouts to Varnishkafka since July 28th - https://phabricator.wikimedia.org/T172681#3505490 (10elukey) [10:37:32] * elukey lunch! [11:02:07] 10Analytics, 10User-Johan: Collect information about how we collect user statistics in one place - https://phabricator.wikimedia.org/T132405#3505554 (10Johan) Not as far as I know. [11:26:09] hellooooo team [11:27:58] hey elukey you around? [11:28:06] milimetric: o/ [11:28:24] cool, so I'm trying to figure out how to read more logs about "oozie job -log 0060876-170621131133576-oozie-oozi-C" [11:28:34] it failed and I don't know why [11:28:51] the error in hue is nondescript spark: https://hue.wikimedia.org/oozie/list_oozie_workflow/0060877-170621131133576-oozie-oozi-W/ [11:29:02] I reran it and that didn't make a difference, same error [11:31:15] milimetric: I'd try with elukey@stat1004:~$ sudo -u hdfs yarn logs -applicationId application_1498042433999_165979 [11:31:32] that's what I was looking for! [11:31:54] \o/ [11:32:15] now I have a question for you people (whenever you have time) [11:32:40] varnishkafka has been throwing some timeouts since July 28th at ~ 18 UTC [11:32:56] and we are likely loosing a bit of data everyday [11:33:10] it correlates with an error that I found in kafka [11:33:13] but really weird to debug [11:33:26] anything that changed around that time that you remeber? Maybe Event Steams? [11:33:40] task is https://phabricator.wikimedia.org/T172681 [11:41:16] 10Analytics, 10Contributors-Analysis, 10DBA, 10Chinese-Sites, 10Patch-For-Review: Data Lake edit data missing for many wikis - https://phabricator.wikimedia.org/T165233#3505597 (10Marostegui) @Milicevic01 @JAllemandou @Nuria we have finished importing all the production shards into the new labs infra: T1... [11:42:06] this is the relevant set of commits in puppet, elukey, I'll browse through them to see if I remember anything: https://github.com/wikimedia/puppet/commits/production?after=d84b9c58a3d86f824f1d2a662d95f98c4f086a1b+174 [11:44:13] already checked those ones :) [11:44:21] I don't really see anything except a pattern that cp40** boxes were tweaked a bit (ulsfo), so maybe is it only limited to ulsfo? [11:47:54] nope https://grafana.wikimedia.org/dashboard/db/varnishkafka?panelId=20&fullscreen&orgId=1&from=now-30d&to=now [12:01:16] nice graph, I see the problem, but yeah, I can't remember anything [12:13:52] we definitely need to add monitoring after this issue, oozie alone is not enough [12:44:12] milimetric, hey do you have 10 mins to help me with router code? :] [12:51:16] hi mforns yea, goin to the cave [12:51:22] computer crashed, would've been there sooner [12:51:28] hey milimetric cool, omw! [13:32:17] 10Analytics, 10Analytics-Wikistats: Unexpected increase in traffic for 4 languages in same region, on smaller projects - https://phabricator.wikimedia.org/T136084#3505786 (10Liuxinyu970226) [14:15:47] milimetric, also another question about style, 2-space tabs or 4-space tabs? [14:28:28] definitely 4-space mforns [14:28:30] :) [14:28:34] k [14:28:55] will change the files I'm editing that are 2-space [14:57:50] (03CR) 10Nuria: "We can abandon this change now that all wikis are in the labs snapshot correct?" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/370322 (owner: 10Milimetric) [15:01:30] (03CR) 10Milimetric: "No, we may want to run _private snapshots from time to time for various reasons, including to double-check the labs snapshots. I think th" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/370322 (owner: 10Milimetric) [15:14:04] nuria_, joal, milimetric, mforns, fdans, elukey: we have a new research scientist on board. please meet dsaezt, joining us from Barcelona (he will work remotely from there). For those of you who will be in Wikimania, you will meet him there. dsaezt: We work very closely with analytics. the bot detection question was theirs. ;) (and, I think ottomata is on vacation. you will see him around when he's back.) [15:15:38] Hi all! :) I'm super happy to be here! [15:16:48] \o/ hi! [15:43:30] dsaezt: nice to met you [15:43:53] Hi nuria! [15:45:06] @dsaezt: yayyyyyy new member of teeeeam Europe!! [15:45:17] @dsaezt: I work from Madrid :) [15:45:49] (Will be 5min late for tasking a-team, moving over to apartment) [15:49:08] 10Analytics-Kanban, 10Operations, 10User-Elukey: Analytics Kafka cluster causing timeouts to Varnishkafka since July 28th - https://phabricator.wikimedia.org/T172681#3506357 (10elukey) From https://apache.googlesource.com/kafka/+/refs/heads/trunk/clients/src/main/java/org/apache/kafka/common/protocol/ApiKeys... [15:51:35] ping fdans [15:51:44] ah sorry, did not see your note [15:54:45] 10Analytics: Weird performance of sqoop job - https://phabricator.wikimedia.org/T172579#3502684 (10Nuria) See: https://phabricator.wikimedia.org/T172633 one of our nodes was slow due to bad network card(?) [15:56:50] 10Analytics-Kanban, 10Operations, 10ops-eqiad, 10User-Elukey: Analytics1034 eth0 negotiated speed to 100Mb/s instead of 1000Mb/s - https://phabricator.wikimedia.org/T172633#3506395 (10Nuria) [16:03:56] 10Analytics, 10Analytics-Wikistats: Unexpected increase in traffic for 4 languages in same region, on smaller projects - https://phabricator.wikimedia.org/T136084#3506422 (10Nuria) [16:03:59] 10Analytics, 10Research: [Open question] Improve bot identification at scale - https://phabricator.wikimedia.org/T138207#3506421 (10Nuria) [16:08:23] 10Analytics, 10Operations, 10Research: Phase out and replace analytics-store (multisource) - https://phabricator.wikimedia.org/T172410#3497641 (10Nuria) Not sure what do we need to do here. What is on analytics store apart from eventlogging and mediawiki databases? [16:11:25] 10Analytics, 10Android-app-feature-Feeds, 10Mobile-Content-Service, 10Pageviews-API, and 4 others: Why top views data of different sources is not the same? - https://phabricator.wikimedia.org/T172379#3496756 (10Nuria) I think @bearND explained it pretty well, data from pageview API is subjected to fluctua... [16:11:34] 10Analytics, 10Android-app-feature-Feeds, 10Mobile-Content-Service, 10Pageviews-API, and 4 others: Why top views data of different sources is not the same? - https://phabricator.wikimedia.org/T172379#3506449 (10Nuria) 05Open>03Resolved [16:23:23] 10Analytics-Kanban, 10RESTBase-API, 10WMF-Legal, 10Patch-For-Review, 10Services (watching): License for pageview data - https://phabricator.wikimedia.org/T170602#3506480 (10Nuria) a:05mforns>03Nuria [16:24:42] fdans, do you have 10 mins to look at routing? [16:25:44] mforns: sure! [16:25:55] a la baticueva! [16:26:01] ok! :] [17:21:37] * elukey off! [18:20:26] 10Analytics, 10Operations, 10Research: Phase out and replace analytics-store (multisource) - https://phabricator.wikimedia.org/T172410#3507261 (10Halfak) Looks like @jcrespo wants to phase out an analytics/dba maintained resource. I guess I'd expect analytics to lead the process of phasing that out. [18:35:17] 10Analytics, 10Contributors-Analysis, 10DBA, 10Chinese-Sites, 10Patch-For-Review: Data Lake edit data missing for many wikis - https://phabricator.wikimedia.org/T165233#3507361 (10Neil_P._Quinn_WMF) >>! In T165233#3503803, @Milimetric wrote: > No rush but just a heads up to @Neil_P._Quinn_WMF, the 2017-0... [19:41:19] 10Analytics, 10Contributors-Analysis, 10DBA, 10Chinese-Sites, 10Patch-For-Review: Data Lake edit data missing for many wikis - https://phabricator.wikimedia.org/T165233#3507677 (10Milimetric) I think I worked out the bugs, should be ready soon unless something else goes wrong. [20:34:38] 10Analytics-Kanban, 10RESTBase-API, 10WMF-Legal, 10Patch-For-Review, 10Services (watching): License for pageview data - https://phabricator.wikimedia.org/T170602#3508063 (10GWicke) a:05Nuria>03GWicke [The PR](https://github.com/wikimedia/restbase/pull/848) is now updated in line with the discussion. [20:51:23] gwicke: I think there are couple sentences that might still need rewording (re:license), commented on PR [20:56:50] gwicke: it could be that you talked to zhou offline about those if so, disregard [20:57:38] we chatted about it for a bit last week [21:36:31] gwicke: ok, then let's merge [21:36:45] gwicke: i do not haver permits to merge right? [21:37:39] gwicke: How did these changes got deployed again? [21:48:44] 10Analytics-Kanban, 10RESTBase-API, 10WMF-Legal, 10Patch-For-Review, 10Services (done): License for pageview data - https://phabricator.wikimedia.org/T170602#3508224 (10mobrovac) 05Open>03Resolved >>! In T170602#3508063, @GWicke wrote: > [The PR](https://github.com/wikimedia/restbase/pull/848) is now... [23:24:30] 10Analytics-Kanban: Use native timestamp types in Data Lake edit data - https://phabricator.wikimedia.org/T161150#3508429 (10Neil_P._Quinn_WMF) Thank you, good to know! Would it make sense to keep this open and stalled while we're waiting? It would put my mind at ease although of course it's not a big deal. [23:54:58] 10Analytics, 10Analytics-Wikistats, 10Operations, 10Wikidata, and 6 others: Create Wikiversity Hindi - https://phabricator.wikimedia.org/T168765#3508516 (10Jayprakash12345)