[03:36:10] 10Analytics, 10Analytics-Wikistats: Reasons for decline and deletion on Wikinews - https://phabricator.wikimedia.org/T203090 (10Tbayer) Phabricator is indeed not the best venue to ask for assistance with research methodology. Here are some suggestions instead: 1. Create a research project page on Meta-wiki as... [05:57:35] 10Analytics, 10Operations, 10ops-eqiad, 10User-Elukey: rack/setup/install analytics-master100[12].eqiad.wmnet - https://phabricator.wikimedia.org/T201939 (10elukey) [06:05:24] !log re-run virtualpageview-hourly-wf-2018-9-2-1 via Hue (failed oozie job to inspect: 0082016-180705103628398-oozie-oozi-W) [06:05:26] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [06:07:07] The geowiki failed job seems a more sneaky issue [06:07:08] org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Expected 1 components, got 2 (month=2018-08/wiki_db=abwiki)) [07:35:45] Hi team - Go banned this weekend [07:38:11] Go banned this weekend? Can't tell if that is a typo, reference that I don't get, or it has words missing, or I just don't get it :D << joal [07:45:11] s/go/got/ - Sorry, didn't notice the type :) [07:45:17] I got banned this weekend [07:45:27] s/type/typo [07:45:38] Will probably stop typing on keyboard for today .... [07:53:11] :) [08:06:11] elukey: problem with geoeditor-load job is the same as before: inconsistent partition-paths for hive repair-table [08:07:32] Namely, folder /wmf/data/raw/mediawiki_private/tables/cu_changes/month=2018-05 doesn't contain any wiki_db=.. partition folder, while it is expected to have some [08:07:57] !log Delete /wmf/data/raw/mediawiki_private/tables/cu_changes/month=2018-05 folder and relaunch mediawiki-geoeditors-load-wf-2018-08 [08:07:58] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [08:11:48] ahhh the thing about directories cleaned up without the wiki_db subdir [08:12:00] correct elukey [08:12:03] completely forgot about it [08:12:21] oozie job sucessful [08:16:14] nice :) [08:24:31] I requested more resources for the analytics labs project [08:46:54] joal: do you know if the 'sniny' labs instance in our analytics project is used? [08:47:08] afaics it was created in 2015 for a Rstudio thing [08:47:11] but nothing on it [08:47:16] (I am doing cleanups) [08:47:36] * joal hasn't used R for a decade [08:47:39] I have no idea [08:48:05] it also runs ubuntu [08:48:08] ahahhah [10:48:59] * elukey lunch! [13:34:47] 10Analytics: Sqoop wbc_entity_usage from all wikis into hadoop (HDFS) - https://phabricator.wikimedia.org/T167290 (10GoranSMilovanovic) 05Open>03Resolved a:03GoranSMilovanovic [13:41:39] joal: o/ [13:41:40] elukey@analytics1003:~$ sudo systemctl list-timers [13:41:40] NEXT LEFT LAST PASSED UNIT ACTIVATES [13:41:43] Tue 2018-09-04 09:14:47 UTC 19h left Mon 2018-09-03 09:14:47 UTC 4h 26min ago systemd-tmpfiles-clean.timer systemd-tmpfiles-clean.service [13:41:46] Tue 2018-09-04 10:00:00 UTC 20h left n/a n/a check_webrequest_partitions.timer check_webrequest_partitions.service [13:41:49] ---^ [14:09:27] (03CR) 10Fdans: Makes closed sites searchable and disables private sites (031 comment) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/456612 (https://phabricator.wikimedia.org/T203105) (owner: 10Fdans) [14:11:02] (03PS2) 10Fdans: Makes closed sites searchable and disables private sites [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/456612 (https://phabricator.wikimedia.org/T203105) [14:16:44] (03PS7) 10Fdans: Add strategy, usability and advisory sites to pageview definition [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/456022 (https://phabricator.wikimedia.org/T187414) [14:18:53] (03CR) 10jerkins-bot: [V: 04-1] Add strategy, usability and advisory sites to pageview definition [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/456022 (https://phabricator.wikimedia.org/T187414) (owner: 10Fdans) [15:20:59] Pchelolo: Hi Pietr - thanks a lot for the merge - Any idea on deploy timing? [15:21:47] 10Analytics: Wikistats: add functions you apply to dimensional data such as "accumulate" - https://phabricator.wikimedia.org/T203180 (10fdans) p:05Triage>03Normal [15:24:05] 10Analytics, 10Operations, 10ops-eqiad: analytics1068 doesn't boot - https://phabricator.wikimedia.org/T203244 (10fdans) p:05Normal>03Triage [15:24:11] 10Analytics, 10Operations, 10ops-eqiad: analytics1068 doesn't boot - https://phabricator.wikimedia.org/T203244 (10fdans) p:05Triage>03Normal [15:30:28] (03CR) 10Nuria: "Please note that tests are failing." [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/456022 (https://phabricator.wikimedia.org/T187414) (owner: 10Fdans) [15:31:51] (03CR) 10Nuria: [V: 032 C: 032] Update WKS2 endpoints to accept project families [analytics/aqs] - 10https://gerrit.wikimedia.org/r/456442 (https://phabricator.wikimedia.org/T203258) (owner: 10Joal) [15:40:00] (03CR) 10Nuria: [C: 032] Makes closed sites searchable and disables private sites [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/456612 (https://phabricator.wikimedia.org/T203105) (owner: 10Fdans) [15:40:12] (03CR) 10Nuria: [C: 032] "Added FAQ entry: https://wikitech.wikimedia.org/wiki/Analytics/Systems/Wikistats2/Metrics/FAQ#The_wiki_I_am_interested_on_does_not_appear_" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/456612 (https://phabricator.wikimedia.org/T203105) (owner: 10Fdans) [15:40:14] (03CR) 10Nuria: [V: 032 C: 032] Makes closed sites searchable and disables private sites [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/456612 (https://phabricator.wikimedia.org/T203105) (owner: 10Fdans) [15:41:51] fdans: wasn't the code that fixed the partition issue when dropping merged a bit ago? [15:42:15] cc joal [15:42:22] fdans: thsi one: https://phabricator.wikimedia.org/T198600 [15:42:23] it has been nuria [15:42:43] joal: but we did not deploy yet , right? [15:42:51] we did deploy [15:43:00] hm - Or at least I thought we had [15:43:07] (03Merged) 10jenkins-bot: Makes closed sites searchable and disables private sites [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/456612 (https://phabricator.wikimedia.org/T203105) (owner: 10Fdans) [15:43:19] joal: we must have not for having gotten error about geowiki crons right? [15:43:29] joal: let me check [15:43:40] nuria: depends when the cron is executed [15:46:38] joal: we deployed cluster on 29th though [15:46:43] joal: right? [15:47:32] joal: per https://wikitech.wikimedia.org/wiki/Server_Admin_Log [15:48:51] nuria: from hdfs crontab on analytics1003 - Those commands are executed on the 16th [15:49:06] nuria: this explains that [15:49:37] joal: ayayay, me no compredou, but elukey sent e-mail today right? For mediawiki-geoeditors-load-coord [15:50:00] joal: ah i see [15:50:06] joal: the loading happen today [15:50:14] joal: the dropping happens on teh 16th [15:50:18] deletion happens on the 16th of the month [15:50:25] yes m'dame [15:50:41] joal: my mistake, i assumed deletion was happening daily [15:51:02] no problemo - I was pretty sure it was that, but it's good we double hecked [15:59:44] 10Analytics-Kanban: Correct end point documentation about parameter specification - https://phabricator.wikimedia.org/T203403 (10Nuria) [16:01:04] (03PS2) 10Nuria: Correct end-point documentation [analytics/aqs] - 10https://gerrit.wikimedia.org/r/456573 (https://phabricator.wikimedia.org/T203403) (owner: 10Joal) [16:01:57] 10Analytics-Kanban: Correct end point documentation about parameter specification - https://phabricator.wikimedia.org/T203403 (10Nuria) https://gerrit.wikimedia.org/r/#/c/456573/ [16:35:06] joal: it's wmf holiday, so tomorrow [16:55:03] 10Analytics, 10Patch-For-Review, 10User-Elukey: Refactor analytics cronjobs to alarm on failure reliably - https://phabricator.wikimedia.org/T172532 (10elukey) [16:59:07] 10Analytics, 10Patch-For-Review, 10User-Elukey: Refactor analytics cronjobs to alarm on failure reliably - https://phabricator.wikimedia.org/T172532 (10elukey) So today I was able to add two systemd timers to analytics1003: ``` elukey@analytics1003:~$ sudo systemctl list-timers NEXT... [17:00:09] interesting --^ [17:00:18] timers seems a good step forward from crons [17:00:28] need to play a bit more with them, especially alarming [17:01:44] * elukey off for today! [17:34:38] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Reading-analysis: Assess impact of ua-parser update on core metrics - https://phabricator.wikimedia.org/T193578 (10fdans) [20:11:06] Hallo [20:11:34] With the help of milimetric I got this file created with report updater: https://analytics.wikimedia.org/datasets/periodic/reports/metrics/published_cx2_translations/ [20:11:42] Now I want to show it on a Dashiki chart. [20:11:52] I created this page: https://meta.wikimedia.org/wiki/Config:Dashiki:CX2Translations [20:12:16] I'm not entirely sure that it's all correct and I'd appreciate a review. [20:12:35] And what do I have to do so that it would actually show up as a chart? [20:12:49] Similarly to https://language-reportcard.wmflabs.org/interlanguage/ , for example? [23:31:22] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Research: 20K events by a single user in the span of 20 mins - https://phabricator.wikimedia.org/T202539 (10Nuria) 05Resolved>03stalled [23:42:06] 10Analytics-Kanban, 10Mobile-Content-Service, 10Reading-Infrastructure-Team-Backlog, 10Wikipedia-Android-App-Backlog, and 2 others: [Feed] Establish criteria for blacklisting likely bot-inflated most-read articles - https://phabricator.wikimedia.org/T143990 (10Nuria) Picking this up again and actually i ta... [23:46:04] 10Analytics, 10Research: [Open question] Improve bot identification at scale - https://phabricator.wikimedia.org/T138207 (10Nuria) [23:46:07] 10Analytics-Kanban, 10Patch-For-Review: Eventlogging's processors stopped working - https://phabricator.wikimedia.org/T200630 (10Nuria)