[03:09:23] log [07:23:52] hello oozie! I know I know you are complaining, let me fix it.. [07:25:26] kbp.wikipedia [07:30:00] (03PS1) 10Elukey: Add kbp.wikipedia to the pageview whitelist [analytics/refinery] - 10https://gerrit.wikimedia.org/r/361033 [07:32:19] aaand uploaded [07:32:35] !log uploaded new pageview whitelist following https://wikitech.wikimedia.org/wiki/Analytics/Team/Oncall#Find_and_fix_pageview_whitelist_exceptions for kbp.wikipedia [07:32:36] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [10:21:59] Hi a-team - Lino is sick, I'll on and off today [10:22:12] joal: take care of Lino! [10:24:18] elukey: quick question, have you added the kbp project to the whitelist manually, or only created the patch? [10:24:50] joal: logged in the chan, I followed https://wikitech.wikimedia.org/wiki/Analytics/Team/Oncall#Find_and_fix_pageview_whitelist_exceptions [10:24:56] Oh sorry elukey - Just saw your email [10:25:09] super :) [10:34:36] * elukey lunch! [12:32:37] (03CR) 10Joal: [V: 032 C: 032] "LGTM ! Merging" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/361033 (owner: 10Elukey) [13:13:10] fdans: there's still some gremlin in my build... [13:13:12] I don't get it [13:14:21] I feel incredibly unproductive working on this, I might switch to the components so I can make some progress and hopefully you can help me with the config [13:14:35] milimetric: You shouldn't have used water on that build stuff ... [13:14:56] aaah! They're activated by water, of course! [13:15:51] milimetric: sure [13:15:51] because it feels just like that - I put out one fire with some water and five more spawn [13:16:04] there's just so many moving parts [13:16:10] Then indeed gremlins was the correct reference :) [13:53:45] 10Analytics, 10DBA: Purge all old data from master - https://phabricator.wikimedia.org/T168414#3374749 (10elukey) Just added to the eventlogging_cleaner script the `--no-whitelist` parameter to support this use case. [14:12:43] 10Analytics-Kanban, 10User-Elukey: dbstore1002 /srv filling up - https://phabricator.wikimedia.org/T168303#3360724 (10elukey) IIUC `du -hsc _log_*` should list the space consumption of the `log` database tables, that is around `1.3T`. This slave will be a target of T156933 so the `eventlogging_cleaner` script... [14:14:18] (03PS2) 10Joal: Add two tables to sqoop on hadoop [analytics/refinery] - 10https://gerrit.wikimedia.org/r/360866 [14:35:43] (03PS18) 10Mforns: UDF to tag requests [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/353287 (https://phabricator.wikimedia.org/T164021) (owner: 10Nuria) [14:41:22] 10Analytics-Kanban, 10Analytics-Wikistats: Implement Wiki Selector Widget - https://phabricator.wikimedia.org/T167675#3374923 (10Milimetric) a:03Milimetric [14:57:51] a-team, I have to attend a delivery in the door, will be 5 minutes late to su [15:01:14] ping mforns [15:24:28] 10Analytics-Kanban: Rename unique_devices_project_wide to unique_devices_per_project_family - https://phabricator.wikimedia.org/T168402#3375008 (10JAllemandou) [15:52:28] fyi, EL spike makes sense: https://grafana-admin.wikimedia.org/dashboard/db/kafka-by-topic?from=1498052342866&to=1498230812170&refresh=5m&orgId=1&var-cluster=analytics-eqiad&var-kafka_brokers=All&var-topic=eventlogging_QuickSurveyInitiation&var-topic=eventlogging_QuickSurveyInstance&var-topic=eventlogging_QuickSurveysResponses [15:52:36] dunno why that doesn't show up on the EL dashboard topics [16:04:36] 10Analytics-Kanban, 10DBA: Purge all old data from master - https://phabricator.wikimedia.org/T168414#3375092 (10Nuria) [16:04:56] ottomata: have a meeting but ready to talk eventlogging flow right after cc mforns [16:05:47] eventlogging flow? [16:06:40] ottomata: eventlogging flow of events being bigger than before taht is [16:06:43] *that [16:09:22] oh, its ok nuria_ it makes sense [16:09:29] its the research survey stuff [16:10:45] ottomata: mmm...but why jump does not show in eventlogging by schema [16:11:29] ah, that i don't know [16:14:02] hey nuria_ ottomata sorry missed the ping [16:14:53] ottomata, I think I know why it does not show... [16:15:02] i agree it is the QuickSurveyInitiation_15278946 [16:15:13] I think metrics that do not span the whole selected datetime range, do not show [16:15:23] if you select last 20 minutes, it shows! [16:16:15] oh haha, ok [16:16:24] and it's huge: 400/min [16:17:10] and that is the rate per second average, so 400/second actually [16:18:36] yeah its huge for EL for sure [16:18:51] joal: do you remember what job writes the '_IMPORTED' flag to webrequest raw? [16:18:54] is it camus? [16:19:19] ottomata: camus checker I think [16:19:26] AHHHH [16:19:28] yes yes [16:19:28] ok thanks [16:19:37] i'm thikning about El refine job flags :) [16:19:52] 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Improve purging for analytics-slave data on Eventlogging - https://phabricator.wikimedia.org/T156933#3375118 (10elukey) Thanks a lot for all the suggestions and feedback. Marcel changed a bit the logic that we were using, replacing LIMIT OFFSET with so... [16:22:03] ottomata: Possibly same deal :) [16:23:26] mforns: just noted all the changes to the script todo, will be ready on monday [16:23:37] going afk now team! Have a good weekend :) [16:31:37] Bye lads Q! [16:32:54] bye! [16:53:51] Hrmm, I asked in another channel but just in case anyone from analytics is still about [16:53:59] im about to install stat1005, the gpu host replacing stat1002 [16:54:07] was wondering if it can be debian strech, or if it needs jessie still? [17:07:21] 10Analytics, 10Analytics-Cluster, 10Operations, 10ops-eqiad, 10Patch-For-Review: rack/setup/install replacement to stat1005 (stat1002 replacement) - https://phabricator.wikimedia.org/T165368#3375220 (10RobH) [17:09:20] nuria_, ping me when/if you can talk about webrequest tagging :] [17:30:34] 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Improve purging for analytics-slave data on Eventlogging - https://phabricator.wikimedia.org/T156933#3375278 (10Volans) @elukey thanks for the update. Which one is the final change to be reviewed then? [17:34:03] 10Analytics, 10Analytics-Cluster, 10Operations: rack/setup/install replacement to stat1005 (stat1002 replacement) - https://phabricator.wikimedia.org/T165368#3375285 (10RobH) p:05Low>03Normal a:05RobH>03Ottomata [17:34:57] 10Analytics, 10Analytics-Cluster, 10Operations: rack/setup/install replacement to stat1005 (stat1002 replacement) - https://phabricator.wikimedia.org/T165368#3264256 (10RobH) System is installed, puppet/salt accepted, and ready to have software/services implemented. Assigning to @Ottomata for implementation. [17:35:33] 10Analytics, 10Analytics-Cluster, 10Operations: rack/setup/install replacement to stat1005 (stat1002 replacement) - https://phabricator.wikimedia.org/T165368#3375291 (10RobH) a:05Ottomata>03RobH Turns out needs jessie, taking back for reimage. [17:38:47] mforns: yt? [17:38:53] nuria_, yep [17:39:02] mforns: ok, let me look at latest chnages [17:39:07] k [17:45:03] mforns: ok, looked at changes about executionStage [17:45:10] aha [17:46:14] so, you think the changes are needed? [17:47:57] mforns: I think we need order if subsequent tags need to use a prior tag yes [17:50:49] nuria_, do you think adding the executionStage is a good solution? [17:51:10] the other option would be not using reflection I guess... [17:51:13] 10Analytics, 10Analytics-Cluster, 10Operations, 10Patch-For-Review: rack/setup/install replacement to stat1005 (stat1002 replacement) - https://phabricator.wikimedia.org/T165368#3264256 (10ops-monitoring-bot) Script wmf_auto_reimage was launched by robh on neodymium.eqiad.wmnet for hosts: ``` ['stat1005.eq... [17:51:20] or not using tagAccumulator [17:51:30] mforns: no, you need both [17:52:09] mforns: to have a tag that says 'it is a pageview and also an apps request', right? [17:52:19] yes [17:52:20] mforns: w/o accumulator there is no way to know [17:52:24] yea [17:52:49] mforns: let me see if there is a way to build the chain that does not require two passes though [17:53:08] nuria_, you mean 2 loops? [17:53:14] mforns: yes [17:53:57] I tried to have a single list, ordered by executionStage, but... java didn't let me :'( [17:54:39] mforns: ya, you need to use a comparator [17:54:43] the thing is the value of executionStage is stored in the Tag class [17:54:54] not only a comparator [17:55:37] you need to extract the executionStage value from the Tag class and then attach it to the Tagger class... [17:55:55] I started implementing a list of pairs, with a comparator [17:56:04] but maaaan... it's too much code for a simple thing [17:56:45] mforns: aha [17:57:35] mforns: let me give it a try but to your question yes, i think that for adding a dependency among tags this is the easiest way> did you talked about it with joal? [17:57:36] that's why I thought that a double loop wouldn't be so bad, especially because it's going to have like very few elements in both levels [17:57:47] mforns: ya, perf wise is no big deal [17:59:12] nuria_, yea Joseph commented on Gerrit, he things the executionStage is less ugly (more maintainable) than order [totally agree] [17:59:21] but he still prefers not using reflection [17:59:26] instead [17:59:39] and hardcode the order in the TaggerChain [17:59:47] 2 mins, please [18:01:13] ya, that prevents us from having tags in packages that are not ours and adds a configuration step, i prefer convention over configuration [18:03:43] fine by me :] [18:16:47] 10Analytics, 10Analytics-Cluster, 10Operations: rack/setup/install replacement stat1006 (stat1003 replacement) - https://phabricator.wikimedia.org/T165366#3375411 (10RobH) p:05Low>03Normal [18:22:27] mforns: let me know if you disagree [18:29:44] nuria_, I'm ok with both... I think none of them is super good: one because it needs configuration, the other because it needs some knowledge of a weird parameter, but I can not see another solution, so I'm fine with how it is now, and also if we were to not use reflection [18:37:19] 10Analytics, 10Analytics-Cluster, 10Operations, 10Patch-For-Review: rack/setup/install replacement to stat1005 (stat1002 replacement) - https://phabricator.wikimedia.org/T165368#3375431 (10ops-monitoring-bot) Completed auto-reimage of hosts: ``` ['stat1005.eqiad.wmnet'] ``` and were **ALL** successful. [18:41:12] 10Analytics: Index integer fields as such for webrequest data in Pivot - https://phabricator.wikimedia.org/T168742#3375438 (10Krinkle) [18:41:43] 10Analytics-Kanban, 10Patch-For-Review: Load webrequest raw data into druid so ops can use it for troubleshooting - https://phabricator.wikimedia.org/T166967#3375451 (10Krinkle) Follow-up tasks: * {T167907} * {T168742} [18:50:12] 10Analytics, 10Analytics-Cluster, 10Operations, 10Patch-For-Review: rack/setup/install replacement to stat1005 (stat1002 replacement) - https://phabricator.wikimedia.org/T165368#3375476 (10RobH) a:05RobH>03Ottomata reimage complete, system ready for you to add to site.pp for specific roles! [18:56:06] 10Analytics, 10Analytics-Cluster, 10Operations: rack/setup/install replacement stat1006 (stat1003 replacement) - https://phabricator.wikimedia.org/T165366#3375499 (10RobH) [19:06:45] 10Analytics-Kanban: Some fields in Pivot should be numbers - https://phabricator.wikimedia.org/T167494#3375541 (10JAllemandou) [19:06:48] 10Analytics: Index integer fields as such for webrequest data in Pivot - https://phabricator.wikimedia.org/T168742#3375546 (10JAllemandou) [19:08:01] 10Analytics-Kanban, 10Patch-For-Review: Load webrequest raw data into druid so ops can use it for troubleshooting - https://phabricator.wikimedia.org/T166967#3375565 (10JAllemandou) >>! In T166967#3375451, @Krinkle wrote: > Follow-up tasks: > * {T167907} > * {T167494} [19:49:34] joal: are you still around? [20:29:39] (03PS26) 10Ottomata: EventLogging JSON -> Hive [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/346291 (https://phabricator.wikimedia.org/T161924) (owner: 10Joal) [20:30:37] (03CR) 10Ottomata: "Woot, I added handling of doneFlags. Because this job handles many partitions at a time, it needs to be responsible for writing out doneF" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/346291 (https://phabricator.wikimedia.org/T161924) (owner: 10Joal) [21:46:18] 10Analytics, 10Analytics-Cluster, 10Operations: rack/setup/install replacement stat1006 (stat1003 replacement) - https://phabricator.wikimedia.org/T165366#3375927 (10RobH) a:05RobH>03Ottomata [21:46:38] 10Analytics, 10Analytics-Cluster, 10Operations: rack/setup/install replacement stat1006 (stat1003 replacement) - https://phabricator.wikimedia.org/T165366#3264224 (10RobH) Assigned to @Ottomata for implementation.