[06:49:23] 10Quarry, 10Community-Wikimetrics, 10DBA, 10Icinga, and 2 others: Evaluate future of wmf puppet module "mysql" - https://phabricator.wikimedia.org/T165625#3362051 (10jcrespo) [06:59:45] gooood morning [07:00:11] still need coffee but I suspect that our datanode partitions on several hadoop nodes are filling up [07:01:57] https://grafana.wikimedia.org/dashboard/db/analytics-hadoop?panelId=25&fullscreen&orgId=1&from=now-90d&to=now [07:02:13] hdfs space used has been increasing steadily during the past three months [07:04:46] (03CR) 10DCausse: [C: 031] [cirrus] Distinguish morelike vs fulltext api search requests [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/345863 (owner: 10DCausse) [07:05:58] joal_: my dear friend I hope that you are working today :D [07:26:15] elukey@stat1002:/mnt/hdfs$ sudo -u hdfs hdfs dfs -du -h / [07:26:15] 25 768 M /system [07:26:15] 1.8 T 5.4 T /tmp [07:26:15] 107.3 T 321.2 T /user [07:26:15] 16.6 T 49.7 T /var [07:26:17] 412.9 T 1.2 P /wmf [07:26:42] in /user there is definitely some easy garbage to trash [07:27:58] from the graph it is interesting to see that without the new nodes we'd have finished the hdfs free space a long time ago :D [07:33:49] (afk for a bit to commute) [07:33:56] Hi elukey [07:34:03] I'm working indeed today [07:34:14] I have some explanations for HDFS filling up :) [07:56:01] elukey: I think we stopped webrequest removal job a while a go and forgot to restart it [07:56:59] elukey: sudo -u hdfs hdfs dfs -du -s -h /wmf/data/wmf/webrequest/webrequest_source=text/*/* [07:57:28] elukey: sudo -u hdfs hdfs dfs -du -s -h /wmf/data/raw/webrequest/webrequest_text/hourly/*/* [07:57:58] I have double checked users usage - I have some recommandations for some users, but nothing major [07:58:29] The real thing is making sure we delete our old data (for space, sure, and more importantly for compliance) [07:59:22] hellooooo [07:59:23] I am back [07:59:24] :D [07:59:26] Heya [07:59:40] Thanks for having noticed the raise in space usage elukey [08:00:01] we definitely need an alarm based on those metrics [08:00:05] correct [08:00:23] elukey: I think we'd also need an alarm on webrequest data not being deleted [08:01:38] elukey: The disk-latency graph BBU effect is impressive ! [08:02:22] joal_: I'll try to come up with one for hadoop nodes too to see if anything changed [08:02:25] :) [08:02:37] elukey: You guy are the man [08:02:53] too kind :) [08:03:05] oh no, never, you know enough of me :) [08:03:07] where is the webrequest cleaner job running? [08:03:13] ahhahaah [08:03:22] hm, I'd say analytics1003, but not sure [08:04:40] checking :) [08:04:47] ah joal_ another good news [08:04:53] I need to reboot all the hosts [08:05:22] elukey: I'd like to take my daily break now, would that be ok (coming back early afternoon and working afternoon [08:05:31] mwarf about reboot [08:07:48] sure! Since there are some alarms firing for hadoop nodes partitions filling up, I'll try to find the job responsible for the deletion and restart it [08:07:51] is it ok? [08:08:20] elukey: please - I think it has been manually stopped by ottomata a while ago [08:09:17] elukey: puppet/modules/role/manifests/analytics_cluster/refinery/job/data_drop.pp [08:09:30] elukey: Dropping now - Thanks for caring that, I'll be back ! [08:22:15] the job is running buuuuuuuut... it has been failing for some weird errors :P [08:22:25] * elukey alarms all the things [08:24:02] ERROR conf.Configuration: error parsing conf file:/etc/hive/conf.analytics-hadoop/hive-site.xml\njava.io.FileNotFoundException: /etc/hive/conf.analytics-hadoop/hive-site.xml [08:24:25] RuntimeError: ('Command: hive --service cli --database wmf_raw ... [08:25:56] so there is no timestamp but [08:25:57] -rw-r--r-- 1 hdfs analytics-admins 3693065 Jun 20 08:15 /var/log/refinery/drop-webrequest-partitions.log [08:26:07] so I guess that those errors are fresh [08:28:25] ahhhhh java.io.FileNotFoundException: /etc/hive/conf.analytics-hadoop/hive-site.xml (Permission denied) [08:49:45] !log manually running /srv/deployment/analytics/refinery/bin/refinery-drop-webrequest-partitions on an1003 to free hdfs space [08:49:46] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [08:51:09] !log manually added the user 'hdfs' to the 'hive' group to be able to run refinery-drop-webrequest-partitions [08:51:10] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [08:51:19] 2017-06-20T08:49:53 INFO Dropping 8512 partitions from table wmf_raw.webrequest [08:51:27] should free some space [09:10:21] https://grafana.wikimedia.org/dashboard/db/analytics-hadoop?panelId=25&fullscreen&orgId=1&from=now-3h&to=now [09:10:24] oh yes [09:10:24] :) [09:49:42] 10Analytics-Tech-community-metrics, 10Developer-Relations (Jul-Sep 2017): Automatically sync mediawiki-identities/wikimedia-affiliations.json DB dump file with the data available on wikimedia.biterg.io - https://phabricator.wikimedia.org/T157898#3362547 (10Albertinisg) The dump is being uploaded everyday at mi... [09:53:23] 10Analytics-Tech-community-metrics, 10Developer-Relations (Apr-Jun 2017): Find out (and fix) why we have a higher number of identity entries than before switching to new Bitergia DB scheme - https://phabricator.wikimedia.org/T168217#3358797 (10Lcanasdiaz) @Aklapper this is something tricky as we discard by pol... [10:27:33] !log reboot kafka1012, analytics1028, aqs1004 for kernel upgrades (canary hosts) [10:27:34] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [10:28:24] a-team: just rebooted --^ for kernel upgrades, will leave them running for a bit to spot any anomalies since the changes to the kernel are important enough. Should be a no-op though, so after lunch I'll keep going with the rest of the hosts [10:40:52] 10Analytics-Kanban: Reboot all the Analytics hosts for kernel upgrades - https://phabricator.wikimedia.org/T168381#3362750 (10elukey) [10:54:24] * elukey lunch! [11:32:38] (03CR) 10Mforns: [C: 04-1] UDF to tag requests (033 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/353287 (https://phabricator.wikimedia.org/T164021) (owner: 10Nuria) [11:53:57] (03PS1) 10Joal: Rename unique devices project-wide [analytics/refinery] - 10https://gerrit.wikimedia.org/r/360327 [11:54:48] !log Deleting old unique_devices data (renamed to unique_devices_per_domain) [11:54:49] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [11:55:22] back [11:55:45] I don't see anything weird in the hosts that I rebooted, so if nobody disagrees I'd proceed with the rest [11:55:50] Hi elukey [11:56:00] I monitored a bit the hosts you mentioned - looked good to me [11:56:11] Thanks again for hving found the error on partition deletion [11:56:26] elukey: How was it possible we had those errors showing up? [11:59:27] joal: I am not sure but I had to do a hack to make everything work (adding the hdfs user to the hive group) [11:59:41] but we absolutely need monitoring [11:59:46] to catch things earlier [11:59:56] elukey: definietly yes [11:59:58] this morning there were already datanode partitions filling up to 100% [12:00:27] That's bad elukey - I'm sorry for that [12:01:07] well it is not your fault joal :D [12:01:16] we as a team should have checked! [12:01:20] (also me as ops :P) [12:01:41] hm - I don't like not noticing when cluster is not happy [12:08:39] elukey: if you have a minute, would you mind restarting pivot? I dropped dataset with no associated config [12:09:20] sure, will do in a sec [12:09:36] joal: we'd need to reboot thorium, stat100[234] and analytics1003 as well [12:09:44] no prob for me [12:09:51] should we announce a maintenance window for tomorrow ? [12:10:21] sure [12:14:21] 10Analytics: Alarm on HDFS space used and related script failures - https://phabricator.wikimedia.org/T168390#3362956 (10elukey) [12:24:05] 10Analytics-Tech-community-metrics, 10Developer-Relations (Apr-Jun 2017): Find out (and fix) why we have a higher number of identity entries than before switching to new Bitergia DB scheme - https://phabricator.wikimedia.org/T168217#3363018 (10Aklapper) [12:28:45] 10Analytics-Tech-community-metrics, 10Developer-Relations (Apr-Jun 2017): Find out (and fix) why we have a higher number of identity entries than before switching to new Bitergia DB scheme - https://phabricator.wikimedia.org/T168217#3363067 (10Aklapper) >>! In T168217#3362570, @Lcanasdiaz wrote: > I don't see... [12:47:29] joal: interesting https://grafana.wikimedia.org/dashboard/db/analytics-hadoop?panelId=39&fullscreen&orgId=1&from=now-3h&to=now [13:01:49] * elukey afk for a bit! [13:08:06] is there any action I need to take for this email? Fatal Error - Oozie Job wikidata-wdqs_extract-wf-2017-6-20-11 [13:09:32] schana: I'd check in Hue what is the status of the job (like why it failed) and re-launch if necessary [13:11:29] elukey: it doesn't look like I can log in to hue with my ldap credentials [13:11:29] schana: I am rebooting the whole cluster for kernel upgrades, so it might have been me [13:11:39] let me check [13:13:57] java.net.ConnectException: Call From analytics1036/10.64.53.15 to analytics1054.eqiad.wmnet:8041 failed on connection exception: java.net.ConnectException: Connection refused; [13:14:36] !log re-run wikidata-wdqs_extract-wf-2017-6-20-11 (failed for connection issues, likely due to reboots) [13:14:37] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:14:44] schana: --^ [13:15:15] thanks elukey [13:16:23] haha, another ambiguity: https://en.wikipedia.org/wiki/Wikipedia:WikiProject [13:16:43] ottomata: o/ [13:16:49] hiii [13:16:57] oh elukey! we need to name that new kafka cluster [13:17:10] i kinda am liking jumbo, but also had another thought about it in my dreams last night [13:17:13] grand? [13:17:15] kafka grand cluster [13:17:20] grand-eqiad grand-codfw [13:17:20] ? [13:17:24] well [13:17:27] no grand-codfw* [13:17:31] buuuuuuuuuuuuu [13:17:33] haha [13:17:36] :D :D :D [13:17:50] I love jumbo [13:17:53] you do? [13:17:55] i like jumbo [13:17:59] +2 [13:18:06] kafka-jumbo1001 [13:18:21] ahahhahaah [13:18:26] it looks awesome [13:19:03] elukey jumbo was becoming my first choice anyway [13:19:24] its short, descriptive (enough) without being limiting, and fun! haha [13:20:12] https://en.wiktionary.org/wiki/jumbo [13:20:26] An especially large or powerful person, animal or thing. [13:22:26] ok let's do it [13:22:28] JUMBOOOOOOOOO [13:22:32] JUMBOTRON [13:28:12] ah there you go oozie! [13:28:19] I thought you were too quiet [13:28:33] hmmm, joal maybe not that useful, but interesting to know about: https://prestodb.io/docs/current/connector/kafka.html [13:28:41] works with json too [13:29:03] wonder if it injests constantly or just consumes when you query? seems weird [13:29:41] !log restart webrequest-load-coord-text and webrequest-load-coord-upload failed jobs due to reboots [13:29:42] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [13:38:06] I thought you guys liked it because of it being a pseudonym of the Boeing 747 [13:38:27] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3363318 (10Ottomata) @mobrovac @Pchelolo, any objections to a new `mediawiki/page/create` schema and even... [13:39:22] 10Analytics, 10Analytics-Cluster, 10Operations, 10ops-eqiad: rack/setup/install new kafka nodes - https://phabricator.wikimedia.org/T167992#3363320 (10Ottomata) We like jumbo! Let's do it. kafka-jumbo100[1-6] [13:39:40] 10Analytics, 10Analytics-Cluster, 10Operations, 10ops-eqiad: rack/setup/install new kafka nodes kafka-jumbo100[1-6] - https://phabricator.wikimedia.org/T167992#3363322 (10Ottomata) [13:40:41] 10Analytics, 10Analytics-Cluster, 10Operations, 10ops-eqiad: rack/setup/install new kafka nodes kafka-jumbo100[1-6] - https://phabricator.wikimedia.org/T167992#3352337 (10Ottomata) [13:42:00] 10Analytics: Send burrow lag statistics to statsd/graphite {hawk} - https://phabricator.wikimedia.org/T120852#3363327 (10Ottomata) Having consumer lag metrics in graphite would be really useful, especially as we encourage more folks to be consumers in the future. So I say yes! [13:53:10] elukey: Sorry missed your ping - Thanks for restarting the oozie jobs [13:54:50] oozie the complainer [13:56:03] :) [13:56:25] I just sent an email to everybody running a process on stat* [13:56:33] will reboot them tomorrow morning [13:58:55] 10Analytics-Kanban: Rename unique_devices_project_wide to unique_devices_per_project_class - https://phabricator.wikimedia.org/T168402#3363399 (10JAllemandou) [13:59:08] 10Analytics-Kanban: Rename unique_devices_project_wide to unique_devices_per_project_class - https://phabricator.wikimedia.org/T168402#3363410 (10JAllemandou) a:03JAllemandou [13:59:37] (03PS2) 10Joal: Rename unique devices project-wide [analytics/refinery] - 10https://gerrit.wikimedia.org/r/360327 (https://phabricator.wikimedia.org/T168402) [14:01:29] joal: forgot today, just restarted pivot [14:01:53] double checking elukey [14:02:13] good on my side elukey :) [14:02:16] Thanks ! [14:02:28] :) [14:02:34] I knoooww ooozieee [14:02:45] today I am causing you a lot of distress [14:02:55] huhu [14:03:59] hey mforns. give me 1 min please [14:04:12] lzia1, no rush :] [14:04:34] I will call you from a separate session. Somehow the link in the calendar doesn't work for me, mforns [14:04:56] k [14:05:00] lzia1: o/ [14:08:14] elukey: hi. :) I'm with Marcel in a call. will ping you in 30 min [14:10:30] lzia1: nono it was just a "hi", nothing more :) [14:26:48] elukey: phew. ;) [14:30:58] sooo, elukey: what's going on? :) [14:32:03] lzia1: having fun rebooting all the analytics hosts! :) [14:32:25] yeah, I'm going to read that email now, elukey. [14:32:27] all good, I've been on the Canary island recently and I am still there [14:32:46] so reboots are slowing bringing me back to work [14:32:48] :D [14:32:51] woooot! working remotely and enjoying or some other reason elukey? [14:33:19] no work, some days of holidays (a friend is getting married) [14:33:29] all good from your side?? [14:33:55] niiice. I'm in Tehran now until Friday, then I fly to Armenia with family to hang out there a bit and hopefully work less and enjoy Armenia a bit more before getting back to full speed madness again. ;) [14:34:21] I'm dying to go to Barcelona. /me looks at mforns. [14:34:30] :) [14:35:01] lzia1, :] awesome! I'm not in Barcelona though, hehehe [14:35:11] I live in Mallorca right now :] [14:35:45] * lzia1 pulls up the map for the new travel destination. :D [14:35:58] hehe [14:36:39] was this always the case, mforns?! :D [14:37:30] lzia1, well, I was born in Barcelona, then I lived for 4 years in Brazil, and then moved to Mallorca, we're here since end 2014 [14:38:17] ok, then expect at least a couple of days of visitors around Christmas. /me makes a note to convince Bernd. :D [14:39:05] lzia1, awesome! we'll be glad to show you the island! [14:39:56] =^.^= [14:46:16] !log re-run failed webrequest-load-text/upload jobs due to reboots [14:46:17] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [14:46:39] just doing the last workers now, boy the cluster is big now :D [14:47:53] elukey: :) [14:52:42] rebooting druid now [15:01:01] ottomata: standup for you rights! [15:01:01] joal: wololoooo [15:01:09] ahhh [15:01:18] Arf [15:36:30] 10Analytics-Tech-community-metrics, 10Developer-Relations (Apr-Jun 2017): Find out (and fix) why we have a higher number of identity entries than before switching to new Bitergia DB scheme - https://phabricator.wikimedia.org/T168217#3363807 (10Albertinisg) >>! In T168217#3361582, @Aklapper wrote: > * The [[ ht... [15:45:24] elukey: https://gerrit.wikimedia.org/r/#/c/360367/ [15:45:49] ottomata: s/hive/hdfs in the commit msg [15:45:50] ? [15:46:14] (03CR) 10Ottomata: [C: 031] Rename unique devices project-wide [analytics/refinery] - 10https://gerrit.wikimedia.org/r/360327 (https://phabricator.wikimedia.org/T168402) (owner: 10Joal) [15:46:35] woops thanks elukey [15:47:03] done [15:47:24] 10Analytics-Tech-community-metrics, 10Developer-Relations (Apr-Jun 2017): When indexing new users, identify identical email addresses and merge identities accordingly in the DB - https://phabricator.wikimedia.org/T151634#3363842 (10Albertinisg) The issue should be solved after the migration. As the `uuids` has... [15:58:03] 10Analytics, 10DBA: Purge all old data from master - https://phabricator.wikimedia.org/T168414#3363866 (10Ottomata) [15:58:28] 10Analytics, 10Analytics-Cluster: Make refinery drop data scripts email analytics-alerts if they fail - https://phabricator.wikimedia.org/T168415#3363883 (10Ottomata) [16:02:48] ok email sent to engineering/analytics [16:02:59] will reboot the stat hosts tomorrow morning [16:03:02] and also thorium [16:03:08] the impact should be minimm [16:03:12] *minimum [16:04:12] !log re-run pageview-druid-hourly-wf-2017-6-20-14 (failed due to druid reboots) [16:04:13] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:05:44] rebooting kafka1013 now [16:08:30] ottomata: I think we need to restart tranquility jobs [16:09:26] joal: ah snap I forgot about those [16:09:47] elukey: I can't connect to druid1001 anymore :( [16:10:19] elukey: druid outage I think [16:10:32] nono joal the host is coming up [16:10:44] let me check [16:10:47] kpivot is not happy though [16:11:49] ah because we have hardcoded in puppet the drud1001 node [16:13:02] k [16:15:37] joal: druid1001 is up now, it needed a slap [16:15:39] :D [16:15:43] :D [16:18:09] !log Rererun pageview-druid-hourly-wf-2017-6-20-14 (failed due to druid reboots) [16:18:10] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:18:55] elukey: Have you restarted druid1003 yet? [16:19:01] all of them [16:19:04] okey [16:23:39] !log Restarted tranquility for banners and netflow on druid1003 [16:23:40] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [16:33:30] elukey: Looks like we're back to regular ops on the cluster [16:33:39] elukey: Thanks a lot for the reboot!P [16:35:01] joal: sadly I need to spent probably tomorrow doing the rest :( [17:39:19] 10Analytics-Kanban: Reboot all the Analytics hosts for kernel upgrades - https://phabricator.wikimedia.org/T168381#3364468 (10elukey) [17:41:57] * elukey off! [17:44:42] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3364537 (10Pchelolo) Oh, this task took a long time to read. > any objections to a new mediawiki/page/cr... [17:50:56] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3364567 (10Ottomata) > As I remember we've used to use the PageContentInsertComplete in the beginning and... [17:54:27] (03PS1) 10Joal: Add new projects to mediawiki labs snapshots [analytics/refinery] - 10https://gerrit.wikimedia.org/r/360393 [18:01:11] !log Rerun webrequest-load-wf-text-2017-6-20-12 after oozie failure [18:01:12] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [18:01:42] Gone for tonight a-team, will see you tomorrow ! [18:01:51] nite jo [18:10:52] laters! [18:14:03] (03CR) 10Ottomata: [V: 032 C: 032] Add new projects to mediawiki labs snapshots [analytics/refinery] - 10https://gerrit.wikimedia.org/r/360393 (owner: 10Joal) [18:45:04] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3364850 (10kaldari) @Ottomata: `PageContentInsertComplete` isn't supposed to capture all revision creates... [18:50:41] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3364904 (10Pchelolo) Oh indeed, we're using `PageContentSaveComplete`, not `PageContentInsertComplete`..... [18:53:09] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3364923 (10Ottomata) Ohhhh, excuse me. Man what bad hook names. `PageContentSaveComplete` is for page u... [19:04:59] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3364980 (10Ottomata) Just examined some of the pages that @Niharika found have 2 revisions with `rev_pare... [19:13:58] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3365046 (10mobrovac) I wonder if also undeletions cause the `rev_parent_id` to be set to `0`. Either way... [19:21:18] wikimedia/mediawiki-extensions-EventLogging#662 (wmf/1.30.0-wmf.6 - ee7c067 : Translation updater bot): The build has errored. [19:21:18] Change view : https://github.com/wikimedia/mediawiki-extensions-EventLogging/compare/wmf/1.30.0-wmf.6 [19:21:18] Build details : https://travis-ci.org/wikimedia/mediawiki-extensions-EventLogging/builds/245074883 [19:21:43] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3365090 (10Ottomata) Are we sure that `PageContentInsertComplete` doesn't suffer from the same problem as... [19:24:50] !log beginning to consume select eventbus event using eventlogging mysql consumer and inserting into eventlogging analytics mysql db [19:24:51] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [19:41:15] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3365166 (10kaldari) The only purpose of `PageContentInsertComplete` is to handle events related to page c... [19:41:30] ottomata: joal: is it fine if I get back to you in 8 hours or so? I want to sleep a bit. ;) [19:41:46] if you prefer to have a response before the end of your day, I'll do it before I sleep. ;) [19:41:57] lzia: yeah no problem, maybe just before the next interview [19:42:11] which is thursday [19:42:16] yup. you should have a response by the time you wake up tomorrow, ottomata. [19:42:31] ok! no hurry no worries lzia, sleep away! [19:42:40] uhu. thanks. ciao ciao. :) [19:42:41] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3365167 (10mobrovac) Good point. This indeed seems to be possible, as there are places in the code that s... [19:45:20] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3365176 (10mobrovac) >>! In T150369#3365166, @kaldari wrote: > The only purpose of `PageContentInsertComp... [19:48:10] 10Analytics, 10Operations, 10Traffic: Increase request limits for GETs to /api/rest_v1/ - https://phabricator.wikimedia.org/T118365#3365183 (10GWicke) >>! In T118365#3349563, @Nuria wrote: >>which matches metrics end points explicitly limited at 100/s per client IP. > > mmm... looking at pageview API dashb... [19:49:34] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3365186 (10Ottomata) On the other hand, using `PageContentInsertComplete` is likely to be more accurate t... [19:51:31] ottomata: i am not opposed to the event, but honestly it sounds like nobody is sure that that hook does what it's supposed to do properly, and i'm kind of opposed to introducing a new event "hoping" that things work they way they should [19:51:51] i.e. i'd prefer to get some data on that hook prior to releasing the new event [19:51:57] shoot i broke eventlogging replication... hang on... [19:52:04] mobrovac: +1 [19:52:22] i'm not to opinionated about it, i generally agree, but they have some tight timeframe [19:56:03] hm i see [19:57:22] alright, ottomata, i guess we can release the event regardless, thinking about it what is important for us is to have a good schema for it [19:57:51] and in the first iteration we can simply say we do not guarantee the events are 100% accurate (which is very lame, but if they are in a hurry...) [19:58:24] +1, and i think basing it on the other schemas will hopefully keep the bikeshedding down :) [19:58:27] should be easy [19:58:32] i'll try to get to that tomorrow [19:58:43] kk cool, thnx! [20:03:12] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3365205 (10Ottomata) @kaldari FYI I just merged eventlogging puppet change to start inserting some of th... [20:03:47] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3365206 (10Pchelolo) Hm I'm wondering if we need a new schema for this or we can reuse the `revision-crea... [20:05:32] Pchelolo: OH NOW you want to reuse the schema! :) [20:05:34] :p [20:05:53] haha ottomata this one would be identical, wouldn't it? [20:05:58] like 100% the same? [20:06:22] not sure yet [20:06:28] maybe witih some fields missing? [20:06:36] LIKE REVISION CREATE IS REVISION SCORE WITH FIELDS MISSING [20:06:38] :D [20:07:40] * mobrovac facepalms [20:07:44] here we go again [20:07:51] hahahah jk [20:08:20] we should definitely implement schema inheritance [20:08:38] i think if we do that, we should ahve a better schema repo/ gui first [20:08:46] maybe even making meta.wm.org able to display the schemas [20:08:52] cause otherwise they are gonna get hard to read [20:13:35] +1 [20:13:55] but you don't want me to design that GUI, because it would then be *ugly* to read :P [20:22:45] haha me neither, but hopefully we can just make it kinda like the eventlogging schema in meta [20:22:49] maybe even use the same extension [20:23:09] https://meta.wikimedia.org/wiki/Schema:Edit [20:23:28] not sure if this qualifies as 'not ugly' though [21:42:12] 10Analytics: Extraneous whitelist items for WikimediaBlogVisit schema - https://phabricator.wikimedia.org/T168475#3365522 (10Tbayer) [21:53:34] 10Analytics-Kanban: Update html language for per-domain uniques - https://phabricator.wikimedia.org/T168477#3365554 (10Milimetric) [23:00:55] 10Analytics, 10Analytics-EventLogging, 10Contributors-Analysis, 10EventBus, and 5 others: Record an event every time a new content namespace page is created - https://phabricator.wikimedia.org/T150369#3365751 (10kaldari) >I like the idea of having a page-creation event, but would really really like us to... [23:16:04] 10Analytics, 10Analytics-EventLogging, 10Community-Tech: Remove EventLogging for cookie blocks - https://phabricator.wikimedia.org/T166247#3365789 (10kaldari) p:05Normal>03Low