[00:24:05] 10Analytics, 10ExternalGuidance, 10Product-Analytics, 10MW-1.33-notes (1.33.0-wmf.18; 2019-02-19), 10Patch-For-Review: Measure the impact of externally-originated contributions - https://phabricator.wikimedia.org/T212414 (10chelsyx) @santhosh, as @Nuria said, eventlogging table goes to hadoop only unless... [01:37:54] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10Performance-Team (Radar): [Bug] Type mismatch between NavigationTiming EL schema and Hive table schema - https://phabricator.wikimedia.org/T214384 (10Nuria) I really wonder if the way to circumvent this bug (that will reappear in other schemas) is to s... [01:50:53] 10Analytics, 10ExternalGuidance, 10Product-Analytics, 10MW-1.33-notes (1.33.0-wmf.18; 2019-02-19), 10Patch-For-Review: Measure the impact of externally-originated contributions - https://phabricator.wikimedia.org/T212414 (10Nuria) @chelsyx you can also see errors like: select * from eventerror where even... [03:58:12] 10Analytics, 10ExternalGuidance, 10Product-Analytics, 10MW-1.33-notes (1.33.0-wmf.18; 2019-02-19), 10Patch-For-Review: Measure the impact of externally-originated contributions - https://phabricator.wikimedia.org/T212414 (10santhosh) >>! In T212414#4961807, @Nuria wrote: > @santhosh tables are created i... [04:10:31] 10Analytics, 10DBA, 10MediaWiki-Database, 10Research, 10Wikidata: Improve interlingual links across wikis through Wikidata IDs - https://phabricator.wikimedia.org/T215616 (10Tbayer) [04:12:17] 10Analytics, 10ExternalGuidance, 10Product-Analytics, 10MW-1.33-notes (1.33.0-wmf.18; 2019-02-19), 10Patch-For-Review: Measure the impact of externally-originated contributions - https://phabricator.wikimedia.org/T212414 (10santhosh) >>! In T212414#4963003, @chelsyx wrote: > I did a quick check on the ta... [06:10:44] 10Analytics, 10Analytics-Kanban, 10DBA, 10Patch-For-Review, 10User-Banyek: Migrate dbstore1002 to a multi instance setup on dbstore100[3-5] - https://phabricator.wikimedia.org/T210478 (10Marostegui) The migration finished. These are the times in UTC from 18th Feb 2019: - Read only on dbstore1002: 05:53... [06:10:55] 10Analytics, 10Analytics-Kanban, 10DBA, 10Patch-For-Review, 10User-Banyek: Migrate dbstore1002 to a multi instance setup on dbstore100[3-5] - https://phabricator.wikimedia.org/T210478 (10Marostegui) [06:43:43] Amir1,addshore o/ - as Manuel wrote staging on dbstore1005 is ready! [06:51:32] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Move AQS to nodejs 10 - https://phabricator.wikimedia.org/T210706 (10elukey) Marko mentioned on IRC that if we have a .travis file in the repo then he can enable Travis on the GH mirror and run nodejs10 there too. [07:00:16] updated https://wikitech.wikimedia.org/wiki/Analytics/Data_access#MariaDB_replicas [07:00:36] my idea is to set the sunset deadline for dbstore1002 in two weeks [07:08:53] 10Analytics: Clean up home dirs for user mkroetzsch - https://phabricator.wikimedia.org/T214501 (10elukey) a:03Smalyshev [07:56:24] 10Analytics, 10Discovery, 10Operations, 10Research: Workflow to be able to move data files computed in jobs from analytics cluster to production - https://phabricator.wikimedia.org/T213976 (10JAllemandou) One note about hadoop blobs: HDFS stores files split in chunks, with those not collocated. If we use... [07:57:45] 10Analytics, 10Analytics-Kanban: Purge wikitext snapshots - https://phabricator.wikimedia.org/T216414 (10JAllemandou) Ahhhh! I didn't get it - The checksum is computed once er argument set, and don't change if the dates worked by script change. This means I can get the checksum manually and set it up in the cr... [07:58:27] 10Analytics, 10Analytics-Kanban: Purge wikitext snapshots - https://phabricator.wikimedia.org/T216414 (10JAllemandou) [07:59:42] Good morning elukey :) [08:00:07] bonjour! [08:00:10] the GPU hates me [08:00:39] I don't think GPU's have feelings, but if it had, I'm sure it'd love you, giving so much attention :) [08:01:00] elukey: I you think I can be of any help, please let me know :S [08:02:10] not sure if you saw https://phabricator.wikimedia.org/T148843#4962357 [08:02:14] elukey: I have a question for you - I'm looking at datapurge in puppet - There are a lot of `cron` in comments while the thing only uses systems-timers [08:02:18] elukey: shall I correct? [08:02:29] ah yes sure [08:03:24] elukey: I had seen that comment yes, and felt like nuria described yesterday - I can read some of those words, yes :) [08:03:36] :) [08:05:46] Ah elukey ! There still is a CRON in there - query_click_retentiong [08:06:05] elukey: do we want to move it? [08:06:08] IIRC it is not ours, but search's? [08:06:12] I didn't move it on purpose [08:06:13] correct elukey [08:06:19] they have set a mailing list as alarming [08:06:20] Ok - will leave it [08:24:08] 10Analytics, 10Analytics-Kanban: Purge wikitext snapshots - https://phabricator.wikimedia.org/T216414 (10mforns) @JAllemandou Yes. The checksum does not change when the dates change, because the --older-than=90 parameter still remains the same. [08:30:14] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Purge wikitext snapshots - https://phabricator.wikimedia.org/T216414 (10JAllemandou) [08:30:41] workers at home in a bit, will probabl lag a bit in answering [08:49:45] 10Analytics, 10Operations, 10Research-management, 10Patch-For-Review, 10User-Elukey: GPU upgrade for stats machine - https://phabricator.wikimedia.org/T148843 (10elukey) https://github.com/RadeonOpenCompute/ROCm/issues/482 is a very similar problem, so I tried a couple of suggestions in here: * export H... [08:57:43] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10Performance-Team (Radar): [Bug] Type mismatch between NavigationTiming EL schema and Hive table schema - https://phabricator.wikimedia.org/T214384 (10mforns) @Nuria I think this could work, but I believe it has a couple drawbacks that we can avoid: -... [09:06:37] 10Analytics, 10Operations, 10Research-management, 10Patch-For-Review, 10User-Elukey: GPU upgrade for stats machine - https://phabricator.wikimedia.org/T148843 (10MoritzMuehlenhoff) >>! In T148843#4963670, @elukey wrote: > What do you think about opening a GH issue to ROCm first to (hopefully) get some fe... [09:32:34] team, I'm going to deploy refinery (not source) to get the updated EL sanitization whitelist for some backfilling I'd like to do [09:35:10] 10Analytics, 10Operations, 10Research-management, 10Patch-For-Review, 10User-Elukey: GPU upgrade for stats machine - https://phabricator.wikimedia.org/T148843 (10elukey) Created https://github.com/RadeonOpenCompute/ROCm/issues/714 [09:38:18] mforns: nice! So my script will get deployed too [09:38:21] feel free to go [09:38:24] elukey, yes! [09:38:27] doing it right now [09:38:28] nothing exploding from the ops side [09:38:35] ok [09:39:24] joal: https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/491246/ seems ready to go, I don't think that I need to inspect all parameters since you tested them, but if you want I can do it [09:39:48] elukey: I feel confident having tested them [09:46:20] ack merging [09:47:46] !log deployed refinery (without refinery-source) until commit 0d7ec1989852d4dd5b1497463fd9509e4f5bdb87 [09:47:48] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [09:48:45] done [09:56:36] mforns: have you deployed the purging patch before deploying refinery? [09:56:53] joal, no... [09:56:56] :S [09:56:57] arf, looks like no [09:57:01] it'll wait [09:57:37] joal, which purging patch? the puppet one? [09:57:55] mforns: This one - https://gerrit.wikimedia.org/r/c/analytics/refinery/+/491252 [09:57:57] did I break something? [09:58:06] everything should be fine :) [09:58:10] it'll wait [09:58:31] joal, tomorrow there's deployment train [09:58:37] but if you want I can redeploy now [09:58:57] mforns: tomorrow is fine - I thought deploy today meant deploy train :) [09:59:09] joal, sorry for not looking into pending patches [09:59:44] no no, just wanted to selfishly deploy the whitelist changes to be able to backfill :P [09:59:47] no problem mforns - my mistake to mix tchuu-tchuu and adhoc deploys :) [09:59:52] heh [10:36:01] 10Analytics, 10Analytics-Kanban, 10User-Marostegui: Migrate users to dbstore100[3-5] - https://phabricator.wikimedia.org/T215589 (10Marostegui) For what is worth, dbstore1002 is now lagging behind on s8 (wikidatawiki) 7 days and it keeps lagging, I doubt it will ever catch up. [10:37:51] 10Analytics, 10Analytics-Kanban, 10Operations, 10Product-Analytics, 10Patch-For-Review: dbstore1002 Mysql errors - https://phabricator.wikimedia.org/T213670 (10Marostegui) For what is worth, dbstore1002 is now lagging behind on s8 (wikidatawiki) 7 days and it keeps lagging, I doubt it will ever catch up.... [10:39:15] 10Analytics, 10Analytics-Kanban, 10Operations, 10Product-Analytics, 10Patch-For-Review: dbstore1002 Mysql errors - https://phabricator.wikimedia.org/T213670 (10Marostegui) p:05High→03Low Reducing priority as the errors on dbstore1002 are not too important anymore as this host shouldn't be used anymor... [10:44:03] 10Analytics, 10Operations, 10ops-eqiad: Decommission dbstore1002 - https://phabricator.wikimedia.org/T216491 (10Marostegui) [10:44:11] 10Analytics, 10Operations, 10ops-eqiad: Decommission dbstore1002 - https://phabricator.wikimedia.org/T216491 (10Marostegui) 05Open→03Stalled p:05Triage→03Normal [10:44:48] 10Analytics, 10Analytics-Kanban, 10DBA, 10Patch-For-Review, 10User-Banyek: Migrate dbstore1002 to a multi instance setup on dbstore100[3-5] - https://phabricator.wikimedia.org/T210478 (10Marostegui) [10:44:51] 10Analytics, 10Operations, 10ops-eqiad: Decommission dbstore1002 - https://phabricator.wikimedia.org/T216491 (10Marostegui) [10:49:02] 10Analytics, 10EventBus, 10Research, 10The-Wikipedia-Library, 10Services (watching): page-links-change stream doesn't capture links on page deletion - https://phabricator.wikimedia.org/T216249 (10Samwalton9) @bmansurov Strange. I just reproduced it at User:Samwalton9/sandbox10 (the most recent deletion l... [11:45:30] 10Analytics: Set up a Kerberos KDC service in production with minimal puppet automation - https://phabricator.wikimedia.org/T212257 (10elukey) [11:46:42] 10Analytics: Set up a Kerberos KDC service in production with minimal puppet automation - https://phabricator.wikimedia.org/T212257 (10elukey) @MoritzMuehlenhoff kerberos1001 created (with role::spare::system), let's coordinate (whenever you have time) about how to proceed :) [11:47:35] joal: ready to merge the data_purge change? [11:51:21] I am reviewing a couple of things though [11:51:22] mmmm [11:54:23] added a couple of comments [12:57:26] * elukey lunch! [13:08:28] 10Analytics, 10EventBus, 10Research, 10The-Wikipedia-Library, 10Services (watching): page-links-change stream doesn't capture links on page deletion - https://phabricator.wikimedia.org/T216249 (10bmansurov) @Samwalton9 thanks for confirming. Now I know that something is wrong with my setup. [14:37:20] It's time to test [14:37:27] (03PS3) 10Joal: Update delete/restore in mediawiki-history [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/485710 (https://phabricator.wikimedia.org/T213603) [14:37:29] (03PS1) 10Joal: Refactor mediawiki-history core data gathering [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/491494 (https://phabricator.wikimedia.org/T216603) [14:42:12] joal: o/ - whenever you have time I have a question about oozie and webrequest refine in the testing cluster [14:42:24] elukey: shoot :) [14:42:58] ack! So I need to start something that refines the webrequest test raw data [14:43:10] what do you prefer? A single coordinator, the bundle, etc..? [14:44:26] elukey: I don't mind - I think easiest might be to use a bundle redacted to a single coord, but can be done otherwise [14:45:04] joal: sure, the bundle looks fine to me. Going to check how to do it, I know that you know it already but I don't :D [14:45:15] so I'll come to you crying soon [14:45:29] elukey: please let me help before you cry ;) [14:45:32] but maybe I can make it! [14:45:44] I'm sure you can, without a tear [14:45:56] let's see if in 30 mins I am done [14:50:02] job_tracker = resourcemanager.analytics.eqiad.wmnet:8032 [14:50:15] resourcemanager.analytics.eqiad.wmnet. 3600 IN CNAME analytics1001.eqiad.wmnet. [14:50:18] analytics1001.eqiad.wmnet. 3600 IN A 10.64.36.118 [14:50:31] mmmm [14:50:42] I guess that the job_tracker thing is not really used [14:54:35] elukey: this still makes me nervous :) [14:58:05] joal: I can update the CNAME in 1 min, but I am wondering what is the purpose of the config since it is broken and we haven't noticed it [15:00:02] elukey: in oozie I think job_tracker is the old name for resource_manager [15:01:44] joal: but is it used by oozie? What I mean is, it should break in some way now no? [15:02:02] elukey: it means oozie will try to use prod RM [15:02:07] IIUC [15:02:43] joal: sure but now it tries to use analytics1001 [15:02:50] that is not really the best choice [15:02:52] :D [15:02:56] indeed [15:03:16] I don't understand how this is even possible :( [15:03:24] old codebase maybe [15:03:28] so either the option it is not used, or it fails in the logs but in a graceful way and we don't notice it [15:04:03] elukey: if you use oozie/webrequest/load/bundle.properties, you have: job_tracker = resourcemanager.analytics.eqiad.wmnet:8032 [15:04:27] Ah !!! But the issue is in CNAME -- MANNN . I 'm slow [15:05:10] ahhhhh sorry I should have mentioned, my bad! [15:05:18] yes exactly it is currently broken [15:05:21] this is why I am asking [15:05:27] Nah, I should have been a bit more careful when reading :) [15:05:53] I assume that oozie default to correct hadoop conf then, but weird indeed ! [15:07:10] the change is very old (~5y ago) https://gerrit.wikimedia.org/r/#/c/analytics/refinery/+/149007/ [15:10:36] elukey: Hey, can you take a look at this: https://gerrit.wikimedia.org/r/c/operations/puppet/+/490085 [15:10:41] 10Analytics, 10Research, 10Article-Recommendation: Generate article recommendations in Hadoop for use in production - https://phabricator.wikimedia.org/T210844 (10Ottomata) @bmansurov There isn't a canonical repository for Oozie job definitions, they can live anywhere. It looks like discovery has their own... [15:11:01] (03CR) 10Addshore: [C: 03+2] Introduce WikimediaDbSectionMapper based on db-eqiad.php config [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/489097 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [15:11:08] (03PS1) 10Addshore: Introduce WikimediaDbSectionMapper based on db-eqiad.php config [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/491508 (https://phabricator.wikimedia.org/T213894) [15:11:13] (03Merged) 10jenkins-bot: Introduce WikimediaDbSectionMapper based on db-eqiad.php config [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/489097 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [15:11:16] (03CR) 10Addshore: [C: 03+2] Introduce WikimediaDbSectionMapper based on db-eqiad.php config [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/491508 (https://phabricator.wikimedia.org/T213894) (owner: 10Addshore) [15:11:19] Amir1: sure, shall I merge? [15:11:34] (03Merged) 10jenkins-bot: Introduce WikimediaDbSectionMapper based on db-eqiad.php config [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/491508 (https://phabricator.wikimedia.org/T213894) (owner: 10Addshore) [15:11:57] I take it as yes :D [15:12:02] it would be fantastic [15:12:03] :D [15:12:09] * addshore just +1ed [15:12:38] Adam +1ed so I cannot do anything else than +2 it [15:13:37] Thanks [15:14:00] running puppet on stat1007 [15:14:07] (03CR) 10Addshore: Add methods for new hosts and changing good_articles.php to use that (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/490088 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [15:14:15] (03CR) 10Addshore: [C: 03+1] "Will merge once the puppet change is in" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/490088 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [15:14:26] done! [15:14:32] (03PS2) 10Addshore: Add methods for new hosts and changing good_articles.php to use that [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/490088 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [15:14:41] (03CR) 10Addshore: [C: 03+2] "Actually only the prod branch change needs to wait for the config update" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/490088 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [15:14:58] addshore: puppet change merged and deployed on stat1007 [15:15:33] (03Merged) 10jenkins-bot: Add methods for new hosts and changing good_articles.php to use that [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/490088 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [15:15:44] (03CR) 10Addshore: Add methods for new hosts and changing good_articles.php to use that (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/490105 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [15:15:48] (03CR) 10Addshore: [C: 03+2] Add methods for new hosts and changing good_articles.php to use that [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/490105 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [15:15:58] elukey: amazing :P [15:15:58] (03Merged) 10jenkins-bot: Add methods for new hosts and changing good_articles.php to use that [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/490105 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [15:16:01] spam spam spam [15:16:25] Amir1: I think we need to prioritize the 2 or so scripts that need to write to a db right to get them working again? :) [15:16:57] (03CR) 10Ladsgroup: Add methods for new hosts and changing good_articles.php to use that (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/490088 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [15:17:28] addshore: yes, that's on my radar [15:19:12] (03CR) 10Addshore: Add methods for new hosts and changing good_articles.php to use that (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/490088 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [15:19:20] :) [15:26:43] (03CR) 10Ladsgroup: Add methods for new hosts and changing good_articles.php to use that (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/490088 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [15:27:54] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 3 others: Provide tools for querying MediaWiki replica databases without having to specify the shard - https://phabricator.wikimedia.org/T212386 (10Ottomata) @elukey, to help with @mpopov's question, could the wrapper have a mo... [15:34:08] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10Performance-Team (Radar): [Bug] Type mismatch between NavigationTiming EL schema and Hive table schema - https://phabricator.wikimedia.org/T214384 (10Ottomata) Thanks yall! The procedure seems great but also especially cautious. This would also work... [15:34:33] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 3 others: Provide tools for querying MediaWiki replica databases without having to specify the shard - https://phabricator.wikimedia.org/T212386 (10elukey) >>! In T212386#4964864, @Ottomata wrote: > @elukey, to help with @mpopo... [15:38:33] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 3 others: Provide tools for querying MediaWiki replica databases without having to specify the shard - https://phabricator.wikimedia.org/T212386 (10Ottomata) > Sure, we can definitely work on a shared sqoop wrapper I don't eve... [15:42:05] 10Analytics: Old job_tracker setting in oozie properties - https://phabricator.wikimedia.org/T216519 (10elukey) [15:42:20] joal: --^ [15:43:16] Thanks elukey - This makes me also think about adding global parameter-values for oozie jobs [15:44:20] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 3 others: Provide tools for querying MediaWiki replica databases without having to specify the shard - https://phabricator.wikimedia.org/T212386 (10elukey) >>! In T212386#4964927, @Ottomata wrote: >> Sure, we can definitely wor... [15:46:28] elukey: about the purging script, shall I go for a bash script? [15:47:08] * joal wonders if intervening between elukey and mforns is a good idea [15:47:11] joal: yep I think it is a better option [15:47:22] ack ! [15:47:32] Marcel seemed ok from what I've read in his last comment [15:48:14] elukey: in `modules/profile/manifests/analytics/refinery/job` ? [15:48:43] milimetric: yt? [15:48:47] hello team [15:48:49] hey nuria [15:49:22] milimetric: did you documented the "new" edit dataset for geo-editors about edits per country [15:49:43] yes nuria, https://wikitech.wikimedia.org/wiki/Analytics/Data_Lake/Edits/Geoeditors#Geoeditors_Edits_Monthly [15:52:11] joal: yep should be ok, not manifest though but "files" [15:52:42] elukey: "templates" - yes my bad [15:53:29] joal: ah ok if you need variables yes [15:54:08] elukey: I was planning to reuse the variables I have set in the current example - ok? [15:56:06] sure [15:58:25] (03CR) 10Milimetric: [V: 03+2] Add mediawiki_wikitext_history to drop-script [analytics/refinery] - 10https://gerrit.wikimedia.org/r/491252 (owner: 10Joal) [15:59:27] milimetric: k, that you, i will note this on teh manager;s updates this week [16:01:40] ottomata: ping? [16:02:04] ping mforns [16:02:22] sorry going! [16:03:05] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 3 others: Provide tools for querying MediaWiki replica databases without having to specify the shard - https://phabricator.wikimedia.org/T212386 (10mpopov) >>! In T212386#4964982, @elukey wrote: >>>! In T212386#4964927, @Ottoma... [16:03:09] (03PS1) 10Elukey: analytics-mysql: add print-target parameter [analytics/refinery] - 10https://gerrit.wikimedia.org/r/491520 (https://phabricator.wikimedia.org/T212386) [16:07:30] 10Analytics, 10EventBus, 10MediaWiki-Core-Testing, 10Quibble, and 4 others: Flaky quibble-vendor-mysql-hhvm-docker test in Jenkins - https://phabricator.wikimedia.org/T216069 (10Pchelolo) This is getting in our way since we can't merge an otherwise ready patchset [16:07:43] (03CR) 10Ottomata: [C: 03+1] analytics-mysql: add print-target parameter [analytics/refinery] - 10https://gerrit.wikimedia.org/r/491520 (https://phabricator.wikimedia.org/T212386) (owner: 10Elukey) [16:10:42] (03CR) 10Elukey: [C: 04-1] "Nope doesn't work ;)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/491520 (https://phabricator.wikimedia.org/T212386) (owner: 10Elukey) [16:16:59] (03PS2) 10Elukey: analytics-mysql: add print-target parameter [analytics/refinery] - 10https://gerrit.wikimedia.org/r/491520 (https://phabricator.wikimedia.org/T212386) [16:18:34] (03PS3) 10Elukey: analytics-mysql: add print-target parameter [analytics/refinery] - 10https://gerrit.wikimedia.org/r/491520 (https://phabricator.wikimedia.org/T212386) [16:20:57] (03PS4) 10Elukey: analytics-mysql: add print-target parameter [analytics/refinery] - 10https://gerrit.wikimedia.org/r/491520 (https://phabricator.wikimedia.org/T212386) [16:21:58] (03CR) 10Elukey: [V: 03+1] "This seems to work :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/491520 (https://phabricator.wikimedia.org/T212386) (owner: 10Elukey) [16:27:06] milimetric: --^ (whenever you have time) [16:27:37] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 2 others: Replace the current multisource analytics-store setup - https://phabricator.wikimedia.org/T172410 (10mpopov) I just noticed that the tables related to the Echo extension are (surprisingly) not yet available in the enw... [16:29:42] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 2 others: Replace the current multisource analytics-store setup - https://phabricator.wikimedia.org/T172410 (10Marostegui) >>! In T172410#4965217, @mpopov wrote: > I just noticed that the tables related to the Echo extension ar... [16:29:46] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 2 others: Replace the current multisource analytics-store setup - https://phabricator.wikimedia.org/T172410 (10jcrespo) Echo extension lives in x1, not on enwiki. [16:30:34] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10Performance-Team (Radar): [Bug] Type mismatch between NavigationTiming EL schema and Hive table schema - https://phabricator.wikimedia.org/T214384 (10Nuria) Looks like we can close this ticket @Milimetric is going to take a look at other schemas to qua... [16:30:48] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10Performance-Team (Radar): [Bug] Type mismatch between NavigationTiming EL schema and Hive table schema - https://phabricator.wikimedia.org/T214384 (10Nuria) 05Open→03Resolved [16:31:02] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 2 others: Replace the current multisource analytics-store setup - https://phabricator.wikimedia.org/T172410 (10elukey) >>! In T172410#4965217, @mpopov wrote: > I just noticed that the tables related to the Echo extension are (s... [16:38:17] 10Analytics, 10Discovery, 10Operations, 10Research: Workflow to be able to move data files computed in jobs from analytics cluster to production - https://phabricator.wikimedia.org/T213976 (10mmodell) FWIW I found it fairly easy to work with swift from a development point of view but getting that experimen... [16:38:49] 10Analytics, 10Discovery, 10Operations, 10Research: Workflow to be able to move data files computed in jobs from analytics cluster to production - https://phabricator.wikimedia.org/T213976 (10Nuria) I think transferring data *seems* that could be taken care of with hadoop's copytolocal right? Issue we wan... [16:49:08] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10Performance-Team (Radar): [Bug] Type mismatch between NavigationTiming EL schema and Hive table schema - https://phabricator.wikimedia.org/T214384 (10mforns) @Ottomata I tried the simple ALTER TABLE, and it works, provided the field you want to change... [16:51:04] 10Analytics, 10Pageviews-API: Add wikimania.wikimedia.org to pageview whitelist - https://phabricator.wikimedia.org/T216525 (10MusikAnimal) [17:18:48] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 2 others: Replace the current multisource analytics-store setup - https://phabricator.wikimedia.org/T172410 (10mpopov) >>! In T172410#4965227, @Marostegui wrote: >>>! In T172410#4965217, @mpopov wrote: >> I just noticed that th... [17:24:14] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 2 others: Replace the current multisource analytics-store setup - https://phabricator.wikimedia.org/T172410 (10jcrespo) While this may look like an annoyance, we don't usually talk about the things that this change improved: *... [17:27:17] 10Analytics, 10ExternalGuidance, 10Product-Analytics, 10MW-1.33-notes (1.33.0-wmf.18; 2019-02-19), 10Patch-For-Review: Measure the impact of externally-originated contributions - https://phabricator.wikimedia.org/T212414 (10Nuria) @santosh, where at? our docs are in wikitech and seem up to date: https://... [17:36:14] 10Analytics, 10Analytics-Kanban, 10Operations, 10hardware-requests: GPU upgrade for stat1005 - https://phabricator.wikimedia.org/T216226 (10RobH) >>! In T216226#4961096, @elukey wrote: > Thanks all for all the detailed info! > > One thought: I found this interesting use case https://www.amd.com/en/case-st... [17:38:30] 10Analytics, 10ExternalGuidance, 10Product-Analytics, 10MW-1.33-notes (1.33.0-wmf.18; 2019-02-19), 10Patch-For-Review: Measure the impact of externally-originated contributions - https://phabricator.wikimedia.org/T212414 (10Nuria) @santhosh I see, that was on the mediawiki.org doc which describe the fron... [17:39:12] 10Analytics, 10Analytics-Kanban, 10Operations, 10hardware-requests: GPU upgrade for stat1005 - https://phabricator.wikimedia.org/T216226 (10EBernhardson) Unfortunately the rx 550 and 560 mentioned have 4GB of memory, which is basically a show stopper. [17:46:22] nuria: yargh, not doing map types now really sucks [17:46:30] its not just custom event fields for ApiAction, etc. [17:46:41] one of the uses is to standardize the http info, e.g. http headers [17:47:18] e.g. https://phabricator.wikimedia.org/T214093#4918832 [17:48:06] it also sucks for normalization, e.g. to_lower() on all header keys [17:48:33] hmmm actually no that is the same, i take that back. [17:55:23] 10Analytics, 10Analytics-Kanban, 10Operations, 10ops-eqiad: confirm gpu form factor in stat1005 - https://phabricator.wikimedia.org/T216528 (10RobH) p:05Triage→03Normal [18:21:55] 10Analytics, 10Analytics-Kanban, 10Operations, 10ops-eqiad: confirm gpu form factor in stat1005 - https://phabricator.wikimedia.org/T216528 (10Cmjohnson) {F28247119} {F28247120} {F28247121} {F28247122} {F28247124} {F28247123} {F28247126} {F28247125} {F28247127} [18:23:47] 10Analytics, 10Analytics-Kanban, 10Operations, 10ops-eqiad: confirm gpu form factor in stat1005 - https://phabricator.wikimedia.org/T216528 (10Cmjohnson) There appears to be power already connected to the GPU The dimensions are 12"L 4" Width 2" Depth. The pictures have the measurements as well [18:27:30] for all the curious, https://phabricator.wikimedia.org/T216528 shows how stat1005 looks [18:27:33] :) [18:35:23] * elukey off! [18:47:00] Heya fdans - Here I am [18:47:25] joal: bc? it's a tiny thing [18:47:31] sure fdans OMW [18:59:40] 10Analytics, 10DBA, 10MediaWiki-Database, 10Research, 10Wikidata: Improve interlingual links across wikis through Wikidata IDs - https://phabricator.wikimedia.org/T215616 (10Isaac) thank you @JAllemandou this is awesome!!! completely unblocks me (i have a bunch of page titles across all the wikipedias an... [19:01:33] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10Performance-Team (Radar): [Bug] Type mismatch between NavigationTiming EL schema and Hive table schema - https://phabricator.wikimedia.org/T214384 (10Ottomata) Ah ok that makes sense. Thanks for doing that! [19:10:26] 10Analytics, 10DBA, 10MediaWiki-Database, 10Research, 10Wikidata: Improve interlingual links across wikis through Wikidata IDs - https://phabricator.wikimedia.org/T215616 (10diego) @JAllemandou , yes. Having this by revision would be great! [19:14:24] 10Analytics, 10Analytics-EventLogging, 10EventBus, 10Security-Team, and 3 others: Modern Event Platform: Stream Intake Service: AJV usage security review - https://phabricator.wikimedia.org/T208251 (10sbassett) Hey @Ottomata - just wanted to check in on the status of all this and if you needed anything els... [19:18:05] 10Analytics, 10DBA, 10MediaWiki-Database, 10Research, 10Wikidata: Improve interlingual links across wikis through Wikidata IDs - https://phabricator.wikimedia.org/T215616 (10Isaac) @diego: my interpretation is that right now in the revision history version, the same wikidb/page ID/title is associated wit... [19:19:53] 10Analytics, 10Analytics-EventLogging, 10EventBus, 10Security-Team, and 3 others: Modern Event Platform: Stream Intake Service: AJV usage security review - https://phabricator.wikimedia.org/T208251 (10Ottomata) Heya! I still need to resolve a couple of things here: CSP and package-lock.json stuff. Its on... [19:20:31] 10Analytics, 10Analytics-EventLogging, 10EventBus, 10Security-Team, and 3 others: Modern Event Platform: Stream Intake Service: AJV usage security review - https://phabricator.wikimedia.org/T208251 (10sbassett) Sounds good, thanks. [19:34:48] 10Analytics, 10Product-Analytics: "Edit" equivalent of pageviews daily available to use in Turnilo and Superset - https://phabricator.wikimedia.org/T211173 (10kzimmerman) @MNeisler Nuria mentioned that @mforns will be testing ways to load datasets related to this ask (as I understand it, he's wrapping up some... [19:40:20] 10Analytics, 10DBA, 10MediaWiki-Database, 10Research, 10Wikidata: Improve interlingual links across wikis through Wikidata IDs - https://phabricator.wikimedia.org/T215616 (10JAllemandou) Thanks @Isaac for reformulating the question I tried to explain above :) @diego: Can you confirm there is value for yo... [19:48:27] (03PS4) 10Joal: Update delete/restore in mediawiki-history [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/485710 (https://phabricator.wikimedia.org/T213603) [19:48:29] (03PS2) 10Joal: Refactor mediawiki-history core data gathering [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/491494 (https://phabricator.wikimedia.org/T216603) [19:49:06] milimetric: Those patches --^ seem to at least not fail when run over data :) [19:50:09] nice joal. I'm still working on checking for that schema mismatch bug, should be done with that relatively soon and I'll take a look at the changes next [19:53:15] 10Analytics, 10Operations, 10Wikimedia-Stream, 10Services (watching): Eventstreams build is broken - https://phabricator.wikimedia.org/T216184 (10Ottomata) Hm, not sure why EventStreams is requiring node-rdkafka@2.5.1. EventStreams itself doesn't require node-rdkafka, its KafkaSSE dependency does. [[ h... [19:58:12] 10Analytics, 10Operations, 10Wikimedia-Stream, 10Services (watching): Eventstreams build is broken - https://phabricator.wikimedia.org/T216184 (10Pchelolo) > KafkaSSE requires ^2.3.4. 2.5.1 satisfies `^2.3.4` :) I think we should lock the node-rdkafka dependency either by removing the `^` or by adding a p... [19:58:18] 10Analytics, 10Readers-Web-Backlog (Tracking): [Bug] Many ReadingDepth validation errors logged - https://phabricator.wikimedia.org/T216063 (10Jdlrobson) Unless I'm misunderstanding this, I'm assuming this is a problem with the ingestion not the delivery? [20:06:11] I'm losing my mind over sqooping some mediawiki data into hive. anyone available to help debug? [20:08:28] I've sqooped successfully before but having a problem right now and I can't figure out what the cause is [20:10:39] bearloga: yes, and I have also lost my mind over that, let's see if I can help [20:11:41] milimetric: awesome! i'll follow-up in private [20:11:48] and thank you! [20:12:12] that's what we're here for, of course [20:12:14] milimetric, bearloga - If you need another pair of eyes, ping me :) [20:28:44] 10Analytics, 10Operations, 10Wikimedia-Stream, 10Services (watching): Eventstreams build is broken - https://phabricator.wikimedia.org/T216184 (10Ottomata) Can we do a package-lock in the EventStreams repo? [20:30:58] 10Analytics, 10Operations, 10Wikimedia-Stream, 10Services (watching): Eventstreams build is broken - https://phabricator.wikimedia.org/T216184 (10Pchelolo) It's still undecided what to do with package-lock (T179229), so maybe let's just freeze the verison? [20:31:34] 10Analytics, 10Readers-Web-Backlog (Tracking): [Bug] Many ReadingDepth validation errors logged - https://phabricator.wikimedia.org/T216063 (10Ottomata) Hm, unless I'm misunderstanding, this is a problem with the proxy sending bad request data? [20:34:19] 10Analytics, 10Readers-Web-Backlog (Tracking): [Bug] Many ReadingDepth validation errors logged - https://phabricator.wikimedia.org/T216063 (10Jdlrobson) >>! In T216063#4966193, @Ottomata wrote: > Hm, unless I'm misunderstanding, this is a problem with the proxy sending bad request data? Then is this a duplic... [20:41:57] 10Analytics-EventLogging, 10Analytics-Kanban, 10EventBus, 10Product-Analytics, and 5 others: Modern Event Platform: Schema Guidelines and Conventions - https://phabricator.wikimedia.org/T214093 (10Ottomata) We need to do some work in Refine (and hopefully eventually with Kafka Connect) to support map types... [20:45:09] 10Analytics, 10Scoring-platform-team: [Discuss] ORES model development and deployment processes - https://phabricator.wikimedia.org/T216246 (10Halfak) Stat machines are not used to deploy to prod for #ores. [20:59:46] 10Analytics-EventLogging, 10Analytics-Kanban, 10EventBus, 10Product-Analytics, and 5 others: Modern Event Platform: Schema Guidelines and Conventions - https://phabricator.wikimedia.org/T214093 (10EBernhardson) >>! In T214093#4966236, @Ottomata wrote: >> Adding a has_cookies boolean field could be useful... [21:09:52] 10Analytics, 10EventBus, 10Wikimedia-production-error: extensions/EventBus/includes/EventBusRCFeedEngine.php:45 PHP Notice: Undefined index: eventServiceName - https://phabricator.wikimedia.org/T216561 (10thcipriani) [21:10:55] 10Analytics, 10EventBus, 10Wikimedia-production-error: extensions/EventBus/includes/EventBusRCFeedEngine.php:45 PHP Notice: Undefined index: eventServiceName - https://phabricator.wikimedia.org/T216561 (10Ottomata) Ah! Didn't realize that would cause a problem. Can fix, will add a check... [21:11:01] 10Analytics, 10EventBus, 10Wikimedia-production-error: extensions/EventBus/includes/EventBusRCFeedEngine.php:45 PHP Notice: Undefined index: eventServiceName - https://phabricator.wikimedia.org/T216561 (10Ottomata) a:03Ottomata [21:25:42] 10Analytics-EventLogging, 10Analytics-Kanban, 10EventBus, 10Product-Analytics, and 5 others: Modern Event Platform: Schema Guidelines and Conventions - https://phabricator.wikimedia.org/T214093 (10mobrovac) The http schema looks good, much simpler than the first version yet clear enough. For lowercasing he... [21:26:25] 10Analytics, 10Operations, 10Wikimedia-Stream, 10Services (watching): Eventstreams build is broken - https://phabricator.wikimedia.org/T216184 (10mobrovac) +1 on freezing the version in package.json in this instance, as this is what we really need. [21:33:28] 10Analytics-EventLogging, 10Analytics-Kanban, 10EventBus, 10Product-Analytics, and 5 others: Modern Event Platform: Schema Guidelines and Conventions - https://phabricator.wikimedia.org/T214093 (10Pchelolo) > EventGate, so that all parts of the system have a standard representation of the headers. Very mu... [21:38:23] 10Analytics-EventLogging, 10Analytics-Kanban, 10EventBus, 10Product-Analytics, and 5 others: Modern Event Platform: Schema Guidelines and Conventions - https://phabricator.wikimedia.org/T214093 (10mobrovac) >>! In T214093#4966568, @Pchelolo wrote: > Very much disagreed... How would you flag properties that... [21:39:54] 10Analytics-EventLogging, 10Analytics-Kanban, 10EventBus, 10Product-Analytics, and 5 others: Modern Event Platform: Schema Guidelines and Conventions - https://phabricator.wikimedia.org/T214093 (10Pchelolo) > Rejecting events that haven't lowercased the headers is another option, albeit not as user-friendl... [21:41:16] 10Analytics-EventLogging, 10Analytics-Kanban, 10EventBus, 10Product-Analytics, and 5 others: Modern Event Platform: Schema Guidelines and Conventions - https://phabricator.wikimedia.org/T214093 (10EBernhardson) >>! In T214093#4966626, @Pchelolo wrote: >> Rejecting events that haven't lowercased the headers... [21:43:01] 10Analytics-EventLogging, 10Analytics-Kanban, 10EventBus, 10Product-Analytics, and 5 others: Modern Event Platform: Schema Guidelines and Conventions - https://phabricator.wikimedia.org/T214093 (10Ottomata) I'd really prefer if we kept as much transformation logic out of EventGate as possible. Right now i... [21:47:39] 10Analytics-EventLogging, 10Analytics-Kanban, 10EventBus, 10Product-Analytics, and 5 others: Modern Event Platform: Schema Guidelines and Conventions - https://phabricator.wikimedia.org/T214093 (10Ottomata) I guess an annoying bit about ^ is that clients will have to fill in the header information manually... [21:53:51] 10Analytics-EventLogging, 10Analytics-Kanban, 10EventBus, 10Product-Analytics, and 5 others: Modern Event Platform: Schema Guidelines and Conventions - https://phabricator.wikimedia.org/T214093 (10Ottomata) > technically there could be 2 different headers that differ only by _ vs -. Good thing we are doin... [22:33:44] 10Analytics, 10EventBus, 10Services (next): EventBusRCFeedFormatter should clean up events from nulls - https://phabricator.wikimedia.org/T216567 (10Pchelolo) [22:37:10] 10Analytics, 10CirrusSearch, 10EventBus, 10WMF-JobQueue, and 4 others: EventBus or CirrusSearch: DomainException from line 353 of /srv/mediawiki/php-1.33.0-wmf.9/vendor/firebase/php-jwt/src/JWT.php: Unknown JSON error: 5 - https://phabricator.wikimedia.org/T212335 (10mobrovac) a:05mobrovac→03None >>! I... [22:37:48] 10Analytics, 10EventBus, 10Services (next): EventBusRCFeedFormatter should clean up events from nulls - https://phabricator.wikimedia.org/T216567 (10Ottomata) Ah! Nice idea. [22:38:30] 10Analytics, 10EventBus, 10Core Platform Team (Modern Event Platform (TEC2)), 10Core Platform Team Backlog (Later), 10Services (next): EventBusRCFeedFormatter should clean up events from nulls - https://phabricator.wikimedia.org/T216567 (10mobrovac) [22:51:15] In beeline I'm getting [22:51:21] Error: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=2) [22:51:29] Any hints for what I'm doing wrong/what that means [22:56:43] seemed to go away when i rewrote my query to not use the IN operator [23:50:17] 10Analytics, 10Dumps-Generation, 10Wikidata: Update wikidata-entities dump generation to fixed day-of-month instead of fixed weekday - https://phabricator.wikimedia.org/T216160 (10Melderick) Hello, As a json dump consumer, I have a script run by crontab to retrieve each week the latest dump. Usually the down...