[00:23:16] (03PS4) 10Nuria: Display exact period of time in widgets instead of "last year" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/493260 (https://phabricator.wikimedia.org/T216105) (owner: 10Fdans) [00:23:41] (03CR) 10Nuria: Display exact period of time in widgets instead of "last year" (032 comments) [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/493260 (https://phabricator.wikimedia.org/T216105) (owner: 10Fdans) [00:25:36] (03CR) 10Nuria: "See if my patch makes sense. I think common functionality like this one it is better done as a filter available app wide.Also, when possib" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/493260 (https://phabricator.wikimedia.org/T216105) (owner: 10Fdans) [02:05:20] 10Analytics, 10Knowledge-Integrity, 10Research, 10Epic, 10Patch-For-Review: Citation Usage: run third round of data collection - https://phabricator.wikimedia.org/T213969 (10RyanSteinberg) Sorry guys, it's hard for me to find time for this project during the week. New review: https://github.com/ryanmax/... [06:22:06] 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review, 10User-Elukey: rack/setup/install labsdb1012.eqiad.wmnet - https://phabricator.wikimedia.org/T215231 (10Marostegui) The data transfer from labsdb1011 to labsdb1012 finished. I have deleted all the following files on labsdb1012 before starting my... [06:59:38] joal: --^ [06:59:41] bonjour! [06:59:51] I have a new cloud-db replica ready for you :D [07:00:08] (thanks to Saint Manuel from Mardrid, protector of the Analytics people) [07:00:15] *Madrid [07:10:36] the current conn limit for our user is 100 [07:10:42] let's see if it is enough or not :) [07:13:55] 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review, 10User-Elukey: rack/setup/install labsdb1012.eqiad.wmnet - https://phabricator.wikimedia.org/T215231 (10elukey) 05Open→03Resolved [07:16:08] 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review: Decommission dbstore1002 - https://phabricator.wikimedia.org/T216491 (10elukey) [07:17:47] 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review: Decommission dbstore1002 - https://phabricator.wikimedia.org/T216491 (10elukey) No complaints or outages after the shutdown of dbstore1002, I think that we are good to keep going with the decom. "@Marostegui this is control tower, you are clear t... [07:22:00] 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review: Decommission dbstore1002 - https://phabricator.wikimedia.org/T216491 (10Marostegui) \o/ Clear to decommission! :) [07:37:29] 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review: Decommission dbstore1002 - https://phabricator.wikimedia.org/T216491 (10Marostegui) [07:38:00] dbstore1002 officially decommed [07:39:00] 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review: Decommission dbstore1002 - https://phabricator.wikimedia.org/T216491 (10Marostegui) a:03RobH Ready for @RobH to do the next steps. @RobH can you give this some priority for the steps that include the power down of this host? It is a trusty host... [07:55:04] (03CR) 10Elukey: [V: 03+2] Add the 'centralauth' db use case in refinery utils.py [analytics/refinery] - 10https://gerrit.wikimedia.org/r/494496 (https://phabricator.wikimedia.org/T212386) (owner: 10Elukey) [08:02:00] morning elukey - kids day today ;) And many thanks to Saint Manuel protector of the Analtics people [08:02:14] I'll start testing this evening, see about perf and all [08:19:46] joal: yep yep! [08:19:50] I was only reporting :) [08:20:06] we can in theory tune the conn limit as we want [08:38:22] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 2 others: Replace the current multisource analytics-store setup - https://phabricator.wikimedia.org/T172410 (10Marostegui) [08:38:28] 10Analytics, 10Analytics-Kanban, 10DBA, 10Patch-For-Review, 10User-Banyek: Migrate dbstore1002 to a multi instance setup on dbstore100[3-5] - https://phabricator.wikimedia.org/T210478 (10Marostegui) 05Open→03Resolved I am going to close this as resolved. Everything is essentially already done as we h... [08:38:40] 10Analytics, 10Analytics-Kanban, 10DBA, 10Patch-For-Review, 10User-Banyek: Migrate dbstore1002 to a multi instance setup on dbstore100[3-5] - https://phabricator.wikimedia.org/T210478 (10Marostegui) [08:39:58] 10Analytics, 10Analytics-Cluster: Upgrade Hive to ≥ 2.0 - https://phabricator.wikimedia.org/T203498 (10elukey) >>! In T203498#5001047, @Neil_P._Quinn_WMF wrote: > It looks like CDH 6.1, which [includes](https://www.cloudera.com/documentation/enterprise/6/release-notes/topics/rg_cdh_61_packaging.html) Hive 2.1.... [08:41:08] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 3 others: Provide tools for querying MediaWiki replica databases without having to specify the shard - https://phabricator.wikimedia.org/T212386 (10elukey) [08:41:42] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 3 others: Provide tools for querying MediaWiki replica databases without having to specify the shard - https://phabricator.wikimedia.org/T212386 (10elukey) >>! In T212386#4974798, @elukey wrote: > All good if we close this task... [08:42:00] 10Analytics, 10Analytics-Kanban, 10User-Marostegui: Migrate users to dbstore100[3-5] - https://phabricator.wikimedia.org/T215589 (10elukey) [08:42:10] 10Analytics, 10Product-Analytics, 10Research, 10WMDE-Analytics-Engineering, and 3 others: Provide tools for querying MediaWiki replica databases without having to specify the shard - https://phabricator.wikimedia.org/T212386 (10elukey) 05Open→03Resolved [08:42:40] 10Analytics, 10Analytics-Kanban, 10User-Marostegui: Migrate users to dbstore100[3-5] - https://phabricator.wikimedia.org/T215589 (10elukey) [09:07:02] (03PS1) 10WMDE-Fisch: Add reference previews beta feature [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494666 (https://phabricator.wikimedia.org/T215891) [09:07:42] (03CR) 10Addshore: [C: 03+2] Add reference previews beta feature [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494666 (https://phabricator.wikimedia.org/T215891) (owner: 10WMDE-Fisch) [09:07:50] (03Merged) 10jenkins-bot: Add reference previews beta feature [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494666 (https://phabricator.wikimedia.org/T215891) (owner: 10WMDE-Fisch) [09:38:01] 10Analytics, 10Operations, 10vm-requests, 10User-Elukey: Create an-tool1005 (Staging environment for Superset) - https://phabricator.wikimedia.org/T217738 (10elukey) p:05Triage→03Normal [09:45:24] 10Analytics, 10Pageviews-API: Add wikimania.wikimedia.org to pageview definition - https://phabricator.wikimedia.org/T216525 (10Billinghurst) [09:49:15] (03CR) 10Fdans: [C: 03+1] Display exact period of time in widgets instead of "last year" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/493260 (https://phabricator.wikimedia.org/T216105) (owner: 10Fdans) [09:55:20] 10Analytics, 10Operations, 10RESTBase, 10Traffic, and 2 others: Verify that hit/miss stats in WebRequest are correct - https://phabricator.wikimedia.org/T215987 (10ema) Is there anything to do here? :-) [10:01:11] 10Analytics, 10Operations, 10vm-requests, 10Patch-For-Review, 10User-Elukey: Create an-tool1005 (Staging environment for Superset) - https://phabricator.wikimedia.org/T217738 (10elukey) ` If you need a private IP, do you need it to be inside the Analytics VLAN? (y/n) y Please enter the correct row. (A,... [10:05:20] 10Analytics, 10Operations, 10vm-requests, 10Patch-For-Review, 10User-Elukey: Create an-tool1005 (Staging environment for Superset) - https://phabricator.wikimedia.org/T217738 (10elukey) To keep archives happy: @MoritzMuehlenhoff is currently testing the Buster debian installer for Ganeti VMs, so please d... [10:12:23] 10Analytics, 10Patch-For-Review, 10User-Elukey: Staging environment for upgrades of superset - https://phabricator.wikimedia.org/T212243 (10elukey) [10:25:08] 10Analytics, 10Patch-For-Review, 10User-Elukey: Staging environment for upgrades of superset - https://phabricator.wikimedia.org/T212243 (10elukey) [10:25:30] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Staging environment for upgrades of superset - https://phabricator.wikimedia.org/T212243 (10elukey) [10:29:49] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Move AQS to nodejs 10 - https://phabricator.wikimedia.org/T210706 (10elukey) Restarted aqs in deployment-prep, we are using nodejs 10 in there. @Milimetric let's chat about the next steps whenever you have time, I think we are really close to deploy! [10:41:44] 10Analytics, 10Research, 10Article-Recommendation, 10Patch-For-Review: Generate article recommendations in Hadoop for use in production - https://phabricator.wikimedia.org/T210844 (10jbond) [11:01:28] (03PS1) 10Ladsgroup: Rewrite user_langauges.php to use babel table and Inclusion–exclusion principle [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494692 (https://phabricator.wikimedia.org/T213894) [11:01:39] (03CR) 10Ladsgroup: [C: 03+2] Rewrite user_langauges.php to use babel table and Inclusion–exclusion principle [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494692 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [11:01:47] (03Merged) 10jenkins-bot: Rewrite user_langauges.php to use babel table and Inclusion–exclusion principle [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494692 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [11:12:11] (03PS1) 10Ladsgroup: Remove WikimediaDb::getPdo() [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494693 (https://phabricator.wikimedia.org/T213894) [11:12:32] (03CR) 10Ladsgroup: [C: 03+2] Remove WikimediaDb::getPdo() [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494693 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [11:12:40] (03Merged) 10jenkins-bot: Remove WikimediaDb::getPdo() [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494693 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [11:38:12] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Superset's rolling average feature results in error message - https://phabricator.wikimedia.org/T213488 (10elukey) @jlinehan I am currently half way through the plan for the staging environment, one thing that we could try to do is to test https://pypi.or... [11:41:57] 10Analytics-Kanban, 10Product-Analytics: Superset Updates - https://phabricator.wikimedia.org/T211706 (10elukey) [11:43:11] Amir1: o/ - all good with the staging db? Did you figure out the issue? [11:43:42] elukey: yup, it was that the PDO got stale so the connection would go away [11:43:47] fixed and everything is back [11:43:53] \o/ [11:43:55] nice [11:44:00] glad that everything is working [11:44:01] basically we are not reading from dbstore1002 anymore [11:44:06] at all [11:44:09] super [11:44:32] (I actually went ahead and rewrote some queries to be way faster) [11:44:42] hahahah [11:46:31] 10Analytics, 10Operations, 10Documentation, 10Patch-For-Review: Remove data from Hadoop's HDFS as part of the user offboard workflow - https://phabricator.wikimedia.org/T200312 (10elukey) 05Open→03Resolved [11:49:03] Amir1: woo! [11:49:28] joal: around today? :) [11:50:38] addshore: I particularly love the babel rewrite, the old code did it through categorylinks but I changed to use babel table (I guess it wasn't there back then) [11:51:04] hahahahahhaaa [11:51:55] addshore: kids day, he'll be available in the evening :) [11:52:25] ack! :) [12:33:46] 10Analytics, 10Analytics-Kanban: Test sqooping from the new dedicated labsdb host - https://phabricator.wikimedia.org/T215550 (10elukey) [12:34:27] Cc: milimetric, nuria - re added it --^ to the kanban [12:34:37] we are ready to test :) [12:44:02] * elukey lunch! [13:06:48] addshore: depending on your request, I might help now, but won't have more than 1/2h I think (kids will wake up soon) [14:21:48] 10Analytics, 10Analytics-Cluster: Upgrade Hive to ≥ 2.0 - https://phabricator.wikimedia.org/T203498 (10Ottomata) > I see migrating to CDH6 as long term goal/plan. CDH6 might be the goal, but if ends up being a brand new install (no clear upgrade path), I think it would still be worth considering other distr... [14:25:02] 10Analytics, 10Analytics-Cluster: Upgrade Hive to ≥ 2.0 - https://phabricator.wikimedia.org/T203498 (10elukey) >>! In T203498#5005248, @Ottomata wrote: >> I see migrating to CDH6 as long term goal/plan. > > CDH6 might be the goal, but if ends up being a brand new install (no clear upgrade path), I think it... [14:37:14] 10Analytics, 10Analytics-Cluster: Upgrade Hive to ≥ 2.0 - https://phabricator.wikimedia.org/T203498 (10Ottomata) Oh interesting right, I think you told me all of ^. Cool. :) [14:41:21] ottomata: o/ [14:41:27] morninggg [14:41:45] I have a cergen question.. are keystores/trustores passwordless? [14:42:34] mmmm probably not, I am now reading the code [14:43:32] Luca, why are you asking? [14:43:40] I'll explain! [14:43:41] :P [14:43:48] so ssl-client/server ask for [14:43:55] 1) keystore key password [14:44:00] 2) keystore password [14:44:04] 3) trustore password [14:48:16] but they can be none (default) [14:48:25] so I probably need to amend my puppet code [14:48:37] I think that only the keystore makes sense to have a pass no? [14:51:14] (03PS1) 10Mforns: Connect to the right shard when querying wiki databases [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) [14:52:35] (03CR) 10jerkins-bot: [V: 04-1] Connect to the right shard when querying wiki databases [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [14:56:07] (03PS2) 10Mforns: Connect to the right shard when querying wiki databases [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) [14:56:41] (03CR) 10Elukey: Connect to the right shard when querying wiki databases (031 comment) [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [14:57:36] (03CR) 10jerkins-bot: [V: 04-1] Connect to the right shard when querying wiki databases [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [14:58:40] (03PS3) 10Mforns: Connect to the right shard when querying wiki databases [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) [14:59:29] (03CR) 10Mforns: Connect to the right shard when querying wiki databases (031 comment) [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [14:59:55] (03CR) 10jerkins-bot: [V: 04-1] Connect to the right shard when querying wiki databases [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [15:00:58] thanks elukey, fixing unit tests now [15:13:20] (03PS4) 10Mforns: Connect to the right shard when querying wiki databases [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) [15:16:44] (03CR) 10jerkins-bot: [V: 04-1] Connect to the right shard when querying wiki databases [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [15:16:59] joal: it was regarding more queries about the strings of labels aliases and description in wikidata, but the question can wait until later ! [15:40:00] (03PS1) 10Mforns: Adapt codebase for new flake8 upgrade [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494752 (https://phabricator.wikimedia.org/T215289) [15:43:53] (03CR) 10jerkins-bot: [V: 04-1] Adapt codebase for new flake8 upgrade [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494752 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [15:45:39] (03PS2) 10Mforns: Adapt codebase for new flake8 upgrade [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494752 (https://phabricator.wikimedia.org/T215289) [15:45:49] (03CR) 10Mforns: "recheck" [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494752 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [15:46:04] (03CR) 10Addshore: [C: 04-1] "needs rebase" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494194 (owner: 10Thiemo Kreuz (WMDE)) [15:46:13] (03PS2) 10Addshore: Add MediaWiki's PHP CodeSniffer rule set [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494190 (owner: 10Thiemo Kreuz (WMDE)) [15:46:38] (03CR) 10Addshore: "perhaps we should make ci run here?" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494190 (owner: 10Thiemo Kreuz (WMDE)) [15:47:42] elukey: o/ [15:47:48] don't get your first comment response [15:49:32] (03PS3) 10Mforns: Adapt codebase for new flake8 upgrade [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494752 (https://phabricator.wikimedia.org/T215289) [15:49:52] (03CR) 10Mforns: "recheck" [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494752 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [15:50:45] ottomata: of the hadoop patch? [15:50:49] ya [15:50:52] ah ok sorry [15:51:17] so what I wanted to say is that if one of the parameter in the hash is not added, then ssl-client/server.xml will not have it [15:51:25] getting the default [15:51:47] so say if you specify only one password then it will be the only one rendered in the config [15:52:09] after a bit of thought I think that we'd only need the keystore pass [15:52:30] so if we specify only that, than we are good [15:52:48] with the possbility of adding the others if needed (say a security review tells us xyz) [15:55:23] does it make sense? [15:55:36] I need to upgrade the docs though [15:56:10] (03CR) 10Mforns: "The jenkins -1 is due to flake8 errors after its upgrade." [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [15:57:02] (03CR) 10Mforns: "@Milimetric, do you think this needs more tests?" [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [15:58:27] we can discuss it during the ops meeting, I have a couple of questions to ask about the certs etc.. [16:01:04] mforns: I am trying to make it through today with a headache, I will try to get to your patch, but no promises [16:01:22] milimetric, don't worry, please rest! [16:01:26] if someone else can review, that'd be great, and if you want to merge and try I'm sure it's great [16:01:38] yes, someone else can! [16:01:39] I wish I could rest, but I have too many meetings I can't back out of [16:02:00] well, you don't have to spend time with this CR at least [16:02:43] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Superset's rolling average feature results in error message - https://phabricator.wikimedia.org/T213488 (10Nuria) Once staging is setup let's go over issues users have opened for 0.29 and see how things look. [16:02:55] 10Analytics, 10Operations, 10vm-requests, 10Patch-For-Review, 10User-Elukey: Create a ganeti VM identical to analytics-tool1003 with Debian Buster - https://phabricator.wikimedia.org/T217640 (10elukey) [16:03:26] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Staging environment for upgrades of superset - https://phabricator.wikimedia.org/T212243 (10elukey) [16:03:29] 10Analytics, 10Operations, 10vm-requests, 10Patch-For-Review, 10User-Elukey: Create a ganeti VM identical to analytics-tool1003 with Debian Buster - https://phabricator.wikimedia.org/T217640 (10elukey) 05Open→03Resolved analytics-tool1004 is up and running, will close this task now and re-open anothe... [16:08:28] 10Analytics, 10Operations, 10decommission, 10ops-eqiad: Decommission dbstore1002 - https://phabricator.wikimedia.org/T216491 (10RobH) [16:10:30] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Superset's rolling average feature results in error message - https://phabricator.wikimedia.org/T213488 (10jlinehan) > Thoughts? I believe that this task can be closed, lemme know if there are other steps todo/missing. Yeah, since the bug this task is re... [16:13:43] mforns, fdans : if any of you can merge this today, we can deploy and fix a very visible bug on wikistats with chu-chu train: https://gerrit.wikimedia.org/r/#/c/analytics/wikistats2/+/493260 [16:14:08] (03CR) 10Fdans: [C: 03+2] Display exact period of time in widgets instead of "last year" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/493260 (https://phabricator.wikimedia.org/T216105) (owner: 10Fdans) [16:14:23] nuria: merged! [16:14:42] joal: yt? [16:18:34] fdans: grasias [16:19:58] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Superset's rolling average feature results in error message - https://phabricator.wikimedia.org/T213488 (10elukey) [16:31:19] (03CR) 10Milimetric: "just a couple of thoughts. Looks good. I noticed you got rid of some config validation and testing for it, but I don't feel strongly abo" (033 comments) [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [16:32:58] thx milimetric ! [16:40:31] (03PS1) 10Tonina Zhelyazkova: Count enables/disables for rollback conf prompt user setting [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494772 (https://phabricator.wikimedia.org/T215796) [16:45:06] (03PS2) 10Tonina Zhelyazkova: Count enables/disables for rollback conf prompt user setting [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494772 (https://phabricator.wikimedia.org/T215796) [16:45:50] (03PS5) 10Mforns: Connect to the right shard when querying wiki databases [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) [16:46:14] (03CR) 10Mforns: Connect to the right shard when querying wiki databases (033 comments) [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [16:48:28] (03CR) 10jerkins-bot: [V: 04-1] Connect to the right shard when querying wiki databases [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [16:59:15] (03Abandoned) 10Neil P. Quinn-WMF: Swap archaïc code to use current tables [analytics/limn-edit-data] - 10https://gerrit.wikimedia.org/r/397628 (owner: 10Jforrester) [17:00:37] ping ottomata , joal standddupppp [17:02:02] srry [17:06:58] (03PS1) 10Elukey: Add analytics-tool1004 to the scap target list [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/494781 [17:07:34] (03CR) 10Elukey: [V: 03+2 C: 03+2] Add analytics-tool1004 to the scap target list [analytics/superset/deploy] - 10https://gerrit.wikimedia.org/r/494781 (owner: 10Elukey) [17:07:56] (03CR) 10Milimetric: [V: 03+2 C: 03+2] Connect to the right shard when querying wiki databases [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494734 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [17:19:09] (03CR) 10Thiemo Kreuz (WMDE): "Sure. Feel free. For now I was running this locally. This is still valuable, in my humble opinion, even without the CI running PHPCS." [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494190 (owner: 10Thiemo Kreuz (WMDE)) [17:30:29] ottomata: qq! so in the superset deploy repo with have artifacts/stretch [17:30:40] the stretch part is created via profile.sh [17:30:44] now I need buster of course :D [17:30:57] do you see any reason why not reusing the current stretch wheels? [17:31:15] namely - do I need to ./build_wheels.sh on a buster node? [17:31:37] or just copy artifacts/stretch/* to artifacts/buster/ ? [17:34:05] elukey, mforns: are the patches you need to be deployed already merged? [17:34:12] mine is! [17:34:17] ack! [17:34:20] joal, yes mine too [17:34:24] ack++ :) [17:34:29] ;] [17:37:34] 10Analytics, 10Analytics-Kanban, 10Analytics-SWAP, 10Product-Analytics, 10Patch-For-Review: Publishing html files generated on notebook hosts - https://phabricator.wikimedia.org/T217619 (10Ottomata) @chelsyx you should now be able to put files in /srv/published-datasets on notebook hosts. Files there wi... [17:41:03] 10Analytics, 10Operations, 10decommission, 10ops-eqiad: Decommission dbstore1002 - https://phabricator.wikimedia.org/T216491 (10ops-monitoring-bot) wmf-decommission-host was executed by robh for dbstore1002.eqiad.wmnet and performed the following actions: - Revoked Puppet certificate - Removed from PuppetD... [17:41:15] 10Analytics, 10Operations, 10decommission, 10ops-eqiad: Decommission dbstore1002 - https://phabricator.wikimedia.org/T216491 (10RobH) [17:47:03] 10Analytics, 10Operations, 10decommission, 10ops-eqiad: Decommission dbstore1002 - https://phabricator.wikimedia.org/T216491 (10RobH) a:05RobH→03Cmjohnson [18:07:46] Wow - refinery is long to deploy nowadays (31mins) [18:08:11] !log Refinery deployed using scap [18:08:13] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [18:08:16] joal: a lot of nodes :D [18:08:22] true, true [18:08:29] elukey: could we do them in parallel? [18:09:20] joal: there should be the scap option to do all of them after the canary no? [18:09:25] if not we can surely check [18:09:43] elukey: when scap asked if I wanted to do them all, I said yes :) [18:09:56] And it all went, but not sure if it has been done in parallel or in sequence [18:10:44] mmm weird [18:10:52] I think it should have been all in parallel in theory [18:11:00] but 31 mins are a lot [18:11:08] maybe one of the host was very bus :) [18:11:12] +y [18:11:25] could be yes [18:12:23] (03CR) 10Elukey: [C: 03+1] "LGTM! Left one nit but not a blocker :)" (031 comment) [analytics/reportupdater] - 10https://gerrit.wikimedia.org/r/494752 (https://phabricator.wikimedia.org/T215289) (owner: 10Mforns) [18:13:53] !log Refinery deployed onto hadoop [18:13:54] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [18:23:24] milimetric: any idea aoubt why I encounter failures when trying to access labswiki on either analytics-mysql or labsdb, whenlabswiki is the wikitech dbname in sitematrix? [18:23:59] joal: I don't remember exactly, but yes, that db is super weird [18:24:08] there's something different about it, I forget what [18:24:18] it may not be replicated to the replicas [18:24:22] right [18:24:30] I'd ask in -cloud [18:24:37] this explains that then :) Meaning we won't get wikitech in WKS2 [18:24:45] potentially not, yeah [18:24:59] well, but it might just be somewhere else that's easy to get to if we configure it [18:28:01] milimetric: https://phabricator.wikimedia.org/T167973 [18:28:19] We need to wait for that one to be done before being able to sqoop from it [18:28:25] I'm gonna create a task [18:29:36] joal: makes sense, and thanks for the history lesson [18:29:46] I knew of the existence of the history, not the content [18:29:47] :) [18:30:21] milimetric: cloud-team hyper-knowlegeable - as usual :) [18:31:26] 10Analytics: Add wikitech (labswiki) to the sqoop list - https://phabricator.wikimedia.org/T217792 (10JAllemandou) [18:32:52] 10Analytics: Add wikitech (labswiki) to the sqoop list - https://phabricator.wikimedia.org/T217792 (10JAllemandou) [18:34:13] 10Analytics: Add wikitech (labswiki) to the sqoop list - https://phabricator.wikimedia.org/T217792 (10JAllemandou) [18:34:48] nuria: so that you know --> https://phabricator.wikimedia.org/T217792 [18:39:56] elukey: still here? [18:42:25] heya addshore - sorry forgot to ping you before [18:45:09] joal: yep [18:45:32] elukey: sorry to bother - I don't manage to access labsdb1012 :( [18:45:41] elukey: I use the labs user we use for sqooping [18:45:48] connection times out [18:46:14] can you telnet fine to the port? [18:46:43] 3306, right? [18:46:59] nope [18:47:30] host is found, but no connection [18:49:35] elukey: could be a firewall issue? [18:50:24] checking [18:51:33] yeah 3306 [18:52:23] ah snap joal [18:52:30] from what host are you running the sqoop? [18:52:55] an-coord1001, but I was testing from stat1004 [18:53:22] THE GREAT FIREWALL IS WATCHING US [18:53:23] :D [18:53:30] :) [18:53:53] lemme see if I can fix it [19:00:35] joal: what about now? [19:00:57] what the hell still not working [19:01:23] I am pretty sure that there might be some VLAN trick to do [19:03:15] I think that the dbproxies are the only hosts allowed to talk with the labsdb VLAN [19:03:25] meanwhile we are attempting to do a direct connection [19:05:02] Mwahh :( [19:05:05] That would explain [19:05:23] elukey: It's late for you, let's do that tomorrowe? [19:06:11] I am checking with Arzhel quickly, it will probably need some more work :( [19:06:18] I didn't think about firewallssss [19:06:30] elukey: no big deal, will test tomorrow ;) [19:06:39] or the day after :) [19:10:07] joal: iptables rules [19:10:52] * joal runs to the nearest hidding place [19:10:56] I'll have to ask to Jaime/Manuel [19:11:49] 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review, 10User-Elukey: rack/setup/install labsdb1012.eqiad.wmnet - https://phabricator.wikimedia.org/T215231 (10elukey) 05Resolved→03Open [19:13:17] 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review, 10User-Elukey: rack/setup/install labsdb1012.eqiad.wmnet - https://phabricator.wikimedia.org/T215231 (10elukey) I added the following bit on cr1/cr2: ` elukey@re0.cr1-eqiad# show | compare [edit firewall family inet filter analytics-in4 term my... [19:13:21] joal: to limit the scope, do you think that only allowing an-coord1001 is enough? [19:13:45] elukey: given analytics should be the only users as of now, yes [19:14:11] elukey: however if at some point we want to provide sqooping capacities for others from that host, well it'll have to be more :) [19:14:27] for now an-coord1001 is fine for me [19:15:05] I think that the cleanest solution is probably a special dbproxy domain [19:15:09] 10Analytics, 10Analytics-EventLogging: ChangesListFilters events invalid because of integer namespace field - https://phabricator.wikimedia.org/T196903 (10Catrope) 05Open→03Resolved a:03Catrope Done by {9c2ba0cb38c71c72d639535f60baeb186c2e5ae4} (and then obviated by {T210907}) [19:15:19] so we'd go through it and avoid any special firewall [19:15:29] seems legit elukey! [19:15:42] 10Analytics: Handle long-term retention of ChangesListFilters and ChangesListFilterGrouping schemas - https://phabricator.wikimedia.org/T185009 (10Catrope) 05Open→03Invalid Since {T210907} is now done, this is no longer needed [19:15:48] 10Analytics, 10DBA, 10Operations, 10ops-eqiad, and 2 others: rack/setup/install labsdb1012.eqiad.wmnet - https://phabricator.wikimedia.org/T215231 (10elukey) [19:16:05] elukey: we'll name it: domain-of-the-beast :) [19:16:55] 10Analytics, 10DBA, 10Operations, 10ops-eqiad, and 2 others: rack/setup/install labsdb1012.eqiad.wmnet - https://phabricator.wikimedia.org/T215231 (10elukey) @jcrespo @Marostegui thoughts? What would it be best in your opinion? I'd prefer another dbproxy-based domain but not sure how complicated to create/... [19:17:13] done --^ [19:18:52] great :) [19:19:07] Thanks elukey see you tomorrow ;) [19:20:00] o/ [19:20:02] * elukey off! [19:36:17] (03CR) 10Thiemo Kreuz (WMDE): "I don't think it makes sense to rebase this when none of the patches this is based on is merged yet. Please review and +2 this first. I wi" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494194 (owner: 10Thiemo Kreuz (WMDE)) [19:36:18] (03CR) 10Thiemo Kreuz (WMDE): "I don't think it makes sense to rebase this when none of the patches this is based on is merged yet. Please review and +2 this first. I wi" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494194 (owner: 10Thiemo Kreuz (WMDE)) [20:00:09] 10Analytics, 10Product-Analytics: Standardize datetimes/timestamps in the Data Lake - https://phabricator.wikimedia.org/T212529 (10Neil_P._Quinn_WMF) >>! In T212529#4981128, @Ottomata wrote: > Thanks Neil! > > The general convention we've been trying to move forward to over the years is: > > - .*dt fields ar... [20:13:02] 10Analytics, 10Knowledge-Integrity, 10Research, 10Epic, 10Patch-For-Review: Citation Usage: run third round of data collection - https://phabricator.wikimedia.org/T213969 (10bmansurov) > If so, perhaps instead of relying on markup for this notion of external-ness, we should consider calculating it based... [20:21:10] 10Analytics, 10Analytics-Cluster: Upgrade Hive to ≥ 2.0 - https://phabricator.wikimedia.org/T203498 (10Neil_P._Quinn_WMF) @Ottomata and @elukey, thanks for the great context! Everything you said makes sense, so I'll keep in mind not to expect this anytime soon 😁 >! In T203498#5004073, @elukey wrote: >What we... [21:02:01] (03CR) 10Ottomata: Update LogHelper for parameterization (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/493237 (owner: 10Joal) [21:02:37] (03CR) 10Ottomata: Add JsonSchemaConverter to spark package (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/492756 (https://phabricator.wikimedia.org/T215442) (owner: 10Joal) [21:03:01] (03CR) 10Ottomata: Add JsonSchemaConverter to spark package (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/492756 (https://phabricator.wikimedia.org/T215442) (owner: 10Joal) [22:10:22] (03PS9) 10Ottomata: Event(Logging) schema loader [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/492399 (https://phabricator.wikimedia.org/T215442) [22:11:56] joal: I'll catch you during the EU day :) [22:12:04] (03PS1) 10Ottomata: Add SparkSchemaLoader capabilities to Refine and RefineTarget [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/494831 (https://phabricator.wikimedia.org/T215442) [22:18:00] 10Analytics, 10Analytics-Kanban, 10EventBus, 10MW-1.33-notes (1.33.0-wmf.21; 2019-03-12), 10Patch-For-Review: Make Refine use JSONSchemas of event data to support Map types and proper types for integers vs decimals - https://phabricator.wikimedia.org/T215442 (10Ottomata) [22:24:03] (03PS2) 10Ottomata: Add SparkSchemaLoader capabilities to Refine and RefineTarget [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/494831 (https://phabricator.wikimedia.org/T215442) [23:13:44] 10Analytics, 10Operations, 10decommission, 10ops-eqiad, and 2 others: Decommission analytics100[1,2] - https://phabricator.wikimedia.org/T205507 (10RobH) analytics1001:asw2-c-eqiad:ge-4/0/16 analytics1002:asw2-d-eqiad:ge-8/0/7 [23:19:23] 10Analytics, 10Operations, 10decommission, 10ops-eqiad, and 2 others: Decommission analytics100[1,2] - https://phabricator.wikimedia.org/T205507 (10RobH) ` [edit interfaces interface-range vlan-analytics1-d-eqiad] member xe-7/0/3 { ... } + member ge-9/0/4; + member ge-9/0/5; + member ge-9/0/6... [23:20:45] 10Analytics, 10Operations, 10decommission, 10ops-eqiad, and 2 others: Decommission analytics100[1,2] - https://phabricator.wikimedia.org/T205507 (10ops-monitoring-bot) wmf-decommission-host was executed by robh for analytics1001.eqiad.wmnet and performed the following actions: - Revoked Puppet certificate... [23:21:01] 10Analytics, 10Operations, 10decommission, 10ops-eqiad, and 2 others: Decommission analytics100[1,2] - https://phabricator.wikimedia.org/T205507 (10ops-monitoring-bot) wmf-decommission-host was executed by robh for analytics1002.eqiad.wmnet and performed the following actions: - Revoked Puppet certificate... [23:22:02] 10Analytics, 10Operations, 10decommission, 10ops-eqiad, 10User-Elukey: Decommission analytics100[1,2] - https://phabricator.wikimedia.org/T205507 (10RobH) a:05RobH→03None [23:34:34] 10Analytics, 10Operations, 10decommission, 10ops-eqiad, and 2 others: Decommission analytics100[1,2] - https://phabricator.wikimedia.org/T205507 (10RobH) a:03Cmjohnson [23:38:20] 10Analytics, 10Operations, 10decommission, 10ops-eqiad, 10User-Elukey: Decommission analytics100[1,2] - https://phabricator.wikimedia.org/T205507 (10RobH) [23:50:24] (03CR) 10Nuria: Event(Logging) schema loader (032 comments) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/492399 (https://phabricator.wikimedia.org/T215442) (owner: 10Ottomata) [23:56:11] (03CR) 10Nuria: Add SparkSchemaLoader capabilities to Refine and RefineTarget (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/494831 (https://phabricator.wikimedia.org/T215442) (owner: 10Ottomata)