[01:12:06] <groceryheist>	 Looking for some help with https://phabricator.wikimedia.org/T221890
[02:41:39] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Discovery, and 5 others: Rewrite Avro schemas (ApiAction, CirrusSearchRequestSet) as JSONSchema and produce to EventGate - https://phabricator.wikimedia.org/T214080 (10bd808) >>! In T214080#5132742, @Ottomata wrote: > @bd808 what do we need to do...
[03:48:30] <wikibugs>	 10Analytics, 10Product-Analytics, 10Epic, 10User-Elukey: Add wikidata ids to data lake tables - https://phabricator.wikimedia.org/T221890 (10Nuria) Requested noted, the easiest that we could do for now would be to  sqoop the page_props tables so they are available in hadoop,  We are working on other change...
[04:58:48] <wikibugs>	 10Analytics, 10Product-Analytics, 10Epic, 10User-Elukey: Add wikidata ids to data lake tables - https://phabricator.wikimedia.org/T221890 (10Groceryheist) Thank you Nuria. Are you saying that we'll be able to sqoop the prop_tables in May at the earliest?  Would it be okay to lookup a sizable number of page...
[06:21:41] <elukey>	 Krenair: o/ just seen the ping - no idea about the instance.. is there any trace about who created it? I think that your plan of shutoff + kill is good!
[06:24:51] <wikibugs>	 10Analytics-Kanban, 10Better Use Of Data, 10Product-Analytics: Superset Updates - https://phabricator.wikimedia.org/T211706 (10elukey)
[07:06:14] <joal>	 Good morning :)
[07:08:35] <elukey>	 bonjour!
[07:09:30] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10EventBus, 10Core Platform Team (Modern Event Platform (TEC2)), and 3 others: Schema Registry HTTP Service - https://phabricator.wikimedia.org/T219552 (10elukey)
[07:09:35] <wikibugs>	 10Analytics-Kanban, 10EventBus, 10Operations, 10netops, 10Patch-For-Review: Allow analytics VLAN to reach schema.svc.$site.wmnet - https://phabricator.wikimedia.org/T221690 (10elukey) 05Open→03Resolved To keep archives happy:  ` term schema {     from {         destination-address {             /* sc...
[07:41:45] <wikibugs>	 (03PS3) 10Thiemo Kreuz (WMDE): Update all code to use the short array syntax [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494192
[07:41:48] <wikibugs>	 (03PS3) 10Thiemo Kreuz (WMDE): Apply all trivial auto-fixes to the PHP code style [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494194
[07:41:51] <wikibugs>	 (03PS3) 10Thiemo Kreuz (WMDE): Add missing limits to explode() and fix PHPDoc tags [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494196
[07:43:47] <wikibugs>	 (03PS1) 10Thiemo Kreuz (WMDE): Update MediaWiki CodeSniffer to version 25.0.0 [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/506596
[07:45:47] <wikibugs>	 (03CR) 10Thiemo Kreuz (WMDE): "I manually rebased this chain of (now 4) patches. Do you think it's possible to get a review for this? Who would be able to do this, other" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494194 (owner: 10Thiemo Kreuz (WMDE))
[07:48:46] <wikibugs>	 (03CR) 10Thiemo Kreuz (WMDE): [V: 03+2] "I manually reviewed this again, checked that the rebase really does not contain any unwanted mistakes, and did run phpcs locally again." [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494194 (owner: 10Thiemo Kreuz (WMDE))
[07:49:45] <wikibugs>	 (03CR) 10Thiemo Kreuz (WMDE): [V: 03+2] "I manually reviewed this again, checked that the rebase really does not contain any unwanted mistakes, and did run phpcs locally again." [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494192 (owner: 10Thiemo Kreuz (WMDE))
[08:17:00] <elukey>	 joal: https://github.com/dpgaspar/Flask-AppBuilder/issues/965
[08:17:03] <elukey>	 very interesting
[08:17:42] <joal>	 Nice
[08:18:10] <elukey>	 ah! that code was added by Andrew!
[08:18:17] <elukey>	 https://github.com/dpgaspar/Flask-AppBuilder/pull/663/commits/2a08f21a2d6006c4850a1e6460abc2c00bfa871c
[08:18:32] <elukey>	 chances are that upstream will fix it
[08:18:57] <elukey>	 going to send a pull request just in case
[08:18:58] <elukey>	 :P
[08:21:12] <wikibugs>	 (03CR) 10WMDE-Fisch: [C: 03+2] Update all code to use the short array syntax [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494192 (owner: 10Thiemo Kreuz (WMDE))
[08:21:33] <wikibugs>	 (03Merged) 10jenkins-bot: Update all code to use the short array syntax [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494192 (owner: 10Thiemo Kreuz (WMDE))
[08:26:43] <wikibugs>	 10Analytics-Kanban, 10Better Use Of Data, 10Product-Analytics: Superset Updates - https://phabricator.wikimedia.org/T211706 (10elukey)
[08:27:10] <elukey>	 done!
[08:54:26] <elukey>	 ah joal I have a question for you whenever you have time - is there a small job that we can use as first candidate to run with the new user? (the one replacing 'hdfs')
[08:54:44] <joal>	 There is elukey :)
[08:54:54] <joal>	 it requires reading webrequest
[08:56:38] <joal>	 oozie/aqs/hourly
[08:56:41] <elukey>	 that should be perfect since the new system user will need to be in analytics-privatedata
[08:57:01] <joal>	 This is the job extracting AQS requests info for stats
[08:57:29] <joal>	 Easy and not critical :)
[08:57:35] <elukey>	 very nice then :)
[08:57:49] <elukey>	 I am still trying to figure out the best user name :D
[09:06:19] <elukey>	 so in theory 'analytics' seems the most 'agnostic' one but we'd need to replace the deployment user
[09:07:18] <joal>	 elukey: I like this way,even if it requires a bit more work about the deployment one 
[09:11:07] <elukey>	 I am currently wondering if there is a way to use 'analytics' for both deploy and hdfs
[09:13:28] <wikibugs>	 (03CR) 10Ladsgroup: [C: 03+2] Apply all trivial auto-fixes to the PHP code style [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494194 (owner: 10Thiemo Kreuz (WMDE))
[09:13:47] <wikibugs>	 (03Merged) 10jenkins-bot: Apply all trivial auto-fixes to the PHP code style [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494194 (owner: 10Thiemo Kreuz (WMDE))
[09:21:41] <wikibugs>	 (03CR) 10Ladsgroup: [C: 03+2] Add missing limits to explode() and fix PHPDoc tags [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494196 (owner: 10Thiemo Kreuz (WMDE))
[09:22:26] <wikibugs>	 (03Merged) 10jenkins-bot: Add missing limits to explode() and fix PHPDoc tags [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/494196 (owner: 10Thiemo Kreuz (WMDE))
[09:23:54] <elukey>	 ah no right I forgot, we can't use both
[09:25:46] <wikibugs>	 (03CR) 10Ladsgroup: [C: 03+2] Update MediaWiki CodeSniffer to version 25.0.0 [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/506596 (owner: 10Thiemo Kreuz (WMDE))
[09:26:07] <wikibugs>	 (03Merged) 10jenkins-bot: Update MediaWiki CodeSniffer to version 25.0.0 [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/506596 (owner: 10Thiemo Kreuz (WMDE))
[09:45:49] <elukey>	 joal: I have created https://gerrit.wikimedia.org/r/#/c/operations/puppet/+/506609/, lemme know if it makes sense
[09:55:46] <elukey>	 joal: thanks! Ack to merge/deploy it? We can disable puppet and try on one host first
[09:56:31] <joal>	 elukey: dumb question before
[09:56:45] <joal>	 elukey: will puppet change all files ownership
[09:56:47] <joal>	 ?
[09:57:08] <joal>	 elukey: And, should we deploy on Friday? :-P
[09:57:11] <elukey>	 it seems that there is a chown exec but I need to verify how good it is
[09:57:42] <elukey>	 ah it does /bin/chown -R ${chown_user} ${chown_target}
[09:57:45] <elukey>	 that should be enough
[09:58:25] <elukey>	 ah no it is not
[09:58:34] <elukey>	 since currently we have analytics:analytics
[09:58:44] <elukey>	 so the group needs to be changed as well in theory
[09:58:45] <joal>	 elukey: just so that I understand - puppet executes the above chown for all files defined to be owned by the given user?
[09:58:48] <elukey>	 but we can do it manually
[09:59:11] <elukey>	 joal: no directly for the target, that it is the repository
[09:59:24] <joal>	 elukey: I meant repos yes
[09:59:26] <elukey>	 $chown_target = "/srv/deployment/${pkg_root}"
[10:00:16] <joal>	 Ah
[10:00:19] <joal>	 makes sense :)
[10:00:36] <elukey>	 we can wait for Andrew's +1 and monday to be sure
[10:02:04] <elukey>	 yeah let's do it so we also know if 'analytics-deploy' is a good name
[10:02:12] <joal>	 elukey: as you prefer - I'm always poin
[10:02:22] <joal>	 ting to Friday deploy, whoever it is to do it :)
[10:03:33] <elukey>	 yep yep same rule for everybody
[10:03:59] <elukey>	 there is also the performance/asoranking that I didn't take into account
[10:04:06] <elukey>	 it uses the same user/key
[10:04:14] <elukey>	 will modify the change and add Gilels
[10:04:14] <joal>	 :(
[10:04:17] <elukey>	 *Gilles
[10:04:56] <elukey>	 shouldn't be problematic, the user in there should be used only for file ownership
[10:07:59] <elukey>	 aaand done
[10:15:01] <wikibugs>	 (03PS1) 10Ladsgroup: build: add php-parallel-lint [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/506627 (https://phabricator.wikimedia.org/T179963)
[10:15:12] <wikibugs>	 (03CR) 10Ladsgroup: [C: 03+2] build: add php-parallel-lint [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/506627 (https://phabricator.wikimedia.org/T179963) (owner: 10Ladsgroup)
[10:15:34] <wikibugs>	 (03Merged) 10jenkins-bot: build: add php-parallel-lint [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/506627 (https://phabricator.wikimedia.org/T179963) (owner: 10Ladsgroup)
[10:17:57] <wikibugs>	 (03CR) 10Ladsgroup: [C: 03+2] Track number of Schemas [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/501327 (https://phabricator.wikimedia.org/T216835) (owner: 10Hoo man)
[10:22:39] <wikibugs>	 (03PS3) 10Ladsgroup: Track number of Schemas [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/501327 (https://phabricator.wikimedia.org/T216835) (owner: 10Hoo man)
[10:22:50] <wikibugs>	 (03CR) 10Ladsgroup: [C: 03+2] Track number of Schemas [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/501327 (https://phabricator.wikimedia.org/T216835) (owner: 10Hoo man)
[10:23:23] <wikibugs>	 (03Merged) 10jenkins-bot: Track number of Schemas [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/501327 (https://phabricator.wikimedia.org/T216835) (owner: 10Hoo man)
[10:24:21] <wikibugs>	 (03PS2) 10Ladsgroup: Count number of Wikidata edits by namespace [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/502169 (https://phabricator.wikimedia.org/T218901) (owner: 10Hoo man)
[10:24:47] <wikibugs>	 (03CR) 10Ladsgroup: [C: 03+2] Count number of Wikidata edits by namespace [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/502169 (https://phabricator.wikimedia.org/T218901) (owner: 10Hoo man)
[10:25:05] <wikibugs>	 (03Merged) 10jenkins-bot: Count number of Wikidata edits by namespace [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/502169 (https://phabricator.wikimedia.org/T218901) (owner: 10Hoo man)
[10:26:26] <wikibugs>	 (03PS2) 10Ladsgroup: Track number of links to Wikidata entity namespaces [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/502179 (https://phabricator.wikimedia.org/T218903) (owner: 10Hoo man)
[10:28:35] <wikibugs>	 (03CR) 10Ladsgroup: [C: 03+2] Track number of links to Wikidata entity namespaces [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/502179 (https://phabricator.wikimedia.org/T218903) (owner: 10Hoo man)
[10:28:52] <wikibugs>	 (03Merged) 10jenkins-bot: Track number of links to Wikidata entity namespaces [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/502179 (https://phabricator.wikimedia.org/T218903) (owner: 10Hoo man)
[10:29:40] <wikibugs>	 (03PS1) 10Ladsgroup: Update all code to use the short array syntax [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/506635
[10:30:02] <wikibugs>	 (03CR) 10Ladsgroup: [C: 03+2] Update all code to use the short array syntax [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/506635 (owner: 10Ladsgroup)
[10:30:25] <wikibugs>	 (03Merged) 10jenkins-bot: Update all code to use the short array syntax [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/506635 (owner: 10Ladsgroup)
[11:19:27] <wikibugs>	 10Analytics, 10Operations, 10Research-management, 10Patch-For-Review, 10User-Elukey: Remove computational bottlenecks in stats machine via adding a GPU that can be used to train ML models - https://phabricator.wikimedia.org/T148843 (10elukey)
[11:26:59] <wikibugs>	 10Analytics, 10Operations, 10hardware-requests, 10netops, and 2 others: Upgrade kafka-jumbo100[1-6] to 10G NICs (if possible) - https://phabricator.wikimedia.org/T220700 (10elukey) Summary of what would be needed:  * kafka-jumbo1001 (A1) -> 10G card + relocation to a 10G rack * kafka-jumbo1002 (A2) -> 10G...
[11:27:44] <wikibugs>	 10Analytics, 10Operations, 10ops-eqiad, 10User-Elukey: Check if a GPU fits in any of the remaining stat or notebook hosts - https://phabricator.wikimedia.org/T220698 (10elukey) a:03Cmjohnson
[11:49:41] * elukey lunch!
[13:25:02] <Krenair>	 elukey, hi
[13:25:19] <elukey>	 o/
[13:25:47] <Krenair>	 not sure if you saw my discussion with otto yesterday
[13:26:16] <Krenair>	 was wondering if you might know what (if anything) uses deployment-conf03.deployment-prep.eqiad.wmflabs
[13:27:15] <elukey>	 <elukey> Krenair: o/ just seen the ping - no idea about the instance.. is there any trace about who created it? I think
[13:27:30] <elukey>	 uff sorry half paste
[13:27:52] <elukey>	 anyway, I wrote that I think your plan about shutoff temporarily + final kill seems good
[13:27:55] <Krenair>	 oh, sorry, looks like my ZNC doesn't have that much buffered
[13:28:46] <elukey>	 with netstat -tuap I don't see a lot of valuable conns on the host
[13:29:03] <Krenair>	 otto created it which is why I asked him first, be he hadn't logged in since 2015
[13:29:09] <elukey>	 ahahhaah
[13:29:20] <Krenair>	 under his own user anyway
[13:29:28] <elukey>	 it is not running anything afaics, I think that you are free to kill it
[13:29:36] <Krenair>	 cool
[13:29:46] <elukey>	 ah no there is one etcd instance
[13:29:49] <elukey>	 lemme check to be sure
[13:30:05] <Krenair>	 there is, but it is using a cert from the old puppetmaster
[13:31:14] <elukey>	 then nuke it :)
[13:31:23] <Krenair>	 great, thanks elukey 
[13:31:45] <elukey>	 thank you for cleaning up old stuff :)
[14:31:22] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10QuickSurveys, 10Readers-Web-Backlog (Tracking): QuickSurveys EventLogging missing ~10% of interactions - https://phabricator.wikimedia.org/T220627 (10phuedx) >>! In T220627#5125187, @Nuria wrote: > Moving to radar as further steps of code chnages to Quicksurveys to...
[14:53:30] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10User-Elukey: Change permissions for daily traffic anomaly reports on stat1007 - https://phabricator.wikimedia.org/T219546 (10elukey) @ssingh ping :)
[15:33:35] <wikibugs>	 (03PS8) 10Mforns: Make saltrotate store salts with timestamps as file name. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/484250 (https://phabricator.wikimedia.org/T212014)
[15:48:05] <wikibugs>	 (03PS1) 10Lucas Werkmeister (WMDE): Apply all trivial auto-fixes to the PHP code style [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/506698
[15:48:08] <wikibugs>	 (03PS1) 10Lucas Werkmeister (WMDE): Add missing limits to explode() and fix PHPDoc tags [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/506699
[15:48:11] <wikibugs>	 (03PS1) 10Lucas Werkmeister (WMDE): Update MediaWiki CodeSniffer to version 25.0.0 [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/506700
[15:48:14] <wikibugs>	 (03PS1) 10Lucas Werkmeister (WMDE): Fix typo in author name "Adddshore" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/506701
[15:48:59] <wikibugs>	 (03CR) 10Lucas Werkmeister (WMDE): [C: 04-1] "Let’s not merge these before Monday. (Unfortunately I don’t have -2 rights on the production branch.)" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/506698 (owner: 10Lucas Werkmeister (WMDE))
[16:01:23] <nuria>	 elukey: standup? cc milimetric 
[16:02:08] <milimetric>	 i'm still out sick, Nuria, doctor's appointment Monday, but I'm doing some admin stuff like catching up on email and adding reviewers since I was out all week
[16:02:22] <milimetric>	 I don't know what I have, but I'm glad all of you are remote :)
[16:02:36] <nuria>	 milimetric: Ok, GET BETTER!
[16:03:44] <milimetric>	 I actually took POISON IVY today (and it worked, look it up it's so weird)
[16:38:26] <nuria>	 milimetric: just do not take a lot maybe?
[16:38:29] <nuria>	 neilpquinn: yt?
[16:40:58] <wikibugs>	 10Analytics, 10Analytics-Data-Quality, 10Analytics-Kanban, 10Product-Analytics: Many revision events in mediawiki_history have missing page and namespace information - https://phabricator.wikimedia.org/T221338 (10Nuria) I am puzzled on how edits are within couple percentages but active editors are so diffe...
[16:47:50] <fdans>	 a-team moved the vetting page to the main namespace
[16:47:51] <fdans>	 https://wikitech.wikimedia.org/wiki/Analytics/AQS/Wikistats_2/Vetting_data_lake_metrics_for_project_families
[16:48:12] <mforns>	 k
[16:51:13] <nuria>	 fdans: shoudl be under data quality 
[16:51:33] <nuria>	 fdans:  Please see https://wikitech.wikimedia.org/wiki/Analytics/AQS/Wikistats_2/Data_Quality
[16:51:55] <nuria>	 fdans: this ( i think) shoudl be a top page that links to the new one you did and josephs's prior work
[16:52:48] <fdans>	 nuria: oook I see
[17:11:15] <wikibugs>	 (03PS9) 10Mforns: Make saltrotate store salts with timestamps as file name. [analytics/refinery] - 10https://gerrit.wikimedia.org/r/484250 (https://phabricator.wikimedia.org/T212014)
[17:13:49] <wikibugs>	 (03CR) 10Mforns: "OK @Ottomata I think this is ready for CR again. I refactored the script to be local and also added a set of unit tests." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/484250 (https://phabricator.wikimedia.org/T212014) (owner: 10Mforns)
[17:49:20] <neilpquinn>	 nuria: I'm here now—what's up?
[18:07:46] <wikibugs>	 10Analytics, 10Analytics-Kanban, 10User-Elukey: Change permissions for daily traffic anomaly reports on stat1007 - https://phabricator.wikimedia.org/T219546 (10ssingh) Hi @elukey, thanks for the ping.  The reports were not being generated so I have made some more changes today (updating paths); I will update...
[18:09:56] <elukey>	 so I was able to pin point the commit that broke the world map in superset
[18:09:59] <elukey>	 https://github.com/apache/incubator-superset/issues/7006
[18:10:18] <elukey>	 I really hope that now upstream comes back with "Ah sorry the fix is X"
[18:10:21] <elukey>	 :D
[18:10:44] <elukey>	 going afk now, havea  good weekend people!
[18:11:39] <mforns>	 byeee luca
[19:42:34] <wikibugs>	 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10EventBus, and 4 others: Modern Event Platform: Deploy instance of EventGate service that produces events to kafka main - https://phabricator.wikimedia.org/T218346 (10fsero) as @akosiaris said i would prefer to not include {{.Release.namespace}} b...