[00:12:27] 10Analytics, 10WMDE-Analytics-Engineering: Pyspark2 fails to read.csv when run with spark2-submit - https://phabricator.wikimedia.org/T217156 (10GoranSMilovanovic) @Ottomata Yes, I am sure I was running with `--master yarn --deploy-mode client `. [00:45:50] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Superset's rolling average feature results in error message - https://phabricator.wikimedia.org/T213488 (10Tbayer) >>! In T213488#4985201, @elukey wrote: ... > > Also @HaeB, do you have an example of dashboard that I can use to trigger this issue? I trie... [02:16:31] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Superset's rolling average feature results in error message - https://phabricator.wikimedia.org/T213488 (10Nuria) >Actually it works for me too now, on the same view where I previously encountered this error Next time let's please add the view that encou... [04:14:24] 10Analytics, 10ExternalGuidance, 10Product-Analytics, 10MW-1.33-notes (1.33.0-wmf.18; 2019-02-19), 10Patch-For-Review: Measure the impact of externally-originated contributions - https://phabricator.wikimedia.org/T212414 (10chelsyx) I did another round of test and confirm that the eventlogging works as w... [06:35:42] 10Analytics, 10Analytics-Kanban, 10Operations, 10Product-Analytics, 10Patch-For-Review: dbstore1002 Mysql errors - https://phabricator.wikimedia.org/T213670 (10Marostegui) MySQL just crashed again: ` Thread pointer: 0x0x0 Attempting backtrace. You can use the following information to find out where mysql... [06:50:12] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Move AQS to nodejs 10 - https://phabricator.wikimedia.org/T210706 (10elukey) @Milimetric done! I haven't rolled restart aqs yet (still using the old nodejs interpreter), so we can test (if you are ok) a deployment via scap that should 1) deploy the new cod... [07:31:07] (03PS1) 10Nuria: [WIP] Eventlogging schema loader quick and dirty refactor on top of andrew's changes [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/493157 [07:31:57] (03CR) 10jerkins-bot: [V: 04-1] [WIP] Eventlogging schema loader quick and dirty refactor on top of andrew's changes [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/493157 (owner: 10Nuria) [08:29:27] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review, 10User-Elukey: Hadoop Yarn stores a ton of znodes related to running/old applications - https://phabricator.wikimedia.org/T216952 (10elukey) [08:29:39] so I discovered in --^ that each znode in /rmstore/ZKRMStateRoot/RMAppRoot/etc.. [08:29:56] has also one child (or more) [08:30:02] dedicated to app attempts [08:30:07] so it is not only 10K znodes [08:30:14] but probably 20/30k [08:33:44] so I will need to be really careful, probably, when deleting all those znodes [08:33:48] maybe in batches [09:16:00] 10Analytics, 10Beta-Cluster-Infrastructure, 10User-Elukey: TCP connections between analytics and deployment-prep - https://phabricator.wikimedia.org/T208870 (10elukey) 05Open→03Resolved [09:22:40] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Patch-For-Review: eventlogging fails flake8 due to new upstream version, breaking CI - https://phabricator.wikimedia.org/T212396 (10hashar) I went ahead and just self merged all the patches that fix up flake8. They were all trivial / test only as... [09:42:06] (03PS2) 10WMDE-Fisch: Update userprop scripts to new db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493086 (https://phabricator.wikimedia.org/T216613) [10:30:01] (03CR) 10Ladsgroup: [C: 03+1] "LGTM" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493086 (https://phabricator.wikimedia.org/T216613) (owner: 10WMDE-Fisch) [10:30:07] 10Analytics, 10Analytics-Kanban: [Bug] Type mismatch for a few other schemas - https://phabricator.wikimedia.org/T216771 (10phuedx) [10:30:11] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Readers-Web-Backlog: Popups schema has the wrong type for popupDelay - https://phabricator.wikimedia.org/T217110 (10phuedx) 05Open→03Resolved a:03phuedx @Tbayer pointed me at this task during our 1:1 yesterday. AFAICT it's the inferred type tha... [10:31:51] mforns: o/ [10:32:46] mforns: to answer your questions about the content of the translate message key, take a look to https://meta.wikimedia.org/wiki/Special:Translate [10:32:59] it basically supports message groups and their metadata [10:33:09] to support translators [11:07:19] * elukey lunch + errands! [11:11:24] (03PS6) 10Addshore: Fixes for new multisource db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/492988 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [11:11:27] (03CR) 10Addshore: Fixes for new multisource db setup (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/492988 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [11:12:01] (03CR) 10Addshore: [C: 03+2] Fixes for new multisource db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/492988 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [11:12:07] (03PS1) 10Addshore: Fixes for new multisource db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493198 (https://phabricator.wikimedia.org/T213894) [11:12:10] (03Merged) 10jenkins-bot: Fixes for new multisource db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/492988 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [11:12:14] (03CR) 10Addshore: [V: 03+2 C: 03+2] Fixes for new multisource db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493198 (https://phabricator.wikimedia.org/T213894) (owner: 10Addshore) [11:12:15] (03CR) 10jerkins-bot: [V: 04-1] Fixes for new multisource db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493198 (https://phabricator.wikimedia.org/T213894) (owner: 10Addshore) [11:12:21] (03PS2) 10Addshore: Fix connecting to the right port of multisource db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493013 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [11:12:23] (03Merged) 10jenkins-bot: Fixes for new multisource db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493198 (https://phabricator.wikimedia.org/T213894) (owner: 10Addshore) [11:12:34] (03CR) 10Addshore: [C: 03+2] Fix connecting to the right port of multisource db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493013 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [11:12:45] (03PS1) 10Addshore: Fix connecting to the right port of multisource db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493200 (https://phabricator.wikimedia.org/T213894) [11:12:51] (03CR) 10Addshore: [V: 03+2 C: 03+2] Fix connecting to the right port of multisource db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493200 (https://phabricator.wikimedia.org/T213894) (owner: 10Addshore) [11:13:04] (03Merged) 10jenkins-bot: Fix connecting to the right port of multisource db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493013 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [11:13:06] (03Merged) 10jenkins-bot: Fix connecting to the right port of multisource db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493200 (https://phabricator.wikimedia.org/T213894) (owner: 10Addshore) [11:13:08] (03PS3) 10Addshore: Update userprop scripts to new db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493086 (https://phabricator.wikimedia.org/T216613) (owner: 10WMDE-Fisch) [11:13:13] 10Analytics, 10WMDE-Analytics-Engineering: Pyspark2 fails to read.csv when run with spark2-submit - https://phabricator.wikimedia.org/T217156 (10GoranSMilovanovic) @Ottomata The proposed solution T217156#4985379, unfortunately, does not deliver: ` spark2-submit --master yarn --deploy-mode client --num-executo... [11:13:14] (03CR) 10Addshore: [V: 03+2 C: 03+2] Update userprop scripts to new db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493086 (https://phabricator.wikimedia.org/T216613) (owner: 10WMDE-Fisch) [11:13:19] (03PS1) 10Addshore: Update userprop scripts to new db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493201 (https://phabricator.wikimedia.org/T216613) [11:13:27] (03Merged) 10jenkins-bot: Update userprop scripts to new db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493086 (https://phabricator.wikimedia.org/T216613) (owner: 10WMDE-Fisch) [11:13:29] (03CR) 10Addshore: [V: 03+2 C: 03+2] Update userprop scripts to new db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493201 (https://phabricator.wikimedia.org/T216613) (owner: 10Addshore) [11:13:35] (03Merged) 10jenkins-bot: Update userprop scripts to new db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493201 (https://phabricator.wikimedia.org/T216613) (owner: 10Addshore) [11:21:58] (03CR) 10Thiemo Kreuz (WMDE): Update userprop scripts to new db setup (032 comments) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493086 (https://phabricator.wikimedia.org/T216613) (owner: 10WMDE-Fisch) [11:23:37] 10Analytics-EventLogging, 10Analytics-Kanban, 10Patch-For-Review: [EventLogging Sanitization] Enable older-than-90-day purging of unsanitized EL database (event) in Hive - https://phabricator.wikimedia.org/T209503 (10Miriam) @mforns yes, that is correct :) Thanks! [11:37:05] (03PS1) 10Ladsgroup: Move reads to multi-source db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493207 (https://phabricator.wikimedia.org/T213894) [11:39:34] (03PS1) 10Thiemo Kreuz (WMDE): lib: Update and add PHPDoc tags to all lib classes [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493209 [11:40:30] (03PS1) 10Thiemo Kreuz (WMDE): Fix typo in three similar userprops scripts [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493210 [11:40:56] (03PS1) 10Ladsgroup: Fix typo [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493211 [11:42:29] (03PS1) 10Thiemo Kreuz (WMDE): Add strict type hints to several scripts and lib classes [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493212 [11:42:35] (03CR) 10jerkins-bot: [V: 04-1] Add strict type hints to several scripts and lib classes [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493212 (owner: 10Thiemo Kreuz (WMDE)) [11:43:50] (03CR) 10Ladsgroup: Update userprop scripts to new db setup (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493086 (https://phabricator.wikimedia.org/T216613) (owner: 10WMDE-Fisch) [11:45:30] (03CR) 10Thiemo Kreuz (WMDE): Fix typo (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493211 (owner: 10Ladsgroup) [11:46:45] (03CR) 10Thiemo Kreuz (WMDE): "recheck" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493212 (owner: 10Thiemo Kreuz (WMDE)) [11:48:27] (03PS2) 10Ladsgroup: Move reads to multi-source db setup [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493207 (https://phabricator.wikimedia.org/T213894) [11:48:48] (03CR) 10Thiemo Kreuz (WMDE): [C: 04-1] "I don't know much about this codebase yet, but what I see in this patch looks good. I'm even willing to hit this +2 button, in case nobody" (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493207 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [11:48:53] (03Abandoned) 10Ladsgroup: Fix typo [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493211 (owner: 10Ladsgroup) [11:49:42] (03CR) 10Thiemo Kreuz (WMDE): [C: 03+1] "Ah, there is the fixup already. Thanks!" [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493207 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [11:49:47] (03CR) 10Ladsgroup: Move reads to multi-source db setup (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493207 (https://phabricator.wikimedia.org/T213894) (owner: 10Ladsgroup) [12:00:59] (03CR) 10Ladsgroup: Update userprop scripts to new db setup (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493086 (https://phabricator.wikimedia.org/T216613) (owner: 10WMDE-Fisch) [12:10:59] (03PS1) 10Thiemo Kreuz (WMDE): Introduce caching in WikimediaDb lib class [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493214 [12:25:49] (03CR) 10WMDE-Fisch: [C: 03+2] Fix typo in three similar userprops scripts [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493210 (owner: 10Thiemo Kreuz (WMDE)) [12:25:57] (03Merged) 10jenkins-bot: Fix typo in three similar userprops scripts [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493210 (owner: 10Thiemo Kreuz (WMDE)) [12:27:12] (03CR) 10WMDE-Fisch: [C: 03+2] lib: Update and add PHPDoc tags to all lib classes [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493209 (owner: 10Thiemo Kreuz (WMDE)) [12:27:21] (03Merged) 10jenkins-bot: lib: Update and add PHPDoc tags to all lib classes [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493209 (owner: 10Thiemo Kreuz (WMDE)) [12:32:06] (03CR) 10WMDE-Fisch: [C: 03+1] Introduce caching in WikimediaDb lib class [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493214 (owner: 10Thiemo Kreuz (WMDE)) [12:37:19] (03CR) 10Thiemo Kreuz (WMDE): Update userprop scripts to new db setup (031 comment) [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493086 (https://phabricator.wikimedia.org/T216613) (owner: 10WMDE-Fisch) [13:55:20] 10Analytics-EventLogging, 10Analytics-Kanban, 10Patch-For-Review: [EventLogging Sanitization] Enable older-than-90-day purging of unsanitized EL database (event) in Hive - https://phabricator.wikimedia.org/T209503 (10mforns) @Miriam Thanks! [13:57:32] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10Patch-For-Review: eventlogging fails flake8 due to new upstream version, breaking CI - https://phabricator.wikimedia.org/T212396 (10mforns) a:05mforns→03hashar [14:10:58] hey team :] [14:31:07] 10Analytics-EventLogging, 10Analytics-Kanban, 10Patch-For-Review: [EventLogging Sanitization] Enable older-than-90-day purging of unsanitized EL database (event) in Hive - https://phabricator.wikimedia.org/T209503 (10mforns) Here's the plan for tomorrow: 1. Collect the names of all tables in the event databa... [14:40:16] 10Analytics, 10Analytics-Kanban, 10Patch-For-Review: Update wikimedia-history revision data with deleted field (and find it a new name?) - https://phabricator.wikimedia.org/T178587 (10Neil_P._Quinn_WMF) >>! In T178587#4979913, @JAllemandou wrote: > the `page_is_deleted` info is present on page data now, but... [14:44:12] (03PS4) 10Joal: Add change_tags to mediawiki_history [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/492320 [14:44:14] (03PS3) 10Joal: Update mediawiki-reconstruction with log info [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/493012 [14:48:11] elukey: how would we do that rolling restart thing from scap? I'm not too familiar with scap [14:48:19] milimetric: o/ [14:48:22] scap deploy [14:48:25] that's it [14:48:38] oh :) ok [14:48:48] it is part of the worflow that we have set up for aqs :) [14:48:50] so, as of right now I don't see any problems running against Node 10 on my machine [14:48:51] should work fine [14:49:00] so there won't be any new code [14:49:05] but I can just re-deploy the current version [14:49:09] ah! [14:49:25] and then if there are problems I'll fix and deploy again [14:49:26] nono ok I thought that we had to deploy something [14:49:44] what we can do then is something like [14:49:58] oh, hm, well, there are slight updates to the package.json for node version and stuff [14:50:01] we can deploy that [14:50:05] depool aqs1004, upgrade to nodejs10, check locally etc.., restart aqs, repool [14:50:18] ah [14:50:40] wait, we haven't tested node 10 on deployment-aqs01 yet [14:50:49] just my local' [14:50:53] sure sure [14:51:00] ok, so, steps [14:51:07] I was only proposing a plan after the green light [14:51:16] 1. deploy minor non-code changes to aqs [14:51:21] since you were saying that no code change is required (basically the alternative procedure) [14:51:21] (to deployment-prep) [14:51:43] 2. check and fix anything on deployment [14:52:06] 3. do the depool aqs1004 thing you said above [14:52:22] uh... but I should probably finish up the sqoop stuff, that's more urgent [14:52:27] milimetric: what about 1. for prod? [14:52:45] (before aqs1004 depool I mean) [14:52:47] right, that too [14:53:09] will the minor code changes work with nodejs6 too? [14:53:17] (03PS1) 10Joal: Update LogHelper for parameterization [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/493237 [14:53:22] if not we'll have to deploy only to aqs1004 [14:53:29] and depool etc.. [14:53:45] anyway, if you want to work on sqoop it makes sense [14:53:50] we have time for aqs [14:53:51] I think it should but I never tested it [14:54:00] yeah, might wait until tonight or tomorrow [14:54:12] ack then [15:12:29] joal: you around? [15:12:57] I am milimetric, until my kids wake up :) [15:13:20] joal: I think this sqoop thing is too messy and that's why I haven't given it much care [15:13:44] joal: why don't we just not pass the host at all from puppet? [15:14:22] we can hard code the labs host in util.py, and have a function there that just returns the jdbc, with only two parameters, dbname and cluster_name [15:14:33] if it's labs, we add the postfix, use the labs host [15:14:42] if it's prod, we grab the host and port from the config [15:15:05] like, it's super weird to have the sqoop script control the host only sometimes [15:15:35] let's have the script just control the cluster and we figure out everything internally [15:15:53] maybe we can even add dns locator records for enwiki_p etc. [15:16:24] milimetric: works for me, but please let's do it in the main executable, not the sqoop.py utility - Like that the utility is reusable [15:16:45] sounds reasonable milimetric ? [15:16:57] joal: that's what I was saying, in util.py [15:17:20] I'll rip out anything related to labs or connections from both bin/sqoop scripts and python/refinery/sqoop.py [15:17:20] no no milimetric - in main, not util :) [15:17:36] no, disagree [15:17:59] we already look up connection information in util.py [15:18:10] I can move it all into a new db_connection.py or something [15:18:21] but we're already using the functions in util.py [15:18:30] (for the analytics-mysql wrapper) [15:19:34] do we still disagree joal? :) I can do it and we can talk after standup if you like [15:21:11] milimetric: I'll stop bothering you :) So far no use case has proven valid in having the sqoop-util generic, so let's make it the way you suggest [15:21:31] joal: I'm not putting it in sqoop.py! I agree that should be generic! [15:21:34] util.py!!! [15:22:13] milimetric: ok - but called from where? main-sqoop and passed to sqoop-util through config? or called from sqoop-util? [15:22:16] https://github.com/wikimedia/analytics-refinery/blob/master/python/refinery/util.py#L1071 [15:22:32] called from main-sqoop, passed to sqoop as jdbc_connection [15:23:22] this'll be obvious in the code, will just do it. But please, don't give up, I want this to be loved by all :) [15:23:36] YES --^ that's what I wanted :) [15:23:57] Having the host computed in the main-sqoop, not in util-sqoop - Thanks milimetric :) [15:24:14] This'll work super fine, and handle both labs and analytics-prod [15:24:19] I'm on board :) [15:25:32] And thanks for continuing pushing on this milimetric :) [15:25:47] milimetric: Lino says hello, I need to leave you - all good ? [15:25:56] of course, thanks joal [15:26:03] bonsoir Lino! [15:36:45] 10Analytics, 10WMDE-Analytics-Engineering: Pyspark2 fails to read.csv when run with spark2-submit - https://phabricator.wikimedia.org/T217156 (10Ottomata) Hm, perhaps --files works different in pyspark? This is how it works in Scala. Anyway, I got to work by resolving the spark uploaded file paths using `Spa... [15:38:15] (03CR) 10Ottomata: [C: 03+1] Update LogHelper for parameterization (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/493237 (owner: 10Joal) [15:39:38] o/ joal [15:41:12] (03CR) 10Ottomata: "Great, I'm going to squash this into my patch (so we don't have to merge 2) and then work on it from there. Probably will move things aro" [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/493157 (owner: 10Nuria) [15:57:41] (03PS1) 10Fdans: [wip] Display exact period of time in widgets instead of "last year" [analytics/wikistats2] - 10https://gerrit.wikimedia.org/r/493260 (https://phabricator.wikimedia.org/T216105) [16:42:47] 10Analytics, 10WMDE-Analytics-Engineering: Pyspark2 fails to read.csv when run with spark2-submit - https://phabricator.wikimedia.org/T217156 (10GoranSMilovanovic) @Ottomata following ` items = sqlContext.read.csv(SparkFiles.get(itemFile), header=True) ` where `ItemFile` is a filename previously provided i... [16:50:19] 10Analytics, 10MediaWiki-API, 10Reading-Infrastructure-Team-Backlog: Load API request count and latency data from Hadoop to a dashboard - https://phabricator.wikimedia.org/T108414 (10Jhernandez) p:05Triage→03Low @Tgr Can you add a full description about what this is and move to backlog if it is ours? Thanks [16:52:02] Hi milimetric. Was looking through SoS notes and saw "new cloud db replica for analytics and mediawiki history use is not in place yet, sqooping and history reconstruction will speed up next month when it is" Can you tell me more about this? [16:52:18] Is there a task? Will think be another mysql replica or are you working with another tech? [16:52:29] *Will there be [16:52:36] halfak: we're in the process of adding another replica to the clouddb cluster [16:52:48] so same tech, same schema sanitizing, etc [16:53:21] except once it's up, it'll be used to sqoop the snapshot we need to build mediawiki history [16:53:22] Gotcha. Just more capacity? [16:53:35] 10Analytics, 10Contributors-Analysis, 10Product-Analytics: Make an Analytics Data Lake table to provide meta info about wikis - https://phabricator.wikimedia.org/T184576 (10Neil_P._Quinn_WMF) Okay, I've put up an initial version of the this dataset: see [canonical-data/wikis.csv](https://github.com/wikimedia... [16:53:38] Oh! Cool So you can build a sanitized one for labs/the public? [16:53:48] yeah, right now we're throttled both in # of parallel connections and when we can start, so we're like 6 days behind right off the bat [16:54:10] halfak: no the one we build now is already sanitized, it has been from the start [16:54:22] this is just more capacity, dedicated to us so we can go faster / earlier [16:54:32] Gotcha. Makes sense. Thank you :) [16:54:48] halfak: but of course the weird thing is the rest of the month it would unused, so we're trying to think of how to best make use of it, it'll be a beefy box [16:55:14] Can it offload some work from Quarry and other toolforge stuff? [16:55:16] maybe we should load the snapshot of mediawiki history onto it, see how it works with a few indices and expose it from quarry :) [16:55:21] Right! [16:55:23] Yexs. [16:55:25] OMG want [16:55:31] it could in theory, but I think they wanted to keep it separate for ops reasons [16:55:58] yeah, in any case, we're going that way anyway, with Presto being ready to use soon [16:57:30] "soon" == not soon probably [16:57:36] due to labs vs prod issues and bad timings [16:57:45] Gotcha. Presto will be higher performance for analytics queries, right? [16:58:34] * halfak reads https://aws.amazon.com/big-data/what-is-presto/ [16:59:19] lol elastic presto service [16:59:49] ottomata: I'm talking to halfak, he knows what soon means [16:59:58] :) [17:00:20] Oh yeah. "soon" means "it will probably happen and there will be the usual delays" :D [17:00:42] more like unusual delays this time, we were hoping to have it up last/this quarter [17:00:52] but hosting prod data in labs is not easy [17:00:53] gotcha. [17:00:55] ping joal [17:00:56] lots of issues [17:01:03] I can imagine, but it's a great thing to be working on. [17:01:05] ping ottomata [17:01:05] if they had it their way they'd turn off labsdb replication too [17:01:06] Big impact potential [17:01:15] * halfak needs to run to the next meeting [17:01:17] i'm there... [17:01:19] thanks for the additional info [17:01:20] aren't I? [17:01:21] o/ [17:01:27] oh weird it just took forever to load [17:02:17] 10Analytics, 10Product-Analytics, 10Reading List Service, 10Reading-Infrastructure-Team-Backlog, and 3 others: [EPIC] Reading List Sync service analytics - https://phabricator.wikimedia.org/T191859 (10Jhernandez) 05Open→03Stalled p:05Triage→03Lowest Reflecting reality, no-one seems to be interested... [17:12:29] (03PS2) 10Thiemo Kreuz (WMDE): Introduce caching in WikimediaDb lib class [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493214 (https://phabricator.wikimedia.org/T216613) [17:12:47] (03PS2) 10Thiemo Kreuz (WMDE): Add strict type hints to several scripts and lib classes [analytics/wmde/scripts] - 10https://gerrit.wikimedia.org/r/493212 (https://phabricator.wikimedia.org/T216613) [17:13:14] 10Analytics, 10WMDE-Analytics-Engineering: Pyspark2 fails to read.csv when run with spark2-submit - https://phabricator.wikimedia.org/T217156 (10Ottomata) @GoranSMilovanovic can you provide a small pyspark script as well as your spark2-submit command so I can try to see what's going on? [17:16:02] PROBLEM - eventbus grafana alert on icinga2001 is CRITICAL: CRITICAL: EventBus ( https://grafana.wikimedia.org/d/000000201/eventbus ) is alerting: EventBus POST Response Status alert. [17:17:03] seems to be mediawiki.job.cirrusSearchElasticaWrite. MessageSizeTooLargeError: MESSAGE_SIZE_TOO_LARGE [17:18:27] RECOVERY - eventbus grafana alert on icinga2001 is OK: OK: EventBus ( https://grafana.wikimedia.org/d/000000201/eventbus ) is not alerting. [17:24:13] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Superset's rolling average feature results in error message - https://phabricator.wikimedia.org/T213488 (10jlinehan) > Next time let's please add the view that encounters the error to the bug report (besides the error itself). Similarly, it would be help... [17:24:20] as FYI I am shutting down analytics1071, it needs to be moved to rack A5 (from A7) [17:26:04] 10Analytics, 10Analytics-Wikistats, 10Contributors-Analysis, 10Product-Analytics: Update active editor metrics to use consensus definition - https://phabricator.wikimedia.org/T153702 (10Neil_P._Quinn_WMF) 05Open→03Declined Since @ezachte has left the WMF and we're not making further investments in Wiki... [17:28:13] 10Analytics, 10WMDE-Analytics-Engineering: Pyspark2 fails to read.csv when run with spark2-submit - https://phabricator.wikimedia.org/T217156 (10GoranSMilovanovic) pyspark: ` import pyspark from pyspark.sql import SparkSession from pyspark.sql.window import Window from pyspark.sql.functions import rank, col f... [17:30:25] 10Analytics, 10Analytics-EventLogging, 10Analytics-Kanban, 10EventBus, and 3 others: Modern Event Platform: Stream Intake Service: Implementation: Deployment Pipeline - https://phabricator.wikimedia.org/T211247 (10jijiki) @Ottomata LVS config is done :) 😺 [17:39:02] 10Analytics, 10Analytics-EventLogging, 10Front-end-Standards-Group, 10MediaWiki-extensions-WikimediaEvents, and 2 others: Provide a reusable getEditCountBucket function for analytics purposes - https://phabricator.wikimedia.org/T210106 (10Jdlrobson) 05Open→03Stalled We synced on this today. This task i... [17:41:55] 10Analytics, 10Knowledge-Integrity, 10Research, 10Epic, 10Patch-For-Review: Citation Usage: run third round of data collection - https://phabricator.wikimedia.org/T213969 (10bmansurov) @RyanSteinberg 2-day data collection is complete. Please check the data quality and let me know if everything looks good... [17:42:23] !log re-run webrequest-load-wf-upload-2019-2-27-16 (failed due to a shutdown of analytics1071 for hw maintenance) [17:42:25] Logged the message at https://www.mediawiki.org/wiki/Analytics/Server_Admin_Log [17:50:10] ottomata: o/ [17:50:15] analytics1071 is in the new rack [17:50:27] should I merge my change and then restart the namenodes just to be sure? [17:50:45] (to update the hdfs network config) [17:55:28] yeah [17:55:28] Rack: /eqiad/A/7 [17:55:28] 10.64.5.25:50010 (analytics1071.eqiad.wmnet) [17:55:34] this is print topology now [17:55:39] after the puppet run [17:55:41] ok doing it [17:57:15] nuria: milimetric https://github.com/wikimedia/mediawiki-extensions-EventLogging/blob/master/includes/ApiJsonSchema.php [18:02:20] 10Analytics: Public event data incoming from eventgate should go into db that does not get sanitized? - https://phabricator.wikimedia.org/T217271 (10Nuria) [18:04:41] ottomata: that requires a revid parameter [18:08:54] Rack: /eqiad/A/5 [18:08:54] 10.64.5.25:50010 (analytics1071.eqiad.wmnet) [18:08:57] \o/ [18:09:41] master back to an-master1001 [18:09:52] currently restart an-master1002 but should be over soon [18:24:11] (03PS3) 10Milimetric: Use db_mapping to find the hostname [analytics/refinery] - 10https://gerrit.wikimedia.org/r/492209 (https://phabricator.wikimedia.org/T215290) [18:24:35] (03PS1) 10Milimetric: Fix linting errors [analytics/refinery] - 10https://gerrit.wikimedia.org/r/493285 [18:26:17] 10Analytics, 10Analytics-Kanban, 10Product-Analytics: Superset's rolling average feature results in error message - https://phabricator.wikimedia.org/T213488 (10elukey) >>! In T213488#4988715, @jlinehan wrote: >> Next time let's please add the view that encounters the error to the bug report (besides the err... [18:28:13] milimetric: let's change it [18:28:18] we can modify the extension no? [18:29:24] ottomata: yea, but I'm not sure what the right parameter combination is to say "get latest revision" [18:29:51] I'm also ridiculously overbooked, gotta worry about other stuff, so you'd have to do this yourself and I can take a look when I'm less swamped [18:30:51] ottomata: oohh, i had this one and jsonconfig confused [18:30:51] yeah i'll look into it milimetric [18:31:00] milimetric: the right hting is if no revid, get latest [18:31:35] eh, I donno at all if that's right [18:31:45] nuria: I think it did or still does inherit from JsonConfig [18:31:55] ottomata: ya, now a rev id must be passed https://github.com/wikimedia/mediawiki-extensions-EventLogging/blob/master/includes/ApiJsonSchema.php#L75 [18:32:13] milimetric: but it seems if no revision is ( a la wikipedia page) it should give you latest revision [18:32:32] cc ottomata , seems very standard for wikipedia no? [18:32:36] nuria: yeah, the way it's structured right now is it doesn't require the title/titles param [18:32:53] so it needs the revid, and I'm not sure if there's a mutually exclusive require parameter thing [18:32:58] (like in argparse) [18:38:10] joal: small thing but realized we need a change_tag_def table definition in Hive [18:38:22] (and I think to update the current change_tag one) [18:38:40] I'm submitting the ipblocks_restrictions one now, and will test the new sqoop with everything [18:39:45] milimetric: ah i see, but it allows for title https://github.com/wikimedia/mediawiki-extensions-EventLogging/blob/master/includes/ApiJsonSchema.php#L41 [18:40:05] 10Analytics, 10Analytics-Kanban: Update reportupdater to be able to query the new db cluster that will substitute 1002 - https://phabricator.wikimedia.org/T215289 (10Milimetric) a:05Milimetric→03mforns [18:40:14] its doable tho, am trying now [18:40:43] ottomata: k, can cr and test as needed [18:41:18] (03PS1) 10Milimetric: Add sqoop queries for ipblocks_restrictions table [analytics/refinery] - 10https://gerrit.wikimedia.org/r/493286 (https://phabricator.wikimedia.org/T209549) [18:45:45] 10Analytics, 10MediaWiki-API, 10Reading-Infrastructure-Team-Backlog: Load API request count and latency data from Hadoop to a dashboard - https://phabricator.wikimedia.org/T108414 (10Tgr) [18:45:58] milimetric: do you know? simple q: what is the best way in MW to get the latest revision? there are a lot of classes here! [18:46:02] Article? Title? WikiPage? [18:46:08] WikiPage has a getLatest [18:46:18] ah but Article uses it... [18:46:54] the only thing I know is that that's not a simple q, and that no matter what you use, someone will say it's wrong :) [18:47:07] so just use anything and ask for a review [18:47:15] I'd randomly guess Title [18:47:28] haha [18:47:28] ok [18:47:49] ottomata: ask timo the oracle [18:48:10] ottomata: or gergo [18:48:21] ottomata: that would be my strategy [18:53:20] 10Analytics, 10MediaWiki-API, 10Reading-Infrastructure-Team-Backlog: Load API request count and latency data from Hadoop to a dashboard - https://phabricator.wikimedia.org/T108414 (10Tgr) Oops, I think I'm confusing this with {T155478}. [18:57:33] 10Analytics, 10MediaWiki-API, 10Reading-Infrastructure-Team-Backlog: Load API request count and latency data from Hadoop to a dashboard - https://phabricator.wikimedia.org/T108414 (10Tgr) [19:04:00] milimetric: https://gerrit.wikimedia.org/r/c/analytics/refinery/+/491838 [19:05:23] ah, right! [19:05:32] :) [19:05:50] I hope I've been answering fastso that you have not started your patch :S [19:08:33] ottomata: do you have a minute on JsonNodes and LogHelpers? [19:09:22] 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review, 10User-Elukey: rack/setup/install labsdb1012.eqiad.wmnet - https://phabricator.wikimedia.org/T215231 (10ayounsi) [19:13:33] 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review, 10User-Elukey: rack/setup/install labsdb1012.eqiad.wmnet - https://phabricator.wikimedia.org/T215231 (10Cmjohnson) @elukey I moved the host to A6 and updated netbox. Arzhel updated network switch cfg. DNS will need to be updated and then ready f... [19:35:04] 10Analytics, 10Operations, 10ops-eqiad, 10Patch-For-Review, 10User-Elukey: rack/setup/install labsdb1012.eqiad.wmnet - https://phabricator.wikimedia.org/T215231 (10elukey) [19:39:19] * elukey off! [19:44:39] This -- https://xkcd.com/2117/ --reminds me so much of my maths years :) [19:48:09] 10Analytics, 10Analytics-Kanban, 10Operations, 10hardware-requests: GPU upgrade for stat1005 - https://phabricator.wikimedia.org/T216226 (10EBernhardson) I have submitted a support request to an "AMD Expert" and will see what they have to say about GPU's that are physically compatible with our servers. [20:06:34] joal: "Burn the evidence!" [20:08:29] :D [20:20:32] nuria yt? [20:20:42] ottomata: free in 10 mins [20:20:57] k :) [20:36:15] ottomata: here [20:36:20] bc? [20:36:24] ottomata: k [20:37:05] ottomata: on bc [20:37:09] loading... [20:37:12] chrome is doing that thing again... [20:37:24] i switched to firefox but firefoxwon't load hangouts [20:56:59] 10Analytics, 10Analytics-EventLogging, 10Front-end-Standards-Group, 10MediaWiki-extensions-WikimediaEvents, and 2 others: Provide a reusable getEditCountBucket function for analytics purposes - https://phabricator.wikimedia.org/T210106 (10Nuria) Seems to me that a javascript package that bundles methods th... [21:03:34] (03CR) 10Joal: "One nit about a name - Awesomely clean :) Thanks Dan!" (031 comment) [analytics/refinery] - 10https://gerrit.wikimedia.org/r/492209 (https://phabricator.wikimedia.org/T215290) (owner: 10Milimetric) [21:13:32] 10Analytics, 10MediaWiki-API, 10Reading-Infrastructure-Team-Backlog: Load API request count and latency data from Hadoop to a dashboard - https://phabricator.wikimedia.org/T108414 (10Tgr) @Jhernandez, added some description. This originally came about when Developer Relations was planning a pivot towards ext... [21:20:29] (03CR) 10Joal: [V: 03+2 C: 03+2] "Ran unit-tests successfully, looks good, merging." [analytics/refinery] - 10https://gerrit.wikimedia.org/r/493285 (owner: 10Milimetric) [21:20:55] (03CR) 10Joal: [V: 03+2 C: 03+2] "Actually, not merging as it has a parent, but +2 :)" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/493285 (owner: 10Milimetric) [21:21:17] thanks joal, yeah, I chained those, I just wanted to test them all together [21:21:41] no prob milimetric - one name thing on first and good for me - Thanks for that, it looks great :) [21:21:44] testing now, will fix any mistakes and report back on the task [21:21:56] I agree with the name thing, thanks, will fix that [21:22:48] (03CR) 10Joal: "We need more than the hive creation script here: addition to the oozie/mediawiki/history/load job. Let me know if you want me to take over" [analytics/refinery] - 10https://gerrit.wikimedia.org/r/493286 (https://phabricator.wikimedia.org/T209549) (owner: 10Milimetric) [21:23:32] ottomata: would now be an ok time for you? [21:23:58] joal ya sure what's up? [21:24:22] ottomata: scala stuff - JsonNodes (easy), LogHelper (bit more complicateD) [21:24:28] k [21:24:32] bc or here? [21:24:35] good point joal: I'll update the load job [21:24:37] either is good! [21:24:44] cave for fast :) [21:24:49] please ottomata :) [21:25:16] k joining... [21:26:13] 10Analytics, 10MediaWiki-API, 10Reading-Infrastructure-Team-Backlog: Load API request count and latency data from Hadoop to a dashboard - https://phabricator.wikimedia.org/T108414 (10Anomie) >>! In T108414#4989503, @Tgr wrote: > Also, back then Reading Infrastructure was the team closest to owning the API so... [21:30:31] (03Abandoned) 10Milimetric: Switch back to sqooping from logging [analytics/refinery] - 10https://gerrit.wikimedia.org/r/487871 (owner: 10Milimetric) [21:34:03] 10Analytics, 10MediaWiki-API, 10Reading-Infrastructure-Team-Backlog: Load API request count and latency data from Hadoop to a dashboard - https://phabricator.wikimedia.org/T108414 (10Tgr) That's [[https://www.mediawiki.org/wiki/Wikimedia_Audiences/Better_use_of_data|Better use of data]]. I guess @kzimmerman... [21:45:55] (03CR) 10Joal: Update LogHelper for parameterization (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/493237 (owner: 10Joal) [21:46:06] (03PS2) 10Joal: Update LogHelper for parameterization [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/493237 [21:48:32] (03CR) 10Joal: Add JsonSchemaConverter to spark package (031 comment) [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/492756 (https://phabricator.wikimedia.org/T215442) (owner: 10Joal) [22:02:55] (03PS3) 10Joal: Add JsonSchemaConverter to spark package [analytics/refinery/source] - 10https://gerrit.wikimedia.org/r/492756 (https://phabricator.wikimedia.org/T215442) [22:03:06] Done for tonight team - see you tomorrow :) [22:17:06] 10Analytics, 10Analytics-Kanban, 10Product-Analytics, 10Reading Depth, 10Readers-Web-Backlog (Tracking): Whitelist sample flags and page/rev ID fields for ReadingDepth schema - https://phabricator.wikimedia.org/T216096 (10Tbayer) >>! In T216096#4976316, @mforns wrote: > @Tbayer event_sanitized.readingdep... [22:58:58] joal: https://meta.wikimedia.org/wiki/Research:Understanding_Wikidata_Queries fyi, will set up meeting about this with stas