[09:19:51] <gehel>	 dcausse: do you remember what the kafka retention is for the RDF stream generate by the WDQS Streaming Updater?
[09:20:12] <gehel>	 Context: how much lag can we recover from (in the context of the data reload)
[09:21:32] <dcausse>	 gehel: it's one month, last time we did catch up 2.23week in 24hours
[09:22:22] <dcausse>	 it's probable that this time will degrade as the journal grows tho
[09:26:19] <gehel>	 thx
[09:42:39] <gehel>	 I wrote a few things about the data reload failures: https://docs.google.com/document/d/14bI_EZRdGyk7lkiJm1SW8A8N-OvtgnvIEUt52-bukxg/edit#
[09:42:49] <gehel>	 If you have time to review before I share it further...
[09:45:01] <dcausse>	 sure
[10:00:55] <gehel>	 dcausse: I corrected following your comments. Let me know if this is better.
[10:06:06] <dcausse>	 gehel: re: "but the reality is that we do them more like 2-3 times per year" you fixed the comment but I think kept the same frequency
[10:06:44] <gehel>	 dcausse: quick meet? meet.google.com/dkc-xvqx-itm
[10:06:50] <dcausse>	 sure
[10:53:58] <dcausse>	 lunch
[10:59:18] <gehel>	 Lunch 
[14:10:27] <inflatador>	 <o/
[14:14:03] <dcausse>	 o/
[14:34:45] <dcausse>	 have to bump flink 1.16 mem from 2Gb to 5Gb to run without being killed by yarn...
[14:39:16] <inflatador>	 Yikes, I guess that's something we'll have to watch in k8s env too?
[14:40:41] <dcausse>	 yes but tbh in k8s today we give 2G to flink and it's being oomkilled as well
[14:40:57] <dcausse>	 not constantly like what I saw in yarn here
[14:41:35] <dcausse>	 I suppose that flink 1.16 is perhaps a bit more accurate in the mem it manages, esp rocksdb usage
[15:09:17] <gehel>	 weekly update posted: https://app.asana.com/0/0/1203844956020893
[15:26:23] <dcausse>	 will let the job run with flink 1.16 for the week-end and call it good enough if it's still alive
[16:04:03] <ebernhardson>	 \o
[16:04:56] <dcausse>	 o/
[16:18:51] <ebernhardson>	 for mjolnir...i'm thinking to create another mjolnir-deploy repo in gitlab that installs mjolnir from gerrit via a git+https://... reference i na requirements.txt, such that we can use the gitlab-ci conda environment building. seem resonable?
[16:21:54] <dcausse>	 why not move it all to gitlab?
[16:23:19] <dcausse>	 (I probably don't understand what you want to do)
[16:24:16] <ebernhardson>	 could move the entire thing to gitlab, but then we also have to change the deployment to search-loader instances. i guess i was thinking it's less work to create a repo that simply references gerrit
[16:24:40] <ebernhardson>	 search-loaders will still need the msearch daemon even when we deprecate mjolnir-bulk
[16:25:53] <dcausse>	 mjolnir-deploy@gitlab would just be here to test mjolnir with conda?
[16:26:21] <dcausse>	 and deploy the python deps with gitlab-ci
[16:26:48] <ebernhardson>	 yea, it would be for the migration to airflow v2 where we need a conda env built for it
[16:26:50] <dcausse>	 but these deployed python deps won't be used by the search-daemon deployment
[16:27:27] <dcausse>	 as we'd still be using mjolnir-deploy@gerrit for this
[16:27:39] <ebernhardson>	 i suppose i was thinking i moved one of the simplest things, next try one of the more complex pieces and see how it works
[16:28:47] <ebernhardson>	 right, there would basically be two deploy repos, one for the search-loader instances and another for airflow
[16:29:32] <ebernhardson>	 it's probably possible to get search-loaders to use the conda env, but not entirely sure how yet. It would amount to scap scripts that fetch and decompress the env instead of building it i suppose
[16:30:39] <dcausse>	 mjolnir-deploy@gitlab would just be here to make a package, could this be done in the existing discolytics repo instead (at least while we figure out a unified deployment system for both)?
[16:31:47] <ebernhardson>	 hmm, i think to integrate nicely with airflow 2 it would need to build and upload the env to gitlab's project packaging, having a dedicated repo that uses the same ci workflows would be the "obvious" way to go about it
[16:32:19] <ebernhardson>	 maybe one repo can build multiple env's, i suppose i'll have to look closer
[16:38:52] <gehel>	 unmeeting time: https://meet.google.com/xgq-wvik-dkp
[17:05:29] <dcausse>	 going offline, have a nice week-end
[17:30:50] <inflatador>	 heading to lunch, back in ~90
[19:24:10] * ebernhardson is slightly amazed how long it's taking conda to decide what the conflicts are in my attemt to conda-fy mjolnir
[20:26:58] <inflatador>	 back
[21:40:32] <ebernhardson>	 :S "pyspark 3.1.2 requires py4j==0.10.9, but you have py4j 0.10.9.7 which is incompatible."