[07:46:05] <pfischer>	 o/
[07:47:47] <dcausse>	 o/
[08:16:42] <pfischer>	 dcausse: I just checked if the AirFlow DAG for ALIS/SLIS-weekly-weighted-tags-via-kafka got kicked of, but the RestExternalTaskSensor waits for ~8h already (https://airflow-search.wikimedia.org/dags/image_suggestions_weekly/grid?dag_run_id=scheduled__2025-08-25T00%3A00%3A00%2B00%3A00&tab=code). I suspect execution_delta=1d is insufficient since ALIS/SLIS already ran 08-28. So I created the following MR: 
[08:16:42] <pfischer>	 https://gitlab.wikimedia.org/repos/data-engineering/airflow-dags/-/merge_requests/1644
[08:26:40] <dcausse>	 pfischer: looking
[08:33:13] <pfischer>	 here’s the link to ALIS: https://airflow-platform-eng.wikimedia.org/dags/ALIS/grid?dag_run_id=scheduled__2025-08-21T00%3A00%3A00%2B00%3A00
[09:10:14] <dcausse>	 pfischer: wondering if the search dag is not scheduled too early... and if so if it should wait for something in the "future"
[09:16:57] <pfischer>	 dcausse: according to the docs of ExternalTaskSensorbut the sensor is designed to look in the past. So you prefer starting the search DAG early? We could achieve that by modifying the logical execution date of the DAG and setting it to the same as ALIS/SLIS. But running after and accepting older completion states makes it more resilient, don’t you think? Just in case our DAG fails and you have to start it after ALIS 
[09:16:57] <pfischer>	 already finished (like today).
[09:19:23] <dcausse>	 pfischer: definitely, it's something I remember seeing with this dag, it starts sensing from our side while the slis/alis job is not even started... this makes it hard to understand
[09:20:21] <dcausse>	 the slis/alis dag is I think pushing data to a partition that does not corrspond to the data_interval_start
[09:21:57] <dcausse>	 I'm afraid that if we go with delta=6d we might push tags too late the next alis/slis job will run on stale data, the alis job depends on cirrus dumps to compute the delta of tags they need to ship
[09:28:25] <pfischer>	 dcausse: Alright, so we pin ourselves to specific ALIS/SLIS DAGs by specifying execution_date_fn for the external sensor instead?
[09:29:17] <gehel>	 pfischer, dcausse: would you have time for a chat to bring me up to date on what happened lately?
[09:29:36] <gehel>	 In case you do: https://meet.google.com/gbn-vibj-oci
[09:29:57] <dcausse>	 gehel: sorry, I'm in the middle of a deploy and have a meeting with Gabriele right after
[09:30:09] <gehel>	 I'll schedule something for this afternoon
[09:38:05] <dcausse>	 thanks!
[10:04:12] <dcausse>	 lunch
[12:42:55] <pfischer>	 picking up the kid + errands, back in 90’
[15:03:32] <gehel>	 pfischer: triage meetin? https://meet.google.com/eki-rafx-cxi
[15:06:44] <pfischer>	 Sorry, I'am stuck with the kid, my wife got sick
[15:08:39] <gehel>	 Damn! Take care of the kids! And the wife!
[17:16:13] <dcausse>	 dinner