Skip to content

Commit 568ca2b

Browse files
potiukephraimbuddy
authored andcommitted
Fix dag file processor hertbeat to run only if necessary (#30899)
The standalone file processor as of #30278 introduced accidentally an artifficial delay between dag processing by adding heartbeat but missing to set "only_if_necessary" flag to True. If your dag file processing has been fast (faster than the scheduler job_heartbeat_sec) this introduced unnecessary pause between the next dag file processor loop (up until the time passed), it also introduced inflation of the dag_processing_last_duration metrics (it would always show minimum job_heartbeat_sec) Adding "only_if_necessary" flag fixes the problem. Fixes: #30593 Fixes: #30884 (cherry picked from commit 00ab45f)
1 parent 3702656 commit 568ca2b

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

airflow/jobs/dag_processor_job_runner.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ def __init__(
5757
self.processor.heartbeat = lambda: perform_heartbeat(
5858
job=self.job,
5959
heartbeat_callback=empty_callback,
60-
only_if_necessary=False,
60+
only_if_necessary=True,
6161
)
6262

6363
def _execute(self) -> int | None:

0 commit comments

Comments
 (0)