Measuring and comparing DAGs' parse time locally

[AlvaroCavalcante]

It's convenient to parse DAGs locally, as you can easily measure if your code modifications effectively reduce your DAG's parse time!

For this reason, I've created a simple Python library called airflow-parse-bench, that can help you to parse, measure, and compare your DAG parse time on your local machine.

To do so, you just need to install the lib by running the following:

pip install airflow-parse-bench

After that, you can measure your DAG parse time by running this command:

airflow-parse-bench --path your_path/dag_test.py

It will result in a table including the following columns:

• Filename: The name of the Python module containing the DAG. This unique name is the key to store DAG information.

• Current Parse Time: The time (in seconds) taken to parse the DAG.

• Previous Parse Time: The parse time from the previous run.

• Difference: The difference between the current and previous parse times.

• Best Parse Time: The best parse time recorded for the DAG.

If you have any doubts, check the project repository!

[potiuk]

Nice. You likely want to make a PR and add it to https://airflow.apache.org/ecosystem/