RunStatus of mlflow run is "FINISHED" instead of "FAILED" when the kedro run fails

[Galileo-Galilei]

Description

When I launch kedro run and the run fails, the on_pipeline_error closes all the mlflow runs (to avoid interactions with further runs)

Context

I cannot distinguish failed runs from sucessful ones in the mlflow ui.

Steps to Reproduce

Launch a failing pipeline with kedro run.

Expected Result

The mlflow ui should display the run with a red cross

Actual Result

The mlflow ui displays the run with a green tick

Does the bug also happen with the last version on develop?

Yes.

Potential solution:

Replace these lines:

https://github.com/Galileo-Galilei/kedro-mlflow/blob/63dcd501bfe98bebc81f25f70020ff4141c1e91c/kedro_mlflow/framework/hooks/pipeline_hook.py#L193-L194

with

while mlflow.active_run():
    mlflow.end_run(mlflow.entities.RunStatus.FAILED)

or even better, retrieve current run status from mlflow?

[takikadiri]

Good catch !
Since we catch the Error and manually end the run, mlflow do not receive the "error code 1" of the current process. If we no longer end run manually, mlflow will tag the run as FAILED. But since we want to control the pipeline error, we can apply your suggestion (specifiying the status as failed)

[Galileo-Galilei]

Yes, but we need to terminate the run manually when it failed and one use it interactively (in CLI, tis makes no difference because it gets the error code as you say) to avoid further interference.