Ugly crash when using MatplotlibWriter in Kedro 0.19.11

[astrojuanlu]

Description

I got a very ugly crash when using MatplotlibWriter in Kedro 0.19.11.

Confirmed that this works fine with Kedro 0.19.10.

Filing it directly because I think it could be related to kedro-org/kedro#4353, similar to Galileo-Galilei/kedro-mlflow#624 cc @merelcht

Steps to Reproduce

1. Clone https://github.com/kedro-org/kedro-academy/tree/main/kedro-image-classification
2. Install all the dependencies on Apple silicon, Python 3.11
3. kedro run

Expected Result

No errors ✔

Actual Result

Tell us what happens instead.

[02/14/25 18:00:56] INFO     Running node: preview_train_data: preview_train_data([ships_train;labels_train]) -> [preview_train_data]                                          node.py:367
                    WARNING  /Users/juan_cano/Projects/QuantumBlackLabs/kedro-image-classification/src/kedro_image_classification/pipelines/data_processing/nodes.py:33:   warnings.py:110
                             UserWarning: Starting a Matplotlib GUI outside of the main thread will likely fail.                                                                          
                               plt.figure(figsize=(15, 10))                                                                                                                               
                                                                                                                                                                                          
*** Terminating app due to uncaught exception 'NSInternalInconsistencyException', reason: 'NSWindow should only be instantiated on the main thread!'
*** First throw call stack:
(
        0   CoreFoundation                      0x000000018cbbae80 __exceptionPreprocess + 176
        1   libobjc.A.dylib                     0x000000018c6a2cd8 objc_exception_throw + 88
        2   CoreFoundation                      0x000000018cbdf534 _CFBundleGetValueForInfoKey + 0
        3   AppKit                              0x00000001906d0460 -[NSWindow _initContent:styleMask:backing:defer:contentView:] + 260
        4   AppKit                              0x00000001906d0350 -[NSWindow initWithContentRect:styleMask:backing:defer:] + 48
        5   _macosx.cpython-311-darwin.so       0x0000000313747900 -[Window initWithContentRect:styleMask:backing:defer:withManager:] + 52
        6   _macosx.cpython-311-darwin.so       0x000000031374a498 FigureManager_init + 216
        7   libpython3.11.dylib                 0x00000001065ee1a4 wrap_init + 20
        8   libpython3.11.dylib                 0x0000000105eff9b4 wrapperdescr_call + 640
        9   libpython3.11.dylib                 0x0000000105d77eb8 _PyEval_EvalFrameDefault + 183768
        10  libpython3.11.dylib                 0x0000000105ebc588 _PyFunction_Vectorcall + 472
        11  libpython3.11.dylib                 0x0000000105fa4050 slot_tp_init + 292
        12  libpython3.11.dylib                 0x0000000105fa72e4 type_call + 136
        13  libpython3.11.dylib                 0x0000000105d77eb8 _PyEval_EvalFrameDefault + 183768
        14  libpython3.11.dylib                 0x0000000105ebc588 _PyFunction_Vectorcall + 472
        15  libpython3.11.dylib                 0x0000000105f877cc method_vectorcall + 280
        16  libpython3.11.dylib                 0x0000000105d7becc _PyEval_EvalFrameDefault + 200172
        17  libpython3.11.dylib                 0x0000000105ebc588 _PyFunction_Vectorcall + 472
        18  libpython3.11.dylib                 0x0000000105d7becc _PyEval_EvalFrameDefault + 200172
        19  libpython3.11.dylib                 0x0000000105ebc588 _PyFunction_Vectorcall + 472
        20  libpython3.11.dylib                 0x0000000105d7bd3c _PyEval_EvalFrameDefault + 199772
        21  libpython3.11.dylib                 0x0000000105ebc588 _PyFunction_Vectorcall + 472
        22  libpython3.11.dylib                 0x0000000105dcacd4 _PyObject_Call_Prepend + 152
        23  libpython3.11.dylib                 0x0000000105dca9dc slot_tp_call + 104
        24  libpython3.11.dylib                 0x0000000105d7be28 _PyEval_EvalFrameDefault + 200008
        25  libpython3.11.dylib                 0x0000000105ebc588 _PyFunction_Vectorcall + 472
        26  libpython3.11.dylib                 0x0000000105d7bd3c _PyEval_EvalFrameDefault + 199772
        27  libpython3.11.dylib                 0x0000000105ebc588 _PyFunction_Vectorcall + 472
        28  libpython3.11.dylib                 0x0000000105f8780c method_vectorcall + 344
        29  libpython3.11.dylib                 0x0000000106006678 thread_run + 220
        30  libpython3.11.dylib                 0x0000000106613320 pythread_wrapper + 48
        31  libsystem_pthread.dylib             0x000000018ca602e4 _pthread_start + 136
        32  libsystem_pthread.dylib             0x000000018ca5b0fc thread_start + 8
)
libc++abi: terminating due to uncaught exception of type NSException

[astrojuanlu]

Although the real problem here is that matplotlib is not thread safe https://matplotlib.org/stable/users/faq.html#work-with-threads

[astrojuanlu]

Setting MPLBACKEND=agg did the trick

[astrojuanlu]

xref kedro-org/kedro#4486

[merelcht]

This is fixed in Change runner logic to not create pool for sequential runner kedro-org/kedro#4502 and will be part of the next release Kedro 0.19.12