release GIL for detached smol::Executor

[ertosns]

Discussed in PyO3/pyo3#4775

^{Originally posted by ertosns December 6, 2024}
allow_threads doesn't release gil for detached smol::Executor in background, for example the following code block only run in the main thread, and the gil doesn't seem to be released.

async fn executor_run() {
    let ex = Arc::new(smol::Executor::new());
    let fut = ex.spawn(async {
        println!("executor run\n");
    });
    Python::with_gil(|gil| {
            gil.allow_threads(|| {
                fut.detach();
            })
    })
}

[davidhewitt]

Sorry for the slow reply. Can you demonstrate a more complete repro along with how you conclude that the GIL is not released? I see nothing obvious in this snippet.

[ertosns]

i need to following to work without having to sleep in the background until background function is called, if i uncomment the line where it sleeps for 1 sec, it will work. while it should if the gil is released since there is a loop in the background function, and it shouldn't exit.

async fn background(ex: Arc<smol::Executor<'_>>) {
    let fut = ex.clone().spawn(async {
        loop {
            println!("some interesting work!\n");
            async_std::task::sleep(std::time::Duration::from_secs(1)).await;
        }
    });
    Python::with_gil(|gil| {
        gil.allow_threads(|| {
            fut.detach();
        })
    });
    //async_std::task::sleep(std::time::Duration::from_secs(1)).await;
}

#[pyfunction]
fn background_caller(py: Python<'_>) ->  PyResult<Bound<PyAny>> {
    let ex = Arc::new(smol::Executor::new());
    pyo3_async_runtimes::async_std::future_into_py(py, async move {
        let fut = background(ex.clone());
        smol::future::block_on(ex.run(fut));
        Ok(())
    })
}

[davidhewitt]

The call to smol::future::block_on seems suspicious, that will block the calling thread. You probably need to find a different way to await the calling future.

I will move this issue to pyo3-async-runtimes, it does not look like a PyO3 problem.