python: Revive ws-demo

[gasmith]

Revive the python ws-demo, by creating a tokio runtime on demand.

Changes here:

• Create a runtime on demand if one doesn't exist.
• Hoist get_runtime_handle() out of websocket.rs to runtime.rs for possible future reuse.
• Teach get_runtime_handle() to reuse the current runtime, if one exists, instead of always creating a new one.
• Add a shutdown_runtime function, and invoke it from a python atexit hook.
• Teach python to drop the GIL before calling into async code.

Those last two are critical for avoiding SIGABRTs and deadlocks during python program exit. Since we use the pyo3-log forwarder, our tokio threads have a tendency to grab the GIL for logging, which means that we need to drop the GIL before blocking on anything going on in tokio.

Fixes: FG-10384
Fixes: FG-9902

[linear]

FG-10384 Fix python interface to live viz

[bryfox]

👍 I believe the drop/abort handling resolves FG-9902 as well.

use an existing runtime when one exists

Worth a patch release of the crate?

[amacneil]

tested and works for me

getting a protobuf deserialization error in FG, but that seems unrelated to the current PR

[gasmith]

Figured it out.

Turns out the GIL is held when you enter a #[pyfunction] or #[pymethod]. Seems completely obvious in retrospect, but I wasn't thinking about it. We really need to drop that before we do a block_on(future). This is particularly important because of pyo3-log, which grabs the GIL to write logs.

By dropping the GIL at the appropriate times, we no longer need the Runtime::shutdown_timeout hack. We can just drop the Runtime, and everything gets cleaned up.

[linear]

FG-9902 Better handling of server lifetime/closing

[gasmith]

👍 I believe the drop/abort handling resolves FG-9902 as well.

It certainly brings us in line with the rust implementation, so we'll mark it fixed for now. Thanks.

use an existing runtime when one exists

Worth a patch release of the crate?

I don't think so. It's OK to have multiple tokio runtimes. In fact, the isolation might not be such a bad thing. But it does have a cost, and I think our default should be to reuse an existing runtime when we have one.