Skip to content

Calling async Python code from a spawned tokio task #48

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ShaneMurphy2 opened this issue May 21, 2025 · 2 comments
Open

Calling async Python code from a spawned tokio task #48

ShaneMurphy2 opened this issue May 21, 2025 · 2 comments

Comments

@ShaneMurphy2
Copy link

ShaneMurphy2 commented May 21, 2025

I'm trying to build some code that runs saved python scripts on demand. As such, the requests are ran in spawned tokio tasks. All of the documentation I could find shows running asynchrounous Python code in the main tokio task in the main function. Here is a minimal example that I tried, but I get behavior that is difficult to understand.

Minimal Example

use pyo3::prelude::*;
use pyo3::{PyResult, Python};

async fn server() -> PyResult<()> {
    let future = Python::with_gil(|py| -> PyResult<_> {
        let asyncio = py.import("asyncio")?;
        pyo3_async_runtimes::tokio::into_future(
            asyncio.call_method1("sleep", (1.into_pyobject(py).unwrap(),))?,
        )
    })?;

    let _ = future.await?;

    Ok(())
}

#[pyo3_async_runtimes::tokio::main]
async fn main() -> PyResult<()> {
    tokio::spawn(server()).await.unwrap()?;

    Ok(())
}

It's essentially the example here just inside tokio::spawn.

Output

<sys>:0: RuntimeWarning: coroutine 'sleep' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
Traceback (most recent call last):
  File "/opt/homebrew/Cellar/[email protected]/3.13.3/Frameworks/Python.framework/Versions/3.13/lib/python3.13/asyncio/base_events.py", line 719, in run_until_complete
    return future.result()
           ~~~~~~~~~~~~~^^
RuntimeError: no running event loop

There is more output, but I think this is the salient error.

Is this expected behavior? I read the section on how asyncio wants the main thread, but does it also need to be initialized per tokio worker thread? Some architectural context would be nice, or a specific design pattern. If there are changes required to support this, I'm happy to give them a shot.

*Updated to be a more minimal example.

@cpu
Copy link

cpu commented May 21, 2025

In a project with similar requirements I was getting this error until switching to a scope() with the current locals like:

let task_locals = Python::with_gil(pyo3_async_runtimes::tokio::get_current_locals)?;
tokio::spawn(pyo3_async_runtimes::tokio::scope(
    task_locals,
    .....,
));

@ShaneMurphy2
Copy link
Author

ShaneMurphy2 commented May 21, 2025

Hm, I tried something like that, but that only works if the python module and locals are constructed on the main thread. I want something where more like my updated example above.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants