Skip to content

feat: add support for compio runtime #1364

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 14, 2025

Conversation

George-Miao
Copy link
Contributor

@George-Miao George-Miao commented May 8, 2025

  • feat: add compio support
  • feat: add unit test

Checklist

  • Updated guide with pertinent info (may not always apply).
  • Squash down commits to one or two logical commits which clearly describe the work you've done.
  • Unittest is a friend:)

This change is Reviewable

@drmingdrmer drmingdrmer self-requested a review May 8, 2025 12:16
Copy link
Member

@drmingdrmer drmingdrmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this supplementary runtime support!

There should be a CI to ensure it pass the test, similar to:
https://github.com/datafuselabs/openraft/blob/00771c8db3dd274fbc2c7461e56960602b94931a/.github/workflows/ci.yaml#L270

Reviewed 10 of 10 files at r1, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @George-Miao)


rt-compio/src/lib.rs line 44 at r1 (raw file):

    pub fn cancel(mut self) {
        // SAFETY: We are not using the JoinHandle anymore, so we can safely drop it.
        unsafe { ManuallyDrop::drop(&mut self.0) }

Since tasks are cancelled when their JoinHandle is dropped, spawned compio tasks should use detach() to keep them running in the background:

https://docs.rs/compio-runtime/0.7.0/compio_runtime/struct.Task.html#method.detach

OpenRaft only needs background tasks that run to completion without cancellation, so all spawned tasks can be detached. This eliminates the need for ManuallyDrop wrappers around task handles.

Code quote:

    pub fn cancel(mut self) {
        // SAFETY: We are not using the JoinHandle anymore, so we can safely drop it.
        unsafe { ManuallyDrop::drop(&mut self.0) }

Cargo.toml line 63 at r1 (raw file):

    "tests",
    "stores/memstore",
    "rt-compio"

A runtime does not have to be a member. Just put it in the following exclude section.

@George-Miao
Copy link
Contributor Author

Since tasks are cancelled when their JoinHandle is dropped, spawned compio tasks should use detach() to keep them running in the background:

The thing is, detach is a consuming function, meaning the handle will not be usable after detached, and the trait requires the handle to implement Future so the result can be retrieved after completion.

@George-Miao George-Miao requested a review from drmingdrmer May 11, 2025 04:42
@drmingdrmer
Copy link
Member

Since tasks are cancelled when their JoinHandle is dropped, spawned compio tasks should use detach() to keep them running in the background:

The thing is, detach is a consuming function, meaning the handle will not be usable after detached, and the trait requires the handle to implement Future so the result can be retrieved after completion.

Hmm... I missed this point.

Then it seems when CompioJoinHandle is dropped, it should detach the inner task. Using ManuallyDrop just leaks the inner Task if the user does not call cancel

@George-Miao
Copy link
Contributor Author

Then it seems when CompioJoinHandle is dropped, it should detach the inner task

That works too. I just need to wrap it with an Option.

Copy link
Member

@drmingdrmer drmingdrmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r2, 1 of 1 files at r3, 1 of 1 files at r4, all commit messages.
Reviewable status: all files reviewed, 3 unresolved discussions (waiting on @George-Miao)


rt-compio/src/lib.rs line 46 at r4 (raw file):

            task.cancel();
        }
    }

Why does it need a cancel API? OpenRaft does not need to cancel a task.

If it allows user to cancel it, the following poll() will just panic. Without it, everything looks good, right?

Code quote:

    pub fn cancel(mut self) {
        if let Some(task) = self.0.take() {
            task.cancel();
        }
    }

rt-compio/src/lib.rs line 73 at r4 (raw file):

impl<T> FusedFuture for CompioJoinHandle<T> {
    fn is_terminated(&self) -> bool {
        self.0.is_none()

Without cancel(), it does not need to be fused any more.

Did I miss anything?

Code quote:

impl<T> FusedFuture for CompioJoinHandle<T> {
    fn is_terminated(&self) -> bool {
        self.0.is_none()

@George-Miao
Copy link
Contributor Author

The internal handle is pub, so everyone can access and alter the state. I can remove all these and hide inner to users if you feel ok.

@drmingdrmer
Copy link
Member

The internal handle is pub, so everyone can access and alter the state. I can remove all these and hide inner to users if you feel ok.

Yes, it would be better. There is no need to leak internal non-required features to users.

One more public features just introduces additional maintenance burden.

@George-Miao George-Miao requested a review from drmingdrmer May 13, 2025 18:36
@drmingdrmer drmingdrmer force-pushed the feat/rt-compio branch 2 times, most recently from fcc055c to 7216e4c Compare May 14, 2025 06:46
drmingdrmer
drmingdrmer previously approved these changes May 14, 2025
Copy link
Member

@drmingdrmer drmingdrmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

Reviewed 1 of 1 files at r6, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @George-Miao)

@George-Miao
Copy link
Contributor Author

There was a problem in the code, fixed.

@George-Miao George-Miao requested a review from drmingdrmer May 14, 2025 07:04
drmingdrmer
drmingdrmer previously approved these changes May 14, 2025
Copy link
Member

@drmingdrmer drmingdrmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r8, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @George-Miao)

Implement runtime support for compio asynchronous framework with core
components:

- Add CompioRuntime implementation of AsyncRuntime trait
- Implement MPSC channels, oneshot, watch, and mutex primitives
- Integrate with GitHub Actions workflow for CI testing
- Add std::time::Instant implementation of Instant trait
Copy link
Member

@drmingdrmer drmingdrmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 1 of 1 files at r10, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @George-Miao)

@drmingdrmer drmingdrmer merged commit e5ab722 into databendlabs:main May 14, 2025
35 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants