Skip to content

refactor: no more C::Responder = OneshotResponder for change_membership() #1368

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 16, 2025

Conversation

drmingdrmer
Copy link
Member

@drmingdrmer drmingdrmer commented May 15, 2025

Changelog

refactor: no more C::Responder = OneshotResponder for change_membership()

Before this commit, calling change_membership() required the
RaftTypeConfig::Responder to be configured with OneshotResponder.
This unnecessarily limited applications from using alternative
Responder types.

In this commit, we've removed this restriction from the
change_membership() method, allowing applications to use any responder
type they prefer. Internally, we now store different types of responders
for change_membership() and client_write() operations in an enum.

chore: add comment
  • Improvement

This change is Reviewable

@drmingdrmer drmingdrmer requested a review from schreter May 15, 2025 17:14
schreter
schreter previously approved these changes May 15, 2025
Copy link
Collaborator

@schreter schreter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not particularly thrilled at having an enum with the two responders, but not sure how to do it in a better way w/o yet another memory allocation.

We use our own InplaceBox<dyn Trait, SIZE> in our project to solve similar problem by storing the dynamic object in-place (limited by SIZE), but the stuff is not on crates.io (yet). Would it be potentially interesting for you?

Reviewed 10 of 10 files at r1, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @drmingdrmer)


openraft/src/raft/api/management.rs line 26 at r1 (raw file):

/// Provides management APIs for the Raft system.
///
/// This module contains methods for managing the Raft cluster, including

Same here.


openraft/src/raft/api/app.rs line 26 at r1 (raw file):

/// Provides application-facing APIs for interacting with the Raft system.
///
/// This module contains methods for client operations such as linearizable reads

This is not a module doc comment. Should it be maybe written with //!?

…rship()`

Before this commit, calling `change_membership()` required the
`RaftTypeConfig::Responder` to be configured with `OneshotResponder`.
This unnecessarily limited applications from using alternative
`Responder` types.

In this commit, we've removed this restriction from the
`change_membership()` method, allowing applications to use any responder
type they prefer. Internally, we now store different types of responders
for `change_membership()` and `client_write()` operations in an enum.
Copy link
Member Author

@drmingdrmer drmingdrmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

InplaceBox sounds great. I think the memory layout of InplaceBox would look like the following. I assume it requires an additional usize to store the vtable?

struct InplaceBox<T, const SIZE: usize> {
    storage: [MaybeUninit<u8>; SIZE],
    vtable: usize;
}

Reviewable status: 8 of 10 files reviewed, all discussions resolved (waiting on @schreter)


openraft/src/raft/api/app.rs line 26 at r1 (raw file):

Previously, schreter wrote…

This is not a module doc comment. Should it be maybe written with //!?

Hmm. AI refined these doc for me. I think 'module' here means the following struct but not a mod in rust. Replaced module with struct.

Copy link
Collaborator

@schreter schreter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, something similar:

pub struct InplaceBox<T: ?Sized, const SIZE: usize> {
    object_space: MaybeUninit<[u8; SIZE]>,
    vtable: AssertUnwindSafe<<T as Pointee>::Metadata>,
    _phantom: PhantomData<T>,
}

The vtable can't be usize, since otherwise miri can't track pointer provenance properly (integer-to-pointer cast). So we use pointer metadata instead. We also have several static assertions that check the alignment, size, etc. when constructing it via fn InplaceBox::new<'a, U: Sized + Unsize<T> + 'a>(value: U) -> Self. Unfortunately, our impl requires a few nightly features (but almost all of it is doable also without).

Reviewed 2 of 2 files at r2, all commit messages.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @drmingdrmer)


openraft/src/raft/api/app.rs line 26 at r1 (raw file):

Previously, drmingdrmer (张炎泼) wrote…

Hmm. AI refined these doc for me. I think 'module' here means the following struct but not a mod in rust. Replaced module with struct.

AI, hm... The I in AI is not yet there, hehe :-).

Copy link
Member Author

@drmingdrmer drmingdrmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting. TIL :)

I have a question: when storing a T: Trait as InplaceBox<dyn Trait, _>, the value of T should be copied to object_space. How to make sure the alignment of object_space the same as T? The alignment of [u8; _] is 1 AFAIK.

Reviewed 1 of 2 files at r2.
Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @drmingdrmer)

Copy link
Collaborator

@schreter schreter left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The alignment of Metadata is 8B, since it is a pointer (thus the entire object is 8B aligned). We statically assert that alignment of a concrete T passed to new() is less than or equal to the alignment of Metadata. Alternatively, using u64 instead of u8 and smaller SIZE also solves the issue.

Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @drmingdrmer)

Copy link
Member Author

@drmingdrmer drmingdrmer left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, That's a genius way!

Reviewable status: :shipit: complete! all files reviewed, all discussions resolved (waiting on @drmingdrmer)

@drmingdrmer drmingdrmer merged commit aa745dc into databendlabs:main May 16, 2025
35 checks passed
@drmingdrmer drmingdrmer deleted the 205-reduce-error branch May 16, 2025 17:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants