Skip to content

Make BufferedReader usable on all platforms to support read of append_vec always through File #6876

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Jul 9, 2025

Conversation

kskalski
Copy link

@kskalski kskalski commented Jul 8, 2025

Problem

BufferedReader is excluded from use on non-unix because read_at is a os::unix API, however there is similar API also available on windows, so we could extend its usage.

Summary of Changes

Narrow down platform-specific function call to read at offset and provide unix and windows implementation of it.

Notes:

  • not-unix and not-windows implementation is not provided - a stub that panics could be provided, but the goal is to make BufferedReader universally available and catch any other platform at compile time
  • windows implementation updates the internal file cursor, so file_io shouldn't be used when regular reads from file assume starting at 0 or last used offset

@kskalski kskalski changed the title Attempt to support read_into_buffer on non unix. Support read_into_buffer on windows Jul 8, 2025
@codecov-commenter
Copy link

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 83.3%. Comparing base (f39ea99) to head (2f0ef5c).

Additional details and impacted files
@@            Coverage Diff            @@
##           master    #6876     +/-   ##
=========================================
- Coverage    83.3%    83.3%   -0.1%     
=========================================
  Files         853      853             
  Lines      378508   378511      +3     
=========================================
- Hits       315482   315471     -11     
- Misses      63026    63040     +14     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@kskalski kskalski marked this pull request as ready for review July 8, 2025 11:18
@brooksprumo
Copy link

I dunno... I don't think we should add any support for Windows for anything IMO. We don't actually support running a real validator on Windows, so adding code that requires support and testing doesn't seem like a great idea.

@kskalski
Copy link
Author

kskalski commented Jul 8, 2025

This unifies the codebase and I was hoping to make this code an actual fallback for an even more linux-specific io_uring implementation of buffered reader.

Also I'm wondering it it would be possible to remove more code if this supports windows - e.g. mmap append vec is serving as fallback for not becing able to use BufferedReader

@kskalski kskalski changed the title Support read_into_buffer on windows Make BufferedReader usable on all platforms to support read of append_vec always through File Jul 9, 2025
Copy link

@brooksprumo brooksprumo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@brooksprumo brooksprumo requested a review from cpubot July 9, 2025 18:34
@brooksprumo
Copy link

Adding @cpubot as a reviewer, since he's touched a lot of this code recently.

Copy link

@cpubot cpubot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great :shipit:

@kskalski kskalski merged commit 56f6528 into anza-xyz:master Jul 9, 2025
41 checks passed
@kskalski kskalski deleted the ks/file_io branch July 10, 2025 04:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants