Skip to content

Conversation

@djshow832
Copy link
Collaborator

What problem does this PR solve?

Issue Number: close #940

Problem Summary:
Opening an S3 file takes up to 200ms and makes decoding very slow.

What is changed and how it works:
Split the opening process to another goroutine to speed up reading.

Check List

Tests

  • Unit test
  • Integration test
  • Manual test (add detailed scripts or steps below)
  • No code

Testing traffic replay with dry-run and now it can catch up with the upstream cluster.

Notable changes

  • Has configuration change
  • Has HTTP API interfaces change
  • Has tiproxyctl change
  • Other user behavior changes

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

@ti-chi-bot
Copy link

ti-chi-bot bot commented Oct 12, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@ti-chi-bot ti-chi-bot bot requested review from YangKeao and bb7133 October 12, 2025 12:57
@djshow832 djshow832 marked this pull request as ready for review October 12, 2025 12:57
@codecov-commenter
Copy link

Codecov Report

❌ Patch coverage is 78.72340% with 20 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@8492403). Learn more about missing BASE report.

Files with missing lines Patch % Lines
pkg/sqlreplay/store/rotate.go 78.72% 14 Missing and 6 partials ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##             main     #941   +/-   ##
=======================================
  Coverage        ?   67.53%           
=======================================
  Files           ?      136           
  Lines           ?    13405           
  Branches        ?        0           
=======================================
  Hits            ?     9053           
  Misses          ?     3727           
  Partials        ?      625           
Flag Coverage Δ
unit 67.53% <78.72%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Member

@bb7133 bb7133 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot
Copy link

ti-chi-bot bot commented Oct 12, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: bb7133

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ti-chi-bot
Copy link

ti-chi-bot bot commented Oct 12, 2025

[LGTM Timeline notifier]

Timeline:

  • 2025-10-12 23:26:46.791281805 +0000 UTC m=+50312.868534375: ☑️ agreed by bb7133.

@ti-chi-bot ti-chi-bot bot merged commit db32d61 into pingcap:main Oct 12, 2025
6 checks passed
@djshow832 djshow832 deleted the parallel branch October 13, 2025 00:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Reading S3 and decoding commands in parallel

3 participants