Skip to content

Topic listener parallel 2 #1805

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Jun 10, 2025
Merged

Topic listener parallel 2 #1805

merged 16 commits into from
Jun 10, 2025

Conversation

rekby
Copy link
Member

@rekby rekby commented Jun 9, 2025

Pull request type

Please check the type of change your PR introduces:

  • Bugfix
  • Feature
  • Code style update (formatting, renaming)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • Documentation content changes
  • Other (please describe):

Did

implemented parallelization for Topic Listener

@rekby rekby marked this pull request as draft June 9, 2025 19:06
Copy link

github-actions bot commented Jun 9, 2025

github.com/ydb-platform/ydb-go-sdk/v3/trace

incompatible changes

CoordinationEvents: value changed from 137438953472 to 1099511627776
DatabaseSQLConnEvents: value changed from 536870912 to 4294967296
DatabaseSQLConnectorEvents: value changed from 268435456 to 2147483648
DatabaseSQLEvents: value changed from 4026531840 to 32212254720
DatabaseSQLStmtEvents: value changed from 2147483648 to 17179869184
DatabaseSQLTxEvents: value changed from 1073741824 to 8589934592
DiscoveryEvents: value changed from 8589934592 to 68719476736
RatelimiterEvents: value changed from 68719476736 to 549755813888
RetryEvents: value changed from 4294967296 to 34359738368
SchemeEvents: value changed from 17179869184 to 137438953472
ScriptingEvents: value changed from 34359738368 to 274877906944
TopicEvents: value changed from 263979008 to 2143027200
TopicWriterEvents: value changed from 234881024 to 1879048192
TopicWriterStreamEvents: value changed from 67108864 to 536870912
TopicWriterStreamGrpcMessageEvents: value changed from 134217728 to 1073741824
TopicWriterStreamLifeCycleEvents: value changed from 33554432 to 268435456

compatible changes

Topic.OnListenerClose: added
Topic.OnListenerError: added
Topic.OnListenerInit: added
Topic.OnListenerReceiveMessage: added
Topic.OnListenerRouteMessage: added
Topic.OnListenerSendDataRequest: added
Topic.OnListenerSplitMessage: added
Topic.OnListenerStart: added
Topic.OnListenerUnknownMessage: added
Topic.OnPartitionWorkerHandlerCall: added
Topic.OnPartitionWorkerProcessMessage: added
Topic.OnPartitionWorkerStart: added
Topic.OnPartitionWorkerStop: added
TopicListenerCloseDoneInfo: added
TopicListenerCloseStartInfo: added
TopicListenerErrorInfo: added
TopicListenerEvents: added
TopicListenerInitDoneInfo: added
TopicListenerInitStartInfo: added
TopicListenerPartitionEvents: added
TopicListenerReceiveMessageInfo: added
TopicListenerRouteMessageInfo: added
TopicListenerSendDataRequestInfo: added
TopicListenerSplitMessageInfo: added
TopicListenerStartInfo: added
TopicListenerStreamEvents: added
TopicListenerUnknownMessageInfo: added
TopicListenerWorkerEvents: added
TopicOnListenerClose: added
TopicOnListenerError: added
TopicOnListenerInit: added
TopicOnListenerReceiveMessage: added
TopicOnListenerRouteMessage: added
TopicOnListenerSendDataRequest: added
TopicOnListenerSplitMessage: added
TopicOnListenerStart: added
TopicOnListenerUnknownMessage: added
TopicOnPartitionWorkerHandlerCall: added
TopicOnPartitionWorkerProcessMessage: added
TopicOnPartitionWorkerStart: added
TopicOnPartitionWorkerStop: added
TopicPartitionWorkerHandlerCallDoneInfo: added
TopicPartitionWorkerHandlerCallStartInfo: added
TopicPartitionWorkerProcessMessageDoneInfo: added
TopicPartitionWorkerProcessMessageStartInfo: added
TopicPartitionWorkerStartInfo: added
TopicPartitionWorkerStopDoneInfo: added
TopicPartitionWorkerStopStartInfo: added

summary

Base version: v3.109.0 (master)
Cannot suggest a release version.
Incompatible changes were detected.

@codecov-commenter
Copy link

codecov-commenter commented Jun 9, 2025

Codecov Report

Attention: Patch coverage is 53.18681% with 426 lines in your changes missing coverage. Please review.

Project coverage is 68.89%. Comparing base (9dc2525) to head (d5ff176).

Files with missing lines Patch % Lines
log/topic.go 1.91% 294 Missing and 13 partials ⚠️
...al/topic/topiclistenerinternal/partition_worker.go 79.12% 48 Missing and 9 partials ⚠️
...nal/topic/topiclistenerinternal/stream_listener.go 74.31% 49 Missing and 7 partials ⚠️
...ernal/topic/topiclistenerinternal/event_handler.go 50.00% 2 Missing and 1 partial ⚠️
internal/xsync/unbounded_chan.go 94.91% 2 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #1805      +/-   ##
==========================================
- Coverage   71.27%   68.89%   -2.39%     
==========================================
  Files         383      385       +2     
  Lines       39370    40154     +784     
==========================================
- Hits        28062    27663     -399     
- Misses      10172    11318    +1146     
- Partials     1136     1173      +37     
Flag Coverage Δ
experiment ?
go-1.21.x 68.86% <52.08%> (-0.59%) ⬇️
go-1.24.x 68.57% <53.18%> (-2.68%) ⬇️
integration 53.41% <39.01%> (-0.62%) ⬇️
macOS 40.26% <42.08%> (-0.25%) ⬇️
ubuntu 68.88% <53.18%> (-2.39%) ⬇️
unit 40.32% <43.18%> (-0.19%) ⬇️
windows 40.25% <41.86%> (-0.26%) ⬇️
ydb-24.4 52.47% <39.01%> (-0.68%) ⬇️
ydb-25.1 53.17% <39.01%> (-0.76%) ⬇️
ydb-latest 52.96% <39.01%> (-0.65%) ⬇️
ydb-nightly ?

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@rekby rekby marked this pull request as ready for review June 10, 2025 19:21
@rekby rekby merged commit 1889f16 into master Jun 10, 2025
46 of 48 checks passed
@rekby rekby deleted the topic-listener-parallel-2 branch June 10, 2025 19:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants