Skip to content

fix: ut mirroring tests #6341

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 4 commits into from
Sep 16, 2025
Merged

fix: ut mirroring tests #6341

merged 4 commits into from
Sep 16, 2025

Conversation

fracasula
Copy link
Collaborator

@fracasula fracasula commented Sep 12, 2025 •
edited
Loading

Description

We had data races in the UT mirroring tests because we were changing the actual slice passed to the transformer instead of creating a copy.

Once that was fixed, the tests showed that we were using the wrong golden file (i.e. goldenUtMirrorClientEvents.json should have had the original client events and not the transformer response).

Additionally, if a difference was detected between the original transformer and the mirrored one, we'd upload the original eventList concurrently. To avoid further races we're not passing a hard copy.

Linear Ticket

< Linear Ticket >

Security

  • The code changed/added as part of this pull request won't create any security issues with how the software is being used.

@fracasula fracasula marked this pull request as ready for review September 12, 2025 15:59
@codecov Codecov
Copy link

codecov bot commented Sep 12, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 42.85714% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.50%. Comparing base (077b0fb) to head (4295f7c).
⚠️ Report is 5 commits behind head on master.

Files with missing lines Patch % Lines
processor/processor.go 42.85% 2 Missing and 2 partials ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master    #6341      +/-   ##
==========================================
- Coverage   77.60%   77.50%   -0.10%     
==========================================
  Files         523      523              
  Lines       70252    70251       -1     
==========================================
- Hits        54522    54451      -71     
- Misses      12910    12957      +47     
- Partials     2820     2843      +23

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@fracasula fracasula merged commit a1d14d3 into master Sep 16, 2025
62 checks passed
@fracasula fracasula deleted the utMirroringTests branch September 16, 2025 07:44
@devops-github-rudderstack devops-github-rudderstack mentioned this pull request Sep 16, 2025
Merged
atzoum added a commit that referenced this pull request Sep 16, 2025
@atzoum
chore: sync release v1.59.0 to main branch (#6351)
51dd4f7 
# Description

Syncing release v1.59.0 to main branch

**WARNING: Do NOT rewrite git history and ALWAYS use a "Merge Commit"
for merging!**

**↓↓ Please review and edit commit overrides before merging ↓↓**

BEGIN_COMMIT_OVERRIDE
fix: ut mirroring tests (#6341)
chore: upgrade build-scan-push-action to v1.8.0 (#6350)
END_COMMIT_OVERRIDE
This was referenced Sep 29, 2025
Merged
Merged
mihir20 pushed a commit that referenced this pull request Sep 29, 2025
@devops-github-rudderstack
chore: prerelease 1.60.0-rc.1 (#6391)
1807354 
🤖 I have created a release *beep* *boop*
---


##
[1.60.0-rc.1](v1.59.0...v1.60.0-rc.1)
(2025-09-29)


### Features

* **router:** jobsdb pickup query batch size adapting to throttling
limit ([#6338](#6338))
([c38b971](c38b971))


### Bug Fixes

* add outgoing metrics to proxy flow
([#6355](#6355))
([23a5c81](23a5c81))
* aws session config region
([#6354](#6354))
([23a5c81](23a5c81))
* dedup gauge
([#6359](#6359))
([a562df4](a562df4))
* keydb grpc config
([#6370](#6370))
([cf04743](cf04743))
* missing keydb client stats
([#6360](#6360))
([c42e5c8](c42e5c8))
* naming collision in redis throttling configuration
([#6365](#6365))
([4f87afb](4f87afb))
* set table type to external for glue
([#6386](#6386))
([9017582](9017582))
* ut mirroring tests
([#6341](#6341))
([51dd4f7](51dd4f7))
* **warehouse:** alter namespace col size
([#6379](#6379))
([fbba498](fbba498))
* **warehouse:** skip extract async job failing test
([#6378](#6378))
([89a4d1e](89a4d1e))


### Miscellaneous

* add authentication to reporting client
([#6384](#6384))
([502d2b0](502d2b0))
* add explicit permissions for workflows
([#6381](#6381))
([1d22623](1d22623))
* add readme for async destinaiton module
([#6356](#6356))
([cc80333](cc80333))
* remove deprecated throttling configuration keys
([#6377](#6377))
([8d667b7](8d667b7))
* upgrade build-scan-push-action to v1.8.0
([#6350](#6350))
([51dd4f7](51dd4f7))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
@devops-github-rudderstack devops-github-rudderstack mentioned this pull request Sep 30, 2025
Merged
mihir20 pushed a commit that referenced this pull request Sep 30, 2025
@devops-github-rudderstack
chore: prerelease 1.60.0-rc.2 (#6401)
ddce825 
🤖 I have created a release *beep* *boop*
---


##
[1.60.0-rc.2](v1.59.0...v1.60.0-rc.2)
(2025-09-30)


### Features

* **router:** jobsdb pickup query batch size adapting to throttling
limit ([#6338](#6338))
([c38b971](c38b971))


### Bug Fixes

* add outgoing metrics to proxy flow
([#6355](#6355))
([23a5c81](23a5c81))
* aws session config region
([#6354](#6354))
([23a5c81](23a5c81))
* dedup gauge
([#6359](#6359))
([a562df4](a562df4))
* keydb grpc config
([#6370](#6370))
([cf04743](cf04743))
* missing keydb client stats
([#6360](#6360))
([c42e5c8](c42e5c8))
* naming collision in redis throttling configuration
([#6365](#6365))
([4f87afb](4f87afb))
* set table type to external for glue
([#6386](#6386))
([9017582](9017582))
* ut mirroring tests
([#6341](#6341))
([51dd4f7](51dd4f7))
* **warehouse:** alter namespace col size
([#6379](#6379))
([fbba498](fbba498))
* **warehouse:** skip extract async job failing test
([#6378](#6378))
([89a4d1e](89a4d1e))


### Miscellaneous

* add authentication to reporting client
([#6384](#6384))
([502d2b0](502d2b0))
* add explicit permissions for workflows
([#6381](#6381))
([1d22623](1d22623))
* add readme for async destinaiton module
([#6356](#6356))
([cc80333](cc80333))
* configurable event name trimming for reporting
([#6394](#6394))
([7628efa](7628efa))
* remove deprecated throttling configuration keys
([#6377](#6377))
([8d667b7](8d667b7))
* upgrade build-scan-push-action to v1.8.0
([#6350](#6350))
([51dd4f7](51dd4f7))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
This was referenced Sep 30, 2025
Closed
Merged
mihir20 pushed a commit that referenced this pull request Sep 30, 2025
@devops-github-rudderstack
chore: prerelease 1.60.0-rc.3 (#6405)
6a86d03 
🤖 I have created a release *beep* *boop*
---


##
[1.60.0-rc.3](v1.59.0...v1.60.0-rc.3)
(2025-09-30)


### Features

* **router:** jobsdb pickup query batch size adapting to throttling
limit ([#6338](#6338))
([c38b971](c38b971))


### Bug Fixes

* add outgoing metrics to proxy flow
([#6355](#6355))
([23a5c81](23a5c81))
* aws session config region
([#6354](#6354))
([23a5c81](23a5c81))
* dedup gauge
([#6359](#6359))
([a562df4](a562df4))
* keydb consistent hashing
([#6403](#6403))
([3ee79b2](3ee79b2))
* keydb grpc config
([#6370](#6370))
([cf04743](cf04743))
* missing keydb client stats
([#6360](#6360))
([c42e5c8](c42e5c8))
* naming collision in redis throttling configuration
([#6365](#6365))
([4f87afb](4f87afb))
* set table type to external for glue
([#6386](#6386))
([9017582](9017582))
* ut mirroring tests
([#6341](#6341))
([51dd4f7](51dd4f7))
* **warehouse:** alter namespace col size
([#6379](#6379))
([fbba498](fbba498))
* **warehouse:** skip extract async job failing test
([#6378](#6378))
([89a4d1e](89a4d1e))


### Miscellaneous

* add authentication to reporting client
([#6384](#6384))
([502d2b0](502d2b0))
* add explicit permissions for workflows
([#6381](#6381))
([1d22623](1d22623))
* add readme for async destinaiton module
([#6356](#6356))
([cc80333](cc80333))
* configurable event name trimming for reporting
([#6394](#6394))
([7628efa](7628efa))
* remove deprecated throttling configuration keys
([#6377](#6377))
([8d667b7](8d667b7))
* upgrade build-scan-push-action to v1.8.0
([#6350](#6350))
([51dd4f7](51dd4f7))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
mihir20 pushed a commit that referenced this pull request Oct 1, 2025
@devops-github-rudderstack
chore: release 1.60.0 (#6390)
bc7879c 
🤖 I have created a release *beep* *boop*
---


##
[1.60.0](v1.59.0...v1.60.0)
(2025-09-30)


### Features

* **router:** jobsdb pickup query batch size adapting to throttling
limit ([#6338](#6338))
([c38b971](c38b971))


### Bug Fixes

* add outgoing metrics to proxy flow
([#6355](#6355))
([23a5c81](23a5c81))
* aws session config region
([#6354](#6354))
([23a5c81](23a5c81))
* dedup gauge
([#6359](#6359))
([a562df4](a562df4))
* keydb consistent hashing
([#6403](#6403))
([3ee79b2](3ee79b2))
* keydb grpc config
([#6370](#6370))
([cf04743](cf04743))
* missing keydb client stats
([#6360](#6360))
([c42e5c8](c42e5c8))
* naming collision in redis throttling configuration
([#6365](#6365))
([4f87afb](4f87afb))
* set table type to external for glue
([#6386](#6386))
([9017582](9017582))
* ut mirroring tests
([#6341](#6341))
([51dd4f7](51dd4f7))
* **warehouse:** alter namespace col size
([#6379](#6379))
([fbba498](fbba498))
* **warehouse:** skip extract async job failing test
([#6378](#6378))
([89a4d1e](89a4d1e))


### Miscellaneous

* add authentication to reporting client
([#6384](#6384))
([502d2b0](502d2b0))
* add explicit permissions for workflows
([#6381](#6381))
([1d22623](1d22623))
* add readme for async destinaiton module
([#6356](#6356))
([cc80333](cc80333))
* configurable event name trimming for reporting
([#6394](#6394))
([7628efa](7628efa))
* remove deprecated throttling configuration keys
([#6377](#6377))
([8d667b7](8d667b7))
* upgrade build-scan-push-action to v1.8.0
([#6350](#6350))
([51dd4f7](51dd4f7))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
mihir20 pushed a commit that referenced this pull request Oct 1, 2025
@devops-github-rudderstack @mihir20
chore: release 1.60.0 (#6390)
c461070 
🤖 I have created a release *beep* *boop*
---


##
[1.60.0](v1.59.0...v1.60.0)
(2025-09-30)


### Features

* **router:** jobsdb pickup query batch size adapting to throttling
limit ([#6338](#6338))
([c38b971](c38b971))


### Bug Fixes

* add outgoing metrics to proxy flow
([#6355](#6355))
([23a5c81](23a5c81))
* aws session config region
([#6354](#6354))
([23a5c81](23a5c81))
* dedup gauge
([#6359](#6359))
([a562df4](a562df4))
* keydb consistent hashing
([#6403](#6403))
([3ee79b2](3ee79b2))
* keydb grpc config
([#6370](#6370))
([cf04743](cf04743))
* missing keydb client stats
([#6360](#6360))
([c42e5c8](c42e5c8))
* naming collision in redis throttling configuration
([#6365](#6365))
([4f87afb](4f87afb))
* set table type to external for glue
([#6386](#6386))
([9017582](9017582))
* ut mirroring tests
([#6341](#6341))
([51dd4f7](51dd4f7))
* **warehouse:** alter namespace col size
([#6379](#6379))
([fbba498](fbba498))
* **warehouse:** skip extract async job failing test
([#6378](#6378))
([89a4d1e](89a4d1e))


### Miscellaneous

* add authentication to reporting client
([#6384](#6384))
([502d2b0](502d2b0))
* add explicit permissions for workflows
([#6381](#6381))
([1d22623](1d22623))
* add readme for async destinaiton module
([#6356](#6356))
([cc80333](cc80333))
* configurable event name trimming for reporting
([#6394](#6394))
([7628efa](7628efa))
* remove deprecated throttling configuration keys
([#6377](#6377))
([8d667b7](8d667b7))
* upgrade build-scan-push-action to v1.8.0
([#6350](#6350))
([51dd4f7](51dd4f7))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@atzoum atzoum atzoum approved these changes

@ktgowtham ktgowtham ktgowtham approved these changes

@cisse21 cisse21 Awaiting requested review from cisse21

@mihir20 mihir20 Awaiting requested review from mihir20

@itsmihir itsmihir Awaiting requested review from itsmihir

@satishrudderstack satishrudderstack Awaiting requested review from satishrudderstack

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@fracasula @atzoum @ktgowtham