Skip to content

[CI] AsyncSearchActionIT testCleanupOnFailure failing #126999

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
elasticsearchmachine opened this issue Apr 17, 2025 · 3 comments · Fixed by #127010
Closed

[CI] AsyncSearchActionIT testCleanupOnFailure failing #126999

elasticsearchmachine opened this issue Apr 17, 2025 · 3 comments · Fixed by #127010
Labels
needs:risk Requires assignment of a risk label (low, medium, blocker) :Search Foundations/Search Catch all for Search Foundations Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch >test-failure Triaged test failures from CI

Comments

@elasticsearchmachine
Copy link
Collaborator

Build Scans:

Reproduction Line:

./gradlew ":x-pack:plugin:async-search:internalClusterTest" --tests "org.elasticsearch.xpack.search.AsyncSearchActionIT.testCleanupOnFailure" -Dtests.seed=C8EF182320233484 -Dtests.locale=zh-Hant-MO -Dtests.timezone=Poland -Druntime.java=24

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

java.lang.AssertionError: 
Expected: an empty collection
     but: <[LEAK: ByteBuf.release() was not called before it's garbage-collected. See https://netty.io/wiki/reference-counted-objects.html for more information.
Recent access records: 
#1:
	org.elasticsearch.transport.netty4.WrappedByteBuf.release(WrappedByteBuf.java:1028)
	org.elasticsearch.transport.netty4.TrashingByteBuf.release(TrashingByteBuf.java:37)
	org.elasticsearch.transport.netty4.Netty4Utils$ByteBufRefCounted.decRef(Netty4Utils.java:172)
	org.elasticsearch.common.bytes.ReleasableBytesReference.close(ReleasableBytesReference.java:98)
	org.elasticsearch.transport.InboundPipeline.releasePendingBytes(InboundPipeline.java:140)
	org.elasticsearch.transport.InboundPipeline.doHandleBytes(InboundPipeline.java:97)
	org.elasticsearch.transport.InboundPipeline.handleBytes(InboundPipeline.java:69)
	org.elasticsearch.transport.netty4.Netty4MessageInboundHandler.channelRead(Netty4MessageInboundHandler.java:54)
	io.netty.channel.Abst
[truncated]

Issue Reasons:

  • [main] 2 consecutive failures in step part2
  • [main] 2 consecutive failures in pipeline elasticsearch-intake
  • [main] 2 failures in test testCleanupOnFailure (1.1% fail rate in 177 executions)
  • [main] 2 failures in step part2 (10.0% fail rate in 20 executions)
  • [main] 2 failures in pipeline elasticsearch-intake (10.0% fail rate in 20 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.
@elasticsearchmachine elasticsearchmachine added :Search/Search Search-related issues that do not fall into other categories >test-failure Triaged test failures from CI labels Apr 17, 2025
elasticsearchmachine added a commit that referenced this issue Apr 17, 2025
@elasticsearchmachine
Mute org.elasticsearch.xpack.search.AsyncSearchActionIT testCleanupOn…
ff07ce8 
…Failure #126999
@elasticsearchmachine
Copy link
Collaborator Author

This has been muted on branch main

Mute Reasons:

  • [main] 2 consecutive failures in step part2
  • [main] 2 consecutive failures in pipeline elasticsearch-intake
  • [main] 2 failures in test testCleanupOnFailure (1.1% fail rate in 177 executions)
  • [main] 2 failures in step part2 (10.0% fail rate in 20 executions)
  • [main] 2 failures in pipeline elasticsearch-intake (10.0% fail rate in 20 executions)

Build Scans:

@elasticsearchmachine elasticsearchmachine added Team:Search Meta label for search team needs:risk Requires assignment of a risk label (low, medium, blocker) labels Apr 17, 2025
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/es-search (Team:Search)

original-brownbear added a commit to original-brownbear/elasticsearch that referenced this issue Apr 17, 2025
@original-brownbear
Fix AsyncSearchActionIT tests
080dcd7 
Missed a spot here when moving this to delayed deserialization, we can leak pending batch results here on exceptions.

closes elastic#126994
closes elastic#126995
closes elastic#126975
closes elastic#126999
closes elastic#127001
closes elastic#126974
closes elastic#127008
@original-brownbear original-brownbear mentioned this issue Apr 17, 2025
Merged
@smalyshev smalyshev added :Search Foundations/Search Catch all for Search Foundations and removed :Search/Search Search-related issues that do not fall into other categories labels Apr 17, 2025
@elasticsearchmachine elasticsearchmachine added Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch and removed Team:Search Meta label for search team labels Apr 17, 2025
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/es-search-foundations (Team:Search Foundations)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
needs:risk Requires assignment of a risk label (low, medium, blocker) :Search Foundations/Search Catch all for Search Foundations Team:Search Foundations Meta label for the Search Foundations team in Elasticsearch >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix AsyncSearchActionIT tests original-brownbear/elasticsearch
2 participants
@smalyshev @elasticsearchmachine