Skip to content

Optimise shared-blob-cache evictions #126581

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Changes from 1 commit
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
c86859c
Evict from the shared blob cache asynchronously
nicktindall Apr 10, 2025
c143dba
Only evict from shared cache when index is partial (SharedSnapshotInd…
nicktindall Apr 10, 2025
a62ac00
Update docs/changelog/126581.yaml
nicktindall Apr 10, 2025
6ae1e7c
Merge branch 'main' into evict_from_the_shared_blob_cache_asynchronously
nicktindall Apr 10, 2025
ff3a25d
Fix changelog
nicktindall Apr 10, 2025
2ef16c9
Update x-pack/plugin/blob-cache/src/main/java/org/elasticsearch/blobc…
nicktindall Apr 14, 2025
d3ce506
Fix indenting
nicktindall Apr 14, 2025
bd35686
evictionsRunner -> asyncEvictionsRunner
nicktindall Apr 14, 2025
83c1bda
Merge branch 'main' into evict_from_the_shared_blob_cache_asynchronously
nicktindall Apr 15, 2025
632afbc
Only evict asynchronously for shards we know are not coming back
nicktindall Apr 14, 2025
3274c1c
Merge remote-tracking branch 'origin/main' into evict_from_the_shared…
nicktindall Apr 15, 2025
253dba1
Merge remote-tracking branch 'origin/main' into evict_from_the_shared…
nicktindall Apr 22, 2025
f035f25
Merge branch 'main' into evict_from_the_shared_blob_cache_asynchronously
nicktindall Apr 23, 2025
7ac3220
Propagate IndexRemovalReason to deletion listeners
nicktindall Apr 23, 2025
8e18644
Fix naming (reasonMessage/reason)
nicktindall Apr 23, 2025
410fb35
Fix naming (reasonText/reason)
nicktindall Apr 23, 2025
2372056
Naming
nicktindall Apr 23, 2025
8c91b45
[CI] Auto commit changes from spotless
elasticsearchmachine Apr 23, 2025
87d1ba4
Naming/javadoc
nicktindall Apr 23, 2025
ea43b2d
randomReason()
nicktindall Apr 23, 2025
c6e7a05
Don't evict shards when IndexRemovalReason is FAILURE
nicktindall Apr 23, 2025
7eebc42
javadoc/naming
nicktindall Apr 23, 2025
250df4c
Merge remote-tracking branch 'origin/main' into evict_from_the_shared…
nicktindall May 20, 2025
d3cd806
Make IndexRemovalReason a top-level enum for sharing
nicktindall May 21, 2025
5eabc0f
Fix eviction logic
nicktindall May 21, 2025
53ad877
Comment
nicktindall May 21, 2025
fa17c66
Fix eviction logic
nicktindall May 21, 2025
69e748f
Improve change summary
nicktindall May 21, 2025
f498216
Merge branch 'main' into evict_from_the_shared_blob_cache_asynchronously
nicktindall May 21, 2025
185b390
Merge branch 'main' into evict_from_the_shared_blob_cache_asynchronously
nicktindall May 22, 2025
c2f3b0f
Add tests
nicktindall May 23, 2025
63f4b1f
Work with any number of nodes
nicktindall May 23, 2025
cd09f7d
Randomise number of docs
nicktindall May 23, 2025
69eb2e5
Merge branch 'main' into evict_from_the_shared_blob_cache_asynchronously
nicktindall May 23, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
evictionsRunner -> asyncEvictionsRunner
  • Loading branch information
nicktindall committed Apr 14, 2025
commit bd35686ad3dadd6ca452271db00ca6e47241647f
Original file line number Diff line number Diff line change
Expand Up @@ -338,7 +338,7 @@ private CacheEntry(T chunk) {
private final Runnable evictIncrementer;

private final LongSupplier relativeTimeInNanosSupplier;
private final ThrottledTaskRunner evictionsRunner;
private final ThrottledTaskRunner asyncEvictionsRunner;

public SharedBlobCacheService(
NodeEnvironment environment,
Expand Down Expand Up @@ -399,7 +399,7 @@ public SharedBlobCacheService(
this.blobCacheMetrics = blobCacheMetrics;
this.evictIncrementer = blobCacheMetrics.getEvictedCountNonZeroFrequency()::increment;
this.relativeTimeInNanosSupplier = relativeTimeInNanosSupplier;
this.evictionsRunner = new ThrottledTaskRunner(
this.asyncEvictionsRunner = new ThrottledTaskRunner(
"shared_blob_cache_evictions",
SHARED_CACHE_CONCURRENT_EVICTIONS_SETTING.get(settings),
threadPool.generic()
Expand Down Expand Up @@ -1633,7 +1633,7 @@ public int forceEvict(Predicate<KeyType> cacheKeyPredicate) {

@Override
public void forceEvictAsync(Predicate<KeyType> cacheKeyPredicate) {
evictionsRunner.enqueueTask(new ActionListener<>() {
asyncEvictionsRunner.enqueueTask(new ActionListener<>() {
@Override
public void onResponse(Releasable releasable) {
try (releasable) {
Expand Down
Loading