Skip to content

Commit 60935c6

Browse files
authored
Adjust sizing guidance re. doc count (elastic#97831)
In elastic#87246 we describe some reasons why it's a good idea to limit the doc count of a shard, and we started to do so in elastic#94065, so this commit adjusts the sizing guidance docs to match.
1 parent bc29549 commit 60935c6

File tree

1 file changed

+15
-14
lines changed

1 file changed

+15
-14
lines changed

docs/reference/how-to/size-your-shards.asciidoc

Lines changed: 15 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -140,20 +140,21 @@ Every new backing index is an opportunity to further tune your strategy.
140140

141141
[discrete]
142142
[[shard-size-recommendation]]
143-
==== Aim for shard sizes between 10GB and 50GB
144-
145-
Larger shards take longer to recover after a failure. When a node fails, {es}
146-
rebalances the node's shards across the data tier's remaining nodes. This
147-
recovery process typically involves copying the shard contents across the
148-
network, so a 100GB shard will take twice as long to recover than a 50GB shard.
149-
In contrast, small shards carry proportionally more overhead and are less
150-
efficient to search. Searching fifty 1GB shards will take substantially more
151-
resources than searching a single 50GB shard containing the same data.
152-
153-
There are no hard limits on shard size, but experience shows that shards
154-
between 10GB and 50GB typically work well for logs and time series data. You
155-
may be able to use larger shards depending on your network and use case.
156-
Smaller shards may be appropriate for
143+
==== Aim for shards of up to 200M documents, or with sizes between 10GB and 50GB
144+
145+
There is some overhead associated with each shard, both in terms of cluster
146+
management and search performance. Searching a thousand 50MB shards will be
147+
substantially more expensive than searching a single 50GB shard containing the
148+
same data. However, very large shards can also cause slower searches and will
149+
take longer to recover after a failure.
150+
151+
There is no hard limit on the physical size of a shard, and each shard can in
152+
theory contain up to just over two billion documents. However, experience shows
153+
that shards between 10GB and 50GB typically work well for many use cases, as
154+
long as the per-shard document count is kept below 200 million.
155+
156+
You may be able to use larger shards depending on your network and use case,
157+
and smaller shards may be appropriate for
157158
{enterprise-search-ref}/index.html[Enterprise Search] and similar use cases.
158159

159160
If you use {ilm-init}, set the <<ilm-rollover,rollover action>>'s

0 commit comments

Comments
 (0)