You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
<p><b>(This unit is a reminder of material previously covered in section 5.7)</b></p>
6
-
<p>In Redis 6.2, a new stream trimming strategy was added. It is called <spanclass="code">MINID</span>, and allows us to trim a stream such that entries whose ID is lower than one provided are removed.</p>
7
-
<p>When combined with the default timestamp entry ID strategy, this gives us the ability to trim a stream to a given point in time.</p>
8
-
<p>In common with the <spanclass="code">MAXLEN</span> trimming strategy, <spanclass="code">MINID</span> can be used with both the <ahref="https://redis.io/commands/xadd/" target="_blank" class="page-link"><spanclass="code">XADD</span></a> and <ahref="https://redis.io/commands/xtrim/" target="_blank" class="page-link"><spanclass="code">XTRIM</span></a> commands, as well as with the <spanclass="code">~</span> modifier for approximate trimming.</p>
9
-
<p>Let's explore the <spanclass="code">MINID</span> trimming strategy using a stream where we've added a new entry every 10 minutes for a period of 3 hours between midnight and 2:50am UTC on January 1st 2025.</p>
10
-
<p>First we'll create the stream, giving each entry a single name/value pair in its payload. We store the date/time in <spanclass="code">YYYY-MM-DD HH:MM:SS</span> format so that we can easily see which time the entry ID timestamp represents:</p>
<p>Having created the stream, we can run some checks to make sure that it is the expected length and that the first and last entries represent the time period we are modeling:</p>
52
-
<p><preclass="code">
53
-
127.0.0.1:6379> XLEN demostream
54
-
(integer) 18
55
-
127.0.0.1:6379> XRANGE demostream - + COUNT 1
56
-
1) 1) "1735689600000-0"
57
-
2) 1) "dt"
58
-
2) "2025-01-01 00:00:00"
59
-
127.0.0.1:6379> XREVRANGE demostream + - COUNT 1
60
-
1) 1) "1735699800000-0"
5
+
<h2>Enhancements in Redis 6.2: The MINID Trimming Strategy</h2>
6
+
<p>In Redis 6.2, a new stream trimming strategy was added. It is called <spanclass="code">MINID</span>, and allows us to trim a stream such that entries whose ID is lower than one provided are removed.</p>
7
+
<p>When combined with the default timestamp entry ID strategy, this gives us the ability to trim a stream to a given point in time.</p>
8
+
<p>In common with the <spanclass="code">MAXLEN</span> trimming strategy, <spanclass="code">MINID</span> can be used with both the <ahref="https://redis.io/commands/xadd/" target="_blank" class="page-link"><spanclass="code">XADD</span></a> and <ahref="https://redis.io/commands/xtrim/" target="_blank" class="page-link"><spanclass="code">XTRIM</span></a> commands, as well as with the <spanclass="code">~</span> modifier for approximate trimming.</p>
9
+
<p>Let's explore the <spanclass="code">MINID</span> trimming strategy using a stream where we've added a new entry every 10 minutes for a period of 3 hours between midnight and 2:50am UTC on January 1st 2025.</p>
10
+
<p>First we'll create the stream, giving each entry a single name/value pair in its payload. We store the date/time in <spanclass="code">YYYY-MM-DD HH:MM:SS</span> format so that we can easily see which time the entry ID timestamp represents:</p>
<p>Having created the stream, we can run some checks to make sure that it is the expected length and that the first and last entries represent the time period we are modeling:</p>
52
+
<p><preclass="code">
53
+
127.0.0.1:6379> XLEN demostream
54
+
(integer) 18
55
+
127.0.0.1:6379> XRANGE demostream - + COUNT 1
56
+
1) 1) "1735689600000-0"
61
57
2) 1) "dt"
62
-
2) "2025-01-01 02:50:00"
63
-
</pre></p>
64
-
<h3>Time Based Trimming with XADD</h3>
65
-
<p>Imagine that we want to trim the stream so that only the first hour's entries are removed. Using the <spanclass="code">MINID</span> strategy, we can compute an appropriate timestamp value (in our application code) and use that as an argument to either <spanclass="code">XADD</span> or <spanclass="code">XTRIM</span>.</p>
66
-
<p>Here we're trimming the stream to remove entries older than 01:00:00 on Jan 1st 2025 (<spanclass="code">1735693200000</span>) while adding a new entry for 03:00:00 on Jan 1st 2025 (<spanclass="code">1735700400000</span>) at the same time:</p>
<p>Note that we don't have to add the sequence number to the value for <spanclass="code">MINID</span>. If not supplied, a sequence number of <spanclass="code">-0</span> is assumed.</p>
71
-
<p>Let's see how running this single command has affected the state of the stream:</p>
72
-
<p><preclass="code">
73
-
127.0.0.1:6379> XLEN demostream
74
-
(integer) 13
75
-
127.0.0.1:6379> XRANGE demostream - + COUNT 1
76
-
1) 1) "1735693200000-0"
77
-
2) 1) "dt"
78
-
2) "2025-01-01 01:00:00"
79
-
127.0.0.1:6379> XREVRANGE demostream + - COUNT 1
80
-
1) 1) "1735700400000-0"
81
-
2) 1) "dt"
82
-
2) "2025-01-01 03:00:00"
83
-
</p></pre>
84
-
<p>Before running the command there were 18 entries in the stream. The <spanclass="code">XADD</span> command added one more, and also atomically trimmed the 6 entries whose ID was less than <spanclass="code">1735693200000</span> (those entries covering the time period from 00:00 to 00:50).</p>
85
-
<p>This leaves us with a stream containing 13 entries that span the time period 01:00 to 03:00 on Jan 1st 2025.</p>
86
-
<h3>Time Based Trimming with XTRIM</h3>
87
-
<p>We can also trim the stream without adding a new entry to it. We do this with the <spanclass="code">XTRIM</span> command.</p>
88
-
<p>Let's use <spanclass="code">XTRIM</span> to remove entries older than 02:00 on Jan 1st 2025 UTC from our stream. As before, we need to calculate the milliseond timestamp for that date and time. It is <spanclass="code">1735696800000</span>.</p>
89
-
<p>Next we call <spanclass="code">XTRIM</span> using the <spanclass="code">MINID</span> trimming strategy:</p>
<p>Redis responds with the number of entries that were trimmed from the stream: 6 in this case.</p>
95
-
<p>Note that, in common with <spanclass="code">XADD</span>, we do not have to provide the sequence ID part of the minimum ID to trim to. <spanclass="code">-0</span> is implied unless an explicit sequence ID is provided.</p>
96
-
<p>Let's see how running <spanclass="code">XTRIM</span> has affected the state of the stream:</p>
97
-
<p><preclass="code">
98
-
127.0.0.1:6379> XLEN demostream
99
-
(integer) 7
100
-
127.0.0.1:6379> XRANGE demostream - +
101
-
1) 1) "1735696800000-0"
102
-
2) 1) "dt"
103
-
2) "2025-01-01 02:00:00"
104
-
2) 1) "1735697400000-0"
105
-
2) 1) "dt"
106
-
2) "2025-01-01 02:10:00"
107
-
3) 1) "1735698000000-0"
108
-
2) 1) "dt"
109
-
2) "2025-01-01 02:20:00"
110
-
4) 1) "1735698600000-0"
111
-
2) 1) "dt"
112
-
2) "2025-01-01 02:30:00"
113
-
5) 1) "1735699200000-0"
114
-
2) 1) "dt"
115
-
2) "2025-01-01 02:40:00"
116
-
6) 1) "1735699800000-0"
117
-
2) 1) "dt"
118
-
2) "2025-01-01 02:50:00"
119
-
7) 1) "1735700400000-0"
120
-
2) 1) "dt"
121
-
2) "2025-01-01 03:00:00"
122
-
</pre></p>
123
-
<p>The stream was trimmed to contain only the 7 most recent entries. These cover the time period from 02:00 to 03:00.</p>
124
-
<h3>Additional Resources</h3>
125
-
<p>For more information on the <spanclass="code">MINID</span> trimming strategy, check out the <ahref="https://redis.io/commands/xtrim/" target="_blank" class="page-link"><spanclass="code">XTRIM</span> command page</a> on redis.io.</p>
58
+
2) "2025-01-01 00:00:00"
59
+
127.0.0.1:6379> XREVRANGE demostream + - COUNT 1
60
+
1) 1) "1735699800000-0"
61
+
2) 1) "dt"
62
+
2) "2025-01-01 02:50:00"
63
+
</pre></p>
64
+
<h3>Time Based Trimming with XADD</h3>
65
+
<p>Imagine that we want to trim the stream so that only the first hour's entries are removed. Using the <spanclass="code">MINID</span> strategy, we can compute an appropriate timestamp value (in our application code) and use that as an argument to either <spanclass="code">XADD</span> or <spanclass="code">XTRIM</span>.</p>
66
+
<p>Here we're trimming the stream to remove entries older than 01:00:00 on Jan 1st 2025 (<spanclass="code">1735693200000</span>) while adding a new entry for 03:00:00 on Jan 1st 2025 (<spanclass="code">1735700400000</span>) at the same time:</p>
<p>Note that we don't have to add the sequence number to the value for <spanclass="code">MINID</span>. If not supplied, a sequence number of <spanclass="code">-0</span> is assumed.</p>
71
+
<p>Let's see how running this single command has affected the state of the stream:</p>
72
+
<p><preclass="code">
73
+
127.0.0.1:6379> XLEN demostream
74
+
(integer) 13
75
+
127.0.0.1:6379> XRANGE demostream - + COUNT 1
76
+
1) 1) "1735693200000-0"
77
+
2) 1) "dt"
78
+
2) "2025-01-01 01:00:00"
79
+
127.0.0.1:6379> XREVRANGE demostream + - COUNT 1
80
+
1) 1) "1735700400000-0"
81
+
2) 1) "dt"
82
+
2) "2025-01-01 03:00:00"
83
+
</p></pre>
84
+
<p>Before running the command there were 18 entries in the stream. The <spanclass="code">XADD</span> command added one more, and also atomically trimmed the 6 entries whose ID was less than <spanclass="code">1735693200000</span> (those entries covering the time period from 00:00 to 00:50).</p>
85
+
<p>This leaves us with a stream containing 13 entries that span the time period 01:00 to 03:00 on Jan 1st 2025.</p>
86
+
<h3>Time Based Trimming with XTRIM</h3>
87
+
<p>We can also trim the stream without adding a new entry to it. We do this with the <spanclass="code">XTRIM</span> command.</p>
88
+
<p>Let's use <spanclass="code">XTRIM</span> to remove entries older than 02:00 on Jan 1st 2025 UTC from our stream. As before, we need to calculate the milliseond timestamp for that date and time. It is <spanclass="code">1735696800000</span>.</p>
89
+
<p>Next we call <spanclass="code">XTRIM</span> using the <spanclass="code">MINID</span> trimming strategy:</p>
<p>Redis responds with the number of entries that were trimmed from the stream: 6 in this case.</p>
95
+
<p>Note that, in common with <spanclass="code">XADD</span>, we do not have to provide the sequence ID part of the minimum ID to trim to. <spanclass="code">-0</span> is implied unless an explicit sequence ID is provided.</p>
96
+
<p>Let's see how running <spanclass="code">XTRIM</span> has affected the state of the stream:</p>
97
+
<p><preclass="code">
98
+
127.0.0.1:6379> XLEN demostream
99
+
(integer) 7
100
+
127.0.0.1:6379> XRANGE demostream - +
101
+
1) 1) "1735696800000-0"
102
+
2) 1) "dt"
103
+
2) "2025-01-01 02:00:00"
104
+
2) 1) "1735697400000-0"
105
+
2) 1) "dt"
106
+
2) "2025-01-01 02:10:00"
107
+
3) 1) "1735698000000-0"
108
+
2) 1) "dt"
109
+
2) "2025-01-01 02:20:00"
110
+
4) 1) "1735698600000-0"
111
+
2) 1) "dt"
112
+
2) "2025-01-01 02:30:00"
113
+
5) 1) "1735699200000-0"
114
+
2) 1) "dt"
115
+
2) "2025-01-01 02:40:00"
116
+
6) 1) "1735699800000-0"
117
+
2) 1) "dt"
118
+
2) "2025-01-01 02:50:00"
119
+
7) 1) "1735700400000-0"
120
+
2) 1) "dt"
121
+
2) "2025-01-01 03:00:00"
122
+
</pre></p>
123
+
<p>The stream was trimmed to contain only the 7 most recent entries. These cover the time period from 02:00 to 03:00.</p>
124
+
<h2>Further Enhancements in Redis 6.2: The LIMIT Clause for Approximate Trimming</h2>
125
+
<p>Redis 6.2 also added a <spanclass="code">LIMIT</span> clause, which gives you a finer level of control over the time that stream trimming can take. This works with both the <spanclass="code">XADD</span> and <spanclass="code">XTRIM</span> commands in conjunction with the <spanclass="code">~</span> approximate trimming modifier.</p>
126
+
<p>Imagine that we want our stream length to trend towards a 5000 entry cap, but that each time we add something new we only want to trim at most 1000 entries from the stream at a time. This helps us balance the time taken to run the trimming command with the need for other clients to have their commands executed by Redis.</p>
127
+
<p>First, let's create a very basic stream with 10000 entries in it:</p>
128
+
<p><preclass="code">
129
+
127.0.0.1:6379> DEL demostream
130
+
(integer) 0
131
+
127.0.0.1:6379> 10000 XADD demostream * hello world
132
+
"1680698899048-0"
133
+
"1680698899050-0"
134
+
"1680698899051-0"
135
+
...
136
+
127.0.0.1:6379> XLEN demostream
137
+
(integer) 10000
138
+
</pre></p>
139
+
<p>Now, let's run an <spanclass="code">XTRIM</span> command that trims the stream towards an approximate 5000 entries, removing 1000 at a time at most:</p>
<p>Finally, let's run the command without the <spanclass="code">LIMIT</span> modifier:</p>
154
+
<p><preclass="code">
155
+
127.0.0.1:6379> XTRIM demostream MAXLEN ~ 5000
156
+
(integer) 3000
157
+
127.0.0.1:6379> XLEN demostream
158
+
(integer) 5000
159
+
</pre></p>
160
+
<p>This trims more entries in a single command (5000), but that command takes longer to execute.</p>
161
+
<p>Use the <spanclass="code">LIMIT</span> modifier if you want to have fine control over the throughput in Redis when trimming your stream to an approximate number with either <spanclass="code">XTRIM</span> or <spanclass="code">XADD</span>.</p>
162
+
<p>You can also use the <spanclass="code">LIMIT</span> clause when using the <spanclass="code">MINID</span> trimming strategy.</p>
163
+
<h2>Additional Resources</h2>
164
+
<p>For more information on the <spanclass="code">MINID</span> trimming strategy and <spanclass="code">LIMIT</span> clause, check out the <ahref="https://redis.io/commands/xtrim/" target="_blank" class="page-link"><spanclass="code">XTRIM</span> command page</a> on redis.io.</p>
0 commit comments