CROSSSLOT Keys in request don't hash to the same slot

[shubsaini09]

Description

I want to insert 50k records everyday as a batch operation. I thought MSET or pipeline is the best option but i got an error CROSSSLOT Keys in request don't hash to the same slot. After some research i got to know that mset or pipeline stores in same hash slots. This is done by adding something constant {} in the key.

I want to use MSET or pipeline to get that performance while setting the keys but also store on different hash slots.

I also came across ioredis package which helps in writing to different slots using pipeline. I have not tried because i wanted to stick to official package.

Also, since my Redis cache is serverless, does it really matter whether the data is stored in the same hash slot or spread across different ones? I assume AWS handles this internally now. However, if I were managing a Redis cluster myself, slot distribution would matter—because sending all data to a single slot could create a hotspot and lead to CPU bottlenecks.

I am using AWS Redis Serverless 7.1
I am using node-redis 4.6.13 library to do above.

[PavelPashov]

Hello @shubsaini09,

node-redis performs auto-pipelining by default. See #2798 and this comment

As you’ve already mentioned, another approach is to use hash tags to ensure keys are grouped in the same slot. You can find more information about that here.

[shubsaini09]

@PavelPashov Thanks for the response. I was already aware of Promise.all, but I was trying to avoid it due to the additional complexity of manually creating and managing multiple promises. Using MSET or pipeline() feels much cleaner and more straightforward.

I also checked with AWS Support, and they confirmed that since I'm using serverless Redis, there's no need to explicitly manage hash slotting. AWS handles data distribution across hash slots internally to ensure optimal performance. That response was reassuring—it means I can confidently use MSET without worrying about performance impact.

However, in scenarios where Promise.all is necessary, could you suggest an optimal batch size for the promises and how many total batches/records should ideally be pushed at once?

[PavelPashov]

@shubsaini09 The optimal batch size will vary based on your application infrastructure and payload size, so I'd suggest testing with smaller batches and gradually scaling up. Monitors metrics like throughput, latency, and memory usage across different configurations to find what works best.