clients disconnected with the new keepalive algorithm

[jesferman]

After updating mosquitto to v2.0.21 I get my clients randomly disconnected either at:

• Connection time
• After a random time

With the following message:

1749570348: New connection from 127.0.0.1:45268 on port 8883.
1749570348: Client <unknown> has exceeded timeout, disconnecting.

I tested it in previous versions and it didn't happened. I have been using my mqtt clients for months without this problem.

It seems related to the new keepalive algorithm backported to v2.0.21 by @ralight. See the end of this issue: #3138.

Important: it only happens if you set a max_keepalive value in mosquitto.conf (for my tests I use 60s). If you don't set the parameter, the error doesn't happen.

To reproduce it, set a max_keepalive value in your mosquitto.conf and use mosquitto_pub to send a test command. Repeat the mosquitto_pub command a few times and the error happens approx 1 out of 10 times.

Let me know if you need more information or I can help testing. Thanks!

[jesferman]

I don't have time right now to debug the algorithm but in case it helps to the original developer here is the ChatGPT analysis of the possible cause:

Here’s what’s happening under the hood:

1. Ring-buffer size and indexing

When you set

max_keepalive 60

the broker allocates a ring of size

$$\text{keepalive\_list\_max} = \frac{60 \times 3}{2} + 1 = 91$$

slots.
Each client with keepalive = 60 s is bound into slot

$$(\text{last\_msg\_in} + 1.5 \times 60)\bmod 91 = (\text{last\_msg\_in} + 90)\bmod 91$$

which is exactly where the broker will look when it believes the client’s 1.5× keepalive window has expired.

2. The timing mismatch

• At broker startup (or config reload), last_keepalive_check is initialized to the current time, say T₀.
• When a new client connects at time T₁ > T₀, its last_msg_in is (correctly) set to T₁, and it’s hashed into slot
  $(T₁ + 90)\bmod 91$.
• But the very next time keepalive__check() runs (which iterates every second over all seconds from last_keepalive_check up to now), it will scan the interval [T₀ … T₁) — which, by modular arithmetic, can include exactly the same slot index you just added the client to. As soon as the loop variable i hits a value such that

$$i \bmod 91 = (T₁ + 90)\bmod 91$$

the broker thinks “Aha! this client’s slot has fired,” and immediately disconnects it with a KEEPALIVE timeout.

3. Why you didn’t see it before

• With the default max_keepalive of 65535, your ring is enormous (98 303 slots), so $(T_1 + 1.5×)\bmod\mathrm{ring_size}$ never collides with any of the few seconds that get scanned on connection.
• As soon as you shrink the ring down to something on the order of your keepalive interval (like 91), that collision window appears, and clients get nuked almost immediately or after a “random” short time.

4. In summary

The back-ported O(1) implementation never updates last_keepalive_check when you add a new client, so your newly-connected client can end up in the “to be expired” slot right away if the small ring size wraps around into the current scan window. That is why setting max_keepalive to a low value suddenly causes perfectly well-behaved clients to get dropped with “exceeded timeout” messages.

---

Workarounds / fixes:

• Patch the new code so that:

  1. last_keepalive_check is bumped to db.now_s whenever you keepalive__add() a client, or
  2. the scanning loop skips the very first second after an add, or
  3. the ring buffer is always at least keepalive*3/2 + keepalive_scan_window in size (i.e. leave extra slack).