Queues: Broken locally when using multiple dev commands

[astanciu]

What versions & operating system are you using?

MacOS latest, node 24, wrangler 4.22.0

Please provide a link to a minimal reproduction

https://github.com/astanciu/cf_worker_repro

Describe the Bug

In local development, when I start workers with separate dev commands, I'm unable to successfully send a message from a producer to a queue and have a consumer receive it.

For ex, doing:

# in producer
wrangler dev

# in consumer
wrangler dev

I see no errors in producer, presumably it puts the message on the queue successfully, but don't know how to verify this, I see nothing in the local state.

There is no response from the consumer, the queue() method never gets invoked.

However, if I start both together, using

wrangler dev -c producer/wrangler.jsonc -c consumer.jsonc

...then it works fine.

Please see the full reproduction https://github.com/astanciu/cf_worker_repro

Please provide any relevant error logs

No response

[CarmenPopoviciu]

hi @astanciu

wrangler currently only supports running queue producer/consumer Workers in a single local dev instance, see https://developers.cloudflare.com/queues/configuration/local-development/#separating-producer--consumer-workers. Is that something that you can do in your project? If yes, can you please give that a try and let us know if it solved the issue for you?

thank you 🙏

[astanciu]

Unfortunately, that really isn't an option for us. we have multiple workers that produce messages and some of them need to be individually addressed, some use vite plugin/build steps, etc... Running our entire dev setup from one worker also just feels super hacky.

But also, am I doing something out of the norm here? Shouldn't this be the primary use case, sending messages between two workers? It's very odd that this isn't supported.

Any plans to address this? It has pretty significant ramifications if we can't use queues locally.

[astanciu]

@CarmenPopoviciu why are we closing this issue, is this something CF won't support?

[Destreyf]

@astanciu The issue here is that the way the development environment is setup for wrangler, the "virtual" queue they use for local development is in process/memory afaik, this means it's not really possible to share across separate worker processes.

The only way I have seen to potentially address this, is to specify multiple configs using the wrangler command (as linked by @CarmenPopoviciu) but it may also be possible with vite when specifying the auxiliaryWorkers config property like this: https://developers.cloudflare.com/workers/development-testing/multi-workers/#single-dev-command

[astanciu]

@Destreyf I understand the challenge, but compared to all the other complex things that wrangler / miniflare / workerd are doing, I can only guess that creating a virtual queue would be on the easier side of things.

Why couldn't it be done with some combination of storing the messages on disk, a shared --persist-to location, or the dev registry. And some mechanism to tell the consumers to ingest, which you could do via hidden rpc calls, watching the message store for changes with chokidar, or a myriad of other potential options.

Would find it hard to believe this is a technical issue.