chore: Added confluent-kafka-javascript to benchmarks.

[ShogunPanda]

Fixes #55.

[mcollina]

what are the results?

[ShogunPanda]

Producer:

╔════════════════════════════╤═════════╤═══════════════╤═══════════╤══════════════════════════╗
║ Slower tests               │ Samples │        Result │ Tolerance │ Difference with previous ║
╟────────────────────────────┼─────────┼───────────────┼───────────┼──────────────────────────╢
║ node-rdkafka               │     100 │ 169.94 op/sec │ ± 13.07 % │                          ║
║ confluent-kafka-javascript │     100 │ 171.87 op/sec │ ± 10.94 % │ + 1.14 %                 ║
║ kafkajs                    │     100 │ 377.57 op/sec │ ±  3.69 % │ + 119.68 %               ║
╟────────────────────────────┼─────────┼───────────────┼───────────┼──────────────────────────╢
║ Fastest test               │ Samples │        Result │ Tolerance │ Difference with previous ║
╟────────────────────────────┼─────────┼───────────────┼───────────┼──────────────────────────╢
║ @platformatic/kafka        │     100 │ 457.41 op/sec │ ±  3.88 % │ + 21.15 %                ║
╚════════════════════════════╧═════════╧═══════════════╧═══════════╧══════════════════════════╝

Consumer:

╔══════════════════════════════════════╤═════════╤══════════════════╤═══════════╤══════════════════════════╗
║ Slower tests                         │ Samples │           Result │ Tolerance │ Difference with previous ║
╟──────────────────────────────────────┼─────────┼──────────────────┼───────────┼──────────────────────────╢
║ node-rdkafka (evented)               │   10000 │   6203.41 op/sec │ ± 97.97 % │                          ║
║ confluent-kafka-javascript (evented) │   10000 │   6216.41 op/sec │ ± 97.44 % │ + 0.21 %                 ║
║ node-rdkafka (stream)                │   10000 │   7170.20 op/sec │ ± 81.06 % │ + 15.34 %                ║
║ confluent-kafka-javascript (stream)  │   10000 │   7283.01 op/sec │ ± 81.69 % │ + 1.57 %                 ║
║ kafkajs                              │   10000 │ 142199.72 op/sec │ ± 59.56 % │ + 1852.49 %              ║
╟──────────────────────────────────────┼─────────┼──────────────────┼───────────┼──────────────────────────╢
║ Fastest test                         │ Samples │           Result │ Tolerance │ Difference with previous ║
╟──────────────────────────────────────┼─────────┼──────────────────┼───────────┼──────────────────────────╢
║ @platformatic/kafka                  │   10000 │ 338564.21 op/sec │ ± 50.30 % │ + 138.09 %               ║
╚══════════════════════════════════════╧═════════╧══════════════════╧═══════════╧══════════════════════════╝

[ShogunPanda]

Note, this PR cannot land until confluentinc/confluent-kafka-javascript#306 since confluent-kafka-javascript doesn't currently work on Node 24.

[Dhruv-Garg79]

@ShogunPanda can you add these parameters to confluent kafka producer

'linger.ms': 2,
'batch.size': 12000,
'compression.codec': 'zstd',
'compression.type': 'zstd',
'socket.timeout.ms': 30000,
'socket.keepalive.enable': true,
'socket.nagle.disable': true

In my benchmarks, the results were vastly different after adding these

[ShogunPanda]

@Dhruv-Garg79 Did you add those parameters just to confluent or to others as well?

[Dhruv-Garg79]

The issue is that not all packages support all the parameters internally, even if they display all the parameters in the options. e.g. kafkajs does not support linger.ms, confluent-kafka does not support batching params on the consumer side properly.

For kafkajs, I did add zstd compression using @mongodb-js/zstd package

import { compress, decompress } from '@mongodb-js/zstd';
import { CompressionCodecs, CompressionTypes, Kafka, Partitioners, logLevel } from 'kafkajs';

CompressionCodecs[CompressionTypes.ZSTD] = () => {
	return {
		compress: async encoder => compress(encoder.buffer, 1),
		decompress: async buffer => decompress(buffer),
	};
};

await producer.send({
    topic: 'topic-name',
    compression: CompressionTypes.ZSTD,
    messages: [
        { key: 'key1', value: 'hello world' },
        { key: 'key2', value: 'hey hey!' }
    ],
  })

[ShogunPanda]

I see. Makes sense.
Anyway, unless @mcollina disagrees, I think the benchmark we currently provide are just for the basic bottom case supported by all drivers so I don't think I'll add more here.

[mcollina]

Unless we need it for some reason, I don't think we will work in creating a benchmark tailored to any individual use case.

[Dhruv-Garg79]

@ShogunPanda is it possible to add some documentation on how to run this benchmark locally? will help me to compare each with the best parameters possible (not all libraries have good default settings)

[ShogunPanda]

Nothing special, simply run:

node --experimental-strip-types benchmarks/consumer.ts
node --experimental-strip-types benchmarks/producer.ts

Note that there are scripts in benchmarks/utils folder to setup the local data.

[Dhruv-Garg79]

@ShogunPanda I was able to run producer benchmark, but not consumer one on my local.

I followed these steps:

docker compose -f docker/compose-cluster.yaml up
node --experimental-strip-types benchmarks/consumer.ts
node --experimental-strip-types benchmarks/utils/prepare-consumer-data.ts // in another terminal

The consumer benchmark got stuck for a long time. can you please help?

[Dhruv-Garg79]

Additionally, could you share some insights on how @platformic/kafka is so fast compared to others?
A blog post will be great on this. I already went through some of the blogs here, and they are amazing - https://blog.platformatic.dev

[Dhruv-Garg79]

@ShogunPanda @mcollina these are the results I am getting for producer.

❯ node --experimental-strip-types benchmarks/producer.ts
(node:2735) ExperimentalWarning: Type Stripping is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
╔════════════════════════════╤═════════╤══════════════╤═══════════╤══════════════════════════╗
║ Slower tests               │ Samples │       Result │ Tolerance │ Difference with previous ║
╟────────────────────────────┼─────────┼──────────────┼───────────┼──────────────────────────╢
║ @platformatic/kafka        │     100 │ 18.03 op/sec │ ± 11.20 % │                          ║
║ node-rdkafka               │     100 │ 19.21 op/sec │ ±  6.00 % │ + 6.55 %                 ║
║ confluent-kafka-javascript │     100 │ 20.38 op/sec │ ±  9.26 % │ + 6.09 %                 ║
╟────────────────────────────┼─────────┼──────────────┼───────────┼──────────────────────────╢
║ Fastest test               │ Samples │       Result │ Tolerance │ Difference with previous ║
╟────────────────────────────┼─────────┼──────────────┼───────────┼──────────────────────────╢
║ kafkajs                    │     100 │ 73.40 op/sec │ ±  2.44 % │ + 260.19 %               ║
╚════════════════════════════╧═════════╧══════════════╧═══════════╧══════════════════════════╝

kafkajs seems to be the most stable in terms of performance.

And I only made these changes to benchmark

diff --git a/benchmarks/producer.ts b/benchmarks/producer.ts
index 015fc18..f270d75 100644
--- a/benchmarks/producer.ts
+++ b/benchmarks/producer.ts
@@ -5,7 +5,7 @@ import RDKafka from 'node-rdkafka'
 import { ProduceAcks, Producer, stringSerializers } from '../src/index.ts'
 import { brokers, topic } from './utils/definitions.ts'
 
-const iterations = 10000
+const iterations = 100000
 const batchSize = Math.max(iterations / 100, 1)
 
 function rdkafka (): Promise<Result> {
@@ -76,6 +76,7 @@ function confluentKafka (): Promise<Result> {
   const producer = new ConfluentKafka.Producer({
     'client.id': 'benchmarks',
     'metadata.broker.list': brokers.join(','),
+    'linger.ms': 0,
     dr_cb: true,
     acks: 0
   })

node version - v22.16.0

[Dhruv-Garg79]

For consumer benchmarks with fetch.message.max.bytes: 20000 for all libraries.

❯ node --experimental-strip-types benchmarks/consumer.ts
(node:12539) ExperimentalWarning: Type Stripping is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
╔══════════════════════════════════════╤═════════╤════════════════╤═══════════╤══════════════════════════╗
║ Slower tests                         │ Samples │         Result │ Tolerance │ Difference with previous ║
╟──────────────────────────────────────┼─────────┼────────────────┼───────────┼──────────────────────────╢
║ @platformatic/kafka                  │   10000 │ 5373.43 op/sec │ ±  1.23 % │                          ║
║ confluent-kafka-javascript (evented) │   10000 │ 6071.01 op/sec │ ± 96.05 % │ + 12.98 %                ║
║ node-rdkafka (evented)               │   10000 │ 6075.77 op/sec │ ± 96.47 % │ + 0.08 %                 ║
║ kafkajs                              │   10000 │ 6717.55 op/sec │ ±  3.00 % │ + 10.56 %                ║
║ confluent-kafka-javascript (stream)  │   10000 │ 7602.98 op/sec │ ± 87.29 % │ + 13.18 %                ║
╟──────────────────────────────────────┼─────────┼────────────────┼───────────┼──────────────────────────╢
║ Fastest test                         │ Samples │         Result │ Tolerance │ Difference with previous ║
╟──────────────────────────────────────┼─────────┼────────────────┼───────────┼──────────────────────────╢
║ node-rdkafka (stream)                │   10000 │ 7658.99 op/sec │ ± 87.65 % │ + 0.74 %                 ║
╚══════════════════════════════════════╧═════════╧════════════════╧═══════════╧══════════════════════════╝

with fetch.message.max.bytes: 200000 for all libraries.

❯ node --experimental-strip-types benchmarks/consumer.ts
(node:12926) ExperimentalWarning: Type Stripping is an experimental feature and might change at any time
(Use `node --trace-warnings ...` to show where the warning was created)
╔══════════════════════════════════════╤═════════╤════════════════╤═══════════╤══════════════════════════╗
║ Slower tests                         │ Samples │         Result │ Tolerance │ Difference with previous ║
╟──────────────────────────────────────┼─────────┼────────────────┼───────────┼──────────────────────────╢
║ @platformatic/kafka                  │   10000 │ 1681.86 op/sec │ ±  1.01 % │                          ║
║ node-rdkafka (evented)               │   10000 │ 6066.09 op/sec │ ± 96.82 % │ + 260.68 %               ║
║ confluent-kafka-javascript (evented) │   10000 │ 6121.33 op/sec │ ± 96.71 % │ + 0.91 %                 ║
║ kafkajs                              │   10000 │ 6236.74 op/sec │ ±  2.48 % │ + 1.89 %                 ║
║ node-rdkafka (stream)                │   10000 │ 7577.48 op/sec │ ± 87.68 % │ + 21.50 %                ║
╟──────────────────────────────────────┼─────────┼────────────────┼───────────┼──────────────────────────╢
║ Fastest test                         │ Samples │         Result │ Tolerance │ Difference with previous ║
╟──────────────────────────────────────┼─────────┼────────────────┼───────────┼──────────────────────────╢
║ confluent-kafka-javascript (stream)  │   10000 │ 7711.77 op/sec │ ± 88.65 % │ + 1.77 %                 ║
╚══════════════════════════════════════╧═════════╧════════════════╧═══════════╧══════════════════════════╝

[Dhruv-Garg79]

@mcollina @ShogunPanda what are your thoughts?

[mcollina]

We are busy with other tasks. We will take a look as soon as possible, but it might take a few weeks.