Skip to content

Commit 4e8f9a6

Browse files
fabienrenaudfabienrenaud
committed
Run 2024-04-30
1 parent 3a45115 commit 4e8f9a6

File tree

3 files changed

+554
-47
lines changed

3 files changed

+554
-47
lines changed

‎README.md‎

Lines changed: 53 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,8 @@
44

55
## Purpose
66

7-
This project benchmarks the throughput performance of a variety of Java Json libraries using [JMH](http://openjdk.java.net/projects/code-tools/jmh/).
7+
This project benchmarks the throughput performance of a variety of Java Json libraries
8+
using [JMH](http://openjdk.java.net/projects/code-tools/jmh/).
89
It covers the following libraries:
910

1011
* [avaje-jsonb](https://github.com/avaje/avaje-jsonb)
@@ -32,11 +33,15 @@ It covers the following libraries:
3233
* [underscore-java](https://github.com/javadev/underscore-java)
3334

3435
When available, both databinding and 'stream' (custom packing and unpacking) implementations are tested.
35-
Two different kinds of [models](/src/main/java/com/github/fabienrenaud/jjb/model/) are evaluated with payloads of 1, 10, 100 and 1000 KB size:
36-
* [`Users`](/src/main/java/com/github/fabienrenaud/jjb/model/Users.java): uses primitive types, String, List and simple POJOs; and
36+
Two different kinds of [models](/src/main/java/com/github/fabienrenaud/jjb/model/) are evaluated with payloads of 1, 10,
37+
100 and 1000 KB size:
38+
39+
* [`Users`](/src/main/java/com/github/fabienrenaud/jjb/model/Users.java): uses primitive types, String, List and simple
40+
POJOs; and
3741
* [`Clients`](/src/main/java/com/github/fabienrenaud/jjb/model/Clients.java): adds arrays, enum, UUID, LocalDate
3842

3943
This benchmark is written to:
44+
4045
* randomly generate payloads upon static loading of the JVM/benchmark; the *seed* is also shared across runs
4146
* read data from RAM
4247
* write data to reusable output streams (when possible); this reduces allocation pressure
@@ -46,36 +51,37 @@ Not evaluated are: RAM utilization, compression, payloads > 1 MB.
4651

4752
## Results
4853

49-
The benchmarks are written with [JMH](http://openjdk.java.net/projects/code-tools/jmh/) and for Java 8.
54+
The benchmarks are written with [JMH](http://openjdk.java.net/projects/code-tools/jmh/) and for Java 17.
5055

51-
The results here-below were computed on November the 27th, 2021 with the following libraries and versions:
56+
The results here-below were computed on April the 30th, 2023 with the following libraries and versions:
5257

5358
| Library | Version |
5459
|--------------|----------|
55-
| jackson | 2.13.0 |
56-
| genson | 1.6 |
57-
| fastjson | 1.2.78 |
58-
| gson | 2.8.9 |
59-
| org.json | 20210307 |
60-
| javax-json | 1.1.4 |
61-
| json-io | 4.13.0 |
62-
| flexjson | 3.3 |
60+
| avaje-jsonb | 1.4 |
6361
| boon | 0.34 |
64-
| json-smart | 2.4.7 |
65-
| johnzon | 1.2.15 |
66-
| logansquare | 1.3.7 |
67-
| dsl-json | 1.9.9 |
68-
| simplejson | 1.1.1 |
69-
| nanojson | 1.7 |
62+
| dsl-json | 1.10.0 |
63+
| fastjson | 2.0.27 |
64+
| flexjson | 3.3 |
65+
| genson | 1.6 |
66+
| gson | 2.10.1 |
67+
| jackson | 2.14.2 |
7068
| jodd json | 6.0.3 |
71-
| moshi | 1.12.0 |
72-
| tapestry | 5.7.3 |
73-
| jsoniter | 0.9.23 |
69+
| johnzon | 1.2.19 |
70+
| jakarta | 2.1.1 |
71+
| json-io | 4.14.0 |
72+
| simplejson | 1.1.1 |
73+
| json-smart | 2.4.10 |
74+
| logansquare | 1.3.7 |
7475
| minimal-json | 0.9.5 |
7576
| mjson | 1.4.1 |
76-
| underscore | 1.71 |
77+
| moshi | 1.14.0 |
78+
| nanojson | 1.8 |
79+
| org.json | 20230227 |
7780
| purejson | 1.0.1 |
78-
| yasson | 1.0.9 |
81+
| qson | 1.1.1 |
82+
| tapestry | 5.8.2 |
83+
| underscore | 1.88 |
84+
| yasson | 3.0.2 |
7985

8086
[All graphs and sheets are available in this google doc.][spreadsheet]
8187

@@ -103,19 +109,18 @@ Note: fewer libraries are tested with this model due to lack of support for some
103109
**Serialization performance**
104110
![json serialization performance for primitive types, String, List and simple POJOs, arrays, enum, UUID, LocalDate][graph-clients-ser]
105111

106-
107112
### Benchmark configuration
108113

109114
Tests were run on an [Amazon EC2 c5.xlarge](https://aws.amazon.com/ec2/instance-types/c5/) (4 vCPU, 8 GiB RAM)
110115

111116
JMH info:
112117

113118
```
114-
# JMH version: 1.33
115-
# VM version: JDK 1.8.0_302, OpenJDK 64-Bit Server VM, 25.302-b08
116-
# VM invoker: /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.302.b08-0.amzn2.0.1.x86_64/jre/bin/java
117-
# VM options: -XX:+AggressiveOpts -Xms2g -Xmx2g
118-
# Blackhole mode: full + dont-inline hint (default, use -Djmh.blackhole.autoDetect=true to auto-detect)
119+
# JMH version: 1.35
120+
# VM version: JDK 17.0.6, OpenJDK 64-Bit Server VM, 17.0.6+10-LTS
121+
# VM invoker: /usr/lib/jvm/java-17-amazon-corretto.x86_64/bin/java
122+
# VM options: -Xms2g -Xmx2g --add-opens=java.base/java.time=ALL-UNNAMED
123+
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
119124
# Warmup: 5 iterations, 10 s each
120125
# Measurement: 10 iterations, 3 s each
121126
# Timeout: 10 min per iteration
@@ -129,8 +134,8 @@ JMH info:
129134

130135
Prerequisites:
131136

132-
* JDK 17; and JAVA_HOME set.
133-
* make
137+
* JDK 17; and JAVA_HOME set.
138+
* make
134139

135140
By default, running `./run ser` (`./run deser` respectively) will run
136141
all -- stream and databind -- serialization (deserialization respectively)
@@ -154,12 +159,14 @@ be patient.
154159

155160
Prerequisites:
156161

157-
* JDK 17; and JAVA_HOME set.
158-
* make
159-
* [packer](https://www.packer.io/)
160-
* [awscli](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) and configured via `aws configure`
162+
* JDK 17; and JAVA_HOME set.
163+
* make
164+
* [packer](https://www.packer.io/)
165+
* [awscli](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) and configured
166+
via `aws configure`
161167

162168
Then, simply run:
169+
163170
```
164171
make packer
165172
```
@@ -172,17 +179,17 @@ libraries is welcome.
172179
Adding a JSON library to the benchmark requires little work and you can
173180
find numerous examples in the commit history. For instance:
174181

175-
* Addition of moshi: https://github.com/fabienrenaud/java-json-benchmark/commit/6af2c0a7091b12a9dc768e49499682b97ea57ff6
176-
* Addition of jodd: https://github.com/fabienrenaud/java-json-benchmark/commit/288a4e61496588ed4c0a80e1f107f34f9a2c985c
177-
* Addition of json-simple: https://github.com/fabienrenaud/java-json-benchmark/commit/1e1e559c39a6eddc3dd7d7cea777fc7861415469
178-
182+
* Addition of moshi: https://github.com/fabienrenaud/java-json-benchmark/commit/6af2c0a7091b12a9dc768e49499682b97ea57ff6
183+
* Addition of jodd: https://github.com/fabienrenaud/java-json-benchmark/commit/288a4e61496588ed4c0a80e1f107f34f9a2c985c
184+
* Addition of
185+
json-simple: https://github.com/fabienrenaud/java-json-benchmark/commit/1e1e559c39a6eddc3dd7d7cea777fc7861415469
179186

180187
Pull requests are welcome.
181188

182189

183-
[jmh-results]: /archive/raw-results-2021-11-27.md
184-
[spreadsheet]: https://docs.google.com/spreadsheets/d/18XdXQi7GJmVgWBuo0BAsjbPUHW-qDHge1k4WT8uOEIU/edit?usp=sharing
185-
[graph-users-deser]: https://docs.google.com/spreadsheets/d/e/2PACX-1vSJsmkX9LTVyohgO8R8tZjIxdRCZugLLNeW42TLwsqdZEeNnSo0sGpVVQ2X8G2Ws7Cw9JXN9J46WZGE/pubchart?oid=1217359585&format=image
186-
[graph-users-ser]: https://docs.google.com/spreadsheets/d/e/2PACX-1vSJsmkX9LTVyohgO8R8tZjIxdRCZugLLNeW42TLwsqdZEeNnSo0sGpVVQ2X8G2Ws7Cw9JXN9J46WZGE/pubchart?oid=296776676&format=image
187-
[graph-clients-deser]: https://docs.google.com/spreadsheets/d/e/2PACX-1vSJsmkX9LTVyohgO8R8tZjIxdRCZugLLNeW42TLwsqdZEeNnSo0sGpVVQ2X8G2Ws7Cw9JXN9J46WZGE/pubchart?oid=684555912&format=image
188-
[graph-clients-ser]: https://docs.google.com/spreadsheets/d/e/2PACX-1vSJsmkX9LTVyohgO8R8tZjIxdRCZugLLNeW42TLwsqdZEeNnSo0sGpVVQ2X8G2Ws7Cw9JXN9J46WZGE/pubchart?oid=2004244401&format=image
190+
[jmh-results]: /archive/raw-results-2023-04-30.md
191+
[spreadsheet]: https://docs.google.com/spreadsheets/d/1KB2V8kxtXIgudNO4SpfYqZKn3z5OnFBP6bZVB9K2RUE/edit?usp=sharing
192+
[graph-users-deser]: https://docs.google.com/spreadsheets/d/e/2PACX-1vQjiR_OOk6FXipfmKiZCil9yAdM4BSPViKIOh0lvAEHcD-gVy_lGiUX86VBhr_xrzcz4VsLSHIzq2qY/pubchart?oid=1217359585&format=image
193+
[graph-users-ser]: https://docs.google.com/spreadsheets/d/e/2PACX-1vQjiR_OOk6FXipfmKiZCil9yAdM4BSPViKIOh0lvAEHcD-gVy_lGiUX86VBhr_xrzcz4VsLSHIzq2qY/pubchart?oid=296776676&format=image
194+
[graph-clients-deser]: https://docs.google.com/spreadsheets/d/e/2PACX-1vQjiR_OOk6FXipfmKiZCil9yAdM4BSPViKIOh0lvAEHcD-gVy_lGiUX86VBhr_xrzcz4VsLSHIzq2qY/pubchart?oid=684555912&format=image
195+
[graph-clients-ser]: https://docs.google.com/spreadsheets/d/e/2PACX-1vQjiR_OOk6FXipfmKiZCil9yAdM4BSPViKIOh0lvAEHcD-gVy_lGiUX86VBhr_xrzcz4VsLSHIzq2qY/pubchart?oid=2004244401&format=image

‎archive/all-runs.md‎

Lines changed: 67 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,65 @@
11
Benchmark runs by date
22

3+
## 2023-04-30
4+
5+
| Library | Version |
6+
|--------------|----------|
7+
| avaje-jsonb | 1.4 |
8+
| boon | 0.34 |
9+
| dsl-json | 1.10.0 |
10+
| fastjson | 2.0.27 |
11+
| flexjson | 3.3 |
12+
| genson | 1.6 |
13+
| gson | 2.10.1 |
14+
| jackson | 2.14.2 |
15+
| jodd json | 6.0.3 |
16+
| johnzon | 1.2.19 |
17+
| jakarta | 2.1.1 |
18+
| json-io | 4.14.0 |
19+
| simplejson | 1.1.1 |
20+
| json-smart | 2.4.10 |
21+
| logansquare | 1.3.7 |
22+
| minimal-json | 0.9.5 |
23+
| mjson | 1.4.1 |
24+
| moshi | 1.14.0 |
25+
| nanojson | 1.8 |
26+
| org.json | 20230227 |
27+
| purejson | 1.0.1 |
28+
| qson | 1.1.1 |
29+
| tapestry | 5.8.2 |
30+
| underscore | 1.88 |
31+
| yasson | 3.0.2 |
32+
33+
**`Users` model**
34+
35+
![json deserialization performance for primitive types, String, List and simple POJOs][20230430-graph-users-deser]
36+
![json serialization performance for primitive types, String, List and simple POJOs][20230430-graph-users-ser]
37+
38+
**`Clients` model**
39+
![json deserialization performance for primitive types, String, List and simple POJOs, arrays, enum, UUID, LocalDate][20230430-graph-clients-deser]
40+
![json serialization performance for primitive types, String, List and simple POJOs, arrays, enum, UUID, LocalDate][20230430-graph-clients-ser]
41+
42+
43+
**JMH configuration and hardware**
44+
45+
```
46+
# JMH version: 1.35
47+
# VM version: JDK 17.0.6, OpenJDK 64-Bit Server VM, 17.0.6+10-LTS
48+
# VM invoker: /usr/lib/jvm/java-17-amazon-corretto.x86_64/bin/java
49+
# VM options: -Xms2g -Xmx2g --add-opens=java.base/java.time=ALL-UNNAMED
50+
# Blackhole mode: compiler (auto-detected, use -Djmh.blackhole.autoDetect=false to disable)
51+
# Warmup: 5 iterations, 10 s each
52+
# Measurement: 10 iterations, 3 s each
53+
# Timeout: 10 min per iteration
54+
# Threads: 16 threads, will synchronize iterations
55+
# Benchmark mode: Throughput, ops/time
56+
```
57+
58+
**Links**
59+
60+
* [google spreadsheet][20230430-spreadsheet]
61+
* [raw-results-2023-04-30.md](/archive/raw-results-2023-04-30.md)
62+
363
## 2021-11-27
464

565
| Library | Version |
@@ -46,7 +106,7 @@ Same as below, JMH 1.33.
46106
**Links**
47107

48108
* [google spreadsheet][20211127-spreadsheet]
49-
* [raw-results-2020-03-01.md](/archive/raw-results-2021-11-27.md)
109+
* [raw-results-2021-11-27.md](/archive/raw-results-2021-11-27.md)
50110

51111

52112
## 2020-03-01
@@ -311,3 +371,9 @@ Same as below, JMH 1.15.
311371
[20211127-graph-users-ser]: https://docs.google.com/spreadsheets/d/e/2PACX-1vSJsmkX9LTVyohgO8R8tZjIxdRCZugLLNeW42TLwsqdZEeNnSo0sGpVVQ2X8G2Ws7Cw9JXN9J46WZGE/pubchart?oid=296776676&format=image
312372
[20211127-graph-clients-deser]: https://docs.google.com/spreadsheets/d/e/2PACX-1vSJsmkX9LTVyohgO8R8tZjIxdRCZugLLNeW42TLwsqdZEeNnSo0sGpVVQ2X8G2Ws7Cw9JXN9J46WZGE/pubchart?oid=684555912&format=image
313373
[20211127-graph-clients-ser]: https://docs.google.com/spreadsheets/d/e/2PACX-1vSJsmkX9LTVyohgO8R8tZjIxdRCZugLLNeW42TLwsqdZEeNnSo0sGpVVQ2X8G2Ws7Cw9JXN9J46WZGE/pubchart?oid=2004244401&format=image
374+
375+
[20230430-spreadsheet]: https://docs.google.com/spreadsheets/d/1KB2V8kxtXIgudNO4SpfYqZKn3z5OnFBP6bZVB9K2RUE/edit?usp=sharing
376+
[20230430-graph-users-deser]: https://docs.google.com/spreadsheets/d/e/2PACX-1vQjiR_OOk6FXipfmKiZCil9yAdM4BSPViKIOh0lvAEHcD-gVy_lGiUX86VBhr_xrzcz4VsLSHIzq2qY/pubchart?oid=1217359585&format=image
377+
[20230430-graph-users-ser]: https://docs.google.com/spreadsheets/d/e/2PACX-1vQjiR_OOk6FXipfmKiZCil9yAdM4BSPViKIOh0lvAEHcD-gVy_lGiUX86VBhr_xrzcz4VsLSHIzq2qY/pubchart?oid=296776676&format=image
378+
[20230430-graph-clients-deser]: https://docs.google.com/spreadsheets/d/e/2PACX-1vQjiR_OOk6FXipfmKiZCil9yAdM4BSPViKIOh0lvAEHcD-gVy_lGiUX86VBhr_xrzcz4VsLSHIzq2qY/pubchart?oid=684555912&format=image
379+
[20230430-graph-clients-ser]: https://docs.google.com/spreadsheets/d/e/2PACX-1vQjiR_OOk6FXipfmKiZCil9yAdM4BSPViKIOh0lvAEHcD-gVy_lGiUX86VBhr_xrzcz4VsLSHIzq2qY/pubchart?oid=2004244401&format=image

0 commit comments

Comments
 (0)