The serialization and deserialization performance of Jackson3 is lower than that of Jackson2 ?

[brucelwl]

I conducted benchmark tests on Jackson3 and Jackson2 through JMH, but found that the performance of Jackson3 was much worse than that of Jackson2, which did not meet my expectations for Jackson3. I don't know if my test cases were unreasonable or if there are other reasons. Do you have a stress test report for Jackson3? Here is my test code

@BenchmarkMode(Mode.Throughput) 
@Warmup(iterations = 1) 
@Threads(15)  
@State(Scope.Benchmark) //@State(Scope.Benchmark)
@Measurement(iterations = 2, time = 10, timeUnit = TimeUnit.SECONDS)
//@Measurement(iterations = 2, batchSize = 1024)
@OutputTimeUnit(TimeUnit.MILLISECONDS) 
public class Jackson3BenchmarkTest {

    com.fasterxml.jackson.databind.ObjectMapper jackson2 = new com.fasterxml.jackson.databind.ObjectMapper();
    tools.jackson.databind.ObjectMapper jackson3 = new tools.jackson.databind.ObjectMapper();


    @Setup
    public void setup() throws Exception {

    }

    @Benchmark
    public void jackson2Test(Blackhole blackhole) throws Exception {
        UserInfo userInfo = new UserInfo();
        userInfo.setUsername("aaa");
        userInfo.setTags(Arrays.asList("aaa", "bbb", "中文你好"));

        String json = jackson2.writer().writeValueAsString(userInfo);
        UserInfo userInfo1 = jackson2.readValue(json, UserInfo.class);

        blackhole.consume(userInfo1);
    }

    @Benchmark
    public void jackson3Test(Blackhole blackhole) throws Exception {
        UserInfo userInfo = new UserInfo();
        userInfo.setUsername("aaa");
        userInfo.setTags(Arrays.asList("aaa", "bbb", "中文你好"));

        String json = jackson3.writer().writeValueAsString(userInfo);
        UserInfo userInfo1 = jackson3.readValue(json, UserInfo.class);

        blackhole.consume(userInfo1);
    }

    public static void main(String[] args) throws Exception {
        Options options = new OptionsBuilder().include(Jackson3BenchmarkTest.class.getName())
                //.output("benchmark/jedis-Throughput.log")
                .forks(0)
                .build();
        new Runner(options).run();
    }

}

[cowtowncoder]

@brucelwl Before drawing firm conclusions, I would increase warmup and test iterations to make sure this is not just measuring initialization: Jackson 3.x does more initial processing I think, so initialization would take longer (although even then 4x would be bit unexpected).

Generally I use runtime units of not iterations but seconds of runtime: warmup needs to be typically at least 3 seconds or so (I use 5), test time multiple seconds as well.
If warm up here is simply 1 write, 1 read of tiny object, it is a fraction of millisecond. :)
(apologies if I misread the set up)

Assuming this is corrected as necessary.... It would be good to know:

1. How do reading (deser) and writing (ser) differ -- split tests into two parts
2. What does UserInfo look like? (could you add that in description)

Benchmarks I have been using (https://github.com/FasterXML/jackson-benchmarks/) show 3.x results to be similar to 2.x although not better (with JDK 17), but not substantially worse in most cases
(interestingly enough, tho, JDK 8 is measurably faster for 2.x than 17, due to JDK's unfortunate ASCII String "optimization" -- but that is not different b/w Jackson 2.x and 3.x (3.x won't run on JDK 8 tho))

PS. Also: I would first remove multi-threading for tests (threads=15) when isolating numbers, esp. for small number of iterations. This to get more stable baseline -- and only then try out higher thread counts to see if that matters.

[brucelwl]

@cowtowncoder This is UserInfo code, a very simple entity class,

By the way, I would like to clarify that I am using Openjdk 25

import java.util.ArrayList;
import java.util.List;

public class UserInfo {
    
    String username;

    List<String> tags = new ArrayList<>();
    

    public String getUsername() {
        return username;
    }

    public void setUsername(String username) {
        this.username = username;
    }

    public List<String> getTags() {
        return tags;
    }

    public void setTags(List<String> tags) {
        this.tags = tags;
    }

}

[cowtowncoder]

@brucelwl Ok that's ... Lombok. Probably not relevant but all reproductions need to be Lombok-free (if I was to investigate).

[brucelwl]

@cowtowncoder ok I have updated the UserInfo code and removed the dependency on Lombok

[iProdigy]

Are you getting similar results with other JDK versions?

[brucelwl]

@cowtowncoder Jackson3 uses MethodHandle, which theoretically performs much faster than Jackson2 using reflection, but even if the performance is similar, it can still be difficult to get excited

[cowtowncoder]

Ok. But first things first: getting benchmark to be relevant -- if warmup is insufficient numbers are not useful.

[brucelwl]

@cowtowncoder I have tried adjusting the benchmark preheating conditions as you suggested, but the results are still very poor

[pjfanning]

My experience with MethodHandles is that they can be of similar performance to reflection or sun.misc.Unsafe. They are aimed at safety and not at improved performance as far as I am aware. If you use them in a suboptimal way they can be much slower.

[cowtowncoder]

Testing we did wrt MethodHandles showed modest (barely measurable) change with JDK 17, so unless JDK 25 changes things I would not assume they are cause here.

[cowtowncoder]

I can try tests in jackson-benchmarks with JDK 25 today if I have time to see if I can replicate.

Aside from that:

1. splitting ser/deser for tests would be useful
2. Test JSON seems tiny (not sure how representative) -- just a note, maybe that's realistic but seems bit off