Incompatible Deserialization of Object Field in Couchbase with Shared Model

[jessaardithya]

Description :
We're encountering a deserialization issue when using Object as a field type in a shared model between MongoDB and Couchbase. While MongoDB handles this correctly, Spring Data Couchbase (4.x) fails to deserialize dynamic Object fields properly, especially when the actual content is a Map or custom structure.

This causes the deserialization process to fail or produce incorrect results when retrieving the document via repository.findById(...).

Here is the detail :

Original Code (Before Applying Any Fix) :

Environment Info

Spring Boot : 2.7.2
Spring Data Couchbase : 4.4.2

================
Sample Data Stored in Couchbase

{
  "createAt": {
    "dateTime": "2025-06-16T09:28:50.228722864Z",
    "offset": "+07:00",
    "zone": "Asia/Jakarta"
  },
  "description": "program 25",
  "details": [
    {
      "dId": "1047e6416da047e0bfc3c060b42156b8",
      "denom": 50000,
      "expirePolicy": {
        "date": {
          "dateTime": "2026-05-31T16:59:59Z",
          "offset": "+07:00",
          "zone": "Asia/Jakarta"
        },
        "days": 0,
        "method": "FIX"
      },
      "image": "http://s3.aws.com/bla/bla/bla/logo04.jpg",
      "printingVendor": "vendor",
      "quantity": 10,
      "templateId": "1384064500618047488",
      "type": "physical",
      "useCount": 1
    },
    {
      "dId": "962435cad4404a2292819c4be9de2825",
      "denom": 100000,
      "expirePolicy": {
        "days": 256,
        "method": "IN"
      },
      "image": "http://s3.aws.com/bla/bla/bla/logo04.jpg",
      "quantity": 10,
      "templateId": "1384064500618047488",
      "type": "electronic",
      "useCount": 1
    }
  ],
  "histories": [
    {
      "actDate": {
        "dateTime": "2025-06-16T09:28:50.265235737Z",
        "offset": "+07:00",
        "zone": "Asia/Jakarta"
      },
      "description": "program voucher created.",
      "operation": "submitted"
    },
    {
      "actDate": {
        "dateTime": "2025-06-16T09:28:54.898410761Z",
        "offset": "+07:00",
        "zone": "Asia/Jakarta"
      },
      "data": {
        "content": {
          "programId": "1384102411438731264",
          "toStatus": "generatingVoucherCode"
        },
        "type": "com.couchbase.client.java.json.JsonObject"
      },
      "description": "Generate voucher code by system for program 1384102411438731264",
      "operation": "generatingVoucherCode"
    }
  ],
  "programName": "program 25",
  "status": "generatingVoucherCode",
  "type": "io.vernoss.giftvoucher.model.Giftvoucherprogram"
}

================
Model

1. GiveVoucherPorgram

src/main/java/io/vernoss/giftvoucher/model/Giftvoucherprogram.java
package io.vernoss.giftvoucher.model;

import java.io.Serializable;
import java.time.ZonedDateTime;
import java.util.List;
import lombok.AllArgsConstructor;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.ToString;
import org.springframework.data.annotation.Id;

@Data
@ToString
@NoArgsConstructor
@AllArgsConstructor
public class Giftvoucherprogram implements Serializable {
    @Id
    private String id;
    private String programName;
    private String description;
    private List<GiftVoucherProgramDetail> details;
    private GiftVoucherRestriction restriction;
    private String status;
    private ZonedDateTime createAt = ZonedDateTime.now();
    private String createBy;
    private ZonedDateTime approvaAt;
    private String approvaBy;
    private String approvaReason;
    private String file;
    private List<History> histories;
}

2. GiftVoucherProgramDetail

src/main/java/io/vernoss/giftvoucher/model/GiftVoucherProgramDetail.java
package io.vernoss.giftvoucher.model;

import java.io.Serializable;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.ToString;

@Data
@ToString
@NoArgsConstructor
public class GiftVoucherProgramDetail implements Serializable {
    private String dId;
    private Integer denom;
    private Integer quantity;
    private String type;
    private Integer useCount = 1;
    private GiftVoucherProgramDetailExpirePolicy expirePolicy; 
    private String templateId;
    private String image;
    private String printingVendor;
}

3. GiftVoucherProgramDetailExpirePolicy

src/main/java/io/vernoss/giftvoucher/model/GiftVoucherProgramDetailExpirePolicy.java
package io.vernoss.giftvoucher.model;

import java.io.Serializable;
import java.time.ZonedDateTime;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.ToString;

@Data
@ToString
@NoArgsConstructor
public class GiftVoucherProgramDetailExpirePolicy implements Serializable {
    private String method;
    private int days;
    private ZonedDateTime date;
}

4. History

src/main/java/io/vernoss/giftvoucher/model/History.java
package io.vernoss.giftvoucher.model;

import com.fasterxml.jackson.annotation.JsonFormat;
import com.fasterxml.jackson.annotation.JsonFormat.Shape;
import java.io.Serializable;
import java.time.ZonedDateTime;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.ToString;

@Data
@ToString
@NoArgsConstructor
public class History implements Serializable {

    private String description;
    private String operation;
    private String lastStatus;
    private Object data;
    private String actBy;
    @JsonFormat(shape = Shape.STRING, pattern = "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'", timezone = "UTC")
    private ZonedDateTime actDate = ZonedDateTime.now();
}

Note: data is of type Object, which may contain either a plain string or structured nested JSON (e.g., Map<String, Object>).

5. GiftVoucherRestriction

src/main/java/io/vernoss/giftvoucher/model/GiftVoucherRestriction.java
package io.vernoss.giftvoucher.model;

import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.ToString;

@Data
@ToString
@NoArgsConstructor
public class GiftVoucherRestriction {

    private String storeCode;
    private String locationCode;
    private String companyCode;
    private String productCode;
    private String brandCode;
    private String categoryCode;
}

6. API Response

src/main/java/io/vernoss/giftvoucher/model/ApiResponse.java
package io.vernoss.giftvoucher.model;

import java.io.Serializable;
import lombok.Data;
import lombok.NoArgsConstructor;
import lombok.ToString;

@Data
@ToString
@NoArgsConstructor
public class ApiResponse<T> implements Serializable {

    private String responseCode;
    private String responseMessage;
    private T data;

    public ApiResponse(T payload) {
        this.responseCode = "00";
        this.data = payload;
    }

    public ApiResponse(String responseCode, T payload) {
        this.responseCode = responseCode;
        this.data = payload;
    }

    public ApiResponse(String responseCode, String responseMessage) {
        this.responseCode = responseCode;
        this.responseMessage = responseMessage;
    }
}

================
Service

public ApiResponse<Giftvoucherprogram> find(String id) throws GiftVoucherException {
        Giftvoucherprogram program = this.programRepository.findById(id)
                .orElseThrow(() -> new GiftVoucherException("Gift voucher program id " + id + " does not exists"));
        return new ApiResponse<>(program);
    }

We found another error for field type java.Object while get by Id

Got throw err

java.lang.IllegalStateException: Cannot set property content because no setter, no wither and it's not part of the persistence constructor private com.couchbase.client.java.json.JsonObject()

================
What We've Tried So Far

1. Create custom converter

package io.vernoss.giftvoucher.config;

import com.couchbase.client.core.error.BucketNotFoundException;
import com.couchbase.client.java.Bucket;
import com.couchbase.client.java.Cluster;
import com.couchbase.client.java.env.ClusterEnvironment;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Lazy;
import org.springframework.context.annotation.Primary;
import org.springframework.data.convert.CustomConversions;
import org.springframework.data.couchbase.config.AbstractCouchbaseConfiguration;
import org.springframework.data.couchbase.core.convert.CouchbaseCustomConversions;
import org.springframework.data.couchbase.repository.config.EnableCouchbaseRepositories;

import java.util.List;

@ConditionalOnProperty(name = "spring.data.database.type", havingValue = "couchbase", matchIfMissing = false)
@Slf4j
@Configuration
@EnableCouchbaseRepositories(basePackages = {"io.vernoss.giftvoucher"})
public class CouchbaseConfig extends AbstractCouchbaseConfiguration {

    @Value("${spring.data.couchbase.host}")
    private String host;

    @Value("${spring.data.couchbase.bucket}")
    private String bucket;

    @Value("${spring.data.couchbase.scope}")
    private String scope;

    @Value("${spring.data.couchbase.username}")
    private String username;

    @Value("${spring.data.couchbase.password}")
    private String password;

    @Override
    public String getConnectionString() {
        return host;
    }

    @Override
    public String getUserName() {
        return username;
    }

    @Override
    public String getPassword() {
        return password;
    }

    @Override
    public String getBucketName() {
        return bucket;
    }

    @Override
    public String getScopeName() {
        return scope;
    }

    @Override
    public String typeKey() {
        return "type";
    }

    @Override
    @Bean(destroyMethod = "disconnect")
    public Cluster couchbaseCluster(ClusterEnvironment couchbaseClusterEnvironment) {
        try {
            log.debug("Connecting to Couchbase cluster at " + host);
            return Cluster.connect(getConnectionString(), getUserName(), getPassword());
        } catch (Exception e) {
            log.error("Error connecting to Couchbase cluster", e);
            throw e;
        }
    }

    @Bean
    public Bucket getCouchbaseBucket(Cluster cluster) {
        try {
            if (!cluster.buckets().getAllBuckets().containsKey(getBucketName())) {
                log.error("Bucket with name {} does not exist. Creating it now", getBucketName());
                throw new BucketNotFoundException(bucket);
            }
            return cluster.bucket(getBucketName());
        } catch (Exception e) {
            log.error("Error getting bucket", e);
            throw e;
        }
    }

    @Bean
    @Primary // 🔥 Tambahkan @Primary untuk menghindari konflik dengan MongoDB
    @Lazy // 🔥 Gunakan @Lazy jika ada kemungkinan circular dependency
    public CustomConversions couchbaseCustomConversions() {
        return new CouchbaseCustomConversions(List.of(
              
                new ZonedDateTimeWriteConverter(),
                new ZonedDateTimeReadConverter(),
                new ObjectReadConverter()
        )) {
            @Override
            public boolean isSimpleType(Class<?> clazz) {
                if (java.time.ZonedDateTime.class.isAssignableFrom(clazz)) {
                    return false;
                }
                return super.isSimpleType(clazz);
            }
        };
    }
}

Object Read Converter

package io.vernoss.giftvoucher.config;

import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import java.util.Map;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.core.convert.converter.Converter;
import org.springframework.data.convert.ReadingConverter;
import org.springframework.data.couchbase.core.mapping.CouchbaseDocument;

/**
 *
 * @author skyfy
 */
@ReadingConverter
public class ObjectReadConverter implements Converter<CouchbaseDocument, Object> {

    private static final Logger log = LoggerFactory.getLogger(ObjectReadConverter.class);

    @Override
    public Object convert(CouchbaseDocument source) {
        log.debug("ObjectReadConverter");
        try {
            return new ObjectMapper().readValue(source.toString(), Map.class);
        } catch (JsonProcessingException e) {
            log.error("Failed to convert CouchbaseDocument to JsonNode", e);
            throw new RuntimeException("Conversion error in ObjectReadConverter", e);
        }
    }
}

Jackson Config

package io.vernoss.giftvoucher.config;

import com.fasterxml.jackson.datatype.jsr310.JavaTimeModule;
import io.vernoss.giftvoucher.model.History;
import org.springframework.boot.autoconfigure.jackson.Jackson2ObjectMapperBuilderCustomizer;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class JacksonConfig {
    @Bean
    public Jackson2ObjectMapperBuilderCustomizer addCustomHistorySerializer() {
        return builder -> {
            builder.serializerByType(History.class, new HistorySerializer());
            builder.modules(new JavaTimeModule());
        };
    }
}

History Seliazer

package io.vernoss.giftvoucher.config;

import com.fasterxml.jackson.core.JsonGenerator;
import com.fasterxml.jackson.databind.SerializerProvider;
import com.fasterxml.jackson.databind.ser.std.StdSerializer;
import io.vernoss.giftvoucher.model.History;

import java.io.IOException;

public class HistorySerializer extends StdSerializer<History> {

    public HistorySerializer() {
        this(null);
    }

    public HistorySerializer(Class<History> t) {
        super(t);
    }

    @Override
    public void serialize(History value, JsonGenerator gen, SerializerProvider provider) throws IOException {
        gen.writeStartObject();
        // Write all fields except "data"
        // Example: adjust these lines to match your actual fields
        gen.writeStringField("description", value.getDescription());
        gen.writeStringField("operation", value.getOperation());
        gen.writeStringField("lastStatus", value.getLastStatus());
        gen.writeStringField("actBy", value.getActBy());
        if (value.getActDate() != null) {
            gen.writeStringField("actDate", value.getActDate().toString());
        }
        gen.writeEndObject();
    }
}

We got err

org.springframework.core.convert.ConverterNotFoundException: No converter found capable of converting from type [org.springframework.data.couchbase.core.mapping.CouchbaseDocument] to type [io.vernoss.giftvoucher.model.Giftvoucherprogram]
	at org.springframework.core.convert.support.GenericConversionService.handleConverterNotFound(GenericConversionService.java:322) ~[spring-core-5.3.22.jar:5.3.22]
	at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:195) ~[spring-core-5.3.22.jar:5.3.22]
	at org.springframework.core.convert.support.GenericConversionService.convert(GenericConversionService.java:175) ~[spring-core-5.3.22.jar:5.3.22]
	at org.springframework.data.couchbase.core.convert.MappingCouchbaseConverter.read(MappingCouchbaseConverter.java:234) ~[spring-data-couchbase-4.4.2.jar:4.4.2]
	at org.springframework.data.couchbase.core.convert.MappingCouchbaseConverter.read(MappingCouchbaseConverter.java:200) ~[spring-data-couchbase-4.4.2.jar:4.4.2]
	at org.springframework.data.couchbase.core.convert.MappingCouchbaseConverter.read(MappingCouchbaseConverter.java:85) ~[spring-data-couchbase-4.4.2.jar:4.4.2]
	at org.springframework.data.couchbase.core.CouchbaseTemplateSupport.decodeEntity(CouchbaseTemplateSupport.java:141) ~[spring-data-couchbase-4.4.2.jar:4.4.2]
	at org.springframework.data.couchbase.core.NonReactiveSupportWrapper.lambda$decodeEntity$1(NonReactiveSupportWrapper.java:45) ~[spring-data-couchbase-4.4.2.jar:4.4.2]

I’ve done some research and would like to confirm whether this is a known limitation in Spring Boot 2.7.2 when used with Spring Data Couchbase 4.x.

The issue seems to occur when a model contains a field typed as Object, and the stored data is a structured JSON object (e.g., a Map). In this setup, Spring Data Couchbase fails to deserialize the object correctly.

Instead of mapping the JSON to a generic structure like Map<String, Object>, it attempts to directly instantiate Object.class, which results in either:

An empty object ({}), or

A deserialization failure like:
Cannot construct instance of java.lang.Object (no Creators, like default constructor, exist)

Interestingly, when testing the same structure with a newer Spring version (e.g., Spring Boot 3.5.0), it works as expected.

Notes

Actually, there is a fallback approach using findRaw(), and the end user is aware of it. However, they’re looking for a better solution, preferably by using a custom converter since findRaw() would only be used if there’s no other viable option.

Thank you.

[mikereiche]

If a entity or property is an Object, then spring data can only create a java.lang.Object and since a java.lang.Object has no fields, nothing will be populated. If the entity or property is a JsonObject, then spring data couchbase will create a JsonObject and populate it based on it's converter for JsonObject - provided you are using a version that has such a converter. If I'm not mistaken, that converter was added sometime in 5.x. #1650 Please the latest version of spring-data-couchbase.

[jessaardithya]

Hi @mikereiche ,
Thank you for the clarification.

I'd like to confirm: based on your explanation, the recommended approach would be to change the entity or property type to JsonObject and upgrade to a newer version of spring-data-couchbase, which would also require upgrading the Spring Boot version, is that correct?

However, in our case, the application maintains a shared model that is used by both MongoDB and Couchbase, so changing the field type to JsonObject may not be feasible due to incompatibility with MongoDB.

Do you have any recommendations or best practices for handling this scenario where the model must remain database-agnostic (e.g., keeping the field as Object), but still support proper deserialization in Couchbase?

Appreciate your insights.

[mikereiche]

The recommended approach is to upgrade to the latest version.

Interestingly, when testing the same structure with a newer Spring version (e.g., Spring Boot 3.5.0), it works as expected.

If you want to take whatever changes allow this to work in the later version and back-port them to the older version - you are free to do that.