Skip to content

Spark is unable to write to a table due to malformed serde struct #1297

@rampage644

Description

@rampage644

When trying to write data to the table with Apache Spark 3.5 (iceberg 1.8) it fails with

java.io.UncheckedIOException: org.apache.iceberg.shaded.com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'Failed': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false')
 at [Source: (String)"Failed to deserialize the JSON body into the target type: updates[0].action: unknown variant `assign-uuid`, expected one of `assign-u-u-i-d`, `upgrade-format-version`, `add-schema`, `set-current-schema`, `add-partition-spec`, `set-default-spec`, `add-sort-order`, `set-default-sort-order`, `add-snapshot`, `set-snapshot-ref`, `remove-snapshots`, `remove-snapshot-ref`, `set-location`, `set-properties`, `remove-properties` at line 1 column 77"; line: 1, column: 7]

This is potentially because https://github.com/JanKaul/iceberg-rust/blob/main/iceberg-rust/src/catalog/commit.rs#L96 becomes assign-u-u-i-d in the json serializiation and it also expects this fields instead of assign-uuid.

Metadata

Metadata

Assignees

Labels

icebergstorage subsystem related, apache iceberg related

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions