How to deserialize a JSON array into a struct using serde?

Question

I'm trying to deserialize following JSON snippets into a Vec of struct Shape:

use serde::{Deserialize, Serialize};
use serde_json::{Result, Value};

#[derive(Debug, Serialize, Deserialize)]
struct Shape {  // this struct is not working, for display purpose only
    shape_type: String,
    d0: f64,
    d1: f64,
    d2: f64, //optional, like the case of "dot"
    d3: f64, //optional, like the case of "circle"
}

let json = r#"
  {[
    ["line", 1.0, 1.0, 2.0, 2.0],
    ["circle", 3.0, 3.0, 1.0],
    ["dot", 4.0, 4.0]
  ]}"#;

let data: Vec<Shape> = match serde_json::from_str(json)?;

Obviously, each type of Shape needs a String and different number of f64 to describe it. How should I define the struct of Shape to deserialize the JSON data as above?

Your JSON data isn't valid JSON. You can't have an anonymous array inside of an object. Also, is it possible to modify the JSON format or must it be deserialized in its current form? — pretzelhammer
– pretzelhammer, Commented Dec 25, 2020 at 16:45
My typo. It should be let json = r#"{"shapes": [["line", 1.0, 1.0, 2.0, 2.0], ..."#; Unfortunately I cannot change the JSON format. I tried to change the type of d2 & d3 to Option<f64>, but no luck: "invalid length 4, expected struct Shape with 5 elements" at line of "circle". — Cuteufo
– Cuteufo, Commented Dec 25, 2020 at 17:04

pretzelhammer · Accepted Answer · 2020-12-25 17:03:47Z

7

Assuming you have control over the JSON format I strongly recommend making the Shape type into an enum that can represent multiple shapes and using serde's derive macros to automatically implement Serialize and Deserialize for Shape. Example:

use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize)]
struct Point {
    x: f64,
    y: f64,
}

#[derive(Debug, Serialize, Deserialize)]
#[serde(tag = "type")]
enum Shape {
    Dot { position: Point },
    Line { start: Point, end: Point },
    Circle { center: Point, radius: f64 },
}

fn main() {
    let shapes = vec![
        Shape::Dot {
            position: Point { x: 3.0, y: 4.0 },
        },
        Shape::Line {
            start: Point { x: -2.0, y: 1.0 },
            end: Point { x: 5.0, y: -3.0 },
        },
        Shape::Circle {
            center: Point { x: 0.0, y: 0.0 },
            radius: 7.0,
        },
    ];

    let serialized = serde_json::to_string(&shapes).unwrap();
    println!("serialized = {}", serialized);

    let deserialized: Vec<Shape> = serde_json::from_str(&serialized).unwrap();
    println!("deserialized = {:?}", deserialized);
}

playground

If you absolutely cannot change the JSON format then serde cannot help you. Serializing a shape as a heterogeneous array of strings and floats is a very bizarre choice. You have to manually parse it yourself (or at least use some parser crate to help you) and then manually implement the Deserializer trait for it to turn it into a Shape.

answered Dec 25, 2020 at 17:03

pretzelhammer

15.4k16 gold badges56 silver badges105 bronze badges

Sign up to request clarification or add additional context in comments.

3 Comments

Cuteufo Over a year ago

Good post. I will figure out if it is possible to change the format of JSON. Just curious, this piece of JSON could be very easily parsed with Python json.loads(text). Python's list can contain different type of data, and variable number of items. Is there a data type in rust similar to Python's list?

ecoe Over a year ago

Consider object store query engines, such as Presto or Amazon Athena. REST API for the latter will actually serialize returned rows from a query as a list of lists rather than list of objects. See @Red15 solution - using enum representations. I agree it's not ideal, but still important to showcase how serde still has a solution for this...

Dreamer Over a year ago

@Cuteufo the Rust equivalent to Python's json.loads() is json::parse() from the json crate ( docs.rs/json/latest/json ). For a JSON array, this will return a Vec<> of an enum type that can contain - among other things - a string or a number similar to Python's list.

Red15 · Accepted Answer · 2023-05-02 12:41:31Z

4

Sounds perfectly doable,

See https://serde.rs/enum-representations.html

Your code would use untagged enum representations and would look like:

use serde::{Deserialize, Serialize};

#[derive(Debug, Serialize, Deserialize)]
#[serde(untagged)]
#[allow(non_camel_case_types)]
enum Shape {
    line(String, f64, f64, f64, f64),
    circle(String, f64, f64, f64),
    dot(String, f64, f64),
}

#[derive(Debug, Serialize, Deserialize)]
struct ShapeList {
    shapes: Vec<Shape>
}

fn main() {
    let json = r#"{"shapes": [
        ["line", 1.0, 1.0, 2.0, 2.0],
        ["circle", 3.0, 3.0, 1.0],
        ["dot", 4.0, 4.0],
        ["circle2", 8.0, 3.0, 16.0]
      ]}"#;
    
    let data: ShapeList = serde_json::from_str(json).unwrap();
    println!("{data:#?}");
}

Outputs:

ShapeList {
    shapes: [
        line(
            "line",
            1.0,
            1.0,
            2.0,
            2.0,
        ),
        circle(
            "circle",
            3.0,
            3.0,
            1.0,
        ),
        dot(
            "dot",
            4.0,
            4.0,
        ),
        circle(
            "circle2",
            8.0,
            3.0,
            16.0,
        ),
    ],
}

I slightly modified your data to highlight the recognition of the type is not based on the actual value of that first column but on the "signature" of the array.

A better way would eventually be to write your own Serialize/Deserialize implementation which should be documented quite well on the serde website.

edited May 2, 2023 at 12:41

answered May 2, 2023 at 12:36

Red15

8451 gold badge6 silver badges17 bronze badges

2 Comments

ecoe Over a year ago

Why is allow(non_camel_case_types) necessary? Thanks!

Dreamer Over a year ago

@ecoe it's not necessary, but disables a coding style warning: Rust expects enum variant names to start with an upper-case letter, but the OP has written line, circle and dot with lower-case characters.

Masklinn · Accepted Answer · 2020-12-25 17:10:42Z

2

How should I define the struct of Shape to deserialize the JSON data as above?

You wouldn't, because the serialisation scheme you want doesn't really make sense to rust, and AFAIK serde doesn't support it (not even if you use an enum of tuple variant, tag="type" is not supported for them).

If you really can't or don't want to use a simpler structure & serialisation scheme as described in the other answer, the only option I can see is to implement a custom (de)serialisation scheme.

Especially since the arity changes for each "type", otherwise https://crates.io/crates/serde_tuple would work (although you could always see if skip_serializing_if works with serde_tuple, that would let you suppress the "extra" fields).

answered Dec 25, 2020 at 17:10

Masklinn

43.7k4 gold badges58 silver badges78 bronze badges

1 Comment

ecoe Over a year ago

What about using untagged enum representations? I suppose one danger is it relies on coincidental ordering of the JSON list to match expected types?

Collectives™ on Stack Overflow

How to deserialize a JSON array into a struct using serde?

3 Answers 3

3 Comments

2 Comments

1 Comment

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

3 Answers 3

3 Comments

2 Comments

1 Comment

Your Answer

Sign up or log in

Post as a guest

Related