Deserialize json based on an enum in the json - json

Is it possible to use a value in JSON to determine how to deserialize the rest of the JSON using serde? For example, consider the following code:
use serde::{Serialize, Deserialize};
use serde_repr::*;
#[derive(Serialize_repr, Deserialize_repr, Debug)]
#[repr(u8)]
enum StructType {
Foo = 1,
Bar = 2
}
#[derive(Serialize, Deserialize, Debug)]
struct Foo {
a: String,
b: u8
}
#[derive(Serialize, Deserialize, Debug)]
struct Bar {
x: String,
y: u32,
z: u16
}
#[derive(Serialize, Deserialize, Debug)]
struct AllMyStuff {
type: StructType,
data: //HELP: Not sure what to put here
}
What I'm trying to achieve is deserialization of the data, even if in multiple steps, where the type field in the AllMyStuff determines which type of struct data is present in data. For example, given the following pseudocode, I'd like to ultimately have a Bar struct with the proper data in it:
data = {"type": "2", "data": { "x": "Hello world", "y": "18", "z": "5" } }
// 1) use serde_json to deserialize a AllMyStuff struct, not erroring on the "data" blob
// 2) Now that we know data is of type "2" (or Bar), parse the remaining "data" into a AllMyStuff struct
If steps (1) and (2) are somehow able to be done in a single step, that would be awesome but not needed. I'm not sure what type of type to declare data in the AllMyStuff struct to enable this as well.

You can use serde_json::Value as the type for AllMyStuff::data. It will deserialize any valid json object and also implements Deserialize itself, so it can be further deserialized once the type to deserialize to is known (via AllMyStuff::type). While this requires more intermittent steps and (mostly temporary) types, it saves you from manually implementing Deserialize on an enum AllMyStuff { Foo(Foo), Bar(Bar) }.

I may be missing something, but AllMyStuff looks as if you are trying to manually distinguish between Foo and Bar.
However, Rust, has a built-in way of doing this:
#[derive(Serialize, Deserialize, Debug)]
enum AllMyStuff {
Foo(Foo),
Bar(Bar),
}
Click here to see it in action.

Related

Rust: Serializing a nested json stirng to a BTree<string,string>

I am trying to serialise a nested json to a BTree<string,string>.
I will be using specific elements of this collection to bind to different structs as required.
JSON
{
"data": "some_data",
"key": "some_key",
"nestedValue": {
"timestamp": "0",
"d1": "d1",
"d2": "d2",
"time": 0,
"secindaryNestedValue": [{
"d3": "test1",
"d4": "test2"
},
{
"d3": "test3",
"d4": "test4"
}
]
},
"timestamp": 0
}
I am attempting to serialise this as follows:
let input: BTreeMap<String, String> = serde_json::from_str(INPUT).unwrap();
println!("input -> {:?}",input);
I want to get an output as following:
BTree items
Key Value
data some_data
key some_key
nested_value "{\"d1\":\"d1\",\"d2\":\"d2\",\"time\":0,\"secindaryNestedValue\":[{\"d3\":\"test1\",\"d4\":\"test2\"},{\"d3\":\"test3\",\"d4\":\"test4\"}]}"
timestamp 0
I am doing this so that my nested jsons can be as generic as possible.
In subsequent operations I will be binding the nested json to a struct using serde as follows using the struct :
use serde_derive::Deserialize;
use serde_derive::Serialize;
#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Root {
pub data: String,
pub key: String,
pub nested_value: NestedValue,
pub timestamp: i64,
}
#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct NestedValue {
pub timestamp: String,
pub d1: String,
pub d2: String,
pub time: i64,
pub secindary_nested_value: Vec<SecindaryNestedValue>,
}
#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct SecindaryNestedValue {
pub d3: String,
pub d4: String,
}
I would want to use the nested value later,
Convert the json string to json and bind it to a similar struct.
Open to suggestions on not using a BTree and something better, but my usecase requires me to have the inner nested jsons as a string which I can bind later.
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=4afbad3da5584bc67c9e70ae08f41cee
(Probably not useful to OP after 2 months but giving some input for other readers.)
I strongly suspect you don't want a BTreeMap. These are primarily useful for being sorted, but these JSON key-value pairs are not something which you want to be sorted by the key. You could try a HashMap<String, String>, but again I suspect this isn't really what you want. The closest reasonable answer to your question is probably to use serde_json::Value, which is a generic type that can match any valid JSON. It is implemented as
pub enum Value {
Null,
Bool(bool),
Number(Number),
String(String),
Array(Vec<Value>),
Object(Map<String, Value>),
}
So this JSON would deserialise to an Object, and you can work with the contained map to do whatever you need. If you really want to put the nested value back to a string, use serde_json::to_string to convert it back at whatever point that is necessary. You can subsequently convert a Value to a more specific type like Root using serde_json::from_value.
However I will note that, in my experience, using Value is rarely the best approach. I have only used it when an API I was integrating with sent some arbitrary JSON as a part of some data, and subsequently required that same JSON to be sent back as part of another request. Your exact requirements aren't entirely clear but I will do my best to guess at what you really want:
To serialise (Rust to JSON, which was specified in the question but maybe not really what you meant), your structs shown should convert nicely to the JSON format specified with serde_json::to_string. Based on the rest of the question, it seems like you have some more complex requirements which may mean those need to be changed, but the principle is correct.
To deserialise (JSON to Rust), this depends on the data that you are getting. If you know that the data comes in the format given, you can deserialise it directly to the Root type you have defined, and the nested values will be build correctly as the appropriate types. If you need to modify the data in some way, I would strongly encourage you to do so on the deserialised types rather than on strings. Even if you want to replace some parts of it with other data which you get as a JSON string, it's better to deserialise that data (perhaps to NestedValue, or some other type) and work with those Rust types. This is far more reliable than trying to edit JSON strings. If you need to present some part of the data to an external user or service as JSON, you can serialise it again.
If you aren't sure that the data is going to come in the format you have written (part of your question suggests you need more generic types, perhaps because of this?), you have a couple of options based on what it may be. If some values may be missing, you can use an Option for the deserialisation type: eg. if nestedValue may be missing, use pub nested_value: Option<NestedValue> in the Root struct. If there are some other known patterns it could follow, you can use an enum to match them: eg.
#[derive(Debug, Deserialize)]
#[serde(untagged)] // This tells serde to figure out from the type structure rather than the name.
pub enum RootEnum {
Root {
data: String,
key: String,
nested_value: NestedValue, // This can be used as it was.
timestamp: i64,
}
SomeOtherType {
foo: Bar
}
}
If you do this, you may need a match block to determine how to use the data. If you really don't know what this data may look like, you would need to use Value to deserialise it. However at that point I suspect that you wouldn't be able to use the data anyway, so you should probably attempt to deserialise to a type you know, and do something appropriate with the error you get if it doesn't match. You will need some error handling anyway since the JSON may be invalid.
In any case, this has become a long answer, hopefully it will be useful to someone.

How to (de)serialize a strongly typed JSON dictionary in Serde?

I am writing a Rust application that handles JSON messages from a TypeScript client with a public interface. I have written some code using serde_derive and it works well, but I can't figure out how to implement dictionaries; e.g.:
{
"foo" : { "data" : 42 },
"bar" : { "data" : 1337 }
}
Here the keys are the strings "foo" and "bar" and the dictionary's values follow this schema:
use serde_derive;
use serde_json::Number;
#[derive(Serialize, Deserialize)]
struct DictionaryValue {
data: Number,
}
I am looking to access the JSON data in this manner:
#[derive(Serialize, Deserialize)]
struct Dictionary {
key: String,
value: DictionaryValue,
}
How can I (de)serialize my JSON data into/from Dictionary using Serde?
You have a logic error in your code. The structure in your JSON file describes an associative array but your Dictionary does not support multiple key-value-pairs. As Stargateur stated in the comments, you may use HashMap as Serde has Serialize and Deserialize implementations for HashMap.
Instead of using a single key-value-pair, you can rewrite your Dictionary as
type Dictionary = HashMap<String, DictionaryValue>;
and you can retrieve the data for example by
let dict: Dictionary = serde_json::from_str(json_string).unwrap();
If you now want to wrap everything in a Dictionary-struct it will look like this:
#[derive(Serialize, Deserialize)]
struct Dictionary {
inner: HashMap<String, DictionaryValue>,
}
The problem is, that serde_json now expects
{
"inner": {
"foo" : { "data" : 42 },
"bar" : { "data" : 1337 }
}
}
To get rid of this, you can add the serde(flatten) attribute to Dictionary:
#[derive(Serialize, Deserialize, Debug)]
struct Dictionary {
#[serde(flatten)]
inner: HashMap<String, DictionaryValue>,
}
If HashMap or any BTreeMap from std does not fit your needs, you can also implement your Dictionary on your own. See the docs here and here for more details.

How to use Serde to (de)serialize a tuple struct from JSON named values instead of an array?

I have a JSON object:
{ "min": 2, "max": 15 }
I'd like to parse it to this tuple struct:
#[derive(Serialize, Deserialize, Debug)]
struct TeamSize(pub i64, pub i64);
#[derive(Serialize, Deserialize, Debug)]
struct Match {
size: TeamSize,
}
The current Serde serialization mechanism does not seem to provide the functionality of (de)serializing a tuple structure from named values instead of an array.
The generated (de)serialization mechanism expects the following:
{"size": [2, 15]}
I've tried to use Serde attributes, but I can't find one that does what I want:
#[derive(Serialize, Deserialize, Debug)]
pub struct TeamSize(
#[serde(rename = "min")]
pub i64,
#[serde(rename = "max")]
pub i64
);
How to parse it? Should I implement everything by myself?
I've opened an issue on the Serde repository.
#[derive(Serialize, Deserialize, Debug)]
pub struct TeamSize(
#[serde(rename = "min")]
pub i64,
#[serde(rename = "max")]
pub i64
);
is not valid code, the Serde rename attribute only renames what is being serialized and deserialized, it does not change your code. In a tuple struct (your first one), you can (and must) omit names because you simply access them via self.0 and self.1, but a struct doesn't have a first or a second field, so you must add a name to them.
Like so:
#[derive(Serialize, Deserialize, Debug)]
pub struct TeamSize {
pub max: i64,
pub min: i64,
};
Since the name of your attribute is the name of the JSON property (both min and max), you do not need to use serde(rename). You would have needed it if your Rust struct used the fields value_max and value_min but your JSON still used max and min.
If you absolutely want to parse it to a tuple struct, you must implement Serialize and Deserialize for your custom struct yourself. I don't think it's worth the hassle though, just switch to a struct instead of a tuple struct.

Rust and JSON serialization

If the JSON object is missing some fields, the decode function throws an exception. For example:
extern crate rustc_serialize;
use rustc_serialize::json;
use rustc_serialize::json::Json;
#[derive(RustcDecodable, RustcEncodable, Debug)]
enum MessageType {
PING,
PONG,
OPT,
}
#[derive(RustcDecodable, RustcEncodable, Debug)]
pub struct JMessage {
msg_type: String,
mtype: MessageType,
}
fn main() {
let result3 = json::decode::<JMessage>(r#"{"msg_type":"TEST"}"#);
println!("{:?}", result3);
// this will print `Err(MissingFieldError("mtype"))`
let result = json::decode::<JMessage>(r#"{"msg_type":"TEST", "mtype":"PING"}"#);
println!("{:?}", &result);
// This will print Ok(JMessage { msg_type: "TEST", mtype: PING })
let result2 = Json::from_str(r#"{"msg_type":"TEST", "mtype":"PING"}"#).unwrap();
println!("{:?}", &result2);
// this will print Object({"msg_type": String("TEST"), "mtype": String("PING")})
}
Is there a way to specify that some fields in a struct are optional?
Why does the function from_str not serialize mtype as an enum?
No, there is no such way. For that, you need to use serde. Serde also has lots of other features, but unfortunately it is not as easy to use as rustc_serialize on stable Rust.
Well, how should it? Json::from_str returns a JSON AST, which consists of maps, arrays, strings, numbers and other JSON types. It simply cannot contain values of your enum. And also there is no way to indicate that you want some other type instead of string, naturally.
Regarding the first question, you can use Option. For example:
pub struct JMessage {
msg_type: Option<String>,
mtype: MessageType,
}
Which defaults to None if the field does not exist.

Deserialising JSON object with rustc_serialize: why do I need to implement PartialEq?

I have a file containing a JSON object with unknown keys. I would like to decode this object into a structure but don't understand how to declare this structure.
extern crate rustc_serialize;
use rustc_serialize::json;
use std::collections::BTreeMap;
#[derive(RustcDecodable, Debug)]
struct MyStruct {
foo: u8,
bar: Vec<String>,
}
let raw_json = r#"{
"A": {
"foo": 2,
"bar": ["a", "b"],
},
"C": {
"foo": 1,
"bar": ["c", "d"],
},
:
}"#;
let j: BTreeMap<String, MyStruct> = json::decode(&raw_json).unwrap();
println!("{:?}", j.get("A").unwrap());
The following error occurs:
error: the trait `core::cmp::PartialEq` is not implemented for the type `MyStruct` [E0277]
let j: BTreeMap<String, MyStruct> = json::decode(&raw_json).unwrap();
^~~~~~~~~~~~
Would I have to implement Decodable for MyStruct myself then?
json::decode is defined as:
pub fn decode<T: Decodable>(s: &str) -> DecodeResult<T>
This means that given a string slice, it will attempt to be decoded into a type specified by the user, so long as that type implements Decodable. On the page for Decodable, you can see all the implementations of it, including the one for BTreeMap:
impl<K: Decodable + PartialEq + Ord,
V: Decodable + PartialEq>
Decodable for BTreeMap<K, V>
This shows that in order to decode to a BTreeMap, both the key and value in the map need to be PartialEq. However, I wasn't clear on why that is actually needed. BTreeMap should only require that the key is Ord and not care about the value at all. To that end, I've opened a pull request to remove those bounds, and it was accepted! ^_^ I guess that means that the bounds might have just been a typo originally.
The simple answer as hinted by #Shepmaster is to either implement the PartialEq trait or let it be derived, as the error message indicates:
#[derive(RustcDecodable, PartialEq, Debug)]