I am trying to serialise a nested json to a BTree<string,string>.
I will be using specific elements of this collection to bind to different structs as required.
JSON
{
"data": "some_data",
"key": "some_key",
"nestedValue": {
"timestamp": "0",
"d1": "d1",
"d2": "d2",
"time": 0,
"secindaryNestedValue": [{
"d3": "test1",
"d4": "test2"
},
{
"d3": "test3",
"d4": "test4"
}
]
},
"timestamp": 0
}
I am attempting to serialise this as follows:
let input: BTreeMap<String, String> = serde_json::from_str(INPUT).unwrap();
println!("input -> {:?}",input);
I want to get an output as following:
BTree items
Key Value
data some_data
key some_key
nested_value "{\"d1\":\"d1\",\"d2\":\"d2\",\"time\":0,\"secindaryNestedValue\":[{\"d3\":\"test1\",\"d4\":\"test2\"},{\"d3\":\"test3\",\"d4\":\"test4\"}]}"
timestamp 0
I am doing this so that my nested jsons can be as generic as possible.
In subsequent operations I will be binding the nested json to a struct using serde as follows using the struct :
use serde_derive::Deserialize;
use serde_derive::Serialize;
#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct Root {
pub data: String,
pub key: String,
pub nested_value: NestedValue,
pub timestamp: i64,
}
#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct NestedValue {
pub timestamp: String,
pub d1: String,
pub d2: String,
pub time: i64,
pub secindary_nested_value: Vec<SecindaryNestedValue>,
}
#[derive(Default, Debug, Clone, PartialEq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct SecindaryNestedValue {
pub d3: String,
pub d4: String,
}
I would want to use the nested value later,
Convert the json string to json and bind it to a similar struct.
Open to suggestions on not using a BTree and something better, but my usecase requires me to have the inner nested jsons as a string which I can bind later.
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=4afbad3da5584bc67c9e70ae08f41cee
(Probably not useful to OP after 2 months but giving some input for other readers.)
I strongly suspect you don't want a BTreeMap. These are primarily useful for being sorted, but these JSON key-value pairs are not something which you want to be sorted by the key. You could try a HashMap<String, String>, but again I suspect this isn't really what you want. The closest reasonable answer to your question is probably to use serde_json::Value, which is a generic type that can match any valid JSON. It is implemented as
pub enum Value {
Null,
Bool(bool),
Number(Number),
String(String),
Array(Vec<Value>),
Object(Map<String, Value>),
}
So this JSON would deserialise to an Object, and you can work with the contained map to do whatever you need. If you really want to put the nested value back to a string, use serde_json::to_string to convert it back at whatever point that is necessary. You can subsequently convert a Value to a more specific type like Root using serde_json::from_value.
However I will note that, in my experience, using Value is rarely the best approach. I have only used it when an API I was integrating with sent some arbitrary JSON as a part of some data, and subsequently required that same JSON to be sent back as part of another request. Your exact requirements aren't entirely clear but I will do my best to guess at what you really want:
To serialise (Rust to JSON, which was specified in the question but maybe not really what you meant), your structs shown should convert nicely to the JSON format specified with serde_json::to_string. Based on the rest of the question, it seems like you have some more complex requirements which may mean those need to be changed, but the principle is correct.
To deserialise (JSON to Rust), this depends on the data that you are getting. If you know that the data comes in the format given, you can deserialise it directly to the Root type you have defined, and the nested values will be build correctly as the appropriate types. If you need to modify the data in some way, I would strongly encourage you to do so on the deserialised types rather than on strings. Even if you want to replace some parts of it with other data which you get as a JSON string, it's better to deserialise that data (perhaps to NestedValue, or some other type) and work with those Rust types. This is far more reliable than trying to edit JSON strings. If you need to present some part of the data to an external user or service as JSON, you can serialise it again.
If you aren't sure that the data is going to come in the format you have written (part of your question suggests you need more generic types, perhaps because of this?), you have a couple of options based on what it may be. If some values may be missing, you can use an Option for the deserialisation type: eg. if nestedValue may be missing, use pub nested_value: Option<NestedValue> in the Root struct. If there are some other known patterns it could follow, you can use an enum to match them: eg.
#[derive(Debug, Deserialize)]
#[serde(untagged)] // This tells serde to figure out from the type structure rather than the name.
pub enum RootEnum {
Root {
data: String,
key: String,
nested_value: NestedValue, // This can be used as it was.
timestamp: i64,
}
SomeOtherType {
foo: Bar
}
}
If you do this, you may need a match block to determine how to use the data. If you really don't know what this data may look like, you would need to use Value to deserialise it. However at that point I suspect that you wouldn't be able to use the data anyway, so you should probably attempt to deserialise to a type you know, and do something appropriate with the error you get if it doesn't match. You will need some error handling anyway since the JSON may be invalid.
In any case, this has become a long answer, hopefully it will be useful to someone.
Related
I'm trying to use serde json deserialization to support "choices" using enum, but it doesn't seep to be working (I have python enum background)
let's say I have this json :
{"name": "content", "state": "open"}
and state can be open or closed
in python I would just create an enum and the state type would be that enum eg:
#[derive(Deserialize)]
enum State {
Open(String),
Closed(String),
}
#[derive(Deserialize)]
struct MyStruct {
name: String,
state: State,
}
and the problem is that I don't know how to derserialize open to State::Open and closed to State::Closed
I have looked into implementing my own deserializer, but it seems very complicated and very advanced for me.
is there any straightforward way ?
You should remove the String. Then you'll get another error:
unknown variant `open`, expected `Open` or `Closed`
Because your enum variants are in PascalCase while your JSON is in camelCase (or snake_case, I don't know). To fix that, add #[serde(rename_all = "camelCase")]:
#[derive(Deserialize)]
#[serde(rename_all = "camelCase")]
enum State {
Open,
Closed,
}
I'm writing a client library around this REST server. Octoprint (A server to manage a 3d printer), to be precise.
Here's one of the types i'm working with:
#[derive(Serialize, Deserialize, Debug)]
pub struct JobInfo {
/// The file that is the target of the current print job
pub file: FileInfo,
/// The estimated print time for the file, in seconds
#[serde(rename = "estimatedPrintTime")]
pub estimated_print_time: Option<f64>,
/// The print time of the last print of the file, in seconds
#[serde(rename = "lastPrintTime")]
pub last_print_time: Option<f64>,
/// Information regarding the estimated filament usage of the print job
pub filament: Option<Filament>,
}
Pretty straightforward, Using the multiplicity property defined in the specification of the API, I determined which properties should be considered optional, hence why some of these props are wrapped in options.
Unfortunately the documentation lies a little bit in the way multiplicity works here; here's an example on what a response looks like when the printer is in an offline state. For the sake of brevity, I will omit most of the body of this JSON message and keep just enough to get the point across
{
"job": {
"file": null,
"filepos": null,
"printTime": null,
... etc
},
...
"state": "Offline"
}
Here's the type that I'm expecting for this response:
#[derive(Serialize, Deserialize, Debug)]
pub struct JobInformationResponse {
/// Information regarding the target of the current print job
job: JobInfo,
/// Information regarding the progress of the current print job
progress: ProgressInfo,
/// A textual representation of the current state of the job
/// or connection. e.g. "Operational", "Printing", "Pausing",
/// "Paused", "Cancelling", "Error", "Offline", "Offline after error",
/// "Opening serial connection" ... - please note that this list is not exhaustive!
state: String,
/// Any error message for the job or connection. Only set if there has been an error
error: Option<String>,
}
Now I could just wrap all of these types in Options, but the previous example json wouldn't parse, since technically since job is an object, it's not going to deserialize as None despite the fact that each of it's keys are null. I was wondering if there were some sort of attribute in serde that would be able to handle this weird kind of serialization issue. I'd like to avoid just wrapping every single property in Options just to handle the edge case where the printer is offline
Edit: I guess what I'm trying to say is that I would expect that if all props on a struct in the json representation were null, that the object itself would serialize as None
If you're willing to redesign a little bit, you might be able to do something like this:
#[serde(tag = "state")]
enum JobInformationResponse {
Offline {}
// If a field only appears on one type of response, use a struct variant
Error { error: String },
// If multiple response types share fields, use a newtype variant and a substruct
Printing(JobInformationResponseOnline),
Paused(JobInformationResponseOnline),
// ...
}
struct JobInformationResponseOnline {
job: JobInfo,
progress: ProgressInfo,
}
This works in the Offline case because by default, serde ignores properties that don't fit into any field of the struct/enum variant. So it won't check whether all entries of job are null.
If you have fields that appear in every message, you can further wrap JobInformationResponse (you should probably rename it):
struct JobInformationResponseAll {
field_appears_in_all_responses: FooBar,
#[serde(flatten)]
state: JobInformationResponse // Field name doesn't matter to serde
}
But I'm not sure whether that works for you, since I certainly haven't seen enough of the spec or any real example messages.
To answer your question directly: No, there is no attribute in serde which would allow an all-null map to be de/serialized as None. You'd need two versions of the struct, one without options (to be used in your rust code) and one with (to be used in a custom deserialization function where you first deserialize to the with-options struct and then convert). Might not be worth the trouble.
And a side note: You might be happy to find #[serde(rename_all = "camelCase")] exists.
I'm new to Rust.
I try to write a websocket client.
This is my format message:
"[integer, "string", object]"
Is there away to store all value in Vector something like:
let msg: Vec<interface> = vec![123, "event", AnyObject{}];
How to convert it into string and vice versa.
Thanks in advance
Conceptually speaking you want to use an enum. The type of enums used in rust are called tagged unions. Essentially you can think of them as enums which can hold data.
enum Interface {
Int(i32),
String(&'static str),
Object(AnyObject),
// etc
}
// You can then create a vec of different enum variants
let msg: Vec<Interface> = vec![Interface::Int(123), Interface::String("event"), Interface::Object(AnyObject{})];
Assuming you are referring to JSON, then the recommended solution is to use serde with serde_json. serde_json provides a Value enum you can use to represent JSON data in an unknown layout.
// Copied from serde_json docs
enum Value {
Null,
Bool(bool),
Number(Number),
String(String),
Array(Vec<Value>),
Object(Map<String, Value>),
}
However, you don't need to use Value directly (unless you want to) since serde_json provides a macro to make this easier by letting you format your code like JSON.
use serde_json::{json, Value};
let msg: Value = json!([123, "event", {}]);
// You can then serialize a Value into JSON format
println!("{:?}", msg.to_string());
// Output: "[123,\"event\",{}]"
I recommend reading through the overview since they have a bunch of convenient ways for working with JSON.
Is it possible to use a value in JSON to determine how to deserialize the rest of the JSON using serde? For example, consider the following code:
use serde::{Serialize, Deserialize};
use serde_repr::*;
#[derive(Serialize_repr, Deserialize_repr, Debug)]
#[repr(u8)]
enum StructType {
Foo = 1,
Bar = 2
}
#[derive(Serialize, Deserialize, Debug)]
struct Foo {
a: String,
b: u8
}
#[derive(Serialize, Deserialize, Debug)]
struct Bar {
x: String,
y: u32,
z: u16
}
#[derive(Serialize, Deserialize, Debug)]
struct AllMyStuff {
type: StructType,
data: //HELP: Not sure what to put here
}
What I'm trying to achieve is deserialization of the data, even if in multiple steps, where the type field in the AllMyStuff determines which type of struct data is present in data. For example, given the following pseudocode, I'd like to ultimately have a Bar struct with the proper data in it:
data = {"type": "2", "data": { "x": "Hello world", "y": "18", "z": "5" } }
// 1) use serde_json to deserialize a AllMyStuff struct, not erroring on the "data" blob
// 2) Now that we know data is of type "2" (or Bar), parse the remaining "data" into a AllMyStuff struct
If steps (1) and (2) are somehow able to be done in a single step, that would be awesome but not needed. I'm not sure what type of type to declare data in the AllMyStuff struct to enable this as well.
You can use serde_json::Value as the type for AllMyStuff::data. It will deserialize any valid json object and also implements Deserialize itself, so it can be further deserialized once the type to deserialize to is known (via AllMyStuff::type). While this requires more intermittent steps and (mostly temporary) types, it saves you from manually implementing Deserialize on an enum AllMyStuff { Foo(Foo), Bar(Bar) }.
I may be missing something, but AllMyStuff looks as if you are trying to manually distinguish between Foo and Bar.
However, Rust, has a built-in way of doing this:
#[derive(Serialize, Deserialize, Debug)]
enum AllMyStuff {
Foo(Foo),
Bar(Bar),
}
Click here to see it in action.
I am trying to deserialize some crypto exchange JSON. The structure of the JSON is predictable but the keys are not. Each time the server adds a new currency it becomes a new key. For example, it looks something like this:
{
"timestamp":"1562405",
"username":"1234",
"BTC":
{"available":"0.00","orders":"0.00000000"},
"BCH":
{"available":"0.0000000","orders":"0.00000000"},
..
..
"GUSD":
{"available":"0.00","orders":"0.00"}
}
I tried the usual approach of defining a struct to deserialize into but every time there is a new currency my program will have an error.
I thought I would just read it all into a Value and then just iterate over it manually to put it all into a struct.
let balance_data: Value = serde_json::from_str(&String::from_utf8_lossy(&body)).unwrap();
println!("balance_data: {:?}", balance__data);
for element in balance_data.iter() {
//push into a proper map...
}
But I can't:
no method named `iter` found for type `serde_json::Value`.
I thought I would put it into a Vec<Value> but this causes a panic:
thread 'main' panicked at 'called ``Result::unwrap()` on an `Err` value: Error("invalid type: map, expected a sequence", line: 1, column: 0)'`
I also experimented with reading it into a HashMap but couldn't quite seem to crack it.
It would be amazing if we could do something like:
let balance_data: RecvBalance = serde_json::from_str(&String::from_utf8_lossy(&body)).unwrap();
#[derive(Debug, Deserialize)]
struct RecvBalance {
timestamp: String,
username: String,
HashMap<String, RecvBalanceData>,
}
#[derive(Debug, Deserialize)]
struct RecvBalanceData {
available: String,
orders: String,
}
Has anyone dealt with this situation? I need a struct with the balance data in it that I can lookup later in my program.
The doc has a related example. You need to put the HashMap and use flatten attribute:
#[derive(Debug, Deserialize)]
struct RecvBalance {
timestamp: String,
username: String,
#[serde(flatten)]
moneys: HashMap<String, RecvBalanceData>,
}