I have json from a vendor api that looks like this:
[
"Crappy vendor unwanted string",
[
{
"key": "val",
"...": "lots more",
},
{
"it": "goes on",
}
]
]
I'm at a loss as to what to do here. Simply passing this as a vec to serde ala How to Deserialize a top level array doesn't work, as I don't know how to reference/pass the inner string and array to other structs.
I can simply do this:
let json_string: String = res.text().await?;
however, that json string is unwieldy and unformatted. Not ideal.
Taking it a step further, I've created the following, which gets me close, but doesn't seem to work:
#[derive(Deserialize)]
pub struct Outer {
pub outer_tite: String,
pub inner_vec: Vec<Inner>,
}
#[derive(Deserialize)]
pub struct Inner {
pub inner_obj: InnerObj,
}
#[derive(Deserialize)]
pub struct InnerObj {
pub key1: String,
pub key2: String,
...
}
impl Outer {
pub async fn get(...) -> Result<Vec<Outer>, ExitFailure> {
let out: Vec<Outer> = rec.json::<Vec<Outer>>().await?;
ok(out)
}
}
The result here is that the first string encountered throws an error:
value: error decoding response body: invalid type: string "Bad vendor string", expected struct Outer at line 2...
Changing the input json is not an option, as it's a vendor endpoint.
I've tried the suggestions from [How can I deserialize JSON with a top-level array using Serde?](How can I deserialize JSON with a top-level array using Serde?) - this does not work for my use case.
You can directly deserialize a struct from a list of elements. The struct fields are deserialized in order. Your example code does not work since you only have one Outer element, containing multiple InnerObj. The Inner struct you added does not match the JSON structure you have shown.
The snippet shows how you can directly deserialize into the Outer struct, which contains the String and the Vec.
use serde::Deserialize;
#[derive(Deserialize)]
pub struct Outer {
pub outer_titel: String,
pub inner_vec: Vec<Inner>,
}
#[derive(Deserialize)]
pub struct Inner {
pub key1: String,
pub key2: String,
}
#[test]
fn deser() {
let j = r#"[
"Crappy vendor unwanted string",
[
{
"key1": "val1",
"key2": "val2"
},
{
"key1": "val1",
"key2": "val2"
}
]
]"#;
let outer: Outer = serde_json::from_str(j).unwrap();
assert_eq!("Crappy vendor unwanted string", outer.outer_titel);
assert_eq!(2, outer.inner_vec.len());
}
Playground
Related
I'm writing for a program that hooks into a web service which sends back JSON.
When a certain property isn't there it provides a empty object, with all its fields as empty strings instead of excluding the value. When the property exists, some of the properties are u64. How can I have it so Serde handles this case?
Rust Structs
#[derive(Clone, Debug, Deserialize)]
struct WebResponse {
foo: Vec<Foo>,
}
#[derive(Clone, Debug, Deserialize)]
struct Foo {
points: Points,
}
#[derive(Clone, Debug, Deserialize)]
struct Points {
x: u64,
y: u64,
name: String,
}
Example JSON
{
"foo":[
{
"points":{
"x":"",
"y":"",
"name":""
}
},
{
"points":{
"x":78,
"y":92,
"name":"bar"
}
}
]
}
Serde supports an interesting selection of attributes that can be used to customize the serialization or deserialization for a type while still using the derived implementation for the most part.
In your case, you need to be able to decode a field that can be specified as one of multiple types, and you don't need information from other fields to decide how to decode the problematic fields. The #[serde(deserialize_with="$path")] annotation is well suited to solve your problem.
We need to define a function that will decode either an empty string or an integer value into an u64. We can use the same function for both fields, since we need the same behavior. This function will use a custom Visitor to be able to handle both strings and integers. It's a bit long, but it makes you appreciate all the work that Serde is doing for you!
extern crate serde;
#[macro_use]
extern crate serde_derive;
extern crate serde_json;
use serde::Deserializer;
use serde::de::{self, Unexpected};
use std::fmt;
#[derive(Clone, Debug, Deserialize)]
struct WebResponse {
foo: Vec<Foo>,
}
#[derive(Clone, Debug, Deserialize)]
struct Foo {
points: Points,
}
#[derive(Clone, Debug, Deserialize)]
struct Points {
#[serde(deserialize_with = "deserialize_u64_or_empty_string")]
x: u64,
#[serde(deserialize_with = "deserialize_u64_or_empty_string")]
y: u64,
name: String,
}
struct DeserializeU64OrEmptyStringVisitor;
impl<'de> de::Visitor<'de> for DeserializeU64OrEmptyStringVisitor {
type Value = u64;
fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
formatter.write_str("an integer or a string")
}
fn visit_u64<E>(self, v: u64) -> Result<Self::Value, E>
where
E: de::Error,
{
Ok(v)
}
fn visit_str<E>(self, v: &str) -> Result<Self::Value, E>
where
E: de::Error,
{
if v == "" {
Ok(0)
} else {
Err(E::invalid_value(Unexpected::Str(v), &self))
}
}
}
fn deserialize_u64_or_empty_string<'de, D>(deserializer: D) -> Result<u64, D::Error>
where
D: Deserializer<'de>,
{
deserializer.deserialize_any(DeserializeU64OrEmptyStringVisitor)
}
fn main() {
let value = serde_json::from_str::<WebResponse>(
r#"{
"foo": [
{
"points": {
"x": "",
"y": "",
"name": ""
}
},
{
"points": {
"x": 78,
"y": 92,
"name": "bar"
}
}
]
}"#,
);
println!("{:?}", value);
}
Cargo.toml:
[dependencies]
serde = "1.0.15"
serde_json = "1.0.4"
serde_derive = "1.0.15"
In str_or_u64, we use an untagged enum to represent either a string or a number. We can then deserialize the field into that enum and convert it to a number.
We annotate the two fields in Points using deserialize_with to tell it to use our special conversion:
use serde::{Deserialize, Deserializer}; // 1.0.124
use serde_json; // 1.0.64
#[derive(Debug, Deserialize)]
struct WebResponse {
foo: Vec<Foo>,
}
#[derive(Debug, Deserialize)]
struct Foo {
points: Points,
}
#[derive(Debug, Deserialize)]
struct Points {
#[serde(deserialize_with = "str_or_u64")]
x: u64,
#[serde(deserialize_with = "str_or_u64")]
y: u64,
name: String,
}
fn str_or_u64<'de, D>(deserializer: D) -> Result<u64, D::Error>
where
D: Deserializer<'de>,
{
#[derive(Deserialize)]
#[serde(untagged)]
enum StrOrU64<'a> {
Str(&'a str),
U64(u64),
}
Ok(match StrOrU64::deserialize(deserializer)? {
StrOrU64::Str(v) => v.parse().unwrap_or(0), // Ignoring parsing errors
StrOrU64::U64(v) => v,
})
}
fn main() {
let input = r#"{
"foo":[
{
"points":{
"x":"",
"y":"",
"name":""
}
},
{
"points":{
"x":78,
"y":92,
"name":"bar"
}
}
]
}"#;
dbg!(serde_json::from_str::<WebResponse>(input));
}
See also:
How to transform fields during deserialization using Serde?
Just adding a note for future viewers: in case it is helpful, I have implemented the solution from the accepted answer and published it as a crate serde-this-or-that.
I've added a section on Performance to explain that an approach with a custom Visitor as suggested, should perform overall much better than a version with an untagged enum, which does also work.
Here is a shortened implementation of the accepted solution above (should have the same result):
use serde::Deserialize;
use serde_json::from_str;
use serde_this_or_that::as_u64;
#[derive(Clone, Debug, Deserialize)]
struct WebResponse {
foo: Vec<Foo>,
}
#[derive(Clone, Debug, Deserialize)]
struct Foo {
points: Points,
}
#[derive(Clone, Debug, Deserialize)]
struct Points {
#[serde(deserialize_with = "as_u64")]
x: u64,
#[serde(deserialize_with = "as_u64")]
y: u64,
name: String,
}
fn main() {
let value = from_str::<WebResponse>(
r#"{
"foo": [
{
"points": {
"x": "",
"y": "",
"name": ""
}
},
{
"points": {
"x": 78,
"y": 92,
"name": "bar"
}
}
]
}"#,
);
println!("{:?}", value);
}
The Cargo.toml would look like:
[dependencies]
serde = { version = "1.0.136", features = ["derive"] }
serde_json = "1.0.79"
serde-this-or-that = "0.4"
I have a JSON object that contains a string that itself is a JSON object. How can I deserialize it?
I'd like to be able to do something like:
#[derive(Deserialize)]
struct B {
c: String,
d: u32,
}
#[derive(Deserialize)]
struct Test {
a: String,
b: B,
}
let object: Test = serde_json::from_str(
r#"
{
"a": "string",
"b": "{\"c\":\"c_string\",\"d\":1234}"
}
"#,
)
.unwrap();
But this panics with invalid type: string "{\"c\":\"c_string\",\"d\":1234}", expected struct B
You can use the serde_with crate with json::nested:
#[derive(Deserialize)]
struct Test {
a: String,
#[serde(with = "serde_with::json::nested")]
b: B,
}
Hello I have a &str type variable. How can I convert it to json? for example I have &str
{"index":0,"name":"AB/CDE/FG/402/test_int4","sts":"on","time":"2021-06-05 03:28:24.044284300 UTC","value":8}
How can I convert this to json
{
"index": 0,
"name": "AB/CDE/FG/402/test_int4",
"sts": "on",
"time": "2021-06-05 03:28:24.044284300 UTC",
"value": 8
}
is there anything like python json.loads() method available or something equivalent can be done in rust?
There's a misconception here. JSON is a serialization format, its always a string. Your two blocks are both JSON and functionally identical. What json.loads() is deserialize it into a value that can be accessed and manipulated natively.
You probably want to use a serde_json::Value if you want to this JSON to be used generically.
use serde_json::Value;
fn main() {
let input = r#"{"index":0,"name":"AB/CDE/FG/402/test_int4","sts":"on","time":"2021-06-05 03:28:24.044284300 UTC","value":8}"#;
let mut object: Value = serde_json::from_str(input).unwrap();
if let Some(name) = object.get_mut("name") {
*name = "new name".into();
}
println!("{}", object);
}
{"index":0,"name":"new name","sts":"on","time":"2021-06-05 03:28:24.044284300 UTC","value":8}
Playground
Or, its typical to create a structure that reflects what the JSON is expected to be and deserialize into that type via serde:
use serde::{Deserialize, Serialize};
#[derive(Serialize, Deserialize, Debug)]
struct Data {
index: i32,
name: String,
sts: String,
time: String,
value: i32
}
fn main() {
let input = r#"{"index":0,"name":"AB/CDE/FG/402/test_int4","sts":"on","time":"2021-06-05 03:28:24.044284300 UTC","value":8}"#;
let mut object: Data = serde_json::from_str(input).unwrap();
object.name = "new name".to_string();
println!("{:#?}", object);
}
Data {
index: 0,
name: "new name",
sts: "on",
time: "2021-06-05 03:28:24.044284300 UTC",
value: 8,
}
Playground
I don't know anything built-in, but this package can do it:
use tinyjson::{JsonParseError, JsonValue};
fn main() -> Result<(), JsonParseError> {
let s = r#"
{
"index":0,
"name":"AB/CDE/FG/402/test_int4",
"sts":"on",
"time":"2021-06-05 03:28:24.044284300 UTC",
"value":8
}
"#;
let m: JsonValue = s.parse()?;
println!("{:?}", m);
Ok(())
}
Result:
Object({
"value": Number(8.0),
"index": Number(0.0),
"name": String("AB/CDE/FG/402/test_int4"),
"time": String("2021-06-05 03:28:24.044284300 UTC"),
"sts": String("on")
})
https://docs.rs/tinyjson
I have a some JSON data that is returned from a web service. The JSON is a top-level array:
[
{
"data": "value1"
},
{
"data": "value2"
},
{
"data": "value3"
}
]
Using serde_derive to make structs I can can deserialize the data contained within the array, however, I am unable to get Serde to deserialize the top-level array.
Am I missing something, or can Serde not deserialize top level-arrays?
You can simply use a Vec:
use serde::{Serialize, Deserialize};
#[derive(Serialize, Deserialize, Debug)]
struct Foo {
data: String,
}
fn main() -> Result<(), serde_json::Error> {
let data = r#"[
{
"data": "value1"
},
{
"data": "value2"
},
{
"data": "value3"
}
]"#;
let datas: Vec<Foo> = serde_json::from_str(data)?;
for data in datas.iter() {
println!("{:#?}", data);
}
Ok(())
}
If you wish, you could also use transparent:
#[derive(Serialize, Deserialize, Debug)]
#[serde(transparent)]
struct Foos {
foos: Vec<Foo>,
}
let foos: Foos = serde_json::from_str(data)?;
This allows to encapsulate your data with your type.
I have the example below, where 2 "unusual" things can happen:
if the status is NOK, the data element will not be included at all
some attributes of the elements inside list might be missing (in the example below, key2 is missing in the second element of list.
Is there a way to de-serialise an irregular JSON string using automatic deriving?
Which is the easiest/better way to handle this kind of irregular JSON in Rust? Can I avoid writing a very complex match based code to check for every possible combination?
extern crate serialize;
static JSON: &'static str = r#"
{
"status": {
"status": "OK"
},
"data": {
"container": {
"key": "value",
"list": [
{
"key1": "value1",
"key2": "value2"
},
{
"key1": "value1"
}
]
}
}
}"#;
#[deriving(Decodable, Show)]
struct Top {
status: Status,
data: Data,
}
#[deriving(Decodable, Show)]
struct Data {
container: Container,
}
#[deriving(Decodable, Show)]
struct Status {
status: String,
}
#[deriving(Decodable, Show)]
struct Container {
key: String,
list: Vec<KeyVals>,
}
#[deriving(Decodable, Show)]
struct KeyVals {
key1: String,
key2: String,
}
fn main() {
let result: Top = match serialize::json::decode(JSON) {
Ok(x) => x,
Err(why) => fail!("Failed decoding the JSON! Reason: {}", why),
};
println!("{}", result);
}
When running the code, it fails because the second element of the list is missing the key2 attribute.
task '<main>' failed at 'Failed decoding the JSON! Reason: MissingFieldError(key2)', hello.rs:56
Thank you
Potentially existing data can be represented via an enum. In the simplest case, with an Option.
I believe using an enum will also solve your problem.
#[deriving(Encodable, Decodable)]
enum Status {
Good(Container),
Bad,
VeryBad
}
If the Container also contains potentially existing data, then you can again use an enum to represent that.