Deserialize JSON with irregular content - json

I have the example below, where 2 "unusual" things can happen:
if the status is NOK, the data element will not be included at all
some attributes of the elements inside list might be missing (in the example below, key2 is missing in the second element of list.
Is there a way to de-serialise an irregular JSON string using automatic deriving?
Which is the easiest/better way to handle this kind of irregular JSON in Rust? Can I avoid writing a very complex match based code to check for every possible combination?
extern crate serialize;
static JSON: &'static str = r#"
{
"status": {
"status": "OK"
},
"data": {
"container": {
"key": "value",
"list": [
{
"key1": "value1",
"key2": "value2"
},
{
"key1": "value1"
}
]
}
}
}"#;
#[deriving(Decodable, Show)]
struct Top {
status: Status,
data: Data,
}
#[deriving(Decodable, Show)]
struct Data {
container: Container,
}
#[deriving(Decodable, Show)]
struct Status {
status: String,
}
#[deriving(Decodable, Show)]
struct Container {
key: String,
list: Vec<KeyVals>,
}
#[deriving(Decodable, Show)]
struct KeyVals {
key1: String,
key2: String,
}
fn main() {
let result: Top = match serialize::json::decode(JSON) {
Ok(x) => x,
Err(why) => fail!("Failed decoding the JSON! Reason: {}", why),
};
println!("{}", result);
}
When running the code, it fails because the second element of the list is missing the key2 attribute.
task '<main>' failed at 'Failed decoding the JSON! Reason: MissingFieldError(key2)', hello.rs:56
Thank you

Potentially existing data can be represented via an enum. In the simplest case, with an Option.
I believe using an enum will also solve your problem.
#[deriving(Encodable, Decodable)]
enum Status {
Good(Container),
Bad,
VeryBad
}
If the Container also contains potentially existing data, then you can again use an enum to represent that.

Related

Parsing json partially with serde_json

I am using serde_json in rust, and I am calling an api and get a very large json in return.
My question is this, is this possible to de-serialize this JSON partially. By partially, I mean to some, but not all properties of the JSON response.
for Example, I have this JSON:
Object {
"age_group": String(""),
"amazon_product_url": String("https://www.amazon.com/dp/0063221489?tag=NYTBSREV-20"),
"article_chapter_link": String(""),
"asterisk": Number(0),
"author": String("Jared Kushner"),
"book_image": String("https://storage.googleapis.com/du-prd/books/images/9780063221482.jpg"),
"book_image_height": Number(500),
"book_image_width": Number(331),
"book_review_link": String(""),
"book_uri": String("nyt://book/e5ec4777-5f2f-5622-9288-9b1d96e8fe1d"),
"buy_links": Array [
Object {
"name": String("Amazon"),
"url": String("https://www.amazon.com/dp/0063221489?tag=NYTBSREV-20"),
},
Object {
"name": String("Apple Books"),
"url": String("https://goto.applebooks.apple/9780063221482?at=10lIEQ"),
},
Object {
"name": String("Barnes and Noble"),
"url": String("https://www.anrdoezrs.net/click-7990613-11819508?url=https%3A%2F%2Fwww.barnesandnoble.com%2Fw%2F%3Fean%3D9780063221482"),
}
}
Then in this case, is it possible to just catch buy_links and amazon_product_url properties and never mind others?
If you only declare the fields you need, yes it is possible to only deserialize a subset of the data:
use serde::{Serialize, Deserialize};
#[derive(Serialize, Deserialize, Debug)]
struct Point {
x: i32,
y: i32,
}
fn test(s: &str)
{
let p: Point = serde_json::from_str(s).unwrap();
println!("{:?}", p);
}
fn main()
{
test("{\"x\":0,\"y\":3}");
test("{\"x\":0,\"y\":2, \"z\": 4}");
}
Output:
Point { x: 0, y: 3 }
Point { x: 0, y: 2 }
As you can see, the "z" field that is present in the second test is ignored.
See play.rust-lang.org example.

Deserialize json array of arrays and strings

I have json from a vendor api that looks like this:
[
"Crappy vendor unwanted string",
[
{
"key": "val",
"...": "lots more",
},
{
"it": "goes on",
}
]
]
I'm at a loss as to what to do here. Simply passing this as a vec to serde ala How to Deserialize a top level array doesn't work, as I don't know how to reference/pass the inner string and array to other structs.
I can simply do this:
let json_string: String = res.text().await?;
however, that json string is unwieldy and unformatted. Not ideal.
Taking it a step further, I've created the following, which gets me close, but doesn't seem to work:
#[derive(Deserialize)]
pub struct Outer {
pub outer_tite: String,
pub inner_vec: Vec<Inner>,
}
#[derive(Deserialize)]
pub struct Inner {
pub inner_obj: InnerObj,
}
#[derive(Deserialize)]
pub struct InnerObj {
pub key1: String,
pub key2: String,
...
}
impl Outer {
pub async fn get(...) -> Result<Vec<Outer>, ExitFailure> {
let out: Vec<Outer> = rec.json::<Vec<Outer>>().await?;
ok(out)
}
}
The result here is that the first string encountered throws an error:
value: error decoding response body: invalid type: string "Bad vendor string", expected struct Outer at line 2...
Changing the input json is not an option, as it's a vendor endpoint.
I've tried the suggestions from [How can I deserialize JSON with a top-level array using Serde?](How can I deserialize JSON with a top-level array using Serde?) - this does not work for my use case.
You can directly deserialize a struct from a list of elements. The struct fields are deserialized in order. Your example code does not work since you only have one Outer element, containing multiple InnerObj. The Inner struct you added does not match the JSON structure you have shown.
The snippet shows how you can directly deserialize into the Outer struct, which contains the String and the Vec.
use serde::Deserialize;
#[derive(Deserialize)]
pub struct Outer {
pub outer_titel: String,
pub inner_vec: Vec<Inner>,
}
#[derive(Deserialize)]
pub struct Inner {
pub key1: String,
pub key2: String,
}
#[test]
fn deser() {
let j = r#"[
"Crappy vendor unwanted string",
[
{
"key1": "val1",
"key2": "val2"
},
{
"key1": "val1",
"key2": "val2"
}
]
]"#;
let outer: Outer = serde_json::from_str(j).unwrap();
assert_eq!("Crappy vendor unwanted string", outer.outer_titel);
assert_eq!(2, outer.inner_vec.len());
}
Playground

How can I parse a JSON array of either strings or objects?

An API I hit has poorly structured JSON. Someone decided that it was a great idea to send back a list that looks like this
features: [
"First one",
"second one",
{
"feature": "third one",
"hasAdditionalImpact": true
},
"forth one"
]
I've figured out a way to get this data into a struct but that was effectively:
struct MyStruct {
SensibleData: String,
SensibleTruthy: bool,
features: serde_json::Value,
}
This doesn't help me normalize and verify the data.
Is there a good way to turn that first object into something like
features: [
{
"feature": "First one",
"hasAdditionalImpact": false
},
{
"feature": "second one",
"hasAdditonalImpact": false
},
{
"feature": "third one",
"hasAdditionalImpact": true
},
{
"feature": "forth one",
"hasAdditionalImpact": false
}
]
I saw type_name might be usable for checking the type and doing post-processing after it's be parsed by serde_json, but I also saw that type_name is for diagnostic purposes so I'd rather not use that for this purpose.
It looks like the features in your JSON have two forms; an explicit object and a simplified form where some fields are defaulted or unnamed. You can model that with an eum like this:
#[derive(Deserialize, Debug)]
#[serde(untagged)]
enum Feature {
Simple(String),
Explicit {
feature: String,
#[serde(rename = "hasAdditionalImpact")]
has_additional_impact: bool,
}
}
(playground)
The #[serde(untagged)] attribute means it will attempt to deserialize into each variant in order until one succeeds.
If the enum is going to be annoying, you can convert them all into the same struct, with default values, using #[serde(from)] and providing a From conversion:
#[derive(Deserialize, Debug)]
#[serde(untagged)]
enum FeatureSource {
Simple(String),
Explicit {
feature: String,
#[serde(rename = "hasAdditionalImpact")]
has_additional_impact: bool,
},
}
#[derive(Deserialize, Debug)]
#[serde(from = "FeatureSource")]
struct Feature {
feature: String,
has_additional_impact: bool,
}
impl From<FeatureSource> for Feature {
fn from(other: FeatureSource) -> Feature {
match other {
FeatureSource::Simple(feature) => Feature {
feature,
has_additional_impact: false,
},
FeatureSource::Explicit {
feature,
has_additional_impact,
} => Feature {
feature,
has_additional_impact,
},
}
}
}
(playground)
FeatureSource is only used as an intermediate representation and converted to Feature before the rest of your code ever sees it.

SWIFT 4 nested JSON Struct - Codable

I'm having issues creating a struct to parse JSON in Swift 4. I'm able to parse small JSONs and JSONDecoder seems to work fine. Just need help to create a struct to parse JSON like that:
{
"main": {
"solutions": [
{
"exersises": [
{
"book_title": "test",
"release_date": "2015-01-12T11:00",
"price": 100,
"additional": [
{
"item1": "test",
"item2": "test",
"number": 1
},
{
"item1": "test2",
"item2": "test2",
"number": 2
}
],
"availability": "Yes",
"item_id": 43534
}
]
}
]
}
}
What kind of struct do I need to get to value of book_title for example?
Its really easy. Your main probem is most likely root element. Let me get first layer or two for you.
let decoded = try JSONDecoder().decode(MainJSON.self, from: data)
class MainJSON: Codable {
var main:SolutionJSON?
}
class SolutionJSON: Codable {
var exercises:[ExercisesJSON]?
}
class ExercisesJSON: Codable {
var bookTitle: String?
var releaseDate: String?
var price: Double?
... etc
enum CodingKeys: String, CodingKey {
case bookTitle = "book_title"
case releaseDate = "release_date"
case price = "price"
}
}
ExerciseJSON also uses Codable interface which lets remap json properties into swift properties if they don't match. Hope this helps.
i prefer to give a general solution not only for this condition
it is very simple just download and run this MACOS APP from GITHUB
run it in your mac by XCODE and but your JSON in it,
it will make Models for any complex JSON
notes
1 if JSON keys have a capital character in the first it will be small
, so after copying model you need to change it like the JSON
2 if two JSON objects have the same structure and the same key names it will be only one model

How can I deserialize JSON with a top-level array using Serde?

I have a some JSON data that is returned from a web service. The JSON is a top-level array:
[
{
"data": "value1"
},
{
"data": "value2"
},
{
"data": "value3"
}
]
Using serde_derive to make structs I can can deserialize the data contained within the array, however, I am unable to get Serde to deserialize the top-level array.
Am I missing something, or can Serde not deserialize top level-arrays?
You can simply use a Vec:
use serde::{Serialize, Deserialize};
#[derive(Serialize, Deserialize, Debug)]
struct Foo {
data: String,
}
fn main() -> Result<(), serde_json::Error> {
let data = r#"[
{
"data": "value1"
},
{
"data": "value2"
},
{
"data": "value3"
}
]"#;
let datas: Vec<Foo> = serde_json::from_str(data)?;
for data in datas.iter() {
println!("{:#?}", data);
}
Ok(())
}
If you wish, you could also use transparent:
#[derive(Serialize, Deserialize, Debug)]
#[serde(transparent)]
struct Foos {
foos: Vec<Foo>,
}
let foos: Foos = serde_json::from_str(data)?;
This allows to encapsulate your data with your type.