How to serialise structs into io::Write in a loop - json

I need to do a simple read/process/write in Rust like this:
#[derive(serde::Deserialize)]
struct Incoming {
first: String,
last: String,
}
#[derive(serde::Serialize)]
struct Outgoing {
name: String,
}
// Keep the read/write traits as generic as possible
fn stream_things<R: std::io::Read, W: std::io::Write>(reader: R, writer: W) {
let incoming: Vec<Incoming> = serde_json::from_reader(reader).unwrap();
for a in incoming {
let b = Outgoing {
name: format!("{} {}", a.first, a.last),
};
serde_json::to_writer(writer, &b).unwrap();
}
}
fn main() {
stream_things(std::io::stdin(), std::io::stdout());
}
This does not compile because:
error[E0382]: use of moved value: `writer`
--> src/main.rs:20:31
|
13 | fn stream_things<R: std::io::Read, W: std::io::Write>(reader: R, writer: W) {
| -- ------ move occurs because `writer` has type `W`, which does not implement the `Copy` trait
| |
| help: consider further restricting this bound: `W: Copy +`
...
20 | serde_json::to_writer(writer, &b).unwrap();
| ^^^^^^ value moved here, in previous iteration of loop
What is the correct way to write to std::io::Write in a loop?
Also how to correctly do it with serde's to_writer?
See the plaground.

Given W is io::Write, then &mut W is also io::Write:
impl<'_, W: Write + ?Sized> Write for &'_ mut W
so the following compiles:
fn stream_things<R: std::io::Read, W: std::io::Write>(reader: R, mut writer: W) {
let incoming: Vec<Incoming> = serde_json::from_reader(reader).unwrap();
for a in incoming {
let b = Outgoing {
name: format!("{} {}", a.first, a.last),
};
serde_json::to_writer(&mut writer, &b).unwrap();
}
}

Related

How do I fix "the method `clone` exists but the following trait bounds were not satisfied" when cloning a struct with generics and a function pointer? [duplicate]

This question already has answers here:
Deriving a trait results in unexpected compiler error, but the manual implementation works
(2 answers)
Closed 2 years ago.
I'm writing a function that will create a menu interface in the terminal. To use the function, it needs to be passed a Vec<menu_option>
#[derive(Clone)]
pub struct menu_option<'a, T> {
pub command: &'a str,
pub description: &'a str,
pub action: menu_option_action<'a, T>,
}
#[derive(Clone)]
pub enum menu_option_action<'a, T> {
sub_menu(Vec<menu_option<'a, T>>),
callback(fn(state: T) -> T),
leave,
}
When I design the menu, it can have multiple levels and there can be repeats of a menu_action. Namely, each layer has a "leave"/"return":
// In my actual code I construct a vector of `menu_option`s
fn example() {
type StateType = i32;
let mut state: StateType = 88;
let leave_option: menu_option<&mut StateType> = menu_option::<&mut StateType> {
command: "l",
description: "leave",
action: menu_option_action::leave,
};
let a = leave_option.clone();
}
error[E0599]: no method named `clone` found for struct `menu_option<'_, &mut i32>` in the current scope
--> src/lib.rs:24:26
|
2 | pub struct menu_option<'a, T> {
| -----------------------------
| |
| method `clone` not found for this
| doesn't satisfy `menu_option<'_, &mut i32>: std::clone::Clone`
...
24 | let a = leave_option.clone();
| ^^^^^ method not found in `menu_option<'_, &mut i32>`
|
= note: the method `clone` exists but the following trait bounds were not satisfied:
`&mut i32: std::clone::Clone`
which is required by `menu_option<'_, &mut i32>: std::clone::Clone`
= help: items from traits can only be used if the trait is implemented and in scope
= note: the following trait defines an item `clone`, perhaps you need to implement it:
candidate #1: `std::clone::Clone`
How do I fix this?
I tried searching online and couldn't find a solution, at least, not one that worked for me (what I found was old Rust, I think)
I don't know what stops the clone because the only use of the T type is a function pointer which I thought was cloneable.
It works when you implement Clone yourself on menu_option & menu_option_action. By default, since your struct/enum has a type parameter, the macro expansion of the #[derive(Clone)] attribute will restrain the clone implementation for your type to T: Clone.
In your case, not only is this requirement not needed, but it is also very unlikely to be ever respected (e.g. &mut T does not implement Clone). By implementing Clone by hand, you can get rid of the T: Clone requirement and it works!
Note that function pointers implement Copy which is why the menu_option_action::callback(*f) works:
type StateType = i32;
pub struct menu_option<'a, T> {
pub command: &'a str,
pub description: &'a str,
pub action: menu_option_action<'a, T>,
}
impl<'a, T> Clone for menu_option<'a, T> {
fn clone(&self) -> Self {
menu_option {
command: self.command.clone(),
description: self.description.clone(),
action: self.action.clone(),
}
}
}
pub enum menu_option_action<'a, T> {
sub_menu(Vec<menu_option<'a, T>>),
callback(fn(T) -> T),
leave,
}
impl<'a, T> Clone for menu_option_action<'a, T> {
fn clone(&self) -> Self {
match self {
menu_option_action::sub_menu(sub) => menu_option_action::sub_menu(sub.to_vec()),
menu_option_action::callback(f) => menu_option_action::callback(*f),
menu_option_action::leave => menu_option_action::leave,
}
}
}
fn main() {
let mut state: StateType = 88;
let leave_option: menu_option<&mut StateType> = menu_option {
command: "l",
description: "leave",
action: menu_option_action::leave,
};
let a = leave_option.clone();
}

Rust: enable Serde to distinguish between undefined & null [duplicate]

I'd like to use Serde to parse some JSON as part of a HTTP PATCH request. Since PATCH requests don't pass the entire object, only the relevant data to update, I need the ability to tell between a value that was not passed, a value that was explicitly set to null, and a value that is present.
I have a value object with multiple nullable fields:
struct Resource {
a: Option<i32>,
b: Option<i32>,
c: Option<i32>,
}
If the client submits JSON like this:
{"a": 42, "b": null}
I'd like to change a to Some(42), b to None, and leave c unchanged.
I tried wrapping each field in one more level of Option:
#[derive(Debug, Deserialize)]
struct ResourcePatch {
a: Option<Option<i32>>,
b: Option<Option<i32>>,
c: Option<Option<i32>>,
}
playground
This does not make a distinction between b and c; both are None but I'd have wanted b to be Some(None).
I'm not tied to this representation of nested Options; any solution that can distinguish the 3 cases would be fine, such as one using a custom enum.
Building off of E_net4's answer, you can also create an enum for the three possibilities:
#[derive(Debug)]
enum Patch<T> {
Missing,
Null,
Value(T),
}
impl<T> Default for Patch<T> {
fn default() -> Self {
Patch::Missing
}
}
impl<T> From<Option<T>> for Patch<T> {
fn from(opt: Option<T>) -> Patch<T> {
match opt {
Some(v) => Patch::Value(v),
None => Patch::Null,
}
}
}
impl<'de, T> Deserialize<'de> for Patch<T>
where
T: Deserialize<'de>,
{
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
Option::deserialize(deserializer).map(Into::into)
}
}
This can then be used as:
#[derive(Debug, Deserialize)]
struct ResourcePatch {
#[serde(default)]
a: Patch<i32>,
}
Unfortunately, you still have to annotate each field with #[serde(default)] (or apply it to the entire struct). Ideally, the implementation of Deserialize for Patch would handle that completely, but I haven't figured out how to do that yet.
Quite likely, the only way to achieve that right now is with a custom deserialization function. Fortunately, it is not hard to implement, even to make it work for any kind of field:
fn deserialize_optional_field<'de, T, D>(deserializer: D) -> Result<Option<Option<T>>, D::Error>
where
D: Deserializer<'de>,
T: Deserialize<'de>,
{
Ok(Some(Option::deserialize(deserializer)?))
}
Then each field would be annotated as thus:
#[serde(deserialize_with = "deserialize_optional_field")]
a: Option<Option<i32>>,
You also need to annotate the struct with #[serde(default)], so that empty fields are deserialized to an "unwrapped" None. The trick is to wrap present values around Some.
Serialization relies on another trick: skipping serialization when the field is None:
#[serde(deserialize_with = "deserialize_optional_field")]
#[serde(skip_serializing_if = "Option::is_none")]
a: Option<Option<i32>>,
Playground with the full example. The output:
Original JSON: {"a": 42, "b": null}
> Resource { a: Some(Some(42)), b: Some(None), c: None }
< {"a":42,"b":null}
Building up on Shepmaster's answer and adding serialization.
use serde::ser::Error;
use serde::{Deserialize, Deserializer};
use serde::{Serialize, Serializer};
// #region ------ JSON Absent support
// build up on top of https://stackoverflow.com/a/44332837
/// serde Valueue that can be Absent, Null, or Valueue(T)
#[derive(Debug)]
pub enum Maybe<T> {
Absent,
Null,
Value(T),
}
#[allow(dead_code)]
impl<T> Maybe<T> {
pub fn is_absent(&self) -> bool {
match &self {
Maybe::Absent => true,
_ => false,
}
}
}
impl<T> Default for Maybe<T> {
fn default() -> Self {
Maybe::Absent
}
}
impl<T> From<Option<T>> for Maybe<T> {
fn from(opt: Option<T>) -> Maybe<T> {
match opt {
Some(v) => Maybe::Value(v),
None => Maybe::Null,
}
}
}
impl<'de, T> Deserialize<'de> for Maybe<T>
where
T: Deserialize<'de>,
{
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
let d = Option::deserialize(deserializer).map(Into::into);
d
}
}
impl<T: Serialize> Serialize for Maybe<T> {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
match self {
// this will be serialized as null
Maybe::Null => serializer.serialize_none(),
Maybe::Value(v) => v.serialize(serializer),
// should have been skipped
Maybe::Absent => Err(Error::custom(
r#"Maybe fields need to be annotated with:
#[serde(default, skip_serializing_if = "Maybe::is_Absent")]"#,
)),
}
}
}
// #endregion --- JSON Absent support
And then you can use it this way:
#[derive(Serialize, Deserialize, Debug)]
struct Rect {
#[serde(default, skip_serializing_if = "Maybe::is_absent")]
stroke: Maybe<i32>,
w: i32,
#[serde(default, skip_serializing_if = "Maybe::is_absent")]
h: Maybe<i32>,
}
// ....
let json = r#"
{
"stroke": null,
"w": 1
}"#;
let deserialized: Rect = serde_json::from_str(json).unwrap();
println!("deserialized = {:?}", deserialized);
// will output: Rect { stroke: Null, w: 1, h: Absent }
let serialized = serde_json::to_string(&deserialized).unwrap();
println!("serialized back = {}", serialized);
// will output: {"stroke":null,"w":1}
I wish Serde had a built-in way to handle JSON's null and absent states.
Update 2021-03-12 - Updated to Maybe::Absent as it is more JSON and SQL DSL idiomatic.
The catch with this approach is that we can express:
type | null with the default Option<type>
type | null | absent with Maybe<type>
But we cannot express
type | absent
The solution would be to refactor Maybe to just have ::Present(value) and ::Absent and support Maybe<Option<type>> for the type | null | absent. So this will give us full coverage.
type | null with the default Option<type>
type | absent with Maybe<type>
type | absent | null with Maybe<Option<type>>
I am trying to implement this without adding a #[serde(deserialize_with = "deserialize_maybe_field")] but not sure it is possible. I might be missing something obvious.

How do I write a Serde Visitor to convert an array of arrays of strings to a Vec<Vec<f64>>?

I need to deserialize a JSON into a struct that has a Vec<Vec<f64>> field. The JSON has strings for numbers so I need a custom deserializer to convert the strings to f64 during the deserialization.
A sample JSON that I'd like to deserialize:
{
"values": [["2", "1.4"], ["8.32", "1.5"]]
}
My struct is this:
#[derive(Deserialize)]
struct Payload {
#[serde(default, deserialize_with = "from_array_of_arrays_of_strs")]
values: Vec<Vec<f64>>,
}
I saw you could probably do this with visitors in the examples of Serde, so I've implemented this visitor:
fn from_array_of_arrays_of_strs<'de, T, D>(deserializer: D) -> Result<Vec<Vec<f64>>, D::Error>
where
T: Deserialize<'de>,
D: Deserializer<'de>,
{
struct F64Visitor(PhantomData<fn() -> Vec<Vec<f64>>>);
impl<'de> Visitor<'de> for F64Visitor {
type Value = Vec<Vec<f64>>;
fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
formatter.write_str("a nonempty sequence of numbers")
}
#[inline]
fn visit_str<E>(self, value: &str) -> Result<f64, E>
where
E: serde::de::Error,
{
self.visit_string(String::from(value))
}
#[inline]
fn visit_string<E>(self, value: String) -> Result<f64, E> {
Ok(value.parse::<f64>().unwrap())
}
#[inline]
fn visit_seq<V, T>(self, mut visitor: V) -> Result<Vec<T>, V::Error>
where
V: SeqAccess<'de>,
{
let mut vec = Vec::new();
while let Some(elem) = try!(visitor.next_element()) {
vec.push(elem);
}
Ok(vec)
}
}
let visitor = F64Visitor(PhantomData);
deserializer.deserialize_seq(visitor)
}
playground
The compiler complains that visit_str and visit_string have an incompatible type for the trait:
error[E0053]: method `visit_str` has an incompatible type for trait
--> src/main.rs:32:9
|
32 | / fn visit_str<E>(self, value: &str) -> Result<f64, E>
33 | | where
34 | | E: serde::de::Error,
35 | | {
36 | | self.visit_string(String::from(value))
37 | | }
| |_________^ expected struct `std::vec::Vec`, found f64
|
= note: expected type `fn(from_array_of_arrays_of_strs::F64Visitor, &str) -> std::result::Result<std::vec::Vec<std::vec::Vec<f64>>, E>`
found type `fn(from_array_of_arrays_of_strs::F64Visitor, &str) -> std::result::Result<f64, E>`
error[E0053]: method `visit_string` has an incompatible type for trait
--> src/main.rs:40:9
|
40 | / fn visit_string<E>(self, value: String) -> Result<f64, E> {
41 | | Ok(value.parse::<f64>().unwrap())
42 | | }
| |_________^ expected struct `std::vec::Vec`, found f64
|
= note: expected type `fn(from_array_of_arrays_of_strs::F64Visitor, std::string::String) -> std::result::Result<std::vec::Vec<std::vec::Vec<f64>>, E>`
found type `fn(from_array_of_arrays_of_strs::F64Visitor, std::string::String) -> std::result::Result<f64, E>`
error[E0049]: method `visit_seq` has 2 type parameters but its trait declaration has 1 type parameter
--> src/main.rs:45:21
|
45 | fn visit_seq<V, T>(self, mut visitor: V) -> Result<Vec<T>, V::Error>
| ^^^^^^ found 2 type parameters, expected 1
I think I don't have the correct understanding of how visitors work. Can I have only one visitor for deserializing the array of arrays of strings, or do I need one visitor for deserializing the arrays and one visitor for deserializing the strings to f64?
I've read:
How to transform fields before deserialization using serde?
Is there is a simpler way to convert a type upon deserialization?
As already described in How to transform fields before deserialization using serde?, the easiest solution is to introduce a newtype for your string-as-a-floating-point-value. You can then implement Deserialize for that, leveraging existing implementations of Deserialize and string parsing:
extern crate serde;
#[macro_use]
extern crate serde_derive;
extern crate serde_json;
use serde::de::{Deserialize, Deserializer, Error, Unexpected};
#[derive(Debug, Deserialize)]
struct Payload {
#[serde(default)]
values: Vec<Vec<Value>>,
}
#[derive(Debug)]
struct Value(f64);
impl<'de> Deserialize<'de> for Value {
fn deserialize<D>(deserializer: D) -> Result<Value, D::Error>
where D: Deserializer<'de>
{
let s: &str = Deserialize::deserialize(deserializer)?;
s.parse()
.map(Value)
.map_err(|_| D::Error::invalid_value(Unexpected::Str(s), &"a floating point number as a string"))
}
}
fn main() {
let input = r#"
{
"values": [["2", "1.4"], ["8.32", "1.5"]]
}
"#;
let out: Payload = serde_json::from_str(input).unwrap();
println!("{:?}", out);
}
I prefer this solution because in many cases I want that new type to play a role in my system.
If you really, truly need to deserialize once and to exactly a Vec<Vec<f64>>, you have to implement two visitors. One will deserialize the outer Vec, one will deserialize the inner Vec. We will reuse the previous Value newtype, but the inner visitor will strip it away. The outer visitor will do the same thing for a newtype around the inner visitor:
extern crate serde;
#[macro_use]
extern crate serde_derive;
extern crate serde_json;
use serde::de::{Deserialize, Deserializer, Error, SeqAccess, Unexpected, Visitor};
use std::fmt;
#[derive(Debug, Deserialize)]
struct Payload {
#[serde(default, deserialize_with = "from_array_of_arrays_of_strs")]
values: Vec<Vec<f64>>,
}
fn from_array_of_arrays_of_strs<'de, D>(deserializer: D) -> Result<Vec<Vec<f64>>, D::Error>
where
D: Deserializer<'de>,
{
struct OuterVisitor;
impl<'de> Visitor<'de> for OuterVisitor {
type Value = Vec<Vec<f64>>;
fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
formatter.write_str("a nonempty sequence of a sequence of numbers")
}
#[inline]
fn visit_seq<V>(self, mut visitor: V) -> Result<Self::Value, V::Error>
where
V: SeqAccess<'de>,
{
let mut vec = Vec::new();
while let Some(Inner(elem)) = try!(visitor.next_element()) {
vec.push(elem);
}
Ok(vec)
}
}
deserializer.deserialize_seq(OuterVisitor)
}
struct Inner(Vec<f64>);
impl<'de> Deserialize<'de> for Inner {
fn deserialize<D>(deserializer: D) -> Result<Inner, D::Error>
where
D: Deserializer<'de>,
{
struct InnerVisitor;
impl<'de> Visitor<'de> for InnerVisitor {
type Value = Inner;
fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
formatter.write_str("a nonempty sequence of numbers")
}
#[inline]
fn visit_seq<V>(self, mut visitor: V) -> Result<Inner, V::Error>
where
V: SeqAccess<'de>,
{
let mut vec = Vec::new();
while let Some(Value(elem)) = try!(visitor.next_element()) {
vec.push(elem);
}
Ok(Inner(vec))
}
}
deserializer.deserialize_seq(InnerVisitor)
}
}
struct Value(f64);
impl<'de> Deserialize<'de> for Value {
fn deserialize<D>(deserializer: D) -> Result<Value, D::Error>
where
D: Deserializer<'de>,
{
let s: &str = Deserialize::deserialize(deserializer)?;
s.parse().map(Value).map_err(|_| {
D::Error::invalid_value(Unexpected::Str(s), &"a floating point number as a string")
})
}
}
fn main() {
let input = r#"
{
"values": [["2", "1.4"], ["8.32", "1.5"]]
}
"#;
let out: Payload = serde_json::from_str(input).unwrap();
println!("{:?}", out);
}
Parsing JSON files with strings instead of numbers is also possible without writing a visitor yourself.
use serde_with::{serde_as, DisplayFromStr};
#[serde_as]
#[derive(Debug, serde::Deserialize)]
struct Payload {
#[serde_as(as = "Vec<Vec<DisplayFromStr>>")]
#[serde(default)]
values: Vec<Vec<f64>>,
}
let j = serde_json::json!({
"values": [["2", "1.4"], ["8.32", "1.5"]]
});
let p: Payload = serde_json::from_value(j)?;
assert_eq!(p.values, vec![vec![2.0, 1.4], vec![8.32, 1.5]]);
The annotation means that we have a Vec inside a Vec and the innermost elements should be deserialized using FromStr and serialized using Display. The annotation is supports any type with Display and FromStr implementations, so it can also be used on u64 or on Url types.

rustc-serialize custom enum decoding

I have a JSON structure where one of the fields of a struct could be either an object, or that object's ID in the database. Let's say the document looks like this with both possible formats of the struct:
[
{
"name":"pebbles",
"car":1
},
{
"name":"pebbles",
"car":{
"id":1,
"color":"green"
}
}
]
I'm trying to figure out the best way to implement a custom decoder for this. So far, I've tried a few different ways, and I'm currently stuck here:
extern crate rustc_serialize;
use rustc_serialize::{Decodable, Decoder, json};
#[derive(RustcDecodable, Debug)]
struct Car {
id: u64,
color: String
}
#[derive(Debug)]
enum OCar {
Id(u64),
Car(Car)
}
#[derive(Debug)]
struct Person {
name: String,
car: OCar
}
impl Decodable for Person {
fn decode<D: Decoder>(d: &mut D) -> Result<Person, D::Error> {
d.read_struct("root", 2, |d| {
let mut car: OCar;
// What magic must be done here to get the right OCar?
/* I tried something akin to this:
let car = try!(d.read_struct_field("car", 0, |r| {
let r1 = Car::decode(r);
let r2 = u64::decode(r);
// Compare both R1 and R2, but return code for Err() was tricky
}));
*/
/* And this got me furthest */
match d.read_struct_field("car", 0, u64::decode) {
Ok(x) => {
car = OCar::Id(x);
},
Err(_) => {
car = OCar::Car(try!(d.read_struct_field("car", 0, Car::decode)));
}
}
Ok(Person {
name: try!(d.read_struct_field("name", 0, Decodable::decode)),
car: car
})
})
}
}
fn main() {
// Vector of both forms
let input = "[{\"name\":\"pebbles\",\"car\":1},{\"name\":\"pebbles\",\"car\":{\"id\":1,\"color\":\"green\"}}]";
let output: Vec<Person> = json::decode(&input).unwrap();
println!("Debug: {:?}", output);
}
The above panics with an EOL which is a sentinel value rustc-serialize uses on a few of its error enums. Full line is
thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: EOF', src/libcore/result.rs:785
What's the right way to do this?
rustc-serialize, or at least its JSON decoder, doesn't support that use case. If you look at the implementation of read_struct_field (or any other method), you can see why: it uses a stack, but when it encounters an error, it doesn't bother to restore the stack to its original state, so when you try to decode the same thing differently, the decoder is operating on an inconsistent stack, eventually leading to an unexpected EOF value.
I would recommend you look into Serde instead. Deserializing in Serde is different: instead of telling the decoder what type you're expecting, and having no clear way to recover if a value is of the wrong type, Serde calls into a visitor that can handle any of the types that Serde supports in the way it wants. This means that Serde will call different methods on the visitor depending on the actual type of the value it parsed. For example, we can handle integers to return an OCar::Id and objects to return an OCar::Car.
Here's a full example:
#![feature(custom_derive, plugin)]
#![plugin(serde_macros)]
extern crate serde;
extern crate serde_json;
use serde::de::{Deserialize, Deserializer, Error, MapVisitor, Visitor};
use serde::de::value::MapVisitorDeserializer;
#[derive(Deserialize, Debug)]
struct Car {
id: u64,
color: String
}
#[derive(Debug)]
enum OCar {
Id(u64),
Car(Car),
}
struct OCarVisitor;
#[derive(Deserialize, Debug)]
struct Person {
name: String,
car: OCar,
}
impl Deserialize for OCar {
fn deserialize<D>(deserializer: &mut D) -> Result<Self, D::Error> where D: Deserializer {
deserializer.deserialize(OCarVisitor)
}
}
impl Visitor for OCarVisitor {
type Value = OCar;
fn visit_u64<E>(&mut self, v: u64) -> Result<Self::Value, E> where E: Error {
Ok(OCar::Id(v))
}
fn visit_map<V>(&mut self, visitor: V) -> Result<Self::Value, V::Error> where V: MapVisitor {
Ok(OCar::Car(try!(Car::deserialize(&mut MapVisitorDeserializer::new(visitor)))))
}
}
fn main() {
// Vector of both forms
let input = "[{\"name\":\"pebbles\",\"car\":1},{\"name\":\"pebbles\",\"car\":{\"id\":1,\"color\":\"green\"}}]";
let output: Vec<Person> = serde_json::from_str(input).unwrap();
println!("Debug: {:?}", output);
}
Output:
Debug: [Person { name: "pebbles", car: Id(1) }, Person { name: "pebbles", car: Car(Car { id: 1, color: "green" }) }]
Cargo.toml:
[dependencies]
serde = "0.7"
serde_json = "0.7"
serde_macros = "0.7"

Unable to tackle optional fields in JSON with Rustc-serialize

I am trying to deserialize JSON to Rust structure using rustc_serialize. The problem is that certain JSONs have some optional fields, i.e., may or may not be present. The moment the first absent field is encountered, the decoder seems to bail out and not consider subsequent fields, even if they are present. Is there a way to overcome this?
Here is the code:
extern crate rustc_serialize;
#[derive(Debug)]
struct B {
some_field_0: Option<u64>,
some_field_1: Option<String>,
}
impl rustc_serialize::Decodable for B {
fn decode<D: rustc_serialize::Decoder>(d: &mut D) -> Result<Self, D::Error> {
Ok(B {
some_field_0: d.read_struct_field("some_field_0", 0, |d| rustc_serialize::Decodable::decode(d)).ok(),
some_field_1: d.read_struct_field("some_field_1", 0, |d| rustc_serialize::Decodable::decode(d)).ok(),
})
}
}
fn main() {
{
println!("--------------------------------\n1st run - all field present\n--------------------------------");
let json_str = "{\"some_field_0\": 1234, \"some_field_1\": \"There\"}".to_string();
let obj_b: B = rustc_serialize::json::decode(&json_str).unwrap();
println!("\nJSON: {}\nDecoded: {:?}", json_str, obj_b);
}
{
println!("\n\n--------------------------------\n2nd run - \"some_field_1\" absent\n---------------------------------");
let json_str = "{\"some_field_0\": 1234}".to_string();
let obj_b: B = rustc_serialize::json::decode(&json_str).unwrap();
println!("\nJSON: {}\nDecoded: {:?}", json_str, obj_b);
}
{
println!("\n\n--------------------------------\n3rd run - \"some_field_0\" absent\n---------------------------------");
let json_str = "{\"some_field_1\": \"There\"}".to_string();
let obj_b: B = rustc_serialize::json::decode(&json_str).unwrap();
println!("\nJSON: {}\nDecoded: {:?}", json_str, obj_b);
}
}
and here's the output:
--------------------------------
1st run - all field present
--------------------------------
JSON: {"some_field_0": 1234, "some_field_1": "There"}
Decoded: B { some_field_0: Some(1234), some_field_1: Some("There") }
--------------------------------
2nd run - "some_field_1" absent
---------------------------------
JSON: {"some_field_0": 1234}
Decoded: B { some_field_0: Some(1234), some_field_1: None }
--------------------------------
3rd run - "some_field_0" absent
---------------------------------
JSON: {"some_field_1": "There"}
Decoded: B { some_field_0: None, some_field_1: None }
Notice that the third run produces an unexpected result. When the decoder fails to find some_field_0 it fails on all subsequent tokens, even though some_field_1 is present.
There's something wrong with your Decodable implementation. Using the automatically-generated implementation works:
#[derive(Debug, RustcDecodable)]
struct B {
some_field_1: Option<String>,
some_field_0: Option<u64>,
}
JSON: {"some_field_1": "There"}
Decoded: B { some_field_1: Some("There"), some_field_0: None }
Using the generated implementation is the right thing to do if you can. If you cannot, here's the right implementation:
impl rustc_serialize::Decodable for B {
fn decode<D: rustc_serialize::Decoder>(d: &mut D) -> Result<Self, D::Error> {
Ok(B {
some_field_0: try!(d.read_struct_field("some_field_0", 0, |d| rustc_serialize::Decodable::decode(d))),
some_field_1: try!(d.read_struct_field("some_field_1", 0, |d| rustc_serialize::Decodable::decode(d))),
})
}
}
The important change is the use of try!. Decoding can fail. By using ok, you were saying that a failed decoding was actually a success, albeit a successful decoding of a None.