If the JSON object is missing some fields, the decode function throws an exception. For example:
extern crate rustc_serialize;
use rustc_serialize::json;
use rustc_serialize::json::Json;
#[derive(RustcDecodable, RustcEncodable, Debug)]
enum MessageType {
PING,
PONG,
OPT,
}
#[derive(RustcDecodable, RustcEncodable, Debug)]
pub struct JMessage {
msg_type: String,
mtype: MessageType,
}
fn main() {
let result3 = json::decode::<JMessage>(r#"{"msg_type":"TEST"}"#);
println!("{:?}", result3);
// this will print `Err(MissingFieldError("mtype"))`
let result = json::decode::<JMessage>(r#"{"msg_type":"TEST", "mtype":"PING"}"#);
println!("{:?}", &result);
// This will print Ok(JMessage { msg_type: "TEST", mtype: PING })
let result2 = Json::from_str(r#"{"msg_type":"TEST", "mtype":"PING"}"#).unwrap();
println!("{:?}", &result2);
// this will print Object({"msg_type": String("TEST"), "mtype": String("PING")})
}
Is there a way to specify that some fields in a struct are optional?
Why does the function from_str not serialize mtype as an enum?
No, there is no such way. For that, you need to use serde. Serde also has lots of other features, but unfortunately it is not as easy to use as rustc_serialize on stable Rust.
Well, how should it? Json::from_str returns a JSON AST, which consists of maps, arrays, strings, numbers and other JSON types. It simply cannot contain values of your enum. And also there is no way to indicate that you want some other type instead of string, naturally.
Regarding the first question, you can use Option. For example:
pub struct JMessage {
msg_type: Option<String>,
mtype: MessageType,
}
Which defaults to None if the field does not exist.
Related
I am working on some code that takes a struct returned by an external library, serializes it to json, and serializes the json to protobuf using pbjson. The external library uses serde and implements Serialize, but the json that is returned is snake case. The problem is that pbjson is expecting the json to be camelcase.
How can I get a camelcase version of the serde json object? (ie configure the external library to use something like #[serde(rename_all = "camelCase")] or to convert the json keys to camelcase?)
Note: I am working with many remote structs that in total add up to almost 2k lines of code. I would like to avoid recreating these types locally if possible.
If I understand correctly, you want something like this? Basically you just need to turn the foreign items into items of your own type and then serialize those.
// foreign struct
#[derive(Serialize, Deserialize)]
struct Foreign {
the_first_field: u32,
the_second_field: String,
}
#[derive(Serialize)]
#[serde(rename_all = "camelCase")]
struct Mine {
the_first_field: u32,
the_second_field: String,
}
impl From<Foreign> for Mine {
fn from(
Foreign {
the_first_field,
the_second_field,
}: Foreign,
) -> Self {
Self {
the_first_field,
the_second_field,
}
}
}
fn main() {
// only used to construct the foreign items
let in_json = r#"[{"the_first_field": 1, "the_second_field": "second"}]"#;
let foreign_items = serde_json::from_str::<Vec<Foreign>>(in_json).unwrap();
let mine_items = foreign_items.into_iter().map(Mine::from).collect::<Vec<_>>();
let out_json = serde_json::to_string(&mine_items).unwrap();
println!("{}", out_json); // [{"theFirstField":1,"theSecondField":"second"}]
}
I'm using serde to deserialize a JSON file and one of it's values is a String.
To read it, I'm using:
#[serde(default)]
pub key: Option<String>,
because in the JSON file I can have null (then Option handles) or not even pass it (#serde(default)] handles it).
The problem I'm having is that when in the null case, the Option is returning None, which is giving me trouble later. I have to later match the Strings and convert to an i8 like this:
let mut transformed: i8 = 0;
if key.as_ref().unwrap() == "H" {
transformed = 1;
}
else {
transformed = -1; // Case that I'm looking for when null in JSON
}
I searched for match practices to handle the None, but it's also giving me trouble with the String vs &str problem, so I'm looking for a way of when deserializing, assign an empty String "" instead of None, so later I can compare in the same way I'm already doing.
Also would appreciate less verbose solution to directly parse and assign an i8.
As mentioned by mcarton in the comments, the easiest solution is to stick with using an Option<String> in you struct and use .unwrap_or_default() in the code consuming the data. If this isn't an option for you, you can provide a custom deserializer using the #[serde(deserialize_with=...)] attribute:
use serde_json::from_str;
use serde::{Deserialize, Deserializer, Serialize};
#[derive(Serialize, Deserialize, Debug)]
struct A {
#[serde(deserialize_with = "null_to_default")]
key: String,
}
fn null_to_default<'de, D, T>(de: D) -> Result<T, D::Error>
where
D: Deserializer<'de>,
T: Default + Deserialize<'de>,
{
let key = Option::<T>::deserialize(de)?;
Ok(key.unwrap_or_default())
}
fn main() {
let a: A = from_str(r#"{"key": null}"#).unwrap();
dbg!(a);
}
(Playground)
Is it possible to use a value in JSON to determine how to deserialize the rest of the JSON using serde? For example, consider the following code:
use serde::{Serialize, Deserialize};
use serde_repr::*;
#[derive(Serialize_repr, Deserialize_repr, Debug)]
#[repr(u8)]
enum StructType {
Foo = 1,
Bar = 2
}
#[derive(Serialize, Deserialize, Debug)]
struct Foo {
a: String,
b: u8
}
#[derive(Serialize, Deserialize, Debug)]
struct Bar {
x: String,
y: u32,
z: u16
}
#[derive(Serialize, Deserialize, Debug)]
struct AllMyStuff {
type: StructType,
data: //HELP: Not sure what to put here
}
What I'm trying to achieve is deserialization of the data, even if in multiple steps, where the type field in the AllMyStuff determines which type of struct data is present in data. For example, given the following pseudocode, I'd like to ultimately have a Bar struct with the proper data in it:
data = {"type": "2", "data": { "x": "Hello world", "y": "18", "z": "5" } }
// 1) use serde_json to deserialize a AllMyStuff struct, not erroring on the "data" blob
// 2) Now that we know data is of type "2" (or Bar), parse the remaining "data" into a AllMyStuff struct
If steps (1) and (2) are somehow able to be done in a single step, that would be awesome but not needed. I'm not sure what type of type to declare data in the AllMyStuff struct to enable this as well.
You can use serde_json::Value as the type for AllMyStuff::data. It will deserialize any valid json object and also implements Deserialize itself, so it can be further deserialized once the type to deserialize to is known (via AllMyStuff::type). While this requires more intermittent steps and (mostly temporary) types, it saves you from manually implementing Deserialize on an enum AllMyStuff { Foo(Foo), Bar(Bar) }.
I may be missing something, but AllMyStuff looks as if you are trying to manually distinguish between Foo and Bar.
However, Rust, has a built-in way of doing this:
#[derive(Serialize, Deserialize, Debug)]
enum AllMyStuff {
Foo(Foo),
Bar(Bar),
}
Click here to see it in action.
I want to deserialize the chemical elements JSON file from Bowserinator on github using Serde. For this I created a structure with all the needed fields and derived the needed macros:
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct Element {
name: String,
appearance: String,
atomic_mass: f64,
boil: f64,
category: String,
#[serde(default)]
color: String,
density: f64,
discovered_by: String,
melt: f64,
#[serde(default)]
molar_heat: f64,
named_by: String,
number: String,
period: u32,
phase: String,
source: String,
spectral_img: String,
summary: String,
symbol: String,
xpos: u32,
ypos: u32,
}
This works fine until it gets to fields which contain a "null" value.
E.g. for the field "color": null, in Helium.
The error message I get is { code: Message("invalid type: unit value, expected a string"), line: 8, column: 17 } for this field.
I experimented with the #[serde(default)] Macro. But this only works when the field is missing in the JSON file, not when there is a null value.
I like to do the deserialization with the standard macros avoiding to program a Visitor Trait. Is there a trick I miss?
A deserialization error occurs because the struct definition is incompatible with the incoming objects: the color field can also be null, as well as a string, yet giving this field the type String forces your program to always expect a string. This is the default behaviour, which makes sense. Be reminded that String (or other containers such as Box) are not "nullable" in Rust. As for a null value not triggering the default value instead, that is just how Serde works: if the object field wasn't there, it would work because you have added the default field attribute. On the other hand, a field "color" with the value null is not equivalent to no field at all.
One way to solve this is to adjust our application's specification to accept null | string, as specified by #user25064's answer:
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct Element {
color: Option<String>,
}
Playground with minimal example
Another way is to write our own deserialization routine for the field, which will accept null and turn it to something else of type String. This can be done with the attribute #[serde(deserialize_with=...)].
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct Element {
#[serde(deserialize_with="parse_color")]
color: String,
}
fn parse_color<'de, D>(d: D) -> Result<String, D::Error> where D: Deserializer<'de> {
Deserialize::deserialize(d)
.map(|x: Option<_>| {
x.unwrap_or("black".to_string())
})
}
Playground
See also:
How can I distinguish between a deserialized field that is missing and one that is null?
Any field that can be null should be an Option type so that you can handle the null case. Something like this?
#[derive(Serialize, Deserialize, Debug, Clone)]
pub struct Element {
...
color: Option<String>,
...
}
Based on code from here, when one needs default values to be deserialized if null is present.
// Omitting other derives, for brevity
#[derive(Deserialize)]
struct Foo {
#[serde(deserialize_with = "deserialize_null_default")]
value: String,
}
fn deserialize_null_default<'de, D, T>(deserializer: D) -> Result<T, D::Error>
where
T: Default + Deserialize<'de>,
D: Deserializer<'de>,
{
let opt = Option::deserialize(deserializer)?;
Ok(opt.unwrap_or_default())
}
playground link with full example. This also works for Vec and HashMap.
I'm very new to Rust - and this code is part of an artificial, learning project. Keep that in mind ;)
I have a collection of tuples: &[(i32, String, String, PathBuf)] that are passed into a function designed to write the data to a JSON file.
The problem: when I convert the PathBuf to a &str - the path written to file has unescaped backslash characters, so the JSON is invalid.
Here's the code:
use std::io;
use std::io::prelude::*;
use std::fs::File;
use std::path::PathBuf;
pub fn write_review_queue(ordered_review_queue: &[(i32, String, String, PathBuf)]) -> io::Result<()> {
let output_file = "C:\\Dev\\Temp\\ReviewQueue\\review_queue.json";
let mut buffer = try!(File::create(output_file));
try!(buffer.write("{".to_string().as_bytes()));
let mut is_first_item = true;
for review_item in ordered_review_queue {
if !is_first_item {
try!(buffer.write(",".to_string().as_bytes()));
}
is_first_item = false;
let json_string = "\"ReviewItem\": ".to_string() +
"{\"Index\": " + &review_item.0.to_string() +
", \"ReviewItemName\": \"" + &review_item.1 +
"\", \"ReviewItemPath\": \"" + &review_item.2 +
"\", \"MetadataPath\": \"" + review_item.3.to_str().unwrap() +
"\"}";
try!(buffer.write(json_string.as_bytes()));
}
try!(buffer.write("}".to_string().as_bytes()));
Ok(())
}
And an example of the output:
{
"ReviewItem": {
"Index": 1,
"ReviewItemName": "Crying Cat",
"ReviewItemPath": "C:/Temp",
"MetadataPath": "C:\Dev\Temp\ReviewQueue\Metadata\cryingcat.json"
},
"ReviewItem": {
"Index": 2,
"ReviewItemName": "Rusty Rat",
"ReviewItemPath": "C:/Temp",
"MetadataPath": "C:\Dev\Temp\ReviewQueue\Metadata\rustyrat.json"
}
}
The code that produces the PathBufs for the MetadataPaths is like this:
let metadata_files = metadata_read::read_filenames_from_dir("C:\\Dev\\Temp\\ReviewQueue\\Metadata");
if !metadata_files.is_ok() {
println!("reading metadata filenames failed");
return;
}
let mut metadata_counts = Vec::new();
for file in metadata_files.unwrap() {
let metadata_field_count = metadata_read::count_nonempty_metadata_fields(&file, &keys);
metadata_counts.push(metadata_field_count.unwrap());
}
And the count_nonempty_metadata_fields function:
pub fn count_nonempty_metadata_fields(file_path: &PathBuf, metadata_keys: &[String]) -> Result<(i32, String, String, PathBuf), io::Error>
{
// a bunch of code here...
let path = file_path.to_path_buf();
Ok((key_count, review_item_name, review_item_path, path))
}
If I change the original directory path string to:
let metadata_files = metadata_read::read_filenames_from_dir("C:/Dev/Temp/ReviewQueue/Metadata");
It does change the output, e.g.
{
"ReviewItem": {
"Index": 1,
"ReviewItemName": "Crying Cat",
"ReviewItemPath": "C:/Temp",
"MetadataPath": "C:/Dev/Temp/ReviewQueue/Metadata\cryingcat.json"
},
"ReviewItem": {
"Index": 2,
"ReviewItemName": "Rusty Rat",
"ReviewItemPath": "C:/Temp",
"MetadataPath": "C:/Dev/Temp/ReviewQueue/Metadata\rustyrat.json"
}
}
But it's still not right.
Questions
If I stick with this approach of building up a String in hand-crafted JSON format, how do I get the path content of the PathBufs into a format with either forward-slashes or escaped backslashes? Am I missing something in the API?
Should I being using a Json object to build the data (which will probably be more reliable)? If so, what's the normal way to write the content of a Json object to file?
It's a good idea to never hand-generate any structured format, because eventually the output will become malformed. Additionally, your output has an object with the same two keys. While not invalid, it's likely not what you want.
In this case, you will quickly run into walls with trying to escape quotes and backslashes and maybe apostrophes and ampersands. You also have to keep track of the last item by hand. Let the library do the hard work.
There are two good JSON libraries for Rust: rustc_serialize and serde.
Step one is to create some actual types for your data. Tuples are great, but will you really remember that foo.1 is the name... or was it foo.2?
Once you have that, you can simply output the slice:
extern crate rustc_serialize;
use rustc_serialize::json;
use std::io;
use std::io::prelude::*;
use std::fs::File;
use std::path::PathBuf;
#[derive(RustcEncodable)]
struct Item {
index: i32,
name: String,
path: String,
metadata_path: PathBuf,
}
fn write_review_queue(ordered_review_queue: &[Item]) -> io::Result<()> {
let mut buffer = try!(File::create("/tmp/output"));
write!(buffer, "{}", json::as_json(&ordered_review_queue))
}
fn main() {
let a = [Item { index: 0, name: "He\"llo".into(), path: "Good\\bye".into(), metadata_path: PathBuf::from(r#"C:\path\with'n\special"\chars"#)}];
write_review_queue(&a).expect("Failed");
}
Unfortunately, this prints out the PathBuf in an ugly way:
[{"index":0,"name":"He\"llo","path":"Good\\bye","metadata_path":[67,58,92,112,97,116,104,92,119,105,116,104,39,110,92,115,112,101,99,105,97,108,34,92,99,104,97,114,115]}]
It's important to know that PathBufs are not strings. Specifically, they are platform-dependent abstractions. On Unix-like systems the path is a collection of bytes close-to-but-not UTF-8, and on Windows it is close-to-but-not UCS-2.
You have to decide what lossy transformation is appropriate to convert it to true UTF-8 for your case. I'll use the one built-in to the standard library, to_string_lossy. I also implement ToJson for the type to allow more customization:
extern crate rustc_serialize;
use rustc_serialize::json::{self, ToJson, Json};
use std::io;
use std::io::prelude::*;
use std::fs::File;
use std::path::PathBuf;
use std::collections::BTreeMap;
struct Item {
index: i32,
name: String,
path: String,
metadata_path: PathBuf,
}
impl ToJson for Item {
fn to_json(&self) -> Json {
let mut obj = BTreeMap::new();
obj.insert("Index".to_string(), self.index.to_json());
obj.insert("ReviewItemName".to_string(), self.name.to_json());
obj.insert("ReviewItemPath".to_string(), self.path.to_json());
obj.insert("MetadataPath".to_string(), self.metadata_path.to_string_lossy().to_json());
obj.to_json()
}
}
fn write_review_queue(ordered_review_queue: &[Item]) -> io::Result<()> {
let mut buffer = try!(File::create("/tmp/output"));
write!(buffer, "{}", json::as_json(&ordered_review_queue.to_json()))
}
fn main() {
let a = [Item { index: 0, name: "He\"llo".into(), path: "Good\\bye".into(), metadata_path: PathBuf::from(r#"C:\path\with'n\special"\chars"#)}];
write_review_queue(&a).expect("Failed");
}
Note that this also allows you an opportunity to rename the keys of the object (although the names seem very redundant to me).
[{"Index":0,"MetadataPath":"C:\\path\\with'n\\special\"\\chars","ReviewItemName":"He\"llo","ReviewItemPath":"Good\\bye"}]