Is there a nice way to tentatively deserialize a JSON into different structs? Couldn't find anything in the docs and unfortunately the structs have "tag" to differentiate as in How to conditionally deserialize JSON to two different variants of an enum?
So far my approach has been like this:
use aws_lambda_events::event::{
firehose::KinesisFirehoseEvent, kinesis::KinesisEvent,
kinesis_analytics::KinesisAnalyticsOutputDeliveryEvent,
};
use lambda::{lambda, Context};
use serde_json::Value;
type Error = Box<dyn std::error::Error + Send + Sync + 'static>;
enum MultipleKinesisEvent {
KinesisEvent(KinesisEvent),
KinesisFirehoseEvent(KinesisFirehoseEvent),
KinesisAnalyticsOutputDeliveryEvent(KinesisAnalyticsOutputDeliveryEvent),
None,
}
#[lambda]
#[tokio::main]
async fn main(event: Value, _: Context) -> Result<String, Error> {
let multi_kinesis_event = if let Ok(e) = serde_json::from_value::<KinesisEvent>(event.clone()) {
MultipleKinesisEvent::KinesisEvent(e)
} else if let Ok(e) = serde_json::from_value::<KinesisFirehoseEvent>(event.clone()) {
MultipleKinesisEvent::KinesisFirehoseEvent(e)
} else if let Ok(e) = serde_json::from_value::<KinesisAnalyticsOutputDeliveryEvent>(event) {
MultipleKinesisEvent::KinesisAnalyticsOutputDeliveryEvent(e)
} else {
MultipleKinesisEvent::None
};
// code below is just sample
let s = match multi_kinesis_event {
MultipleKinesisEvent::KinesisEvent(_) => "Kinesis Data Stream!",
MultipleKinesisEvent::KinesisFirehoseEvent(_) => "Kinesis Firehose!",
MultipleKinesisEvent::KinesisAnalyticsOutputDeliveryEvent(_) => "Kinesis Analytics!",
MultipleKinesisEvent::None => "Not Kinesis!",
};
Ok(s.to_owned())
}
You should use the #untagged option.
use serde::{Serialize, Deserialize};
#[derive(Serialize, Deserialize, Debug)]
struct KinesisFirehoseEvent {
x: i32,
y: i32
}
#[derive(Serialize, Deserialize, Debug)]
struct KinesisEvent(i32);
#[derive(Serialize, Deserialize, Debug)]
#[serde(untagged)]
enum MultipleKinesisEvent {
KinesisEvent(KinesisEvent),
KinesisFirehoseEvent(KinesisFirehoseEvent),
None,
}
fn main() {
let event = MultipleKinesisEvent::KinesisFirehoseEvent(KinesisFirehoseEvent { x: 1, y: 2 });
// Convert the Event to a JSON string.
let serialized = serde_json::to_string(&event).unwrap();
// Prints serialized = {"x":1,"y":2}
println!("serialized = {}", serialized);
// Convert the JSON string back to a MultipleKinesisEvent.
// Since it is untagged
let deserialized: MultipleKinesisEvent = serde_json::from_str(&serialized).unwrap();
// Prints deserialized = KinesisFirehoseEvent(KinesisFirehoseEvent { x: 1, y: 2 })
println!("deserialized = {:?}", deserialized);
}
See in playground
Docs for: Untagged
Related
I have an enum:
#[derive(Serialize, Deserialize)]
enum Action {
Join,
Leave,
}
and a struct:
#[derive(Serialize, Deserialize)]
struct Message {
action: Action,
}
and I pass a JSON string:
"{\"action\":0}" // `json_string` var
but when I try deserialzing this like this:
let msg: Message = serde_json::from_str(json_string)?;
I get the error expected value at line 1 column 11.
In the JSON if I were to replace the number 0 with the string "Join" it works, but I want the number to correspond to the Action enum's values (0 is Action::Join, 1 is Action::Leave) since its coming from a TypeScript request. Is there a simple way to achieve this?
You want serde_repr!
Here's example code from the library's README:
use serde_repr::{Serialize_repr, Deserialize_repr};
#[derive(Serialize_repr, Deserialize_repr, PartialEq, Debug)]
#[repr(u8)]
enum SmallPrime {
Two = 2,
Three = 3,
Five = 5,
Seven = 7,
}
fn main() -> serde_json::Result<()> {
let j = serde_json::to_string(&SmallPrime::Seven)?;
assert_eq!(j, "7");
let p: SmallPrime = serde_json::from_str("2")?;
assert_eq!(p, SmallPrime::Two);
Ok(())
}
For your case:
use serde_repr::{Serialize_repr, Deserialize_repr};
#[derive(Serialize_repr, Deserialize_repr)]
#[repr(u8)]
enum Action {
Join = 0,
Leave = 1,
}
use serde::{Serialize, Deserialize};
#[derive(Serialize, Deserialize)]
struct Message {
action: Action,
}
Without adding any extra dependencies, the least verbose way is possibly to use "my favourite serde trick", the try_from and into container attributes. But in this case I feel that custom implementations of Deserialize and Serialize are more appropriate:
impl<'de> Deserialize<'de> for Action {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: serde::Deserializer<'de>,
{
match i8::deserialize(deserializer)? {
0 => Ok(Action::Join),
1 => Ok(Action::Leave),
_ => Err(serde::de::Error::custom("Expected 0 or 1 for action")),
}
}
}
impl Serialize for Action {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: serde::Serializer,
{
serializer.serialize_i8(match self {
Action::Join => 0,
Action::Leave => 1,
})
}
}
The custom implementations only redirect to serializing/deserializing i8. Playground
I'm trying to use csv and serde to read a mixed-delimiter csv-type file in rust, but I'm having a hard time seeing how to use these libraries to accomplish it. Each line looks roughly like:
value1|value2|subvalue1,subvalue2,subvalue3|value4
and would de-serialize to a struct that looks like:
struct Line {
value1:u64,
value2:u64,
value3:Vec<u64>,
value4:u64,
}
Any guidance on how to tell the library that there are two different delimiters and that one of the columns has this nested structure?
Ok, I'm still a beginner in Rust so I can't guarantee that this is good at all- I suspect it could be done more efficiently, but I do have a solution that works-
use csv::{ReaderBuilder};
use serde::{Deserialize, Deserializer};
use serde::de::Error;
use std::error::Error as StdError;
#[derive(Debug, Deserialize)]
pub struct ListType {
values: Vec<u8>,
}
fn deserialize_list<'de, D>(deserializer: D) -> Result<ListType , D::Error>
where D: Deserializer<'de> {
let buf: &str = Deserialize::deserialize(deserializer)?;
let mut rdr = ReaderBuilder::new()
.delimiter(b',')
.has_headers(false)
.from_reader(buf.as_bytes());
let mut iter = rdr.deserialize();
if let Some(result) = iter.next() {
let record: ListType = result.map_err(D::Error::custom)?;
return Ok(record)
} else {
return Err("error").map_err(D::Error::custom)
}
}
struct Line {
value1:u64,
value2:u64,
#[serde(deserialize_with = "deserialize_list")]
value3:ListType,
value4:u64,
}
fn read_line(line: &str) -> Result<Line, Box<dyn StdError>> {
let mut rdr = ReaderBuilder::new()
.delimiter(b'|')
.from_reader(line.as_bytes());
let mut iter = rdr.deserialize();
if let Some(result) = iter.next() {
let record: Line = result?;
return Ok(Line)
} else {
return Err(From::from("error"));
}
}
[EDIT]
I found that the above solution was intolerably slow, but I was able to make performance acceptable by simply manually deserializing the nested type into a fixed size array by-
#[derive(Debug, Deserialize)]
pub struct ListType {
values: [Option<u8>; 8],
}
fn deserialize_farray<'de, D>(deserializer: D) -> Result<ListType, D::Error>
where
D: Deserializer<'de>,
{
let buf: &str = Deserialize::deserialize(deserializer)?;
let mut split = buf.split(",");
let mut dest: CondList = CondList {
values: [None; 8],
};
let mut ind: usize = 0;
for tok in split {
if tok == "" {
break;
}
match tok.parse::<u8>() {
Ok(val) => {
dest.values[ind] = Some(val);
}
Err(e) => {
return Err(e).map_err(D::Error::custom);
}
}
ind += 1;
}
return Ok(dest);
}
I have a JSON structure where one of the fields of a struct could be either an object, or that object's ID in the database. Let's say the document looks like this with both possible formats of the struct:
[
{
"name":"pebbles",
"car":1
},
{
"name":"pebbles",
"car":{
"id":1,
"color":"green"
}
}
]
I'm trying to figure out the best way to implement a custom decoder for this. So far, I've tried a few different ways, and I'm currently stuck here:
extern crate rustc_serialize;
use rustc_serialize::{Decodable, Decoder, json};
#[derive(RustcDecodable, Debug)]
struct Car {
id: u64,
color: String
}
#[derive(Debug)]
enum OCar {
Id(u64),
Car(Car)
}
#[derive(Debug)]
struct Person {
name: String,
car: OCar
}
impl Decodable for Person {
fn decode<D: Decoder>(d: &mut D) -> Result<Person, D::Error> {
d.read_struct("root", 2, |d| {
let mut car: OCar;
// What magic must be done here to get the right OCar?
/* I tried something akin to this:
let car = try!(d.read_struct_field("car", 0, |r| {
let r1 = Car::decode(r);
let r2 = u64::decode(r);
// Compare both R1 and R2, but return code for Err() was tricky
}));
*/
/* And this got me furthest */
match d.read_struct_field("car", 0, u64::decode) {
Ok(x) => {
car = OCar::Id(x);
},
Err(_) => {
car = OCar::Car(try!(d.read_struct_field("car", 0, Car::decode)));
}
}
Ok(Person {
name: try!(d.read_struct_field("name", 0, Decodable::decode)),
car: car
})
})
}
}
fn main() {
// Vector of both forms
let input = "[{\"name\":\"pebbles\",\"car\":1},{\"name\":\"pebbles\",\"car\":{\"id\":1,\"color\":\"green\"}}]";
let output: Vec<Person> = json::decode(&input).unwrap();
println!("Debug: {:?}", output);
}
The above panics with an EOL which is a sentinel value rustc-serialize uses on a few of its error enums. Full line is
thread '<main>' panicked at 'called `Result::unwrap()` on an `Err` value: EOF', src/libcore/result.rs:785
What's the right way to do this?
rustc-serialize, or at least its JSON decoder, doesn't support that use case. If you look at the implementation of read_struct_field (or any other method), you can see why: it uses a stack, but when it encounters an error, it doesn't bother to restore the stack to its original state, so when you try to decode the same thing differently, the decoder is operating on an inconsistent stack, eventually leading to an unexpected EOF value.
I would recommend you look into Serde instead. Deserializing in Serde is different: instead of telling the decoder what type you're expecting, and having no clear way to recover if a value is of the wrong type, Serde calls into a visitor that can handle any of the types that Serde supports in the way it wants. This means that Serde will call different methods on the visitor depending on the actual type of the value it parsed. For example, we can handle integers to return an OCar::Id and objects to return an OCar::Car.
Here's a full example:
#![feature(custom_derive, plugin)]
#![plugin(serde_macros)]
extern crate serde;
extern crate serde_json;
use serde::de::{Deserialize, Deserializer, Error, MapVisitor, Visitor};
use serde::de::value::MapVisitorDeserializer;
#[derive(Deserialize, Debug)]
struct Car {
id: u64,
color: String
}
#[derive(Debug)]
enum OCar {
Id(u64),
Car(Car),
}
struct OCarVisitor;
#[derive(Deserialize, Debug)]
struct Person {
name: String,
car: OCar,
}
impl Deserialize for OCar {
fn deserialize<D>(deserializer: &mut D) -> Result<Self, D::Error> where D: Deserializer {
deserializer.deserialize(OCarVisitor)
}
}
impl Visitor for OCarVisitor {
type Value = OCar;
fn visit_u64<E>(&mut self, v: u64) -> Result<Self::Value, E> where E: Error {
Ok(OCar::Id(v))
}
fn visit_map<V>(&mut self, visitor: V) -> Result<Self::Value, V::Error> where V: MapVisitor {
Ok(OCar::Car(try!(Car::deserialize(&mut MapVisitorDeserializer::new(visitor)))))
}
}
fn main() {
// Vector of both forms
let input = "[{\"name\":\"pebbles\",\"car\":1},{\"name\":\"pebbles\",\"car\":{\"id\":1,\"color\":\"green\"}}]";
let output: Vec<Person> = serde_json::from_str(input).unwrap();
println!("Debug: {:?}", output);
}
Output:
Debug: [Person { name: "pebbles", car: Id(1) }, Person { name: "pebbles", car: Car(Car { id: 1, color: "green" }) }]
Cargo.toml:
[dependencies]
serde = "0.7"
serde_json = "0.7"
serde_macros = "0.7"
I'm attemptin to include the Double value of 0.81 in some JSON generated by NSJSONSerialization. The code is as follows:
let jsonInput = [ "value": 0.81 ]
let data = try NSJSONSerialization.dataWithJSONObject(jsonInput, options: NSJSONWritingOptions.PrettyPrinted)
let json = NSString(data: data, encoding: NSUTF8StringEncoding)!
print( json )
The output is:
{
"value" : 0.8100000000000001
}
But what I'd like to see is:
{
"value" : 0.81
}
How can I make NSJSONSerialization do this?
One further thing that is confusing me here is Swift's handling of the 64bit Double. As in the playground I can also do this:
let eightOne:Double = 0.81
"\(eightOne)"
print( eightOne )
And the output is then as desired with:
0.81
Even though in the playground it shows eightOne as 0.8100000000000001 as far as internal representation goes. However here when it converts to string it chops off the rest.
I'm surely this is solved, as you'd need it sorted for any kind of financial handling (eg. in Java we know we only use BigDecimals when it comes to financial values).
Please help. :)
NOTE: The focus here is on serialization to JSON. Not just a simple call off to NSString( format: "%\(0.2)f", 0.81).
For precise base-10 arithmetic (up to 38 significant digits)
you can use NSDecimalNumber:
let jsonInput = [ "value": NSDecimalNumber(string: "0.81") ]
or
let val = NSDecimalNumber(integer: 81).decimalNumberByDividingBy(NSDecimalNumber(integer: 100))
let jsonInput = [ "value": val ]
Then
let data = try NSJSONSerialization.dataWithJSONObject(jsonInput, options: NSJSONWritingOptions.PrettyPrinted)
let json = NSString(data: data, encoding: NSUTF8StringEncoding)!
print( json )
produces the output
{
"value" : 0.81
}
Manual conversion
You'll need to convert your Double to a Decimal to keep its expected string representation when serializing.
One way to avoid a precision of 16 digits may be to round with a scale of 15:
(0.81 as NSDecimalNumber).rounding(accordingToBehavior: NSDecimalNumberHandler(roundingMode: .plain, scale: 15, raiseOnExactness: false, raiseOnOverflow: true, raiseOnUnderflow: true, raiseOnDivideByZero: true)) as Decimal
JSONSerialization extension for automatic conversion
To automatically and recursively do it for all Double values in your JSON object, being it a Dictionary or an Array, you can use:
import Foundation
/// https://stackoverflow.com/q/35053577/1033581
extension JSONSerialization {
/// Produce Double values as Decimal values.
open class func decimalData(withJSONObject obj: Any, options opt: JSONSerialization.WritingOptions = []) throws -> Data {
return try data(withJSONObject: decimalObject(obj), options: opt)
}
/// Write Double values as Decimal values.
open class func writeDecimalJSONObject(_ obj: Any, to stream: OutputStream, options opt: JSONSerialization.WritingOptions = [], error: NSErrorPointer) -> Int {
return writeJSONObject(decimalObject(obj), to: stream, options: opt, error: error)
}
fileprivate static let roundingBehavior = NSDecimalNumberHandler(roundingMode: .plain, scale: 15, raiseOnExactness: false, raiseOnOverflow: true, raiseOnUnderflow: true, raiseOnDivideByZero: true)
fileprivate static func decimalObject(_ anObject: Any) -> Any {
let value: Any
if let n = anObject as? [String: Any] {
// subclassing children
let dic = DecimalDictionary()
n.forEach { dic.setObject($1, forKey: $0) }
value = dic
} else if let n = anObject as? [Any] {
// subclassing children
let arr = DecimalArray()
n.forEach { arr.add($0) }
value = arr
} else if let n = anObject as? NSNumber, CFNumberGetType(n) == .float64Type {
// converting precision for correct decimal output
value = NSDecimalNumber(value: anObject as! Double).rounding(accordingToBehavior: roundingBehavior)
} else {
value = anObject
}
return value
}
}
private class DecimalDictionary: NSDictionary {
let _dictionary: NSMutableDictionary = [:]
override var count: Int {
return _dictionary.count
}
override func keyEnumerator() -> NSEnumerator {
return _dictionary.keyEnumerator()
}
override func object(forKey aKey: Any) -> Any? {
return _dictionary.object(forKey: aKey)
}
func setObject(_ anObject: Any, forKey aKey: String) {
let value = JSONSerialization.decimalObject(anObject)
_dictionary.setObject(value, forKey: aKey as NSString)
}
}
private class DecimalArray: NSArray {
let _array: NSMutableArray = []
override var count: Int {
return _array.count
}
override func object(at index: Int) -> Any {
return _array.object(at: index)
}
func add(_ anObject: Any) {
let value = JSONSerialization.decimalObject(anObject)
_array.add(value)
}
}
Usage
JSONSerialization.decimalData(withJSONObject: [ "value": 0.81 ], options: [])
Note
If you need fine tuning of decimal formatting, you can check Eneko Alonso answer on Specify number of decimals when serializing currencies with JSONSerialization.
If you have use 'NSDecimalNumber' demand, it is suggested that encapsulate for ease of use and reduce mistakes.
Here's the Demo for you reference, using a simple.The hope can help you!
switch (operatorType) {
case 0:
resultNumber = SNAdd(_cardinalNumberTextField.text, _complementNumberTextField.text);
break;
case 1:
resultNumber = SNSub(_cardinalNumberTextField.text, _complementNumberTextField.text);
break;
case 2:
resultNumber = SNMul(_cardinalNumberTextField.text, _complementNumberTextField.text);
break;
case 3:
resultNumber = SNDiv(_cardinalNumberTextField.text, _complementNumberTextField.text);
break;
}
Github:https://github.com/ReverseScale/DecimalNumberDemo
I have a struct that I want to encode to JSON. This struct contains a field with type Option<i32>. Let's say
extern crate rustc_serialize;
use rustc_serialize::json;
#[derive(RustcEncodable)]
struct TestStruct {
test: Option<i32>
}
fn main() {
let object = TestStruct {
test: None
};
let obj = json::encode(&object).unwrap();
println!("{}", obj);
}
This will give me the output
{"test": null}
Is there a convenient way to omit Option fields with value None? In this case I would like to have the output
{}
If someone arrives here with the same question like I did, serde has now an option skip_serializing_none to do exactly that.
https://docs.rs/serde_with/1.8.0/serde_with/attr.skip_serializing_none.html
It doesn't seem to be possible by doing purely from a struct, so i converted the struct into a string, and then converted that into a JSON object. This method requires that all Option types be the same type. I'd recommend if you need to have variable types in the struct to turn them into string's first.
field_vec and field_name_vec have to be filled with all fields at compile time, as I couldn't find a way to get the field values, and field names without knowing them in rust at run time.
extern crate rustc_serialize;
use rustc_serialize::json::Json;
fn main() {
#[derive(RustcEncodable)]
struct TestStruct {
test: Option<i32>
}
impl TestStruct {
fn to_json(&self) -> String {
let mut json_string = String::new();
json_string.push('{');
let field_vec = vec![self.test];
let field_name_vec = vec![stringify!(self.test)];
let mut previous_field = false;
let mut count = 0;
for field in field_vec {
if previous_field {
json_string.push(',');
}
match field {
Some(value) => {
let opt_name = field_name_vec[count].split(". ").collect::<Vec<&str>>()[1];
json_string.push('"');
json_string.push_str(opt_name);
json_string.push('"');
json_string.push(':');
json_string.push_str(&value.to_string());
previous_field = true;
},
None => {},
}
count += 1;
}
json_string.push('}');
json_string
}
}
let object = TestStruct {
test: None
};
let object2 = TestStruct {
test: Some(42)
};
let obj = Json::from_str(&object.to_json()).unwrap();
let obj2 = Json::from_str(&object2.to_json()).unwrap();
println!("{:?}", obj);
println!("{:?}", obj2);
}
To omit Option<T> fields, you can create an implementation of the Encodable trait (instead of using #[derive(RustcEncodable)]).
Here I updated your example to do this.
extern crate rustc_serialize;
use rustc_serialize::json::{ToJson, Json};
use rustc_serialize::{Encodable,json};
use std::collections::BTreeMap;
#[derive(PartialEq, RustcDecodable)]
struct TestStruct {
test: Option<i32>
}
impl Encodable for TestStruct {
fn encode<S: rustc_serialize::Encoder>(&self, s: &mut S) -> Result<(), S::Error> {
self.to_json().encode(s)
}
}
impl ToJson for TestStruct {
fn to_json(&self) -> Json {
let mut d = BTreeMap::new();
match self.test {
Some(value) => { d.insert("test".to_string(), value.to_json()); },
None => {},
}
Json::Object(d)
}
}
fn main() {
let object = TestStruct {
test: None
};
let obj = json::encode(&object).unwrap();
println!("{}", obj);
let decoded: TestStruct = json::decode(&obj).unwrap();
assert!(decoded==object);
}
It would be nice to implement a custom #[derive] macro which does this automatically for Option fields, as this would eliminate the need for such custom implementations of Encodable.