Rust: enable Serde to distinguish between undefined & null [duplicate] - json

Question

I'd like to use Serde to parse some JSON as part of a HTTP PATCH request. Since PATCH requests don't pass the entire object, only the relevant data to update, I need the ability to tell between a value that was not passed, a value that was explicitly set to null, and a value that is present.
I have a value object with multiple nullable fields:
struct Resource {
a: Option<i32>,
b: Option<i32>,
c: Option<i32>,
}
If the client submits JSON like this:
{"a": 42, "b": null}
I'd like to change a to Some(42), b to None, and leave c unchanged.
I tried wrapping each field in one more level of Option:
#[derive(Debug, Deserialize)]
struct ResourcePatch {
a: Option<Option<i32>>,
b: Option<Option<i32>>,
c: Option<Option<i32>>,
}
playground
This does not make a distinction between b and c; both are None but I'd have wanted b to be Some(None).
I'm not tied to this representation of nested Options; any solution that can distinguish the 3 cases would be fine, such as one using a custom enum.

Answer 1

Building off of E_net4's answer, you can also create an enum for the three possibilities:
#[derive(Debug)]
enum Patch<T> {
Missing,
Null,
Value(T),
}
impl<T> Default for Patch<T> {
fn default() -> Self {
Patch::Missing
}
}
impl<T> From<Option<T>> for Patch<T> {
fn from(opt: Option<T>) -> Patch<T> {
match opt {
Some(v) => Patch::Value(v),
None => Patch::Null,
}
}
}
impl<'de, T> Deserialize<'de> for Patch<T>
where
T: Deserialize<'de>,
{
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
Option::deserialize(deserializer).map(Into::into)
}
}
This can then be used as:
#[derive(Debug, Deserialize)]
struct ResourcePatch {
#[serde(default)]
a: Patch<i32>,
}
Unfortunately, you still have to annotate each field with #[serde(default)] (or apply it to the entire struct). Ideally, the implementation of Deserialize for Patch would handle that completely, but I haven't figured out how to do that yet.

Answer 2

Quite likely, the only way to achieve that right now is with a custom deserialization function. Fortunately, it is not hard to implement, even to make it work for any kind of field:
fn deserialize_optional_field<'de, T, D>(deserializer: D) -> Result<Option<Option<T>>, D::Error>
where
D: Deserializer<'de>,
T: Deserialize<'de>,
{
Ok(Some(Option::deserialize(deserializer)?))
}
Then each field would be annotated as thus:
#[serde(deserialize_with = "deserialize_optional_field")]
a: Option<Option<i32>>,
You also need to annotate the struct with #[serde(default)], so that empty fields are deserialized to an "unwrapped" None. The trick is to wrap present values around Some.
Serialization relies on another trick: skipping serialization when the field is None:
#[serde(deserialize_with = "deserialize_optional_field")]
#[serde(skip_serializing_if = "Option::is_none")]
a: Option<Option<i32>>,
Playground with the full example. The output:
Original JSON: {"a": 42, "b": null}
> Resource { a: Some(Some(42)), b: Some(None), c: None }
< {"a":42,"b":null}

Answer 3

Building up on Shepmaster's answer and adding serialization.
use serde::ser::Error;
use serde::{Deserialize, Deserializer};
use serde::{Serialize, Serializer};
// #region ------ JSON Absent support
// build up on top of https://stackoverflow.com/a/44332837
/// serde Valueue that can be Absent, Null, or Valueue(T)
#[derive(Debug)]
pub enum Maybe<T> {
Absent,
Null,
Value(T),
}
#[allow(dead_code)]
impl<T> Maybe<T> {
pub fn is_absent(&self) -> bool {
match &self {
Maybe::Absent => true,
_ => false,
}
}
}
impl<T> Default for Maybe<T> {
fn default() -> Self {
Maybe::Absent
}
}
impl<T> From<Option<T>> for Maybe<T> {
fn from(opt: Option<T>) -> Maybe<T> {
match opt {
Some(v) => Maybe::Value(v),
None => Maybe::Null,
}
}
}
impl<'de, T> Deserialize<'de> for Maybe<T>
where
T: Deserialize<'de>,
{
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
let d = Option::deserialize(deserializer).map(Into::into);
d
}
}
impl<T: Serialize> Serialize for Maybe<T> {
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
match self {
// this will be serialized as null
Maybe::Null => serializer.serialize_none(),
Maybe::Value(v) => v.serialize(serializer),
// should have been skipped
Maybe::Absent => Err(Error::custom(
r#"Maybe fields need to be annotated with:
#[serde(default, skip_serializing_if = "Maybe::is_Absent")]"#,
)),
}
}
}
// #endregion --- JSON Absent support
And then you can use it this way:
#[derive(Serialize, Deserialize, Debug)]
struct Rect {
#[serde(default, skip_serializing_if = "Maybe::is_absent")]
stroke: Maybe<i32>,
w: i32,
#[serde(default, skip_serializing_if = "Maybe::is_absent")]
h: Maybe<i32>,
}
// ....
let json = r#"
{
"stroke": null,
"w": 1
}"#;
let deserialized: Rect = serde_json::from_str(json).unwrap();
println!("deserialized = {:?}", deserialized);
// will output: Rect { stroke: Null, w: 1, h: Absent }
let serialized = serde_json::to_string(&deserialized).unwrap();
println!("serialized back = {}", serialized);
// will output: {"stroke":null,"w":1}
I wish Serde had a built-in way to handle JSON's null and absent states.
Update 2021-03-12 - Updated to Maybe::Absent as it is more JSON and SQL DSL idiomatic.
The catch with this approach is that we can express:
type | null with the default Option<type>
type | null | absent with Maybe<type>
But we cannot express
type | absent
The solution would be to refactor Maybe to just have ::Present(value) and ::Absent and support Maybe<Option<type>> for the type | null | absent. So this will give us full coverage.
type | null with the default Option<type>
type | absent with Maybe<type>
type | absent | null with Maybe<Option<type>>
I am trying to implement this without adding a #[serde(deserialize_with = "deserialize_maybe_field")] but not sure it is possible. I might be missing something obvious.