WebApi custom JsonConverter not called - json

I implemented a custom JsonConverter for Guids.
If I declare it on the properties (of type Guid) of the class serialized like so
[JsonConverter(typeof(JsonGuidConverter))]
then it gets called and works fine.
However, I'd like to use it "automatically", without needed the attributes, so I do this:
var formatters = GlobalConfiguration.Configuration.Formatters;
var jsonFormatter = formatters.JsonFormatter;
jsonFormatter.SerializerSettings.Converters.Add(new JsonGuidConverter());
Unfortunately this results in my converter never getting called. I'm using WebApi 2.1 in a MVC 5.1 project.
Any ideas?
Edit: here is the converter code
public class JsonGuidConverter : JsonConverter
{
public override bool CanRead
{
get
{
// We only need the converter for writing Guids without dashes, for reading the default mechanism is fine
return false;
}
}
public override bool CanWrite
{
get
{
return true;
}
}
public override bool CanConvert(Type objectType)
{
return objectType == typeof(Guid) || objectType == typeof(Guid?);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, Newtonsoft.Json.JsonSerializer serializer)
{
// We declared above CanRead false so the default serialization will be used
throw new NotImplementedException();
}
public override void WriteJson(JsonWriter writer, object value, Newtonsoft.Json.JsonSerializer serializer)
{
if (value == null || Guid.Empty.Equals(value))
{
writer.WriteValue(string.Empty);
}
else
{
writer.WriteValue(((Guid)value).ToStringNoDashes());
}
}
}
Additional note, not even the CanRead/CanWrite properties and CanConvert method are called when trying to use it by adding it to the converters collection.
Maybe it has something to do with how I return data from the webapi controller?
public async Task<IHttpActionResult> GetSettings()
{
...
return Json(something);
}

Since your are using formatters, do not use Json(something) when returning from action, but rather use Content(something) in this case. The Content helper will honor the formatters' settings.
I agree that Json helper is confusing here and something which I wished we never included in our product.

Related

Distinguish between NULL and not present using JSON Merge Patch with NetCore WebApi and System.Text.Json

I want to support partial updates with JSON Merge Patch. The domain model is based on the always valid concept and has no public setters. Therefore I can't just apply the changes to the class types. I need to translate the changes to the specific commands.
Since classes have nullable properties I need to be able to distinguish between properties set to null and not provided.
I'm aware of JSON Patch. I could use patch.JsonPatchDocument.Operations to go through the list of changes. JSON Patch is just verbose and more difficult for the client. JSON Patch requires to use Newtonsoft.Json (Microsoft states an option to change Startup.ConfigureServices to only use Newtonsoft.Json for JSON Patch (https://learn.microsoft.com/en-us/aspnet/core/web-api/jsonpatch?view=aspnetcore-6.0).
Newtonsoft supports IsSpecified-Properties that can be used as a solution for JSON Merge Patch in the DTO classes (How to make Json.NET set IsSpecified properties for properties with complex values?). This would solve the problem, but again requires Newtonsoft. System.Text.Json does not support this feature. There is an open issue for 2 years (https://github.com/dotnet/runtime/issues/40395), but nothing to expect.
There is a post that describes a solution with a custom JsonConverter for Web API (https://github.com/dotnet/runtime/issues/40395). Would this solution still be usable for NetCore?
I was wondering if there is an option to access the raw json or a json object inside the controller method after the DTO object was filled. Then I could manually check if a property was set. Web Api closes the stream, so I can't access the body anymore. It seems there are ways to change that behavior (https://gunnarpeipman.com/aspnet-core-request-body/#comments). It seems quite complicated and feels like a gun that is too big. I also don't understand what changes were made for NetCore 6.
I'm surpised that such a basic problem needs one to jump through so many loops. Is there an easy way to accomplish my goal with System.Text.Json and NetCore 6? Are there other options? Would using Newtonsoft have any other bad side effects?
With the helpful comments of jhmckimm I found
Custom JSON serializer for optional property with System.Text.Json. DBC shows a fantastic solution using Text.Json and Optional<T>. This should be in the Microsoft docs!
In Startup I added:
services.AddControllers()
.AddJsonOptions(o => o.JsonSerializerOptions.DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingDefault)
.AddJsonOptions(o => o.JsonSerializerOptions.Converters.Add(new OptionalConverter()));
Since we use <Nullable>enable</Nullable> and <WarningsAsErrors>nullable</WarningsAsErrors> I adapted the code for nullables.
public readonly struct Optional<T>
{
public Optional(T? value)
{
this.HasValue = true;
this.Value = value;
}
public bool HasValue { get; }
public T? Value { get; }
public static implicit operator Optional<T>(T value) => new Optional<T>(value);
public override string ToString() => this.HasValue ? (this.Value?.ToString() ?? "null") : "unspecified";
}
public class OptionalConverter : JsonConverterFactory
{
public override bool CanConvert(Type typeToConvert)
{
if (!typeToConvert.IsGenericType) { return false; }
if (typeToConvert.GetGenericTypeDefinition() != typeof(Optional<>)) { return false; }
return true;
}
public override JsonConverter CreateConverter(Type typeToConvert, JsonSerializerOptions options)
{
Type valueType = typeToConvert.GetGenericArguments()[0];
return (JsonConverter)Activator.CreateInstance(
type: typeof(OptionalConverterInner<>).MakeGenericType(new Type[] { valueType }),
bindingAttr: BindingFlags.Instance | BindingFlags.Public,
binder: null,
args: null,
culture: null
)!;
}
private class OptionalConverterInner<T> : JsonConverter<Optional<T>>
{
public override Optional<T> Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
T? value = JsonSerializer.Deserialize<T>(ref reader, options);
return new Optional<T>(value);
}
public override void Write(Utf8JsonWriter writer, Optional<T> value, JsonSerializerOptions options) =>
JsonSerializer.Serialize(writer, value.Value, options);
}
}
My test DTO looks like this:
public class PatchGroupDTO
{
public Optional<Guid?> SalesGroupId { get; init; }
public Optional<Guid?> AccountId { get; init; }
public Optional<string?> Name { get; init; }
public Optional<DateTime?> Start { get; init; }
public Optional<DateTime?> End { get; init; }
}
I can now access the fields and check with .HasValue if the value was set. It also works for writing and allows us to stripe fields based on permission.

How to validate JSON using System.Text.Json before deserialization

In .NET Core 5.0, using System.Text.Json.JsonSerializer Deserialize(someJsonFile) i get:
System.Text.Json.JsonException: 'The JSON value could not be converted to System.Guid. Path: $.someGuid | ..
which is expected, since the the someGuid property is of type System.Guid and the value of someGuid in the JSON file/string is:
{
"someGuid": ""
}
which can not be deserialized properly .. (since it's not Guid.Empty)..
To my question.
What's a good and generic implementation to validate the Json before deserializing it (in general)? like TryParse or JsonDocument.Parse? sure, try-catch but that's dirty (imho).
btw: i don't want to use Newtonsoft
thanks for your suggestions (and critics of course).
I created a custom converter using the example is this answer: The JSON value could not be converted to System.Int32
public class StringToGuidConverter : JsonConverter<Guid>
{
public override Guid Read(ref Utf8JsonReader reader, Type type, JsonSerializerOptions options)
{
if (reader.TokenType == JsonTokenType.String)
{
ReadOnlySpan<byte> span = reader.ValueSpan;
if (Utf8Parser.TryParse(span, out Guid guid, out int bytesConsumed) && span.Length == bytesConsumed)
{
return guid;
}
if (Guid.TryParse(reader.GetString(), out guid))
{
return guid;
}
}
return Guid.Empty;
}
public override void Write(Utf8JsonWriter writer, Guid value, JsonSerializerOptions options)
{
writer.WriteStringValue(value.ToString());
}
}
In my case my model to deserialize to can't take Nullable Guid so I return an empty GUID and then validate this in my logic.
Because I'm create a web api using .Net standard, I can't register this in the services at the startup class. But you can register the custom converter using the JsonSerializerOptions property when calling the Deserialize method like this:
var options = new JsonSerializerOptions
{
PropertyNameCaseInsensitive = true,
Converters = { new NERDS.API.Helpers.StringToGuidConverter() }
};
StreamReader reader = new StreamReader(HttpContext.Current.Request.InputStream);
string json = reader.ReadToEnd();
return JsonSerializer.Deserialize<T>(json, options);

JsonConstructorAttribute in System.Text.Json (but not Newtonsoft.Json) results in exception when property and constructor argument types differ

Given a Base64 string, the following sample class will deserialize properly using Newtonsoft.Json, but not with System.Text.Json:
using System;
using System.Text.Json.Serialization;
public class AvatarImage{
public Byte[] Data { get; set; } = null;
public AvatarImage() {
}
[JsonConstructor]
public AvatarImage(String Data) {
//Remove Base64 header info, leaving only the data block and convert it to a Byte array
this.Data = Convert.FromBase64String(Data.Remove(0, Data.IndexOf(',') + 1));
}
}
With System.Text.Json, the following exception is thrown:
must bind to an object property or field on deserialization. Each parameter name must match with a property or field on the object. The match can be case-insensitive.
Apparently System.Text.Json doesn't like the fact the property is a Byte[] but the parameter is a String, which shouldn't really matter because the whole point is that the constructor should be taking care of the assignments.
Is there any way to get this working with System.Text.Json?
In my particular case Base64 images are being sent to a WebAPI controller, but the final object only needs the Byte[]. In Newtonsoft this was a quick and clean solution.
This is apparently a known restriction of System.Text.Json. See the issues:
[JsonSerializer] Relax restrictions on ctor param type to immutable property type matching where reasonable #44428 which is currently labeled with the 6.0.0 Future milestone.
System.Text.Json incorrectly requires construct parameter types to match immutable property types. #47422
JsonConstrutor different behavior between Newtonsoft.Json and System.Text.Json #46480.
Thus (in .Net 5 at least) you will need to refactor your class to avoid the limitation.
One solution would be to add a surrogate Base64 encoded string property:
public class AvatarImage
{
[JsonIgnore]
public Byte[] Data { get; set; } = null;
[JsonInclude]
[JsonPropertyName("Data")]
public string Base64Data
{
private get => Data == null ? null : Convert.ToBase64String(Data);
set
{
var index = value.IndexOf(',');
this.Data = Convert.FromBase64String(index < 0 ? value : value.Remove(0, index + 1));
}
}
}
Note that, ordinarily, JsonSerializer will only serialize public properties. However, if you mark a property with [JsonInclude] then either the setter or the getter -- but not both -- can be nonpublic. (I have no idea why Microsoft doesn't allow both to be private, the data contract serializers certainly support private members marked with [DataMember].) In this case I chose to make the getter private to reduce the chance the surrogate property is serialized by some other serializer or displayed via some property browser.
Demo fiddle #1 here.
Alternatively, you could introduce a custom JsonConverter<T> for AvatarImage
[JsonConverter(typeof(AvatarConverter))]
public class AvatarImage
{
public Byte[] Data { get; set; } = null;
}
class AvatarConverter : JsonConverter<AvatarImage>
{
class AvatarDTO { public string Data { get; set; } }
public override AvatarImage Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
var dto = JsonSerializer.Deserialize<AvatarDTO>(ref reader, options);
var index = dto.Data?.IndexOf(',') ?? -1;
return new AvatarImage { Data = dto.Data == null ? null : Convert.FromBase64String(index < 0 ? dto.Data : dto.Data.Remove(0, index + 1)) };
}
public override void Write(Utf8JsonWriter writer, AvatarImage value, JsonSerializerOptions options) =>
JsonSerializer.Serialize(writer, new { Data = value.Data }, options);
}
This seems to be the easier solution if for simple models, but can become a nuisance for complex models or models to which properties are frequently added.
Demo fiddle #2 here.
Finally, it seems a bit unfortunate that the Data property will have some extra header prepended during deserialization that is not present during serialization. Rather than fixing this during deserialization, consider modifying your architecture to avoid mangling the Data string in the first place.
Implementing the custom converter deserialization using an ExpandoObject can avoid the nested DTO class if desired:
using System.Dynamic;
.
.
.
public override FileEntity Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) {
dynamic obj = JsonSerializer.Deserialize<ExpandoObject>(ref reader, options);
return new FileEntity {
Data = (obj.data == null) ? null : Convert.FromBase64String(obj.data.GetString().Remove(0, obj.data.GetString().IndexOf(',') + 1))
};
}
it makes the custom converter a little more flexible when developing, since the DTO doesn't continually need to grow with the base class being deserialized into. It also makes handling potential nullable properties a bit easier too (over standard JsonElement deserialization, i.e. JsonSerializer.Deserialize< JsonElement >) as such:
public override FileEntity Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) {
dynamic obj = JsonSerializer.Deserialize<ExpandoObject>(ref reader, options);
return new FileEntity {
SomeNullableInt32Property = obj.id?.GetInt32(),
Data = (obj.data?.GetString() == null) ? null : Convert.FromBase64String(obj.data.GetString().Remove(0, obj.data.GetString().IndexOf(',') + 1))
};
}

Override global Json.NET enum handling for one particular enum via decoration

I have a bunch of enums that I want Json.NET to serialize as camelcased strings. I have the following in my Global.asax.cs file and it's working great:
HttpConfiguration config = GlobalConfiguration.Configuration;
config.Formatters.JsonFormatter.SerializerSettings.Converters.Add(new Newtonsoft.Json.Converters.StringEnumConverter { CamelCaseText = true });
This makes it so an enum like this:
public enum FavoriteWebSite {
StackOverflow,
GoogleNews
// Etc
}
will serialize to values like "stackOverflow", "googleNews", etc.
However, I have a couple enums that are bitwise masks. To make this a simple example, suppose one looks like this:
public enum Hobbies {
Walking = 0x01,
Biking = 0x02,
// Etc
}
What happens when I serialize instances of this enum depends on what kind of values are in it. For example:
Hobbies set1 = Hobbies.Walking; // Serializes as "walking" -- bad
Hobbies set2 = Hobbies.Walking | Hobbies.Biking; // Serializes as "3" -- good!
I want to override the serialization on this enum to just serialize as an int, while leaving the global setting to use camelcased strings intact.
I tried removing the global configuration so that enums by default are serialized as ints, then adding only [JsonConverter(typeof(StringEnumConverter))] to the non-bitmask enums. However, this results in PascalCased, rather than CamelCased serialization for those. I didn't see any way to get the CamelCaseText attribute set when using StringEnumConverter in a method decoration like above.
So, to recap, the goal is:
Have single-value enums be serialized as pascalCased strings.
Have bitmask enums be serialized as ints.
Thank you!
Your main difficulty appears to be that you are not decorating your flag enums with FlagsAttribute, like so:
[Flags]
public enum Hobbies
{
Walking = 0x01,
Biking = 0x02,
// Etc
}
This is the recommended best practice for flag enums:
Designing Flag Enums
√ DO apply the System.FlagsAttribute to flag enums. Do not apply this attribute to simple enums.
See also here. If you don't do this, many enum-related .Net utilities may not work as expected for flag enumerations.
Having done this, StringEnumConverter will serialize flag enums with composite values as a set of comma-separated values instead of as the numeric value you are currently seeing:
{
"Hobbies": "walking, biking"
}
If you don't want this and still prefer to see default, numeric values for flag enums in your JSON, you can subclass StringEnumConverter to only convert non-flag enums:
public class NonFlagStringEnumConverter : StringEnumConverter
{
public override bool CanConvert(Type objectType)
{
if (!base.CanConvert(objectType))
return false;
return !HasFlagsAttribute(objectType);
}
static bool HasFlagsAttribute(Type objectType)
{
return Attribute.IsDefined(Nullable.GetUnderlyingType(objectType) ?? objectType, typeof(System.FlagsAttribute));
}
}
Then use it like:
config.Formatters.JsonFormatter.SerializerSettings.Converters.Add(new NonFlagStringEnumConverter { CamelCaseText = true });
This will cause Json.NET to fall back on any global default JSON converter for enums, or to numeric serialization if there is no applicable fallback. Demo fiddle #1 here.
Additionally, if you need to supersede a converter applied at a higher level and force numeric serialization for flag enums, use the following:
public class ForceNumericFlagEnumConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
if (!(Nullable.GetUnderlyingType(objectType) ?? objectType).IsEnum)
return false;
return HasFlagsAttribute(objectType);
}
public override bool CanRead { get { return false; } }
public override bool CanWrite { get { return false; } }
static bool HasFlagsAttribute(Type objectType)
{
return Attribute.IsDefined(Nullable.GetUnderlyingType(objectType) ?? objectType, typeof(System.FlagsAttribute));
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
throw new NotImplementedException();
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
Demo fiddle #2 here.
This blog post explains pretty well that there is not built in way to override global StringEnumConverter. You need write you own converter that does nothing, then when converting JSON.NET will go back to default converter for that type (which for enums is serializing to it's numeric value).
So in case you have global converter:
config.Formatters.JsonFormatter.SerializerSettings.Converters.Add(new StringEnumConverter { CamelCaseText = true });
You can define this ForceDefaultConverter converter
public class ForceDefaultConverter : JsonConverter
{
public override bool CanRead => false;
public override bool CanWrite => false;
public override bool CanConvert(Type objectType) => throw new NotImplementedException();
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer) => throw new NotImplementedException();
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer) => throw new NotImplementedException();
}
And use it on the property you want to override your default StringEnumConverter.
public class ExampleDto
{
[JsonConverter(typeof(ForceDefaultConverter))]
public TestEnum EnumValue { get; set; }
}
or on enum type itself if you want this numeric values when serializing all objects with this enum type.
[JsonConverter(typeof(ForceDefaultConverter))]
public enum TestEnum
{
Foo = 1,
Bar = 2
}

Webapi with entity framework - use customer JSON converter to serialized DbGeography not working

I am using webapi with DbGeography spatial data and want to serialize to json.
By default, DbGeography serializes to null. So I implemented my own converter for it.
Here is what I have so far, but it doesn't seem to work.
Basically, with the following code, my DbGeographyConverter.WriteJson method is never under debug and the Location property is serialized as null
Customer converter:
public class DbGeographyConverter : JsonConverter
{
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
DbGeography contextObj = value as DbGeography;
writer.WriteStartObject();
writer.WritePropertyName("Lat");
serializer.Serialize(writer, contextObj.Latitude);
writer.WritePropertyName("Long");
serializer.Serialize(writer, contextObj.Longitude);
writer.WriteEndObject();
}
public override bool CanConvert(Type objectType)
{
if (objectType == typeof(DbGeography))
{
return true;
}
return false;
}
public override bool CanRead
{
get
{
return true;
}
}
public override bool CanWrite
{
get
{
return true;
}
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
throw new NotImplementedException();
}
Add convert in Global.ascx.cs
protected void Application_Start()
{
GlobalConfiguration.Configuration.Formatters.JsonFormatter.SerializerSettings.Converters.Add(
new DbGeographyConverter()
}
Finally, apply converter to the data model class property
public DataModelClass1
{
[JsonConverter (typeof(DbGeographyConverter))]
public DbGeography Location { get; set; }
}
First, since you're adding your custom converter to the SerializerSettings.Converters collection, you don't need to decorate your DataModelClass1's Location property with the JsonConverterAttribute- The JsonFormatter will run through the aforementioned collection until it finds the derived JsonConverter you added without the attribute.
Now back to your question- which browser are you testing this in and how? If I were to speculate, I'd say you're using either Chrome or Firefox with GET requests, both of which prioritize application/xml over application/json in the accept header they send to the server. For that reason Web API will see that the browsers prefer XML over JSON and the JsonFormatter will never be touched, let alone your custom JsonConverter.
There are a few workarounds to this. On the browser side the easiest way is to make ajax GET requests with jQuery and specify that you want JSON back. On the server side, you can remove application/xml from the SupportedMediaTypes.
I spent quite awhile on this. You'd only use the write method if you wanted to change the default output format of the Json for DbGeography from this
"geography": {
"coordinateSystemId": 4326,
"wellKnownText": "POINT (77.6599502563474 12.9602302518557)"
}
to something else like "77.22, 12.8" - just a single string.
If you're looking to convert a string like that to DbGeography on reading Json from the request, the code below is what you're after
public class DbGeographyConverter : JsonConverter
{
public override bool CanConvert ( Type objectType )
{
return objectType.IsAssignableFrom( typeof( string ) );
}
public override object ReadJson ( JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer )
{
if ( reader.Value == null ) {
return null;
}
return Parser.ToDbGeography( reader.Value.ToString() );
}
public override void WriteJson ( JsonWriter writer, object value, JsonSerializer serializer )
{
// Base serialization is fine
serializer.Serialize( writer, value );
}
}
This is the code for the Converter if you're passing in a string value - 12,99
Which will be you're lat and lng
This sample application has the answer you're looking for.
https://code.msdn.microsoft.com/windowsazure/HTML-ASPNET-Web-API-Bing-58c97f9f
The Converter code from this can be found here
https://github.com/Azure-Samples/SQLDatabase-Spatial-WebAPI-BingMaps/blob/master/SpatialTypesWithWebAPI/Models/DbGeographyConverter.cs