Is there a way to make DataContractJsonSerializer emit the "odata.type" field required when posting an OData entity into a collection that supports multiple entity types (hierarchy per table)?
If I construct DataContractJsonSerializer with a settings object with EmitTypeInformation set to Always, it emits a "__type" field in the output, but that's not the field name needed for OData and the format of the value is wrong as well.
Is there any way to hook into the DataContractJsonSerializer pipeline to inject the desired "odata.type" field into the serialization output?
It would be such a hack to have to parse the serialization output in order to inject the field. How does WCF Data Services do it? Not using DataContractJsonSerializer is my guess.
Have you considered using Json.Net? Json.Net is much more extensible and the scenario that you have can be done using a custom resolver. sample code
class Program
{
static void Main(string[] args)
{
Console.WriteLine(
JsonConvert.SerializeObject(new Customer { Name = "Raghu" }, new JsonSerializerSettings
{
ContractResolver = new CustomContractResolver()
}));
}
}
public class CustomContractResolver : DefaultContractResolver
{
protected override JsonObjectContract CreateObjectContract(Type objectType)
{
JsonObjectContract objectContract = base.CreateObjectContract(objectType);
objectContract.Properties.Add(new JsonProperty
{
PropertyName = "odata.type",
PropertyType = typeof(string),
ValueProvider = new StaticValueProvider(objectType.FullName),
Readable = true
});
return objectContract;
}
private class StaticValueProvider : IValueProvider
{
private readonly object _value;
public StaticValueProvider(object value)
{
_value = value;
}
public object GetValue(object target)
{
return _value;
}
public void SetValue(object target, object value)
{
throw new NotSupportedException();
}
}
}
public class Customer
{
public string Name { get; set; }
}
I can't answer your first two questions, but for the third question, I found on the OData Team blog a link to the OData WCF Data Services V4 library open source code. Downloading that code, you will see that they perform all serialization and deserialization manually. They have 68 files in their two Json folders! And looking through the code they have comments such as:
// This is a work around, needTypeOnWire always = true for client side:
// ClientEdmModel's reflection can't know a property is open type even if it is, so here
// make client side always write 'odata.type' for enum.
So that to me kind of implies there is no easy, clean, simple, elegant way to do it.
I tried using a JavaScriptConverter, a dynamic type, and other stuff, but most of them ended up resorting to using Reflection which just made for a much more complicated solution versus just using a string manipulation approach.
Related
I want to support partial updates with JSON Merge Patch. The domain model is based on the always valid concept and has no public setters. Therefore I can't just apply the changes to the class types. I need to translate the changes to the specific commands.
Since classes have nullable properties I need to be able to distinguish between properties set to null and not provided.
I'm aware of JSON Patch. I could use patch.JsonPatchDocument.Operations to go through the list of changes. JSON Patch is just verbose and more difficult for the client. JSON Patch requires to use Newtonsoft.Json (Microsoft states an option to change Startup.ConfigureServices to only use Newtonsoft.Json for JSON Patch (https://learn.microsoft.com/en-us/aspnet/core/web-api/jsonpatch?view=aspnetcore-6.0).
Newtonsoft supports IsSpecified-Properties that can be used as a solution for JSON Merge Patch in the DTO classes (How to make Json.NET set IsSpecified properties for properties with complex values?). This would solve the problem, but again requires Newtonsoft. System.Text.Json does not support this feature. There is an open issue for 2 years (https://github.com/dotnet/runtime/issues/40395), but nothing to expect.
There is a post that describes a solution with a custom JsonConverter for Web API (https://github.com/dotnet/runtime/issues/40395). Would this solution still be usable for NetCore?
I was wondering if there is an option to access the raw json or a json object inside the controller method after the DTO object was filled. Then I could manually check if a property was set. Web Api closes the stream, so I can't access the body anymore. It seems there are ways to change that behavior (https://gunnarpeipman.com/aspnet-core-request-body/#comments). It seems quite complicated and feels like a gun that is too big. I also don't understand what changes were made for NetCore 6.
I'm surpised that such a basic problem needs one to jump through so many loops. Is there an easy way to accomplish my goal with System.Text.Json and NetCore 6? Are there other options? Would using Newtonsoft have any other bad side effects?
With the helpful comments of jhmckimm I found
Custom JSON serializer for optional property with System.Text.Json. DBC shows a fantastic solution using Text.Json and Optional<T>. This should be in the Microsoft docs!
In Startup I added:
services.AddControllers()
.AddJsonOptions(o => o.JsonSerializerOptions.DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingDefault)
.AddJsonOptions(o => o.JsonSerializerOptions.Converters.Add(new OptionalConverter()));
Since we use <Nullable>enable</Nullable> and <WarningsAsErrors>nullable</WarningsAsErrors> I adapted the code for nullables.
public readonly struct Optional<T>
{
public Optional(T? value)
{
this.HasValue = true;
this.Value = value;
}
public bool HasValue { get; }
public T? Value { get; }
public static implicit operator Optional<T>(T value) => new Optional<T>(value);
public override string ToString() => this.HasValue ? (this.Value?.ToString() ?? "null") : "unspecified";
}
public class OptionalConverter : JsonConverterFactory
{
public override bool CanConvert(Type typeToConvert)
{
if (!typeToConvert.IsGenericType) { return false; }
if (typeToConvert.GetGenericTypeDefinition() != typeof(Optional<>)) { return false; }
return true;
}
public override JsonConverter CreateConverter(Type typeToConvert, JsonSerializerOptions options)
{
Type valueType = typeToConvert.GetGenericArguments()[0];
return (JsonConverter)Activator.CreateInstance(
type: typeof(OptionalConverterInner<>).MakeGenericType(new Type[] { valueType }),
bindingAttr: BindingFlags.Instance | BindingFlags.Public,
binder: null,
args: null,
culture: null
)!;
}
private class OptionalConverterInner<T> : JsonConverter<Optional<T>>
{
public override Optional<T> Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
T? value = JsonSerializer.Deserialize<T>(ref reader, options);
return new Optional<T>(value);
}
public override void Write(Utf8JsonWriter writer, Optional<T> value, JsonSerializerOptions options) =>
JsonSerializer.Serialize(writer, value.Value, options);
}
}
My test DTO looks like this:
public class PatchGroupDTO
{
public Optional<Guid?> SalesGroupId { get; init; }
public Optional<Guid?> AccountId { get; init; }
public Optional<string?> Name { get; init; }
public Optional<DateTime?> Start { get; init; }
public Optional<DateTime?> End { get; init; }
}
I can now access the fields and check with .HasValue if the value was set. It also works for writing and allows us to stripe fields based on permission.
Given a Base64 string, the following sample class will deserialize properly using Newtonsoft.Json, but not with System.Text.Json:
using System;
using System.Text.Json.Serialization;
public class AvatarImage{
public Byte[] Data { get; set; } = null;
public AvatarImage() {
}
[JsonConstructor]
public AvatarImage(String Data) {
//Remove Base64 header info, leaving only the data block and convert it to a Byte array
this.Data = Convert.FromBase64String(Data.Remove(0, Data.IndexOf(',') + 1));
}
}
With System.Text.Json, the following exception is thrown:
must bind to an object property or field on deserialization. Each parameter name must match with a property or field on the object. The match can be case-insensitive.
Apparently System.Text.Json doesn't like the fact the property is a Byte[] but the parameter is a String, which shouldn't really matter because the whole point is that the constructor should be taking care of the assignments.
Is there any way to get this working with System.Text.Json?
In my particular case Base64 images are being sent to a WebAPI controller, but the final object only needs the Byte[]. In Newtonsoft this was a quick and clean solution.
This is apparently a known restriction of System.Text.Json. See the issues:
[JsonSerializer] Relax restrictions on ctor param type to immutable property type matching where reasonable #44428 which is currently labeled with the 6.0.0 Future milestone.
System.Text.Json incorrectly requires construct parameter types to match immutable property types. #47422
JsonConstrutor different behavior between Newtonsoft.Json and System.Text.Json #46480.
Thus (in .Net 5 at least) you will need to refactor your class to avoid the limitation.
One solution would be to add a surrogate Base64 encoded string property:
public class AvatarImage
{
[JsonIgnore]
public Byte[] Data { get; set; } = null;
[JsonInclude]
[JsonPropertyName("Data")]
public string Base64Data
{
private get => Data == null ? null : Convert.ToBase64String(Data);
set
{
var index = value.IndexOf(',');
this.Data = Convert.FromBase64String(index < 0 ? value : value.Remove(0, index + 1));
}
}
}
Note that, ordinarily, JsonSerializer will only serialize public properties. However, if you mark a property with [JsonInclude] then either the setter or the getter -- but not both -- can be nonpublic. (I have no idea why Microsoft doesn't allow both to be private, the data contract serializers certainly support private members marked with [DataMember].) In this case I chose to make the getter private to reduce the chance the surrogate property is serialized by some other serializer or displayed via some property browser.
Demo fiddle #1 here.
Alternatively, you could introduce a custom JsonConverter<T> for AvatarImage
[JsonConverter(typeof(AvatarConverter))]
public class AvatarImage
{
public Byte[] Data { get; set; } = null;
}
class AvatarConverter : JsonConverter<AvatarImage>
{
class AvatarDTO { public string Data { get; set; } }
public override AvatarImage Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
{
var dto = JsonSerializer.Deserialize<AvatarDTO>(ref reader, options);
var index = dto.Data?.IndexOf(',') ?? -1;
return new AvatarImage { Data = dto.Data == null ? null : Convert.FromBase64String(index < 0 ? dto.Data : dto.Data.Remove(0, index + 1)) };
}
public override void Write(Utf8JsonWriter writer, AvatarImage value, JsonSerializerOptions options) =>
JsonSerializer.Serialize(writer, new { Data = value.Data }, options);
}
This seems to be the easier solution if for simple models, but can become a nuisance for complex models or models to which properties are frequently added.
Demo fiddle #2 here.
Finally, it seems a bit unfortunate that the Data property will have some extra header prepended during deserialization that is not present during serialization. Rather than fixing this during deserialization, consider modifying your architecture to avoid mangling the Data string in the first place.
Implementing the custom converter deserialization using an ExpandoObject can avoid the nested DTO class if desired:
using System.Dynamic;
.
.
.
public override FileEntity Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) {
dynamic obj = JsonSerializer.Deserialize<ExpandoObject>(ref reader, options);
return new FileEntity {
Data = (obj.data == null) ? null : Convert.FromBase64String(obj.data.GetString().Remove(0, obj.data.GetString().IndexOf(',') + 1))
};
}
it makes the custom converter a little more flexible when developing, since the DTO doesn't continually need to grow with the base class being deserialized into. It also makes handling potential nullable properties a bit easier too (over standard JsonElement deserialization, i.e. JsonSerializer.Deserialize< JsonElement >) as such:
public override FileEntity Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options) {
dynamic obj = JsonSerializer.Deserialize<ExpandoObject>(ref reader, options);
return new FileEntity {
SomeNullableInt32Property = obj.id?.GetInt32(),
Data = (obj.data?.GetString() == null) ? null : Convert.FromBase64String(obj.data.GetString().Remove(0, obj.data.GetString().IndexOf(',') + 1))
};
}
I need to replace the DateTime serialization for JSON in WCF REST Self Hosted service. Right now, I'm using something like the following code to do it, but it's definitely not the way to go since it requires manipulating each class.
[DataContract]
public class Test
{
[IgnoreDataMember]
public DateTime StartDate;
[DataMember(Name = "StartDate")]
public string StartDateStr
{
get { return DateUtil.DateToStr(StartDate); }
set { StartDate = DateTime.Parse(value); }
}
}
where my utility function DateUtil.DateToStr does all the formatting work.
Is there any easy way to do it without having to touch the attributes on my classes which have the DataContract attribute? Ideally, there would be no attributes, but a couple of lines of code in my configuration to replace the serializer with one where I've overridden DateTime serialization.
Everything that I've found looks like I have to replace huge pieces of the pipeline.
This article doesn't appear to apply because in I'm using WebServiceHost not HttpServiceHost, which not part of the 4.5.1 Framework.
JSON.NET Serializer for WCF REST Services
By default WCF uses DataContractJsonSerializer to serialize data into JSON. Unfortunatelly date from this serializer is in very difficult format to parse by human brain.
"DateTime": "\/Date(1535481994306+0200)\/"
To override this behavior we need to write custom IDispatchMessageFormatter. This class will receive all data which should be returned to requester and change it according to our needs.
To make it happen to the operations in the endpoint add custom formatter - ClientJsonDateFormatter:
ServiceHost host=new ServiceHost(typeof(CustomService));
host.AddServiceEndpoint(typeof(ICustomContract), new WebHttpBinding(), Consts.WebHttpAddress);
foreach (var endpoint in host.Description.Endpoints)
{
if (endpoint.Address.Uri.Scheme.StartsWith("http"))
{
foreach (var operation in endpoint.Contract.Operations)
{
operation.OperationBehaviors.Add(new ClientJsonDateFormatter());
}
endpoint.Behaviors.Add(new WebHttpBehavior());
}
}
ClientJsonDateFormatter is simple class which just applies formatter ClientJsonDateFormatter
public class ClientJsonDateFormatter : IOperationBehavior
{
public void AddBindingParameters(OperationDescription operationDescription, BindingParameterCollection bindingParameters) { }
public void ApplyClientBehavior(OperationDescription operationDescription, ClientOperation clientOperation) { }
public void ApplyDispatchBehavior(OperationDescription operationDescription, DispatchOperation dispatchOperation)
{
dispatchOperation.Formatter = new ResponseJsonFormatter(operationDescription);
}
public void Validate(OperationDescription operationDescription) { }
}
In the formatter we took imput and serialize it with the changed Serializer:
public class ResponseJsonFormatter : IDispatchMessageFormatter
{
OperationDescription Operation;
public ResponseJsonFormatter(OperationDescription operation)
{
this.Operation = operation;
}
public void DeserializeRequest(Message message, object[] parameters)
{
}
public Message SerializeReply(MessageVersion messageVersion, object[] parameters, object result)
{
string json=Newtonsoft.Json.JsonConvert.SerializeObject(result);
byte[] bytes = Encoding.UTF8.GetBytes(json);
Message replyMessage = Message.CreateMessage(messageVersion, Operation.Messages[1].Action, new RawDataWriter(bytes));
replyMessage.Properties.Add(WebBodyFormatMessageProperty.Name, new WebBodyFormatMessageProperty(WebContentFormat.Raw));
return replyMessage;
}
}
And to send information to client we need data writer - RawDataWriter. Its implementation is simple:
class RawDataWriter : BodyWriter
{
byte[] data;
public RawDataWriter(byte[] data)
: base(true)
{
this.data = data;
}
protected override void OnWriteBodyContents(XmlDictionaryWriter writer)
{
writer.WriteStartElement("Binary");
writer.WriteBase64(data, 0, data.Length);
writer.WriteEndElement();
}
}
Applying all code will result in returning date in more friendly format:
"DateTime":"2018-08-28T20:56:48.6411976+02:00"
To show it in practice I created example in the github branch DateTimeFormatter.
Please check also this answer as very likely you also will need it.
There is a limitation in JSON to convert DateTime, specially according to your case.
Please see http://msdn.microsoft.com/en-us/library/bb412170(v=vs.110).aspx
and read the section Dates/Times and JSON
To resolve this problem, I simply changed the type of serialization from JSON to XML for all the calls including DateTime.
After long time discussion ,I have find out the solution for it.
Please Use the following Code to Solve serialized date..
[IgnoreDataMember]
public DateTime? PerformanceDate { get; set; }
[DataMember(EmitDefaultValue = false, Name = "PerformanceDate")]
public string UpdateStartDateStr
{
get
{
if (this.PerformanceDate.HasValue)
return this.PerformanceDate.Value.ToUniversalTime().ToString("s", CultureInfo.InvariantCulture);
else
return null;
}
set
{
// should implement this...
}
}
Ok - I've been beating my head against this for a few of hours now. Time to ask for help.
I have just upgraded my Web application project to ASP.NET MVC 4 RC, and the new WebApi.
My web api method is now returning EMPTY json "{}" - even though my object is fully populated.
I have replace the serializer with my own MediaTypeFormatter that also calls the Newtonsoft Json serializer, just so I can hook in and see things working.
What I see is an object going in to the serializer, and coming out as "{}".
This USED to work before I upgraded.
This is my object
[Serializable]
public class Parameters
{
public string ApplicantName { get; set; }
}
And I am just calling:
var result = JsonConvert.SerializeObject(new Parameters(){ Name = "test" });
I get back
"{}"
Whats going on??
[EDIT]
Someone else having the same problem... after running through the Newtonsoft source code, I can see we're having the exact same problem from a recent change.
http://json.codeplex.com/discussions/357850
Ok - there have been numerous changes, which result is some pretty radical changes to the Json output. These changes also include how custom TypeConverters are applied.
I have written a basic resolver which (for us at least) causes the Newtonsoft serializer to behave more like a basic Serializable object serializer - i.e. serializes all PROPERTIES, and doesnt use custom TypeConverters...
/// <summary>
/// A resolver that will serialize all properties, and ignore custom TypeConverter attributes.
/// </summary>
public class SerializableContractResolver : Newtonsoft.Json.Serialization.DefaultContractResolver
{
protected override IList<Newtonsoft.Json.Serialization.JsonProperty> CreateProperties(Type type, MemberSerialization memberSerialization)
{
var properties = base.CreateProperties(type, memberSerialization);
foreach (var p in properties)
p.Ignored = false;
return properties;
}
protected override Newtonsoft.Json.Serialization.JsonContract CreateContract(Type objectType)
{
var contract = base.CreateContract(objectType);
if (contract is Newtonsoft.Json.Serialization.JsonStringContract)
return CreateObjectContract(objectType);
return contract;
}
}
* REGISTRATION *
In your MvcApplication "Application_Start"...
GlobalConfiguration.Configuration.Formatters
.JsonFormatter.SerializerSettings.ContractResolver =
new SerializableContractResolver()
{
IgnoreSerializableAttribute = true
};
recently I have set up a WCF restful service with EF4.
It all worked out when returning XML format response. however when it comes to JSON, I got 504 Error. unable to return json data, WCF Resful Service .NET 4.0
By digging deeper by using Service Trace Viewer:
I found this error:
'The type 'xxx.DataEntity.AppView'
cannot be serialized to JSON because
its IsReference setting is 'True'. The
JSON format does not support
references because there is no
standardized format for representing
references. To enable serialization,
disable the IsReference setting on the
type or an appropriate parent class of
the type.'
The "AppView" is a complex object class which generated by EF4 from a store procedure.
I spend quite a bit time google how to disable the IsReference, very little result so far.
anyone? with any solutions?
thanks in advance
Code:
[OperationContract]
[WebInvoke(Method = "GET",
BodyStyle = WebMessageBodyStyle.Wrapped,
UriTemplate = "App/{id}/{format}")]
AppView FuncDetail(string id, string format);
public AppView FuncDetail(string id, string format)
{
SetResponseFormat(format);
return AppSvcs.GetById(id);
}
private void SetResponseFormat(string format)
{
if (format.ToLower() == "json")
{
ResponseContext.Format = WebMessageFormat.Json;
}
else
{
ResponseContext.Format = WebMessageFormat.Xml;
}
}
I ran into exactly the same issue. It only happened on one of my service methods where I was trying to return JSON serialised Entity objects. For all my other methods I was returning JSON serialised data transfer objects (DTOs), which are stand-alone and not connected to the Entity framework. I am using DTOs for data posted into methods. Often, the data you send out does not need all the data you store in the model or the database e.g. ID values, updated dates, etc. The mapping is done in the model class, like so:
public partial class Location
{
public static LocationDto CreateLocationDto(Location location)
{
LocationDto dto = new LocationDto
{
Accuracy = location.Accuracy,
Altitude = location.Altitude,
Bearing = location.Bearing
};
return dto;
}
It may seem a bit clunky but it works and it ensures that you only send the data fields you intended to send back. It works for me because I only have 5 or 6 entities but I can see that it would get a bit tedious if you have lots of classes.
I was running into the same problem, as caused by using the auto-generated ADO Entity Models. I have not found a direct fix for this issue, but as a work around, I serialize the response as json explicitly.
So in your example, the AppView FuncDetail looks like this:
public object FuncDetail(string id, string format)
{
SetResponseFormat(format);
// where AppSvc is the object type and the enumerable list of this type is returned by the GetById method, cast it to a json string
return JSONSerializer.ToJson<AppSvc>(AppSvcs.GetById(id));
}
Here are the Serializers that I'm using:
public static class GenericSerializer
{
public static DataTable ToDataTable<T>(IEnumerable<T> varlist)
{
DataTable dtReturn = new DataTable();
// column names
PropertyInfo[] oProps = null;
if (varlist == null) return dtReturn;
foreach (T rec in varlist)
{
// Use reflection to get property names, to create table, Only first time, others will follow
if (oProps == null)
{
oProps = ((Type)rec.GetType()).GetProperties();
foreach (PropertyInfo pi in oProps)
{
Type colType = pi.PropertyType;
if ((colType.IsGenericType) && (colType.GetGenericTypeDefinition()
== typeof(Nullable<>)))
{
colType = colType.GetGenericArguments()[0];
}
dtReturn.Columns.Add(new DataColumn(pi.Name, colType));
}
}
DataRow dr = dtReturn.NewRow();
foreach (PropertyInfo pi in oProps)
{
dr[pi.Name] = pi.GetValue(rec, null) == null ? DBNull.Value : pi.GetValue
(rec, null);
}
dtReturn.Rows.Add(dr);
}
return dtReturn;
}
}
public static class JSONSerializer
{
public static string ToJson<T>(IEnumerable<T> varlist)
{
DataTable dtReturn = GenericSerializer.ToDataTable(varlist);
return GetJSONString(dtReturn);
}
static object RowsToDictionary(this DataTable table)
{
var columns = table.Columns.Cast<DataColumn>().ToArray();
return table.Rows.Cast<DataRow>().Select(r => columns.ToDictionary(c => c.ColumnName, c => r[c]));
}
static Dictionary<string, object> ToDictionary(this DataTable table)
{
return new Dictionary<string, object>
{
{ table.TableName, table.RowsToDictionary() }
};
}
static Dictionary<string, object> ToDictionary(this DataSet data)
{
return data.Tables.Cast<DataTable>().ToDictionary(t => "Table", t => t.RowsToDictionary());
}
public static string GetJSONString(DataTable table)
{
JavaScriptSerializer serializer = new JavaScriptSerializer();
return serializer.Serialize(table.ToDictionary());
}
public static string GetJSONString(DataSet data)
{
JavaScriptSerializer serializer = new JavaScriptSerializer();
return serializer.Serialize(data.ToDictionary());
}}
It is a lot clearer to use Entity Metadata instead of Reflection.
The Metadata is pretty extensive.
another way to do this is to use LINQ to create an anonymous type with the subset of fields that you need from your entity and then use JSON.NET to serialize the collection of anon types that you created in the LINQ statement. then persist that collection out as a string by serializing.