I am having some problems with create with JSON.Net. When I try to parse it, it gives me following error:
Additional text encountered after finished reading JSON content:
I tried validating it with http://json.parser.online.fr/ and it says "SyntaxError: Unexpected token ,".
My JSON is as below:
{"StaffID":"S01","StaffRank":"Manager"},{"StaffID":"S02","StaffRank":"Waiter"}
How to deserialize it?
You need to surround that with square brackets, which denotes that it's an array:
[{"StaffID":"S01","StaffRank":"Manager"},{"StaffID":"S02","StaffRank":"Waiter"}]
As of Release 11.0.1, Json.NET now natively supports parsing comma-delimited JSON in the same way it supports parsing newline delimited JSON:
New feature - Added support for reading multiple comma delimited values with JsonReader.SupportMultipleContent.
Thus the answer to Line delimited json serializing and de-serializing by Yuval Itzchakov should work here also. Define an extension method:
public static partial class JsonExtensions
{
public static IEnumerable<T> FromDelimitedJson<T>(TextReader reader, JsonSerializerSettings settings = null)
{
using (var jsonReader = new JsonTextReader(reader) { CloseInput = false, SupportMultipleContent = true })
{
var serializer = JsonSerializer.CreateDefault(settings);
while (jsonReader.Read())
{
if (jsonReader.TokenType == JsonToken.Comment)
continue;
yield return serializer.Deserialize<T>(jsonReader);
}
}
}
}
Then, given a data model created to hold an individual item in the comma-separated list such as:
public class RootObject
{
public string StaffID { get; set; }
public string StaffRank { get; set; }
}
You can deserialize the JSON string shown like so:
var jsonString = #"{""StaffID"":""S01"",""StaffRank"":""Manager""},{""StaffID"":""S02"",""StaffRank"":""Waiter""}";
var list = JsonExtensions.FromDelimitedJson<RootObject>(new StringReader(jsonString)).ToList();
This approach may be preferable when deserializing a very large sequence of comma-delimited objects from a large file, because it is not necessary to load the entire file into a string then add '[' and ']' to the beginning and end. In Performance Tips: Optimize Memory Usage Newtonsoft recommends deserializing large files directly from a stream, so instead a StreamReader can be passed into JsonExtensions.FromDelimitedJson() which will then stream through the file deserializing each object separately.
Related
I've got following setup: C#, ServiceStack, MariaDB, POCOs with objects and structs, JSON.
The main question is: how to use ServiceStack to store POCOs to MariaDB having complex types (objects and structs) blobbed as JSON and still have working de/serialization of the same POCOs? All of these single tasks are supported, but I had problems when all put together mainly because of structs.
... finally during writing this I found some solution and it may look like I answered my own question, but I still would like to know the answer from more skilled people, because the solution I found is a little bit complicated, I think. Details and two subquestions arise later in the context.
Sorry for the length and for possible misinformation caused by my limited knowledge.
Simple example. This is the final working one I ended with. At the beginning there were no SomeStruct.ToString()/Parse() methods and no JsConfig settings.
using Newtonsoft.Json;
using ServiceStack;
using ServiceStack.DataAnnotations;
using ServiceStack.OrmLite;
using ServiceStack.Text;
using System.Diagnostics;
namespace Test
{
public class MainObject
{
public int Id { get; set; }
public string StringProp { get; set; }
public SomeObject ObjectProp { get; set; }
public SomeStruct StructProp { get; set; }
}
public class SomeObject
{
public string StringProp { get; set; }
}
public struct SomeStruct
{
public string StringProp { get; set; }
public override string ToString()
{
// Unable to use .ToJson() here (ServiceStack does not serialize structs).
// Unable to use ServiceStack's JSON.stringify here because it just takes ToString() => stack overflow.
// => Therefore Newtonsoft.Json used.
var serializedStruct = JsonConvert.SerializeObject(this);
return serializedStruct;
}
public static SomeStruct Parse(string json)
{
// This method behaves differently for just deserialization or when part of Save().
// Details in the text.
// After playing with different options of altering the json input I ended with just taking what comes.
// After all it is not necessary, but maybe useful in other situations.
var structItem = JsonConvert.DeserializeObject<SomeStruct>(json);
return structItem;
}
}
internal class ServiceStackMariaDbStructTest
{
private readonly MainObject _mainObject = new MainObject
{
ObjectProp = new SomeObject { StringProp = "SomeObject's String" },
StringProp = "MainObject's String",
StructProp = new SomeStruct { StringProp = "SomeStruct's String" }
};
public ServiceStackMariaDbStructTest()
{
// This one line is needed to store complex types as blobbed JSON in MariaDB.
MySqlDialect.Provider.StringSerializer = new JsonStringSerializer();
JsConfig<SomeStruct>.RawSerializeFn = someStruct => JsonConvert.SerializeObject(someStruct);
JsConfig<SomeStruct>.RawDeserializeFn = json => JsonConvert.DeserializeObject<SomeStruct>(json);
}
public void Test_Serialization()
{
try
{
var json = _mainObject.ToJson();
if (!string.IsNullOrEmpty(json))
{
var objBack = json.FromJson<MainObject>();
}
}
catch (System.Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
public void Test_Save()
{
var cs = "ConnectionStringToMariaDB";
var dbf = new OrmLiteConnectionFactory(cs, MySqlDialect.Provider);
using var db = dbf.OpenDbConnection();
db.DropAndCreateTable<MainObject>();
try
{
db.Save(_mainObject);
var dbObject = db.SingleById<MainObject>(_mainObject.Id);
}
catch (System.Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
}
}
What (I think) I know / have tried but at first didn't help to solve it myself:
ServiceStack stores complex types in DB as blobbed JSV by default (last paragraph of first section: https://github.com/ServiceStack/ServiceStack.OrmLite), so it is necessary to set it the way it is proposed: MySqlDialect.Provider.StringSerializer = new JsonStringSerializer(); (https://github.com/ServiceStack/ServiceStack.OrmLite#pluggable-complex-type-serializers)=> default JSV changed to JSON.
the ServiceStack's serialization does not work with structs, it is necessary to treat them special way:
a) according to https://github.com/ServiceStack/ServiceStack.Text#c-structs-and-value-types and example https://github.com/ServiceStack/ServiceStack.Text/#using-structs-to-customize-json it is necessary to implement TStruct.ToString() and static TStruct.ParseJson()/ParseJsv() methods.
b) according to https://github.com/ServiceStack/ServiceStack.Text/#typeserializer-details-jsv-format and unit tests https://github.com/ServiceStack/ServiceStack.Text/blob/master/tests/ServiceStack.Text.Tests/CustomStructTests.cs it shall be TStruct.ToString() (the same as in a) and static TStruct.Parse().
Subquestion #1: which one is the right one? For me, ParseJson() was never called, Parse() was. Documentation issue or is it used in other situation?
I implemented option b). Results:
IDbConnection.Save(_mainObject) saved the item to MariaDB. Success.
Through the saving process ToString() and Parse() were called. In Parse, incoming JSON looked this way:
"{\"StringProp\":\"SomeStruct's String\"}". Fine.
Serialization worked. Success.
Deserialization failed. I don't know the reason, but JSON incoming to Parse() was "double-escaped":
"{\\\"StringProp\\\":\\\"SomeStruct's String\\\"}"
Subquestion #2: Why the "double-escaping" in Parse on deserialization?
I tried to solve structs with JsConfig (and Newtonsoft.Json to get proper JSON):
JsConfig<SomeStruct>.SerializeFn = someStruct => JsonConvert.SerializeObject(someStruct);
JsConfig<SomeStruct>.DeSerializeFn = json => JsonConvert.DeserializeObject<SomeStruct>(json);
a) at first without ToString() and Parse() defined in the TStruct. Results:
Save failed: the json input in JsonConvert.DeserializeObject(json) that is used during Save was just type name "WinAmbPrototype.SomeStruct".
De/serialization worked.
b) then I implemented ToString() also using Newtonsoft.Json. During Save ToString() was used instead of JsConfig.SerializeFn even the JsConfig.SerializeFn was still set (maybe by design, I do not judge). Results:
Save failed: but the json input of DeserializeFn called during Save changed, now it was JSV-like "{StringProp:SomeStruct's String}", but still not deserializable as JSON.
De/serialization worked.
Then (during writing this I was still without any solution) I found JsConfig.Raw* "overrides" and tried them:
JsConfig<SomeStruct>.RawSerializeFn = someStruct => JsonConvert.SerializeObject(someStruct);
JsConfig<SomeStruct>.RawDeserializeFn = json => JsonConvert.DeserializeObject<SomeStruct>(json);
a) at first without ToString() and Parse() defined in the TStruct. Results are the same as in 2a.
b) then I implemented ToString(). Results:
BOTH WORKED. No Parse() method needed for this task.
But it is very fragile setup:
if I removed ToString(), it failed (now I understand why, default ToString produced JSON with just type name in 2a, 3a).
if I removed RawSerializeFn setting, it failed in RawDeserializeFn ("double-escaped" JSON).
Is there some simpler solution? I would be very glad if someone points me to better direction.
Acceptable would be maybe two (both of them accessible because of different circumstances):
if I am the TStruct owner: with just pure TStruct.ToString() and static TStruct.Parse() to support out of the box de/serialization and DB by ServiceStack (without different input in Parse()).
if I am a consumer of TStruct with no JSON support implemented and I am without access to its code: until now I did not find the way, if the ToString is not implemented: Save to DB did not work. Maybe would be fine to ensure JsConfig serialize functions are enough for both de/serialization and when used during saving to DB.
And the best one would be without employing other dependency (e.g. Newtonsoft.Json) to serialize structs. Maybe some JsConfig.ShallProcessStructs = true; (WARNING: just a tip, not working as of 2021-04-02) would be fine for such situations.
ServiceStack treats structs like a single scalar value type, just like most of the core BCL Value Types (e.g. TimeSpan, DateTime, etc). Overloading the Parse() and ToString() methods and Struct's Constructor let you control the serialization/deserialization of custom structs.
Docs have been corrected. Structs use Parse whilst classes use ParseJson/ParseJsv
If you want to serialize a models properties I'd suggest you use a class instead as the behavior you're looking for is that of a POCO DTO.
If you want to have structs serailized as DTOs in your RDBMS an alternative you can try is to just use JSON.NET for the complex type serialization, e.g:
public class JsonNetStringSerializer : IStringSerializer
{
public To DeserializeFromString<To>(string serializedText) =>
JsonConvert.DeserializeObject<To>(serializedText);
public object DeserializeFromString(string serializedText, Type type) =>
JsonConvert.DeserializeObject(serializedText, type);
public string SerializeToString<TFrom>(TFrom from) =>
JsonConvert.SerializeObject(from);
}
MySqlDialect.Provider.StringSerializer = new JsonNetStringSerializer();
I'm developing a Rest Client using Spring Boot and Spring Framework (spring-boot-starter-parent 2.1.6.RELEASE)
I have a class representing a response object as shown below:
public class ValidateResponse {
private String ResponseCode;
private String ResponseDesc;
//getters and setters
//constructors using fields
//empty constructor
}
I'm creating a web-hook for an external api and I need to return a JSON object to for a specific endpoint (the JSON object properties must start with uppercase(s)). I'm calling returning the object from a PostMapping method nested in a RequestMapping root path:
#PostMapping("hooks/validate")
public ValidateResponse responseObj(#RequestHeader Map<String, String> headersObj) {
ValidateResponse response = new ValidateResponse("000000", "Success");
logger.info("Endpoint = hooks/validate | Request Headers = {}", headersObj);
return response;
}
However, when I hit the endpoint from postman I'm getting duplicate varialbes
{
"ResponseCode": "000000",
"ResponseDesc": "Success",
"responseCode": "000000",
"responseDesc": "Success"
}
I understand that the pojo-json conversion is handled by spring but I don't understand why the conversion is yielding duplicate variables.
Note: I know the ResponseDesc and the ResponseCode are not declared using the best standards for naming variables (camelCasing).
I've done some digging and according to the Java Language Specification
An identifier is an unlimited-length sequence of Java letters and Java digits, the first of which must be a Java letter.
and
The "Java letters" include uppercase and lowercase ASCII Latin letters A-Z (\u0041-\u005a), and a-z (\u0061-\u007a), and, for historical reasons, the ASCII underscore (_, or \u005f) and dollar sign ($, or \u0024). The $ character should be used only in mechanically generated source code or, rarely, to access pre-existing names on legacy systems.
So, I'm assuming its syntactically correct to define a variable using the Camelcase format [Need clarification on this].
I'm considering having to create the JSON object manually but I'd like to know the cause of this behaviour first. Any pointers are appreciated.
Jackson deserializes all the public fields that it comes across. However if you want Jackson to return the response in your expected element names (in your case elements starting with capital letters), make the fields private and annotate them with the #JsonProperty(expected_name_here). Your class file will typically looks as shown below
public class ValidateResponse {
#JsonProperty("ResponseDesc")
private String responseCode;
#JsonProperty("ResponseDesc")
private String responseDesc;
//getters and setters
//constructors using fields
//empty constructor
}
Note: The getters and setters for these fields should be public, otherwise Jackson won't see anything to deserialize in the class.
public class ValidateResponse {
#JsonProperty("ResponseDesc")
public String responseCode;
#JsonProperty("ResponseDesc")
public String responseDesc;
//getters and setters
//constructors using fields
//empty constructor
}
This must fix your problem, however I do not know the reason as it requires deep Jackson investigation.
EDIT
I found out the reason.
The field got duplicated because in you case you had:
2 public fields named in upper case -> they are to be processed by jackson
2 getters getResponseCode and getResponseDesc -> they are to be resolved
as accessors for properties responseCode and responseDesc accordingly.
Summing this up - you have 4 properties resolved by Jackson. Simply making your fields private will resolve your issue, however I still advise using JsonProperty approach.
I added a com.google.code.gson dependency in the projects pom.xml file to configure Spring Boot to use Gson (instead of the default jackson).
The Json object returned from the hooks/validate endpoint must have its property names starting with a capital letter. Using a java class to generate the response object was resulting to camelCased property names so I resolved to create the JSON response object manually. Here's the code for creating the custom JSON object:
public ResponseEntity<String> responseObj(#RequestHeader Map<String, String> headersObj) {
HttpHeaders responseHeaders = new HttpHeaders();
responseHeaders.setContentType(MediaType.APPLICATION_JSON);
JsonObject response = new JsonObject();
response.addProperty("ResponseCode", "00000000");
response.addProperty("ResponseDesc" , "Success");
logger.info("Endpoint = hooks/validate | Request Headers = {}", headersObj);
return ResponseEntity.ok().headers(responseHeaders).body(response.toString());
}
Note The JSON object is returned as a String so the response from the endpoint must have an additional header to define MediaType to inform the calling system that the response is in JSON format:
responseHeaders.setContentType(MediaType.APPLICATION_JSON);
then add the header to the response:
return ResponseEntity.ok().headers(responseHeaders).body(response.toString());
I am trying to serialize a string that is returned from a http response and I am using netstandard1.0. Not a lot of serializing functions work in this framework, but I finally found a working function. Here is my code so far:
HttpResponseMessage Response = // initialized else where
var jsonTask = Response.Content.ReadAsStringAsync();
if (!jsonTask.IsCompleted) jsonTask.RunSynchronously();
string json = jsonTask.Result;
Data = JsonConvert.DeserializeObject<MyModel>(json);
However this does not deserialize I get from the http response. It throws an error that the DeserializeObject function is looking for a different format. When I run Result.Content.ReadAsStringAsync(), I get the result in the following format.
"[{\"key\":\"Password\",\"errors\":[\"The Password field is required.\"]},{\"key\":\"UserName\",\"errors\":[\"The UserName field is required.\"]},{\"key\":\"OrganizationUserName\",\"errors\":[\"The OrganizationUserName field is required.\"]}]"
Does anyone know how to deserialize this format?
If you define your MyModel as follows:
public class MyModel
{
public string key { get; set; }
public List<string> errors { get; set; }
}
You can deserialize as follows:
var list = JsonConvert.DeserializeObject<List<MyModel>>(json);
Notes:
I generated the c# definition for MyModel by uploading your JSON to http://json2csharp.com/.
The reason for the exception you are seeing trying to deserialize directly to MyModel is that your outer JSON container is an array, not an object. As explained in the standard, JSON has two types of container:
An array which is an ordered collection of values. An array begins with [ (left bracket) and ends with ] (right bracket). Values are separated by , (comma).
An object which is an unordered set of name/value pairs. An object begins with { (left brace) and ends with } (right brace).
In the Json.NET Serialization Guide: IEnumerable, Lists, and Arrays it is explained that JSON arrays are converted from and to .Net types implementing IEnumerable. So that's what you need to do.
If you know the array will contain no more than one element, you can use SingleOrDefault() to extract that single element:
Data = list.SingleOrDefault();
However, in the example included in you question, the outer array has 3 items, so this is not appropriate.
Sample fiddle.
When a Json string contains 'key1:value1', this can be converted to the Dictionary type.
But in my case, it also contains an array of strings along with the above key:value, ie:
{k1:v1; "key2\":[{\"Key11\":{\"key21\":\"Val21\",\"key22\":\"val22\"}]
(The Json data contains some strings and some arrays.)
When I use Dictionary<string, string[]> or Dictionary<string, ArrayList> -- it is failing at the value as string - cannot convert string to string[], etc.
Still Dictionary<string, object> can be used, but is there any better way to handle this?
thanks
Phani
If you don't know the structure at compile-time, then there's no other way to serialize a JSON string-- it has to be Dictionary<string,object>. However, if you're using C# 4.0, you can use DynamicObject. Since dynamic typing defers type resolution until runtime, if you serialize using this approach, you can treat your serialized object as strongly-typed (albeit without compile-time support). That means you can use JSON-style dot notation to access properties:
MyDynamicJsonObject.key2
To accomplish this, you can inherit from DynamicObject, and implement the TryGetMember method (quoted from this link, which has a full implementation):
public class DynamicJsonObject : DynamicObject
{
private IDictionary<string, object> Dictionary { get; set; }
public DynamicJsonObject(IDictionary<string, object> dictionary)
{
this.Dictionary = dictionary;
}
public override bool TryGetMember(GetMemberBinder binder, out object result)
{
result = this.Dictionary[binder.Name];
if (result is IDictionary<string, object>)
{
result = new DynamicJsonObject(result as IDictionary<string, object>);
}
else if (result is ArrayList && (result as ArrayList) is IDictionary<string, object>)
{
result = new List<DynamicJsonObject>((result as ArrayList).ToArray().Select(x => new DynamicJsonObject(x as IDictionary<string, object>)));
}
else if (result is ArrayList)
{
result = new List<object>((result as ArrayList).ToArray());
}
return this.Dictionary.ContainsKey(binder.Name);
}
}
Note that dynamic typing currently doesn't support indexer notation, so for arrays, you'll need to implement a workaround using notation like this:
MyDynamicJsonObject.key2.Item(0)
Your example is not valid JSON, e.g. every key and value should be surrounded by ", e.g. {"k1":"v1"}, the number of opening and closing curly brackets must match, if you escape the " character by using \" you must add a another " character, e.g. "key2\""
Use a tool such as JSONLint to validate that your JSON is correct.
I can't seem to find anything in the OpenRasta docs or tutorials that shows how to use arbitrary JSON objects (i.e. objects not predefined using C# classes) for both receiving from and responding back to the client.
One way to do it would be to use JsonValue and write a custom codec that would just use the (de)serialization features provided by JsonValue. That should be pretty straightforward and less than 50 lines of code, but I wondered if there isn't anything built into OpenRasta?
(One downside of JsonValue is that MS has not yet released it, so you can't yet deploy it to customers (see 1. "Additional Use Rights"). But in cases where that matters, any other Json library, like Json.NET can be used.)
I have written, like most people, a very simple codec that supports dynamics as inputs and outputs to handlers using json.net. You can also register that codec with an anonymous type and it works brilliantly. You end up with this:
public object Post(dynamic myCustomer) {
return new { response = myCustomer.Id };
}
I just implemented a JSON codec using JsonFx. It goes like this:
using System.IO;
using System.Text;
using JsonFx.Json;
namespace Example
{
[global::OpenRasta.Codecs.MediaType("application/json")]
public class JsonFXCodec : global::OpenRasta.Codecs.IMediaTypeWriter, global::OpenRasta.Codecs.IMediaTypeReader
{
public void WriteTo(object entity, global::OpenRasta.Web.IHttpEntity response, string[] codecParameters)
{
JsonWriter json = new JsonWriter();
using (TextWriter w = new StreamWriter(response.Stream, Encoding.UTF8))
{
json.Write(entity, w);
}
}
public object ReadFrom(global::OpenRasta.Web.IHttpEntity request, global::OpenRasta.TypeSystem.IType destinationType, string destinationName)
{
JsonReader json = new JsonReader();
using (TextReader r = new StreamReader(request.Stream, Encoding.UTF8))
{
return json.Read(r, destinationType.StaticType);
}
}
public object Configuration { get; set; }
}
}
If it is registered for "object" then it seems to work for any class:
ResourceSpace.Has.ResourcesOfType<object>()
.WithoutUri
.TranscodedBy<JsonFXCodec>();