Generate JSON from Nustache with proper escaping - json

I would like to use Nustache to generate JSON to talk to a specific webservice.
I do not want to do proper JSON-building using Newtonsoft or something like that because the specs for this webservice come as textfiles with placeholders. I agree that this is silly. So it makes sense to copy/massage/paste them into a template-format and hopefully make fewer mistakes.
But of course Nustache has no notion of what makes valid JSON.
With a template like
{ "foo": "{{bar}}" }
and a value for bar that needs escaping in JSON, say it includes curly brackets or an innocent backslash the result is string-replacy-correct, but not valid JSON.
Is there a way to tell Nustache that I want the output to be JSON and have it escape strings as it replaces them?
Or would you recommend doing a helper that can manage the escaping and put that on all the placeholders?
Thanks for reading and thinking.

I did not find a wholly satisfactory answer but a workable solution.
My workaround is a Nustache helper that takes care of the quoting and escaping. The ugly bit is that I need to specify the helper in the template in each instance:
{ "foo": "{{json bar}}" }
The helper implementation is trivial and can be listed fully here. For the actual work it delegates to JsonConvert from Newtonsoft JSON:
public class JSONHelpers
{
public static void Register()
{
if (!Helpers.Contains("json"))
{
Helpers.Register("json", JSON);
}
}
private static void JSON(RenderContext ctx, IList<object> args, IDictionary<string, object> options,
RenderBlock fn, RenderBlock inverse)
{
object input = args[0];
if (input == null)
{
ctx.Write("null");
}
else
{
string text = input.ToString();
string json = JsonConvert.ToString(text);
ctx.Write(json);
}
}
}
Hopefully this comes useful to someone else.

Related

How to use ServiceStack to store POCOs to MariaDB having complex types (objects and structs) blobbed as JSON?

I've got following setup: C#, ServiceStack, MariaDB, POCOs with objects and structs, JSON.
The main question is: how to use ServiceStack to store POCOs to MariaDB having complex types (objects and structs) blobbed as JSON and still have working de/serialization of the same POCOs? All of these single tasks are supported, but I had problems when all put together mainly because of structs.
... finally during writing this I found some solution and it may look like I answered my own question, but I still would like to know the answer from more skilled people, because the solution I found is a little bit complicated, I think. Details and two subquestions arise later in the context.
Sorry for the length and for possible misinformation caused by my limited knowledge.
Simple example. This is the final working one I ended with. At the beginning there were no SomeStruct.ToString()/Parse() methods and no JsConfig settings.
using Newtonsoft.Json;
using ServiceStack;
using ServiceStack.DataAnnotations;
using ServiceStack.OrmLite;
using ServiceStack.Text;
using System.Diagnostics;
namespace Test
{
public class MainObject
{
public int Id { get; set; }
public string StringProp { get; set; }
public SomeObject ObjectProp { get; set; }
public SomeStruct StructProp { get; set; }
}
public class SomeObject
{
public string StringProp { get; set; }
}
public struct SomeStruct
{
public string StringProp { get; set; }
public override string ToString()
{
// Unable to use .ToJson() here (ServiceStack does not serialize structs).
// Unable to use ServiceStack's JSON.stringify here because it just takes ToString() => stack overflow.
// => Therefore Newtonsoft.Json used.
var serializedStruct = JsonConvert.SerializeObject(this);
return serializedStruct;
}
public static SomeStruct Parse(string json)
{
// This method behaves differently for just deserialization or when part of Save().
// Details in the text.
// After playing with different options of altering the json input I ended with just taking what comes.
// After all it is not necessary, but maybe useful in other situations.
var structItem = JsonConvert.DeserializeObject<SomeStruct>(json);
return structItem;
}
}
internal class ServiceStackMariaDbStructTest
{
private readonly MainObject _mainObject = new MainObject
{
ObjectProp = new SomeObject { StringProp = "SomeObject's String" },
StringProp = "MainObject's String",
StructProp = new SomeStruct { StringProp = "SomeStruct's String" }
};
public ServiceStackMariaDbStructTest()
{
// This one line is needed to store complex types as blobbed JSON in MariaDB.
MySqlDialect.Provider.StringSerializer = new JsonStringSerializer();
JsConfig<SomeStruct>.RawSerializeFn = someStruct => JsonConvert.SerializeObject(someStruct);
JsConfig<SomeStruct>.RawDeserializeFn = json => JsonConvert.DeserializeObject<SomeStruct>(json);
}
public void Test_Serialization()
{
try
{
var json = _mainObject.ToJson();
if (!string.IsNullOrEmpty(json))
{
var objBack = json.FromJson<MainObject>();
}
}
catch (System.Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
public void Test_Save()
{
var cs = "ConnectionStringToMariaDB";
var dbf = new OrmLiteConnectionFactory(cs, MySqlDialect.Provider);
using var db = dbf.OpenDbConnection();
db.DropAndCreateTable<MainObject>();
try
{
db.Save(_mainObject);
var dbObject = db.SingleById<MainObject>(_mainObject.Id);
}
catch (System.Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
}
}
What (I think) I know / have tried but at first didn't help to solve it myself:
ServiceStack stores complex types in DB as blobbed JSV by default (last paragraph of first section: https://github.com/ServiceStack/ServiceStack.OrmLite), so it is necessary to set it the way it is proposed: MySqlDialect.Provider.StringSerializer = new JsonStringSerializer(); (https://github.com/ServiceStack/ServiceStack.OrmLite#pluggable-complex-type-serializers)=> default JSV changed to JSON.
the ServiceStack's serialization does not work with structs, it is necessary to treat them special way:
a) according to https://github.com/ServiceStack/ServiceStack.Text#c-structs-and-value-types and example https://github.com/ServiceStack/ServiceStack.Text/#using-structs-to-customize-json it is necessary to implement TStruct.ToString() and static TStruct.ParseJson()/ParseJsv() methods.
b) according to https://github.com/ServiceStack/ServiceStack.Text/#typeserializer-details-jsv-format and unit tests https://github.com/ServiceStack/ServiceStack.Text/blob/master/tests/ServiceStack.Text.Tests/CustomStructTests.cs it shall be TStruct.ToString() (the same as in a) and static TStruct.Parse().
Subquestion #1: which one is the right one? For me, ParseJson() was never called, Parse() was. Documentation issue or is it used in other situation?
I implemented option b). Results:
IDbConnection.Save(_mainObject) saved the item to MariaDB. Success.
Through the saving process ToString() and Parse() were called. In Parse, incoming JSON looked this way:
"{\"StringProp\":\"SomeStruct's String\"}". Fine.
Serialization worked. Success.
Deserialization failed. I don't know the reason, but JSON incoming to Parse() was "double-escaped":
"{\\\"StringProp\\\":\\\"SomeStruct's String\\\"}"
Subquestion #2: Why the "double-escaping" in Parse on deserialization?
I tried to solve structs with JsConfig (and Newtonsoft.Json to get proper JSON):
JsConfig<SomeStruct>.SerializeFn = someStruct => JsonConvert.SerializeObject(someStruct);
JsConfig<SomeStruct>.DeSerializeFn = json => JsonConvert.DeserializeObject<SomeStruct>(json);
a) at first without ToString() and Parse() defined in the TStruct. Results:
Save failed: the json input in JsonConvert.DeserializeObject(json) that is used during Save was just type name "WinAmbPrototype.SomeStruct".
De/serialization worked.
b) then I implemented ToString() also using Newtonsoft.Json. During Save ToString() was used instead of JsConfig.SerializeFn even the JsConfig.SerializeFn was still set (maybe by design, I do not judge). Results:
Save failed: but the json input of DeserializeFn called during Save changed, now it was JSV-like "{StringProp:SomeStruct's String}", but still not deserializable as JSON.
De/serialization worked.
Then (during writing this I was still without any solution) I found JsConfig.Raw* "overrides" and tried them:
JsConfig<SomeStruct>.RawSerializeFn = someStruct => JsonConvert.SerializeObject(someStruct);
JsConfig<SomeStruct>.RawDeserializeFn = json => JsonConvert.DeserializeObject<SomeStruct>(json);
a) at first without ToString() and Parse() defined in the TStruct. Results are the same as in 2a.
b) then I implemented ToString(). Results:
BOTH WORKED. No Parse() method needed for this task.
But it is very fragile setup:
if I removed ToString(), it failed (now I understand why, default ToString produced JSON with just type name in 2a, 3a).
if I removed RawSerializeFn setting, it failed in RawDeserializeFn ("double-escaped" JSON).
Is there some simpler solution? I would be very glad if someone points me to better direction.
Acceptable would be maybe two (both of them accessible because of different circumstances):
if I am the TStruct owner: with just pure TStruct.ToString() and static TStruct.Parse() to support out of the box de/serialization and DB by ServiceStack (without different input in Parse()).
if I am a consumer of TStruct with no JSON support implemented and I am without access to its code: until now I did not find the way, if the ToString is not implemented: Save to DB did not work. Maybe would be fine to ensure JsConfig serialize functions are enough for both de/serialization and when used during saving to DB.
And the best one would be without employing other dependency (e.g. Newtonsoft.Json) to serialize structs. Maybe some JsConfig.ShallProcessStructs = true; (WARNING: just a tip, not working as of 2021-04-02) would be fine for such situations.
ServiceStack treats structs like a single scalar value type, just like most of the core BCL Value Types (e.g. TimeSpan, DateTime, etc). Overloading the Parse() and ToString() methods and Struct's Constructor let you control the serialization/deserialization of custom structs.
Docs have been corrected. Structs use Parse whilst classes use ParseJson/ParseJsv
If you want to serialize a models properties I'd suggest you use a class instead as the behavior you're looking for is that of a POCO DTO.
If you want to have structs serailized as DTOs in your RDBMS an alternative you can try is to just use JSON.NET for the complex type serialization, e.g:
public class JsonNetStringSerializer : IStringSerializer
{
public To DeserializeFromString<To>(string serializedText) =>
JsonConvert.DeserializeObject<To>(serializedText);
public object DeserializeFromString(string serializedText, Type type) =>
JsonConvert.DeserializeObject(serializedText, type);
public string SerializeToString<TFrom>(TFrom from) =>
JsonConvert.SerializeObject(from);
}
MySqlDialect.Provider.StringSerializer = new JsonNetStringSerializer();

Gson - nice serialisation of Kotlin enums with a value

I have an enum with values
enum class Foo(val serverVal:Int) {
BAR(1),
BUG(2)
}
that I would like to use with a class:
data class C1(val fooVal:Foo)
I want to be able to serialise it with a code that looks as close as can be to:
Gson().toJson(C1(Foo.BAR))
That would yield
{"fooVal":1}
Instead of the default conversion, which is, of course {"fooVal":"BAR"}. If it was a string value I could have used #SerializedName, but I can't because it's an Int, not a String.
Is there a simple way to add something to Foo, in order to show Gson how to take the Int value from Foo entries rather than their name?
Examples I saw in Java include EnumAdapterFactory and TypeAdapter which are quite cumbersome and defeat the purpose of a pretty code.
I would love to get a solution that is maintained inside the data structure.
enum class SomeEnum(val i:Int) {
V1(1), V2(10), V3(5);
companion object {
#SerializeMeLikeThis
fun toJson() = i;
#DeserializeMeLikeThat
fun fromJson(v:Int) = values().find{it.i == v}?:whatever
}
}
An ugly way but still encapsulated-ish
data class C2(#SerializedName("foo") var serverFoo:Int) {
// Becomes a nightmare with many params.
constructor(f:Foo) : this(F.serverVal)
var foo:Foo
get() = Foo.values().find{serverFoo == it.serverVal}?:whatever
set(v) { serverFoo = v.serverVal }
}
So it can be called
Gson().toJson(C2(BAR))
and
// Result: Foo.BAR
Gson().fromJson("""{"foo":"1"}""", C2::class.java).foo
Well, you can live on it...but it... :-(
Any nice way?
Thanks

Gson: Parsing String[]

I'm using Gson library to parse JSON objects. In particular, I've got a JSON like this:
{
"key": ["string1", "string2"]
}
and I would like to parse it in a simple String[], without building a specific object. I tried this way:
gson.fromJson(json, String[].class);
but I had an error: "Expected BEGIN_OBJECT but was BEGIN_ARRAY", I guess because of the presence of the key. Any ideas how should I fix it?
Create a class that has a key property that has type of String[] and deserialize to that.
public class Thing
{
private String[] key;
public String[] getKey() {
return key;
}
}
Thing thing = gson.fromJson(json, Thing.class);
Since tvanfosson answer is perfect, I should not add anything but in the comment you asked is it's possible to avoid creating the Thing class. Yes, it is but I think is more fragile. I'm going to show you how to do with this code:
String json = "{\"key\": [\"string1\", \"string2\"]}";
String mJson = json.replace("{\"key\":", "").replace("}","");
String[] strings = new Gson().fromJson(mJson, String[].class);
System.out.println(Arrays.asList(strings));
Of course this code runs without errors and avoids you additional classes, but think to what happens if there's some carriage return inside. It breaks, unless you user a regexp inside the replace invocation. At this point I prefer to add a class instead of thinking of right regexp and let Gson do the whole work.
I added this as response and not as comment to have enough space to explain myself, but this answer should not be taken as right response but instead as poor hack to use. It pays better to understand a bit more how Gson reasons.

Json.Net deserialization, keeping some properties unserialized

I have a json string looking something like this:
{
"Bar": {
"Pressure":42,
"ServesDrinks":false
},
"Baz":[
{
"Pressure":9001,
"ServesDrinks":true
},
{
"Pressure":0,
"ServesDrinks":false
}
]
}
And I would like to de-serialize this into a class looking something like this:
public class Foo
{
public Bar Bar { get; set; }
/// <summary>
/// Raw serialized (un-de-serialized) json string of the Baz list
/// </summary>
public string Baz { get; set; }
}
Basically not de-serializing the Baz value and instead storing the raw string.
I.e. I want the Baz string value after serialization to be:
"[{\"Pressure\":9001,\"ServesDrinks\":true},{\"Pressure\":0,\"ServesDrinks\":false}]"
I am using Json.Net but have not managed to figure out how to do this.
In the ideal case I would just like to put a magic attribute on my Baz property that retains the raw json.
I have seen a very similar question here:
Partially deserialize with JSON.NET, keeping some fields raw
But I did understand how to go with that JObject approach.
(If it is of any help, I know that the data I want to retain is a list)
Thankful for any help!
Based on the answer to the other question, it seems that the solution in JSON.Net for cases like this is to use the appropriate J* type and use their ToString() and Parse() methods to work with the JSON string.
For JS lists, the appropriate type seems to be JArray, not JObject, which is used for {} style Javascript objects.
Please note that I know little about JSON.Net, and perhaps there is a different solution which would avoid the need to use Parse() and ToString() everywhere. Somebody with a better knowledge of that library might give a better answer.

How can I pass complex objects as arguments to a RESTful service?

I have successfully set up a quick test of creating a "REST-like" service that returns an object serialized to JSON, and that was quite easy and quick (based on this article).
But while returning JSON-ified objects was easy as peach, I have yet to see any examples dealing with input parameters that are not primitives. How can I pass in a complex object as an argument? I am using Apache CXF, but examples using other frameworks like Jackson are welcome too :)
Client side would probably be something like building a javascript object, pass it into JSON.stringify(complexObj), and pass that string as one of the parameters.
The service would probably look something like this
#Service("myService")
class RestService {
#GET
#Produces("application/json")
#Path("/fooBar")
public Result fooBar(#QueryParam("foo") double foo, #QueryParam("bar") double bar,
#QueryParam("object") MyComplex object) throws WebServiceException {
...
}
}
Sending serialized objects as parameters would probably quickly touch the 2KB URL-limit imposed by Internet Explorer. Would you recommend using POST in these cases, and would I need to change much in the function definitions?
After digging a bit I quickly found out there are basically two options:
Option 1
You pass a "wrapper object" containing all the other parameters to the service. You might need to annotate this wrapper class with JAXB annotations like #XmlRootElement in order for this to work with the Jettison based provider, but if you use Jackson in stead there is no need. Just set the content type to the right type and the right message body reader will be invoked.
This will only work for POST type services of course (AFAIK).
Example
This is just an example of turning the service mentioned in the original question into one using a wrapper object.
#Service("myService")
class RestService {
#POST
#Produces("application/json")
#Path("/fooBar")
public Result fooBar(
/**
* Using "" will inject all form params directly into a ParamsWrapper
* #see http://cxf.apache.org/docs/jax-rs-basics.html
*/
#FormParam("") FooBarParamsWrapper wrapper
) throws WebServiceException {
doSomething(wrapper.foo);
}
}
class ParamsWrapper {
double foo, bar;
MyComplexObject object;
}
Option 2
You can provide some special string format that you pack your objects into and then implement either a constructor taking a string, a static valueOf(String s) or a static fromString(String s) in the class that will take this string and create an object from it. Or quite similar, create a ParameterHandler that does exactly the same.
AFAIK, only the second version will allow you to call your services from a browser using JSONP (since JSONP is a trick restricted to GET). I chose this route to be able to pass arrays of complex objects in the URI.
As an example of how this works, take the following domain class and service
Example
#GET
#Path("myService")
public void myService(#QueryParam("a") MyClass [] myVals) {
//do something
}
class MyClass {
public int foo;
public int bar;
/** Deserializes an Object of class MyClass from its JSON representation */
public static MyClass fromString(String jsonRepresentation) {
ObjectMapper mapper = new ObjectMapper(); //Jackson's JSON marshaller
MyClass o= null;
try {
o = mapper.readValue(jsonRepresentation, MyClass.class );
} catch (IOException e) {
throw new WebApplicationException()
}
return o;
}
}
A URI http://my-server.com/myService?a={"foo":1, "bar":2}&a={"foo":100, "bar":200} would in this case be deserialized into an array composed of two MyClass objects.
2019 comment:
Seeing that this answer still gets some hits in 2019, I feel I should comment. In hindsight, I would not recomment option 2, as going through these steps just to be able to be able to do GET calls adds complexity that's probably not worth it. If your service takes such complex input, you will probably not be able to utilize client side caching anyway, due to the number of permutations of your input. I'd just go for configuring proper Cross-Origin-Sharing (CORS) headers on the server and POST the input. Then focus on caching whatever you can on the server.
The accepted answer is missing #BeanParam. See
https://docs.jboss.org/resteasy/docs/3.0-rc-1/javadocs/javax/ws/rs/BeanParam.html
for further details. It allows you to define query params inside a wrapper object.
E.g.
public class TestPOJO {
#QueryParam("someQueryParam")
private boolean someQueryParam;
public boolean isSomeQueryParam() {
return someQueryParam;
}
public boolean setSomeQueryParam(boolean value) {
this.someQueryParam = value;
}
}
... // inside the Resource class
#GET
#Path("test")
public Response getTest(#BeanParam TestPOJO testPOJO) {
...
}
the best and simplest solution is to send your object as a json string and in server side implement a method which will decode that json and map to the specified object as per your need.. and yes it`s better to use POST.