Serialize Range with Jackson - json

I have a Range that I need to serialize in a certain format, namely "[lower,upper)". To do so I have written a basic serializer:
public class RangeSerializer extends StdSerializer<Range<?>> {
#Override
public void serialize(final Range<?> value, final JsonGenerator gen, final SerializerProvider provider) throws IOException {
if (value != null) {
gen.writeRaw('"');
gen.writeRaw('[');
provider.defaultSerializeValue(value.lowerEndpoint(), gen);
gen.writeRaw(',');
provider.defaultSerializeValue(value.upperEndpoint(), gen);
gen.writeRaw(')');
gen.writeRaw('"');
}
}
(Note that in reality the serializer handles the various possibilities of Range such as closed/open ranges, the possibility of unbounded ranges at either end, etc. but for the purposes of my question that's not relevant so I've removed it to keep the code simple).
My problem is that by falling back to the default serializer for each class I end up with quotes in the wrong place. For example, if I had a Range<String> with an entry "[foo,bar)" then when serializing it I obtain "["foo","bar")". I need the result without quotes around the lower and upper endpoint values.
I understand that the additional quotes are because gen.writeString() in the underlying serializer doesn't realize that it's already in a string. Is there some way to let the generator know this, or an alternative way to achieve what I'm attempting to do?
Note that Range<?> really is generic, so I can't hard-code the serialization of the values. It needs to work for Range<Integer>, Range<String>, Range<DateTime> and anything else.

I was unable to get your approach of using a single generator to serialize everything working. You might be able to use a DelegatingJsonGenerator and hook certain calls, but I decided the way shown below was significantly simpler (at the price of minor performance loss).
Here's a Spock test class demonstrating how I was able to get this working.
#Grab('com.fasterxml.jackson.core:jackson-databind:2.8.1')
#Grab('org.spockframework:spock-core:1.0-groovy-2.4')
import com.fasterxml.jackson.databind.ObjectMapper
import com.fasterxml.jackson.databind.SerializationFeature
import com.fasterxml.jackson.databind.annotation.JsonSerialize
import com.fasterxml.jackson.databind.util.StdConverter
import spock.lang.Specification
import spock.lang.Unroll
class RangeSerializationTest extends Specification {
static class Range<T> {
T lower
T upper
}
#JsonSerialize(converter = RangeConverter)
static interface RangeMixin {
}
static class RangeConverter extends StdConverter<Range, String> {
private static final mapper = new ObjectMapper().disable(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS)
#Override
String convert(Range value) {
def lower = mapper.convertValue(value.lower, String)
def upper = mapper.convertValue(value.upper, String)
return "[$lower,$upper)"
}
}
#Unroll
def 'range of #lower.class'() {
given:
def mapper = new ObjectMapper()
mapper.addMixIn(Range, RangeMixin)
expect:
mapper.writeValueAsString(new Range(lower: lower, upper: upper)) == expectedJson
where:
lower | upper | expectedJson
'abc' | 'def' | '"[abc,def)"'
123 | 456 | '"[123,456)"'
new Date(123) | new Date(456) | '"[1970-01-01T00:00:00.123+0000,1970-01-01T00:00:00.456+0000)"'
}
}

Related

How to use ServiceStack to store POCOs to MariaDB having complex types (objects and structs) blobbed as JSON?

I've got following setup: C#, ServiceStack, MariaDB, POCOs with objects and structs, JSON.
The main question is: how to use ServiceStack to store POCOs to MariaDB having complex types (objects and structs) blobbed as JSON and still have working de/serialization of the same POCOs? All of these single tasks are supported, but I had problems when all put together mainly because of structs.
... finally during writing this I found some solution and it may look like I answered my own question, but I still would like to know the answer from more skilled people, because the solution I found is a little bit complicated, I think. Details and two subquestions arise later in the context.
Sorry for the length and for possible misinformation caused by my limited knowledge.
Simple example. This is the final working one I ended with. At the beginning there were no SomeStruct.ToString()/Parse() methods and no JsConfig settings.
using Newtonsoft.Json;
using ServiceStack;
using ServiceStack.DataAnnotations;
using ServiceStack.OrmLite;
using ServiceStack.Text;
using System.Diagnostics;
namespace Test
{
public class MainObject
{
public int Id { get; set; }
public string StringProp { get; set; }
public SomeObject ObjectProp { get; set; }
public SomeStruct StructProp { get; set; }
}
public class SomeObject
{
public string StringProp { get; set; }
}
public struct SomeStruct
{
public string StringProp { get; set; }
public override string ToString()
{
// Unable to use .ToJson() here (ServiceStack does not serialize structs).
// Unable to use ServiceStack's JSON.stringify here because it just takes ToString() => stack overflow.
// => Therefore Newtonsoft.Json used.
var serializedStruct = JsonConvert.SerializeObject(this);
return serializedStruct;
}
public static SomeStruct Parse(string json)
{
// This method behaves differently for just deserialization or when part of Save().
// Details in the text.
// After playing with different options of altering the json input I ended with just taking what comes.
// After all it is not necessary, but maybe useful in other situations.
var structItem = JsonConvert.DeserializeObject<SomeStruct>(json);
return structItem;
}
}
internal class ServiceStackMariaDbStructTest
{
private readonly MainObject _mainObject = new MainObject
{
ObjectProp = new SomeObject { StringProp = "SomeObject's String" },
StringProp = "MainObject's String",
StructProp = new SomeStruct { StringProp = "SomeStruct's String" }
};
public ServiceStackMariaDbStructTest()
{
// This one line is needed to store complex types as blobbed JSON in MariaDB.
MySqlDialect.Provider.StringSerializer = new JsonStringSerializer();
JsConfig<SomeStruct>.RawSerializeFn = someStruct => JsonConvert.SerializeObject(someStruct);
JsConfig<SomeStruct>.RawDeserializeFn = json => JsonConvert.DeserializeObject<SomeStruct>(json);
}
public void Test_Serialization()
{
try
{
var json = _mainObject.ToJson();
if (!string.IsNullOrEmpty(json))
{
var objBack = json.FromJson<MainObject>();
}
}
catch (System.Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
public void Test_Save()
{
var cs = "ConnectionStringToMariaDB";
var dbf = new OrmLiteConnectionFactory(cs, MySqlDialect.Provider);
using var db = dbf.OpenDbConnection();
db.DropAndCreateTable<MainObject>();
try
{
db.Save(_mainObject);
var dbObject = db.SingleById<MainObject>(_mainObject.Id);
}
catch (System.Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
}
}
What (I think) I know / have tried but at first didn't help to solve it myself:
ServiceStack stores complex types in DB as blobbed JSV by default (last paragraph of first section: https://github.com/ServiceStack/ServiceStack.OrmLite), so it is necessary to set it the way it is proposed: MySqlDialect.Provider.StringSerializer = new JsonStringSerializer(); (https://github.com/ServiceStack/ServiceStack.OrmLite#pluggable-complex-type-serializers)=> default JSV changed to JSON.
the ServiceStack's serialization does not work with structs, it is necessary to treat them special way:
a) according to https://github.com/ServiceStack/ServiceStack.Text#c-structs-and-value-types and example https://github.com/ServiceStack/ServiceStack.Text/#using-structs-to-customize-json it is necessary to implement TStruct.ToString() and static TStruct.ParseJson()/ParseJsv() methods.
b) according to https://github.com/ServiceStack/ServiceStack.Text/#typeserializer-details-jsv-format and unit tests https://github.com/ServiceStack/ServiceStack.Text/blob/master/tests/ServiceStack.Text.Tests/CustomStructTests.cs it shall be TStruct.ToString() (the same as in a) and static TStruct.Parse().
Subquestion #1: which one is the right one? For me, ParseJson() was never called, Parse() was. Documentation issue or is it used in other situation?
I implemented option b). Results:
IDbConnection.Save(_mainObject) saved the item to MariaDB. Success.
Through the saving process ToString() and Parse() were called. In Parse, incoming JSON looked this way:
"{\"StringProp\":\"SomeStruct's String\"}". Fine.
Serialization worked. Success.
Deserialization failed. I don't know the reason, but JSON incoming to Parse() was "double-escaped":
"{\\\"StringProp\\\":\\\"SomeStruct's String\\\"}"
Subquestion #2: Why the "double-escaping" in Parse on deserialization?
I tried to solve structs with JsConfig (and Newtonsoft.Json to get proper JSON):
JsConfig<SomeStruct>.SerializeFn = someStruct => JsonConvert.SerializeObject(someStruct);
JsConfig<SomeStruct>.DeSerializeFn = json => JsonConvert.DeserializeObject<SomeStruct>(json);
a) at first without ToString() and Parse() defined in the TStruct. Results:
Save failed: the json input in JsonConvert.DeserializeObject(json) that is used during Save was just type name "WinAmbPrototype.SomeStruct".
De/serialization worked.
b) then I implemented ToString() also using Newtonsoft.Json. During Save ToString() was used instead of JsConfig.SerializeFn even the JsConfig.SerializeFn was still set (maybe by design, I do not judge). Results:
Save failed: but the json input of DeserializeFn called during Save changed, now it was JSV-like "{StringProp:SomeStruct's String}", but still not deserializable as JSON.
De/serialization worked.
Then (during writing this I was still without any solution) I found JsConfig.Raw* "overrides" and tried them:
JsConfig<SomeStruct>.RawSerializeFn = someStruct => JsonConvert.SerializeObject(someStruct);
JsConfig<SomeStruct>.RawDeserializeFn = json => JsonConvert.DeserializeObject<SomeStruct>(json);
a) at first without ToString() and Parse() defined in the TStruct. Results are the same as in 2a.
b) then I implemented ToString(). Results:
BOTH WORKED. No Parse() method needed for this task.
But it is very fragile setup:
if I removed ToString(), it failed (now I understand why, default ToString produced JSON with just type name in 2a, 3a).
if I removed RawSerializeFn setting, it failed in RawDeserializeFn ("double-escaped" JSON).
Is there some simpler solution? I would be very glad if someone points me to better direction.
Acceptable would be maybe two (both of them accessible because of different circumstances):
if I am the TStruct owner: with just pure TStruct.ToString() and static TStruct.Parse() to support out of the box de/serialization and DB by ServiceStack (without different input in Parse()).
if I am a consumer of TStruct with no JSON support implemented and I am without access to its code: until now I did not find the way, if the ToString is not implemented: Save to DB did not work. Maybe would be fine to ensure JsConfig serialize functions are enough for both de/serialization and when used during saving to DB.
And the best one would be without employing other dependency (e.g. Newtonsoft.Json) to serialize structs. Maybe some JsConfig.ShallProcessStructs = true; (WARNING: just a tip, not working as of 2021-04-02) would be fine for such situations.
ServiceStack treats structs like a single scalar value type, just like most of the core BCL Value Types (e.g. TimeSpan, DateTime, etc). Overloading the Parse() and ToString() methods and Struct's Constructor let you control the serialization/deserialization of custom structs.
Docs have been corrected. Structs use Parse whilst classes use ParseJson/ParseJsv
If you want to serialize a models properties I'd suggest you use a class instead as the behavior you're looking for is that of a POCO DTO.
If you want to have structs serailized as DTOs in your RDBMS an alternative you can try is to just use JSON.NET for the complex type serialization, e.g:
public class JsonNetStringSerializer : IStringSerializer
{
public To DeserializeFromString<To>(string serializedText) =>
JsonConvert.DeserializeObject<To>(serializedText);
public object DeserializeFromString(string serializedText, Type type) =>
JsonConvert.DeserializeObject(serializedText, type);
public string SerializeToString<TFrom>(TFrom from) =>
JsonConvert.SerializeObject(from);
}
MySqlDialect.Provider.StringSerializer = new JsonNetStringSerializer();

Grails: setting transient fields in the map constructor

I'm trying to persist Maps of properties as single JSON-encoded columns, as shown in this question.
The problem I'm having is that apparently transient properties cannot be set in the default map constructor. Given any transient field:
class Test {
//...
String foo
static transients = ['foo']
}
It seems that the map constructor (which Grails overrides in various ways) simply discards transient fields:
groovy:000> t = new Test(foo:'bar')
===> Test : (unsaved)
groovy:000> t.foo
===> null
While direct assignment (through the setter method) works as expected:
groovy:000> c.foo = 'bar'
===> bar
groovy:000> c.foo
===> bar
Is there a way to make the map constructor accept transient fields?
Or rather: is there a better way to persist a Map as a single JSON-encoded DB field, rather than the method shown in the linked question?
Here's the complete example:
import grails.converters.JSON
class JsonMap {
Map data
String dataAsJSON
static transients = ['data']
def afterLoad() { data = JSON.parse(dataAsJSON) }
def beforeValidate() { dataAsJSON = data as JSON }
}
I can set data using the setter (which will then be converted into dataAsJSON) but not using the map constructor.
The map constructor in GORM uses the data binding mechanism, and transient properties are not data-bindable by default. But you can override this using the bindable constraint
class Test {
//...
String foo
static transients = ['foo']
static constraints = {
foo bindable:true
}
}
I've also replied to your original question, that you don't need json conversion to achieve what you need. However, If you need json conversion badly, why don't you implement it in your getters/setters?
class Test {
String propsAsJson
static transients = ['props']
public Map getProps() {
return JSON.parse(propsAsJson)
}
public void setProps(Map props) {
propsAsJson = props as JSON
}
}
//So you can do
Test t = new Test(props : ["foo" : "bar"])
t.save()
In this way you encapsulate the conversion stuff, and in DB you have your properties as Json.
You can simplify your case by adding the JSON-conversion methods to your domain class, they should have nothing to do with GORMing:
class Test {
String title
void titleFromJSON( json ){
title = json.toStringOfSomeKind()
}
def titleAsJSON(){
new JSON( title )
}
}

How can I wrap a JSON response in Spring

Suppose I have two sets of controllers in Spring:
/jsonapi1/*
/jsonapi2/*
both of which return objects that are to be interpretted as JSON text.
I'd like some kind of filter to wrap the responses from one set of these controllers so that:
the original response is contained within another object.
For example, if /jsonapi1/count returns:
{"num_humans":123, "num_androids":456}
then the response should be wrapped and returned as follows:
{ "status":0,
"content":{"num_humans":123, "num_androids":456}
}
if an exception happens in the controller, then filter should catch the exception and report it as follows
{ "status":5,
"content":"Something terrible happened"
}
The responses from the other controllers are returned unchanged.
We're currently customizing a MappingJackson2HttpMessageConverter passed to WebMvcConfigurerAdapter.configureMessageConverters in order to perform the above tasks. Works great except that it doesn't seem possible for this approach to be selective about the URLs (or controller classes) it applies to.
Is it possible to apply these kinds of wrappers to individual controller classes or URLs?
Update: Servlet filters look like a solution. Is it possible chose which filter gets applied to which controller methods, or which URLs?
I was struggling on this for multiple days. The solution by #Misha didn't work for me. I was able to finally get this working using ControllerAdvice and ResponseBodyAdvice.
ResponseBodyAdvice allows to inject custom transformation logic on the response returned by a controller but before it is converted to HttpResponse and committed.
This is how my controller method looks:
#RequestMapping("/global/hallOfFame")
public List<HallOfFame> getAllHallOfFame() {
return hallOfFameService.getAllHallOfFame();
}
Now i wanted to add some standard fields around the response like devmessage and usermessage. That logic goes into the ResponseAdvice:
#ControllerAdvice
public class TLResponseAdvice implements ResponseBodyAdvice<Object> {
#Override
public boolean supports(MethodParameter returnType, Class<? extends HttpMessageConverter<?>> converterType) {
return true;
}
#Override
public Object beforeBodyWrite(Object body, MethodParameter returnType, MediaType selectedContentType,
Class<? extends HttpMessageConverter<?>> selectedConverterType, ServerHttpRequest request,
ServerHttpResponse response) {
// TODO Auto-generated method stub
final RestResponse<Object> output = new RestResponse<>();
output.setData(body);
output.setDevMessage("ResponseAdviceDevMessage");
output.setHttpcode(200);
output.setStatus("Success");
output.setUserMessage("ResponseAdviceUserMessage");
return output;
}
}
The entity classes look like this:
#Setter // All lombok annotations
#Getter
#ToString
public class RestResponse<T> {
private String status;
private int httpcode;
private String devMessage;
private String userMessage;
private T data;
}
#Entity
#Data // Lombok
public class HallOfFame {
#Id
private String id;
private String name;
}
To handle exceptions, simply create another ControllerAdvice with ExceptionHandler. Use the example in this link.
Advantages of this solution:
It keeps your controllers clean. You can support any return type from your controller methods.
Your controller return type class does not need to extend some base class as required by the AOP approach.
You do not need to hack your way through Spring filters by using HttpServletResponseWrappers. They come up with a performance penalty.
EDIT - 17th September 2019
To handle exceptions use #ExceptionHandler. Refer code below.
#ExceptionHandler(Exception.class)
#ResponseBody
public MyResponseEntity<Object> handleControllerException(HttpServletRequest request, Throwable ex) {
// default value
int httpCode = HttpStatus.INTERNAL_SERVER_ERROR.value();
if(ex instanceof ResourceNotFoundException) {
httpCode = HttpStatus.NOT_FOUND.value();
}
...
}
The way I understand your question, you have exactly three choices.
Option #1
Manually wrap your objects in simple SuccessResponse, ErrorResponse, SomethingSortOfWrongResponse, etc. objects that have the fields you require. At this point, you have per-request flexibility, changing the fields on one of the response wrappers is trivial, and the only true drawback is code repetition if many of the controller's request methods can and should be grouped together.
Option #2
As you mentioned, and filter could be designed to do the dirty work, but be wary that Spring filters will NOT give you access to request or response data. Here's an example of what it might look like:
#Component
public class ResponseWrappingFilter extends GenericFilterBean {
#Override
public void doFilter(
ServletRequest request,
ServletResponse response,
FilterChain chain) {
// Perform the rest of the chain, populating the response.
chain.doFilter(request, response);
// No way to read the body from the response here. getBody() doesn't exist.
response.setBody(new ResponseWrapper(response.getStatus(), response.getBody());
}
}
If you find a way to set the body in that filter, then yes, you could easily wrap it up. Otherwise, this option is a dead end.
Option #3
A-ha. So you got this far. Code duplication is not an option, but you insist on wrapping responses from your controller methods. I'd like to introduce the true solution - aspect-oriented programming (AOP), which Spring supports fondly.
If you're not familiar with AOP, the premise is as follows: you define an expression that matches (like a regular expression matches) points in the code. These points are called join points, while the expressions that match them are called pointcuts. You can then opt to execute additional, arbitrary code, called advice, when any pointcut or combination of pointcuts are matched. An object that defines pointcuts and advice is called an aspect.
It's great for expressing yourself more fluently in Java. The only drawback is weaker static type checking. Without further ado, here's your response-wrapping in aspect-oriented programming:
#Aspect
#Component
public class ResponseWrappingAspect {
#Pointcut("within(#org.springframework.stereotype.Controller *)")
public void anyControllerPointcut() {}
#Pointcut("execution(* *(..))")
public void anyMethodPointcut() {}
#AfterReturning(
value = "anyControllerPointcut() && anyMethodPointcut()",
returning = "response")
public Object wrapResponse(Object response) {
// Do whatever logic needs to be done to wrap it correctly.
return new ResponseWrapper(response);
}
#AfterThrowing(
value = "anyControllerPointcut() && anyMethodPointcut()",
throwing = "cause")
public Object wrapException(Exception cause) {
// Do whatever logic needs to be done to wrap it correctly.
return new ErrorResponseWrapper(cause);
}
}
The final result will be the non-repeating response wrapping that you seek. If you only want some or one controller receive this effect, then update the pointcut to match methods only within instances of that controller (rather than any class holding the #Controller annotation).
You'll need to include some AOP dependencies, add the AOP-enabling annotation in a configuration class, and make sure something component-scans the package this class is in.
Simplest way i manage custom responses from controllers is by utilising the Map variable.
so your code ends up looking like:
public #ResponseBody Map controllerName(...) {
Map mapA = new HashMap();
mapA.put("status", "5");
mapA.put("content", "something went south");
return mapA;
}
beauty of is is that you can configure it any thousand ways.
Currently i use for object transmition, custom exception handling and data reporting, too easy.
Hope this helps
I am also using AOP with #Around. Developed a custom annotation and using that for point cut. I am using a global Response. It has the status, Message and data which is of type List of type
List <? extends parent> dataList
( which can solve your class cast exception). All the entities extends this Parent class. This way I can set all the data into my List.
Also I am using the message key as param with the custom annotation and setting it in action.
Hope this helps.

jackson arraynode

I have a input json object that contains a array as shown below:
[{"key11":"value11","key12":"value12"},
{"key21":"value21","key22":"value22"},
...
{"keyn1":"valuen1","keyn2":"valuen2"}]
I would like to use jackson parser ,firstly, to identify the number of rows and secondly, print only the values on the screen. I want it to be as lightweight as possible. Can anyone point me to a direction?
Assuming that a row is one of the entries in a ArrayNode the following simple approach may be useful. It uses the JsonNode abstraction instead of a series of nested Map objects which I personally prefer since JsonNode provides a series of utility methods that are helpful when dealing with this kind of data (data where the structure is possibly unknown or very dynamic so that it can't be easily transformed to a POJO).
The testcase below illustrates how to find the number of rows and how to print the values. To get hold of the values the method JsonNode.elements() is used and the number of rows is simply a call to the size()-method.
public class ArrayNodeTest {
#Test
public void verifySizeAndPrintRows() throws IOException {
final String jsonStr =
"[{\"key11\":\"value11\",\"key12\":\"value12\"},\n" +
"{\"key21\":\"value21\",\"key22\":\"value22\"},\n" +
"{\"keyn1\":\"valuen1\",\"keyn2\":\"valuen2\"}]";
final ObjectMapper mapper = new ObjectMapper();
final JsonNode jsonNode = mapper.readTree(jsonStr);
// Verify size
Assert.assertEquals(3, jsonNode.size());
// And print rows
for (final JsonNode row : jsonNode) {
final Iterable<JsonNode> iterable = () -> row.elements();
iterable.forEach(elem -> System.out.println(elem.asText()));
}
}
}
to identify the number of rows
What's a "row"? Is a key/value pair a row? Is an element of the JSON array (no matter how many key/value pairs it contains) a row?
print only the values on the screen
Jackson has nothing to do with printing anything on any screens. Jackson can be used to populate a Java data structure from the input JSON, and the populated Java data structure can then be used however you want.
Given the JSON structure in the original question, a simple solution would be to bind to a list (or array) of maps, and then just iterate through the list of maps, accessing all of the values. Following is such an example.
import java.io.File;
import java.util.Map;
import org.codehaus.jackson.map.ObjectMapper;
public class JacksonFoo
{
public static void main(String[] args) throws Exception
{
ObjectMapper mapper = new ObjectMapper();
Map<String, String>[] maps = mapper.readValue(new File("input.json"), Map[].class);
for (Map<String, String> map : maps)
{
for (Map.Entry<String, String> entry : map.entrySet())
{
System.out.println(entry.getValue());
}
}
}
}
Output:
value11
value12
value21
value22
valuen1
valuen2

Dependency Injection and JavaScriptConverter.Deserialize

My application needs to combine extensive use of dependency injection with the use of JSON as a public API. This apparently leads to the need for a custom JavaScriptConverter.
Right now, my JavaScriptConverter's Deserialize method looks like this:
public override object Deserialize(IDictionary<string, object> dictionary, Type type, JavaScriptSerializer serializer)
{
var result = IocHelper.GetForType(type);
return result;
}
This hands back the appropriate class. Unfortunately, it fails to populate the class members with the applicable values. What I'm missing is a way to tell the Serializer, "Here's the type you asked for. Now fill it in."
The solution I used was to switch from JavaScriptSerializer to Newtonsoft's JSON converter
I was able to get a working round trip by writing a single CustomCreationConverter:
public class JsonDomainConverter : CustomCreationConverter<object>
{
public JsonDomainConverter()
{
}
public override bool CanConvert(Type objectType)
{
return objectType.IsInterface;
}
public override object Create(Type objectType)
{
return IocHelper.GetForType(objectType);
}
}
No doubt this same approach is possible with JavaScriptSerializer, I just couldn't figure out how to make it work. With the Newtonsoft stuff, it took a couple hours at the most, and just a couple lines of code.