Jersey/Genson: Unmarschalling single object array - json

Similar to Jersey: Json array with 1 element is serialized as object BUT on the client side. E.g. I recieve a JSON object where a field is an array regulary, but in case there is only one element, it is a single object.
{"fileInfo":[{"fileName":"weather.arff","id":"10"},"fileName":"supermarket.arff","id":"11"}]}
versus
{"fileInfo":{"fileName":"weather.arff","id":"10"}}
I'm parsing/unmarshalling the JSON using Jersey/Genson. Of course, if the JSON doesnt match the target class I recieve an error (such as expected [ but read '{' )
I've read a lot about this bug and how to avoid when creating JSON objects on the SERVER side, but I found nothing about how to handle this issus when dealing on the CLIENT side.
As always, I prefere the most codeless possibility if there are several solutions...
BTW: Moxy works but it does not marshal native Object-type objects which is another requirement...

Update
Starting with Genson 1.3 release you can achieve it by enabling permissiveParsing:
Genson genson = new GensonBuilder().usePermissiveParsing(true).create();
Answer
Uh, do you know what library produces this on server side? I am curious to see who is responsible for all those badly structured jsons out there...
It is not yet supported in Genson. Originally because IMO people should not produce such dynamic json. Anyway, I opened an issue - this can be easily done, you can expect it to be present in the release coming next week.
Otherwise here is a way to achieve it without breaking the existing mechanisms.
You need to register a Factory that will use Gensons collections factory to create an instance of its standard collection converter. Then you will wrap this converter in another one that will handle the object to array logic. Here is the code (not codeless..., but if you wait a bit you won't have to code :)).
import com.owlike.genson.convert.DefaultConverters.CollectionConverterFactory;
import com.owlike.genson.convert.DefaultConverters.CollectionConverterFactory;
class SingleObjectAsCollectionFactory implements Factory<Converter<Collection>> {
// get the default factory
Factory<Converter<Collection<?>>> defaultFactory = CollectionConverterFactory.instance;
#Override
public Converter<Collection> create(Type type, Genson genson) {
// obtain an instance of the correct default converter for this type
final CollectionConverter defaultConverter = (CollectionConverter) defaultFactory.create(type, genson);
// wrap it in your own converter
return new Converter<Collection>() {
#Override
public void serialize(Collection object, ObjectWriter writer, Context ctx) throws Exception {
defaultConverter.serialize(object, writer, ctx);
}
#Override
public Collection deserialize(ObjectReader reader, Context ctx) throws Exception {
if (reader.getValueType() == ValueType.OBJECT) {
Object object = defaultConverter.getElementConverter().deserialize(reader, ctx);
Collection result = defaultConverter.create();
result.add(object);
return result;
} else return defaultConverter.deserialize( reader, ctx );
}
};
}
}
And then register it
Genson genson = new GensonBuilder()
.withConverterFactory(new SingleObjectAsCollectionFactory())
.create();

Related

Unmarshalling with Jackson "The Json input stream must start with an array of Json objects"

I'm getting an error when unmarshalling files that only contain a single JSON object: "IllegalStateException: The Json input stream must start with an array of Json objects"
I can't find any workaround and I don't understand why it has to be so.
#Bean
public ItemReader<JsonHar> reader(#Value("file:${json.resources.path}/*.json") Resource[] resources) {
log.info("Processing JSON resources: {}", Arrays.toString(resources));
JsonItemReader<JsonHar> delegate = new JsonItemReaderBuilder<JsonHar>()
.jsonObjectReader(new JacksonJsonObjectReader<>(JsonHar.class))
.resource(resources[0]) //FIXME had to force this, but fails anyway because the file is "{...}" and not "[...]"
.name("jsonItemReader")
.build();
MultiResourceItemReader<JsonHar> reader = new MultiResourceItemReader<>();
reader.setDelegate(delegate);
reader.setResources(resources);
return reader;
}
I need a way to unmarshall single object files, what's the point in forcing arrays (which I won't have in my use case)??
I don't understand why it has to be so.
The JsonItemReader is designed to read an array of objects because batch processing is usually about handling data sources with a lot of items, not a single item.
I can't find any workaround
JsonObjectReader is what you are looking for: You can implement it to read a single json object and use it with the JsonItemReader (either at construction time or using the setter). This is not a workaround but a strategy interface designed for specific use cases like yours.
Definitely not ideal #thomas-escolan. As #mahmoud-ben-hassine pointed, ideal would be to code a custom reader.
In case some new SOF users stumble on this question, I leave here a code example on how to do it
Though this may not be ideal, this is how I handled the situation:
#Bean
public ItemReader<JsonHar> reader(#Value("file:${json.resources.path}/*.json") Resource[] resources) {
log.info("Processing JSON resources: {}", Arrays.toString(resources));
JsonItemReader<JsonHar> delegate = new JsonItemReaderBuilder<JsonHar>()
.jsonObjectReader(new JacksonJsonObjectReader<>(JsonHar.class))
.resource(resources[0]) //DEBUG had to force this because of NPE...
.name("jsonItemReader")
.build();
MultiResourceItemReader<JsonHar> reader = new MultiResourceItemReader<>();
reader.setDelegate(delegate);
reader.setResources(Arrays.stream(resources)
.map(WrappedResource::new) // forcing the bride to look good enough
.toArray(Resource[]::new));
return reader;
}
#RequiredArgsConstructor
static class WrappedResource implements Resource {
#Delegate(excludes = InputStreamSource.class)
private final Resource resource;
#Override
public InputStream getInputStream() throws IOException {
log.info("Wrapping resource: {}", resource.getFilename());
InputStream in = resource.getInputStream();
BufferedReader reader = new BufferedReader(new InputStreamReader(in, UTF_8));
String wrap = reader.lines().collect(Collectors.joining())
.replaceAll("[^\\x00-\\xFF]", ""); // strips off all non-ASCII characters
return new ByteArrayInputStream(("[" + wrap + "]").getBytes(UTF_8));
}
}

Dart: Generic which has .fromJson constructor

In Swift I'm used to setting up a protocol JSONInitializable which defines an init?(json: JSON) meaning that all classes and structs that conform to the protocol are initializable with a JSON object.
Is it true that this isn't possible in Dart (with an abstract class) because static methods and factory initializers are not inherited from an abstract class?
The reason I'm asking is because I'm writing a lot of similar methods for API GET requests, which could easily be merged into one, e.g.:
static Future<T> get<T extends JSONInitializable>(int id) async {
final resourceName = T; // TODO: transform to snake case
final uri = Uri.parse("$kApiHostname/api/$resourceName/$id");
final response = await _secureGet(uri);
if (response == null) {
return null;
}
final responseJson = json.decode(response.body);
final model = T.fromJson(responseJson);
return model;
}
But for that to work I'd need to constrain T to a protocol/interface that defines the .fromJson() initializer.
The feature you want is not available (or planned) in Dart, but there have been discussions on it.
Give this issue a thumbs up: https://github.com/dart-lang/language/issues/356
A MANUAL workaround could be having a map of serializers,deserializers like:
//Register object 1 in singleton
JsonMapper.register<MyObject>(JsonObjectMapper(
(mapper, map) => MyObject.fromJson(map),
(mapper, instance) => instance.toJson(),
));
//Register object 2 in singleton...
...
This way you can deserialize and serialize your objects as long as you have them registered without resorting to it's generic type.
The AUTOMATIC (technically code generated) way would be using a package like simple_json to help you workaround this with a little code generation so you don't mess up registering and eliminating mappers.
And the big plus is that the actual code that transforms your objecto from and to JSON is not stored in the object itself but in the generated classes, thus pushing the responsability of serializaition deserailization from the object into the generated code (in a decoupled manner).
Take my advice with a grain of salt as with both approaches you lose static type checking of wether a type has a registered mapper and can be converted.

ServiceStack.Text CSV serialization of IEnumerable<object> ignores custom serialization functions

Firstly, please forgive any rookie mistakes here - I'm not a regular poster I'm afraid.
Now on to the nitty gritty...
I am trying to use ServiceStack.Text to serialize objects to CSV. If I keep it simple, everything works as expected when serializing objects of a known type.
However I want to serialize many objects and I don't know the type at runtime so I am writing a reusable component where all data is treated as a System.Object. We already do this same routine for Json serialization without problems. But CsvSerializer appears to handle objects differently during serialization.
Sample code
public void TestIEnumerableObjectSerialization()
{
var data = GenerateSampleData();
JsConfig<DateTime>.SerializeFn =
time => new DateTime(time.Ticks, DateTimeKind.Utc).ToString("yyyy-MM-dd HH:mm:ss");
var csv = CsvSerializer.SerializeToCsv(data);
Console.WriteLine(csv);
Assert.Equal("DateTime\r\n"
+ "2017-06-14 00:00:00\r\n"
+ "2017-01-31 01:23:45\r\n",
csv);
}
object[] GenerateSampleData()
{
return new object[] {
new POCO
{
DateTime = new DateTime(2017,6,14)
},
new POCO
{
DateTime = new DateTime(2017,1,31, 01, 23, 45)
}
};
}
public class POCO
{
public DateTime DateTime { get; set; }
}
The result of this code is that the custom serialization function is not invoked, and the DateTime is written out using the standard ToString() method.
The cause?
The CsvWriter.Write method is inspecting the type of the records and if the type is Object it is treated as a Dictionary<string, object> and CsvDictionaryWriter generates the output.
In turn, CsvDictionaryWriter uses the ToCsvField() extension method to write each property a record.
The problem is that ToCsvField() converts the value of each property to a string using ToString() meaning no custom serialization is performed.
JsonSerializer uses TypeSerializer.SerializeToString(text) to serialize the properties of an Object using any configured custom serialization functions; but this doesn't happen with CsvSerializer.
A possible solution?
Without complicating CsvSerializer, the ToCsvField() extension method could be updated to use TypeSerializer to handle the serialization to a string. Here is what I've been testing with so far:
public static object ToCsvField(this object text)
{
var textSerialized = TypeSerializer.SerializeToString(text).StripQuotes();
return textSerialized == null || !CsvWriter.HasAnyEscapeChars(textSerialized)
? textSerialized
: string.Concat
(
CsvConfig.ItemDelimiterString,
textSerialized.Replace(CsvConfig.ItemDelimiterString, CsvConfig.EscapedItemDelimiterString),
CsvConfig.ItemDelimiterString
);
}
So far I haven't come across an issue with this change, although someone may prefer not to allocate a new intermediate variable before the return statement.
Hopefully that is enough information, so on to my questions...
Has anyone else experienced this issue?
Am I doing something wrong and should I be serializing Objects a different way?
If this is a suitable fix/implementation of TypeSerializer, what are the chances of this being addressed in an update to ServiceStack.Text? I would raise an issue on GitHub but the ServiceStack.Text repo doesn't let me raise issues.
Thanks in advance.

Spring MVC Test, MockMVC: Conveniently convert objects to/from JSON

I am used to JAX-RS and would like to have similar comfort when sending requests using Spring MVC and working with the responses, i.e. on the client side inside my tests.
On the server (controller) side I'm quite happy with the automatic conversion, i.e. it suffices to just return an object instance and have JSON in the resulting HTTP response sent to the client.
Could you tell me how to work around the manual process of converting objectInstance to jsonString or vice versa in these snippets? If possible, I'd also like to skip configuring the content type manually.
String jsonStringRequest = objectMapper.writeValueAsString(objectInstance);
ResultActions resultActions = mockMvc.perform(post(PATH)
.contentType(MediaType.APPLICATION_JSON)
.content(jsonStringRequest)
)
String jsonStringResponse = resultActions.andReturn().getResponse().getContentAsString();
Some objectInstanceResponse = objectMapper.readValue(jsonStringResponse, Some.class);
For comparison, with JAX-RS client API I can easily send an object using request.post(Entity.entity(objectInstance, MediaType.APPLICATION_JSON_TYPE) and read the response using response.readEntity(Some.class);
if you have lot's of response objects, you could create some generic JsonToObject mapper-factory. It could be then used to detect the object type from a generic response (all response objects inherit from the same generic class) and respond/log properly from a bad mapping attempt.
I do not have a code example at hand, but as a pseudocode:
public abstract GenericResponse {
public String responseClassName = null;
// get/set
}
In the server code, add the name of the actual response object to this class.
The JsonToObject factory
public ConverterFactory<T> {
private T objectType;
public ConverterFactory(T type) {
objectType = type;
}
public T convert(String jsonString) {
// Type check
GenericResponse genResp = mapper.readValue(result.getResponse().getContentAsString(),
GenericResponse.class);
if (objectType.getClass().getSimpleName().equals(genResp.getResponseClassName())) {
// ObjectMapper code
return mapper.readValue(result.getResponse().getContentAsString(),
objectType.class);
} else {
// Error handling
}
}
}
I think this could be extended to be used with annotation to do more automation magic with the response. (start checking with BeanPostProcessor)
#Component
public class AnnotationWorker implements BeanPostProcessor {
#Override
public Object postProcessBeforeInitialization(final Object bean, String name) throws BeansException {
ReflectionUtils.doWithFields(bean.getClass(), field -> {
// make the field accessible if defined private
ReflectionUtils.makeAccessible(field);
if (field.getAnnotation(MyAnnotation.class) != null) {
field.set(bean, log);
}
});
return bean;
}
}
The above code snippet is copied from my current project and it injects to fields, you need to change it so, that it works for methods, eg ... where you may need it.
Having this all implemented may be tricky and can't say it necessarily works even, but it's something to try if you don't mind a bit of educative work.

How can I pass complex objects as arguments to a RESTful service?

I have successfully set up a quick test of creating a "REST-like" service that returns an object serialized to JSON, and that was quite easy and quick (based on this article).
But while returning JSON-ified objects was easy as peach, I have yet to see any examples dealing with input parameters that are not primitives. How can I pass in a complex object as an argument? I am using Apache CXF, but examples using other frameworks like Jackson are welcome too :)
Client side would probably be something like building a javascript object, pass it into JSON.stringify(complexObj), and pass that string as one of the parameters.
The service would probably look something like this
#Service("myService")
class RestService {
#GET
#Produces("application/json")
#Path("/fooBar")
public Result fooBar(#QueryParam("foo") double foo, #QueryParam("bar") double bar,
#QueryParam("object") MyComplex object) throws WebServiceException {
...
}
}
Sending serialized objects as parameters would probably quickly touch the 2KB URL-limit imposed by Internet Explorer. Would you recommend using POST in these cases, and would I need to change much in the function definitions?
After digging a bit I quickly found out there are basically two options:
Option 1
You pass a "wrapper object" containing all the other parameters to the service. You might need to annotate this wrapper class with JAXB annotations like #XmlRootElement in order for this to work with the Jettison based provider, but if you use Jackson in stead there is no need. Just set the content type to the right type and the right message body reader will be invoked.
This will only work for POST type services of course (AFAIK).
Example
This is just an example of turning the service mentioned in the original question into one using a wrapper object.
#Service("myService")
class RestService {
#POST
#Produces("application/json")
#Path("/fooBar")
public Result fooBar(
/**
* Using "" will inject all form params directly into a ParamsWrapper
* #see http://cxf.apache.org/docs/jax-rs-basics.html
*/
#FormParam("") FooBarParamsWrapper wrapper
) throws WebServiceException {
doSomething(wrapper.foo);
}
}
class ParamsWrapper {
double foo, bar;
MyComplexObject object;
}
Option 2
You can provide some special string format that you pack your objects into and then implement either a constructor taking a string, a static valueOf(String s) or a static fromString(String s) in the class that will take this string and create an object from it. Or quite similar, create a ParameterHandler that does exactly the same.
AFAIK, only the second version will allow you to call your services from a browser using JSONP (since JSONP is a trick restricted to GET). I chose this route to be able to pass arrays of complex objects in the URI.
As an example of how this works, take the following domain class and service
Example
#GET
#Path("myService")
public void myService(#QueryParam("a") MyClass [] myVals) {
//do something
}
class MyClass {
public int foo;
public int bar;
/** Deserializes an Object of class MyClass from its JSON representation */
public static MyClass fromString(String jsonRepresentation) {
ObjectMapper mapper = new ObjectMapper(); //Jackson's JSON marshaller
MyClass o= null;
try {
o = mapper.readValue(jsonRepresentation, MyClass.class );
} catch (IOException e) {
throw new WebApplicationException()
}
return o;
}
}
A URI http://my-server.com/myService?a={"foo":1, "bar":2}&a={"foo":100, "bar":200} would in this case be deserialized into an array composed of two MyClass objects.
2019 comment:
Seeing that this answer still gets some hits in 2019, I feel I should comment. In hindsight, I would not recomment option 2, as going through these steps just to be able to be able to do GET calls adds complexity that's probably not worth it. If your service takes such complex input, you will probably not be able to utilize client side caching anyway, due to the number of permutations of your input. I'd just go for configuring proper Cross-Origin-Sharing (CORS) headers on the server and POST the input. Then focus on caching whatever you can on the server.
The accepted answer is missing #BeanParam. See
https://docs.jboss.org/resteasy/docs/3.0-rc-1/javadocs/javax/ws/rs/BeanParam.html
for further details. It allows you to define query params inside a wrapper object.
E.g.
public class TestPOJO {
#QueryParam("someQueryParam")
private boolean someQueryParam;
public boolean isSomeQueryParam() {
return someQueryParam;
}
public boolean setSomeQueryParam(boolean value) {
this.someQueryParam = value;
}
}
... // inside the Resource class
#GET
#Path("test")
public Response getTest(#BeanParam TestPOJO testPOJO) {
...
}
the best and simplest solution is to send your object as a json string and in server side implement a method which will decode that json and map to the specified object as per your need.. and yes it`s better to use POST.