I have a Jersey (1.x) based REST service. It uses Jackson 2.4.4 to generate JSON responses. I need to add a newline character at the end of response (cURL users complain that there's no new line in responses). I am using Jersey pretty-print feature (SerializationFeature.INDENT_OUTPUT).
current: {\n "prop" : "value"\n}
wanted: {\n "prop" : "value"\n}\n
I tried using a custom serializer. I need to add \n only at the end of the root object. Serializer is defined per data type, which means, if an instance of such class is nested in a response, I will get \n in the middle of my JSON.
I thought of subclassing com.fasterxml.jackson.core.JsonGenerator.java, overriding close() where i'd add writeRaw('\n'), but that feels very hacky.
Another idea would be to add Servlet filter which would re-write the response from Jersey Filter, adding the \n and incrementing the contentLenght by 1. Seems not only hacky, but also inefficient.
I could also give up Jersey taking care of serializing the content and do ObjectMapper.writeValue() + "\n", but this is quite intrusive to my code (need to change many places).
What is the clean solution for that problem?
I have found these threads for the same problem, but none of them provides solution:
http://markmail.org/message/nj4aqheqobmt4o5c
http://jackson-users.ning.com/forum/topics/add-newline-after-object-serialization-in-jersey
Update
Finally I went for #arachnid's solution with NewlineAddingPrettyPrinter (also bumper Jackson version to 2.6.2). Sadly, it does not work out of the box with Jaskson as JAX-RS Json provider. Changed PrettyPrinter in ObjectMapper does not get propagated to JsonGenerator (see here why). To make it work, I had to add ResponseFilter which adds ObjectWriterModifier (now I can easily toggle between pretty-print and minimal, based on input param ):
#Provider
public class PrettyPrintFilter extends BaseResponseFilter {
public ContainerResponse filter(ContainerRequest request, ContainerResponse response) {
ObjectWriterInjector.set(new PrettyPrintToggler(true));
return response;
}
final class PrettyPrintToggler extends ObjectWriterModifier {
private static final PrettyPrinter NO_PRETTY_PRINT = new MinimalPrettyPrinter();
private final boolean usePrettyPrint;
public PrettyPrintToggler(boolean usePrettyPrint) {
this.usePrettyPrint = usePrettyPrint;
}
#Override
public ObjectWriter modify(EndpointConfigBase<?> endpoint, MultivaluedMap<String, Object> responseHeaders,
Object valueToWrite, ObjectWriter w, JsonGenerator g) throws IOException {
if (usePrettyPrint) g.setPrettyPrinter(new NewlineAddingPrettyPrinter());
else g.setPrettyPrinter(NO_PRETTY_PRINT);
return w;
}
}
}
Actually, wrapping up (not subclassing) JsonGenerator isn't too bad:
public static final class NewlineAddingJsonFactory extends JsonFactory {
#Override
protected JsonGenerator _createGenerator(Writer out, IOContext ctxt) throws IOException {
return new NewlineAddingJsonGenerator(super._createGenerator(out, ctxt));
}
#Override
protected JsonGenerator _createUTF8Generator(OutputStream out, IOContext ctxt) throws IOException {
return new NewlineAddingJsonGenerator(super._createUTF8Generator(out, ctxt));
}
}
public static final class NewlineAddingJsonGenerator extends JsonGenerator {
private final JsonGenerator underlying;
private int depth = 0;
public NewlineAddingJsonGenerator(JsonGenerator underlying) {
this.underlying = underlying;
}
#Override
public void writeStartObject() throws IOException {
underlying.writeStartObject();
++depth;
}
#Override
public void writeEndObject() throws IOException {
underlying.writeEndObject();
if (--depth == 0) {
underlying.writeRaw('\n');
}
}
// ... and delegate all the other methods of JsonGenerator (CGLIB can hide this if you put in some time)
}
#Test
public void append_newline_after_end_of_json() throws Exception {
ObjectWriter writer = new ObjectMapper(new NewlineAddingJsonFactory()).writer();
assertThat(writer.writeValueAsString(ImmutableMap.of()), equalTo("{}\n"));
assertThat(writer.writeValueAsString(ImmutableMap.of("foo", "bar")), equalTo("{\"foo\":\"bar\"}\n"));
}
A servlet filter isn't necessarily too bad either, although recently the ServletOutputStream interface has been more involved to intercept properly.
I found doing this via PrettyPrinter problematic on earlier Jackson versions (such as your 2.4.4), in part because of the need to go through an ObjectWriter to configure it properly: only fixed in Jackson 2.6. For completeness, this is a working 2.5 solution:
#Test
public void append_newline_after_end_of_json() throws Exception {
// Jackson 2.6:
// ObjectMapper mapper = new ObjectMapper()
// .setDefaultPrettyPrinter(new NewlineAddingPrettyPrinter())
// .enable(SerializationFeature.INDENT_OUTPUT);
// ObjectWriter writer = mapper.writer();
ObjectMapper mapper = new ObjectMapper();
ObjectWriter writer = mapper.writer().with(new NewlineAddingPrettyPrinter());
assertThat(writer.writeValueAsString(ImmutableMap.of()), equalTo("{}\n"));
assertThat(writer.writeValueAsString(ImmutableMap.of("foo", "bar")),
equalTo("{\"foo\":\"bar\"}\n"));
}
public static final class NewlineAddingPrettyPrinter
extends MinimalPrettyPrinter
implements Instantiatable<PrettyPrinter> {
private int depth = 0;
#Override
public void writeStartObject(JsonGenerator jg) throws IOException, JsonGenerationException {
super.writeStartObject(jg);
++depth;
}
#Override
public void writeEndObject(JsonGenerator jg, int nrOfEntries) throws IOException, JsonGenerationException {
super.writeEndObject(jg, nrOfEntries);
if (--depth == 0) {
jg.writeRaw('\n');
}
}
#Override
public PrettyPrinter createInstance() {
return new NewlineAddingPrettyPrinter();
}
}
Not yet tested but the following should work:
public class MyObjectMapper extends ObjectMapper {
_defaultPrettyPrinter = com.fasterxml.jackson.core.util.MinimalPrettyPrinter("\n");
// AND/OR
#Override
protected PrettyPrinter _defaultPrettyPrinter() {
return new com.fasterxml.jackson.core.util.MinimalPrettyPrinter("\n");
}
}
public class JerseyConfiguration extends ResourceConfig {
...
MyObjectMapper mapper = new MyObjectMapper();
mapper.enable(SerializationFeature.INDENT_OUTPUT); //enables pretty printing
// create JsonProvider to provide custom ObjectMapper
JacksonJaxbJsonProvider provider = new JacksonJaxbJsonProvider();
provider.setMapper(mapper);
register(provider); //register so that jersey use it
}
Do not know if this is the "cleanest" solution but it feels less hacky than the others.
Should produce something like
{\n "root" : "1"\n}\n{\n "root2" : "2"\n}
But it seems that does not work if there is only one root element.
Idea is from https://gist.github.com/deverton/7743979
Related
I'm currently working on a project where I need to fetch a large amount of data from the Database and parse it into a specific Json format, I already have built my custom Serializers and Its working properly when i pass a List to Gson. But as I was already working with Streams from my JPA Layer, I thought I could pass the Stream down to the Gson parser so that it could transform it directly to my Json data. But I'm getting an empty Json object instead of a correctly populated one.
So, if anyone could point to me a way to make Gson work with Java 8 Streams or if this isn't possible currently.. i could not find anything on Google, so i came to Stackoverflow.
You could use JsonWriter to streaming your data to output stream:
public void writeJsonStream(OutputStream out, Stream<DataObject> data) throws IOException {
try(JsonWriter writer = new JsonWriter(new OutputStreamWriter(out, "UTF-8"))) {
writer.setIndent(" ");
writer.beginArray();
data.forEach(d -> {
d.beginObject();
d.name("yourField").value(d.getYourField());
....
d.endObject();
});
writer.endArray();
}
}
Note that you're in charge of controling the json structure.
That is, if your DataObject contains nested Object, you have to write beginObject()/endObject() respectively. The same goes for nested array.
It is not as trivial as one would expect, but it can be done in a generic way.
When you look into the Javadoc to TypeAdapterFactory, they provide a very simplistic way of writing a TypeAdapterFactory for a custom type. Alas, it does not work as expected because of problems with element type detection. The proper way to do this can be found in Gson-internal CollectionTypeAdapterFactory. It is quite complex, but taking what's necessary one can come up with something like that:
final class StreamTypeAdapterFactory implements TypeAdapterFactory {
#SuppressWarnings("unchecked")
#Override
public <T> TypeAdapter<T> create(Gson gson, TypeToken<T> typeToken) {
Type type = typeToken.getType();
Class<? super T> rawType = typeToken.getRawType();
if (!Stream.class.isAssignableFrom(rawType)) {
return null;
}
Type elementType = ExtraGsonTypes.getStreamElementType(type, rawType);
TypeAdapter<?> elementAdapter = gson.getAdapter(TypeToken.get(elementType));
return (TypeAdapter<T>) new StreamTypeAdapter<>(elementAdapter);
}
private static class StreamTypeAdapter<E> extends TypeAdapter<Stream<E>> {
private final TypeAdapter<E> elementAdapter;
StreamTypeAdapter(TypeAdapter<E> elementAdapter) {
this.elementAdapter = elementAdapter;
}
public void write(JsonWriter out, Stream<E> value) throws IOException {
out.beginArray();
for (E element : iterable(value)) {
elementAdapter.write(out, element);
}
out.endArray();
}
public Stream<E> read(JsonReader in) throws IOException {
Stream.Builder<E> builder = Stream.builder();
in.beginArray();
while (in.hasNext()) {
builder.add(elementAdapter.read(in));
}
in.endArray();
return builder.build();
}
}
private static <T> Iterable<T> iterable(Stream<T> stream) {
return stream::iterator;
}
}
The ExtraGsonTypes is a special class that I used to circumvent package-private access to $Gson$Types.getSupertype method. It's a hack that works if you're not using JDK 9's modules - you simply place this class in the same package as $Gson$Types:
package com.google.gson.internal;
import java.lang.reflect.*;
import java.util.stream.Stream;
public final class ExtraGsonTypes {
public static Type getStreamElementType(Type context, Class<?> contextRawType) {
return getContainerElementType(context, contextRawType, Stream.class);
}
private static Type getContainerElementType(Type context, Class<?> contextRawType, Class<?> containerSupertype) {
Type containerType = $Gson$Types.getSupertype(context, contextRawType, containerSupertype);
if (containerType instanceof WildcardType) {
containerType = ((WildcardType)containerType).getUpperBounds()[0];
}
if (containerType instanceof ParameterizedType) {
return ((ParameterizedType) containerType).getActualTypeArguments()[0];
}
return Object.class;
}
}
(I filed an issue about that in GitHub)
You use it in the following way:
Gson gson = new GsonBuilder()
.registerTypeAdapterFactory(new StreamTypeAdapterFactory())
.create();
System.out.println(gson.toJson(Stream.of(1, 2, 3)));
I have CSV files without headers. Since I'm using 'useMaps' I want to specify the headers dynamically. If I set headers statically and then use in route it works fine as below Approach 1 -
#Component
public class BulkActionRoutes extends RouteBuilder {
#Override
public void configure() throws Exception {
CsvDataFormat csv = new CsvDataFormat(",");
csv.setUseMaps(true);
ArrayList<String> list = new ArrayList<String>();
list.add("DeviceName");
list.add("Brand");
list.add("status");
list.add("type");
list.add("features_c");
list.add("battery_c");
list.add("colors");
csv.setHeader(list);
from("direct:bulkImport")
.convertBodyTo(String.class)
.unmarshal(csv)
.split(body()).streaming()
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
GenericObjectModel model = null;
HashMap<String, String> csvRecord = (HashMap<String, String>)exchange.getIn().getBody();
}
});
}
}
However, if the list is passed via Camel headers as below then it does not work Approach 2 -
#Component
public class BulkActionRoutes extends RouteBuilder {
#Override
public void configure() throws Exception {
CsvDataFormat csv = new CsvDataFormat(",");
csv.setUseMaps(true);
from("direct:bulkImport")
.convertBodyTo(String.class)
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
ArrayList<String> fileHeaders = (ArrayList<String>)headers.get(Constants.FILE_HEADER_LIST);
if (fileHeaders != null && fileHeaders.size() > 0) {
csv.setHeader(fileHeaders);
}
}
})
.unmarshal(csv)
.split(body()).streaming()
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
GenericObjectModel model = null;
HashMap<String, String> csvRecord = (HashMap<String, String>)exchange.getIn().getBody();
}
});
}
}
What could be missing in the Approach 2?
The big difference between approach 1 and 2 is the scope.
In approach 1 you fully configure the CSV data format. This is all done when the Camel Context is created, since the data format is shared within the Camel Context. When messages are processed, it is the same config for all messages.
In approach 2 you just configure the basics globally. The header configuration is within the route and therefore can change for every single message. Every message would overwrite the header configuration of the context-global data format instance.
Without being sure about this, I guess that it is not possible to change a context-global DataFormat inside the routes.
What would you expect (just for example) when messages are processed in parallel? They would overwrite the header config against each other.
As an alternative, you could use a POJO where you can do your dynamic marshal / unmarshal from Java code.
In my rest client i am passing the below JSON request data:
{
"jobName":"test1",
"source":{ "name":"prod1","type":"teradata"},
"target":{ "name":"prod2","type":"teradata"},
"objects":{ "name":"table1"}<br/>
}
junkdata ; ##%$##%
So the extra "junkdata ; ##%$##%" not got validated by the rest client or by the spring jackson out-of-the box message converter.
I did debug the code, the spring HttpServletRequest body has the complete data including the junk data. As such its not failing, the spring is ignoring the junk data and converting the starting JSON data into Java object.
I did try by adding annotations like #JsonFormat for #RequestBody in rest controller calls (#RestController). But its not validating seems the Spring out-of-the box message converter jackson is not validating the incoming request JSON data properly.
Now this issue (failing on trailing tokens or data) is fixed in the spring jackson https://github.com/FasterXML/jackson-databind/issues/1583
using DeserializationFeature.FAIL_ON_TRAILING_TOKENS, Here is the code sample to fix the issue:
#Configuration
public class RestControllerConfiguration extends WebMvcConfigurationSupport
{
#Override
public void configureMessageConverters(List<HttpMessageConverter<?>> converters) {
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.configure(DeserializationFeature.FAIL_ON_TRAILING_TOKENS, true);
converters.add(new MappingJackson2HttpMessageConverter(objectMapper));
}
}
After working on different ways i got the solution using Google gson.jar, #Pete yes i have validate the JSON input which is invalid.
The google gson api is validating it properly, we need to use the custom message converter to validate it in the rest WebMvcConfigurationSupport class.
#Configuration
#ComponentScan(basePackages = { "com.teradata.datamovement.rest.controllers",
"com.teradata.rest.controller" })
public class RestControllerConfiguration extends WebMvcConfigurationSupport
{
#Override
public void configureMessageConverters(List<HttpMessageConverter<?>> converters) {
log.debug("Adding custom message converter.");
converters.add(new AbstractHttpMessageConverter<Object>(MediaType.APPLICATION_JSON, new MediaType("application", "*+json")){
#Override
protected Object readInternal(Class<? extends Object> clazz,
HttpInputMessage inputMessage) throws IOException, HttpMessageNotReadableException {
try{
log.debug("Converting and validating the http request body data.");
String httpRequestBody = convertStreamToString(inputMessage.getBody());
log.debug("Http request body data:"+httpRequestBody);
return new Gson().fromJson(httpRequestBody, clazz);
}
catch(JsonSyntaxException e){
throw new HttpMessageNotReadableException("Invalid input JSON data: " + e.getMessage(), e);
}
}
private String convertStreamToString(InputStream is) throws IOException {
if (is != null) {
Writer writer = new StringWriter();
char[] buffer = new char[1024];
try {
Reader reader = new BufferedReader(new InputStreamReader(is, "UTF-8"));
int n;
while ((n = reader.read(buffer)) != -1) {
writer.write(buffer, 0, n);
}
} finally {
is.close();
}
return writer.toString();
} else {
return "";
}
}
#Override
protected boolean supports(Class clazz) {
return true;
}
#Override
protected void writeInternal(Object t, HttpOutputMessage outputMessage)
throws IOException, HttpMessageNotWritableException {
outputMessage.getBody().write(new Gson().toJson(t).getBytes());
}
});
}
}
But the weird thing i have noticed is that its working only if i make it as anonymous class or by adding the class with in the same file. If i create this custom message converter out side this RestControllerConfiguration.java file, then its not validating it.
Here is the example:
{
"jobName":"test1",
"source":{ "name":"prod1","type":"teradata"},
"target":{ "name":"prod2","type":"teradata"},
"objects":{ "name":"table1"}
}
junkdata ; ##%$##%
This will get validated, and will throw error like
{"message":"Invalid input JSON data: com.google.gson.stream.MalformedJsonException: Expected EOF at line 7 column 1; nested exception is com.google.gson.JsonSyntaxException: com.google.gson.stream.MalformedJsonException: Expected EOF at line 7 column 1"}
I found an answer similar to this question but it isn't working when posting JSON data. I have the following:
#ControllerAdvice
public class ControllerConfig {
#InitBinder
public void initBinder ( WebDataBinder binder ) {
StringTrimmerEditor stringtrimmer = new StringTrimmerEditor(true);
binder.registerCustomEditor(String.class, stringtrimmer);
}
}
I know that the code is being reached during binding via debugging but when I pass in data like:
{ "companyId": " ABC "}
ABC isn't actually being trimmed during binding. My guess is that this only works with request params and not raw JSON bodies but not sure about that. If that is the case, is there something I can do that is similar?
Create this JsonDeserializer
public class WhiteSpaceRemovalDeserializer extends JsonDeserializer<String> {
#Override
public String deserialize(JsonParser jp, DeserializationContext ctxt) {
// This is where you can deserialize your value the way you want.
// Don't know if the following expression is correct, this is just an idea.
return jp.getCurrentToken().asText().trim();
}
}
and set this to your property
#JsonDeserialize(using=WhiteSpaceRemovalSerializer.class)
public void setAString(String aString) {
// body
}
Try this,
Create a class.
Annotate the class with #JsonComponent
extend the JsonDeserializer
and, add your trimming logic in the overridden method,
this will automatically trim the whitespaces in the json request, when it hits the controller, no external properties needed to activate this.
#JsonComponent
public class WhiteSpaceRemover extends JsonDeserializer<String> {
#Override
public String deserialize(JsonParser arg0, DeserializationContext arg1)
throws IOException, JsonProcessingException {
return arg0.getValueAsString().trim();
}
}
I have JAXB objects created from a schema. While marshalling, the xml elements are getting annotated with ns2. I have tried all the options that exist over the net for this problem, but none of them works. I cannot modify my schema or change package-info.java. Please help
After much research and tinkering I have finally managed to achieve a solution to this problem. Please accept my apologies for not posting links to the original references - there are many and I wasn't taking notes - but this one was certainly useful.
My solution uses a filtering XMLStreamWriter which applies an empty namespace context.
public class NoNamesWriter extends DelegatingXMLStreamWriter {
private static final NamespaceContext emptyNamespaceContext = new NamespaceContext() {
#Override
public String getNamespaceURI(String prefix) {
return "";
}
#Override
public String getPrefix(String namespaceURI) {
return "";
}
#Override
public Iterator getPrefixes(String namespaceURI) {
return null;
}
};
public static XMLStreamWriter filter(Writer writer) throws XMLStreamException {
return new NoNamesWriter(XMLOutputFactory.newInstance().createXMLStreamWriter(writer));
}
public NoNamesWriter(XMLStreamWriter writer) {
super(writer);
}
#Override
public NamespaceContext getNamespaceContext() {
return emptyNamespaceContext;
}
}
You can find a DelegatingXMLStreamWriter here.
You can then filter the marshalling xml with:
// Filter the output to remove namespaces.
m.marshal(it, NoNamesWriter.filter(writer));
I am sure there are more efficient mechanisms but I know this one works.
For me, only changing the package-info.java class worked like a charm, exactly as zatziky stated :
package-info.java
#javax.xml.bind.annotation.XmlSchema
(namespace = "http://example.com",
xmlns = {#XmlNs(prefix = "", namespaceURI = "http://example.com")},
elementFormDefault = javax.xml.bind.annotation.XmlNsForm.QUALIFIED)
package my.package;
import javax.xml.bind.annotation.XmlNs;
You can let the namespaces be written only once. You will need a proxy class of the XMLStreamWriter and a package-info.java. Then you will do in your code:
StringWriter stringWriter = new StringWriter();
XMLStreamWriter writer = new Wrapper((XMLStreamWriter) XMLOutputFactory
.newInstance().createXMLStreamWriter(stringWriter));
JAXBContext jaxbContext = JAXBContext.newInstance(Collection.class);
Marshaller jaxbMarshaller = jaxbContext.createMarshaller();
jaxbMarshaller.setProperty(Marshaller.JAXB_FORMATTED_OUTPUT, Boolean.TRUE);
jaxbMarshaller.marshal(books, writer);
System.out.println(stringWriter.toString());
Proxy class (the important method is "writeNamespace"):
class WrapperXMLStreamWriter implements XMLStreamWriter {
private final XMLStreamWriter writer;
public WrapperXMLStreamWriter(XMLStreamWriter writer) {
this.writer = writer;
}
//keeps track of what namespaces were used so that not to
//write them more than once
private List<String> namespaces = new ArrayList<String>();
public void init(){
namespaces.clear();
}
public void writeStartElement(String localName) throws XMLStreamException {
init();
writer.writeStartElement(localName);
}
public void writeStartElement(String namespaceURI, String localName) throws XMLStreamException {
init();
writer.writeStartElement(namespaceURI, localName);
}
public void writeStartElement(String prefix, String localName, String namespaceURI) throws XMLStreamException {
init();
writer.writeStartElement(prefix, localName, namespaceURI);
}
public void writeNamespace(String prefix, String namespaceURI) throws XMLStreamException {
if(namespaces.contains(namespaceURI)){
return;
}
namespaces.add(namespaceURI);
writer.writeNamespace(prefix, namespaceURI);
}
// .. other delegation method, always the same pattern: writer.method() ...
}
package-info.java:
#XmlSchema(elementFormDefault=XmlNsForm.QUALIFIED, attributeFormDefault=XmlNsForm.UNQUALIFIED ,
xmlns = {
#XmlNs(namespaceURI = "http://www.w3.org/2001/XMLSchema-instance", prefix = "xsi")})
package your.package;
import javax.xml.bind.annotation.XmlNs;
import javax.xml.bind.annotation.XmlNsForm;
import javax.xml.bind.annotation.XmlSchema;
You can use the NamespacePrefixMapper extension to control the namespace prefixes for your use case. The same extension is supported by both the JAXB reference implementation and EclipseLink JAXB (MOXy).
http://wiki.eclipse.org/EclipseLink/Release/2.4.0/JAXB_RI_Extensions/Namespace_Prefix_Mapper
Every solution requires complex overwriting or annotations which seems not to work with recent version. I use a simpler approach, just by replacing the annoying namespaces. I wish Google & Co would use JSON and get rid of XML.
kml.marshal(file);
String kmlContent = FileUtils.readFileToString(file, "UTF-8");
kmlContent = kmlContent.replaceAll("ns2:","").replace("<kml xmlns:ns2=\"http://www.opengis.net/kml/2.2\" xmlns:ns3=\"http://www.w3.org/2005/Atom\" xmlns:ns4=\"urn:oasis:names:tc:ciq:xsdschema:xAL:2.0\" xmlns:ns5=\"http://www.google.com/kml/ext/2.2\">", "<kml>");
FileUtils.write(file, kmlContent, "UTF-8");