SpringBatch ItemProcessor: process a List<?> not one Item - json

I have to modify a process made with SpringBatch, the procedure it's easy.
Actually, the program reads records from a database and exports the results to XML files (one by each table)
Now, I want to write JSON files instead XML files, I didn't find how to make it possible, but reading and reading I have something close to that I want.
I wrote an ItemProcessor class like this
#Component("jSONObjectProcessor")
public class JSONObjectProcessor implements ItemProcessor<Object, String> {
private Gson gson = new Gson();
private List<Object> array = new ArrayList<Object>();
#Override
public String process(Object item) throws Exception {
array.add(item);
return gson.toJson(array);
}
}
Obviously, if I have 6 items; this going to return 6 List, like it does right now
1st item
[
{
"number":0,
"string":"abc",
"desc":"abcdefg"
}
]
2nd item
[
{
"number":0,
"string":"abc",
"desc":"abcdefg"
},
{
"number":1000,
"string":"xyz",
"desc":"uvwxyz"
}
]
//more lists by the total of items
To write the files I'm using org.springframework.batch.item.file.FlatFileItemWriter class.
I want to find the way to return all the items in a List and give it JSON form and write this json in the file. I'm in the correct way or there are another, more elegant form. It's possible?
Thanks!
Update
I have made the changes (thanks #Sanj), but I miss the comma (,) delimiter between each object.
My ItemWriter it's defined like this
<bean id="itemWriterRegConstantes" scope="step"
class="org.springframework.batch.item.file.FlatFileItemWriter">
<property name="resource" value="#{jobParameters['fileOutput']}" />
<property name="shouldDeleteIfExists" value="true" />
<property name="lineAggregator">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineAggregator">
<property name="delimiter" value="," />
</bean>
</property>
<property name="footerCallback" ref="headerFooterCallback" />
<property name="headerCallback" ref="headerFooterCallback" />
</bean>
The output file now looks like this
[
{"number":0,"string":"abcd","desc":"efgh"} //no comma
{"number":1000,"string":"xyz","valor":"xyzw"}
]
How add it?
Additionaly, how can I print all the content in a single line? (to minify the content) It's possible?
My Solution
I had to create my own class (really I made change to the FlatFileItemWriter SpringBatch class, it's here
Output: a file with an one line JSON array content.
Thanks!

Return single json for every item from ItemProcessor
#Component("jSONObjectProcessor")
public class JSONObjectProcessor implements ItemProcessor<Object, String> {
private Gson gson = new Gson();
#Override
public String process(Object item) throws Exception {
return gson.toJson(item);
}
}
Create header and footer call backs. Basically they will be used to start and close the array respectively.
public class JSONHeaderFooterCallBack implements FlatFileHeaderCallback, FlatFileFooterCallback{
#Override
public void writeHeader(Writer writer) throws IOException {
writer.write("[" + System.getProperty("line.separator"));
}
#Override
public void writeFooter(Writer writer) throws IOException {
writer.write("]");
}
}
Associate the callbacks with FlatFileItemWriter
FlatFileItemWriter<String> writer = new FlatFileItemWriter<String>();
//Setting header and footer.
JSONHeaderFooterCallBack headerFooterCallback = new JSONHeaderFooterCallBack();
writer.setHeaderCallback(headerFooterCallback);
writer.setFooterCallback(headerFooterCallback);
writer.setResource(new FileSystemResource(System.getProperty("user.dir") + "/output.json"));
Now you can use "writer" to write all records as a json array to a file.
---update--
Use CustomLineAggregator to append comma at the end of every record:
public class CustomLineAggregator<String> implements LineAggregator<String> {
#Override
public String aggregate(String item) {
return item+",";
}
}

Related

Wrap a primitive returned from a Spring controller into json

For now, I am using something like this:
#RequestBody
#RequestMapping("whatever")
public ObjectWrapper<Integer> foo() {
return new ObjectWrapper<>(42);
}
What I would like to do is to rewrite the method in the following way
#RequestBody
#RequestMapping("whatever")
public int foo() {
return 42;
}
and get 42 (or any other primitive) wrapped into ObjectWrapper before it gets serialized (by Jackson) and gets written into response. I wonder if it is actually possible and, if so, how to do that.
As I have misunderstood your question, I updated my answer:
public class CustomObjectMapper extends ObjectMapper {
public CustomObjectMapper() {
super();
super.configure(SerializationFeature.WRAP_ROOT_VALUE, true);
}
}
Add to default message converter:
<mvc:message-converters>
<bean class="org.springframework.http.converter.json.MappingJackson2HttpMessageConverter">
<property name="objectMapper" ref="jacksonObjectMapper" />
</bean>
</mvc:message-converters>
<bean id="jacksonObjectMapper" class="com.mysite.CustomObjectMapper" />
However this might not produce the output you desired.
Best thing is to write your own serializer and use it with your custom object mapper and wrap primitives in your serializer.
Here is something related: https://github.com/FasterXML/jackson-databind/issues/34

Spring Data Rest - Configure pagination

Using Spring Data REST with JPA in version 2.1.0.
How can I configure the pagination in order to have the page argument starting at index 1 instead of 0 ?
I have tried setting a custom HateoasPageableHandlerMethodArgumentResolver with an mvc:argument-resolvers, but that doesn't work:
<mvc:annotation-driven>
<mvc:argument-resolvers>
<bean class="org.springframework.data.web.HateoasPageableHandlerMethodArgumentResolver">
<property name="oneIndexedParameters" value="true"/>
</bean>
</mvc:argument-resolvers>
</mvc:annotation-driven>
Note that this behaviour is perfectly coherent with the documentation for mvc:argument-resolver that says:
Using this option does not override the built-in support for
resolving handler method arguments. To customize the built-in support
for argument resolution configure RequestMappingHandlerAdapter
directly.
But how can I achieve this ? If possible, in a clean and elegant way ?
The easiest way to do so is to subclass RepositoryRestMvcConfiguration and include your class into your configuration:
class CustomRestMvcConfiguration extends RepositoryRestMvcConfiguration {
#Override
#Bean
public HateoasPageableHandlerMethodArgumentResolver pageableResolver() {
HateoasPageableHandlerMethodArgumentResolver resolver = super.pageableResolver();
resolver.setOneIndexedParameters(true);
return resolver;
}
}
In your XML configuration, replace:
<bean class="….RepositoryRestMvcConfiguration" />
with
<bean class="….CustomRestMvcConfiguration" />
or import the custom class instead of the standard one in your JavaConfig file.
I have configured the RequestMappingHandlerAdapter using a BeanPostProcessor, however I believe that's neither clean, nor elegant. That looks more like a hack. There must be a better way ! I'm giving the code below just for reference.
public class RequestMappingHandlerAdapterCustomizer implements BeanPostProcessor {
public Object postProcessAfterInitialization(Object bean, String beanName) throws BeansException {
if (bean instanceof RequestMappingHandlerAdapter) {
RequestMappingHandlerAdapter adapter = (RequestMappingHandlerAdapter)bean;
List<HandlerMethodArgumentResolver> customArgumentResolvers = adapter.getCustomArgumentResolvers();
if(customArgumentResolvers != null) {
for(HandlerMethodArgumentResolver customArgumentResolver : customArgumentResolvers) {
if(customArgumentResolver instanceof HateoasPageableHandlerMethodArgumentResolver) {
HateoasPageableHandlerMethodArgumentResolver hateoasPageableHandlerMethodArgumentResolver = (HateoasPageableHandlerMethodArgumentResolver)customArgumentResolver;
hateoasPageableHandlerMethodArgumentResolver.setOneIndexedParameters(true);
}
}
}
}
return bean;
}
public Object postProcessBeforeInitialization(Object bean, String beanName)
throws BeansException {
return bean;
}
}
<beans ...>
<bean class="util.spring.beanpostprocessors.RequestMappingHandlerAdapterCustomizer" />
</beans>
I use to do it using a customizer, which is something that they keep adding for more and more components with every new version:
#Bean
public PageableHandlerMethodArgumentResolverCustomizer pageableResolverCustomizer() {
return resolver -> resolver.setOneIndexedParameters(true);
}
You can put this in any #Configuration class, but ideally you should put it (with any other customization) in one that implements RepositoryRestConfigurer.

How to exclude certain modelAttributes in JSON output (using ContentNegotiatingViewResolver)

I have a Spring MVC 3.1.0 project and I have configured a ContentNegotiatingViewResolver bean to automatically generate JSON output for a given endpoint (which uses org.springframework.web.servlet.view.json.MappingJacksonJsonView).
I have a few controller methods which add data to the JSP page (via model.addAttribute("foo", fooService.getFoo());) that I don't want to appear in the JSON output.
I have tried adding a #JsonIgnore annotation to my service method getFoo() (which returns a Map<String, String>) but it doesn't work. I still see the foo object being marshalled in my JSON output when I hit that controller.
Can anyone suggest another way of achieving this or tell me why the JsonIgnore annotation is not working?
MappingJacksonJsonView serializes all the contents of the model into a json - all the objects that you have placed in your model object, so it does not matter if you have marked one of the service methods with #JsonIgnore, as long it ends up in the model which it does because of the call to model.addAttribute("foo".. it would get serialized. The fix could be simply to not add the model attribute, or to use #ResponseBody which will give you control over the specific response object that is being serialized.
Another option is to specify the exact keys that you will be using when configuring MappingJacksonJsonView:
<bean class="org.springframework.web.servlet.view.json.MappingJacksonJsonView" name="jsonView">
<property name="modelKeys">
<set>
<value>model1</value>
<value>model2</value>
</set>
</property>
</bean>
Extend MappingJackson2JsonView class and override filterMap(Map model) method.
In the filterMap method filter out keys with the name of modelAttributes you need to exclude.
public class MappingJackson2JsonViewExt extends MappingJackson2JsonView {
private static final Set<String> EXCLUDED_KEYS = new HashSet<>();
public static void excludeModelKey(final String key) {
EXCLUDED_KEYS.add(key);
}
#Override
protected Object filterModel(final Map<String, Object> model) {
final Map<String, Object> filteredModel = model.entrySet().stream()
.filter(e -> {
final String key = e.getKey();
return !EXCLUDED_KEYS.contains(key);
})
.collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue));
return super.filterModel(filteredModel);
}
}

When using Spring MVC for REST, how do you enable Jackson to pretty-print rendered JSON?

While developing REST services using Spring MVC, I would like render JSON 'pretty printed' in development but normal (reduced whitespace) in production.
If you are using Spring Boot 1.2 or later the simple solution is to add
spring.jackson.serialization.INDENT_OUTPUT=true
to the application.properties file. This assumes that you are using Jackson for serialization.
If you are using an earlier version of Spring Boot then you can add
http.mappers.json-pretty-print=true
This solution still works with Spring Boot 1.2 but it is deprecated and will eventually be removed entirely. You will get a deprecation warning in the log at startup time.
(tested using spring-boot-starter-web)
I had an answer when I posted this question, but I thought I'd post it anyway in case there are better alternative solutions. Here was my experience:
First thing's first. The MappingJacksonHttpMessageConverter expects you to inject a Jackson ObjectMapper instance and perform Jackson configuration on that instance (and not through a Spring class).
I thought it would be as easy as doing this:
Create an ObjectMapperFactoryBean implementation that allows me to customize the ObjectMapper instance that can be injected into the MappingJacksonHttpMessageConverter. For example:
<bean id="jacksonHttpMessageConverter" class="org.springframework.http.converter.json.MappingJacksonHttpMessageConverter">
<property name="objectMapper">
<bean class="com.foo.my.ObjectMapperFactoryBean">
<property name="prettyPrint" value="${json.prettyPrint}"/>
</bean>
</property>
</bean>
And then, in my ObjectMapperFactoryBean implementation, I could do this (as has been documented as a solution elsewhere on SO):
ObjectMapper mapper = new ObjectMapper();
mapper.configure(SerializationConfig.Feature.INDENT_OUTPUT, isPrettyPrint());
return mapper;
But it didn't work. And trying to figure out why is a nightmare. It is a major test of patience to figure Jackson out. Looking at its source code only confuses you further as it uses outdated and obtuse forms of configuration (integer bitmasks for turning on/off features? Are you kidding me?)
I essentially had to re-write Spring's MappingJacksonHttpMessageConverter from scratch, and override its writeInternal implementation to be the following:
#Override
protected void writeInternal(Object o, HttpOutputMessage outputMessage)
throws IOException, HttpMessageNotWritableException {
JsonEncoding encoding = getEncoding(outputMessage.getHeaders().getContentType());
JsonGenerator jsonGenerator =
getObjectMapper().getJsonFactory().createJsonGenerator(outputMessage.getBody(), encoding);
try {
if (this.prefixJson) {
jsonGenerator.writeRaw("{} && ");
}
if (isPrettyPrint()) {
jsonGenerator.useDefaultPrettyPrinter();
}
getObjectMapper().writeValue(jsonGenerator, o);
}
catch (JsonGenerationException ex) {
throw new HttpMessageNotWritableException("Could not write JSON: " + ex.getMessage(), ex);
}
}
The only thing I added to the existing implementation is the following block:
if (isPrettyPrint()) {
jsonGenerator.useDefaultPrettyPrinter();
}
isPrettyPrint() is just a JavaBeans compatible getter w/ matching setter that I added to my MappingJacksonHttpMessageConverter subclass.
Only after jumping through these hoops was I able to turn on or off pretty printing based on my ${json.prettyPrint} value (that is set as a property depending on how the app is deployed).
I hope this helps someone out in the future!
When you are using Jackson 2.0.0, you can do it in a way Les wanted to.
I currently use RC3 and the configuration seems to be working as expected.
ObjectMapper jacksonMapper = new ObjectMapper();
jacksonMapper.configure(SerializationFeature.INDENT_OUTPUT, true);
translates
{"foo":"foo","bar":{"field1":"field1","field2":"field2"}}
into
{
"foo" : "foo",
"bar" : {
"field1" : "field1",
"field2" : "field2"
}
}
Might I suggest this approach, it is valid with Spring 4.0.x and possibly older versions.
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import java.util.List;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.http.converter.HttpMessageConverter;
import org.springframework.http.converter.json.MappingJackson2HttpMessageConverter;
import org.springframework.web.servlet.config.annotation.EnableWebMvc;
import org.springframework.web.servlet.config.annotation.WebMvcConfigurerAdapter;
#Configuration
#EnableWebMvc
public class WebMvcConfig extends WebMvcConfigurerAdapter {
#Bean
public MappingJackson2HttpMessageConverter mappingJackson2HttpMessageConverter() {
MappingJackson2HttpMessageConverter mappingJackson2HttpMessageConverter = new MappingJackson2HttpMessageConverter();
mappingJackson2HttpMessageConverter.setObjectMapper(objectMapper());
return mappingJackson2HttpMessageConverter;
}
#Bean
public ObjectMapper objectMapper() {
ObjectMapper objMapper = new ObjectMapper();
objMapper.enable(SerializationFeature.INDENT_OUTPUT);
return objMapper;
}
#Override
public void configureMessageConverters(List<HttpMessageConverter<?>> converters) {
super.configureMessageConverters(converters);
converters.add(mappingJackson2HttpMessageConverter());
}
}
Thanks to Willie Wheeler for the solution: Willie Wheeler's Spring blog
How do I make Jackson pretty-print the JSON content it generates?
Here's a simple example:
Original JSON Input:
{"one":"AAA","two":["BBB","CCC"],"three":{"four":"DDD","five":["EEE","FFF"]}}
Foo.java:
import java.io.FileReader;
import org.codehaus.jackson.map.ObjectMapper;
import org.codehaus.jackson.map.ObjectWriter;
public class Foo
{
public static void main(String[] args) throws Exception
{
ObjectMapper mapper = new ObjectMapper();
MyClass myObject = mapper.readValue(new FileReader("input.json"), MyClass.class);
// this is Jackson 1.x API only:
ObjectWriter writer = mapper.defaultPrettyPrintingWriter();
// ***IMPORTANT!!!*** for Jackson 2.x use the line below instead of the one above:
// ObjectWriter writer = mapper.writer().withDefaultPrettyPrinter();
System.out.println(writer.writeValueAsString(myObject));
}
}
class MyClass
{
String one;
String[] two;
MyOtherClass three;
public String getOne() {return one;}
void setOne(String one) {this.one = one;}
public String[] getTwo() {return two;}
void setTwo(String[] two) {this.two = two;}
public MyOtherClass getThree() {return three;}
void setThree(MyOtherClass three) {this.three = three;}
}
class MyOtherClass
{
String four;
String[] five;
public String getFour() {return four;}
void setFour(String four) {this.four = four;}
public String[] getFive() {return five;}
void setFive(String[] five) {this.five = five;}
}
Output:
{
"one" : "AAA",
"two" : [ "BBB", "CCC" ],
"three" : {
"four" : "DDD",
"five" : [ "EEE", "FFF" ]
}
}
If this approach doesn't exactly fit your needs, if you search the API docs v1.8.1 for "pretty", it'll turn up the relevant components available. If you use API version 2.x then look instead at the newer API 2.1.0 docs.
Pretty print will be enable by adding and configure the MappingJackson2HttpMessageConverter converter. Disable prettyprint within production environment.
Message converter configuration
<mvc:annotation-driven>
<mvc:message-converters>
<bean id="jacksonHttpMessageConverter"
class="org.springframework.http.converter.json.MappingJackson2HttpMessageConverter">
<property name="prettyPrint" value="${json.prettyPrint}" />
</bean>
</mvc:message-converters>
</mvc:annotation-driven>
Based on baeldung this could be a nice idea using java 8:
#Override
public void extendMessageConverters(List<HttpMessageConverter<?>> converters) {
Optional<HttpMessageConverter<?>> converterFound;
converterFound = converters.stream().filter(c -> c instanceof AbstractJackson2HttpMessageConverter).findFirst();
if (converterFound.isPresent()) {
final AbstractJackson2HttpMessageConverter converter;
converter = (AbstractJackson2HttpMessageConverter) converterFound.get();
converter.getObjectMapper().enable(SerializationFeature.INDENT_OUTPUT);
converter.getObjectMapper().enable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES);
}
}
I had trouble getting the custom MappingJacksonHttpMessageConverter to work as suggested above but I was finally able to get it to work after struggling w/ the configuration. From the code stand point I did exactly what was mentioned above but I had to add the following configuration to my springapp-servlet.xml to get it to work.
I hope this helps others who are looking to implement the same.
<bean class="org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter">
<property name="messageConverters">
<list>
<ref bean="jsonConverter" />
</list>
</property>
</bean>
<bean id="jsonConverter" class="com.xxx.xxx.xxx.common.PrettyPrintMappingJacksonHttpMessageConverter">
<property name="supportedMediaTypes" value="application/json" />
<property name="prettyPrint" value="true" />
</bean>
Jackson 2 has a nicer API, agreed, but it won't resolve this problem in a Spring MVC environment given Spring MVC uses ObjectMapper#writeValue(JsonGenerator, Object) to write objects out as JSON. This writeValue variant does not apply ObjectMapper serialization features such as INDENT_OUTPUT in either Jackson 1.x or 2.0.
I do think this is somewhat confusing. Since we use the ObjectMapper to construct JsonGenerators, I'd expect returned generators to be initialized based on configured ObjectMapper settings. I reported this as a issue against Jackson 2.0 here: https://github.com/FasterXML/jackson-databind/issues/12.
Les's suggestion of calling JsonGenerator#useDefaultPrettyPrinter based on the value of a prettyPrint flag is about the best we can do at the moment. I've gone ahead and created a Jackson2 HttpMessageConverter that does this based on the enabled status of the INDENT_OUTPUT SerializationFeature: https://gist.github.com/2423129.
I would make that a rendering issue, not the concern of the REST service.
Who's doing the rendering? Let that component format the JSON. Maybe it can be two URLs - one for production and another for development.

Spring3 REST Web Services with Jackson JSONViews

I got a plain spring3 web project set up and have a controller method like this:
#RequestMapping(method = RequestMethod.GET, value = "/book/{id}", headers = "Accept=application/json,application/xml")
public #ResponseBody
Book getBook(#PathVariable final String id)
{
logger.warn("id=" + id);
return new Book("12345", new Date(), "Sven Haiges");
}
It returns a new book object which will be transformed to JSON or XML because of the transformers I setup in the spring config:
<bean class="org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter">
<property name="messageConverters">
<list>
<ref bean="jsonConverter" />
<ref bean="marshallingConverter" />
</list>
</property>
</bean>
JSON generation (and XML) all works, but I would like to be able to define multiple views for the data. For example I'd like to specify a detailed view with less properties in the exposed JSON/XML and a detailed view with the full set of properties.
Using Jackson's ObjectMapper this is possible like this:
objectMapper.writeValueUsingView(out, beanInstance, ViewsPublic.class);
Is there a way I can configure Spring to use a specific VIEW (detailed/summary)? The only way to achieve this right now is to use different DTOs returned from my controller methods.
Thanx!
If you need that level of control, then you need to do it yourself.
So rather than using #ResponseBody, instead use your own ObjectMapper to write the response manually, e.g.
private final ObjectMapper objectMapper = new ObjectMapper();
#RequestMapping(method = RequestMethod.GET, value = "/book/{id}", headers = "Accept=application/json,application/xml")
public void getBook(#PathVariable final String id, HttpServletResponse httpResponse)
{
logger.warn("id=" + id);
Book book = new Book("12345", new Date(), "Sven Haiges");
objectMapper.writeValueUsingView(httpResponse.getWriter(), book, ViewsPublic.class);
}
By the way, writeValueUsingView is deprecated in the current version of JSON (see javadoc).