SpringBatch - how to set up via java config the JsonLineMapper for reading a simple json file - json

How to change from "setLineTokenizer(new DelimitedLineTokenizer()...)" to "JsonLineMapper" in the first code below? Basicaly, it is working with csv but I want to change it to read a simple json file. I found some threads here asking about complex json but this is not my case. Firstly I thought that I should use a very diferent approach from csv way, but after I read SBiAch05sample.pdf (see the link and snippet at the bottom), I understood that FlatFileItemReader can be used to read json format.
In almost similiar question, I can guess that I am not in the wrong direction. Please, I am trying to find the simplest but elegant and recommended way for fixing this snippet code. So, the wrapper below, unless I am really obligated to work this way, seems to go further. Additionally, the wrapper seems to me more Java 6 style than my tentative which takes advantage of anonimous method from Java 7 (as far as I can judge from studies). Please, any advise is higly appreciated.
//My Code
#Bean
#StepScope
public FlatFileItemReader<Message> reader() {
log.info("ItemReader >>");
FlatFileItemReader<Message> reader = new FlatFileItemReader<Message>();
reader.setResource(new ClassPathResource("test_json.js"));
reader.setLineMapper(new DefaultLineMapper<Message>() {
{
setLineTokenizer(new DelimitedLineTokenizer() {
{
setNames(new String[] { "field1", "field2"...
//Sample using a wrapper
http://www.manning.com/templier/SBiAch05sample.pdf
import org.springframework.batch.item.file.LineMapper;
import org.springframework.batch.item.file.mapping.JsonLineMapper;
import com.manning.sbia.ch05.Product;
public class WrappedJsonLineMapper implements LineMapper<Product> {
private JsonLineMapper delegate;
public Product mapLine(String line, int lineNumber) throws Exception {
Map<String,Object> productAsMap
= delegate.mapLine(line, lineNumber);
Product product = new Product();
product.setId((String)productAsMap.get("id"));
product.setName((String)productAsMap.get("name"));
product.setDescription((String)productAsMap.get("description"));
product.setPrice(new Float((Double)productAsMap.get("price")));
return product;
}
public void setDelegate(JsonLineMapper delegate) {
this.delegate = delegate;
}
}

Really you have two options for parsing JSON within a Spring Batch job:
Don't create a LineMapper, create a LineTokenizer. Spring Batch's DefaultLineMapper breaks up the parsing of a record into two phases, parsing the record and mapping the result to an object. The fact that the incoming data is JSON vs a CSV only impacts the parsing piece (which is handled by the LineTokenizer). That being said, you'd have to write your own LineTokenizer to parse the JSON into a FieldSet.
Use the provided JsonLineMapper. Spring Batch provides a LineMapper implementation that uses Jackson to deserialize JSON objects into java objects.
In either case, you can't map a LineMapper to a LineTokenizer as they accomplish two different things.

Related

How to write a efficient POJO Serializer with field renaming in Flink?

I am in the process of writing a new Flink job that ingests JSON data from a Kafka source. I am "stuck" with using Flink 1.15.2. But apart from this very free. I would like to stick to the DataStream API, mostly of a learning experience.
My data looks like this:
{ "SomeField": 123, "someOtherField": 4554, "another_different_field": 34543}
As you can see there are multiple different naming schemas present. Inside of my Flink job and further downstream I would like to clean this up. But how to do this efficiently? Create my own POJOSerializer?
Or just the type hints? E.g. something like this:
public class MyDataTypeInfoFactory extends TypeInfoFactory<MyData> {
#Override
public TypeInformation<MyData> createTypeInfo(
Type t, Map<String, TypeInformation<?>> genericParameters) {
return Types.POJO(MyData.class, new HashMap<>(...)); // some code here?
}
}
Or go with JSONKeyValueDeserializationSchema and then map the objects?
Any pointers?

cucumber.runtime.CucumberException: Not a Map or List type

If I am using a data inside the Data Table which is not type of List or Map then it is giving me error
cucumber.runtime.CucumberException: Not a Map or List type
This is how I am testing this. I am testing it bottom to top i.e. write function > then write step def > write feature file (just for testing purpose).
java function:
public String getScenarioName(Scenario scenario) {
System.out.println("scenario.getName().toString());
}
step def:
#And("^Get current scenario name$")
public void get_current_scenario_name(Scenario scenario) {
System.out.println(getScenarioName(scenario));
}
Feature file:
Scenario: Title of your scenario
Given I have a scenario
Then Get current scenario name
|scenario|
Since I'm using Scenario interface as a parameter I have to use it across, in function, in steps and in feature file.
Note: Please don't judge by the weird scenario, I am only testing it.
I have gone through below links, but it doesn't help me. It keep giving me same error.
https://github.com/cucumber/cucumber-jvm/issues/741
http://grasshopper.tech/340/ >> this one I couldn't implement, didn't quiet understand it.
You're looking somewhat in the wrong direction. To print out the scenario name in a given step you have to capture it in a before hook before printing it in a step:
import cucumber.api.Scenario;
import cucumber.api.java.Before;
import cucumber.api.java.en.And;
public class ExampleSteps {
private Scenario scenario;
#Before
public void capture_scenario(Scenario scenario){
this.scenario = scenario;
}
#And("^get current scenario name$")
public void get_current_scenario_name() {
System.out.println(this.scenario.getName());
}
}
Then this scenario will print My Scenario.
Scenario: My Scenario
And I get the current scenario name

My RCP Client is not returning a deep copy of an object

I have been working on an RCP Client to handle weather data.
What i do is 2 things, first i scraped the JSON i will be using and put it into a dart file. See: https://dartpad.dartlang.org/a9c1fe8ce34c608eaa28
My server.dart page, will import the weather data, and then carry out the following:
import "dart:io";
import "weather_data.dart";
import "dart:convert";
import "package:rpc/rpc.dart";
find ApiServer _apiServer = new ApiServer(prettyPrint:true);
main() async {
Weather w = new Weather(WeatherJson);
TestServer ts = new TestServer(w);
_apiServer.addApi(ts);
HttpServer server = await HttperServer.bind(InternetAddress.ANY_IP_V4, 12345);
server.listen(_apiServer.httpRequestHandler);
}
class Weather{
Map weather;
Weather(this.weather){
Map get daily => weather["daily"];
}
}
#ApiClass(name:"test_server", version: 'v1', description: 'This is a test server api to ping for some quick sample data.')
class TestServer {
Weather myWeather;
TestServer(this.myWeather){
}
#ApiMethod(method:'GET', path: 'daily')
Map<String, Object> getDaily(){
return myWeather.daily;
}
}
So, the server starts correctly, and i will go to localhost:12345/test_server/v1/daily and it will return this:
{
"summary": {},
"icon": {},
"data": {}
}
which is not correct. If you look up the JSON data, summary and icon are both strings and data is an array. They are also empty, and should contain the data i wanted to return.
Why does this occur? Is it because i am returning a Map<String, Object>? I was trying to set it up to be: Map<String, dynamic> but the dart compiler didnt like it.
How do i get this data to return the correct dataset?
The Dart website for RPC is located at: https://github.com/dart-lang/rpc
and you can see that under methods, the return value of a method can be either an instance of a class or a future. That makes sense as per usual, so I set it to be a Map<String,Object> though trying to be vague about it by saying: Map was not sufficient.
Edit:
When doing this mostly in dart pad without RPC, it seems to work correctly, by a sample of: https://dartpad.dartlang.org/3f6dc5779617ed427b75
This leads me to believe something is wrong with the Parsing tool as it seems the return type in dartpad allows to return Map, Map<String, Object>, and Map<String, dynamic>.
Having had a quick look at the RPC package README here https://pub.dartlang.org/packages/rpc, it seems that methods marked as Api methods (with #ApiMethod) should return an instance of a class with simple fields such as:
class ResourceMessage {
int id;
String name;
int capacity;
}
The RPC package will take that instance and serialize it into JSON based on the field names.
From the README:
The MyResponse class must be a non-abstract class with an unnamed
constructor taking no required parameters. The RPC backend will
automatically serialize all public fields of the the MyResponse
instance into JSON ...
You are returning a nested Map representation of the JSON you want the RPC operation to emit and would guess that the RPC package does not handle it as you are expecting it to.
Re: this from your question:
This leads me to believe something is wrong with the Parsing tool as
it seems the return type in dartpad allows to return Map, Map, and Map.
There is no 'parsing' on JSON going on on your example. The data you have is a set of nested literal Dart Maps, Lists and Strings with the same structure as the JSON it was derived from. It just happens to look like JSON.
In your example you are just selecting and printing a sub-map of your data map (data['daily']), which prints out the String that results from calling toString() - which is recursive so you get the contents of all the nested maps and lists within it.
So it's not a 'deep copy' issue, but a difference in how toString() and the RPC code processes a set of nested maps.
BTW: the return type of your getDaily() method is immaterial. What is returned is just a Map whatever the declared return type of the method is. Remember types in Dart are optional and there for editors and compilers to spot potentially incorrect code. See https://www.dartlang.org/docs/dart-up-and-running/ch02.html#variables.
I am going to piggyback off of #Argenti Apparatus here as there was a lot of information gained from him.
Long story short, the required return type of the method:
#ApiMethod(method:'GET', path: 'daily')
Map<String,Object> getDaily(){ // <-- Map<String,Object>
return myWeather.daily;
}
is the error.
I went through and updated the method signature to be Map<String,String> and it parsed it entirely correct. It did not parse the object as a string, but actually parsed it as a full recursed object.
I went through and for the sake of code cleanliness also changed signatures of Weather properties to reflect what they actually were, Map<String,Object> as well.
All in all, When defining it to be an value type of Object, it was returning curly braces, but setting it as a String parsed it correctly.
I ran it through JSLint to confirm it is correct as well.
I gave a +1 to the helper, I had to dig deeper into the code to see WHY it wasnt doing a Map correctly.
This also I feel, is plausibly a bug in RPC Dart.

How can I pass complex objects as arguments to a RESTful service?

I have successfully set up a quick test of creating a "REST-like" service that returns an object serialized to JSON, and that was quite easy and quick (based on this article).
But while returning JSON-ified objects was easy as peach, I have yet to see any examples dealing with input parameters that are not primitives. How can I pass in a complex object as an argument? I am using Apache CXF, but examples using other frameworks like Jackson are welcome too :)
Client side would probably be something like building a javascript object, pass it into JSON.stringify(complexObj), and pass that string as one of the parameters.
The service would probably look something like this
#Service("myService")
class RestService {
#GET
#Produces("application/json")
#Path("/fooBar")
public Result fooBar(#QueryParam("foo") double foo, #QueryParam("bar") double bar,
#QueryParam("object") MyComplex object) throws WebServiceException {
...
}
}
Sending serialized objects as parameters would probably quickly touch the 2KB URL-limit imposed by Internet Explorer. Would you recommend using POST in these cases, and would I need to change much in the function definitions?
After digging a bit I quickly found out there are basically two options:
Option 1
You pass a "wrapper object" containing all the other parameters to the service. You might need to annotate this wrapper class with JAXB annotations like #XmlRootElement in order for this to work with the Jettison based provider, but if you use Jackson in stead there is no need. Just set the content type to the right type and the right message body reader will be invoked.
This will only work for POST type services of course (AFAIK).
Example
This is just an example of turning the service mentioned in the original question into one using a wrapper object.
#Service("myService")
class RestService {
#POST
#Produces("application/json")
#Path("/fooBar")
public Result fooBar(
/**
* Using "" will inject all form params directly into a ParamsWrapper
* #see http://cxf.apache.org/docs/jax-rs-basics.html
*/
#FormParam("") FooBarParamsWrapper wrapper
) throws WebServiceException {
doSomething(wrapper.foo);
}
}
class ParamsWrapper {
double foo, bar;
MyComplexObject object;
}
Option 2
You can provide some special string format that you pack your objects into and then implement either a constructor taking a string, a static valueOf(String s) or a static fromString(String s) in the class that will take this string and create an object from it. Or quite similar, create a ParameterHandler that does exactly the same.
AFAIK, only the second version will allow you to call your services from a browser using JSONP (since JSONP is a trick restricted to GET). I chose this route to be able to pass arrays of complex objects in the URI.
As an example of how this works, take the following domain class and service
Example
#GET
#Path("myService")
public void myService(#QueryParam("a") MyClass [] myVals) {
//do something
}
class MyClass {
public int foo;
public int bar;
/** Deserializes an Object of class MyClass from its JSON representation */
public static MyClass fromString(String jsonRepresentation) {
ObjectMapper mapper = new ObjectMapper(); //Jackson's JSON marshaller
MyClass o= null;
try {
o = mapper.readValue(jsonRepresentation, MyClass.class );
} catch (IOException e) {
throw new WebApplicationException()
}
return o;
}
}
A URI http://my-server.com/myService?a={"foo":1, "bar":2}&a={"foo":100, "bar":200} would in this case be deserialized into an array composed of two MyClass objects.
2019 comment:
Seeing that this answer still gets some hits in 2019, I feel I should comment. In hindsight, I would not recomment option 2, as going through these steps just to be able to be able to do GET calls adds complexity that's probably not worth it. If your service takes such complex input, you will probably not be able to utilize client side caching anyway, due to the number of permutations of your input. I'd just go for configuring proper Cross-Origin-Sharing (CORS) headers on the server and POST the input. Then focus on caching whatever you can on the server.
The accepted answer is missing #BeanParam. See
https://docs.jboss.org/resteasy/docs/3.0-rc-1/javadocs/javax/ws/rs/BeanParam.html
for further details. It allows you to define query params inside a wrapper object.
E.g.
public class TestPOJO {
#QueryParam("someQueryParam")
private boolean someQueryParam;
public boolean isSomeQueryParam() {
return someQueryParam;
}
public boolean setSomeQueryParam(boolean value) {
this.someQueryParam = value;
}
}
... // inside the Resource class
#GET
#Path("test")
public Response getTest(#BeanParam TestPOJO testPOJO) {
...
}
the best and simplest solution is to send your object as a json string and in server side implement a method which will decode that json and map to the specified object as per your need.. and yes it`s better to use POST.

Json <-> Java serialization that works with GWT [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I am looking for a simple Json (de)serializer for Java that might work with GWT. I have googled a bit and found some solutions that either require annotate every member or define useless interfaces. Quite a boring. Why don't we have something really simple like
class MyBean {
...
}
new GoodSerializer().makeString(new MyBean());
new GoodSerializer().makeObject("{ ... }", MyBean.class)
Take a look at GWT's Overlay Types. I think this is by far the easiest way to work with JSON in GWT. Here's a modified code example from the linked article:
public class Customer extends JavaScriptObject {
public final native String getFirstName() /*-{
return this.first_name;
}-*/;
public final native void setFirstName(String value) /*-{
this.first_name = value;
}-*/;
public final native String getLastName() /*-{
return this.last_name;
}-*/;
public final native void setLastName(String value) /*-{
this.last_name = value;
}-*/;
}
Once you have the overlay type defined, it's easy to create a JavaScript object from JSON and access its properties in Java:
public static final native Customer buildCustomer(String json) /*-{
return eval('(' + json + ')');
}-*/;
If you want the JSON representation of the object again, you can wrap the overlay type in a JSONObject:
Customer customer = buildCustomer("{'Bart', 'Simpson'}");
customer.setFirstName("Lisa");
// Displays {"first_name":"Lisa","last_name":"Simpson"}
Window.alert(new JSONObject(customer).toString());
Another thing to try is the new AutoBean framework introduced with GWT 2.1.
You define interfaces for your beans and a factory that vends them, and GWT generates implementations for you.
interface MyBean {
String getFoo();
void setFoo(String foo);
}
interface MyBiggerBean {
List<MyBean> getBeans();
void setBeans(List<MyBean> beans>;
}
interface Beanery extends AutoBeanFactory{
AutoBean<MyBean> makeBean();
AutoBean<MyBiggerBean> makeBigBean();
}
Beanery beanFactory = GWT.create(Beanery.class);
void go() {
MyBean bean = beanFactory.makeBean().as();
bean.setFoo("Hello, beans");
}
The AutoBeanCodex can be used to serialize them to and from json.
AutoBean<MyBean> autoBean = AutoBeanUtils.getAutoBean(bean);
String asJson = AutoBeanCodex.encode(autoBean).getPayload();
AutoBean<MyBean> autoBeanCloneAB =
AutoBeanCodex.decode(beanFactory, MyBean.class, asJson );
MyBean autoBeanClone = autoBeanCloneAB.as();
assertTrue(AutoBeanUtils.deepEquals(autoBean, autoBeanClone));
They work on the server side too — use AutoBeanFactoryMagic.create(Beanery.class) instead of GWT.create(Beanery.class).
The simplest way would be to use GWT's built-in JSON API. Here's the documentation. And here is a great tutorial on how to use it.
It's as simple as this:
String json = //json string
JSONValue value = JSONParser.parse(json);
The JSONValue API is pretty cool. It lets you chain validations as you extract values from the JSON object so that exceptions will be thrown if anything's amiss with the format.
It seems that I found the right answer to my question
I figured out that bean to json and json to bean conversion in GWT isn't a trivial task. Known libraries would not work because GWT would require their full source code and this source code must use only Java classes that are amoung emulated by GWT. Also, you cannot use reflection in GWT. Very tough requirements!
I found the only existing solution named gwt-jsonizer. It uses a custom Generator class and requires a satellite interface for each "jsonable" bean. Unfortunately, it does not work without patching on the latest version of GWT and has not been updated for a long time.
So, I personally decided that it is cheaper and faster to make my beans khow how to convert themselves to and from json. Like this:
public class SmartBean {
private String name;
public String getName() { return name; }
public void setName(String value) { name = value; }
public JSONObject toJson() {
JSONObject result = new JSONObject();
result.put("name", new JSONString(this.name));
return result;
}
public void fromJson(JSONObject value) {
this.name = value.get("name").isString().stringValue();
}
}
JSONxxxx are GWT built-in classes that provide low-level json support.
RestyGWT is a powerful library for encoding or decoding Java Object to JSON in GWT:
import javax.ws.rs.POST;
...
public interface PizzaOrderCodec extends JsonEncoderDecoder<PizzaOrder> {
}
Then:
// GWT will implement the interface for you
PizzaOrderCodec codec = GWT.create(PizzaOrderCodec.class);
// Encoding an object to json
PizzaOrder order = ...
JSONValue json = codec.encode(order);
// decoding an object to from json
PizzaOrder other = codec.decode(json);
It has also got several easy to use API for consuming Restful web services.
Have a nice time.
Check this:
GWT Professional JSON Serializer:
http://code.google.com/p/gwtprojsonserializer/
!Works with GWT 2.0+!
json.org/java seems to be included with GWT these days:
gwt-servlet-deps.jar\org\json\
Or, this project seems to be comprehensive:
http://code.google.com/p/piriti/
In Google Web Toolkit Applications, pages 510 to 522, the author, Ryan Dewsbury, shows how to use GWT code generation to do serialization to and from XML and JSON documents.
You can download the code here; you want the chapter 10 code bundles, and then you want to look in the src/com/gwtapps/serialization package. I did not see a license for this code, but have emailed the author to see what he says. I'll update this if he replies.
Issues with this solution:
You have to add a marker interface on all your objects that you want serialized (he uses java.io.Serializable but I imagine you could use others--if you are using hibernate for your backend, your pojos might already be tagged like this).
The code only supports string properties; it could be extended.
The code is only written for 1.4 and 1.5.
So, this is not an out of the box solution, but a great starting point for someone to build a JSON serializer that fits with GWT. Combine that with a JSON serializer on the server side, like json-lib and you're good to go.
I also found this project (again, some marker interface is required).
Try this serializer from Google Code: http://code.google.com/p/json-io/
If you need to write or read JSON format in Java, this is the tool to use. No need to create extra classes, etc. Convert a Java object graph to JSON format in one call. Do the opposite - create a JSON String or Stream to Java objects. This is the fastest library I have seen yet to do this. It is faster than ObjectOutputStream and ObjectInputStream in most cases, which use binary format.
Very handy utility.
You may want to checkout this project https://gerrit.googlesource.com/gwtjsonrpc/
It's a library created in order to support a code review system for Android, Gerrit, but it's a stand-alone module meant to be embedded into any GWT project, not just Gerrit.
A reasonable tutorial is probably the README in the top level of the directory. It's quite similar to standard GWT RPC but it uses JSON encoding. It also has built-in XSRF protection.
I seem to be answering this question a lot...
There's a page on code.google.com titled Using GWT for JSON Mashups. It's (unfortunately) way over my head, as I'm not that familiar with GWT, so it may not be helpful.
OK, I deleted my previous answer because it turned out to be exactly what you didn't want.
I don't know how well it works with GWT, but we use the json-lib library to serialize objects in a normal Java project where I work.
It can create a JSONObject directly from a JavaBean, then use the resulting JSONObject's toString() method to get the actual JSON string back.
Likewise, it can also turn JSON back into a JavaBean.
Not sure if Jackson would work for you.
I don't know if there's GWT-specific you are looking for; if not it should work.
But its serialization/deserialization works quite well, like:
// read json, create object
ObjectMapper mapper = new ObjectMapper();
MyBean bean = mapper.readValue(jsonAsString, MyBean.class);
// and write out
StringWriter sw = new StringWriter();
mapper.writeValue(sw, user);
String jsonOut = sw.toString();
You do need accessors (getX() to serialize, setX() to deserialize; can annotate methods with other names), but that's about it.