I'm using Flink to process some JSON-format data coming from some Data Source.
For now, my process is quite simple: extract each element from the JSON-format data and print them into log file.
Here is my piece of code:
// create proper deserializer to deserializer the JSON-format data into ObjectNode
PravegaDeserializationSchema<ObjectNode> adapter = new PravegaDeserializationSchema<>(ObjectNode.class, new JavaSerializer<>());
// create connector to receive data from Pravega
FlinkPravegaReader<ObjectNode> source = FlinkPravegaReader.<ObjectNode>builder()
.withPravegaConfig(pravegaConfig)
.forStream(stream)
.withDeserializationSchema(adapter)
.build();
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
DataStream<ObjectNode> dataStream = env.addSource(source).name("Pravega Stream");
dataStream.???.print();
Saying that the data coming from Pravega is like this: {"name":"titi", "age":18}
As I said, for now I simply need to extract name and age and print them.
So how could I do this?
As my understanding, I need to make some customized codes at ???. I might need to create a custom POJO class which contains ObjectNode. But I don't know how. I've read the official doc of Flink and also tried to google about how to create a custom POJO for Flink but I can't still figure out clearly.
Could you please show me an example?
Why don't You simply use something more meaningful instead of JavaSerializer? Perhaps something from here.
You could then create a POJO with the fields you want to use and simply deserialize JSON data to Your POJO instead of ObjectNode
Also, if there is some specific reason that You need to have ObjectNode on deserialization then You can simply do something like :
//I assume You have created the class named MyPojo
dataStream.map(new MapFunction<ObjectNode, MyPojo>() {
ObjectMapper mapper = new ObjectMapper();
#Override
public MyPojo map(final ObjectNode value) throws Exception {
mapper.readValue(value.asText(), MyPojo.class)
}
})
Related
I'm really stuck with cast Rabbit response to POJO. How to do it properly? On sprin.io just no practical example.
So I'm trying to do it with this
#Bean
public DefaultClassMapper typeMapper() {
DefaultClassMapper typeMapper = new DefaultClassMapper();
Map<String, Class> idClassMapping = new HashMap<String, Class>();
idClassMapping.put("range", Loan.class);
typeMapper.setIdClassMapping(idClassMapping);
//typeMapper.setDefaultType(Loan.class);
return typeMapper;
}
#Bean
public MessageConverter messageConverter(DefaultClassMapper defaultClassMapper){
JsonMessageConverter jsonMessageConverter = new JsonMessageConverter();
jsonMessageConverter.setClassMapper(defaultClassMapper);
return jsonMessageConverter;
}
"range" actually fake value from example. Also this really doesn't work type problemn on .setIdClassMapping() . Also I can't use default mapper because serve send header without type hinting field. And I have no control on this remote server. Data format always JSON.
Caused by: org.springframework.amqp.support.converter.MessageConversionException: failed to convert Message content. Could not resolve __TypeId__ in header
Any suggestions working example for marshalling/demarshalling Java objects. I have completely different class for send back value from my code. I'm using java 8.
Just write your own ClassMapper - don't use the default one if your decision criteria to choose the class type is not compatible with its internals.
Or, you can subclass the DefaultClassMapper and override getClassIdFieldName() to tell it which message property to use.
I would like to de-serialize a JSON string and conduct object mutations including replicating nodes, adding new nodes to arrays, and changing the value of text nodes. I read that JsonNodes are for reading and ObjectNodes are for editting.
The only thing I could find to attempt was:
root = mapper.readTree(apiResponseTemplate);
ObjectNode rootTwo = mapper.valueToTree(root);
I'm using jackson 1.9.12
How can I do this? Thanks!
To parse the JSON string you could use something like this.
ObjectMapper mapper = new ObjectMapper();
JsonNode node = mapper.readTree(jsonText);
To add new nodes:
ObjectNode objNode= mapper.createObjectNode();
objNode.put("NodeName", "NodeValue");
To add new ArrayNodes:
objNode.putArray("NodeName");
JsonNode has many useful methods like has("NodeName"), path("NodeName"), etc.
I have a grails object that I am converting using def json = object as JSON. After I have it converted I want to add one more property called pin to the JSON which looks like the following.
[location:[lat:23.03, lon:72.58]]
Only way to do this so far seems like following
Serialize the DomainClass to JSON using grails.converters.json
Convert the JSON to string
Create JSONBoject using the string from Step 2
Add the property
Convert it back to String
Any other way to do this using grails.converters.json? I have tried using Gson but I do not want to go that route because I am getting many Circular Reference Errors
Try this:
domainInstance.properties + [pin: pinInstance] as JSON
I recently needed to do a similar thing. Some caveats:
This is using Grails 2.4.5
I use MongoDB as a backend. As such, I created an object marshaller for MongoDB domain classes. It is printed below, and you can wrap a similar marshaller for your domain class(es):
Marshaller:
class MongodbObjectMarshaller implements ObjectMarshaller<JSON> {
#Override
boolean supports(Object o) { return o?.properties?.dbo }
#Override
void marshalObject(Object obj, JSON converter) throws
ConverterException {
Map propertiesToOutput = obj.properties.dbo
propertiesToOutput.remove("_id") //don't print org.bson.types.ObjectId
propertiesToOutput.remove("version") //don't print gorm verson column
converter.build {
_id obj.id.toString()
propertiesToOutput.each{ entry ->
"$entry.key" entry.value
}
}
}
}
What that marshaller does, it allow in JSON output any of the domain class's properties. obj.properties.dbo is special to MongoDB, but for a regular domain class, you can just grab the properties and exclude the ones you don't need.
Then in my controller, this works:
domainInstance.pin = [location:[lat:23.03, lon:72.58]]
def content = tacticalCard as JSON
because my marshaller now picks up the pin property and prints it.
I need to pass a map back to the web application.
I'm used to encapsulating the map in a JSONObject
http://json.org/java/
But since I am using Spring and Jackson Haus.
is there an easier way to maintain the pojo? May I can just annotate the MAP ?
Jackson has com.fasterxml.jackson.core.JsonNode, and specific subtypes like ObjectNode.
These form so-called Tree Model, which is one of 3 ways to handle JSON with Jackson -- some other libraries (like org.json) only offer this way.
So you should be able to just use JsonNode instead; there is little point in using org.json library; it is slow, and has outdated API.
Alternatively you can just use java.util.Map, and return that. Jackson can handle standard Lists, Maps and other JDK types just fine.
If you need to manipulate the output, ie, you don't want to provide all the fields of the object you can use JSonArray:
#RequestMapping(value = "/api/users", method = RequestMethod.GET)
public
#ResponseBody
String listUsersJson(ModelMap model) throws JSONException {
JSONArray userArray = new JSONArray();
for (User user : userRepository.findAll()) {
JSONObject userJSON = new JSONObject();
userJSON.put("id", user.getId());
userJSON.put("firstName", user.getFirstName());
userJSON.put("lastName", user.getLastName());
userJSON.put("email", user.getEmail());
userArray.put(userJSON);
}
return userArray.toString();
}
Use the example from here
Otherwise if you add jackson to your dependencies and set the controller method anotatted with #ResponseBody the response will automatically mapped to JSON. Check here for a simple example.
Is it possible to configure jackson to always add the type of the serialized object to the generated json output.
For example:
package org.acme;
class ClassA
{
String a;
String b;
}
and I want the generated json to be:
["org.acme.ClassA",{"a":"str1","b":"str2"}]
You can do that with enableDefaultTyping() of the ObjectMapper
e.g.
mapper.enableDefaultTyping(DefaultTyping.OBJECT_AND_NON_CONCRETE);
See ObjectMapper API
If your are free to change from Jackson and do not especially need the format to match the one your are showing you can try Genson http://code.google.com/p/genson.
For example if your requirement is to be able to deserialize interfaces or abstract classes based on the original type of the object you serialized you can do:
interface Entity {}
static class Person implements Entity {}
Genson genson = new Genson.Builder().setWithClassMetadata(true).create();
// json will be equal to {"#class":"my.package.Person"}
String json = genson.serialize(new Person());
// and now Genson is able to deserialize it back to Person using the information
// in the Json Object
Person person = (Person) genson.deserialize(json, Entity.class);
Another nice feature is the ability to define aliases for your classes, so you show less information in the json stream but also this allows you to do refactoring without worring of existing json streams (for example if you store it in a database).
Genson genson = new Genson.Builder().addAlias("person", Person.class).create();
// json value is {"#class": "person"}
String json = genson.serialize(new Person());
Have a look at the wiki.