API publisher added new field to their response object that isn't in my model classes. Is there a way to loosen up the mapper to ignore unknown fields? I still want to use my old legacy model classes to parse, but now I get an exception...
Switch to Jackson JSON Processor and do this:
ObjectMapper mapper = new ObjectMapper();
// THIS IS WHAT I WAS LOOKING FOR TO HANDLE IN XSTREAM!!!!!!
mapper.configure(org.codehaus.jackson.map.DeserializationConfig.Feature.FAIL_ON_UNKNOWN_PROPERTIES, false);
User user = mapper.readValue(new File("user.json"), User.class);
Related
I'm using Flink to process some JSON-format data coming from some Data Source.
For now, my process is quite simple: extract each element from the JSON-format data and print them into log file.
Here is my piece of code:
// create proper deserializer to deserializer the JSON-format data into ObjectNode
PravegaDeserializationSchema<ObjectNode> adapter = new PravegaDeserializationSchema<>(ObjectNode.class, new JavaSerializer<>());
// create connector to receive data from Pravega
FlinkPravegaReader<ObjectNode> source = FlinkPravegaReader.<ObjectNode>builder()
.withPravegaConfig(pravegaConfig)
.forStream(stream)
.withDeserializationSchema(adapter)
.build();
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
DataStream<ObjectNode> dataStream = env.addSource(source).name("Pravega Stream");
dataStream.???.print();
Saying that the data coming from Pravega is like this: {"name":"titi", "age":18}
As I said, for now I simply need to extract name and age and print them.
So how could I do this?
As my understanding, I need to make some customized codes at ???. I might need to create a custom POJO class which contains ObjectNode. But I don't know how. I've read the official doc of Flink and also tried to google about how to create a custom POJO for Flink but I can't still figure out clearly.
Could you please show me an example?
Why don't You simply use something more meaningful instead of JavaSerializer? Perhaps something from here.
You could then create a POJO with the fields you want to use and simply deserialize JSON data to Your POJO instead of ObjectNode
Also, if there is some specific reason that You need to have ObjectNode on deserialization then You can simply do something like :
//I assume You have created the class named MyPojo
dataStream.map(new MapFunction<ObjectNode, MyPojo>() {
ObjectMapper mapper = new ObjectMapper();
#Override
public MyPojo map(final ObjectNode value) throws Exception {
mapper.readValue(value.asText(), MyPojo.class)
}
})
I'm trying to serialize/deserialize a Scala class to JSON using Jackson ObjectMapper. The serialization works fine, but I was getting type exceptions trying to read the JSON back in. I fixed most of those by adding appropriate annotations, but it's not working for my Map members... it seems like Jackson is trying to treat the keys in the JSON object as properties in a class instead of keys in a map. (I believe this is different than other questions like this one since they are calling readValue on the map contents directly.)
Here's my ObjectMapper setup:
val mapper = new ObjectMapper() with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
Here's what my annotated class and member look like:
#JsonInclude(JsonInclude.Include.NON_DEFAULT)
class MyClass extends Serializable {
#JsonDeserialize(
as = classOf[mutable.HashMap[String, Long]],
keyAs = classOf[java.lang.String],
contentAs = classOf[java.lang.Long]
)
val counts = mutable.Map.empty[String, Long]
}
If I give it some JSON like:
{"counts":{"foo":1,"bar":2}}
And read it with mapper.readValue[MyClass](jsonString)
I get an exception like UnrecognizedPropertyException: Unrecognized field "foo" (class mutable.HashMap), not marked as ignorable.
I tried adding DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES to my mapper configuration but that didn't seem to do anything in this case, and I'm not sure that kind of global setting is desirable.
How do I convince Jackson to treat the strings "foo" and "bar" as keys in the map member field and not as properties in the HashMap class? It seems to have done the right thing automatically writing it out.
Also worth noting: the deserialization appears to work fine in a quick out/in unit test to a temp file or a string variable, but not when I try to run the whole application and it reads the JSON its previously written. I don't know why it seems to work in the test, as far as I know it's making the same readValue call.
I made one simple test like this:
case class TestClass (counts: mutable.HashMap[String, Long])
And I converted it like:
val objectMapper = new ObjectMapper() with ScalaObjectMapper
objectMapper.registerModule(DefaultScalaModule)
val r3 = objectMapper.readValue("{\"counts\":{\"foo\":1,\"bar\":2}}", classOf[TestClass])
And apparently it works for me. Maybe it's something about the version you're using of Jackson, or Scala. Have you tried different versions of Jackson for example?
Try jsoniter-scala and you will enjoy how it can be handy, safely, and efficient to parse and serialize JSON these days with Scala: https://github.com/plokhotnyuk/jsoniter-scala
One of it's crucial features is an ability to generate codecs in compile time and even print their sources. You will have no any runtime magic like reflection or byte code replacement that will affect your code.
My problem was a race condition due to not using chaining in my mapper configuration singleton.
My old code was more like this:
private var mapper: ObjectMapper with ScalaObjectMapper = _
def getMapper: ObjectMapper with ScalaObjectMapper = {
if (mapper == null) {
mapper = new ObjectMapper() with ScalaObjectMapper
mapper.registerModule(DefaultScalaModule)
mapper.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
}
mapper
}
As you can see, if one thread initializes the mapper, but hasn't yet disabled unknown properties failure, a second thread could return and use a mapper that hasn't had that flag set yet, which explains why I was seeing the error only some of the time.
The correct code uses chaining so that the mapper singleton is set with all of the configuration:
private var mapper: ObjectMapper = _
def getMapper: ObjectMapper = {
if (mapper == null) {
mapper = new ObjectMapper()
.registerModule(DefaultScalaModule)
.disable(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES)
}
mapper
}
(I also removed the experimental ScalaObjectMapper mix-in.)
I am newbie to play framework.
I had seen chapter 'JSON' in play framework documentation
It guide me to use case class, not normal class
so It seemed not to support json to class mapping
is it true?
"sorry for poor expression skill, I'm not a native"
What is exactly your problem ? Here is how it works in Java, it must be very close in Scala.
Let's say you have a class MyClass and and instance myObject of this class.
If you want to serialize :
JsonNode json = play.libs.Json.toJson(myObject);
And if you want to deserialize :
ObjectMapper mapper = new ObjectMapper();
mapper.configure(DeserializationFeature.FAIL_ON_UNKNOWN_PROPERTIES, false);
MyClass myObject = mapper.readValue(jsonNode.toString(), MyClass.class);
Obviously, you'll need to handle JsonParseException, JsonMappingException... to send a human readable message to your end user.
I have seen the responses from many other posts but would like to understand if there is a better way to do the same thing.
Requirement:-
I am using restTemplate to talk to web service which returns JSON output which is dynamic. As a consumer I don't want to access all fields but is interested in few of them. I am using Spring framework with Jackson parser and found the way of accessing it
String response = restTemplate.getForObject(targetUrl, String.class);
System.out.println(response);
ObjectMapper mapper = new ObjectMapper();
JsonNode rootNode = mapper.readValue(response, JsonNode.class);
JsonNode uri = rootNode.get("uri");
System.out.println(uri.asText());
Do you know any better way to do it? Mapping to java Object is something that I dont want to do as the json output is not in my control
If your RestTemplate is configured with the default HttpMessageConverters, which is provided by Jackson2ObjectMapperBuilder, you may directly get a JsonNode from restTemplate.getForObject.
For example,
ArrayNode resources = restTemplate.getForObject("/resources", ArrayNode.class);
Or,
ObjectNode resource = restTemplate.getForObject("/resources/123", ObjectNode.class);
Note that ArrayNode and ObjectNode are sub-classes of JsonNode.
Is it possible to configure jackson to always add the type of the serialized object to the generated json output.
For example:
package org.acme;
class ClassA
{
String a;
String b;
}
and I want the generated json to be:
["org.acme.ClassA",{"a":"str1","b":"str2"}]
You can do that with enableDefaultTyping() of the ObjectMapper
e.g.
mapper.enableDefaultTyping(DefaultTyping.OBJECT_AND_NON_CONCRETE);
See ObjectMapper API
If your are free to change from Jackson and do not especially need the format to match the one your are showing you can try Genson http://code.google.com/p/genson.
For example if your requirement is to be able to deserialize interfaces or abstract classes based on the original type of the object you serialized you can do:
interface Entity {}
static class Person implements Entity {}
Genson genson = new Genson.Builder().setWithClassMetadata(true).create();
// json will be equal to {"#class":"my.package.Person"}
String json = genson.serialize(new Person());
// and now Genson is able to deserialize it back to Person using the information
// in the Json Object
Person person = (Person) genson.deserialize(json, Entity.class);
Another nice feature is the ability to define aliases for your classes, so you show less information in the json stream but also this allows you to do refactoring without worring of existing json streams (for example if you store it in a database).
Genson genson = new Genson.Builder().addAlias("person", Person.class).create();
// json value is {"#class": "person"}
String json = genson.serialize(new Person());
Have a look at the wiki.