Getting exception when deserialize a long from a stored mongdb item - json

I'm using the MongoD Java Driver to store an object with a long field as follows:
public static class WithLong {
long time = System.currentTimeMillis();
}
public static void main(String[] args) throws CAAException, InterruptedException {
MongoClient client = new MongoClient("localhost",27017);
MongoDatabase db = client.getDatabase("somedb");
MongoCollection<Document> itemCollection = db.getCollection("mycollection");
itemCollection.drop();
itemCollection = db.getCollection("mycollection");
WithLong rd = new WithLong();
try {
Gson gson = new Gson();
String json = gson.toJson(rd);
System.out.println("json=" + json);
Document doc = Document.parse(json);
itemCollection.insertOne(doc);
String id = doc.get("_id").toString();
doc = itemCollection.find(new Document().append("_id", new ObjectId(id))).first();
System.out.println("doc json=" + doc.toJson());
rd = gson.fromJson(doc.toJson(), WithLong.class);
System.out.println(rd);
} finally {
db.drop();
client.close();
}
}
The json created by Gson and that is used to create the Document that I insert is (from the first println()):
json={"time":1458154511859}
but the json that comes back when I look up the doc is as follows and can't be handled by Gson (from the 2nd println()).
doc json={ "_id" : { "$oid" : "56e9ac0fb032d9320c57bece" }, "time" : { "$numberLong" : "1458154511859" } }
This causes the following exception in Gson.fromJson():
Exception in thread "main" com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected a long but was BEGIN_OBJECT at line 1 column 62
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:176)
at com.google.gson.Gson.fromJson(Gson.java:803)
...
Apparently it can deserialize $oid, but not $numberLong. So how am I supposed to deserialize this Document back into a WithLong instance? Thanks

Related

networknt schema validator is not consistent for "AutoDetect JsonSchema", "Load schema from URL" or "Load schema from Loc"

I tried to validate custom JsonSchema with 3 ways
AutoDetect JsonSchema From JsonNode - No errors, but 11 WARN
JsonSchema From Url - 2 Errors, No WARN
JsonSchema From String contents - 2 Errors, No WARN
I am expecting all 3 schema factory to provide similar result. At this moment, I am not sure which one is correct approach.
Why AutoDetect JsonSchema is not failing?
Based on our requirement, "AutoDetect JsonSchema From JsonNode" is best fit for us.
Detail to replicate the issue:
Sample code -
public class ValidateJsonSchemaTest {
private ObjectMapper mapper = new ObjectMapper();
private JsonNode testJsonSchema = null;
private String DRAFT_7_SCHEMA_URI="https://json-schema.org/draft-07/schema";
private static final Logger LOGGER = LoggerFactory.getLogger(ValidateJsonSchemaTest.class);
#Before
public void setup() throws IOException {
InputStream resourceAsStream = getClass().getClassLoader().getResourceAsStream("json-validation-poc/test-custom-schema.json");
String content = IOUtils.toString(resourceAsStream, Charset.defaultCharset());
testJsonSchema = mapper.readTree(content);
}
#Test
public void testAutoDetectJsonSchemaFromJsonNode() {
JsonSchema schema = getJsonSchemaFromJsonNodeAutomaticVersion(testJsonSchema);
Set<ValidationMessage> errors = schema.validate(testJsonSchema);
if(errors.size() > 0) {
LOGGER.error("testAutoDetectJsonSchemaFromJsonNode :: Validation Error %S", errors);
}
Assert.assertEquals("Error size", 0, errors.size());
}
#Test
public void testJsonSchemaFromUrl() throws URISyntaxException {
JsonSchema schema = getV7JsonSchemaFromUrl();
Set<ValidationMessage> errors = schema.validate(testJsonSchema);
if(errors.size() > 0) {
LOGGER.error("testJsonSchemaFromUrl :: Validation Error '{}'", errors);
}
Assert.assertEquals("Error size", 0, errors.size());
}
#Test
public void testJsonSchemaFromStringContent() throws IOException {
JsonSchema schema = getV7JsonSchemaFromStringContent();
Set<ValidationMessage> errors = schema.validate(testJsonSchema);
if(errors.size() > 0) {
LOGGER.error("testJsonSchemaFromStringContent :: Validation Error '{}'", errors);
}
Assert.assertEquals("Error size", 0, errors.size());
}
protected JsonSchema getJsonSchemaFromJsonNodeAutomaticVersion(JsonNode jsonNode) {
VersionFlag versionFlag = SpecVersionDetector.detect(jsonNode);
JsonSchemaFactory factory = JsonSchemaFactory.getInstance(versionFlag);
JsonSchema schema = factory.getSchema(jsonNode);
return schema;
}
protected JsonSchema getV7JsonSchemaFromUrl() throws URISyntaxException {
JsonSchemaFactory factory = JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V7);
JsonSchema schema = factory.getSchema(new URI(DRAFT_7_SCHEMA_URI));
return schema;
}
protected JsonSchema getV7JsonSchemaFromStringContent() throws IOException {
InputStream resourceAsStream = getClass().getClassLoader().getResourceAsStream("json-validation-poc/json-draft-07-schema.json");
String schemaContent = IOUtils.toString(resourceAsStream, Charset.defaultCharset());
JsonSchemaFactory factory = JsonSchemaFactory.getInstance(SpecVersion.VersionFlag.V7);
return factory.getSchema(schemaContent);
}
}
Refer ValidateJsonSchemaTest.java for more details.
version used for testing: com.networknt:json-schema-validator:1.0.66
Screenshot of test failure:
Log screenshot:
Reported this issue to Github-networknt/json-schema-validator as well

How to test a Datastream with jsonobject in Apache Flink

I am new to testing and i am trying to write a unit test cases on a Flink Datastream which takes input a jsonobject and passes the json object to a processfuntion and it returns a valid or invalid jsonobject when certain rule conditions are met below is the junit test case, below i am trying to compare the output jsonobject from process function with the jsonobject of the input file
#Test
public void testcompareInputAndOutputDataJSONSignal() throws Exception {
org.json.JSONObject jsonObject = toJsonObject();
String input = jsonObject.toString();
String output = JSONDataStreamOutput();
assertEquals(mapper.readTree(input), mapper.readTree(output));
}
below is my toJSONObject and JSONDataStream meathods
public static JSONObject toJsonObject() throws IOException, ParseException {
JSONParser jsonParser = new JSONParser();
FileReader fileReader = new FileReader(getFileFromResources("input.json"));
JSONObject obj = (JSONObject) jsonParser.parse(fileReader);
return obj;
}
public String SignalDataStreamOutput() throws Exception {
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
DataStream<JSONObject> validSignal = env.fromElements(toJsonObject())
.process(new JsonFilter());
String outputFolder = "output";
validSignal.writeAsText(outputFolder).setParallelism(1);
env.execute();
String content = new String(Files.readAllBytes(Paths.get("output.txt")));
return content;
}
What i am doing is i am converting a jsonfile to jsonobject using the toJSONObject method and sending to a data stream using SignalDataStreamOutput method which will intern send it to a process function in JsonFilter class and validate it against a set of rules and if it's valid it will return a jsonobject and when trying to access the jsonobject directly from stream i am getting value like org.apache.flink#994jdkeiri so i am trying to write the output to a file and trying to read it back to a string and comparing it in test method but this is a work around process and i found a link to use Mockito framework here i changed it to use json object like below
final Collector<JSONObject> collectorMock = (Collector<JSONObject>)Mockito.mock(JsonFilter.class);
final Context contextMock = Mockito.mock(Context.class);
#Test
public void testcompareInputAndOutputDataForValidSignal() throws Exception {
org.json.JSONObject jsonObject = convertToJsonObject();
Mockito.verify(collectorMock).collect(jsonObject);
}
but the above approach is also not working can you suggest me simplified approach to test the json object

how to parse JSON file using rest API and spring boot

I'm new to this, and I want to read JSON file imported with rest api and parse it with spring boot.
I worked with CSV file with his method :
#RequestMapping(value = "/import", method = RequestMethod.POST)
public String handleFileUpload(#RequestParam("file") MultipartFile multipartFile) throws IOException {
String name = multipartFile.getOriginalFilename();
System.out.println("File name: "+name);
byte[] bytes = multipartFile.getBytes();
System.out.println("File uploaded content:\n" + new String(bytes));
return "file uploaded";
}
Now i want to parse a JSON File :
[
{
"name":"John",
"city":"Berlin",
"cars":[
"audi",
"bmw"
],
"job":"Teacher"
},
{
"name":"Mark",
"city":"Oslo",
"cars":[
"VW",
"Toyata"
],
"job":"Doctor"
}
]
I have try it to parse this file with java and it works for me but I dont know how to get it with rest api
This method to parse te file JSON and it works
public static void main(String[] args) throws FileNotFoundException,
IOException, ParseException {
JSONParser parser = new JSONParser();
JSONArray jsonArray = (JSONArray) parser.parse(new FileReader(
"D:/PFE 2018/testjsonfile.json"));
for (Object o : jsonArray) {
JSONObject person = (JSONObject) o;
String strName = (String) person.get("name");
System.out.println("Name::::" + strName);
String strCity = (String) person.get("city");
System.out.println("City::::" + strCity);
JSONArray arrays = (JSONArray) person.get("cars");
for (Object object : arrays) {
System.out.println("cars::::" + object);
}
String strJob = (String) person.get("job");
System.out.println("Job::::" + strJob);
System.out.println();
}
}
now how to reuse this methode with rest api
It depends what you want to do with your JSON (it's not entirely clear in your question).
In general, good practice with Spring Boot is to use Jackson either:
binding your JSON with a POJO if you expect your JSON to have a known format or
mapping your JSON in a tree.
Examples for the described behaviours can be found in this article for instance.

Null check with Long does not work when extracted from JSON using GSON Library

I have a Data class as below
Class Data {
private Long check;
private Long cost;
}
My jsonString looks like
"data": { "cost": 5 }
when I do
Data data = gson.fromJson(jsonString, Data.class);
and I check for null for check like below
if (data.check == null) {//print error message}
It does not work. the statements at {//print error message} are not executed. Any idea whats happening?
If you have a json sample such as:
{
"data": {
"cost": 5
}
}
it means you have an object which has an inner object named as data. So you have to define the inner pojo(Data) and its holder pojo(i.e DataHolder):
class DataHolder {
private Data data;
}
class Data {
private Long check;
private Long cost;
}
Then you can deserialize your json string to a java instance:
GsonBuilder gsonBuilder = new GsonBuilder();
Gson gson = gsonBuilder.create();
String jsonString = "{\"data\": { \"cost\": 5 }}";
DataHolder dataHolder = gson.fromJson(jsonString, DataHolder.class);

How to configure Gson to serialize a set of JSR-303 ConstraintViolation objects?

I can't seem to find out how to serialize Hibernate's implementation of constraint violations using Gson.
Here's what I've tried so far.
Approach 1
MyPojo aPojo = new MyPojo();
Gson gson = new Gson();
Set<ConstraintViolation<MyPojo>> violations = validator.validate(aPojo);
System.out.println(gson.toJson(violations));
Fails with this error:
java.lang.UnsupportedOperationException:
Attempted to serialize java.lang.Class: com.bar.baz.MyPojo.
Forgot to register a type adapter?
at com.google.gson.internal.bind.TypeAdapters$1.write(TypeAdapters.java:67)
at com.google.gson.internal.bind.TypeAdapters$1.write(TypeAdapters.java:61)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:89)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:195)
at com.google.gson.internal.bind.ObjectTypeAdapter.write(ObjectTypeAdapter.java:107)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:96)
at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:60)
at com.google.gson.Gson.toJson(Gson.java:593)
at com.google.gson.Gson.toJson(Gson.java:572)
at com.google.gson.Gson.toJson(Gson.java:527)
at com.google.gson.Gson.toJson(Gson.java:507)
Approach 2
Gson gson = new Gson();
Set<ConstraintViolation<MyPojo>> violations = validator.validate(MyPojo);
System.out.println(
gson.toJson(violations,
new TypeToken<ConstraintViolation<MyPojo>>() {}.getType())
);
Fails by not serializing MyPojo's properties:
Output: {}.
Approach 3
I was expecting this approach to delegate serialization to my custom Serializer but it still fails:
GsonBuilder builder = new GsonBuilder();
builder.registerTypeAdapter(
new TypeToken<ConstraintViolation<MyPojo>>() {}.getType(),
new JsonSerializer<ConstraintViolation<MyPojo>>() {
#Override
public JsonElement serialize(ConstraintViolation<MyPojo> src, Type typeOfSrc, JsonSerializationContext context) {
JsonObject result = new JsonObject();
result.addProperty("aTestProperty", "A Test Value");
return result;
}
});
Gson gson = builder.create();
Set<ConstraintViolation<MyPojo>> violations = validator.validate(MyPojo);
System.out.println(gson.toJson(violations));
However it fails with this error:
java.lang.UnsupportedOperationException:
Attempted to serialize java.lang.Class:
com.bar.baz.MyPojo.
Forgot to register a type adapter?
at com.google.gson.internal.bind.TypeAdapters$1.write(TypeAdapters.java:67)
at com.google.gson.internal.bind.TypeAdapters$1.write(TypeAdapters.java:61)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.write(ReflectiveTypeAdapterFactory.java:89)
at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.write(ReflectiveTypeAdapterFactory.java:195)
at com.google.gson.internal.bind.ObjectTypeAdapter.write(ObjectTypeAdapter.java:107)
at com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.write(TypeAdapterRuntimeTypeWrapper.java:68)
at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:96)
at com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.write(CollectionTypeAdapterFactory.java:60)
at com.google.gson.Gson.toJson(Gson.java:593)
at com.google.gson.Gson.toJson(Gson.java:572)
at com.google.gson.Gson.toJson(Gson.java:527)
at com.google.gson.Gson.toJson(Gson.java:507)
Approach 4
Looking at the error message, I though this might work:
GsonBuilder builder = new GsonBuilder();
builder.registerTypeAdapter(
new TypeToken<ConstraintViolation<MyPojo>>() {}.getType(),
new JsonSerializer<ConstraintViolation<MyPojo>>() {
#Override
public JsonElement serialize(ConstraintViolation<MyPojo> src, Type typeOfSrc, JsonSerializationContext context) {
JsonObject result = new JsonObject();
result.addProperty("aTestProperty", "A Test Value");
return result;
}
});
builder.registerTypeAdapter(
new TypeToken<MyPojo>() {}.getType(),
new JsonSerializer<MyPojo>() {
#Override
public JsonElement serialize(MyPojo src, Type typeOfSrc, JsonSerializationContext context) {
JsonObject result = new JsonObject();
result.addProperty("anotherTestProperty", "Another Test Value");
return result;
}
});
Gson gson = builder.create();
Set<ConstraintViolation<MyPojo>> violations = validator.validate(MyPojo);
System.out.println(gson.toJson(violations));
But it fails with a similar error.
Approach 5: Working but ugly
The only thing that I've managed to make work is to register the serializer with the type of the vendor (Hibernate) specific implementation for ConstraintViolation:
Set<ConstraintViolation<MyPojo>> violations = validator.validate(MyPojo);
GsonBuilder builder = new GsonBuilder();
builder.registerTypeAdapter(
new TypeToken<ConstraintViolationImpl>() {}.getType(),
new JsonSerializer<ConstraintViolation<MyPojo>>() {
#Override
public JsonElement serialize(ConstraintViolation<MyPojo> src, Type typeOfSrc, JsonSerializationContext context) {
JsonObject result = new JsonObject();
result.addProperty("aTestProperty", "A Test Value");
return result;
}
});
Gson gson = builder.create();
System.out.println(gson.toJson(violations));
Is there a way to make this work without relying on the concrete implementation of ConstraintViolation (i.e. org.hibernate.validator.internal.engine.ConstraintViolationImpl)?
There doesn't seem to be a reasonable approach to serialize javax.validation.ConstraintViolation objects. In fact, even Jackson errs while trying to serialize the set:
Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: fromIndex(0) > toIndex(-1) (through reference chain: java.util.HashSet[0]->org.hibernate.validator.internal.engine.ConstraintViolationImpl["propertyPath"]->org.hibernate.validator.internal.engine.path.PathImpl["pathWithoutLeafNode"]->org.hibernate.validator.internal.engine.path.PathImpl["pathWithoutLeafNode"]->org.hibernate.validator.internal.engine.path.PathImpl["pathWithoutLeafNode"])
at com.fasterxml.jackson.databind.JsonMappingException.wrapWithPath(JsonMappingException.java:232)
For the time being, I just convert the set of errors into a set of custom POJOs I've written and serialize that instead.
Custom ValidationError POJO:
public class ValidationError {
private String className;
private String propertyPath;
private String errorMessage;
public static Set<ValidationError> fromViolations(Set violations) {
Set<ValidationError> errors = new HashSet<ValidationError>();
for (Object o : violations) {
ConstraintViolation v = (ConstraintViolation) o;
ValidationError error = new ValidationError();
error.setClassName(v.getRootBeanClass().getSimpleName());
error.setErrorMessage(v.getMessage());
error.setPropertyPath(v.getPropertyPath().toString());
errors.add(error);
}
return errors;
}
public String getClassName() {
return className;
}
public void setClassName(String className) {
this.className = className;
}
public String getPropertyPath() {
return propertyPath;
}
public void setPropertyPath(String propertyPath) {
this.propertyPath = propertyPath;
}
public String getErrorMessage() {
return errorMessage;
}
public void setErrorMessage(String errorMessage) {
this.errorMessage = errorMessage;
}
#Override
public String toString() {
return "ValidationError{" +
"className='" + className + '\'' +
", propertyPath='" + propertyPath + '\'' +
", errorMessage='" + errorMessage + '\'' +
'}';
}
}
Sample usage:
Set<ConstraintViolation<MyBean>> violations = validator.validate(myBean);
Set<ValidationError> errors = ValidationError.fromViolations(violations);
GsonBuilder gsonBuilder = new GsonBuilder();
Gson gson = gsonBuilder.create();
System.out.println(gson.toJson(errors));
Update
For the sake of record, it is worth mentioning that XStream can serialize the set of constraint violations like a charm:
XStream xstream = new XStream(new JettisonMappedXmlDriver());
xstream.setMode(XStream.NO_REFERENCES);
System.out.println(xstream.toXML(violations));
However the generated the object graph is way too much verbose and is not suitable for use in production anyway. You can see the sample output here.