Spring - Return Raw JSON without double serialization - json

I know there are other posts similar to this, but I haven't found any that help me find a solution for this particular case.
I am trying to return a HashMap<String, Object> from my Controller.
The Object part is a JSON string, but its being double serialized and not returned as a raw JSON string, thus not ending up with extra quotations and escape characters.
Controller function:
#RequestMapping(method = RequestMethod.GET, produces = MediaType.APPLICATION_JSON_VALUE)
public HashMap<String, Object> heartbeat(){
String streamInfo = service.getStreamInfo();
String streamCursorInfo = service.getStreamCursorInfo();
String topicInfo = service.getTopicInfo();
String greeting = "This is a sample app for using Spring Boot with MapR Streams.";
HashMap<String, Object> results = new HashMap();
results.put("greeting", greeting);
results.put("streamInfo", streamInfo);
results.put("streamCursorInfo", streamCursorInfo);
results.put("topicInfo", topicInfo);
return results;
}
Service function:
private String performCURL(String[] command){
StringBuilder stringBuilder = new StringBuilder();
try{
ProcessBuilder processBuilder = new ProcessBuilder(command);
Process p = processBuilder.start();
BufferedReader reader = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = null;
while((line = reader.readLine()) != null){
stringBuilder.append(line);
}
}
catch(Exception e){
LOGGER.error(ExceptionUtils.getRootCauseMessage(e));
}
return stringBuilder.toString();
}
The cURL command I run already returns a raw JSON string. So im just trying to add it to the HashMap to be returned in the heartbeat response.
But every time I run this, my output looks like:
{
"greeting": "This is a sample app for using Spring Boot with MapR Streams.",
"streamCursorInfo": "{\"timestamp\":1538676344564,\"timeofday\":\"2018-10-04 02:05:44.564 GMT-0400 PM\",\"status\":\"OK\",\"total\":1,\"data\":[{\"consumergroup\":\"MapRDBConsumerGroup\",\"topic\":\"weightTags\",\"partitionid\":\"0\",\"produceroffset\":\"44707\",\"committedoffset\":\"10001\",\"producertimestamp\":\"2018-10-03T05:57:27.128-0400 PM\",\"consumertimestamp\":\"2018-09-21T12:35:51.654-0400 PM\",\"consumerlagmillis\":\"1056095474\"}]}",
...
}
If i return only the single string, such as streamInfo then it works fine and doesnt add the extra quotes and escape chars.
Can anyone explain what im missing or need to do to prevent this double serialization?

Instead of returning a HashMap, create an object like this:
public class HeartbeatResult {
private String greeting;
... //other fields here
#JsonRawValue
private String streamCursorInfo;
... //getters and setters here (or make the object immutable by having just a constructor and getters)
}
With #JsonRawValue Jackson will serialize the string as is. See https://www.baeldung.com/jackson-annotations for more info.

streamCursorInfo is a string, not an object => the serialization will escape the " character.
If you are able to return the object containing the data, it will work out of the box. If what you have is just a String, I suggest to serialize it to JsonNode and add it in your response
ObjectMapper objectMapper = new ObjectMapper();
JsonNode streamCursorInfo = objectMapper.readTree(service.getStreamInfo())
results.put("streamCursorInfo", streamCursorInfo);

Related

jackson reading in non-existent and null values to “” and marshalling out “” to non-existent values?

I read through this post Jackson: deserializing null Strings as empty Strings which has this cool trick
mapper.configOverride(String.class)
.setSetterInfo(JsonSetter.Value.forValueNulls(Nulls.AS_EMPTY));
THEN on the flipside, I read through this post Jackson serialization: ignore empty values (or null) which has this cool trick
objectMapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
This is VERY VERY close except I really don't want incoming data to be null in any case. I have the following code printing 4 situations with the above settings BUT want to fix the null piece so any json we unmarshal into java results in
public class MapperTest {
private static final Logger log = LoggerFactory.getLogger(MapperTest.class);
private ObjectMapper mapper = new ObjectMapper();
public MapperTest() {
mapper.setSerializationInclusion(JsonInclude.Include.NON_NULL);
mapper.configOverride(String.class)
.setSetterInfo(JsonSetter.Value.forValueNulls(Nulls.AS_EMPTY));
}
public static void main(String[] args) throws JsonProcessingException {
new MapperTest().start();
}
private void start() throws JsonProcessingException {
//write out java color=null resulting in NO field...
String val = mapper.writeValueAsString(new Something());
log.info("val="+val);
Something something = mapper.readValue(val, Something.class);
log.info("value='"+something.getColor()+"'");
//write out java color="" resulting in NO field...
Something s = new Something();
s.setColor("");
String val2 = mapper.writeValueAsString(new Something());
log.info("val="+val2);
String temp = "{\"color\":null,\"something\":0}";
Something something2 = mapper.readValue(temp, Something.class);
log.info("value2='"+something2.getColor()+"'");
}
}
The output is then
INFO: val={"something":0}
INFO: value='null'
INFO: val={"something":0}
INFO: value2=''
NOTE: The value = 'null' is NOT what I desire and want that to also be empty string. Notice that if customers give a color:'null', it does result in empty string. Non-existence should result in the same thing for us "".
This is a HUGE win in less mistakes in this area 'for us' I mean.
thanks,
Dean

How to test a Datastream with jsonobject in Apache Flink

I am new to testing and i am trying to write a unit test cases on a Flink Datastream which takes input a jsonobject and passes the json object to a processfuntion and it returns a valid or invalid jsonobject when certain rule conditions are met below is the junit test case, below i am trying to compare the output jsonobject from process function with the jsonobject of the input file
#Test
public void testcompareInputAndOutputDataJSONSignal() throws Exception {
org.json.JSONObject jsonObject = toJsonObject();
String input = jsonObject.toString();
String output = JSONDataStreamOutput();
assertEquals(mapper.readTree(input), mapper.readTree(output));
}
below is my toJSONObject and JSONDataStream meathods
public static JSONObject toJsonObject() throws IOException, ParseException {
JSONParser jsonParser = new JSONParser();
FileReader fileReader = new FileReader(getFileFromResources("input.json"));
JSONObject obj = (JSONObject) jsonParser.parse(fileReader);
return obj;
}
public String SignalDataStreamOutput() throws Exception {
final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
DataStream<JSONObject> validSignal = env.fromElements(toJsonObject())
.process(new JsonFilter());
String outputFolder = "output";
validSignal.writeAsText(outputFolder).setParallelism(1);
env.execute();
String content = new String(Files.readAllBytes(Paths.get("output.txt")));
return content;
}
What i am doing is i am converting a jsonfile to jsonobject using the toJSONObject method and sending to a data stream using SignalDataStreamOutput method which will intern send it to a process function in JsonFilter class and validate it against a set of rules and if it's valid it will return a jsonobject and when trying to access the jsonobject directly from stream i am getting value like org.apache.flink#994jdkeiri so i am trying to write the output to a file and trying to read it back to a string and comparing it in test method but this is a work around process and i found a link to use Mockito framework here i changed it to use json object like below
final Collector<JSONObject> collectorMock = (Collector<JSONObject>)Mockito.mock(JsonFilter.class);
final Context contextMock = Mockito.mock(Context.class);
#Test
public void testcompareInputAndOutputDataForValidSignal() throws Exception {
org.json.JSONObject jsonObject = convertToJsonObject();
Mockito.verify(collectorMock).collect(jsonObject);
}
but the above approach is also not working can you suggest me simplified approach to test the json object

Cannot pass string to AWS Lambda using InvokeRequest.withPayload() method

I have a lambda function that accepts a String as an input parameter. When running the lambda function I get the following error:
Can not deserialize instance of java.lang.String out of START_OBJECT token\n
This is what my code too call it looks like:
InvokeRequest request = new InvokeRequest();
final String payload = "";
request.withFunctionName(FUNCTION_NAME).withPayload((String) null);
InvokeResult invokeResult = lambdaClient.invoke(request);
Assert.assertEquals(new String (invokeResult.getPayload().array(), "UTF-8"), "Success");
And this is what my handler looks like:
public String handleRequest(String s, Context context) {}
Now the contents of the string don't matter, it could be null it could be anything. I don't use the input. The obvious solution is to remove it, but because of an annotation generator i'm using I can't do that. I've tried a ByteBuffer input, String, empty String, JSON String {\"s\":\"s\"} but nothing seems to work. I believe I need to pass in a string (i.e no {}). But since I'm using InvokeRequest I don't believe I can do that. Any suggestions would be greatly appreciated.
It works by passing a JSON valid String.
String payload = "{ \"subject\" : \"content\"}";
request.withFunctionName(functionName)
.withPayload(payload);
At the receiving end you have to map it from Object to String again if that's what you want. Here I used Jackson ObjectMapper for this.
ObjectMapper objectMapper = new ObjectMapper();
try {
String payload = objectMapper.writeValueAsString(input);
} catch (JsonProcessingException e) {
e.printStackTrace();
}

How to deserialize Jackson Json NULL String to Date with JsonFormat

I have looked a lot but still couldn't get the answer so far, any help is really appreciated!
I have a simple String to Date field mapping and try to read a JSON string to Java object.
#JsonInclude(value=Include.NON_EMPTY)
#JsonFormat(shape=JsonFormat.Shape.STRING, pattern="dd-MMM-yyyy", timezone="PST")
protected Date eolAnnounceDate;
However I am getting the following exception if the JSON string value is empty.
Can you someone tell me how to get around this? I have tried a few options but they are all for serialization.
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.setSerializationInclusion(Include.NON_NULL);
objectMapper.setSerializationInclusion(Include.NON_EMPTY);
Exception :
java.lang.IllegalArgumentException: Failed to parse Date value 'NULL' (format: "dd-MMM-yyyy"): Unparseable date: "NULL"
com.fasterxml.jackson.databind.deser.std.DateDeserializers$DateBasedDeserializer._parseDate(DateDeserializers.java:180)
com.fasterxml.jackson.databind.deser.std.DateDeserializers$DateDeserializer.deserialize(DateDeserializers.java:279)
com.fasterxml.jackson.databind.deser.std.DateDeserializers$DateDeserializer.deserialize(DateDeserializers.java:260)
com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:464)
com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:98)
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295)
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121)
com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:230)
com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:207)
com.fasterxml.jackson.databind.deser.std.CollectionDeserializer.deserialize(CollectionDeserializer.java:23)
com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:464)
com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:98)
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:295)
com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:121)
com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:2888)
com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2034)
com.cisco.cre.dao.impl.ElasticsearchDAOImpl.getListByIdsFilter(ElasticsearchDAOImpl.java:94)
Thanks
- Atul
Your problem is not that a null value is passed in the JSON. The problem is that the JSON contains a string that has the value "NULL".
So, in order to fix this there are a number of available approaches. I think that the following two will work for this case.
Alternative 1: Fix the JSON
The first alternative is to fix the JSON so that it does not contain the the string value "NULL" and instead contain the value null (not quoted) or simply skip it.
Imagine the following POJO:
public class DatePojo {
#JsonInclude(value= JsonInclude.Include.NON_EMPTY)
#JsonFormat(shape=JsonFormat.Shape.STRING, pattern="dd-MMM-yyyy", timezone="PST")
#JsonProperty("date")
private Date date;
}
The following test shows that valid dates, empty values and null values work:
#Test
public void testJson() throws IOException {
String jsonWithValidDate = "{\"date\":\"12-Jun-1982\"}";
String jsonWithNoDate = "{}";
String jsonWithNullDate = "{\"date\":null}";
ObjectMapper mapper = new ObjectMapper();
final DatePojo pojoWithValidDate = mapper.readValue(jsonWithValidDate, DatePojo.class);
final DatePojo pojoWithNoDate = mapper.readValue(jsonWithNoDate, DatePojo.class);
final DatePojo pojoWithNullDate = mapper.readValue(jsonWithNullDate, DatePojo.class);
Assert.assertNotNull(pojoWithValidDate.date);
Assert.assertNull(pojoWithNoDate.date);
Assert.assertNull(pojoWithNullDate.date);
}
However, if you pass along the value "NULL" the test fails since "NULL" can not be parsed as a date:
#Test(expected = JsonMappingException.class)
public void testInvalidJson() throws IOException {
String jsonWithNullString = "{\"date\":\"NULL\"}";
ObjectMapper mapper = new ObjectMapper();
mapper.readValue(jsonWithNullString, DatePojo.class); // Throws the exception
Assert.fail();
}
Alternative 2: provide your own converter that handles "NULL"
If it is not possible to fix the JSON (as described in alternative 1) you can provide your own converter.
Setup your pojo like this instead:
public class DatePojo {
#JsonProperty("date")
#JsonDeserialize(converter = MyDateConverter.class)
private Date date;
}
And provide a converter along the lines of:
public class MyDateConverter extends StdConverter<String, Date> {
#Override
public Date convert(final String value) {
if (value == null || value.equals("NULL")) {
return null;
}
try {
return new SimpleDateFormat("dd-MMM-yyyy").parse(value);
} catch (ParseException e) {
throw new IllegalStateException("Unable to parse date", e);
}
}
}
Then, you should be all set. The following test passes:
#Test
public void testJson() throws IOException {
String jsonWithValidDate = "{\"date\":\"12-Jun-1982\"}";
String jsonWithNoDate = "{}";
String jsonWithNullDate = "{\"date\":null}";
String jsonWithNullString = "{\"date\":\"NULL\"}"; // "NULL"
ObjectMapper mapper = new ObjectMapper();
final DatePojo pojoWithValidDate = mapper.readValue(jsonWithValidDate, DatePojo.class);
final DatePojo pojoWithNoDate = mapper.readValue(jsonWithNoDate, DatePojo.class);
final DatePojo pojoWithNullDate = mapper.readValue(jsonWithNullDate, DatePojo.class);
final DatePojo pojoWithNullStr = mapper.readValue(jsonWithNullString, DatePojo.class); // Works
Assert.assertNotNull(pojoWithValidDate.date);
Assert.assertNull(pojoWithNoDate.date);
Assert.assertNull(pojoWithNullDate.date);
Assert.assertNull(pojoWithNullStr.date); // Works
}
IMO, the best approach is to use alternative 1 where you simply change the JSON.

parse JSON with gson and GsonBuilder()

String jsons = "{'appname':'application', 'Version':'0.1.0', 'UUID':'300V', 'WWXY':'310W', 'ABCD':'270B', 'YUDE':'280T'}";
This is my json string. How can i parse it to GsonBuilder() that i will get object back? I try few thinks but none works.
I also read https://sites.google.com/site/gson/gson-user-guide
public class YourObject {
private String appname;
private String Version;
private String UUID;
private String WWXY;
private String ABCD;
private String YUDE;
//getters/setters
}
parse to Object
YourObject parsed = new Gson().fromJson(jsons, YourObject.class);
or
YourObject parsed = new GsonBuilder().create().fromJson(jsons, YourObject.class);
minor test
String jsons = "{'appname':'application', 'Version':'0.1.0', 'UUID':'300V', 'WWXY':'310W', 'ABCD':'270B', 'YUDE':'280T'}";
YourObject parsed = new Gson().fromJson(jsons, YourObject.class);
works well
EDIT
in this case use JsonParser
JsonObject object = new JsonParser().parse(jsons).getAsJsonObject();
object.get("appname"); // application
object.get("Version"); // 0.1.0
JSON uses double quotes ("), not single ones, for strings so the JSON you have there is invalid. That's likely the cause of any issues you're having converting it to an object.