I am persisting a JSON object to MongoDB here is relevant snippet:
Viewed: [
-{
ViewedID: "8992ade400a"
Dockey: "3323aba3233"
PID: "32399a"
actionsTaken: "email|direct message|report seeker"
viewDate: "01-APR-2014"
MessageSent: "true"
-Message: [
-{
MessageID: "123aca323"
Delivered: "True"
Opened: "True"
ClickThroughRate: "NotBad"
MessageDate: "02-APR-2014"
-Response: [
-{
ResponseId: "a323a9da"
ResponseDate: "23-APR-2014"
}
]
}
Here is how I am setting the JSON on my persisting object:
trackingData.setEventdata((DBObject)JSON.parse(tracker.getEventData().toString()));
where tracker.getEventData() returns ObjectNode when I persist the DBObject to mongo I see the dates such "viewDate" : "01-APR-2014" and "ResponseDate" : "23-APR-2014" as Strings.
I need these attributes to be converted to type date so I may query against these dates. Is there anyway to specify that these be handled as date objects either before or after parsing that JSON to DBObject?
Cheers,
Will
The com.mongodb.util.JSON.parse method supports Extended JSON format, so if the JSON was actually formatted that way then this would automatically be cast as a date.
But since it isn't then you would either need to implement a custom parser that handles the "date" elements in your JSON or otherwise just manipulate the BSON before sending it on to MongoDB.
The simplest way is just to manipulate, with something like as shown in this example:
try {
DBObject content = (DBObject)com.mongodb.util.JSON.parse("{ \"date\": \"01-APR-2014\" }");
System.out.println(content);
SimpleDateFormat sdf = new SimpleDateFormat("dd-MMM-yyyy");
sdf.setTimeZone(TimeZone.getTimeZone("GMT"));
content.put("date",sdf.parse((String) content.get("date")));
System.out.println(content);
} catch (ParseException e) {
e.printStackTrace();
}
So after processing the .parse() you can basically do that date conversion for each relevant date field in your source. Also keep in mind that the data is going to be stored with dates as "UTC/GMT", so make sure that the Timezone you are getting from the source is correctly represented.
Related
I am looking at extracting the root element of a JSON document. It looks like this is possible neither using JsonPointer nor JsonPath as my attempts to look up for such an expression has been unsuccessful. Any tips would be appreciated. TIA.
Sample document:
{
"MESSAGE1_ROOT_INPUT": {
"CTRL_SEG": "test"
}
}
The below using gson 2.9.0:
$.*~
produces:
{"CTRL_SEG": "test"}
while JSONPath Online produces this:
[
"MESSAGE1_ROOT_INPUT"
]
The attempt is to get text "MESSAGE1_ROOT_INPUT" using JsonPath/JsonPointer expression(s). Note that, extracting this the traditional (substring or regex on a stringified json text) way, would preferably be my last resort.
Background: We are building an API service that accepts JSON documents with different roots. Such as, MESSAGE2_ROOT_INPUT, MESSAGE3_ROOT_INPUT, etc. It is based on this, the routing of a message further will occur.
Supported/Employed Languages: Java/GSON Library/RegEx
Gson does not natively support JSONPath or JSON Pointer. However, you can quite efficiently obtain the name of the first property using JsonReader:
public static String getFirstPropertyName(Reader reader) throws IOException {
// Don't have to call JsonReader.close(); that would just close the provided reader
JsonReader jsonReader = new JsonReader(reader);
jsonReader.beginObject();
return jsonReader.nextName();
}
There are however two things to keep in mind:
This only reads the beginning of the JSON document; it neither verifies that the complete JSON document has valid syntax, nor checks if there might be more top-level properties
This consumes some data from the Reader; to further process the data you have to buffer the data to allow re-reading it again (you can also first store the JSON in a String and pass a StringReader to JsonReader)
I need to be able to PUT a JSON data from POSTMAN, with no fixed format, store it in database(in MYSQL preferably with datatype: JSON) and then send a GET request to get the same value back. The data can be uniquely identified by an id that I'll be sending as a path variable.
I cannot define an entity class as the JSON will have no fixed format. How do I proceed?
#PutMapping("/sample/{sampleNo}")
public ResponseEntity<Object> addSampleData(
#ApiParam("Sampleto PUT") #PathVariable Long sampleNo, #RequestBody String sampleBody) {
if (sampleBody!= null) {
sampleService.save(new Sample(sampleNo, sampleBody));
return new ResponseEntity<>(HttpStatus.OK);
} else {
return new ResponseEntity<>(HttpStatus.BAD_REQUEST);
}
}
Not sure how to proceed with the "save" functionality so that the data is stored as a JSON object, because even if I've used datatype JSON n mySQL, GET returns a string with "\".
{
"sampleNo": 1,
"sampleData": "{\"sample\": \"test\", \"sampleNo\": \"20\"}"
}
Example: PUT request body
{
"field1": "test",
"field2": 20
}
GET response body expected
{
"field1": "test",
"field2": 20
}
P.S: I don't need to process the data, so it doesn't matter how I store it, I just need to be able to GET it back in the same format(JSON) it arrived(in the PUT req).
sampleData is returned as JSON String, so it is reasonable to include escape sequence \":
This is a valid JSON String:
String json = "{\"sample\": \"test\", \"sampleNo\": \"20\"}";
This does not compile:
String invalidJson = "{"sample": "test", "sampleNo": "20"}";
Solution:
In order to have the expected response, you have to map your MySQL JSON column into an Object.
There are several ways to achieve this, take a look at a solution here. It maps JSON column into a Map<String, Object>.
The Json body I want to deserialize is something like below
{
"status": "OK",
"place_id": "07e0a63d862b2982e4c4b7e655f148d2",
"scope": "APP"
}
Below is the Json body I want to build from above Json after deserializing it
{
"place_id": "07e0a63d862b2982e4c4b7e655f148d2"
}
Because your JSON data appears to be rather small you could use Gson's JsonParser.parseString(String) method to parse the data into an in-memory representation, then copy the relevant JSON object member to a new JsonObject and serialize that to JSON using Gson.toJson(JsonElement):
JsonObject original = JsonParser.parseString(json).getAsJsonObject();
JsonObject copy = new JsonObject();
// Might also have to add some validation to make sure the member
// exists and is a string
copy.add("place_id", original.get("place_id"));
String transformedJson = new Gson().toJson(copy);
If this solution turns out to not be efficient enough for you, you could also have a look at JsonReader and JsonWriter.
I am in the process of converting a really old Grails app to the latest version (3.3.0). Things have been a bit frustrating, but I'm pretty close to migrating everything except the JSON and XML marshallers which were previously registered in my BootStrap init.
The previous marshaller registering looked like this:
// register JSON marshallers at startup in all environments
MarshallerUtils.registerMarshallers()
This was defined like this:
class MarshallerUtils {
// Registers marshaller logic for various types that
// aren't supported out of the box or that we want to customize.
// These are used whenever the JSON or XML converters are called,
// e.g. return model as JSON
static registerMarshallers() {
final dateTimeFormatter = ISODateTimeFormat.dateTimeNoMillis()
final isoDateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ")
// register marshalling logic for both XML and JSON converters
[XML, JSON].each { converter ->
// This overrides the marshaller from the joda time plugin to
// force all DateTime instances to use the UTC time zone
// and the ISO standard "yyyy-mm-ddThh:mm:ssZ" format
converter.registerObjectMarshaller(DateTime, 10) { DateTime it ->
return it == null ? null : it.toString(dateTimeFormatter.withZone(org.joda.time.DateTimeZone.UTC))
}
converter.registerObjectMarshaller(Date, 10) { Date it ->
return it == null ? null : isoDateFormat.format(it)
}
converter.registerObjectMarshaller(TIMESTAMP, 10) { TIMESTAMP it ->
return it == null ? null : isoDateFormat.format(it.dateValue())
}
}
}
}
During the migration, I ended up converting all instances of org.joda.time.DateTime to java.time.ZonedDateTime:
class MarshallerUtils {
// Registers marshaller logic for various types that
// aren't supported out of the box or that we want to customize.
// These are used whenever the JSON or XML converters are called,
// e.g. return model as JSON
static registerMarshallers() {
final dateTimeFormatter = DateTimeFormatter.ISO_ZONED_DATE_TIME
final isoDateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH:mm:ss.SSSZ")
// register marshalling logic for both XML and JSON converters
[XML, JSON].each { converter ->
// This overrides the marshaller from the java.time to
// force all DateTime instances to use the UTC time zone
// and the ISO standard "yyyy-mm-ddThh:mm:ssZ" format
converter.registerObjectMarshaller(ZonedDateTime, 10) { ZonedDateTime it ->
return it == null ? null : it.toString(dateTimeFormatter.withZone(ZoneId.of("UTC")))
}
converter.registerObjectMarshaller(Date, 10) { Date it ->
return it == null ? null : isoDateFormat.format(it)
}
converter.registerObjectMarshaller(TIMESTAMP, 10) { TIMESTAMP it ->
return it == null ? null : isoDateFormat.format(it.dateValue())
}
}
}
}
Unfortunately, after the upgrade to Grails 3.3.0, this marshaller registering doesn't seem to be used at all, no matter what I try to do.
I do know that there is a new "JSON Views" way of doing things, but this particular service has many endpoints, and I don't want to write custom converters and ".gson" templates for all of them, if everything is already in the format I need. I just need the responses to be in JSON and the dates to behave property (be formatted strings).
Instead, what I am finding (compared to the previous behavior, is that the properties which utilize ZonedDateTime are "exploded" in my JSON output. There is an insane amount of garbage date object information that is not needed, and it is not formatted as a simple string as I expect.
I have tried a few things (mostly per recommendations in the offical latest Grails documentation) ---
Custom Converters
Default Date Format
Adding configurations for grails views in application.yml:
views:
json:
generator:
dateFormat: "yyyy-MM-dd'T'HH:mm:ss.SSSZ"
locale: "en/US"
timeZone: "GMT"
Creating this path under "src":
src/main/resources/META-INF/services/grails.plugin.json.builder.JsonGenerator$Converter
And adding a Converter to my domain class which is named in the file above^:
class MultibeamFileConverter implements JsonGenerator.Converter {
final DateTimeFormatter isoDateFormat = DateTimeFormatter.ofPattern("yyyy-MM-dd'T'HH:mm:ss.SSSZ").withZone(ZoneId.of("UTC"));
#Override
boolean handles(Class<?> type) {
MultibeamFile.isAssignableFrom(type)
}
#Override
Object convert(Object value, String key) {
MultibeamFile multibeamFile = (MultibeamFile)value
multibeamFile.startTime.format(isoDateFormat)
multibeamFile.endTime.format(isoDateFormat)
return multibeamFile
}
}
In my controller, I have changed:
return multibeamCatalogService.findFiles(cmd, params)
To this (in order to get JSON output in the browser as before):
respond multibeamCatalogService.findFiles(cmd, params), formats: ['json', 'xml']
Unfortuantely, most permutations I can think to try of the above have resulted in errors such as "Could not resolve view". Otherwise, when I am getting a response, the major issue is that the date is not formatted as a string. This function was previously performed by the Marshaller.
I am getting pretty frustrated. Can someone please tell me how to format ZonedDateTime as a simple string (e.g. - "2009-06-21T00:00:00Z") in my JSON output instead of a giant object like this? Simply converting to java.util.Date causes the "Could not resolve view" error to show up again; consequently, that expects me to make a ".gson" view which never ends up showing the format I expect or is empty.
"startTime": {
"dayOfMonth": 26,
"dayOfWeek": {
"enumType": "java.time.DayOfWeek",
"name": "FRIDAY"
},
"dayOfYear": 207,
"hour": 0,
"minute": 0,
"month": {
"enumType": "java.time.Month",
"name": "JULY"
},
"monthValue": 7,
"nano": 0,
"offset": {
"id": "-06:00",
"rules": {
"fixedOffset": true,
"transitionRules": [],
"transitions": []
},
"totalSeconds": -21600
}, ... // AND SO ON FOR_EVAH
The simple answer is to format a ZonedDateTime object you call .format(DateTimeFormatter). It depends what format you want. You can specify your own or use some of the predefined ones in DateTimeFormatter.
I too though would love to know if there's an easy way to say "for every endpoint display it as json". The only way I've found so far is to have this in every controller class, which isn't too bad but seems silly. I'm using respond followed by a return in my controller methods.
static responseFormats = ['json'] // This is needed for grails to indicate what format to use for respond.
Though I still see the error logged, but rest api still appears to work, "Could not resolve view" for any endpoint I hit.
I think the title of this is wrong but can't create a title that reflects, in the abstract, what I want to achieve.
I am writing a function which calls a service and retrieves data as a JSON string. The function parses the string with a JSON type provider. Under certain conditions I want to amend properties on that JSON object and then return the string of the amended object. So if the response from the call was
{"property1" : "value1","property2" : "value2", "property3": "value3" }
I want to change property3 to a new value and then return the JSON string.
If the JsonProvider was mutable this would be an exercise like:
type JsonResponse =
JsonProvider<""" {"property1" : "value1",
"property2" : "value2",
"property3": "value3" } """>
let jsonResponse = JsonResponse.Parse(response)
jsonResponse.Property3 <- "new value"
jsonResponse.ToString()
However, this does not work as the property cannot be set. I am trying to ascertain the best way to resolve this. I am quite happy to initialise a new object based on the original response but with amended parameters but I am not sure if there is an easy way to achieve this.
For reference, the JSON object is much more involved than the flat example given and contains a deep hierarchy.
Yes, you would need to create a new object, changing the bits you want and using the existing object's values for the rest. We added write APIs for both the XML and JSON type providers a while back. You will notice the types representing your JSON have constructors on them. You can see an example of this in use at the bottom of this link