Postgres, jsonb & jsonb_set - json

Depending on the HTTP_USER_AGENT I have to return a very specialized formatted version of a json structure to the client.
The json object is generated as usual with standard postgres functions.
Lets assume the generated json looks similar to this:
{
"return_code" : 0,
"payload" : {
"name" : "smith",
"age": 17,
"address" :{
"street" : "<whatever>",
"city" : "<anycity>"
}
}
}
Now under some circumstances I have to return this json in the following format:
{
"return_code" : 0,
"payload" : "{"name" : "smith", "age": 17, "address" :"{"street" : "<whatever>", "city" : "<anycity>"}"}"
}
As you can see the nested payload object should be returned as a string - masking ignored here for better readability.
Further else the address property should also be returned as a string, not as a json object.
My postgres code that should do this is simply:
response := jsonb_set(response, '{payload}', to_jsonb((response->'payload')::text));
But the result from the code above looks like:
{
"return_code" : 0,
"payload" : "{"name" : "smith", "age": 17, "address" :{"street" : "<whatever>", "city" : "<anycity>"}}"
}
Consider the quotes are missing (just two) for the address-object.
How can I fix this?
Thank You!

It seems to me that you need a further level of escaping / quoting on the address property, because you have:
Your substitution: { "payload" : { ... } } -> { "payload" : "{ ... }" }
Extra substitution: { "address": { ... } } -> { "address": "{ ... }" }
You'll need to do this before the existing line, so I think what you want is this (requires an extra jsonb variable, payload):
payload := jsonb_set(response->'payload', '{address}', to_jsonb((response->'payload'->address)::text));
response := jsonb_set(response, '{payload}', to_jsonb(payload::text));

Related

How to retrieve all key-value pairs avoiding key duplication from JSON in Groovy script

I am totally new to groovy script and would like some help to solve this out. I have a JSON response I want to manipulate and get desired parameters back by avoiding duplication. The Json response does not have indexes like 0,1,2.. that I can iterate through.
Here is the response that I want to work with:
{
"AuthenticateV2" : {
"displayName" : "Verification of authentication",
"description" : "notification ",
"smsTemplate" : "authentication.v2.0_sms",
"emailHeaderTemplate" : "v2.0_header",
"emailBodyTemplate" : "html",
"parameters" : {
"displayName" : "USER_DISPLAY_NAME",
"actionTokenURL" : "VERIFICATION_LINK",
"customToken" : "VERIFICATION_CODE"
},
"supportedPlans" : [
"connectGo"
]
},
"PasswordRecovery" : {
"displayName" : "Verification of password recovery",
"description" : "notification",
"smsTemplate" : "recovery.v1.0_sms",
"emailHeaderTemplate" : "recovery.v1.0_header",
"emailBodyTemplate" : "recovery.v1.0_body_html",
"parameters" : {
"displayName" : "USER_DISPLAY_NAME",
"actionTokenURL" : "VERIFICATION_LINK",
"customToken" : "VERIFICATION_CODE",
"adminInitiated" : false,
"authnId" : "AUTHENTICATION_IDENTIFIER",
"authnType" : "EMAIL",
"user" : {
"displayName" : "USER_DISPLAY_NAME"
}
},
"supportedPlans" : [
"connectGo"
]
},
"PasswordReset" : {
"displayName" : "password reset",
"description" : "notification",
"smsTemplate" : "recovery.v1.0_sms",
"emailHeaderTemplate" : "recovery.v1.0_header",
"emailBodyTemplate" : "html",
"parameters" : {
"displayName" : "USER_DISPLAY_NAME",
"user" : {
"displayName" : "USER_DISPLAY_NAME"
}
}
The expected output that I want to have:
{
"displayName" : "USER_DISPLAY_NAME",
"actionTokenURL" : "VERIFICATION_LINK",
"customToken" : "VERIFICATION_CODE",
"customToken" : "VERIFICATION_CODE",
"adminInitiated" : false,
"authnId" : "AUTHENTICATION_IDENTIFIER",
"authnType" : "EMAIL"
}
I need to retrieve all fields under parameters tag and also want to avoid duplication
You should first get familiar with parsing and producing JSON in Groovy.
Then, assuming the provided response is a valid JSON (it's not - there are 2 closing curlies (}) missing at the end) to get all the parameters keys merged into one JSON we have to convert the JSON string into a Map object first using JsonSlurper:
def validJsonResponse = '<your valid JSON string>'
Map parsedResponse = new JsonSlurper().parseText(validJsonResponse) as Map
Now, when we have a parsedResponse map we can iterate over all the root items in the response and transform them into the desired form (which is all the unique parameters keys) using Map::collectEntries method:
Map uniqueParameters = parsedResponse.collectEntries { it.value['parameters'] }
Finally, we can convert the uniqueParameters result back into a pretty printed JSON string using JsonOuput:
println JsonOutput.prettyPrint(JsonOutput.toJson(uniqueParameters))
After applying all the above we'll get the output
{
"displayName": "USER_DISPLAY_NAME",
"actionTokenURL": "VERIFICATION_LINK",
"customToken": "VERIFICATION_CODE",
"adminInitiated": false,
"authnId": "AUTHENTICATION_IDENTIFIER",
"authnType": "EMAIL",
"user": {
"displayName": "USER_DISPLAY_NAME"
}
}
If you want to get rid of user entry from the final output just remove it from the resulting uniqueParameters map (uniqueParameters.remove('user')) before converting it back to JSON string.

My JSON parsing fails

Im using SwiftyJSON, Alamofire_SwiftyJSON and Alamofire libraries. I'm trying to parse my response but it returns nothing("definitions" fails to print).
JSON isn't empty it has got the response.
So, my code looks like the following (nothing fails, it all compiles)
Alamofire.request(request).responseSwiftyJSON { dataResponse in
if let JSON = dataResponse.result.value {
print(JSON)
if let definitions = JSON["results"]["lexicalEntries"]["entries"]["senses"]["definitions"].string {
print(definitions)
print("Hello")
}}
}
My response model looks like (this is not the whole response, that's just what I want to reach :
{
"results" : [
{
"language" : "en",
"id" : "ace",
"type" : "headword",
"lexicalEntries" : [
{
"language" : "en",
"entries" : [
{
"etymologies" : [
"Middle English (denoting the ‘one’ on dice): via Old French from Latin as ‘unity, a unit’"
],
"grammaticalFeatures" : [
{
"type" : "Number",
"text" : "Singular"
}
],
"homographNumber" : "000",
"senses" : [
{
"definitions" : [
"a playing card with a single spot on it, ranked as the highest card in its suit in most card games"
I think that my problem is in, should I add any parentheses or any symbols?
if let definitions = JSON["results"]["lexicalEntries"]["entries"]["senses"]["definitions"].string
You are not indexing the corresponding arrays in your JSON response, to access the definitions array you can simply use
JSON['results'][0]['lexicalEntries'][0]['entries'][0]['senses'][0]['definitions']

Search inside JSON with Elastic

I have an index/type in ES which has the following type of records:
body "{\"Status\":\"0\",\"Time\":\"2017-10-3 16:39:58.591\"}"
type "xxxx"
source "11.2.21.0"
The body field is a JSON.So I want to search for example the records that have in their JSON body Status:0.
Query should look something like this(it doesn't work):
GET <host>:<port>/index/type/_search
{
"query": {
"match" : {
"body" : "Status:0"
}
}
}
Any ideas?
You have to change the analyser settings of your index.
For the JSON pattern you presented you will need to have a char_filter and a tokenizer which remove the JSON elements and then tokenize according to your needs.
Your analyser should contain a tokenizer and a char_filter like these ones here:
{
"tokenizer" : {
"type": "pattern",
"pattern": ","
},
"char_filter" : [ {
"type" : "mapping",
"mappings" : [ "{ => ", "} => ", "\" => " ]
} ],
"text" : [ "{\"Status\":\"0\",\"Time\":\"2017-10-3 16:39:58.591\"}" ]
}
Explanation: the char_filter will remove the characters: { } ". The tokenizer will tokenize by the comma.
These can be tested using the Analyze API. If you execute the above JSON against this API you will get these tokens:
{
"tokens" : [ {
"token" : "Status:0",
"start_offset" : 2,
"end_offset" : 13,
"type" : "word",
"position" : 0
}, {
"token" : "Time:2017-10-3 16:39:58.591",
"start_offset" : 15,
"end_offset" : 46,
"type" : "word",
"position" : 1
} ]
}
The first token ("Status:0") which is retrieved by the Analyze API is the one you were using in your search.

Matching data in JsonPath with wiremock

I'm trying to create mocks for my login procedure. I use POST method with a couple of fields and login object (with login, password, etc.)
For that I'm using JsonPath. Code below:
{
"request": {
"method": "POST",
"url": "/login",
"bodyPatterns" : [
{"matchesJsonPath" : "$.method"},
{"matchesJsonPath" : "$.params[?(#.clientVersion == "1")]"},
{"matchesJsonPath" : "$.params.login"},
{"matchesJsonPath" : "$.params.password"}
]
},
"response": {
"status": 200,
"bodyFileName": "login.json"
}
}
I'm checking the clientVersion because it's similar to the examples.
My problem is, that with te given POST JSON:
{
"method": "login",
"params": {
"clientVersion": "1",
"login": "test#test.com",
"password": "681819535da188b6ef2"
}
}
I receive 404.
However, when I change
{"matchesJsonPath" : "$.params[?(#.clientVersion == "1")]"},
to normal
{"matchesJsonPath" : "$.params.clientVersion"},
everything works just fine.
So - how to check inside wiremock, using matchesJsonPath if given field equals some value?
How to apply it to the root field like method in my case?
And while we're at it - I had similar problems with checking if the value is not null. I tried to apply regular expressions and such - no luck.
It's working in my case :
wiremock:
"request": {
"urlPathPattern": "/api/authins-portail-rs/authins/inscription/infosperso",
"bodyPatterns" : [
{"matchesJsonPath" : "$[?(#.nir == '123456789')]"},
{"matchesJsonPath" : "$[?(#.nomPatronyme == 'aubert')]"},
{"matchesJsonPath" : "$[?(#.prenoms == 'christian')]"},
{"matchesJsonPath" : "$[?(#.dateNaissance == '01/09/1952')]"}
],
"method": "POST"
}
Json:
{
"nir": "123456789",
"nomPatronyme": "aubert",
"prenoms": "christian",
"dateNaissance": "01/09/1952"
}
Following worked for me.
"matchesJsonPath" : "$.rootItem.itemA[0].item..[?(#.fieldName=='file')]"
Json :
{
"rootItem" : {
"itemA" : [
{
"item" : {
"fieldName" : "file",
"name" : "test"
}
}
]
}
}
Wiremock
{
"request" : {
"urlPattern" : "/testjsonpath",
"method" : "POST",
"bodyPatterns" : [ {
"matchesJsonPath" : "$.rootItem.itemA[0].item..[?(#.fieldName=='file')]"
} ]
},
"response" : {
"status" : 200,
"body" : "{\"result\": \"success\"}",
"headers" : {
"Content-Type" : "application/json"
}
}
}
Update Wiremock. It should work with newer versions >= 2.0.0-beta. Its JsonPath dependency was very outdated (GitHub #261).
Using the double dots operator is semantically not the same, as the filter will also match for elements with the same name deeper down the tree.
try with double dots operator (recursive)
$..params[?(#.clientVersion == "1")]

Mongo Search by Json Name

I have a json document which has a structure as is displayed below:
{
"125": {
"name": "sample name one",
"age" : 22,
},
"126": {
"name": "sample name two",
"age" : 27,
},
"127": {
"name": "sample name three",
"age" : 21,
}
}
I want to return the object represented by 125, i.e. the following object :
{
"name": "sample name one",
"age" : 22,
}
I have tried the following query from the console, but I cannot get the result I need:
db.persons.find({"125": {$exists: true}}).
I would like to know how to query the database to return the data that I need. Thank you for any help.
db.persons.find({"125": {$exists: true}})
would return whole document, that has "125" key, not only the "125" key.
If You need to return only the "125", You should use projection, passing it as second argument to find:
db.persons.find({"125": {$exists: true}}, {"125": 1})
then You will get documents containing "125" key, without any other keys (except _id of that document)
To omit _id of document, You need to use:
db.persons.find({"125": {$exists: true}}, {"125": 1, _id: 0})
EDIT:
Did You consider using array for those subdocuments with repeating schema? It would be more natural, idiomatic, and also give You some additional possibilities.
In mongo search by JSON key / field / name, use $exists operator
db.things.find( { '125' : { $exists : true } } );