Deserialize dynamic json object with arrays to postgres table Using postgres - json

"code1": {
"obj1": [ {
"sample_name": "1",
"sample_code": "1",
"sample_serial": "01",
"parameters": [
{
"param1": "param1",
"param2": "param2",
"param3": "param3",
"param4": "param4",
"param5": "param5",
"param6": "param6"
}],
]
}
This is my sample json and I would like to deserialize it dynamically in backend using postgres .I am able to deserialize for array with "Testdata" but for code "code1": {
"obj1" and so on I am not able to deserialize it . Code 1 , obj1 ,obj2 are dynamically created in Json.
Any help would be appreciated. Thanks
I tried to DE serialize the array to table . I am expecting to DE serialize the object with arrays . Also I am able to get all keys from json_object_keys(jsondata) as key in but not able to get the data

There are many ways to deserialize a json data depending on its own structure.
Here is one possible solution :
SELECT a->>'key' AS key, a->>'value' AS value
FROM jsonb_path_query('{"code1": { "obj1": [ { "sample_name": "1", "sample_code": "1", "sample_serial": "01", "parameters": [ { "param1": "param1", "param2": "param2", "param3": "param3", "param4": "param4", "param5": "param5", "param6": "param6" }]}]}}' :: jsonb, '$.** ? (#.type() == "object").keyvalue()') AS a
Result :
key
value
code1
{"obj1": [{"parameters": [{"param1": "param1", "param2": "param2", "param3": "param3", "param4": "param4", "param5": "param5", "param6": "param6"}], "sample_code": "1", "sample_name": "1", "sample_serial": "01"}]}
obj1
[{"parameters": [{"param1": "param1", "param2": "param2", "param3": "param3", "param4": "param4", "param5": "param5", "param6": "param6"}], "sample_code": "1", "sample_name": "1", "sample_serial": "01"}]
parameters
[{"param1": "param1", "param2": "param2", "param3": "param3", "param4": "param4", "param5": "param5", "param6": "param6"}]
sample_code
1
sample_name
1
sample_serial
01
parameters
[{"param1": "param1", "param2": "param2", "param3": "param3", "param4": "param4", "param5": "param5", "param6": "param6"}]
sample_code
1
sample_name
1
sample_serial
01
param1
param1
param2
param2
param3
param3
param4
param4
param5
param5
param6
param6
param1
param1
param2
param2
param3
param3
param4
param4
param5
param5
param6
param6
see dbfiddle

Related

How to resolve a recursive nested tree in jsonb?

So I have a flattened tree like this:
[{
aid: "id3"
atype: ""
data: ["id1", "id2"]
},
{
aid: "id1"
atype: ""
data: ["id3", "id2"]
},
{
aid: "id2"
atype: ""
bdata: {aid: "id4", atype: "nested", data: ["id1", "id3"]}
data: []
}]
I want to gather that tree and resolve ids into data with recursion loops into something like this (say we start from "id3"):
{
aid: "id3"
payload: "1"
data: [
"id1":{
aid: "id1"
atype: ""
data: ["id3":Null, "id2":Null]
},
"id2":{
aid: "id2"
atype: ""
bdata: {aid: "id4", atype: "nested", data: ["id1":Null, "id3":Null]}
data: []
}]
}
So that we would get breadth-first search and resolve some field into "value": "object with that field" on first entrance and "value": Null
So How can one implement such a list to a tree in Postgres JSONb\ JSON?
I understand one could do it using something like PL/Python yet I failed to see a PL/Python function example that could do such a thing as not putting all JSON records into ram...
{'id1': {'aid': 'id1', 'atype': '', 'data': ['id3', 'id2']},
'id2': {'aid': 'id2',
'atype': '',
'bdata': {'aid': 'id4', 'atype': 'nested', 'data': ['id1', 'id3']},
'data': []},
'id3': {'aid': 'id3', 'atype': '', 'data': ['id1', 'id2']}}
Is not really usable for me - So I need a real tree.
There is a problem with your approach.
Let's consider following json.
{
"aid": "id3",
"atype": "",
"data": ["id1", "id2"]
},
{
"aid": "id4",
"atype": "",
"data": ["id1", "id2"]
}
Here, id1 and id2 data need to be copied again for id4, It creates lot of duplicate data.
Instead of that, convert your data to dictionary.
You can access the data just by calling the id’s like “id1”, “id2” and so on.
data_map = {}
for data in data_array:
data_map.__setitem__(data['aid'], data)
Following will be sample output.
{
"id3": {
"aid": "id3",
"atype": "",
"data": [
"id1", "id2"
]
}
This way you can access the id’s directly and don’t have duplicates in the structure.
Complete sample code
import json
json_data = '''
[
{
"aid": "id3",
"atype": "",
"data": ["id1", "id2"]
},
{
"aid": "id1",
"atype": "",
"data": ["id3", "id2"]
},
{
"aid": "id2",
"atype": "",
"bdata": {"aid": "id4", "atype": "nested", "data": ["id1", "id3"]},
"data": []
}
]
'''
data_array = json.loads(json_data)
data_map = {}
for data in data_array:
data_map.__setitem__(data['aid'], data)
print(data_map)

SQL JSON - Append Json to Json

I Have this json:
{"keyvalue": {"head": {"id": ""},"column": {"id": ""},"degrees": {"id": ""}}}
How do i add this json to the key "keyvalue":
"somekey": { "id" : "" }
so my json looks like this:
{"keyvalue": {"head": {"id": ""},"column": {"id": ""},"degrees": {"id": ""}, "somekey": { "id" : "" }}}
i tried this:
SELECT JSON_MODIFY('{"keyvalue": {"head": {"id": ""},"column": {"id": ""},"degrees": {"id": ""}}}', 'append $', json_query(N' {"somekey": {"id" : ""}}'))
FROM PL_Table
WHERE PL_Id = 6;
but nothing changed
UPDATE
i have this now:
update PL_PageLayout
set PL_Json = json_modify('{
"keyvalue": {
"obj1": {
"id": ""
},
"obj2": {
"id": ""
},
"obj3": {
"id": ""
}
}
}', 'append $.keyvalue.content', '{"id" : "ddd"}')
FROM PL_PageLayout
WHERE PL_Id = 6;
Output is:
{"keyvalue": {"obj1": {"id": ""},"obj2": {"id": ""},"obj3": {"id": ""},"content":["{\"id\" : \"ddd\"}"]}}
but the
"content":["{\"id\" : \"ddd\"}"]
needs to be
"content":{\"id\" : \"ddd\"}
The reason for this result is that with append optional modifier, the new value is appended to the array referenced by the path. You also need to use JSON_QUERY() to get a properly formatted JSON, because JSON_MODIFY escapes all special characters in the new value if the type of the value is varchar or nvarchar.
You may try with the following approach, without using append:
DECLARE #json nvarchar(max) = N'{"keyvalue": {"head": {"id": ""},"column": {"id": ""},"degrees": {"id": ""}}}'
SELECT JSON_MODIFY(
#json,
'$.keyvalue.somekey',
JSON_QUERY(N'{"id" : ""}')
)
Result:
{"keyvalue": {"head": {"id": ""},"column": {"id": ""},"degrees": {"id": ""},"somekey":{"id" : ""}}}

How to get value from JsValue?

For Example:
My Db stores following Json. Form Following json I need to extract the value of particular field.
"student": [
{
"name": "Xyz",
"college": "abc",
"student_id":{
"$oid": "59a9314f6d0000920962e247"
}},
{
"name": "DDD",
"college": "opop",
"student_id":{
"$oid": "59a9314f6d0000920962e257"
}}
]
How can I pick only the value of "$oid" and save json in following way:
"student": [
{
"name": "Xyz",
"college": "abc",
"student_id":
"59a9314f6d0000920962e247"
},
{
"name": "DDD",
"college": "opop",
"student_id":
"59a9314f6d0000920962e257"
}
]
In my scenario, I'm reading it from client side as -
String Json = null;
JsonNode body = request().body().asJson();
Json = body.toString();
Logger.info(Json);
String role = body.get("role").get("role").asText();
users.firstName = body.get("firstName").asText();
users.lastName = body.get("lastName").asText();
You need to change the definition of JsonNode body = request().body().asJson(); and the other code as per your scenario to get it from db.
You will need to replace "student_id" jsValue with string value as below:
val original: JsValue = Json.parse(
""" {"student": [
{
"name": "Xyz",
"college": "abc",
"student_id":{
"$oid": "59a9314f6d0000920962e247"
}},
{
"name": "DDD",
"college": "opop",
"student_id":{
"$oid": "59a9314f6d0000920962e257"
}}
]}""")
val changed = original.as[JsObject] ++ Json.obj(
"student" -> Json.arr {
original.transform((__ \ 'student)
.json.pick[JsArray])
.getOrElse(Json.arr())
.value.map(e => {
val value = e.transform((__ \ 'student_id \ '$oid).json.pick[JsString]).get
e.as[JsObject] ++ Json.obj("student_id" -> value)
})
})
println(Json.stringify(changed))
//Result:
{"student":[[{"name":"Xyz","college":"abc","student_id":"59a9314f6d0000920962e247"},{"name":"DDD","college":"opop","student_id":"59a9314f6d0000920962e257"}]]}

Is there simple way/method to extract the element from json in scala?

Supposed I have the following simple json string:
val jsonString="""{
| "result": {
| "header": ["time-stamp", "id-number", "call-number", "trial-number", "ratio"],
| "data": [
| ["2017-08-29 00:00:00", "111550", "16", "10", "0.79"],
| ["2017-08-29 00:00:00", "111551", "15", "18", "0.55"],
| ["2017-08-29 00:00:00", "111552", "13", "16", "0.35"]
| ],
| "paging": { "a": 5, "b": 10, "c": 11}
| }
|}""".stripMargin
Now I would like to extract the data in the below form:
//List[List[String]]
List(
List("2017-08-29 00:00:00", "111550", "16", "10", "0.79"),
List("2017-08-29 00:00:00", "111551", "15", "18", "0.55"),
List("2017-08-29 00:00:00", "111552", "13", "16", "0.35")
)
My trial:
scala> import play.api.libs.json._
scala> val json=Json.parse(jsonString)
scala> val jsonTransformer = (__ \ 'result \ 'data).json.pick[JsArray]
scala> val dataArray = json.transform(jsonTransformer).get
//dataArray: json.JsArray = [["2017-08-29 00:00:00","111550","16","10","0.79"], ["2017-08-29 00:00:00","111551","15","18","0.55"], ["2017-08-29 00:00:00","111552","13","16","0.35"]]
scala> val data = dataArray.value.map(_.as[JsArray]).map(_.value).toList.map(_.toList)
//data: List[List[json.JsValue]] = List(List("2017-08-29 00:00:00", "111550", "16", "10", "0.79"), List("2017-08-29 00:00:00", "111551", "15", "18", "0.55"), List("2017-08-29 00:00:00", "111552", "13", "16", "0.35"))
According to the REPL value, I know the type of data is List[List[json.JsValue]], instead of List[List[String]].
So I would like to know how to deal with problem in simple way. Thanks sincerely!
Using Jackson Library
scala> import org.json4s.jackson.JsonMethods
scala> val parsedMap = JsonMethods.parse(jsonString).values.asInstanceOf[Map[String, Any]].head._2.asInstanceOf[Map[String,Any]]
scala> parsedMap.get("data").get.asInstanceOf[List[List[String]]]
(Json.parse(jsonString) \ "result" \ "data").as[List[List[String]]]
I think you need cast String also
val data = dataArray.value
.map(_.as[JsArray])
.map(_.value)
.map(_.toList.map(_.as[String]))
.toList
or
val data = dataArray.as[List[List[String]]]
Enjoy!

Stripping model pk and fields text from Django queryset json output

In a view of django, I want to output the queryset converted to json without the model, pk, and field text.
my view code:
s = serializers.serialize('json', Item.objects.get(id=actuators_id)])
o = s.strip("[]")
return HttpResponse(o, content_type="application/json")
What I get is this:
{"model": "actuators.acutatoritem", "pk": 1, "fields": {"name": "Environment Heater", "order": 1, "controlid": "AAHE", "index": "1", "param1": "", "param2": "", "param3": "", "current_state": "unknown"}}
What I spend all day NOT getting is this:
{"name": "Environment Heater", "order": 1, "controlid": "AAHE", "index": "1", "param1": "", "param2": "", "param3": "", "current_state": "unknown"}
I can I strip the model, pk, and field text from my output????
Use simplejson to convert the qs to a python dict {}
import simplejson
s = serializers.serialize('json', Item.objects.filter(id=actuators_id)])
js = simplejson.loads(s)
//select the key needed and return the response
s = js[0]['fields']
return HttpResponse(str(s), content_type="application/json")