JSON queries in PostgreSQL 9.6 - json

presumed the following data:
select * from orderlist ;
id | orders | orderenvelope_id
----+--------------------------------------------+------------------
14 | { +| 13
| "orders" : [ { +|
| "orderType" : "OfferPurchaseOrder", +|
| "duration" : 1494413009450, +|
| "currencyCode" : "EUR" +|
| }, { +|
| "orderType" : "CustomerCreationOrder",+|
| "customerData" : { +|
| "customerType" : "PERSONAL_ACCOUNT",+|
| "contactData" : { +|
| "contactQuality" : "VALID", +|
| "firstName" : "Peter", +|
| "lastName" : "Pan" +|
| } +|
| } +|
| } ] +|
| } |
I want to get the 'OfferPurchaseOrder'; therefore the following SELECT was used:
select id, orderenvelope_id, o from orderlist list, json_array_elements(list.orders->'orders') as o where o->>'orderType' = 'OfferPurchaseOrder';
id | orderenvelope_id | o
----+------------------+-----------------------------------------
14 | 13 | { +
| | "orderType" : "OfferPurchaseOrder",+
| | "duration" : 1494413009450, +
| | "currencyCode" : "EUR" +
| | }
It looks as if works like a charm, only one thing: I want to integrate with Hibernate, so the colum should be named 'orders' instead of 'o' (as it was in the initial select); otherwise Hibernate will not be able to map the things properly.
Aside of this, the 'reduced' JSON-list should be in there, so the desired result should look like this:
id | orderenvelope_id | orders |
----+------------------+----------------------------------------+
14 | 13 | "orders" : [{ +
| | "orderType" : "OfferPurchaseOrder",+
| | "duration" : 1494413009450, +
| | "currencyCode" : "EUR" +
| | } +
| |]
Any hints?
Thx and regards
el subcomandante

If you can move to jsonb type than query can looks like:
WITH x AS (
SELECT id, orderenvelope_id, o
FROM orderlist list, jsonb_array_elements(list.orders->'orders') as o
WHERE o->>'orderType' = 'OfferPurchaseOrder'
)
SELECT id, orderenvelope_id, jsonb_set('{}'::jsonb, '{orders}'::text[], jsonb_agg(o))
FROM x
GROUP BY 1,2
;
However, if you can't use jsonb just cast text to json:
WITH x AS (
SELECT id, orderenvelope_id, o
FROM orderlist list, json_array_elements(list.orders->'orders') as o
WHERE o->>'orderType' = 'OfferPurchaseOrder'
)
SELECT id, orderenvelope_id, ('{"orders": ' || json_agg(o) ||'}')::json
FROM x
GROUP BY 1,2
;

Related

JSONB Array Output - Postgres 11.5

I have a JSONB string in this format
{
"RouteId": "90679754-89f5-48d7-99e1-5192bf0becf9",
"Started": "2019-11-20T21:24:33.7294486Z",
"RouteName": "ProcessRequestsAndPublishResponse",
"MachineName": "5CG8134NJW-LA",
"ChildProfiles": [
{
"ApiMethod": "ProcessApiRequest",
"ExecuteType": null,
"DurationMilliseconds": 2521.4,
},
{
"ApiMethod": "PublishShipViaToQueue",
"ExecuteType": null,
"DurationMilliseconds": 0.6,
}
],
"DataBaseTimings": null,
"DurationMilliseconds": 2522.6
}
How do I get the output in this format
| RouteName | Metrics | Time | TotalDuration |
---------------------------------------------------------------------------------------------
| ProcessRequestsAndPublishResponse | ProcessApiRequest | 2521.4 | 2522.6 |
| ProcessRequestsAndPublishResponse | PublishShipViaToQueue | 0.6 | 2522.6 |
---------------------------------------------------------------------------------------------
Any help on this is appreciated
How do you also extend this in case there are different arrays. Sorry fairly new to the JSONB world.
{
"RouteId": "af2e9cba-11ae-43a9-813c-d24ea574ee62",
"RouteName": "GenerateRequestAndPublishToQueue",
"ChildProfiles": [
{
"ApiMethod": "PublishShipViaRequestToQueue",
"DurationMilliseconds": 0.1,
}
],
"DataBaseTimings": [
{
"ExecuteType": "OpenAsync",
"DurationMilliseconds": 0.1
},
{
"ExecuteType": "Reader",
"DurationMilliseconds": 72.1
},
{
"ExecuteType": "Close",
"DurationMilliseconds": 15.9
}
],
"DurationMilliseconds": 88.6
}
The required output is something like this
| RouteName | Metrics | Time | TotalDuration |
--------------------------------------------------------------------------------------------------------
| GenerateRequestAndPublishToQueue | PublishShipViaRequestToQueue | 0.1 | 88.6 |
| GenerateRequestAndPublishToQueue | OpenAsync | 0.1 | 88.6 |
| GenerateRequestAndPublishToQueue | Reader | 72.1 | 88.6 |
| GenerateRequestAndPublishToQueue | Close | 15.9 | 88.6 |
---------------------------------------------------------------------------------------------------------
You can do a lateral join and use jsonb_to_recordset() to expand the inner json array as an inline table:
select
js ->> 'RouteName' RouteName,
xs."ApiMethod" Metrics,
xs."DurationMilliseconds" "Time",
js ->> 'DurationMilliseconds' TotalDuration
from t
cross join lateral jsonb_to_recordset( js -> 'ChildProfiles')
as xs("ApiMethod" text, "DurationMilliseconds" numeric)
Demo on DB Fiddlde:
routename | metrics | Time | totalduration
:-------------------------------- | :-------------------- | -----: | :------------
ProcessRequestsAndPublishResponse | ProcessApiRequest | 2521.4 | 2522.6
ProcessRequestsAndPublishResponse | PublishShipViaToQueue | 0.6 | 2522.6

How to covert MySQL rows into JSON response using node.js?

I'm trying to create a mark sheet application for Schools.
I would like to display the marks entered by the teachers in the Angular web application.
I have a MySQL table like this,
+---------+-------------+--------------+-------+
| Roll_no | Subject | Category | Marks |
+---------+-------------+--------------+-------+
| 1 | English | HomeWork 1 | 10 |
| 1 | English | HomeWork 2 | 10 |
| 1 | Science | HomeWork 1 | 10 |
| 1 | Science | HomeWork 2 | 10 |
| 2 | English | HomeWork 1 | 10 |
| 2 | English | HomeWork 2 | 10 |
| 2 | Science | HomeWork 1 | 10 |
| 2 | Science | HomeWork 2 | 10 |
+---------+-------------+--------------+-------+
How to convert this into a JSON response like this,
[
{
"Roll_no" : 1,
"Marks" : [
{
"English" : [ "HomeWork 1" : "10" , "HomeWork 2" : "10" ] ,
"Science" : [ "HomeWork 1" : "10" , "HomeWork 2" : "10" ]
}
},
{
"Roll_no" : 2,
"Marks" : [
{
"English" : [ "HomeWork 1" : "10" , "HomeWork 2" : "10" ] ,
"Science" : [ "HomeWork 1" : "10" , "HomeWork 2" : "10" ]
}
}
]
using SQL queries and node.js?
I'm using,
node.js V10.0
Angular V7.0
Thanks in advance.

How to make nested JSON response in Go?

I am new in Go and need some help.
In my PostgreSQL database I have 4 table. They called: surveys, questions, options and surveys_questions_options.
They looks like this:
surveys table:
| survey_id (uuid4) | survey_name (varchar) |
|--------------------------------------|-----------------------|
| 0cf1cf18-d5fd-474e-a8be-754fbdc89720 | April |
| b9fg55d9-n5fy-s7fe-s5bh-856fbdc89720 | May |
questions table:
| question_id (int) | question_text (text) |
|-------------------|------------------------------|
| 1 | What is your favorite color? |
options table:
| option_id (int) | option_text (text) |
|-------------------|--------------------|
| 1 | red |
| 2 | blue |
| 3 | grey |
| 4 | green |
| 5 | brown |
surveys_questions_options table combines data from all three previous tables:
| survey_id | question_id | option_id |
|--------------------------------------|-------------|-----------|
| 0cf1cf18-d5fd-474e-a8be-754fbdc89720 | 1 | 1 |
| 0cf1cf18-d5fd-474e-a8be-754fbdc89720 | 1 | 2 |
| 0cf1cf18-d5fd-474e-a8be-754fbdc89720 | 1 | 3 |
| b9fg55d9-n5fy-s7fe-s5bh-856fbdc89720 | 1 | 3 |
| b9fg55d9-n5fy-s7fe-s5bh-856fbdc89720 | 1 | 4 |
| b9fg55d9-n5fy-s7fe-s5bh-856fbdc89720 | 1 | 5 |
How can I make nested JSON response in Go? I use GORM library. I want a JSON response like this:
[
{
"survey_id": "0cf1cf18-d5fd-474e-a8be-754fbdc89720",
"survey_name": "April",
"questions": [
{
"question_id": 1,
"question_text": "What is your favorite color?",
"options": [
{
"option_id": 1,
"option_text": "red"
},
{
"option_id": 2,
"option_text": "blue"
},
{
"option_id": 3,
"option_text": "grey"
},
]
}
]
},
{
"survey_id": "b9fg55d9-n5fy-s7fe-s5bh-856fbdc89720",
"survey_name": "May",
"questions": [
{
"question_id": 1,
"question_text": "What is your favorite color?",
"options": [
{
"option_id": 3,
"option_text": "grey"
},
{
"option_id": 4,
"option_text": "green"
},
{
"option_id": 5,
"option_text": "brown"
},
]
}
]
}
]
My models looks like this:
type Survey struct {
SurveyID string `gorm:"primary_key" json:"survey_id"`
SurveyName string `gorm:"not null" json:"survey_name"`
Questions []Question
}
type Question struct {
QuestionID int `gorm:"primary_key" json:"question_id"`
QuestionText string `gorm:"not null;unique" json:"question_text"`
Options []Option
}
type Option struct {
OptionID int `gorm:"primary_key" json:"option_id"`
OptionText string `gorm:"not null;unique" json:"option_text"`
}
I'm not sure abour GORM part, but with JSON you need to add struct tags on the nested objects as well:
type Survey struct {
...
Questions []Question `json:"questions"`
}
type Question struct {
...
Options []Option `json:"options"`
}
We're missing some scope from your code, and so it's quite hard to point you in the right direction. Are you asking about querying GORM so you get []Survey, or are you asking about marshalling []Survey? Anyway, you should add the tag to Questions too, as slomek replied.
However, try this:
To fetch nested data in m2m relation
type Survey struct {
gorm.Model
SurveyID string `gorm:"primary_key" json:"survey_id"`
SurveyName string `gorm:"not null" json:"survey_name"`
Questions []*Question `gorm:"many2many:survey_questions;"`
}
surveys := []*model.Survey{}
db := dbSession.Where(&model.Survey{SurveyID: id}).Preload("Questions").Find(&surveys)

PostgreSQL Creating JSON Column From Existing Data

Given data like the below:
+---+------------+------------
|id | change | date
+---+------------+------------
| 1 | name | 2018-06-20
| 2 | address | 2018-06-20
| 3 | email | 2018-06-20
| 4 | email | 2018-06-21
| 5 | address | 2018-06-22
| 6 | address | 2018-06-23
I'm trying to create a view that summarises the above into a single json column with data like:
{"name":["2018-06-20"], "address":["2018-06-20","2018-06-22","2018-06-23"], "email":["2018-06-20","2018-06-21"]}
I have been trying to figure it out using the array_aggr, array_to_json, json_agg, array_build_object functions but I can't seem to get it quite right.
I hope someone can help.
Cheers
You should use jsonb aggregates twice for two levels:
select jsonb_pretty(jsonb_object_agg(change, dates))
from (
select change, jsonb_agg(date) as dates
from my_table
group by change
) s
jsonb_pretty
-----------------------
{ +
"name": [ +
"2018-06-20" +
], +
"email": [ +
"2018-06-20",+
"2018-06-21" +
], +
"address": [ +
"2018-06-20",+
"2018-06-22",+
"2018-06-23" +
] +
}
(1 row)
Note that jsonb_pretty() is unnecessary, used only for a nice output.

how to extract data from json using oracle text index

I have a table, which has an Oracle text index. I created the index because I need an extra fast search. The table contains JSON data. Oracle json_textcontains works very poorly so I tried to play with CONTAINS (json_textcontains is rewritten to CONTAINS actually if we have a look into query plan).
I want to find all jsons by given class_type and id of value but Oracle looks all over JSON without looking that class_type and id should be in one JSON section i.e. it deals with JSON not like structured data but like a huge string.
Well formatted JSON looks like this:
{
"class":[
{
"class_type":"ownership",
"values":[{"nm":"id","value":"1"}]
},
{
"class_type":"country",
"values":[{"nm":"id","value":"640"}]
},
,
{
"class_type":"features",
"values":[{"nm":"id","value":"15"},{"nm":"id","value":"20"}]
}
]
}
The second one which shouldn't be found looks like this:
{
"class":[
{
"class_type":"ownership",
"values":[{"nm":"id","value":"18"}]
},
{
"class_type":"country",
"values":[{"nm":"id","value":"11"}]
},
,
{
"class_type":"features",
"values":[{"nm":"id","value":"7"},{"nm":"id","value":"640"}]
}
]
}
Please see how to reproduce what I'm trying to achieve:
create table perso.json_data(id number, data_val blob);
insert into perso.json_data
values(
1,
utl_raw.cast_to_raw('{"class":[{"class_type":"ownership","values":[{"nm":"id","value":"1"}]},{"class_type":"country","values":[{"nm":"id","value":"640"}]},{"class_type":"features","values":[{"nm":"id","value":"15"},{"nm":"id","value":"20"}]}]}')
);
insert into perso.json_data values(
2,
utl_raw.cast_to_raw('{"class":[{"class_type":"ownership","values":[{"nm":"id","value":"18"}]},{"class_type":"country","values":[{"nm":"id","value":"11"}]},{"class_type":"features","values":[{"nm":"id","value":"7"},{"nm":"id","value":"640"}]}]}')
)
;
commit;
ALTER TABLE perso.json_data
ADD CONSTRAINT check_is_json
CHECK (data_val IS JSON (STRICT));
CREATE INDEX perso.json_data_idx ON json_data (data_val)
INDEXTYPE IS CTXSYS.CONTEXT
PARAMETERS ('section group CTXSYS.JSON_SECTION_GROUP SYNC (ON COMMIT)');
select *
from perso.json_data
where ctxsys.contains(data_val, '(640 INPATH(/class/values/value)) and (country inpath (/class/class_type))')>0
The query returns 2 rows but I expect to get only the record where id = 1.
How can I use a full text index with the ability to search without the error I highlighted, without using JSON_TABLE?
There is no options to put data in relational format.
Thanks in advance.
Please don't use the text index directly to try to solve this kind of problem. It's not what it's designed for..
In 12.2.0.1.0 this should work for you (and yes it does use a specialized version of the text index under the covers, but it also applies selective post filtering to ensure the results are correct)..
SQL> create table json_data(id number, data_val blob)
2 /
Table created.
SQL> insert into json_data values(
2 1,utl_raw.cast_to_raw('{"class":[{"class_type":"ownership","values":[{"nm":"id","value":"1"}]},{"class_type":"cou
ntry","values":[{"nm":"id","value":"640"}]},{"class_type":"features","values":[{"nm":"id","value":"15"},{"nm":"id","valu
e":"20"}]}]}')
3 )
4 /
1 row created.
Execution Plan
----------------------------------------------------------
--------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------------
| 0 | INSERT STATEMENT | | 1 | 100 | 1 (0)| 00:00:01 |
| 1 | LOAD TABLE CONVENTIONAL | JSON_DATA | | | | |
--------------------------------------------------------------------------------------
SQL> insert into json_data values(
2 2,utl_raw.cast_to_raw('{"class":[{"class_type":"ownership","values":[{"nm":"id","value":"18"}]},{"class_type":"co
untry","values":[{"nm":"id","value":"11"}]},{"class_type":"features","values":[{"nm":"id","value":"7"},{"nm":"id","value
":"640"}]}]}')
3 )
4 /
1 row created.
Execution Plan
----------------------------------------------------------
--------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
--------------------------------------------------------------------------------------
| 0 | INSERT STATEMENT | | 1 | 100 | 1 (0)| 00:00:01 |
| 1 | LOAD TABLE CONVENTIONAL | JSON_DATA | | | | |
--------------------------------------------------------------------------------------
SQL> commit
2 /
Commit complete.
SQL> ALTER TABLE json_data
2 ADD CONSTRAINT check_is_json
3 CHECK (data_val IS JSON (STRICT))
4 /
Table altered.
SQL> CREATE SEARCH INDEX json_SEARCH_idx ON json_data (data_val) for JSON
2 /
Index created.
SQL> set autotrace on explain
SQL> --
SQL> set lines 256 trimspool on pages 50
SQL> --
SQL> select ID, json_query(data_val, '$' PRETTY)
2 from JSON_DATA
3 /
ID
----------
JSON_QUERY(DATA_VAL,'$'PRETTY)
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----------------
1
{
"class" :
[
{
"class_type" : "ownership",
"values" :
[
{
"nm" : "id",
"value" : "1"
}
]
},
{
"class_type" : "country",
"values" :
[
{
"nm" : "id",
"value" : "640"
}
]
},
{
"class_type" : "features",
"values" :
[
{
"nm" : "id",
"value" : "15"
},
{
"nm" : "id",
"value" : "20"
}
]
}
]
}
2
{
"class" :
[
ID
----------
JSON_QUERY(DATA_VAL,'$'PRETTY)
------------------------------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------------------------------
----------------
{
"class_type" : "ownership",
"values" :
[
{
"nm" : "id",
"value" : "18"
}
]
},
{
"class_type" : "country",
"values" :
[
{
"nm" : "id",
"value" : "11"
}
]
},
{
"class_type" : "features",
"values" :
[
{
"nm" : "id",
"value" : "7"
},
{
"nm" : "id",
"value" : "640"
}
]
}
]
}
Execution Plan
----------------------------------------------------------
Plan hash value: 3213740116
-------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 2 | 4030 | 3 (0)| 00:00:01 |
| 1 | TABLE ACCESS FULL| JSON_DATA | 2 | 4030 | 3 (0)| 00:00:01 |
-------------------------------------------------------------------------------
Note
-----
- dynamic statistics used: dynamic sampling (level=2)
SQL> select ID, to_clob(data_val)
2 from json_data
3 where JSON_EXISTS(data_val,'$?(exists(#.class?(#.values.value == $VALUE && #.class_type == $TYPE)))' passing '640'
as "VALUE", 'country' as "TYPE")
4 /
ID TO_CLOB(DATA_VAL)
---------- --------------------------------------------------------------------------------
1 {"class":[{"class_type":"ownership","values":[{"nm":"id","value":"1"}]},{"class_
type":"country","values":[{"nm":"id","value":"640"}]},{"class_type":"features","
values":[{"nm":"id","value":"15"},{"nm":"id","value":"20"}]}]}
Execution Plan
----------------------------------------------------------
Plan hash value: 3248304200
-----------------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-----------------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 2027 | 4 (0)| 00:00:01 |
|* 1 | TABLE ACCESS BY INDEX ROWID| JSON_DATA | 1 | 2027 | 4 (0)| 00:00:01 |
|* 2 | DOMAIN INDEX | JSON_SEARCH_IDX | | | 4 (0)| 00:00:01 |
-----------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter(JSON_EXISTS2("DATA_VAL" FORMAT JSON , '$?(exists(#.class?(#.values.value
== $VALUE && #.class_type == $TYPE)))' PASSING '640' AS "VALUE" , 'country' AS "TYPE"
FALSE ON ERROR)=1)
2 - access("CTXSYS"."CONTAINS"("JSON_DATA"."DATA_VAL",'{640} INPATH
(/class/values/value) and {country} INPATH (/class/class_type)')>0)
Note
-----
- dynamic statistics used: dynamic sampling (level=2)
SQL> select ID, TO_CLOB(DATA_VAL)
2 from JSON_DATA d
3 where exists (
4 select 1
5 from JSON_TABLE(
6 data_val,
7 '$.class'
8 columns (
9 CLASS_TYPE VARCHAR2(32) PATH '$.class_type',
10 NESTED PATH '$.values.value'
11 columns (
12 "VALUE" VARCHAR2(32) path '$'
13 )
14 )
15 )
16 where CLASS_TYPE = 'country' and "VALUE" = '640'
17 )
18 /
ID TO_CLOB(DATA_VAL)
---------- --------------------------------------------------------------------------------
1 {"class":[{"class_type":"ownership","values":[{"nm":"id","value":"1"}]},{"class_
type":"country","values":[{"nm":"id","value":"640"}]},{"class_type":"features","
values":[{"nm":"id","value":"15"},{"nm":"id","value":"20"}]}]}
Execution Plan
----------------------------------------------------------
Plan hash value: 1621266031
-------------------------------------------------------------------------------------
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
-------------------------------------------------------------------------------------
| 0 | SELECT STATEMENT | | 1 | 2027 | 32 (0)| 00:00:01 |
|* 1 | FILTER | | | | | |
| 2 | TABLE ACCESS FULL | JSON_DATA | 2 | 4054 | 3 (0)| 00:00:01 |
|* 3 | FILTER | | | | | |
|* 4 | JSONTABLE EVALUATION | | | | | |
-------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
1 - filter( EXISTS (SELECT 0 FROM JSON_TABLE( :B1, '$.class' COLUMNS(
"CLASS_TYPE" VARCHAR2(32) PATH '$.class_type' NULL ON ERROR , NESTED PATH
'$.values.value' COLUMNS( "VALUE" VARCHAR2(32) PATH '$' NULL ON ERROR ) ) )
"P" WHERE "CTXSYS"."CONTAINS"(:B2,'({country} INPATH (/class/class_type))
and ({640} INPATH (/class/values/value))')>0 AND "P"."CLASS_TYPE"='country'
AND "P"."VALUE"='640'))
3 - filter("CTXSYS"."CONTAINS"(:B1,'({country} INPATH
(/class/class_type)) and ({640} INPATH (/class/values/value))')>0)
4 - filter("P"."CLASS_TYPE"='country' AND "P"."VALUE"='640')
Note
-----
- dynamic statistics used: dynamic sampling (level=2)
SQL>