Extract key from JSON string in MySQL - mysql

My table contains string in json format. I need to get the sum and average of each key.
+----+------------------------------------------------------------------------------------+------------+
| id | json_data | subject_id |
+----+------------------------------------------------------------------------------------+------------+
| 1 | {"id": "a", "value": "30"}, {"id": "b", "value": "20"}, {"id": "c", "value": "30"} | 1 |
+----+------------------------------------------------------------------------------------+------------+
| 2 | {"id": "a", "value": "40"}, {"id": "b", "value": "50"}, {"id": "c", "value": "60"} | 1 |
+----+------------------------------------------------------------------------------------+------------+
| 3 | {"id": "a", "value": "20"} | 1 |
+----+------------------------------------------------------------------------------------+------------+
Expected result is
{"id": "a", "sum": 90, "avg": 30},
{"id": "b", "sum": 70, "avg": 35},
{"id": "c", "sum": 120, "avg": 40}
I've tried
SELECT (
JSON_OBJECT('id', id, 'sum', sum_data, 'avg', avg_data)
) FROM (
SELECT
JSON_EXTRACT(json_data, "$.id") as id,
SUM(JSON_EXTRACT(json_data, "$.sum_data")) as sum_data,
AVG(JSON_EXTRACT(json_data, "$.avg_data")) as avg_data
FROM Details
GROUP BY JSON_EXTRACT(json_data, "$.id")
) as t
But no luck. How can I sort this out?

Input json needs to correct
create table json_sum (id int primary key auto_increment, json_data json);
insert into json_sum values (0,'[{"id": "a", "value": "30"}, {"id": "b", "value": "20"}, {"id": "c", "value": "30"}]');
insert into json_sum values (0,'[{"id": "a", "value": "40"}, {"id": "b", "value": "50"}, {"id": "c", "value": "60"}]');
insert into json_sum values (0,'[{"id": "a", "value": "20"}]');
select
json_object("id", jt.id, "sum", sum(jt.value), "avg", avg(jt.value))
from json_sum, json_table(json_data, "$[*]" columns (
row_id for ordinality,
id varchar(10) path "$.id",
value varchar(10) path "$.value")
) as jt
group by jt.id
Output:
json_object("id", jt.id, "sum", sum(jt.value), "avg", avg(jt.value))
{"id": "a", "avg": 30.0, "sum": 90.0}
{"id": "b", "avg": 35.0, "sum": 70.0}
{"id": "c", "avg": 45.0, "sum": 90.0}

Related

How to get array/object number value in postgreSQL?

Is it possible to get array/object number values.
I have a table called tableA:
create table "tableA" (
"_id" serial,
"userId" integer,
"dependentData" jsonb);
INSERT INTO "tableA"
("_id", "userId", "dependentData")
VALUES('55555', '1191', '[{"_id": 133, "type": "radio", "title": "questionTest7", "question": "questionTest7", "response": {"value": ["option_11"]}, "dependentQuestionResponse": [{"_id": 278, "type": "text", "title": "questionTest8", "question": "questionTest8", "response": {"value": ["street no 140"]}, "dependentQuestionResponse": []}]}, {"_id": 154, "type": "dropdown", "title": "questionTest8", "question": "questionTest8", "response": {"value": ["option_14"]}, "dependentQuestionResponse": []}]');
Array number is to be fetched. Output should be require below.
_id
userId
array/object
55555
1191
[0,0,1]
You can try something like this
select id, user_id,
(
with
base as (
select o1, oo1.a oo1 from (
select jsonb_array_elements(t.a) o1
from (select depend_data as a) t
) o
left join lateral (select a from jsonb_array_elements(o.o1-> 'dependentQuestionResponse') a) oo1 on true
)
select json_agg(nn) from (
select dense_rank() over(order by b.o1) - 1 nn, b.o1 from base b
union all
select dense_rank() over(order by b.o1) - 1 nn, b.oo1 from base b where oo1 is not null
order by nn
) tz
) as array_object
from
(select 55555 as id,
1191 as user_id,
'[{"_id": 133, "type": "radio", "title": "questionTest7", "question": "questionTest7", "response": {"value": ["option_11"]}, "dependentQuestionResponse": [
{"_id": 278, "type": "text", "title": "questionTest8", "question": "questionTest8", "response": {"value": ["street no 140"]},"dependentQuestionResponse": []}]},
{"_id": 154, "type": "dropdown", "title": "questionTest8", "question": "questionTest8", "response": {"value": ["option_14"]}, "dependentQuestionResponse": []}]'::jsonb as depend_data) t

how to filter data by Json string column in specific field

I have a Ms Sql Table. In this table, I store data in json format in my column named "DynamicData". What I want to do is to filter on this table. But to do this filtering according to different fields on the json data in my column named "dynamicData".
e.g;
Id | Title | DynamicData
---------------------------------------------------------------- ------------------------------
1 | Title1 | { "Id": 1, "Score": 5, "Message": "Msg1", Types: ["a", "b", "c"]}
2 | Title2 | { "Id": 2, "Score": 3, "Message": "Msg2", Types: ["z", "k", "c"]}
3 | Title3 | { "Id": 3, "Score": 1, "Message": "Msg3", Types: ["a", "b", "c"]}
4 | Title4 | { "Id": 4, "Score": 4, "Message": "Msg4", Types: ["h", "n", "f"]}
Exactly what I want; querying in different ways according to the fields in this json data.
For example;
Retrieve the record whose "score" field is "5".
Bring the record with "Score" field "5" and "Message" field "Msg1".
Fetch the record with the "Score" field "5" or the "Types" field "h" or "n".
There may be different and somewhat more complex inquiries similar to these. I searched but couldn't find what I wanted. Do you have a suggestion or a sample application on how I can make such inquiries?
If you are SQL Server 2016+ you can use OPENJSON()
Here's an example of how you can filter on the Score:
DECLARE #testdata TABLE
(
[Id] INT
, [Title] NVARCHAR(20)
, [DynamicData] NVARCHAR(MAX)
);
INSERT INTO #testdata (
[Id]
, [Title]
, [DynamicData]
)
VALUES ( 1, 'Title1', '{ "Id": 1, "Score": 5, "Message": "Msg1", "Types": ["a", "b", "c"]}' )
, ( 2, 'Title2', '{ "Id": 2, "Score": 3, "Message": "Msg2", "Types": ["z", "k", "c"]}' )
, ( 3, 'Title3', '{ "Id": 3, "Score": 1, "Message": "Msg3", "Types": ["a", "b", "c"]}' )
, ( 4, 'Title4', '{ "Id": 4, "Score": 4, "Message": "Msg4", "Types": ["h", "n", "f"]}' );
SELECT *
FROM #testdata [td]
CROSS APPLY OPENJSON([td].[DynamicData])
WITH (
[Id] INT
, [Score] INT
, [Message] NVARCHAR(20)
, [Types] NVARCHAR(MAX) AS JSON
) AS [dd]
WHERE [dd].[Score] = 5
Giving you the results of:
Id Title DynamicData Id Score Message Types
----------- -------------------- ------------------------------------------------------------------- ----------- ----------- ------- -----
1 Title1 { "Id": 1, "Score": 5, "Message": "Msg1", "Types": ["a", "b", "c"]} 1 5 Msg1 ["a", "b", "c"]
To get to "Types" we'll use anther CROSS APPLY since "Types" is an array:
SELECT *
FROM #testdata [td]
CROSS APPLY OPENJSON([td].[DynamicData])
WITH (
[Id] INT
, [Score] INT
, [Message] NVARCHAR(20)
, [Types] NVARCHAR(MAX) AS JSON
) AS [dd]
CROSS APPLY OPENJSON([Types]) dt --Add another cross apply for types since it is an array
WHERE [dd].[Score] = 5
OR dt.[Value] IN ('h','n') --Then we can filter on it
Giving results of:
Id Title DynamicData Id Score Message Types key value type
----------- -------------------- ---------------------------------------------------------------------- ----------- ----------- --------- ---------------- ---- ------- ----
1 Title1 { "Id": 1, "Score": 5, "Message": "Msg1", "Types": ["a", "b", "c"]} 1 5 Msg1 ["a", "b", "c"] 0 a 1
1 Title1 { "Id": 1, "Score": 5, "Message": "Msg1", "Types": ["a", "b", "c"]} 1 5 Msg1 ["a", "b", "c"] 1 b 1
1 Title1 { "Id": 1, "Score": 5, "Message": "Msg1", "Types": ["a", "b", "c"]} 1 5 Msg1 ["a", "b", "c"] 2 c 1
4 Title4 { "Id": 4, "Score": 4, "Message": "Msg4", "Types": ["h", "n", "f"]} 4 4 Msg4 ["h", "n", "f"] 0 h 1
4 Title4 { "Id": 4, "Score": 4, "Message": "Msg4", "Types": ["h", "n", "f"]} 4 4 Msg4 ["h", "n", "f"] 1 n 1
You see there are duplicates, with the CROSS APPLY on "Types", each element in that array is now it's own row, shown in the Value column.
You will have to make adjustments for your specific requirement, but this should get you moving in the right direction.
If you can't use JSON there is always Ngrams8k.
DECLARE #table TABLE (ID INT IDENTITY, Title VARCHAR(20), DynamicData VARCHAR(1000));
INSERT #table(Title,DynamicData) VALUES
('Title1', '{"Id": 1, "Score": 5, "Message": "Msg1", Types: ["a", "b", "c"]}'),
('Title2', '{"Id": 2, "Score": 3, "Message": "Msg2", Types: ["z", "k", "c"]}'),
('Title3', '{"Id": 3, "Score": 1, "Message": "Msg3", Types: ["a", "b", "c"]}'),
('Title4', '{"Id": 4, "Score": 4, "Message": "Msg4", Types: ["h", "n", "f"]}');
DECLARE #score INT = 5, #type CHAR(1) = 'b'
SELECT t.*
FROM #table AS t
CROSS APPLY PerfLib.samd.ngrams8K(t.DynamicData,7) AS ng
CROSS APPLY (VALUES(SUBSTRING(t.DynamicData,ng.Position+8,8000))) AS pfx(Txt)
CROSS APPLY (VALUES(CHARINDEX('[',t.DynamicData),
CHARINDEX(']',t.DynamicData))) AS tp(Lft,Rgt)
CROSS APPLY (VALUES(REPLACE(SUBSTRING(
t.DynamicData,tp.Lft+1,tp.Rgt-tp.Lft-1),'"',''))) AS tl(List)
CROSS APPLY (VALUES(SUBSTRING(pfx.Txt,1,CHARINDEX(',',pfx.Txt)-1))) AS f(Score)
WHERE ng.Token = '"Score"'
AND f.Score = #score
AND EXISTS (SELECT 1 FROM STRING_SPLIT(tl.List,',') AS split
WHERE LTRIM(split.[value]) = #type);
Returns:
ID Title DynamicData Position
--- ------ ---------------------------------------------------------------- --------
1 Title1 {"Id": 1, "Score": 5, "Message": "Msg1", Types: ["a", "b", "c"]} 11

PostgreSQL query : distinct count of jsonb values column

I have a table with the name mainapp_project_data which has a jsonb column project_user_data
TABLE
public | mainapp_project_data | table | admin
select project_user_data from mainapp_project_data;
project_user_data
-----------------------------------------------------------------------------------------------------------------
[{"name": "john", "age": "21", "gender": "M"}, {"name": "randy", "age": "23", "gender": "M"}]
[{"name": "donald", "age": "31", "gender": "M"}, {"name": "wick", "age": "32",
"gender": "M"}]
[{"name": "orton", "age": "18", "gender": "M"}, {"name": "russel", "age": "55",
"gender": "M"}]
[{"name": "angelina", "age": "open", "gender": "F"}, {"name": "josep", "age": "21",
"gender": "M"}]
(4 rows)
(END)
I would like to count the distinct values of keys gender and age of JSON.
output format : [{key:count(repeated_values)}]
filtering on `gender` : [{"M":7},{"F":1}]
filtering on `age` : [{"21":2},{"23":1},{"31":1}.....]
WITH flat AS (
SELECT
kv.key,
-- make into a JSON object with a single value and count, e.g., '{"M": 7}'
jsonb_build_object(kv.value, COUNT(*)) AS val_count
FROM mainapp_project_data AS mpd
-- Flatten the JSON arrays into single objects per row
CROSS JOIN LATERAL jsonb_array_elements(mpd.project_user_data) AS unarrayed(udata)
-- Convert to a long, flat list of key-value pairs
CROSS JOIN LATERAL jsonb_each_text(unarrayed.udata) AS kv(key, value)
GROUP BY kv.key, kv.value
)
SELECT
-- de-deplicated object keys
flat.key,
-- aggregation of all values and counts per key
jsonb_agg(flat.val_count) AS value_counts
FROM flat
GROUP BY flat.key
Returns
key | value_counts
--------+---------------------------------------------------------------------------------------------------------------------
gender | [{"M": 7}, {"F": 1}]
name | [{"josep": 1}, {"russel": 1}, {"orton": 1}, {"donald": 1}, {"wick": 1}, {"john": 1}, {"randy": 1}, {"angelina": 1}]
age | [{"18": 1}, {"32": 1}, {"21": 2}, {"23": 1}, {"open": 1}, {"31": 1}, {"55": 1}]
This will provide any key-value pair instance count. If you just want genders and ages, just add a where clause before the first GROUP BY clause.
WHERE kv.key IN ('gender', 'age')
Does something like this work for you?
postgres=# select count(*), (foo->'gender')::text as g from (select json_array_elements(project_user_data) as foo from mainapp_project_data) as j group by (foo->'gender')::text;
count | g
-------+-----
7 | "M"
1 | "F"
(2 rows)
postgres=# select count(*), (foo->'age')::text as g from (select json_array_elements(project_user_data) as foo from mainapp_project_data) as j group by (foo->'age')::text;
count | g
-------+--------
2 | "21"
1 | "32"
1 | "open"
1 | "23"
1 | "18"
1 | "55"
1 | "31"
(7 rows) ```

How to make pgsql return the json array

everyone , I face some issue to convert the data into json object. There is a table called milestone with the following data:
id name parentId
a test1 A
b test2 B
c test3 C
I want to convert the result into a json type in Postgres:
[{"id": "a", "name": "test1", "parentId": "A"}]
[{"id": "b", "name": "test2", "parentId": "B"}]
[{"id": "c", "name": "test3", "parentId": "C"}]
if there are anyone know how to handle , please let me know , thanks all
You can get each row of the table as simple json object with to_jsonb():
select to_jsonb(m)
from milestone m
to_jsonb
-----------------------------------------------
{"id": "a", "name": "test1", "parentid": "A"}
{"id": "b", "name": "test2", "parentid": "B"}
{"id": "c", "name": "test3", "parentid": "C"}
(3 rows)
If you want to get a single element array for each row, use jsonb_build_array():
select jsonb_build_array(to_jsonb(m))
from milestone m
jsonb_build_array
-------------------------------------------------
[{"id": "a", "name": "test1", "parentid": "A"}]
[{"id": "b", "name": "test2", "parentid": "B"}]
[{"id": "c", "name": "test3", "parentid": "C"}]
(3 rows)
You can also get all rows as a json array with jsonb_agg():
select jsonb_agg(to_jsonb(m))
from milestone m
jsonb_agg
-----------------------------------------------------------------------------------------------------------------------------------------------
[{"id": "a", "name": "test1", "parentid": "A"}, {"id": "b", "name": "test2", "parentid": "B"}, {"id": "c", "name": "test3", "parentid": "C"}]
(1 row)
Read about JSON Functions and Operators in the documentation.
You can use ROW_TO_JSON
From Documentation :
Returns the row as a JSON object. Line feeds will be added between
level-1 elements if pretty_bool is true.
For the query :
select
row_to_json(tbl)
from
(select * from tbl) as tbl;
You can check here in DEMO

Postgresql: How to get values from arrays of JSONB?

We have official documentation which unclear for me without examples.
I have JSON loke this structure, its a bit bigger but the same structure, more JSONs inside the array:
WITH
responses AS (
SELECT *
FROM (VALUES ((1,
'[
{
"id": "13",
"alias": "r1",
"title": "quest",
"answer": "5",
"question": "qq",
"answer_id": 10048
},
{
"id": "24",
"alias": "q6",
"title": "quest",
"answer": "yes",
"question": "quest",
"answer_id": 10094
}
]' :: JSON),
(2,
'[
{
"id": "13",
"alias": "r1",
"title": "quest",
"answer": "-1",
"question": "qq",
"answer_id": 10048
},
{
"id": "24",
"alias": "q6",
"title": "quest",
"answer": "no",
"question": "quest",
"answer_id": 10094
}
]' :: JSON))
) TEST(id,val)
)
SELECT * from responses
I could transform to flat structure:
id| question| answer
-----------
1, 'r1', 5
1, 'q6', yes
2, 'r1', -1
2, 'q6', no
But I need to get result in this way
id| r1| q6
----------
1, 5, yes
2, -1, no
How can I get result like last one?
There are some issues with your WITH query.
First, there is no need to do "select * from (values (..))" – VALUES is itself a query which you can use without SELECT:
test=# values (1, 'zz'), (2, 'bbb');
column1 | column2
---------+---------
1 | zz
2 | bbb
(2 rows)
Next, there is an issue with parentheses. Compare the previous query with this one:
test=# values ((1, 'zz'), (2, 'bbb'));
column1 | column2
---------+---------
(1,zz) | (2,bbb)
(1 row)
If you place additional parentheses like this, you got just one row of two columns, and values in these columns being of "anonymous" record type (its Postgres type magic, very powerful, but it's different story, not needed here).
So let's fix the first part of your CTE:
with responses(id, val) AS (
values
(1, '
[
{
"id": "13",
"alias": "r1",
"title": "quest",
"answer": "5",
"question": "qq",
"answer_id": 10048
},
{
"id": "24",
"alias": "q6",
"title": "quest",
"answer": "yes",
"question": "quest",
"answer_id": 10094
}
]'::json
), (2, '
[
{
"id": "13",
"alias": "r1",
"title": "quest",
"answer": "-1",
"question": "qq",
"answer_id": 10048
},
{
"id": "24",
"alias": "q6",
"title": "quest",
"answer": "no",
"question": "quest",
"answer_id": 10094
}
]'::json
)
)
select * from responses;
Now we can use json_array_elements(..) function to extract JSON elements from the JSON arrays:
with responses(id, val) AS (
values
...
)
select id, json_array_elements(val)
from responses;
Let's use it to construct the "second stage" of our CTE:
...
), extra(id, elem) as (
select id, json_array_elements(val)
from responses
)
...
And finally, we'll get the needed result like this:
...
select
id,
elem->>'id' as json_id,
elem->>'alias' as alias,
elem->>'question' as question,
elem->>'answer' as answer
from extra;
Entire query:
with responses(id, val) AS (
values
(1, '
[
{
"id": "13",
"alias": "r1",
"title": "quest",
"answer": "5",
"question": "qq",
"answer_id": 10048
},
{
"id": "24",
"alias": "q6",
"title": "quest",
"answer": "yes",
"question": "quest",
"answer_id": 10094
}
]'::json
), (2, '
[
{
"id": "13",
"alias": "r1",
"title": "quest",
"answer": "-1",
"question": "qq",
"answer_id": 10048
},
{
"id": "24",
"alias": "q6",
"title": "quest",
"answer": "no",
"question": "quest",
"answer_id": 10094
}
]'::json
)
), extra(id, elem) as (
select id, json_array_elements(val)
from responses
)
select
id as row_id,
elem->>'id' as json_id,
elem->>'alias' as alias,
elem->>'question' as question,
elem->>'answer' as answer
from extra;
Result:
row_id | id | alias | question | answer
--------+----+-------+----------+--------
1 | 13 | r1 | qq | 5
1 | 24 | q6 | quest | yes
2 | 13 | r1 | qq | -1
2 | 24 | q6 | quest | no
(4 rows)
This is a bit different from what you wanted. What you wanted cannot be achieved with pure SQL since you want to have dynamic column names in the output -- those "r1" and "q6" must be extracted from JSON data dynamically. It's possible with plpgsql though, or with tablefunc extension to create a pivoted table, let me know if you need it.