Postgresql merge rows with same key (hstore or json) - json

I have a table like this:
+--------+--------------------+
| ID | Attribute |
+--------+--------------------+
| 1 |"color" => "red" |
+--------+--------------------+
| 1 |"color" => "green" |
+--------+--------------------+
| 1 |"shape" => "square" |
+--------+--------------------+
| 2 |"color" => "blue" |
+--------+--------------------+
| 2 |"color" => "black" |
+--------+--------------------+
| 2 |"flavor" => "sweat" |
+--------+--------------------+
| 2 |"flavor" => "salty" |
+--------+--------------------+
And I want to run some postgres query that get a result table like this:
+--------+------------------------------------------------------+
| ID | Attribute |
+--------+------------------------------------------------------+
| 1 |"color" => "red, green", "shape" => "square" |
+--------+------------------------------------------------------+
| 2 |"color" => "blue, black", "flavor" => "sweat, salty" |
+--------+------------------------------------------------------+
The attribute column can either be hstore or json format. I wrote it in hstore for an example, but if we cannot achieve this in hstore, but in json, I would change the column to json.
I know that hstore does not support one key to multiple values, when I tried some merge method, it only kept one value for each key. But for json, I didn't find anything that supports multiple value merge like this neither. I think this can be done by function merging values for the same key into a string/text and add it back to the key/value pair. But I'm stuck in implementing it.
Note: if implement this in some function, ideally any key such as color, shape should not appear in the function since keys can be expanded dynamically.
Does anyone have any idea about this? Any advice or brainstorm might help. Thank you!

Just a note before anything else: in your desidered output I would use some proper json and not that kind of lookalike. So a correct output according to me would be:
+--------+----------------------------------------------------------------------+
| ID | Attribute |
+--------+----------------------------------------------------------------------+
| 1 | '{"color":["red","green"], "flavor":[], "shape":["square"]}' |
+--------+----------------------------------------------------------------------+
| 2 | '{"color":["blue","black"], "flavor":["sweat","salty"], "shape":[]}' |
+--------+----------------------------------------------------------------------+
A PL/pgSQL function which parses the json attributes and executes a dynamic query would do the job, something like that:
CREATE OR REPLACE FUNCTION merge_rows(PAR_table regclass) RETURNS TABLE (
id integer,
attributes json
) AS $$
DECLARE
ARR_attributes text[];
VAR_attribute text;
ARR_query_parts text[];
BEGIN
-- Get JSON attributes names
EXECUTE format('SELECT array_agg(name ORDER BY name) AS name FROM (SELECT DISTINCT json_object_keys(attribute) AS name FROM %s) AS s', PAR_table) INTO ARR_attributes;
-- Write json_build_object() query part
FOREACH VAR_attribute IN ARRAY ARR_attributes LOOP
ARR_query_parts := array_append(ARR_query_parts, format('%L, array_remove(array_agg(l.%s), null)', VAR_attribute, VAR_attribute));
END LOOP;
-- Return dynamic query
RETURN QUERY EXECUTE format('
SELECT t.id, json_build_object(%s) AS attributes
FROM %s AS t,
LATERAL json_to_record(t.attribute) AS l(%s)
GROUP BY t.id;',
array_to_string(ARR_query_parts, ', '), PAR_table, array_to_string(ARR_attributes, ' text, ') || ' text');
END;
$$ LANGUAGE plpgsql;
I've tested it and it seems to work, it returns a json with. Here is my test code:
CREATE TABLE mytable (
id integer NOT NULL,
attribute json NOT NULL
);
INSERT INTO mytable (id, attribute) VALUES
(1, '{"color":"red"}'),
(1, '{"color":"green"}'),
(1, '{"shape":"square"}'),
(2, '{"color":"blue"}'),
(2, '{"color" :"black"}'),
(2, '{"flavor":"sweat"}'),
(2, '{"flavor":"salty"}');
SELECT * FROM merge_rows('mytable');
Of course you can pass the id and attribute column names as parameters as well and maybe refine the function a bit, this is just to give you an idea.
EDIT : If you're on 9.4 please consider using jsonb datatype, it's much better and gives you room for improvements. You would just need to change the json_* functions to their jsonb_* equivalents.

If you just want this for display purposes, this might be enough:
select id, string_agg(key||' => '||vals, ', ')
from (
select t.id, x.key, string_agg(value, ',') vals
from t
join lateral each(t.attributes) x on true
group by id, key
) t
group by id;
If you are not on 9.4, you can't use the lateral join:
select id, string_agg(key||' => '||vals, ', ')
from (
select id, key, string_agg(val, ',') as vals
from (
select t.id, skeys(t.attributes) as key, svals(t.attributes) as val
from t
) t1
group by id, key
) t2
group by id;
This will return:
id | string_agg
---+-------------------------------------------
1 | color => red,green, shape => square
2 | color => blue,black, flavor => sweat,salty
SQLFiddle: http://sqlfiddle.com/#!15/98caa/2

Related

Json hash save into relational database in rows and columns

I want to read the values from json and need to create a new json so is there any way that
we can save json in table and columns in oracle that will help to perform calculation on that. calculation is too complax.
Here is the json sample and json has many hash and
{
"agri_Expense": {
"input": 6000,
"max": 7500,
"check": 7500
},
"income3": {
"Hiring_income": 239750
},
"Operational_Cost1": [
{
"Field_input3": 10000,
"Minimum": "0.05",
"Check_Input": 26750,
"Tractor_Cost": "Maintenance"
}
]
}
You do not need PL/SQL, and can do it entirely in SQL.
I want to read the values from json [...] so is there any way that
we can save json in table and columns in oracle
Yes, use SQL to create a table:
CREATE TABLE table_name ( json_column CLOB CHECK ( json_column IS JSON ) )
and then INSERT the value there:
INSERT INTO table_name ( json_column ) VALUES (
'{'
|| '"agri_Expense": {"input": 6000,"max": 7500,"check": 7500},'
|| '"income3": {"Hiring_income": 239750},'
|| '"Operational_Cost1": [{"Field_input3": 10000,"Minimum": "0.05","Check_Input": 26750,"Tractor_Cost": "Maintenance"}]'
|| '}'
)
then, if you want individual values, SELECT using JSON_TABLE:
SELECT j.*
FROM table_name t
CROSS JOIN JSON_TABLE(
t.json_column,
'$'
COLUMNS (
agri_expense_input NUMBER PATH '$.agri_Expense.input',
agri_expense_max NUMBER PATH '$.agri_Expense.max',
agri_expense_check NUMBER PATH '$.agri_Expense.check',
income3_hiring_income NUMBER PATH '$.income3.Hiring_income',
NESTED PATH '$.Operational_Cost1[*]'
COLUMNS (
oc1_field_input3 NUMBER PATH '$.Field_input3',
oc1_minimum NUMBER PATH '$.Minimum',
oc1_check_input NUMBER PATH '$.Check_Input'
)
)
) j
Which outputs:
AGRI_EXPENSE_INPUT | AGRI_EXPENSE_MAX | AGRI_EXPENSE_CHECK | INCOME3_HIRING_INCOME | OC1_FIELD_INPUT3 | OC1_MINIMUM | OC1_CHECK_INPUT
-----------------: | ---------------: | -----------------: | --------------------: | ---------------: | ----------: | --------------:
6000 | 7500 | 7500 | 239750 | 10000 | .05 | 26750
db<>fiddle here

MySQL JSON object insert

I have the following JSON:
{
"params" : {
"A" : 200.5,
"B" : 70.2
}
}
And the following table:
CREATE TABLE `params` (
`param` varchar(255),
`value` float
) ENGINE=MyISAM DEFAULT CHARSET=utf8;
Is there a way to make mysql function with one insert query without using "while do" directly to insert all the parameters into table like this:
-------+-------
| param | value |
|-------+-------|
| A | 200.5|
| B | 70.2|
---------------
If you are running MySQL 8.0, you can use json_keys() to dynamically extract the keys from the json subobject as a json array, and then use json_table() to turn it to rows. You can then extract the values.
Consider:
insert into `params`
with t as (select '{"params": { "A": 200.5, "B": 70.2 } }' js)
select x.k, json_extract(js, concat('$.params.', x.k)) v
from
t
cross join json_table(
json_keys(js->"$.params"),
"$[*]" columns(k varchar(255) path "$")
) as x
Demo on DB Fiddle
Content of the table after running the query:
param | value
:---- | ----:
A | 200.5
B | 70.2
We can insert multiple rows with a single query using insert into. we need to construct this query using your JSON object
INSERT INTO params(param,value)
VALUES('A',200.5), ('B', 70.2);

Parsing JSon column value

I am trying to parse a json column from a table called Json_table which has 2 different types of elements,
keyfield | json_column | SomeotherField1 | SomeotherField2
----------------------------------------------------------------
keyfield1, | {"jField1":"Value1", ..."jField10":"Value10", "MapField":[{"Key": "key1", "Value":"Keyvalue1"}, {"Key": "key2", "Value":"Keyvalue2"}] | someothervalue | someothervalue
Using get_json_object function I can easily reach to jField1 to jField10 and MapField. But I dont know how i can parse MapField into further separate columns.
SELECT keyfield, get_json_object(json_column, '$.jField1') as jField1, get_json_object(json_column, '$.jField2') as jField2
FROM Json_table
I need to write query that would provide me results like this
Select Keyfield, jField1, jField2, .. , jField10, Key1, Key2 From Json_table
result as:
keyfield1 | Value1 | ... | Value10 | Keyvalue1 | Keyvalue2
I want to get all individual elements from within "Mapfield" part.
You can access MapField array by indexes, $.MapField[0].Value,
$.MapField[1].Value
with cte as (
select string('{"jField1":"Value1","jField10":"Value10", "MapField":[{"Key": "key1", "Value":"Keyvalue1"}, {"Key": "key2", "Value":"Keyvalue2"}]}')as jsn_col)
select get_json_object(jsn_col,'$.jField1')jField1,
get_json_object(jsn_col,'$.MapField[0].Value')key1,
get_json_object(jsn_col,'$.MapField[1].Value')key2 from cte;
+-------+---------+---------+
|jField1|key1 |key2 |
+-------+---------+---------+
|Value1 |Keyvalue1|Keyvalue2|
+-------+---------+---------+

Aggregating subquery keys to build a complex object

I have a table that describes a list of products made for a given order, in this every row on this table has to have a an id of the product and the reason why it was purchased,
I would like to build a json response that amounts to an array of locations the ordered product is for, within that, an array of unique product codes and with that an array of reasons why that unique product was ordered.
I've only been able to get the topmost part of my query defined but the join and sub-select nature of the request is actually getting me in a bit of a fiddle. Is this kind of thing actually possible in plpgsql?
Additionally I'd like to join product_order.reason on product_order_reason.id and retrieve the longform_text inside the table associated with this row but I figure the bigger thing is to get the return at all and that's where i've been left stumped.
product
id |name |cost |cost_rate|
--------|------------------------|------|---------|
WALLC |Wall Clock | 15.00|SINGLE |
MIRR |Mirror | 25.00|SINGLE |
KEY |Door Keys | 5.00|SINGLE |
KEYFOB |Key Fob | 40.00|SINGLE |
product_order
product_id|quantity|location |quote_detail_quote_id |is_primary_order|reason|
----------|--------|----------|------------------------------------|----------------|------|
MIRR | 2|floor_0 |C7D33FED-CB15-5796-DC7D-A7BCEA8923C5|true | 1|
KEYF | 3|floor_0 |C7D33FED-CB15-5796-DC7D-A7BCEA8923C5|true | 2|
WALLC | 3|floor_1 |C7D33FED-CB15-5796-DC7D-A7BCEA8923C5|true | 1|
WALLC | 3|floor_1 |C7D33FED-CB15-5796-DC7D-A7BCEA8923C5|true | 3|
product_order_reason
------------------------------------------------
id (varchar, pk) | shortform_text(varchar) | longform_text(varchar)
------------------------------------------------
id|shortform_text |longform_text |
--|-------------------------------------|-----------------------------------------------------|
1|Employee Room |Standard employee room with no window |
2|Meeting Room |Standard Meeting Room |
3|Mirror |Additional Mirror Request |
create
or replace
function get_breakdown_v1_0_0(p_quote_id character varying,
p_location character varying,
p_product_code character varying) returns json language plpgsql as $function$ declare row_count smallint := 0;
begin
raise notice 'Location: %',
p_location;
raise notice 'Product: %',
p_product_code;
-- Perform santiy check on quote_id so that the json does not include a null result.
select
count(*) into
strict row_count
from
quote_detail
where
quote_id = p_quote_id;
if row_count = 0 then raise 'Quote ID % not found',
p_quote_id
using ERRCODE = '02000';
-- SQL standard no_data
elseif row_count > 1 then raise 'Too many rows returned for ID %',
p_quote_id
using ERRCODE = 'P0003';
-- PL/pgSQL too_many_rows
end if;
-- Returns an object comprised of unique values for locations, where not null and their associated products
return (
select
jsonb_build_object ('locations',jsonb_agg( jsonb_build_object( 'area', location, 'items', items)))
from
(
select
location,
jsonb_agg(jsonb_build_object ('code', product_id, 'reasons', reason)) as items
from
product_order
where
(quote_detail_quote_id = p_quote_id)
and (location = p_location
or p_location is null)
and (product_id = p_product_code
or p_product_code is null)
group by
location) a );
end $function$ ;
Desired response;
{
"area": "floor_0",
"items": [
{
"code": "WALLC",
"reasons": [
{
"quantity": 2,
"reason_code": "Standard Employee Room"
},
{
"quantity": 2,
"reason_code": "Standard Cubicle"
}
]
},
{
"code": "MIRR",
"reasons": [
{
"quantity": 3,
"reason_code": "Meeting Room"
}
]
}
]
}]
Alright I think I have something for you. The idea is to build one of the arrays at a time and carry the necessary remaining info to the outer queries for further array building. You can add your constraints for quote_detail_quote_id , location, and product_id to the innermost query's WHERE clause.
SQLFiddle to show it in action.
This may take some studying:
SELECT json_build_object('area', t3.location, 'items', t3.code_json)
FROM
(
SELECT t2.location
, array_to_json(array_agg(jsonb_build_object('code', t2.product_id, 'reasons', t2.qty_reason_json))) AS code_json
FROM
(
SELECT t.location
, t.product_id
, array_to_json(array_agg(jsonb_build_object('quantity', t.quantity, 'reason_code', t.longform_text))) AS qty_reason_json
FROM
(
SELECT po.product_id
, po.quantity
, po.location
, po.reason
, por.longform_text
FROM product_order po
JOIN product_order_reason por ON (por.id = po.reason)
WHERE quote_detail_quote_id = 'C7D33FED'
) t
GROUP BY t.location, t.product_id
) t2
GROUP BY t2.location
) t3
;

How to search JSON array in MySQL?

Let's say I have a JSON column named data in some MySQL table, and this column is a single array. So, for example, data may contain:
[1,2,3,4,5]
Now I want to select all rows which have a data column where one of its array elements is greater than 2. Is this possible?
I tried the following, but seems it is always true regardless of the values in the array:
SELECT * from my_table
WHERE JSON_EXTRACT(data, '$[*]') > 2;
You may search an array of integers as follows:
JSON_CONTAINS('[1,2,3,4,5]','7','$') Returns: 0
JSON_CONTAINS('[1,2,3,4,5]','1','$') Returns: 1
You may search an array of strings as follows:
JSON_CONTAINS('["a","2","c","4","x"]','"x"','$') Returns: 1
JSON_CONTAINS('["1","2","3","4","5"]','"7"','$') Returns: 0
Note: JSON_CONTAINS returns either 1 or 0
In your case you may search using a query like so:
SELECT * from my_table
WHERE JSON_CONTAINS(data, '2', '$');
SELECT JSON_SEARCH('["1","2","3","4","5"]', 'one', "2") is not null
is true
SELECT JSON_SEARCH('["1","2","3","4","5"]', 'one', "6") is not null
is false
Since MySQL 8 there is a new function called JSON_TABLE.
CREATE TABLE my_table (id INT, data JSON);
INSERT INTO my_table VALUES
(1, "[1,2,3,4,5]"),
(2, "[0,1,2]"),
(3, "[3,4,-10]"),
(4, "[-1,-2,0]");
SELECT DISTINCT my_table.*
FROM my_table, JSON_TABLE(data, "$[*]" COLUMNS(nr INT PATH '$')) as ids
WHERE ids.nr > 2;
+------+-----------------+
| id | data |
+------+-----------------+
| 1 | [1, 2, 3, 4, 5] |
| 3 | [3, 4, -10] |
+------+-----------------+
2 rows in set (0.00 sec)
I use a combination of JSON_EXTRACT and JSON_CONTAINS (MariaDB):
SELECT * FROM table WHERE JSON_CONTAINS(JSON_EXTRACT(json_field, '$[*].id'), 11, '$');
I don't know if we found the solution.
I found with MariaDB a way, to search path in a array. For example, in array [{"id":1}, {"id":2}], I want find path with id equal to 2.
SELECT JSON_SEARCH('name_field', 'one', 2, null, '$[*].id')
FROM name_table
The result is:
"$[1].id"
The asterisk indicate searching the entire array
This example works for me with mysql 5.7 above
SET #j = '{"a": [ "8428341ffffffff", "8428343ffffffff", "8428345ffffffff", "8428347ffffffff","8428349ffffffff", "842834bffffffff", "842834dffffffff"], "b": 2, "c": {"d": 4}}';
select JSON_CONTAINS(JSON_EXTRACT(#j , '$.a'),'"8428341ffffffff"','$') => returns 1
notice about " around search keyword, '"8428341ffffffff"'
A possible way is to deal with the problem as string matching. Convert the JSON to string and match.
Or you can use JSON_CONTAINS.
You can use JSON extract to search and select data
SELECT data, data->"$.id" as selectdata
FROM table
WHERE JSON_EXTRACT(data, "$.id") = '123'
#ORDER BY c->"$.name";
limit 10 ;
SET #doc = '[{"SongLabels": [{"SongLabelId": "111", "SongLabelName": "Funk"}, {"SongLabelId": "222", "SongLabelName": "RnB"}], "SongLabelCategoryId": "test11", "SongLabelCategoryName": "曲风"}]';
SELECT *, JSON_SEARCH(#doc, 'one', '%un%', null, '$[*].SongLabels[*].SongLabelName')FROM t_music_song_label_relation;
result: "$[0].SongLabels[0].SongLabelName"
SELECT song_label_content->'$[*].SongLabels[*].SongLabelName' FROM t_music_song_label_relation;
result: ["Funk", "RnB"]
I have similar problem, search via function
create function searchGT(threshold int, d JSON)
returns int
begin
set #i = 0;
while #i < json_length(d) do
if json_extract(d, CONCAT('$[', #i, ']')) > threshold then
return json_extract(d, CONCAT('$[', #i, ']'));
end if;
set #i = #i + 1;
end while;
return null;
end;
select searchGT(3, CAST('[1,10,20]' AS JSON));
This seems to be possible with to JSON_TABLE function. It's available in mysql version 8.0 or mariadb version 10.6.
With this test setup
CREATE TEMPORARY TABLE mytable
WITH data(a,json) AS (VALUES ('a','[1]'),
('b','[1,2]'),
('c','[1,2,3]'),
('d','[1,2,3,4]'))
SELECT * from data;
we get the following table
+---+-----------+
| a | json |
+---+-----------+
| a | [1] |
| b | [1,2] |
| c | [1,2,3] |
| d | [1,2,3,4] |
+---+-----------+
It's possible to select every row from mytable wich has a value greater than 2 in the json array with this query.
SELECT * FROM mytable
WHERE TRUE IN (SELECT val > 2
FROM JSON_TABLE(json,'$[*]'
columns (val INT(1) path '$')
) as json
)
Returns:
+---+-----------+
| a | json |
+---+-----------+
| c | [1,2,3] |
| d | [1,2,3,4] |
+---+-----------+