using a Postgres-Db as source for json-documents, I need to convert two columns from a table to an JSON-object.
So I have the columns "color_id", "language" and "name" in a table of colors:
color_id | language | name
1 | "de" | "blau"
1 | "en" | "blue"
1 | "fr" | "bleu"
And I'd like to produce a JSON-object like:
{
"de": "blau",
"fr": "bleu",
"en": "blue"
}
I started with
SELECT
array_to_json(array_agg((language::text, name::text))),
color_id
FROM colors
GROUP BY color_id;
which unfortunately produced
array to json | color_id
"[{"f1":"de","f2":"blau"} |
, {"f1":"en","f2":"blue"} | 1
, {"f1":"fr","f2":"bleu"}]" |
I'd think it would be simple - more or less -, but found myself at a dead end of misleading results and syntax errors.
Kind regards, Dominik
Use jsonb_object_agg():
with data(color_id, language, name) as (
values
(1, 'de', 'blau'),
(1, 'en', 'blue'),
(1, 'fr', 'bleu')
)
select color_id, jsonb_object_agg(language, name)
from data
group by 1;
color_id | jsonb_object_agg
----------+--------------------------------------------
1 | {"de": "blau", "en": "blue", "fr": "bleu"}
(1 row)
The function jsonb_object_agg() was introduced in Postgres 9.5.
In Postgres 9.4 use json_object_agg() instead.
In Postgres 9.3 you have to construct the result using string functions:
select
color_id,
format('{%s}', string_agg(format('"%s": "%s"', language, name), ', '))::json
from data
group by 1;
For key value pair and Proper JSON data this can be used
with data(color_id, language, name) as (
values
(1, 'de', 'blau'),
(1, 'en', 'blue'),
(1, 'fr', 'bleu')
)
SELECT color_id,
json_agg((SELECT x FROM (SELECT language, name) AS x)) AS json_data
FROM data
group by "data".color_id;
Related
With this select:
SELECT '{"key1": "value1", "key2": "value2"}'::jsonb
can we transform the result object into the following array of objects keeping their keys and values respectively?
[{"key1": "value1"}, {"key2": "value2"}]
In my case, I have many-2-many relationship of tags and operations, and the following query with aggregation of tags into json object for each operation:
SELECT ot.operation_id, json_object_agg(t.name, t.value) AS tagsjson
FROM tag t
INNER JOIN operation_tag ot ON t.tag_id = ot.tag_id
GROUP BY ot.operation_id
This works fine except format of the result: I need array of tags [{name : value},...] instead of whole object containing tags as it's properties:
If I understand you correctly you are in a situation like the following:
your source table contains id, keys and values like the below
create table test_k_v (id int, k varchar, v varchar);
insert into test_k_v values (1, 'key1', 'value1');
insert into test_k_v values (1, 'key2', 'value2');
insert into test_k_v values (2, 'key3', 'value3');
insert into test_k_v values (2, 'key4', 'value4');
insert into test_k_v values (2, 'key5', 'value5');
which results in
defaultdb=> select * from test_k_v;
id | k | v
----+------+--------
1 | key1 | value1
1 | key2 | value2
2 | key3 | value3
2 | key4 | value4
2 | key5 | value5
(5 rows)
If so, you can create an array with all the tags with the following query:
select id, array_agg(json_build_object(k, v)) agg_res
from test_k_v
group by id;
result
id | agg_res
----+---------------------------------------------------------------------------------
2 | {"{\"key3\" : \"value3\"}","{\"key4\" : \"value4\"}","{\"key5\" : \"value5\"}"}
1 | {"{\"key1\" : \"value1\"}","{\"key2\" : \"value2\"}"}
(2 rows)
I have a column which contains a JSON value of different lengths
["The Cherries:2.50","Draw:3.25","Swansea Jacks:2.87"]
I want to split them and store into a JSON like so:
[
{
name: "The Cherries",
odds: 2.50
},
{
name: "Draw",
odds: 3.25
},
{
name: "Swansea",
odds: 2.87
},
]
What I did right now is looping and splitting them in the UI which to me is quite heavy for the client. I want to parse and split them all in a single query.
If you are running MySQL 8.0, you can use json_table() to split the original arrayto rows, and then build new objects and aggregate them with json_arrayagg().
We need a primary key column (or set of columns) so we can properly aggreate the generated rows, I assumed id:
select
t.id,
json_arrayagg(json_object(
'name', substring(j.val, 1, locate(':', j.val) - 1),
'odds', substring(j.val, locate(':', j.val) + 1)
)) new_js
from mytable t
cross join json_table(t.js, '$[*]' columns (val varchar(500) path '$')) as j
group by t.id
Demo on DB Fiddle
Sample data:
id | js
-: | :-------------------------------------------------------
1 | ["The Cherries:2.50", "Draw:3.25", "Swansea Jacks:2.87"]
Query results:
id | new_js
-: | :----------------------------------------------------------------------------------------------------------------------
1 | [{"name": "The Cherries", "odds": "2.50"}, {"name": "Draw", "odds": "3.25"}, {"name": "Swansea Jacks", "odds": "2.87"}]
You can use json_table to create rows from the json object.
Just replace table_name with your table name and json with the column that contains json
SELECT json_arrayagg(json_object('name',SUBSTRING_INDEX(person, ':', 1) ,'odds',SUBSTRING_INDEX(person, ':', -1) ))
FROM table_name,
JSON_TABLE(json, '$[*]' COLUMNS (person VARCHAR(40) PATH '$') people;
Here is a Db fiddle you can refer
https://dbfiddle.uk/?rdbms=mysql_8.0&fiddle=801de9f067e89a48d45ef9a5bd2d094a
I have a PostgreSQL table with unique key/value pairs, which were originally in a JSON format, but have been normalized and melted:
key | value
-----------------------------
name | Bob
address.city | Vancouver
address.country | Canada
I need to turn this into a hierarchical JSON:
{
"name": "Bob",
"address": {
"city": "Vancouver",
"country": "Canada"
}
}
Is there a way to do this easily within SQL?
jsonb_set() almost does everything for you, but unfortunately it can only create missing leafs (i.e. missing last keys on a path), but not whole missing branches. To overcome this, here is a modified version of it, which can set values on any missing levels:
create function jsonb_set_rec(jsonb, jsonb, text[])
returns jsonb
language sql
as $$
select case
when array_length($3, 1) > 1 and ($1 #> $3[:array_upper($3, 1) - 1]) is null
then jsonb_set_rec($1, jsonb_build_object($3[array_upper($3, 1)], $2), $3[:array_upper($3, 1) - 1])
else jsonb_set($1, $3, $2, true)
end
$$;
Now you only need to apply this function one-by-one to your rows, starting with an empty json object: {}. You can do this with either recursive CTEs:
with recursive props as (
(select distinct on (grp)
pk, grp, jsonb_set_rec('{}', to_jsonb(value), string_to_array(key, '.')) json_object
from eav_tbl
order by grp, pk)
union all
(select distinct on (grp)
eav_tbl.pk, grp, jsonb_set_rec(json_object, to_jsonb(value), string_to_array(key, '.'))
from props
join eav_tbl using (grp)
where eav_tbl.pk > props.pk
order by grp, eav_tbl.pk)
)
select distinct on (grp)
grp, json_object
from props
order by grp, pk desc;
Or, with a custom aggregate defined as:
create aggregate jsonb_set_agg(jsonb, text[]) (
sfunc = jsonb_set_rec,
stype = jsonb,
initcond = '{}'
);
your query could became as simple as:
select grp, jsonb_set_agg(to_jsonb(value), string_to_array(key, '.'))
from eav_tbl
group by grp;
https://rextester.com/TULNU73750
There are no ready to use tools for this. The function generates a hierarchical json object based on a path:
create or replace function jsonb_build_object_from_path(path text, value text)
returns jsonb language plpgsql as $$
declare
obj jsonb;
keys text[] := string_to_array(path, '.');
level int := cardinality(keys);
begin
obj := jsonb_build_object(keys[level], value);
while level > 1 loop
level := level- 1;
obj := jsonb_build_object(keys[level], obj);
end loop;
return obj;
end $$;
You also need the aggregate function jsonb_merge_agg(jsonb) described in this answer. The query:
with my_table (path, value) as (
values
('name', 'Bob'),
('address.city', 'Vancouver'),
('address.country', 'Canada'),
('first.second.third', 'value')
)
select jsonb_merge_agg(jsonb_build_object_from_path(path, value))
from my_table;
gives this object:
{
"name": "Bob",
"first":
{
"second":
{
"third": "value"
}
},
"address":
{
"city": "Vancouver",
"country": "Canada"
}
}
The function do not recognize json arrays.
I can't really think of something simpler, although I think there should be an easier way.
I assume there is some additional column that can be used to bring the keys that belong to one "person" together, I used p_id for that in my example.
select p_id,
jsonb_object_agg(k, case level when 1 then v -> k else v end)
from (
select p_id,
elements[1] k,
jsonb_object_agg(case cardinality(elements) when 1 then ky else elements[2] end, value) v,
max(cardinality(elements)) as level
from (
select p_id,
"key" as ky,
string_to_array("key", '.') as elements, value
from kv
) t1
group by p_id, k
) t2
group by p_id;
The innermost query just converts the dot notation to an array for easier access later.
The next level then builds JSON objects depending on the "key". For the "single level" keys, it just uses key/value, for the others it uses the second element + the value and then aggregates those that belong together.
The second query level returns the following:
p_id | k | v | level
-----+---------+--------------------------------------------+------
1 | address | {"city": "Vancouver", "country": "Canada"} | 2
1 | name | {"name": "Bob"} | 1
2 | address | {"city": "Munich", "country": "Germany"} | 2
2 | name | {"name": "John"} | 1
The aggregation done in the second step, leaves one level too much for the "single element" keys, and that's what we need level for.
If that distinction wasn't made, the final aggregation would return {"name": {"name": "Bob"}, "address": {"city": "Vancouver", "country": "Canada"}} instead of the wanted: {"name": "Bob", "address": {"city": "Vancouver", "country": "Canada"}}.
The expression case level when 1 then v -> k else v end essentially turns {"name": "Bob"} back to "Bob".
So, with the following sample data:
create table kv (p_id integer, "key" text, value text);
insert into kv
values
(1, 'name','Bob'),
(1, 'address.city','Vancouver'),
(1, 'address.country','Canada'),
(2, 'name','John'),
(2, 'address.city','Munich'),
(2, 'address.country','Germany');
then query returns:
p_id | jsonb_object_agg
-----+-----------------------------------------------------------------------
1 | {"name": "Bob", "address": {"city": "Vancouver", "country": "Canada"}}
2 | {"name": "John", "address": {"city": "Munich", "country": "Germany"}}
Online example: https://rextester.com/SJOTCD7977
create table kv (key text, value text);
insert into kv
values
('name','Bob'),
('address.city','Vancouver'),
('address.country','Canada'),
('name','John'),
('address.city','Munich'),
('address.country','Germany');
create view v_kv as select row_number() over() as nRec, key, value from kv;
create view v_datos as
select k1.nrec, k1.value as name, k2.value as address_city, k3.value as address_country
from v_kv k1 inner join v_kv k2 on (k1.nrec + 1 = k2.nrec)
inner join v_kv k3 on ((k1.nrec + 2= k3.nrec) and (k2.nrec + 1 = k3.nrec))
where mod(k1.nrec, 3) = 1;
select json_agg(json_build_object('name',name, 'address', json_build_object('city',address_city, 'country', address_country)))
from v_datos;
In the query below, $isComplete and $isValid are returned as a string. However, they are saved as boolean values. How can I get the boolean representation of these fields to be returned?
query =
"SELECT
data #>> '{id}' AS id,
data #>> '{name}' AS name,
data #>> '{curator}' AS curator,
data #> '{$isValid}' as \"$isValid\",
data #> '{customer}' as customer,
data #> '{$createdTS}' as \"$createdTS\",
data #> '{$updatedTS}' as \"$updatedTS\",
data #> '{$isComplete}' as \"$isComplete\",
(count(keys))::numeric as \"numProducts\"
FROM
appointment_intakes,
LATERAL jsonb_object_keys(data #> '{products}') keys
GROUP BY id"
Simply cast a text to boolean:
create table jsonb_test (id int, data jsonb);
insert into jsonb_test values
(1, '{"is_boolean" : true}'),
(2, '{"is_boolean" : false}');
select id, data, (data->>'is_boolean')::boolean as is_boolean
from jsonb_test
where (data->>'is_boolean')::boolean
id | data | is_boolean
----+------------------------+------------
1 | {"is_boolean": true} | t
(1 row)
Note that you can also cast other json text values to boolean, examples:
insert into jsonb_test values
(3, '{"is_boolean" : "true"}'),
(4, '{"is_boolean" : "false"}'),
(5, '{"is_boolean" : "t"}'),
(6, '{"is_boolean" : "f"}'),
(7, '{"is_boolean" : "on"}'),
(8, '{"is_boolean" : "off"}');
select id, data, (data->>'is_boolean')::boolean as is_boolean
from jsonb_test
where (data->>'is_boolean')::boolean
id | data | is_boolean
----+------------------------+------------
1 | {"is_boolean": true} | t
3 | {"is_boolean": "true"} | t
5 | {"is_boolean": "t"} | t
7 | {"is_boolean": "on"} | t
(4 rows)
Read about valid literals for boolean type in the documentation.
Update
Postgres 11 adds casts from JSONB scalars to numeric and boolean data types. This query will work only for regular boolean JSONB scalars (i.e. true or false):
select id, data, (data->'is_boolean')::boolean as is_boolean
from jsonb_test
where (data->'is_boolean')::boolean
I have a table like this:
+--------+--------------------+
| ID | Attribute |
+--------+--------------------+
| 1 |"color" => "red" |
+--------+--------------------+
| 1 |"color" => "green" |
+--------+--------------------+
| 1 |"shape" => "square" |
+--------+--------------------+
| 2 |"color" => "blue" |
+--------+--------------------+
| 2 |"color" => "black" |
+--------+--------------------+
| 2 |"flavor" => "sweat" |
+--------+--------------------+
| 2 |"flavor" => "salty" |
+--------+--------------------+
And I want to run some postgres query that get a result table like this:
+--------+------------------------------------------------------+
| ID | Attribute |
+--------+------------------------------------------------------+
| 1 |"color" => "red, green", "shape" => "square" |
+--------+------------------------------------------------------+
| 2 |"color" => "blue, black", "flavor" => "sweat, salty" |
+--------+------------------------------------------------------+
The attribute column can either be hstore or json format. I wrote it in hstore for an example, but if we cannot achieve this in hstore, but in json, I would change the column to json.
I know that hstore does not support one key to multiple values, when I tried some merge method, it only kept one value for each key. But for json, I didn't find anything that supports multiple value merge like this neither. I think this can be done by function merging values for the same key into a string/text and add it back to the key/value pair. But I'm stuck in implementing it.
Note: if implement this in some function, ideally any key such as color, shape should not appear in the function since keys can be expanded dynamically.
Does anyone have any idea about this? Any advice or brainstorm might help. Thank you!
Just a note before anything else: in your desidered output I would use some proper json and not that kind of lookalike. So a correct output according to me would be:
+--------+----------------------------------------------------------------------+
| ID | Attribute |
+--------+----------------------------------------------------------------------+
| 1 | '{"color":["red","green"], "flavor":[], "shape":["square"]}' |
+--------+----------------------------------------------------------------------+
| 2 | '{"color":["blue","black"], "flavor":["sweat","salty"], "shape":[]}' |
+--------+----------------------------------------------------------------------+
A PL/pgSQL function which parses the json attributes and executes a dynamic query would do the job, something like that:
CREATE OR REPLACE FUNCTION merge_rows(PAR_table regclass) RETURNS TABLE (
id integer,
attributes json
) AS $$
DECLARE
ARR_attributes text[];
VAR_attribute text;
ARR_query_parts text[];
BEGIN
-- Get JSON attributes names
EXECUTE format('SELECT array_agg(name ORDER BY name) AS name FROM (SELECT DISTINCT json_object_keys(attribute) AS name FROM %s) AS s', PAR_table) INTO ARR_attributes;
-- Write json_build_object() query part
FOREACH VAR_attribute IN ARRAY ARR_attributes LOOP
ARR_query_parts := array_append(ARR_query_parts, format('%L, array_remove(array_agg(l.%s), null)', VAR_attribute, VAR_attribute));
END LOOP;
-- Return dynamic query
RETURN QUERY EXECUTE format('
SELECT t.id, json_build_object(%s) AS attributes
FROM %s AS t,
LATERAL json_to_record(t.attribute) AS l(%s)
GROUP BY t.id;',
array_to_string(ARR_query_parts, ', '), PAR_table, array_to_string(ARR_attributes, ' text, ') || ' text');
END;
$$ LANGUAGE plpgsql;
I've tested it and it seems to work, it returns a json with. Here is my test code:
CREATE TABLE mytable (
id integer NOT NULL,
attribute json NOT NULL
);
INSERT INTO mytable (id, attribute) VALUES
(1, '{"color":"red"}'),
(1, '{"color":"green"}'),
(1, '{"shape":"square"}'),
(2, '{"color":"blue"}'),
(2, '{"color" :"black"}'),
(2, '{"flavor":"sweat"}'),
(2, '{"flavor":"salty"}');
SELECT * FROM merge_rows('mytable');
Of course you can pass the id and attribute column names as parameters as well and maybe refine the function a bit, this is just to give you an idea.
EDIT : If you're on 9.4 please consider using jsonb datatype, it's much better and gives you room for improvements. You would just need to change the json_* functions to their jsonb_* equivalents.
If you just want this for display purposes, this might be enough:
select id, string_agg(key||' => '||vals, ', ')
from (
select t.id, x.key, string_agg(value, ',') vals
from t
join lateral each(t.attributes) x on true
group by id, key
) t
group by id;
If you are not on 9.4, you can't use the lateral join:
select id, string_agg(key||' => '||vals, ', ')
from (
select id, key, string_agg(val, ',') as vals
from (
select t.id, skeys(t.attributes) as key, svals(t.attributes) as val
from t
) t1
group by id, key
) t2
group by id;
This will return:
id | string_agg
---+-------------------------------------------
1 | color => red,green, shape => square
2 | color => blue,black, flavor => sweat,salty
SQLFiddle: http://sqlfiddle.com/#!15/98caa/2