I have a table with a json column that contains an array of objects, like the following:
create table test_json (json_id int not null primary key, json_data json not null) select 1 as json_id, '[{"category":"circle"},{"category":"square", "qualifier":"def"}]' as json_data union select 2 as json_id, '[{"category":"triangle", "qualifier":"xyz"},{"category":"square"}]' as json_data;
+---------+----------------------------------------------------------------------------------------+
| json_id | json_data |
+--------------------------------------------------------------------------------------------------+
| 1 | [{"category":"circle"}, {"category":"square", "qualifier":"def"}] |
| 2 | [{"category":"triangle", "qualifier":"xyz"}, {"category":"square"}] |
+---------+----------------------------------------------------------------------------------------+
I'd like to be able to query this table to look for any rows (json_id's) that contain a json object in the array with both a "category" value of "square" and no "qualifier" property.
The sample table above is just a sample and I'm looking for a query that would work over hundreds of rows and hundreds of objects in the json array.
In MySQL 8.0, you would use JSON_TABLE() for this:
mysql> select json_id, j.* from test_json, json_table(json_data, '$[*]' columns (
category varchar(20) path '$.category',
qualifier varchar(10) path '$.qualifier')) as j
where j.category = 'square' and j.qualifier is null;
+---------+----------+-----------+
| json_id | category | qualifier |
+---------+----------+-----------+
| 2 | square | NULL |
+---------+----------+-----------+
It's not clear why you would use JSON for this at all. It would be better to store the data in the normal manner, one row per object, with category and qualifier as individual columns.
A query against normal columns is a lot simpler to write, and you can optimize the query easily with an index:
select * from mytable where category = 'square' and qualifier is null;
I found another solution using only MySQL 5.7 JSON functions:
select json_id, json_data from test_json
where json_extract(json_data,
concat(
trim(trailing '.category' from
json_unquote(json_search(json_data, 'one', 'square'))
),
'.qualifier')
) is null
This assumes the value 'square' only occurs as a value for a "category" field. This is true in your simple example, but I don't know if it will be true in your real data.
Result:
+---------+------------------------------------------------------------------------+
| json_id | json_data |
+---------+------------------------------------------------------------------------+
| 2 | [{"category": "triangle", "qualifier": "xyz"}, {"category": "square"}] |
+---------+------------------------------------------------------------------------+
I still think that it's a CodeSmell anytime you reference JSON columns in a condition in the WHERE clause. I understood your comment that this is a simplified example, but regardless of the JSON structure, if you need to do search conditions, your queries will be far easier to develop if your data is stored in conventional columns in normalized tables.
Your request is not clear. Both of your SQL records has not such properties but your JSON object has. Maybe you try to find any record that has such object. So the following is your answer:
create table test_json (json_id int not null primary key, json_data json not null) select 1 as json_id, '[{"category":"circle", "qualifier":"abc"},{"category":"square", "qualifier":"def"}]' as json_data union select 2 as json_id, '[{"category":"triangle", "qualifier":"xyz"},{"category":"square"}]' as json_data;
select * from test_json;
select * from test_json where 'square' in (JSON_EXTRACT(json_data, '$[0].category'),JSON_EXTRACT(json_data, '$[1].category'))
AND (JSON_EXTRACT(json_data, '$[0].qualifier') is NULL || JSON_EXTRACT(json_data, '$[1].qualifier') is NULL);
See Online Demo
Also see JSON Function Reference
Related
I have a "table" named users with a column name "group_in" which stores an array like [1,2,3].
I am trying to remove an element with a value (eg. 1) for a specified row (eg id:1).
Before:
id| group_in
1 | **[1,2,3]**
2 | [1,3]
After:
id | group_in
1 | **[2,3]**
2 | [1,3]
I have tried the following:-
Update users
SET group_in = JSON_REMOVE(group_in,JSON_UNQUOTE(JSON_search(group_in, 'one', 1)))
where id = 1
but I got back is
id | group_in
1 | null
2 | [1,3]
Screenshot of my table and query result for your reference.
My table
Result gotten
Please help me if you know how to solve it
Thank you 🙏
I believe JSON_SEARCH works on strings, not sure, if it was extended to integer searches as well.
One way is to flatten the array and then recombine it while excluding the value as needed.
Query -
update users set group_in = (select new_grp from (
select id,json_arrayagg(grp) new_grp from users,
JSON_TABLE(group_in, "$[*]" COLUMNS(grp INT PATH '$')) as grp_id
where id=1
and grp<>1
group by id
)X
)
where id=1;
Refer fiddle here.
Try this,
UPDATE `channels` SET `group_in` = JSON_REMOVE(`group_in`, '$[2]') WHERE `id` = 1 ;
I have a table with JSON data like this:
{"a": [{"color": "blue", "value": 15}, {"color": "red", "value": 30}]}
I need to get the "value" that is inside the same object of "blue".
I thought to use the code below:
SELECT JSON_EXTRACT(my_data, '$.a[0].value');
The problem is that the "blue" object can be in any index of the array.
So, is there a way to retrieve the index first and then i'll query using the right index?
UPDATE
The Barmar's answer works but it needs to wrap in JSON_UNQUOTE()
Use JSON_SEARCH() to find the path to blue.
SELECT JSON_EXTRACT(my_data, JSON_UNQUOTE(REPLACE(JSON_SEARCH(my_data, 'one', 'blue'), '.color', '.value')))
JSON_SEARCH will return a string like $.a[0].color. REPLACE changes that to $.a[0].value, then you extract that element.
DEMO
Here's an example of using JSON_TABLE():
select j.* from d, json_table(d.data, '$.a[*]' columns (
color varchar(20) path '$.color',
value int path '$.value')
) as j;
+-------+-------+
| color | value |
+-------+-------+
| blue | 15 |
| red | 30 |
+-------+-------+
You can then apply conditions in the WHERE clause, as if you had stored the data in a normal table.
select j.* from d, json_table(d.data, '$.a[*]' columns (
color varchar(20) path '$.color',
value int path '$.value')
) as j
where j.color = 'blue';
+-------+-------+
| color | value |
+-------+-------+
| blue | 15 |
+-------+-------+
This requires you to write a complex query like this EVERY TIME you query the JSON data.
One wonders if it would have been easier to store the JSON in a normal table from the start.
I often recommend to MySQL users that storing data as JSON makes more work for you, if you need to make SQL expressions to reference individual fields within the JSON. I wouldn't use JSON in these cases, I'd explode the JSON array into rows, and the JSON fields into columns of a set of normal tables. Then you can write simpler queries, you can optimize with indexes, and you can use constraints and data types properly.
JSON is the most easily misused feature of the recent MySQL releases.
I have a table
X
- id
- name (eg: 'testing')
- array_column_name (eg: ["value1","value2"])
I want a query that will return the following output.
_____________________
| column1 | column2 |
| testing | value1 |
| testing | value2 |
|_________|_________|
Essentially what I need is to convert the array to rows.
I tried using
SELECT name, JSON_EXTRACT(#'array_column_name', '*') FROM X;
But I am not sure what to put as the second parameter of the JSON_EXTRACT since my columns contain a plain array instead array of objects.
If you are using MySQL >= 8.0.4, you can use JSON_TABLE, otherwise, to achieve what you need will be somewhat less simple:
SELECT
`X`.`name` `column1`,
`der`.`column2`
FROM
`X`,
JSON_TABLE(
`array_column_name`,
'$[*]'
COLUMNS(
`column2` VARCHAR(255) PATH '$'
)
) `der`;
See db-fiddle.
I have a table called slices with some simple json objects that looks like this:
id | payload | metric_name
---|---------------------------------------|------------
1 | {"a_percent":99.97,"c_percent":99.97} | metric_c
2 | {"a_percent":98.37,"c_percent":97.93} | metric_c
many records of this. I am trying to get this:
a_percent | c_percent
----------|----------
99.97 | 99.97
98.37 | 97.93
I am creating the type and using json_populate_recordset along with json_agg in the following fashion:
CREATE TYPE c_history AS(
"a_percent" NUMERIC(5, 2),
"c_percent" NUMERIC(5, 2)
);
SELECT * FROM
json_populate_recordset(
NULL :: c_history,
(
SELECT json_agg(payload::json) FROM slices
WHERE metric_name = 'metric_c'
)
);
The clause select json_agg(...) by itself produces a nice array of json objects, as expected:
[{"a_percent":99.97,"c_percent":99.97}, {"a_percent":98.37,"c_percent":97.93}]
But when I run it inside json_populate_recordset, I get Error : ERROR: must call json_populate_recordset on an array of objects.
What am I doing wrong?
This is a variant of #TimBiegeleisen's solution with the function json_populate_record() used in a from clause:
select id, r.*
from slices,
lateral json_populate_record(null::c_history, payload) r;
See rextester or SqlFiddle.
You don't need to use json_agg, since it appears you want to get the set of a_percent and c_percent values for each id in a separate record. Rather just call json_populate_recordset as follows:
SELECT id, (json_populate_record(null::c_history, payload)).* FROM slices
I have a Redshift table that looks like this:
id | metadata
---------------------------------------------------------------------------
1 | [{"pet":"dog"},{"country":"uk"}]
2 | [{"pet":"cat"}]
3 | []
4 | [{"country":"germany"},{"education":"masters"},{"country":"belgium"}]
All array elements have just one field.
There is no guarantee that a particular field will feature in any of an array's elements.
A field name can be repeated in an array
The array elements can be in any order
I am wanting to get back a table that looks like this:
id | field | value
------------------------
1 | pet | dog
1 | country | uk
2 | pet | cat
4 | country | germany
4 | education | masters
4 | country | belgium
I can then combine this with my queries on the rest of the input table.
I have tried playing around with the Redshift JSON functions, but without being able to write functions/use loops/have variables in Redshift, I really can't see a way to do this!
Please let me know if I can clarify anything else.
Thanks to this inspired blog post, I've been able to craft a solution. This is:
Create a look-up table to effectively 'iterate' over the elements of each array. The number of rows in this table has be equal to or greater than the maximum number of elements of arrays. Let's say this is 4 (it can be calculated using SELECT MAX(JSON_ARRAY_LENGTH(metadata)) FROM input_table):
CREATE VIEW seq_0_to_3 AS
SELECT 0 AS i UNION ALL
SELECT 1 UNION ALL
SELECT 2 UNION ALL
SELECT 3
);
From this, we can create one row per JSON element:
WITH exploded_array AS (
SELECT id, JSON_EXTRACT_ARRAY_ELEMENT_TEXT(metadata, seq.i) AS json
FROM input_table, seq_0_to_3 AS seq
WHERE seq.i < JSON_ARRAY_LENGTH(metadata)
)
SELECT *
FROM exploded_array;
Producing:
id | json
------------------------------
1 | {"pet":"dog"}
1 | {"country":"uk"}
2 | {"pet":"cat"}
4 | {"country":"germany"}
4 | {"education":"masters"}
4 | {"country":"belgium"}
However, I was needing to extract the field names/values. As I can't see any way to extract JSON field names using Redshift's limited functions, I'll do this using a regular expression:
WITH exploded_array AS (
SELECT id, JSON_EXTRACT_ARRAY_ELEMENT_TEXT(metadata, seq.i) AS json
FROM input_table, seq_0_to_3 AS seq
WHERE seq.i < JSON_ARRAY_LENGTH(metadata)
)
SELECT id, field, JSON_EXTRACT_PATH_TEXT(json, field)
FROM (
SELECT id, json, REGEXP_SUBSTR(json, '[^{"]\\w+[^"]') AS field
FROM exploded_array
);
There is generic version for CREATE VIEW seq_0_to_3. Let's call it CREATE VIEW seq_0_to_n. This can be generated by
CREATE VIEW seq_0_to_n AS (
SELECT row_number() over (
ORDER BY TRUE)::integer - 1 AS i
FROM <insert_large_enough_table> LIMIT <number_less_than_table_entries>);
This helps in generating large sequences as a view.
It's now possible in Redshift to treat strings in either array format [] or json format {} as parsable json structures. First let's make a temp table based on your data:
create temporary table #t1 (id int, json_str varchar(100));
truncate table #t1;
insert into #t1 values (1, '[{"pet":"dog"},{"country":"uk"}]');
insert into #t1 values (2, '[{"pet":"cat"}]');
insert into #t1 values (3, '[]');
insert into #t1 values (4, '[{"country":"germany"},{"education":"masters"},{"country":"belgium"}]');
This creation of a common table expression (cte) will be used to implicitly convert the json_str field into a formal json structure of SUPER type. If the table's field were already SUPER type, we could skip this step.
drop table if exists #t2;
create temporary table #t2 as
with cte as
(select
x.id,
json_parse(x.json_str) as json_str -- convert string to SUPER structure
from
#t1 x
)
select
x.id
,unnested
from
cte x, x.json_str as unnested -- an alias of cte and x.json_str is required!
order by
id
;
Now we have an exploded list of key/value pairs to easily extract:
select
t2.id
,json_key -- this is the extracted key
,cast(json_val as varchar) as json_val -- eleminates the double quote marks
from
#t2 t2, unpivot t2.unnested as json_val at json_key --"at some_label" (e.g. json_key) will extract the key
order by
id
A different way to render the info is to allow the parsing engine to turn keys into columns. This isn't what you asked for, but potentially interesting:
select
id
,cast(t2.unnested.country as varchar) -- data is already parsed into rows, so it's directly addressable now
,cast(t2.unnested.education as varchar)
,cast(t2.unnested.pet as varchar)
from
#t2 t2
;
If you want more info on this, use a search engine to search for parsing the SUPER data type. If the data already existed as SUPER in the Redshift table, these latter 2 queries would work natively against the table, no need for a temp table.