Postgresql - How to query nested array elements - json

I have JSON array data in PostgreSQL 13 table. I want to query this table to see all the nested array data in the output. I tried the below query, but it's not giving the expected output.
select
json_data::json -> 'Rows' -> 0 -> 'Values' ->> 0 as Lid
,json_data::json -> 'Rows' -> 0 -> 'Values' ->> 1 as L2LicenseId
,json_data::json -> 'Rows' -> 1 -> 'Values' ->> 0 as Lid
,json_data::json -> 'Rows' -> 1 -> 'Values' ->> 1 as L2LicenseId
from test;
Can someone please help me?
Sample Data
CREATE TABLE IF NOT EXISTS test
(
json_data text
);
INSERT INTO test (json_data) VALUES ('{"Origin":"api","Topic":"licenses","Timestamp":"2023-02-07T12:46:42.2568898+00:00","Columns":["LId","L2LicenseId","SfdcAccountId","SfdcLineItemId","SL","Quantity","StartDate","EndDate","DisplayName","ProductPrimaryKey"],"Schema":["string","string","string","string","string","int32","datetime","datetime","string","string"],"Rows":[{"Values":["1234","123456","ACC_","PurchaseT","SKU-0000","1","2023-01-09T00:00:00.0000000","2024-01-08T00:00:00.0000000","Automation with 5 users","lc11dev.my-dev.com"]},{"Values":["8967","8967-e567","fihikelo","Addon_00000490_2nd_GB","SKU-0490","3","2023-01-01T00:00:00.0000000","2023-01-22T00:00:00.0000000","Automation, Data 5GB","mygreattest01311433.my-dev.com"]}]}')
Expected Output
DB FIDDLE

You can do it using PostgreSQL function jsonb_array_elements (or json_array_elements). This function extracts all Json array elements like as rows view.
select
a2.value -> 'Values' ->> 0 as Lid,
a2.value -> 'Values' ->> 1 as L2LicenseId
from test a1
cross join jsonb_array_elements(a1.json_data::jsonb->'Rows') a2
-- Result:
lid | l2licenseid |
--- -+-------------+
1234 | 123456 |
8967 | 8967-e567 |

Related

Converting a mysql dict column using JSON_Table

Edited
I have a mysql table "prop" with a column "detail" that contains a dict field.
fnum details
55 '{"a":"3"},{"b":"2"},{"d":"1"}'
I have tried to convert this to a table. using this
SELECT p.fnum, deets.*
FROM prop p
JOIN JSON_TABLE( p.details,
'$[*]'
COLUMNS (
idx FOR ORDINALITY,
a varChar(10) PATH '$.a',
b varchar(20) PATH '$.b'
d varchar(45) PATH '$.d',
)
) deets
I have tried various paths including $.*. I am expecting the following:
fnum a b d
55 3 2 1
also if I have 2 rows such as
fnum details
55 '{"a":"3"},{"b":"2"},{"d":"1"}'
56 '{"c":"car"}'
should generate the following
fnum a b d c
55 3 2 1 null
56 null null null car
Your details data is not valid JSON, because it doesn't have [ ] delimiting the array.
Demo:
mysql> create table prop (fnum int, details json);
mysql> insert into prop select 55, '{"a":"3"},{"b":"2"},{"d":"1"}';
ERROR 3140 (22032): Invalid JSON text: "The document root must
not be followed by other values." at position 9 in value for column
'prop.details'.
mysql> insert into prop select 55, '[{"a":"3"},{"b":"2"},{"d":"1"}]';
Query OK, 1 row affected (0.00 sec)
Records: 1 Duplicates: 0 Warnings: 0
It's worth using the JSON data type instead of storing JSON in a text column, because the JSON data type ensures that the document is valid JSON format. It must be valid JSON to use the JSON_TABLE() function or any other JSON function.
Also your query has some syntax mistakes with respect to commas:
SELECT p.fnum, deets.*
FROM prop p
JOIN JSON_TABLE( p.details,
'$[*]'
COLUMNS (
idx FOR ORDINALITY,
a varChar(10) PATH '$.a',
b varchar(20) PATH '$.b' <-- missing comma
d varchar(45) PATH '$.d', <-- extra comma
)
) deets

SQL IN query behavior

Executing the following SQL statement;
select '2312' in ('2312,254,2111') as result1, 2312 in ('2312,254,2111') as result2
I am getting the following result
+---------+---------+
| result1 | result2 |
+---------+---------+
| 0 | 1 |
+---------+---------+
I would expect the opposite result. Having result1 to be true and result2 to be false. Could someone explain why?
Using IN() with a CSV-string as parameter is not supported.
It should be IN ('2312','254','2111') instead of IN ('2312,254,2111')
The reason for the observed behaviour is an implicit type conversion happening. Look:
SELECT 2312 IN ('2312,254,2111') -- result: 1
SELECT 2312 IN ('254,2312,2111') -- result: 0 -- interesting
SELECT 2312 = '2312,254,2111' -- result: 1 << see ??
SELECT 2312 = '254,2312,2111' -- result: 0
Only the first number in the string is relevant. The rest is ignored due to the implicit type conversion.
Also,
SELECT '2312' in ('2312,254,2111') -- result: 0
is false, because no type conversion happens here. '2312' does not equal the only string '2312,254,2111' in the value list and hence the IN() operator return false.
If you use a list of values for IN() instead of a CSV-string, everything works als expected:
SELECT
2312 IN ('2312','254','2111') -- result: 1
, '2312' IN ('2312','254','2111') -- result: 1
, 254 IN ('2312','254','2111') -- result: 1
, '254' IN ('2312','254','2111') -- result: 1
, 2312 IN (2312,254,2111) -- result: 1
, '2312' IN (2312,254,2111) -- result: 1
, 254 IN (2312,254,2111) -- result: 1
, '254' IN (2312,254,2111) -- result: 1
From the manual:
Implicit type conversion may produce nonintuitive results:
mysql> SELECT 'a' IN (0), 0 IN ('b');
-> 1, 1
In both cases, the comparison values are converted to floating-point values, yielding 0.0 in each case, and a comparison result of 1 (true).

Postgres select value by key from json in a list

Given the following:
create table test (
id int,
status text
);
insert into test values
(1,'[]'),
(2,'[{"A":"d","B":"c"}]'),
(3,'[{"A":"g","B":"f"}]');
Is it possible to return?
id A B
1 null null
2 d c
3 g f
I am attempting something like this:
select id,
status::json ->> 0 #> "A" from test
Try this to address your specific example :
SELECT id, (status :: json)#>>'{0,A}' AS A, (status :: json)#>>'{0,B}' AS B
FROM test
see the result
see the manual :
jsonb #>> text[] → text
Extracts JSON sub-object at the specified path as text.
'{"a": {"b": ["foo","bar"]}}'::json #>> '{a,b,1}' → bar
This does it:
SELECT id,
(status::json->0)->"A" as A,
(status::json->0)->"B" as B
FROM test;

MySQL match pattern and select number and letters

I have a list of IDs which are created in various third party applications systems and manually added to our system. I need to try and auto increment these IDs based on the largest number. The values are either entirely a number or any number of letters followed by any number of numbers.
For example:
Array ( [works_id] => MD001 [num] => 0 )
Array ( [works_id] => WX9834V [num] => 0 )
Array ( [works_id] => WK009 [num] => 0 )
Array ( [works_id] => W4KHA2 [num] => 0 )
Array ( [works_id] => MD001 [num] => 0 )
Array ( [works_id] => DE1234 [num] => 0 )
Array ( [works_id] => 99 [num] => 99 )
Array ( [works_id] => 100 [num] => 100 )
In the above example, I would need to return 'DE' and 1234 as 1234 is the largest number which matches the pattern (WX9834V does not match as it is LLNNNNL)
So far I have tried:
SELECT works_id, CAST(works_id as UNSIGNED) as num
FROM table
WHERE (works_id REGEXP '^[a-zA-Z]+[0-9]' or works_id REGEXP '^[0-9]+$')
But this returns all rows and returns 0 for the number part unless it is only made up of numbers - how can I return only 'DE' and 1234 from the above?
From the comments, I understant that your primary intent is to select the records that do match your format spec (possibly characters at the beginning of the string, then mandatory numbers until the end of string).
The problem with you current query is that the first regexp, '^[a-zA-Z]+[0-9]' is too permissive: it does allow non-numbers characters at the end of the field, and would be better written '^[a-zA-Z]+[0-9]+$'
Bottom line, the two regexes can be combined into one:
SELECT works_id
FROM mytable
WHERE works_id REGEXP '^[a-zA-Z]*[0-9]+$'
The regexp means:
^ beginning of the string
[a-zA-Z]* 0 to N letters
[0-9]+ at least one digit
$ end of string
In this db fiddle with your test data, this returns:
| works_id |
| -------- |
| MD001 |
| WK009 |
| MD001 |
| 99 |
| 100 |
NB : in MySQL pre-8.0, splitting the string in order to find the max numerical pain is hard to do, since functions such as REGEXP_REPLACE are not available. It is probably easier to do this in your application (unless you have a very large numbers of matching records...). You can have a look at this post or this other one for solutions that mostly rely on MySQL functions.

Map JSON to columns and rows in PostgreSQL

I'm trying to map JSON data to columns. Everything I need is contained in data array. Example:
{"data":
[
{"stamp":1348249585,"date":"2012-09-21 17:46","blur":"blurs/1.jpg","img":["imgs/1.jpg",[1600,1200]],"thumb":["thumbs/1.jpg",[150,113]]},
{"stamp":1375607177,"date":"2013-08-04 09:06","blur":"blurs/2.jpg","img":["imgs/2.jpg",[1600,1200]],"thumb":["thumbs/2.jpg",[150,113]]},
{"stamp":1376242046,"date":"2013-08-11 17:27","blur":"blurs/3.jpg","img":["imgs/3.jpg",[1600,1200]],"thumb":["thumbs/3.jpg",[150,113]]},
...
Currently, I am using #>> operator with dynamically generated condition:
1) Calculate number of elements in data array
2) Create varchar array condition to match every "row"
3) Process elements on individual rows.
My solution is:
select
json_row,
json_row#>>'{stamp}' as stamp,
json_row#>>'{date}' as date,
json_row#>>'{img,0}' as img,
json_row#>>'{img,1,0}' as img_width,
json_row#>>'{img,1,1}' as img_height,
json_row#>>'{thumb,0}' as thumb,
json_row#>>'{thumb,1,0}' as thumb_width,
json_row#>>'{thumb,1,1}' as thumb_height,
json_row#>>'{thumb,2,0}' as th_x1,
json_row#>>'{thumb,2,1}' as th_y1,
json_row#>>'{thumb,3,0}' as th_x2,
json_row#>>'{thumb,3,1}' as th_y2,
json_row#>>'{blur}'
from
(
select
(gjson#>>c.cond)::json json_row
from
gallery_json
cross join (
select ('{data,'|| generate_series(0,
(select json_array_length((gjson#>>'{data}')::json) from gallery_json) - 1) || '}')::varchar[] cond) c
) rd
This works and I can live with it. But, given that this is my first exercise with JSON in PostgreSQL I would like to ask if there is better way to map similar JSON structure to rows. I think that I am supposed to use json_populate_recordset, but did not succeed so far.
SQLFiddle does not work currently, sample data:
--drop table if exists gallery_json;
create table gallery_json(gjson json);
insert into gallery_json (gjson)
select '{"data":[
{"stamp":1348249585,"date":"2012-09-21 17:46","blur":"blurs/1.jpg","img":["imgs/1.jpg",[1600,1200]],"thumb":["thumbs/1.jpg",[150,113]]},
{"stamp":1376659268,"date":"2013-08-16 13:21","blur":"blurs/7.jpg","img":["imgs/7.jpg",[1600,539]],"thumb":["thumbs/7.jpg",[267,112],[332,112],[32,0]]},
{"stamp":1376666907,"date":"2013-08-16 15:28","blur":"blurs/8.jpg","img":["imgs/8.jpg",[1600,1200]],"thumb":["thumbs/8.jpg",[150,113]]},
{"stamp":1379016669,"date":"2013-09-12 20:11","blur":"blurs/11.jpg","img":["imgs/11.jpg",[1600,590]],"thumb":["thumbs/11.jpg",[267,112],[304,112],[18,0]]},
{"stamp":1383304027,"date":"2013-11-01 11:07","blur":"blurs/17.jpg","img":["imgs/17.jpg",[1600,1200]],"thumb":["thumbs/17.jpg",[150,113]]}]
,"blur":[600,336],"thumb":{"min":[150,112],"max":[267,200]}}'::json
SQL Fiddle
with data as (
select json_array_elements(gjson -> 'data') as data
from gallery_json
)
select
(data -> 'stamp')::text::bigint as stamp,
(data -> 'date')::text::timestamp as date,
(data -> 'blur')::text as blur,
(data -> 'img' -> 0)::text as img,
(data -> 'img' -> 1 -> 0)::text::int as img_width,
(data -> 'img' -> 1 -> 1)::text::int as img_height,
(data -> 'thumb' -> 0)::text as thumb,
(data -> 'thumb' -> 1 -> 0)::text::int as thumb_width,
(data -> 'thumb' -> 1 -> 1)::text::int as thumb_height,
(data -> 'thumb' -> 2 -> 0)::text::int as th_x1,
(data -> 'thumb' -> 2 -> 1)::text::int as th_y1,
(data -> 'thumb' -> 3 -> 0)::text::int as th_x2,
(data -> 'thumb' -> 3 -> 1)::text::int as th_y2
from data