In the database (postgres 10) i work with at the moment i have a lot of columns with json format strings (text fields)
I have a hard time to search,filter and join with values inside thoose strings.
I created an example in fiddle and i hope someone can help me solve it.
https://dbfiddle.uk/?rdbms=postgres_13&fiddle=d003c1cea832e35696260b10c6b4c047
Here is my two problems i found tricky.
In table object -> configuration field i have "date_kg_later" field. It could be [] or hold more dates. In my select i want to get the highest date if it's not empty.
In table object -> object_type_cd refers on a value inside the records string in code_table. Here i want to get the title from that string back.
My goal is an output like this:
id name object_type_name date
1000 Headphones tech 2022-04-30
1001 Pencil null null
Your first question : if the json structure of your configuration field conforms the example you provide, then a solution can be :
Before Postgres 12 :
SELECT Max((c.elt->>'date_from') :: date)
FROM object
CROSS JOIN LATERAL json_array_elements(configuration->'date_kg_later') AS c(elt)
GROUP BY your_object_table_key
From and after Postgres 12 :
SELECT Max(d :: text :: date)
FROM object
CROSS JOIN LATERAL jsonb_array_elements((configuration :: jsonb)->'date_kg_later', '$[*].date_from') AS d
GROUP BY your_object_table_key
Your second question is unclear to me, needs more explanation.
Related
I want to join a column with a JSON value. The problem is the JSON value is an array and I want to join the resulting UUID value from the other column to whatever matches with the JSON Array. The table name that has the JSON column(column named staffdep) is department and the other table name is staff which has the staffId column.
staffdepID column's value would be like
{"departmentID":[],"staffID":["109ec36a-42bd-42fe-9b1f-c4f479c48fda","109ec36a-42bd-42fe-9b1f-c4f479c48fda"]}
staff id column would have a unique uuid for each row. For example like '109ec36a-42bd-42fe-9b1f-c4f479c48fda', '84dfbc00-0ff4-4689-98de-1d7496bb9da1'.
The extract of the query I used was,
from department d
join staff s on s.staffId = (d.staffdep -> 'staffID' ->> 0)::uuid
The issue with the above query is as I said above, the equivalent UUID of staffID might not always be the first value in the JSON array under d.staffdep. I would need a solution for this.
Any help is highly appreciated. Thanks
You can use a JSON path condition as the join condition:
from department d
join staff s on jsonb_path_exists(d.staffdep, '$.staffID[*] ? (# == $id)', jsonb_build_object('id', s.staffid::text));
I want to join a column with a json value. The problem is the json value is within square brackets and its a uuid. Table name that has the json column(column named json) is department and the other table name is staff. The json column value would be like below,
{"title":"Manager","alternativeTitle":null,"departmentIds":["c8098u43-7d9a-3789-gt56-r78009v4r345"]}
I would like to query the departmentIds from the json column and join it with staffdepartmentID column in the staff table.
My query for the join
from staff s
join department d on d.json ->> departmentIds::json = s.staffdepartmentID
The problem I am facing is that I dont know how to remove those square brackets. Any help is highly appreciated. Thanks
Square braquets within a json data correspond to an array.
You can access any element of the array based on its position starting with 0 for the first element : array->0
So for your query you can do :
from staff s
inner join department d
on d.json -> 'departmentIds'->>0 = s.staffdepartmentID :: text
Im having problems querying data from json fields.
I have some json format columns saved as text in my postgres database (version 10)
Sometimes i need to be able to join two tables on json values.
I have no idea how i can do this..
Here is an simple example.
In my select i want to output the fruit and the color.
I have the color_cd number inside a json in the fruits table and i can find the color inside another json in the code_table.
My wished output should be like this
Fruit_ID Name Color
1000 Pear Green
1001 Banana Yellow
Fiddle link --> https://dbfiddle.uk/?rdbms=postgres_13&fiddle=3f989db0524e288183619bab63fc9add
Your column records on table code_table has problem in json data and i fixed problem and change to below format:
{"color_cd":{"30":{"code":"30","color":"yellow"},"55":{"code":"55","color":"green"},"60":{"code":"60","color":"red"}}}
You can see query structure and result in dbfiddle
select
f.id,
f.name,
j_cd.value ->> 'color' as color
from
code_table ct
cross join jsonb_each(records::jsonb -> 'color_cd') j_cd
inner join fruits f on f.type_cd :: jsonb ->> 'color_cd' = j_cd.key::text
where
ct.name = 'color_cd'
From a CSV file (with a header and a pipe delimiter) I've got the following content which contains a JSON column (with a collection inside), like this:
ProductId|IngestTime|ProductOrders
9180|20171025145034|[{"OrderId":"299","Location":"NY"},{"OrderId":"499","Location":"LA"}]
8251|20171026114034|[{"OrderId":"1799","Location":"London"}]
What I need is to create a SELECT Hive query which returns:
ProductId IngestTime OrderId OrderLocation
9180 20171025145034 299 NY
9180 20171025145034 499 LA
8251 20171026114034 1799 London
So far, I tried many combinations by using 'explode', 'get_json_object' and so on, but I still haven't found the right SQL query.
Have you got a solution ?
Thanks a lot for your help :-)
I was having similar kind of requirement. The solution from this link helped me solve it.
BTW, below is the query for your requirement assuming all the columns in your DB_TABLE are of type 'String'.
SELECT ProductId,
IngestTime,
split(split(results,",")[0],':')[1] AS OrderId,
regexp_replace(split(split(results,",")[1],':')[1], "[\\]|}]", "") AS OrderLocation
FROM
(SELECT ProductId,
IngestTime,
split(translate(ProductOrders, '"\\[|]|\""',''), "},") AS r
FROM DB_TABLE) t1 LATERAL VIEW explode(r) rr AS results
Datamodel
A person is represented in the database as a meta table row with a name and with multiple attributes which are stored in the data table as key-value pair (key and value are in separate columns).
Simplified data-model
Now there is a query to retrieve all users (name) with all their attributes (data). The attributes are returned as JSON object in a separate column. Here is an example:
name data
Florian { "age":25 }
Markus { "age":25, "color":"blue" }
Thomas {}
The SQL command looks like this:
SELECT
name,
json_object_agg(d.key, d.value) AS data,
FROM meta AS m
JOIN (
JOIN d.fk_id, d.key, d.value AS value FROM data AS d
) AS d
ON d.fk_id = m.id
GROUP BY m.name;
Problem
Now the problem I am facing is, that users like Thomas which do not have any attributes stored in the key-value table, are not shown with my select function. This is because it does only a JOIN and no LEFT OUTER JOIN.
If I would use LEFT OUTER JOIN then I run into the problem, that json_object_agg try's to aggregate NULL values and dies with an error.
Approaches
1. Return empty list of keys and values
So I tried to check if the key-column of a user is NULL and return an empty array so json_object_agg would just create an empty JSON object.
But there is not really a function to create an empty array in SQL. The nearest thing I found was this:
select '{}'::text[];
In combination with COALESCE the query looks like this:
json_object_agg(COALESCE(d.key, '{}'::text[]), COALESCE(d.value, '{}'::text[])) AS data
But if I try to use this I get following error:
ERROR: COALESCE types text and text[] cannot be matched
LINE 10: json_object_agg(COALESCE(d.key, '{}'::text[]), COALES...
^
Query failed
PostgreSQL said: COALESCE types text and text[] cannot be matched
So it looks like that at runtime d.key is a single value and not an array.
2. Split up JSON creation and return empty list
So I tried to take json_object_agg and replace it with json_object which does not aggregate the keys for me:
json_object(COALESCE(array_agg(d.key), '{}'::text[]), COALESCE(array_agg(d.value), '{}'::text[])) AS data
But there I get the error that null value not allowed for object key. So COALESCE does not check that the array is empty.
Qustion
So, is there a function to check if a joined column is empty, and if yes return just a simple JSON object?
Or is there any other solution which would solve my problem?
Use left join with coalesce(). As default value use '{}'::json.
select name, coalesce(d.data, '{}'::json) as data
from meta m
left join (
select fk_id, json_object_agg(d.key, d.value) as data
from data d
group by 1
) d
on m.id = d.fk_id;
name | data
---------+------------------------------------
Florian | { "age" : "25" }
Marcus | { "age" : "25", "color" : "blue" }
Thomas | {}
(3 rows)