Join tables with values from JSON - json

Im having problems querying data from json fields.
I have some json format columns saved as text in my postgres database (version 10)
Sometimes i need to be able to join two tables on json values.
I have no idea how i can do this..
Here is an simple example.
In my select i want to output the fruit and the color.
I have the color_cd number inside a json in the fruits table and i can find the color inside another json in the code_table.
My wished output should be like this
Fruit_ID Name Color
1000 Pear Green
1001 Banana Yellow
Fiddle link --> https://dbfiddle.uk/?rdbms=postgres_13&fiddle=3f989db0524e288183619bab63fc9add

Your column records on table code_table has problem in json data and i fixed problem and change to below format:
{"color_cd":{"30":{"code":"30","color":"yellow"},"55":{"code":"55","color":"green"},"60":{"code":"60","color":"red"}}}
You can see query structure and result in dbfiddle
select
f.id,
f.name,
j_cd.value ->> 'color' as color
from
code_table ct
cross join jsonb_each(records::jsonb -> 'color_cd') j_cd
inner join fruits f on f.type_cd :: jsonb ->> 'color_cd' = j_cd.key::text
where
ct.name = 'color_cd'

Related

Join a json value with a column in postgresql

I want to join a column with a json value. The problem is the json value is within square brackets and its a uuid. Table name that has the json column(column named json) is department and the other table name is staff. The json column value would be like below,
{"title":"Manager","alternativeTitle":null,"departmentIds":["c8098u43-7d9a-3789-gt56-r78009v4r345"]}
I would like to query the departmentIds from the json column and join it with staffdepartmentID column in the staff table.
My query for the join
from staff s
join department d on d.json ->> departmentIds::json = s.staffdepartmentID
The problem I am facing is that I dont know how to remove those square brackets. Any help is highly appreciated. Thanks
Square braquets within a json data correspond to an array.
You can access any element of the array based on its position starting with 0 for the first element : array->0
So for your query you can do :
from staff s
inner join department d
on d.json -> 'departmentIds'->>0 = s.staffdepartmentID :: text

Unnesting stringified JSON with two sets of double quotes

I have a python script that polls an API and stores the response in a BQ table with layered, stringified JSON field that looks like so:
"{""inputs"": {""Layer1"": {""Layer2"": {""Layer3"": {""item1"": 0.7, ""item2"": 10.0, ""item3"": 0.14}}}}"
I created the following query to unnest:
ro.id,
json_extract_scalar(s,
'$.item1') AS item1,
json_extract_scalar(s,
'$.item2') AS item2,
json_extract_scalar(s,
'$.item3') AS item3
FROM
`project.dataset.table` ro
LEFT JOIN
UNNEST(json_extract_array(response)) AS r
LEFT JOIN
UNNEST (json_extract_array("Layer1")) AS o
LEFT JOIN
UNNEST (json_extract_array("Layer2")) AS sc
LEFT JOIN
UNNEST (json_extract_array("Layer3")) AS s
The query runs and builds the table as expected, but all of the data from the unnested fields is null.
I do not have control over how the script stores the data, so I can only leverage SQL to solve this problem. Are the two sets of double quotes my main issue here? If so, what is the best way to address that?
Consider below approach
SELECT
json_extract_scalar(items,'$.item1') AS item1,
json_extract_scalar(items,'$.item2') AS item2,
json_extract_scalar(items,'$.item3') AS item3,
FROM `project.dataset.table` ro,
UNNEST([struct(json_extract(response, '$.inputs.Layer1.Layer2.Layer3') as items)])
if applied to sample data in your question - output is

Join with JSON fields

In the database (postgres 10) i work with at the moment i have a lot of columns with json format strings (text fields)
I have a hard time to search,filter and join with values inside thoose strings.
I created an example in fiddle and i hope someone can help me solve it.
https://dbfiddle.uk/?rdbms=postgres_13&fiddle=d003c1cea832e35696260b10c6b4c047
Here is my two problems i found tricky.
In table object -> configuration field i have "date_kg_later" field. It could be [] or hold more dates. In my select i want to get the highest date if it's not empty.
In table object -> object_type_cd refers on a value inside the records string in code_table. Here i want to get the title from that string back.
My goal is an output like this:
id name object_type_name date
1000 Headphones tech 2022-04-30
1001 Pencil null null
Your first question : if the json structure of your configuration field conforms the example you provide, then a solution can be :
Before Postgres 12 :
SELECT Max((c.elt->>'date_from') :: date)
FROM object
CROSS JOIN LATERAL json_array_elements(configuration->'date_kg_later') AS c(elt)
GROUP BY your_object_table_key
From and after Postgres 12 :
SELECT Max(d :: text :: date)
FROM object
CROSS JOIN LATERAL jsonb_array_elements((configuration :: jsonb)->'date_kg_later', '$[*].date_from') AS d
GROUP BY your_object_table_key
Your second question is unclear to me, needs more explanation.

How to create an empty JSON object in postgresql?

Datamodel
A person is represented in the database as a meta table row with a name and with multiple attributes which are stored in the data table as key-value pair (key and value are in separate columns).
Simplified data-model
Now there is a query to retrieve all users (name) with all their attributes (data). The attributes are returned as JSON object in a separate column. Here is an example:
name data
Florian { "age":25 }
Markus { "age":25, "color":"blue" }
Thomas {}
The SQL command looks like this:
SELECT
name,
json_object_agg(d.key, d.value) AS data,
FROM meta AS m
JOIN (
JOIN d.fk_id, d.key, d.value AS value FROM data AS d
) AS d
ON d.fk_id = m.id
GROUP BY m.name;
Problem
Now the problem I am facing is, that users like Thomas which do not have any attributes stored in the key-value table, are not shown with my select function. This is because it does only a JOIN and no LEFT OUTER JOIN.
If I would use LEFT OUTER JOIN then I run into the problem, that json_object_agg try's to aggregate NULL values and dies with an error.
Approaches
1. Return empty list of keys and values
So I tried to check if the key-column of a user is NULL and return an empty array so json_object_agg would just create an empty JSON object.
But there is not really a function to create an empty array in SQL. The nearest thing I found was this:
select '{}'::text[];
In combination with COALESCE the query looks like this:
json_object_agg(COALESCE(d.key, '{}'::text[]), COALESCE(d.value, '{}'::text[])) AS data
But if I try to use this I get following error:
ERROR: COALESCE types text and text[] cannot be matched
LINE 10: json_object_agg(COALESCE(d.key, '{}'::text[]), COALES...
^
Query failed
PostgreSQL said: COALESCE types text and text[] cannot be matched
So it looks like that at runtime d.key is a single value and not an array.
2. Split up JSON creation and return empty list
So I tried to take json_object_agg and replace it with json_object which does not aggregate the keys for me:
json_object(COALESCE(array_agg(d.key), '{}'::text[]), COALESCE(array_agg(d.value), '{}'::text[])) AS data
But there I get the error that null value not allowed for object key. So COALESCE does not check that the array is empty.
Qustion
So, is there a function to check if a joined column is empty, and if yes return just a simple JSON object?
Or is there any other solution which would solve my problem?
Use left join with coalesce(). As default value use '{}'::json.
select name, coalesce(d.data, '{}'::json) as data
from meta m
left join (
select fk_id, json_object_agg(d.key, d.value) as data
from data d
group by 1
) d
on m.id = d.fk_id;
name | data
---------+------------------------------------
Florian | { "age" : "25" }
Marcus | { "age" : "25", "color" : "blue" }
Thomas | {}
(3 rows)

Is it possible to condense the result of multiple rows into a set object on one row in an SQL query?

I'm refactoring some code, and looking at ways to try and improve both readability and performance.
One item that bugs me is situations where I need a join statement with multiple objects on one side. For example...
Foo Schema Bar 2 Schema
-------------- ---------------
id id
data fooId
data
Result from Search:
---------------------
id barId fooData
1 1 ...
1 2 ...
1 3 ...
2 4 ...
3 5 ...
My end result, when querying for object Foo, needs to be an object Foo containing the id's (or objects fetched based on the id's) that are related.
Currently, I wind up having to condense multiple rows at the PHP level, adding bar id's to Foo until the foo id changes. It's a bit ugly, but it does work. What'd I'd like to reduce my result set to would be:
Result from Search:
---------------------
id barIds fooData
1 [1,2,3] ...
2 4 ...
3 5 ...
Is there any way to do this at the SQL level? (As a note, I'm not looking for the literal string 1, 2, 3, I want an array consisting of the id's 1, 2, and 3 -- but if I have to take a string and then transform, I can do)
As an aside, my intent is to combine this with the PDO::fetch_class to let me instantiate the class in a single line instead of spending time writing many lines of cookie-cutter code to load the properties of the class.
Sounds like you're looking into using GROUP_CONCAT. This will combine all the bar ids together. Something like this:
SELECT F.Id, GROUP_CONCAT(B.Id) BarIds, F.data
FROM Foo F
INNER JOIN Bar B ON F.Id = B.FooId
GROUP BY F.Id
If you'd like to get your exact format, try using CONCAT:
CONCAT('[',GROUP_CONCAT(B.Id),']') BarIds
Here's a SQL Fiddle Demo.
--EDIT--
If there is a concern over the length of characters stored by default with GROUP_CONCAT (check this link out), another alternative approach would be to mimic the behavior of GROUP_CONCAT by doing something like this:
SELECT Id, BarIds, Data
FROM (
SELECT F.Id,
MAX(#barIdsCombined:=IF(#prevFoodId=F.Id,
CONCAT(#barIdsCombined,',',B.Id),
B.Id)) BarIds,
F.data,
#prevFoodId:=F.Id
FROM Foo F
INNER JOIN Bar B ON F.Id = B.FooId
JOIN (SELECT #barIdsCombined:='') t
GROUP BY F.Id
) t