Map JSON to columns and rows in PostgreSQL - json

I'm trying to map JSON data to columns. Everything I need is contained in data array. Example:
{"data":
[
{"stamp":1348249585,"date":"2012-09-21 17:46","blur":"blurs/1.jpg","img":["imgs/1.jpg",[1600,1200]],"thumb":["thumbs/1.jpg",[150,113]]},
{"stamp":1375607177,"date":"2013-08-04 09:06","blur":"blurs/2.jpg","img":["imgs/2.jpg",[1600,1200]],"thumb":["thumbs/2.jpg",[150,113]]},
{"stamp":1376242046,"date":"2013-08-11 17:27","blur":"blurs/3.jpg","img":["imgs/3.jpg",[1600,1200]],"thumb":["thumbs/3.jpg",[150,113]]},
...
Currently, I am using #>> operator with dynamically generated condition:
1) Calculate number of elements in data array
2) Create varchar array condition to match every "row"
3) Process elements on individual rows.
My solution is:
select
json_row,
json_row#>>'{stamp}' as stamp,
json_row#>>'{date}' as date,
json_row#>>'{img,0}' as img,
json_row#>>'{img,1,0}' as img_width,
json_row#>>'{img,1,1}' as img_height,
json_row#>>'{thumb,0}' as thumb,
json_row#>>'{thumb,1,0}' as thumb_width,
json_row#>>'{thumb,1,1}' as thumb_height,
json_row#>>'{thumb,2,0}' as th_x1,
json_row#>>'{thumb,2,1}' as th_y1,
json_row#>>'{thumb,3,0}' as th_x2,
json_row#>>'{thumb,3,1}' as th_y2,
json_row#>>'{blur}'
from
(
select
(gjson#>>c.cond)::json json_row
from
gallery_json
cross join (
select ('{data,'|| generate_series(0,
(select json_array_length((gjson#>>'{data}')::json) from gallery_json) - 1) || '}')::varchar[] cond) c
) rd
This works and I can live with it. But, given that this is my first exercise with JSON in PostgreSQL I would like to ask if there is better way to map similar JSON structure to rows. I think that I am supposed to use json_populate_recordset, but did not succeed so far.
SQLFiddle does not work currently, sample data:
--drop table if exists gallery_json;
create table gallery_json(gjson json);
insert into gallery_json (gjson)
select '{"data":[
{"stamp":1348249585,"date":"2012-09-21 17:46","blur":"blurs/1.jpg","img":["imgs/1.jpg",[1600,1200]],"thumb":["thumbs/1.jpg",[150,113]]},
{"stamp":1376659268,"date":"2013-08-16 13:21","blur":"blurs/7.jpg","img":["imgs/7.jpg",[1600,539]],"thumb":["thumbs/7.jpg",[267,112],[332,112],[32,0]]},
{"stamp":1376666907,"date":"2013-08-16 15:28","blur":"blurs/8.jpg","img":["imgs/8.jpg",[1600,1200]],"thumb":["thumbs/8.jpg",[150,113]]},
{"stamp":1379016669,"date":"2013-09-12 20:11","blur":"blurs/11.jpg","img":["imgs/11.jpg",[1600,590]],"thumb":["thumbs/11.jpg",[267,112],[304,112],[18,0]]},
{"stamp":1383304027,"date":"2013-11-01 11:07","blur":"blurs/17.jpg","img":["imgs/17.jpg",[1600,1200]],"thumb":["thumbs/17.jpg",[150,113]]}]
,"blur":[600,336],"thumb":{"min":[150,112],"max":[267,200]}}'::json


SQL Fiddle
with data as (
select json_array_elements(gjson -> 'data') as data
from gallery_json
)
select
(data -> 'stamp')::text::bigint as stamp,
(data -> 'date')::text::timestamp as date,
(data -> 'blur')::text as blur,
(data -> 'img' -> 0)::text as img,
(data -> 'img' -> 1 -> 0)::text::int as img_width,
(data -> 'img' -> 1 -> 1)::text::int as img_height,
(data -> 'thumb' -> 0)::text as thumb,
(data -> 'thumb' -> 1 -> 0)::text::int as thumb_width,
(data -> 'thumb' -> 1 -> 1)::text::int as thumb_height,
(data -> 'thumb' -> 2 -> 0)::text::int as th_x1,
(data -> 'thumb' -> 2 -> 1)::text::int as th_y1,
(data -> 'thumb' -> 3 -> 0)::text::int as th_x2,
(data -> 'thumb' -> 3 -> 1)::text::int as th_y2
from data

Related

Postgresql - How to query nested array elements

I have JSON array data in PostgreSQL 13 table. I want to query this table to see all the nested array data in the output. I tried the below query, but it's not giving the expected output.
select
json_data::json -> 'Rows' -> 0 -> 'Values' ->> 0 as Lid
,json_data::json -> 'Rows' -> 0 -> 'Values' ->> 1 as L2LicenseId
,json_data::json -> 'Rows' -> 1 -> 'Values' ->> 0 as Lid
,json_data::json -> 'Rows' -> 1 -> 'Values' ->> 1 as L2LicenseId
from test;
Can someone please help me?
Sample Data
CREATE TABLE IF NOT EXISTS test
(
json_data text
);
INSERT INTO test (json_data) VALUES ('{"Origin":"api","Topic":"licenses","Timestamp":"2023-02-07T12:46:42.2568898+00:00","Columns":["LId","L2LicenseId","SfdcAccountId","SfdcLineItemId","SL","Quantity","StartDate","EndDate","DisplayName","ProductPrimaryKey"],"Schema":["string","string","string","string","string","int32","datetime","datetime","string","string"],"Rows":[{"Values":["1234","123456","ACC_","PurchaseT","SKU-0000","1","2023-01-09T00:00:00.0000000","2024-01-08T00:00:00.0000000","Automation with 5 users","lc11dev.my-dev.com"]},{"Values":["8967","8967-e567","fihikelo","Addon_00000490_2nd_GB","SKU-0490","3","2023-01-01T00:00:00.0000000","2023-01-22T00:00:00.0000000","Automation, Data 5GB","mygreattest01311433.my-dev.com"]}]}')
Expected Output
DB FIDDLE

You can do it using PostgreSQL function jsonb_array_elements (or json_array_elements). This function extracts all Json array elements like as rows view.
select
a2.value -> 'Values' ->> 0 as Lid,
a2.value -> 'Values' ->> 1 as L2LicenseId
from test a1
cross join jsonb_array_elements(a1.json_data::jsonb->'Rows') a2
-- Result:
lid | l2licenseid |
--- -+-------------+
1234 | 123456 |
8967 | 8967-e567 |

Postgres select value by key from json in a list

Given the following:
create table test (
id int,
status text
);
insert into test values
(1,'[]'),
(2,'[{"A":"d","B":"c"}]'),
(3,'[{"A":"g","B":"f"}]');
Is it possible to return?
id A B
1 null null
2 d c
3 g f
I am attempting something like this:
select id,
status::json ->> 0 #> "A" from test

Try this to address your specific example :
SELECT id, (status :: json)#>>'{0,A}' AS A, (status :: json)#>>'{0,B}' AS B
FROM test
see the result
see the manual :
jsonb #>> text[] → text
Extracts JSON sub-object at the specified path as text.
'{"a": {"b": ["foo","bar"]}}'::json #>> '{a,b,1}' → bar

This does it:
SELECT id,
(status::json->0)->"A" as A,
(status::json->0)->"B" as B
FROM test;

How do I declare an 'operator' that takes 2 arguments and uses letters rather than symbols?

I want to be able to make operators such as found in VB(.net) like shown below
Console.WriteLine ( 16 Mod 2 )
produces an output shown below
0
As you can see, this makes code slightly easier to read. How would I go about making functions that do this?
I have tried the following format
equal :: Integer -> Integer -> bool
x `equal` y = x == y
and I get the following error
*Main> 5 equal 5
<interactive>:1:1: error:
* Non type-variable argument
in the constraint: Num ((Integer -> Integer -> Bool) -> t -> t1)
(Use FlexibleContexts to permit this)
* When checking the inferred type
it :: forall t t1.
(Num ((Integer -> Integer -> Bool) -> t -> t1), Num t) =>
t1
*Main>
What is going wrong and how do I do it correctly?

You need to use backticks around equal when you call it, exactly as you did to define it.
5 `equal` 5
The way you wrote it, Haskell thinks you're trying to pass equal as an argument to 5, rather than the other way around.

Writing a function from definitions?

First off, I would like to make you all aware that I'm very new to Haskell, so to increase knowledge etc, I've been trying out questions and I'm pretty stuck on one. I think I'm nearly there but some more experienced advice would be appreciated. Here's the question:
A sporting team is represented by it's name and the amount of points they scored in their last games like so ("Newcastle",[3,3,3,0]). This data is modelled by the type definitions:
type TName = String
type Points = [Int]
type Team = (TName,Points)
From this I have to define the following function that orders one team higher than another if their points sum is greater:
sortPoints :: [Team] -> [Team]
This is what I've tried:
sortPoints :: [Team] -> [Team]
sortPoints [_,()] -> []
sortPoints [_,(x:xs)] = sum[x|x<-xs]
Once I get to here, I'm not too sure how to go about adding the conditions to check the points sum, any pointers would be greatly appreciated as I'm still coming to terms with a lot of Haskell features.

Note: This post is written in literate Haskell. You can save it as Team.lhs and try it. That being said, it's basically a longer version of Carsten's comment. If you still try to figure things out, use hoogle and look for the functions, although it's fine if you manage things with sortBy solely first.
First of all, we're going to work on lists, so you want to import Data.List.
> module Team where
> import Data.List
It contains a function called sortBy:
sortBy :: (a -> a -> Ordering) -> [a] -> [a]
The first argument of sortBy should be a function that compares two elements of your list and returns
LT if the first is less than the second,
EQ if the both are equal,
GT if the first is greater than the second.
So we need something that that takes two teams and returns their ordering:
> -- Repeating your types for completeness
> type TName = String
> type Points = [Int]
> type Team = (TName, Points)
>
> compareTeams :: Team -> Team -> Ordering
Now, you want to compare teams based on their sum of their points. You don't need their name, so you can capture just the second part of the pair:
> compareTeams (_, s1) (_, s2) =
We need the sum of the points, so we define sum1 and sum2 to be the respective sums of the teams:
> let sum1 = sum s1
> sum2 = sum s2
Now we can compare those sums:
in if sum1 < sum2
then LT
else if sum1 == sum2
then EQ
else GT
However, that's rather verbose, and there's already a function that has type Ord a => a -> a -> Ordering. It's called compare and part of the Prelude:
> in sum1 `compare` sum2
That's a lot more concise. Now we can define sortTeams easily:
> sortTeams :: [Team] -> [Team]
> sortTeams = sortBy compareTeams
And that's it, we're done!
Fine, I lied, we're not 100% done. The module Data.Ord contains a function called comparing that's rather handy:
comparing :: Ord b => (a -> b) -> a -> a -> Ordering
comparing f x y = f x `compare` f y -- or similar
Together with snd and sum you can define sortTeams in a single line:
sortTeams = sortBy (comparing $ sum . snd)
The alternative on mentioned by Carsten is on from Data.Function:
sortTeams = sortBy (compare `on` sum . snd)

Declaration of this haskell method

Im trying to understand Haskell and I have a question: What is the type of this function and how do you call it.
two f(a,b) = f a b

If we take, for example, arguments of type Int, then the type of two is like this:
two :: (Int -> Int -> Int) -> (Int, Int) -> Int
two f (a,b) = f a b
example:
two (*) (3,4)
12
Explanation:
You are taking a function that takes 2 arguments (Int -> Int -> Int)and a tuple (Int, Int) and applying that function to a and b.
The actual type, when not constrained, is actually like this:
:t two
two :: (t1 -> t2 -> t) -> (t1, t2) -> t
So for example other things are possible:
two (++) ("he","llo")
"hello"
(etc etc.)