Clean way to query complex JSON in Postgresql - json

I have JSON data stored in a JSONB field in my postgresql 9.5 DB.
Is there a way of making sub-objects columns without knowing which column is a sub-object?
JSON example in question:
{
"a":1,
"b":[1,2,3],
"c":"bar",
"d":{
"key1":"value1",
"key2":"value2"
}
}
I can use the following to get all of the keys from a JSON object.
SELECT * FROM json_object_keys('{"a":1,"b":[1,2,3],"c":"bar", "d":{"key1":"value1", "key2":"value2"}}')
At that point I can then use json_to_record() but I would like to split the column out to their own separate fields.
select * from json_to_record('{"a":1,"b":[1,2,3],"c":"bar", "d":{"key1":"value1", "key2":"value2"}}') as x(a int, b text, c text, d text)
gets me
a| b | c | d
1| [1,2,3] | bar | {"key1":"value1", "key2":"value2"}
Is there a way to get something like this back, preferably in a single query?
--------------------------------------------------------------------
a| b | c | d | key1 | key2
1| [1,2,3] | bar | {"key1":"value1", "key2":"value2"} |value1 |value2

WITH t(v) AS ( VALUES
('{
"a":1,
"b":[1,2,3],
"c":"bar",
"d":{
"key1":"value1",
"key2":"value2"
}
}'::JSONB)
)
SELECT x1.*,x2.* FROM t,
jsonb_to_record(v) as x1(a int,b text,c text,d jsonb),
jsonb_to_record(v->'d') as x2(key1 text,key2 text);
Result:
a | b | c | d | key1 | key2
---+-----------+-----+--------------------------------------+--------+--------
1 | [1, 2, 3] | bar | {"key1": "value1", "key2": "value2"} | value1 | value2
(1 row)

Related

Print key, value pairs from nested MYSQL json

I have extracted a mysql json dictionary strucutre and I wish to get all the values associated with the keys alpha and beta; however I also wish to print the key too. The structure of the dictionary is:
results =
{1:
{"a": {"alpha": 1234,
"beta": 2345},
"b": {"alpha": 1234,
"beta": 2345},
"c": {"alpha": 1234,
"beta": 2345},
},
2:
{"ab": {"alpha": 1234,
"beta": 2345},
"ac": {"alpha": 1234,
"beta": 2345},
"bc": {"alpha": 1234,
"beta": 2345},
},
3:
{"abc": {"alpha": 1234,
"beta": 2345}
}
"random_key": "not_interested_in_this_value"
}
So far I have been had some succes extracting the data I wish using:
SELECT JSON_EXTRACT alpha, beta FROM results;
This gave me the alpha and beta columns; however, I ideally would like to assoicate each value with their key to get:
+-------+---------+---------+
| key | alpha | beta |
+-------+---------+---------+
| a | 1234. | 2345. |
| b | 1234. | 2345. |
| c | 1234. | 2345. |
| ab | 1234. | 2345. |
| ac | 1234. | 2345. |
| bc | 1234. | 2345. |
| abc | 1234. | 2345. |
+-------+---------+---------+
I am very new to mysql and any help is appreciated.
First of all, what you posted is not valid JSON. You can use integers as values, but you can't use integers as keys in objects. Also you have a few spurious , symbols. I had to fix these mistakes before I could insert the data into a table to test.
I was able to solve this using MySQL 8.0's JSON_TABLE() function in the following way:
select
j2.`key`,
json_extract(results, concat('$."',j1.`key`,'"."',j2.`key`,'".alpha')) as alpha,
json_extract(results, concat('$."',j1.`key`,'"."',j2.`key`,'".beta')) as beta
from mytable
cross join json_table(json_keys(results), '$[*]' columns (`key` int path '$')) as j1
cross join json_table(json_keys(json_extract(results, concat('$."',j1.`key`,'"'))), '$[*]' columns (`key` varchar(3) path '$')) as j2
where j2.`key` IS NOT NULL;
Output:
+------+-------+------+
| key | alpha | beta |
+------+-------+------+
| a | 1234 | 2345 |
| b | 1234 | 2345 |
| c | 1234 | 2345 |
| ab | 1234 | 2345 |
| ac | 1234 | 2345 |
| bc | 1234 | 2345 |
| abc | 1234 | 2345 |
+------+-------+------+
If you find this sort of query too difficult, I would encourage you to reconsider whether you want to store data in JSON.
If I were you, I'd store data in normal rows and columns, then the query would be a lot simpler and easier to write and maintain.

MySQL JSON with arbitrary keys to table

There is a map nested in a large json payload like
{
"map": {
"key1": "value1",
"key2": "value2",
"key3": "value3"
},
// more stuff
}
I would like to generate a table like that:
+------#--------+
| Key | Value |
+------#--------+
| key1 | value1 |
| key2 | value2 |
| key3 | value3 |
+------#--------+
The only thing I can think of is writing a stored function that loops over JSON_KEYS to convert all key value pairs into
[{"key":"key1", "value":"value1"}, {"key":"key2", "value":"value2"}, ...]
which makes the task trivial with JSON_TABLE.
Is there a faster and more elegant way?
Here's a solution:
select j.key, json_unquote(json_extract(m.data, concat('$.map.', j.key))) as value from mytable as m
cross join json_table(json_keys(m.data, '$.map'), '$[*]' columns (`key` varchar(10) path '$')) as j
Output with your sample data:
+------+--------+
| key | value |
+------+--------+
| key1 | value1 |
| key2 | value2 |
| key3 | value3 |
+------+--------+
If that query seems inelegant or hard to maintain, you're probably right. You shouldn't store data in JSON if you want simple or elegant queries.
your doing well and even for enterprise project doing this way

Append data to JSON array in Redshift

I have a Redshift table in which one of the columns is a JSON array. I would like to append some data into that array. Eg:
id | col1 | col2
1 | A | {"key": []}
2 | B | {"key": []}
3 | B | {"key": ['A']}
4 | B | {"key": ['A', 'B']}
I would like to create a statement like UPDATE <table> SET col2 = <something> where col1 = 'B' so that I get:
id | col1 | col2
1 | A | {"key": []}
2 | B | {"key": ['C']}
3 | B | {"key": ['A', 'C']}
4 | B | {"key": ['A', 'B', 'C']}
You'd have to write your own User Defined Function (UDF), passing in the current value of the column and the element you would like to add, then passing back the result.
Hwoever, you really should avoid JSON columns in Amazon Redshift if at all possible. They cannot take advantage of all the features that make Redshift great (columnar, SORTKEY, etc). Plus, you'll have problems like this that are not in the normal realm of a relational database.

Convert Parquet to JSON [duplicate]

I have the following sample DataFrame:
a | b | c |
1 | 2 | 4 |
0 | null | null|
null | 3 | 4 |
And I want to replace null values only in the first 2 columns - Column "a" and "b":
a | b | c |
1 | 2 | 4 |
0 | 0 | null|
0 | 3 | 4 |
Here is the code to create sample dataframe:
rdd = sc.parallelize([(1,2,4), (0,None,None), (None,3,4)])
df2 = sqlContext.createDataFrame(rdd, ["a", "b", "c"])
I know how to replace all null values using:
df2 = df2.fillna(0)
And when I try this, I lose the third column:
df2 = df2.select(df2.columns[0:1]).fillna(0)
df.fillna(0, subset=['a', 'b'])
There is a parameter named subset to choose the columns unless your spark version is lower than 1.3.1
Use a dictionary to fill values of certain columns:
df.fillna( { 'a':0, 'b':0 } )

Postgres update JSON field

I've got several Postgres 9.4 tables that contain data like this:
| id | data |
|----|-------------------------------------------|
| 1 | {"user": "joe", "updated-time": 123} |
| 2 | {"message": "hi", "updated-time": 321} |
I need to transform the JSON column into something like this
| id | data |
|----|--------------------------------------------------------------|
| 1 | {"user": "joe", "updated-time": {123, "unit":"millis"}} |
| 2 | {"message": "hi", "updated-time": {321, "unit":"millis"}} |
Ideally it would be easy to apply the transformation to multiple tables. Tables that contain the JSON key data->'updated-time' should be updated, and ones that do not should be skipped. Thanks!
You can use the || operator to merge two jsonb objects together.
select '{"foo":"bar"}'::jsonb || '{"baz":"bar"}'::jsonb;
= {"baz": "bar", "foo": "bar"}