I have a field varchar(65000) column in my AWS Redshift database which is used to store JSON strings. The JSON key/value pairs change frequently and I need to be able to run a daily report to retrieve all key/value data from the column.
For example:
create table test.json(json varchar(65000));
insert into test.json
select '{"animal_id": 1, "name": "harry", "animal_type": "cat", "age": 2, "location": "oakland"}' union
select '{"animal_id": 2, "name": "louie","animal_type": "dog", "age": 4}' union
select '{"animal_id": 3, "gender": "female"}' union
select '{"animal_id": 4, "size": "large"}' ;
With the above data I can write the below query to get the attributes I know are there however if a new attribute is added tomorrow, my report query will not pick up that new key/value pair. Is there any way to do a SELECT * type query on this table?
SELECT
json_extract_path_text(JSON,'animal_id') animal_id,
json_extract_path_text(JSON,'name') name,
json_extract_path_text(JSON,'animal_type') animal_type,
json_extract_path_text(JSON,'location') location,
json_extract_path_text(JSON,'age') age,
json_extract_path_text(JSON,'gender') gender,
json_extract_path_text(JSON,'size') size
FROM test.json
ORDER BY animal_id;
It is not possible to do what you want using your current schema with plain SQL.
If you can have application logic when creating your SQL query, you could dynamically create the SELECT statement.
Option A
Load the whole JSON in your app, parse it and obtain the required information this way.
Option B
When storing values in your database, parse the JSON object and add the discovered keys to another table. When querying your Redshift cluster, load this list of values and generate the appropriate SQL statement using this information.
Here's hoping these workarounds can be applied to your situation.
Related
Situation
I have a table in a MariaDB database. This table has a LONGTEXT column which is used to store a JSON array (read more about this topic in MariaDB JSON Data Type).
Question
I would like to extract values from the JSON array, based on a certain key. How do I achieve this with MariaDB (or MySQL)?
Example
Here's the simplified table thing (just for demo purposes):
id
thing_name
examples
0
fruit
[{"color": "green","title": "Apple"},{"color": "orange","title": "Orange"},{"color": "yellow","title": "Banana"}]
1
car
[{"color": "silver","title": "VW"},{"color": "black","title": "Bentley"},{"color": "blue","title": "Tesla"}]
My goal is to extract all title values from the JSON array.
You can use JSON_EXTRACT for this task (works for both MariaDB and MySQL). This function also supports wildcards, as described in the docs:
Paths can contain * or ** wildcards
Depending on whether you have multiple levels of data (e.g. single document vs array), either a single or double asterisk wildcard should be used:
JSON_EXTRACT(json,'$**.key')
json is a valid JSON document (e.g. a column), key is the lookup key used.
For your example
In order to find all title values in your JSON array, use the following query:
SELECT id, thing_name, JSON_EXTRACT(examples, '$**.title') as examples_titles FROM thing
id
thing_name
examples_titles
0
fruit
["Apple", "Orange", "Banana"]
1
car
["VW", "Bentley", "Tesla"]
I have datatype JSON stored on postgress db, e g. column:jsondata.
Each row on db table represent one JSON data of the same format,e.q.
{
"ID": "001",
"Name": "Britney",
"DebtAmount": "100.23"
}
There are multiple records with the above data with different ID.
How do I do JSON query to get the total sum of Debt Amount accross multiple records?
Thanks alot for your help.
If there is already existing solution, please point me to it.
You can use json_each_text() function in order to extract key-value pairs from the JSON object after fixing the current value in order to get a valid JSON value, and apply a SUM() aggregation through filtering out the key names DebtAmount along with applying float convertion for the values such as
SELECT SUM(value::float)
FROM t,
LATERAL json_each_text(js)
WHERE key = 'DebtAmount'
Demo
With MySQL 5.7 new features involving JSON has emerged. Among these features is the ability to query the fields in the JSON object as it is stored in the database.
My object looks like this.
{
"color": [
{"WHITE" :{ "size": [
{"S": [{"Price" : "31"},
{"discountPrice" : "13" }]}
]}},
{"BLACK" :{ "size": [
{"S": "69"},
{"M": "31"},
{"L": "55.666"}
]}}
]}
I want to query this as if it was regular tabular data, to this end I tried the following query to no avail.
select json_extract(Sku, '$.color[0]') from CRAWL.DAILYDATA;
I want to explode this into a format that looks more like a traditional RDBMS.
Any ideas?
In order to get data out of a json object as values, you need to get all the way down to the values. For instance, if you wanted to pull all of the values like they are regular RDBMS columns:
select json_extract(Sku, '$.color[0].WHITE.size[0].S[0].price') as price,
json_extract(Sku, '$.color[0].WHITE.size[0].S[0].discountPrice') as discountPrice
from CRAWL.DAILYDATA;
Of course, you need to know exactly what you're looking for in the object. This is the price of having a schema-less object like json. In principle, you could define a mysql function that would use combinations of
json_contains_path
and
json_extract
to make sure the path you are looking for exists, and otherwise it returns null. Personally though, if you want the RDBMS quality, why not just force it into a form where you can put the values directly into mysql tables? This is, of course, why RDBMS's exist. If you can't put it into such a form, you're going to be stuck with searching your json as above.
DBMS: MySQL 5.6
I have a table tbl of which column json stores JSON-like text, the type of column is text. The column json looks like
{"id": "123", "name": "foo", "age": "20"}
I tried to select rows with the condition json.id = '123'. The query select * from tbl where json like '%"id": "123"%' failed.
I found MySQL 5.6 not supporting Json functions. So how to use Json in the WHERE clause?
Append
The schema that storing a JSON in a single column is definitely not so reasonable. But I cannot modify the schema since the business has run for a while. The version of MySQL is out of same concern. So I think a workaround is needed.
use LOCATE
Select * from tbl where
LOCATE('"id": "123"','{"id": "123", "name": "foo", "age": "20"}') >0
Try this, common_schema is used for json parsing,
SELECT json
FROM tbl
WHERE common_schema.extract_json_value(json ,'id')
LIKE "123%"
select * from tbl where json like '%\"id\": \"123\"%'
try this. I have escaped " character.
I want to save the ruby hash in the database like
Metrics table has name,values fields.
metrics.create("Registered", '{"Gender": "Male", "Age": 21}')
I want the query should run like this.
select count(*) from metrics where name=“Registered” and values.age > 20
As per best of my knowledge it will not work. But Is there any possibility to achieve this?
If the database used is postgresql, you can have a json column named values and then use json_extract_path_text(values, age) to extract 'age'
Reference for json_extract_path_text can be seen here