I am attempting to extract from my json object
hits = [{“title”: “Facebook”,
“domain”: “facebook.com”},
{“title”: “Linkedin”,
“domain”: “linkedin.com”}]
When I use:
json_extract(hits,'$.title') as title,
nothing is returned. I would like the result to be: [Facebook, Linkedin].
However, when I extract by a scalar value, ex.:
json_extract_scalar(hits,'$[0].title') as title,
it works and Facebook is returned.
hits contains a lot of values, so I need to use json_extract in order to get all of them, so I can't do each scalar individually. Any suggestions to fix this would be greatly appreciated.
I get INVALID_FUNCTION_ARGUMENT: Invalid JSON path: '$.title' as an error for $.title (double stars). When I try unnest I get INVALID_FUNCTION_ARGUMENT: Cannot unnest type: varchar as an error and INVALID_FUNCTION_ARGUMENT: Cannot unnest type: json. I get SYNTAX_ERROR: line 26:19: Column '$.title' cannot be resolved when I try double quotes
Correct json path to exract all titles is $.[*].title (or $.*.title), though it is not supported by athena. One option is to cast your json to array of json and use transform on it:
WITH dataset AS (
SELECT * FROM (VALUES
(JSON '[{"title": "Facebook",
"domain": "facebook.com"},
{"title": "Linkedin",
"domain": "linkedin.com"}]')
) AS t (json_string))
SELECT transform(cast(json_string as ARRAY(JSON)), js -> json_extract_scalar(js, '$.title'))
FROM dataset
Output:
_col0
[Facebook, Linkedin]
Fits you have an array. So $.title doesn't exist see below
Second, you have not a valid json, is must have double quotes " like the example shows
SET #a := '[{
"title": "Facebook",
"domain": "facebook.com"
},
{
"title": "Linkedin",
"domain": "linkedin.com"
}
]'
SELECT json_extract(#a,'$[0]') as title
| title |
| :---------------------------------------------- |
| {"title": "Facebook", "domain": "facebook.com"} |
SELECT JSON_EXTRACT(#a, "$[0].title") AS 'from'
| from |
| :--------- |
| "Facebook" |
SELECT #a
| #a |
| :-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| [{<br> "title": "Facebook",<br> "domain": "facebook.com"<br> },<br> {<br><br> "title": "Linkedin",<br> "domain": "linkedin.com"<br> }<br>] |
db<>fiddle here
Related
Consider this table:
DROP TABLE IF EXISTS `example`;
CREATE TABLE `example` (
`id` int NOT NULL AUTO_INCREMENT,
`content` json NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
And these rows:
INSERT INTO example(content)
VALUES (
'[ { "date": "1617210148", "name": "John", "status": "0" },
{ "date": "1617210148", "name": "Jack", "status": "0" },
{ "date": "1617210148", "name": "Henry", "status": "0" }]'
);
I'd like to update the value of the key status where name = Jack to 1
The result would be:
{ "date": "1617210148", "name": "Jack", "status": "1" }
How can I do this using JSON_REPLACE() or JSON_SET() in a SQL query (I'm aiming for a partial update since I'm using MySQL 8.0.25)?
This is very awkward, nearly impossible with MySQL's JSON functions.
You can use JSON_REPLACE() or JSON_SET(), but both require that you know the path to the field you want to change. So in this case, we can see that the array element is $[1] but if you didn't know that, you couldn't use this solution.
mysql> select json_pretty(json_replace(content, '$[1].status', '1')) as j
from example\G
*************************** 1. row ***************************
j: [
{
"date": "1617210148",
"name": "John",
"status": "0"
},
{
"date": "1617210148",
"name": "Jack",
"status": "1"
},
{
"date": "1617210148",
"name": "Henry",
"status": "0"
}
]
The question like yours has come up before on Stack Overflow, for example JSON update single value in MySQL table. The solution in that case depends on you knowing which array element your pseudo-record exists in.
You can get the path to a JSON element with JSON_SEARCH(), but you can only search by value, not by a key/value pair. If "Jack" occurs in some other field, that would also be found.
mysql> select json_unquote(json_search(content, 'one', 'Jack')) as path from example;
+-----------+
| path |
+-----------+
| $[1].name |
+-----------+
To search for a key/value pair, you need to use JSON_TABLE() and that requires you upgrade to MySQL 8.0. And that doesn't tell you the path to the element, it only allows you to return a specific row out of the array.
mysql> select j.* from example cross join json_table(content, '$[*]' columns(
date int unsigned path '$.date',
name varchar(10) path '$.name',
status int path '$.status')
) as j where name = 'Jack';
+------------+------+--------+
| date | name | status |
+------------+------+--------+
| 1617210148 | Jack | 0 |
+------------+------+--------+
Here's a trick: You can extract the name field, and that turns into an array of those values:
mysql> select json_extract(content, '$[*].name') as a from example;
+---------------------------+
| a |
+---------------------------+
| ["John", "Jack", "Henry"] |
+---------------------------+
Then you can search that array to get the array position:
mysql> select json_search(json_extract(content, '$[*].name'), 'one', 'Jack') as root from example;
+--------+
| root |
+--------+
| "$[1]" |
+--------+
Some unquoting and adding .status and you can get the full path to the field you want to update:
mysql> select concat(json_unquote(json_search(json_extract(content, '$[*].name'), 'one', 'Jack')), '.status') as path from example;
+-------------+
| path |
+-------------+
| $[1].status |
+-------------+
Now use it in a JSON_SET() call:
mysql> select json_pretty(
json_set(content,
concat(json_unquote(json_search(json_extract(content, '$[*].name'), 'one', 'Jack')), '.status'),
'1')) as newcontent
from example\G
*************************** 1. row ***************************
newcontent: [
{
"date": "1617210148",
"name": "John",
"status": "0"
},
{
"date": "1617210148",
"name": "Jack",
"status": "1"
},
{
"date": "1617210148",
"name": "Henry",
"status": "0"
}
]
Using this in an UPDATE looks like this:
mysql> update example set content = json_set(content, concat(json_unquote(json_search(json_extract(content, '$[*].name'), 'one', 'Jack')), '.status'), '1');
That's a long way to go. Now compare how difficult that is to:
UPDATE content SET status = 1 WHERE name = 'Jack';
Storing data in a JSON document when you eventually want to use SQL expressions to search or update individual fields within the JSON document is a costly mistake. It increases the complexity of any code you write to do it, and the developer who needs to take over maintenance of your code after you have moved on to another project will curse your name.
How could I extract nested json using sqlite json-extract or other sqlite json command ?
Here I'd like to extract given_id
"invoices": [{
........
"items": [{
"given_id": "TBC0003B",
...
}
]
}
]
Thanks.
In SQLite you can use json_extract() as follows:
select json_extract(my_json_col, '$.invoices[0].items[0].given_id') my_given_id from mytable
This gives you the given_id attribute of the first element of the items array under first element of the invoices array.
Demo on DB Fiddle:
with mytable as (select '{
"invoices": [{
"items": [{ "given_id": "TBC0003B" }]
}]
}' my_json_col)
select json_extract(my_json_col, '$.invoices[0].items[0].given_id') my_given_id from mytable
| my_given_id |
| :---------- |
| TBC0003B |
I have a column in a mysql database called params, this contains json data. See sample below;
{
"menu_text": 1,
"menu-meta_description": "My Website",
"enable_page_title": "0",
"page_title_heading": "h2"
}
I only want to update the enable_page_title key to 1, for every record in the table. I need to leave all other json values intact.
How can I achieve this?
You can use JSON modification function JSON_SET():
select json_set(js, "$.enable_page_title", 1) new_js from t;
Demo on DB Fiddle:
with t as (
select '{
"menu_text": 1,
"menu-meta_description": "My Website",
"enable_page_title": "0",
"page_title_heading": "h2"
}' js
)
select json_pretty(json_set(js, "$.enable_page_title", 1)) new_js from t;
| new_js |
| -------------------------------------------- |
| {
"menu_text": 1,
"enable_page_title": 1,
"page_title_heading": "h2",
"menu-meta_description": "My Website"
} |
I have a problem querying a csv file with header like this:
ID, NOMBRE, APELLIDOS, PROVINCIA
12121212, MARIA, LIONZA, MADRID
12312312, JAIMITO, PEREZ, ALMERÍA
13131313, BRUNO, DIAZ, MALAGA
23423423, HARLEY, QUINN, BARCELONA
I am doing it over apache drill UI v1.8.
When a do the following query
SELECT * FROM dfs.'path_to_file/clientes.csv' it works perfectly and it returns a table like this:
But if I do a query specifying the columns names (the headers of the csv file) then some columns are empty for any reason I haven´t figured out yet and is driving me crazy
p.e. this query
SELECT ID, NOMBRE FROM dfs.'path_to_file/clientes.csv'
Return this
Also I edited the dfs plugin and added the property extractHeader to true
...
"csv": {
"type": "text",
"extensions": [
"csv"
],
"extractHeader": true,
"delimiter": ","
},
...
So what I'm doing wrong? why I can query by ID but not by the other fields (header names) like NOMBRE or PROVINCIA. Do you have any idea?
I believe the issue you are encountering is due to the spaces in the header fields.
To verify this hypothesis, try this query (Note the spaces and back ticks in the field names):
SELECT `ID `, `NOMBRE ` FROM <your file>
The easy fix is to remove the spaces in the header.
Seems like a bug.
Removed "extractHeader from the dfs plugin.
...
"csv": {
"type": "text",
"extensions": [
"csv"
],
"delimiter": ","
},
...
and tried with older approach
select columns[0] as id ,columns[1] as NOMBRE from `a.csv`;
Output:
+-----------+-----------+
| id | NOMBRE |
+-----------+-----------+
| ID | NOMBRE |
| 12121212 | MARIA |
| 12312312 | JAIMITO |
| 13131313 | BRUNO |
| 23423423 | HARLEY |
+-----------+-----------+
Works fine.
I have scenario to parse the json data which is of one column in table.
Issue is that below Response column as json generated by Datastore backup to BigQuery. It has '\'attached to every data.
Reponse": "[
{
\"questionId\":5121566669012992,
\"answereId\":0,
\"answeredText\":\"Summer\"
},{
\"questionId\":5166851730440192,
\"answereId\":0,
\"answeredText\":\"Barcelona\"
},{
\"questionId\":6304057064947712,
\"answereId\":0,
\"answeredText\":\"Kitesurf\"
}
]"
How do I parse the below to get value for questionId using BigQuery?
JSON_EXTRACT cannot return REPEATED field, it can only do one match - hence no support for *
you can get the first position using hardcoded indexes as
SELECT JSON_EXTRACT_SCALAR('[
{
\"questionId\":5121566669012992,
\"answereId\":0,
\"answeredText\":\"Summer\"
},{
\"questionId\":5166851730440192,
\"answereId\":0,
\"answeredText\":\"Barcelona\"
},{
\"questionId\":6304057064947712,
\"answereId\":0,
\"answeredText\":\"Kitesurf\"
}
]', '$[0].questionId') AS str;
This returns:
+-----+------------------+---+
| Row | str | |
+-----+------------------+---+
| 1 | 5121566669012992 | |
+-----+------------------+---+