I have a database with a JSON array in column extra_fields. I need to extract one value by id.
The JSONs looks something like this, though the number of objects is random for each row:
[
{"id":"1","value":"48768"},
{"id":"2","value" ["String","http:someurl"]},
{"id":"5","value":"somevalue"},
{"id":"6","value":""},
{"id":"8","value":"Op-Ed"},
{"id":"9","value":"8111,13498,15408"},
{"id":"10","value":"30"},
{"id":"11","value":"This is the target string"}
]
I can extract an array of ids with:
SELECT extra_fields->>"$[*].id" FROM esqt7_k2_items;
I can extract an array of values with:
SELECT extra_fields->>"$[*].value" FROM esqt7_k2_items;
I can extract a single value at the nth zero-ordered object position with:
SELECT extra_fields->>"$[2].value" FROM esqt7_k2_items;
But the problem is that there's a variable number of objects and I specifically need the value of object id = 11. JSON_EXTRACT doesn't seem to support filtering, as every variation of a JSONPATH seems to fail. $..[?(#.id=11)].value works as a JSONPATH, but this (and many variants) fails:
SELECT extra_fields->"$.[?(#.id=11)].value" FROM esqt7_k2_items; #FAILS
This is a migration project and the JSON stuff is just how Joomla chose to implement their extra fields. Kind of a pain if you ask me.
One option you can consider (be careful about performance issues):
mysql> SELECT VERSION();
+-----------+
| VERSION() |
+-----------+
| 5.7.19 |
+-----------+
1 row in set (0.00 sec)
mysql> DROP TABLE IF EXISTS `esqt7_k2_items`;
Query OK, 0 rows affected (0.00 sec)
mysql> CREATE TABLE IF NOT EXISTS `esqt7_k2_items` (
-> `id` BIGINT UNSIGNED NOT NULL AUTO_INCREMENT PRIMARY KEY,
-> `extra_fields` JSON NOT NULL
-> );
Query OK, 0 rows affected (0.00 sec)
mysql> INSERT INTO `esqt7_k2_items`
-> (`extra_fields`)
-> VALUES
-> ('
'> [
'> {"id": "1", "value": "48768"},
'> {"id": "2", "value": ["String","http:someurl"]},
'> {"id": "5", "value": "somevalue"},
'> {"id": "6", "value": ""},
'> {"id": "8", "value": "Op-Ed"},
'> {"id": "9", "value": "8111,13498,15408"},
'> {"id": "10", "value": "30"},
'> {"id": "11", "value": "This is the target string"}
'> ]
'> '),
-> ('
'> [
'> {"id": "1", "value": ""},
'> {"id": "9", "value": "ONE This is the target string"}
'> ]
'> '),
-> ('
'> [
'> {"id": "6", "value": ""},
'> {"id": "11", "value": "TWO This is the target string"}
'> ]
'> ');
Query OK, 3 rows affected (0.00 sec)
Records: 3 Duplicates: 0 Warnings: 0
mysql> SET #`search_id` := '11';
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT `der`.`id`, `der`.`value`
-> FROM (
-> SELECT
-> `id`,
-> JSON_UNQUOTE(
-> JSON_EXTRACT(`extra_fields`,
-> CONCAT((SELECT
-> JSON_UNQUOTE(
-> JSON_SEARCH(`extra_fields` ->> '$[*].id', 'one', #`search_id`)
-> )
-> FROM `esqt7_k2_items` `esqt7_k2_items_in`
-> WHERE `esqt7_k2_items_out`.`id` = `esqt7_k2_items_in`.`id`
-> ), ".value"
-> )
-> )
-> ) `value`
-> FROM `esqt7_k2_items` `esqt7_k2_items_out`
-> ) `der`
-> WHERE `der`.`value` IS NOT NULL;
+----+-------------------------------+
| id | value |
+----+-------------------------------+
| 1 | This is the target string |
| 3 | TWO This is the target string |
+----+-------------------------------+
2 rows in set (0.00 sec)
See db-fiddle.
Simpliest way i found to get
the value of object id = 11
its by finding the right "index" of the needed object, doing a bit of PHP.
Firstly we get all objects Ids, by your simple request:
> SELECT extra_fields->>"$[*].id" FROM esqt7_k2_items;
then in PHP
$rightIndex = array_search($ids, '11');
and then you get the right object by passing this $rightIndex :
SELECT extra_fields->>"$[$rightIndex].value" FROM esqt7_k2_items;
Simple as hell! The same thing you can do anywhere else, for example, when you need to UPDATE an object in array, or its value/values, etc..
That is, Cheers!
Related
Consider this table:
DROP TABLE IF EXISTS `example`;
CREATE TABLE `example` (
`id` int NOT NULL AUTO_INCREMENT,
`content` json NOT NULL,
PRIMARY KEY (`id`)
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb3;
And these rows:
INSERT INTO example(content)
VALUES (
'[ { "date": "1617210148", "name": "John", "status": "0" },
{ "date": "1617210148", "name": "Jack", "status": "0" },
{ "date": "1617210148", "name": "Henry", "status": "0" }]'
);
I'd like to update the value of the key status where name = Jack to 1
The result would be:
{ "date": "1617210148", "name": "Jack", "status": "1" }
How can I do this using JSON_REPLACE() or JSON_SET() in a SQL query (I'm aiming for a partial update since I'm using MySQL 8.0.25)?
This is very awkward, nearly impossible with MySQL's JSON functions.
You can use JSON_REPLACE() or JSON_SET(), but both require that you know the path to the field you want to change. So in this case, we can see that the array element is $[1] but if you didn't know that, you couldn't use this solution.
mysql> select json_pretty(json_replace(content, '$[1].status', '1')) as j
from example\G
*************************** 1. row ***************************
j: [
{
"date": "1617210148",
"name": "John",
"status": "0"
},
{
"date": "1617210148",
"name": "Jack",
"status": "1"
},
{
"date": "1617210148",
"name": "Henry",
"status": "0"
}
]
The question like yours has come up before on Stack Overflow, for example JSON update single value in MySQL table. The solution in that case depends on you knowing which array element your pseudo-record exists in.
You can get the path to a JSON element with JSON_SEARCH(), but you can only search by value, not by a key/value pair. If "Jack" occurs in some other field, that would also be found.
mysql> select json_unquote(json_search(content, 'one', 'Jack')) as path from example;
+-----------+
| path |
+-----------+
| $[1].name |
+-----------+
To search for a key/value pair, you need to use JSON_TABLE() and that requires you upgrade to MySQL 8.0. And that doesn't tell you the path to the element, it only allows you to return a specific row out of the array.
mysql> select j.* from example cross join json_table(content, '$[*]' columns(
date int unsigned path '$.date',
name varchar(10) path '$.name',
status int path '$.status')
) as j where name = 'Jack';
+------------+------+--------+
| date | name | status |
+------------+------+--------+
| 1617210148 | Jack | 0 |
+------------+------+--------+
Here's a trick: You can extract the name field, and that turns into an array of those values:
mysql> select json_extract(content, '$[*].name') as a from example;
+---------------------------+
| a |
+---------------------------+
| ["John", "Jack", "Henry"] |
+---------------------------+
Then you can search that array to get the array position:
mysql> select json_search(json_extract(content, '$[*].name'), 'one', 'Jack') as root from example;
+--------+
| root |
+--------+
| "$[1]" |
+--------+
Some unquoting and adding .status and you can get the full path to the field you want to update:
mysql> select concat(json_unquote(json_search(json_extract(content, '$[*].name'), 'one', 'Jack')), '.status') as path from example;
+-------------+
| path |
+-------------+
| $[1].status |
+-------------+
Now use it in a JSON_SET() call:
mysql> select json_pretty(
json_set(content,
concat(json_unquote(json_search(json_extract(content, '$[*].name'), 'one', 'Jack')), '.status'),
'1')) as newcontent
from example\G
*************************** 1. row ***************************
newcontent: [
{
"date": "1617210148",
"name": "John",
"status": "0"
},
{
"date": "1617210148",
"name": "Jack",
"status": "1"
},
{
"date": "1617210148",
"name": "Henry",
"status": "0"
}
]
Using this in an UPDATE looks like this:
mysql> update example set content = json_set(content, concat(json_unquote(json_search(json_extract(content, '$[*].name'), 'one', 'Jack')), '.status'), '1');
That's a long way to go. Now compare how difficult that is to:
UPDATE content SET status = 1 WHERE name = 'Jack';
Storing data in a JSON document when you eventually want to use SQL expressions to search or update individual fields within the JSON document is a costly mistake. It increases the complexity of any code you write to do it, and the developer who needs to take over maintenance of your code after you have moved on to another project will curse your name.
I have JSON like:
{"1": "6", "2": "10", "3": "12"}
And i would like to get LAST key and value using MySQL query to get output like:
3x12
3 is the last key, and 12 is the last key value...
Is there any MySQL query to do that? I know using reading whole MySQL filed value as posted above and then while loop and if key is last print its value and key...but if is possible in MySQL query to get this output?
I im using this php that reads MySQL field value and get last key and value...but i don't know how to do it in mysql:
$json = json_decode('{"1": "6", "2": "10", "3": "12"}', true);
$value = end($json);
$key = key($json);
echo 'KEY: '.$key.'...VALUE: '.$value;
You can try something like the following, adjust as necessary:
mysql> SELECT VERSION();
+-----------+
| VERSION() |
+-----------+
| 5.7.20 |
+-----------+
1 row in set (0.00 sec)
mysql> SET #`json` := '
'> {
'> "1": "6",
'> "2": "10",
'> "3": "12"
'> }
'> ';
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT
-> CONCAT(
-> JSON_UNQUOTE(#`key`),
-> 'x',
-> JSON_UNQUOTE(
-> JSON_EXTRACT(#`json`,
-> CONCAT('$.', #`key`)
-> )
-> )
-> ) `value`
-> FROM (
-> SELECT #`key` := JSON_EXTRACT(
-> JSON_KEYS(#`json`),
-> CONCAT('$[', JSON_LENGTH(#`json`) - 1, ']')
-> )
-> ) `init`;
+-------+
| value |
+-------+
| 3x12 |
+-------+
1 row in set (0.00 sec)
See db-fiddle.
Having this json:
{
f1: "abc",
f2: [
{id: 1, val:"a"},
{id: 2, val:"b"},
{id: 3, val:"c"}
],
f3: [
"a",
"b",
"c"
]
}
Update:
As an example:
SELECT JSON_SEARCH(
'{"f1": "abc", "f2": [{"id": "1", "val":"a"},{"id": "2", "2":"b"},{"id": 3, "val":"2"}], "f3": ["a","b","c"]}',
'all', '1', null, '$.f2[*].id');
returns the needed path for f2->id==1
I can then use
select json_set(
'{"f1": "abc", "f2": [{"id": "1", "val":"a"},{"id": "2", "2":"b"},{"id": 3, "val":"2"}], "f3": ["a","b","c"]}',
'$.f2[0].val', 'd');
to update the data.
but
SELECT JSON_SEARCH(
'{"f1": "abc", "f2": [{"id": 1, "val":"a"},{"id": "2", "2":"b"},{"id": 3, "val":"2"}], "f3": ["a","b","c"]}',
'all', '1', null, '$.f2[*].id');
won't find f2->id==1 . Neither does
SELECT JSON_SEARCH(
'{"f1": "abc", "f2": [{"id": 1, "val":"a"},{"id": "2", "2":"b"},{"id": 3, "val":"2"}], "f3": ["a","b","c"]}',
'all', 1, null, '$.f2[*].id');
=========
update 2:
I will just save the IDs as string... But right now i have onother problem:
SELECT JSON_SEARCH(
'{"mm": [{"id":"1","field":"test","value":33}]}',
'one', '1', null, '$.mm[*].id') as path;
is working
SELECT * FROM document_data where document_id=5;
update document_data set data=JSON_SET(data, '$.mm', json_array()) where document_id=5;
update document_data set data=JSON_ARRAY_APPEND(data, '$.mm', '{"id":"1","field":"test","value":33}') where document_id=5;
SELECT JSON_SEARCH(data, 'one', '1', null, '$.mm[*].id') as path from document_data where id='5';
is not working. Seems to be the quoting.
Can someone help?
==========
how can i update f2 where id==2 using JSON_SET?
Tried everything but i can't seem to figure out how to do it.
Many thanks
Rene
JSON_SEARCH, by design, seems to search only strings, see WL#7909: Server side JSON functions :: JSON_SEARCH.
One option, very unintuitive is to use something like (be careful with performance problems):
mysql> SELECT VERSION();
+-----------+
| VERSION() |
+-----------+
| 5.7.20 |
+-----------+
1 row in set (0.00 sec)
mysql> SET #`json` := '{
'> "f1": "abc",
'> "f2": [
'> {"id": 1, "val": "a"},
'> {"id": 2, "val": "b"},
'> {"id": 3, "val": "c"}
'> ],
'> "f3": ["a", "b", "c"]
'> }',
-> #`value` := 2,
-> #`base_path` := '$.f2';
Query OK, 0 rows affected (0.00 sec)
mysql> SELECT JSON_SEARCH(
-> REPLACE(
-> REPLACE(
-> REPLACE(
-> JSON_EXTRACT(#`json`, CONCAT(#`base_path`, '[*].id')),
-> ', ', '","'),
-> '[', '["'),
-> ']', '"]'),
-> 'one', #`value`) INTO #`path`;
Query OK, 1 row affected (0.00 sec)
mysql> SELECT #`path`;
+---------+
| #`path` |
+---------+
| "$[1]" |
+---------+
1 row in set (0.00 sec)
mysql> SELECT
-> CONCAT(
-> REPLACE(
-> JSON_UNQUOTE(#`path`),
-> '$',
-> #`base_path`
-> ),
-> '.val') INTO #`path`;
Query OK, 1 row affected (0.00 sec)
mysql> SELECT #`path`;
+-------------+
| #`path` |
+-------------+
| $.f2[1].val |
+-------------+
1 row in set (0.00 sec)
mysql> SELECT JSON_SET(#`json`, #`path`, 'd');
+-------------------------------------------------------------------------------------------------------------------+
| JSON_SET(#`json`, #`path`, 'd') |
+-------------------------------------------------------------------------------------------------------------------+
| {"f1": "abc", "f2": [{"id": 1, "val": "a"}, {"id": 2, "val": "d"}, {"id": 3, "val": "c"}], "f3": ["a", "b", "c"]} |
+-------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
See db-fiddle.
I have MySQL table consisting of 3 columns having type of (cookie vachar, userdata json, userprefernce json). I am trying to load the data from csv but it gives the following error :
Error:
Error Code: 3140 Invalid JSON text: Invalid value at position 0 in value for column userdata'
I have tried to validate the JSON it looks fine. Now I couldn't understand what is the problem
Sample Row:
ZwpBHCrWObHE61rSOpp9dkUfJ, '{"bodystyle": {"SUV/MUV": 2}, "budgetsegment": {"EP": 2}, "models": {"Grand Cherokee": 1, "XC90": 1}}', '{"bodystyle": "SUV/MUV", "budgetsegment": "EP", "models": "Grand Cherokee,XC90"}'
I can't reproduce the problem:
File: /path/to/file/data.csv
ZwpBHCrWObHE61rSOpp9dkUfJ, '{"bodystyle": {"SUV/MUV": 2}, "budgetsegment": {"EP": 2}, "models": {"Grand Cherokee": 1, "XC90": 1}}', '{"bodystyle": "SUV/MUV", "budgetsegment": "EP", "models": "Grand Cherokee,XC90"}'
MySQL Command Line:
mysql> \! lsb_release --description
Description: Ubuntu 16.04.2 LTS
mysql> SELECT VERSION();
+-----------+
| VERSION() |
+-----------+
| 5.7.19 |
+-----------+
1 row in set (0.00 sec)
mysql> DROP TABLE IF EXISTS `table`;
Query OK, 0 rows affected (0.00 sec)
mysql> CREATE TABLE IF NOT EXISTS `table` (
-> `cookie` VARCHAR(25) NOT NULL,
-> `userdata` JSON NOT NULL,
-> `userprefernce` JSON NOT NULL
-> );
Query OK, 0 rows affected (0.01 sec)
mysql> LOAD DATA LOCAL INFILE '/path/to/file/data.csv'
-> INTO TABLE `table`
-> FIELDS TERMINATED BY ', '
-> OPTIONALLY ENCLOSED BY '\''
-> LINES TERMINATED BY '\n';
Query OK, 1 row affected (0.00 sec)
Records: 1 Deleted: 0 Skipped: 0 Warnings: 0
mysql> SELECT
-> `cookie`,
-> `userdata`,
-> `userprefernce`
-> FROM
-> `table`\G
*************************** 1. row ***************************
cookie: ZwpBHCrWObHE61rSOpp9dkUfJ
userdata: {"models": {"XC90": 1, "Grand Cherokee": 1}, "bodystyle": {"SUV/MUV": 2}, "budgetsegment": {"EP": 2}}
userprefernce: {"models": "Grand Cherokee,XC90", "bodystyle": "SUV/MUV", "budgetsegment": "EP"}
1 row in set (0.00 sec)
Given the following JSON object stored in MariaDB/MySQL:
SET #j = '{
"thing": {
"sub_things": [
{
"attribute": [
{ "1": 40 },
{ "5": 25 },
{ "13": 35 }
]
},
{
"attribute": [
{ "2": 50 },
{ "7": 50 }
]
}
]
}
}'
How would I query this to return sub_things where one of the objects in the attribute array has a particular key e.g. where the key is 13 this should return the first sub_thing.
Thanks!
I'm not sure if in a single query you can achieve what you need.
The following stored procedure, perhaps, can give you some ideas (stored procedure works for MariaDB and for MySQL):
> DROP PROCEDURE IF EXISTS `JSON_query_based_on_key`;
Query OK, 0 rows affected (0.01 sec)
> DELIMITER //
> CREATE PROCEDURE `JSON_query_based_on_key`(
-> `json` TEXT,
-> `key` VARCHAR(5)
-> )
-> BEGIN
-> DECLARE `sub_things_current` INT
-> DEFAULT JSON_LENGTH(`json`, '$.thing.sub_things') - 1;
->
-> WHILE (`sub_things_current` > -1) DO
-> IF NOT JSON_CONTAINS_PATH(
-> `json`,
-> 'one',
-> CONCAT('$.thing.sub_things[', `sub_things_current`, '].attribute[*]."', `key`, '"')
-> ) THEN
-> SET `json` := JSON_REMOVE(
-> `json`,
-> CONCAT('$.thing.sub_things[', `sub_things_current`, ']'));
-> END IF;
-> SET `sub_things_current` := `sub_things_current` - 1;
-> END WHILE;
->
-> SELECT JSON_EXTRACT(`json`, '$.thing');
-> END//
Query OK, 0 rows affected (0.00 sec)
> DELIMITER ;
> CALL `JSON_query_based_on_key`('{
'> "thing": {
'> "sub_things": [
'> {
'> "attribute": [
'> { "1": 40 },
'> { "5": 25 },
'> { "13": 35 }
'> ]
'> },
'> {
'> "attribute": [
'> { "2": 50 },
'> { "7": 50 }
'> ]
'> }
'> ]
'> }
'> }', '13');
+---------------------------------------------------------------------+
| JSON_EXTRACT(`json`, '$.thing') |
+---------------------------------------------------------------------+
| {"sub_things": [{"attribute": [{"1": 40}, {"5": 25}, {"13": 35}]}]} |
+---------------------------------------------------------------------+
1 row in set (0.00 sec)
Query OK, 0 rows affected (0.00 sec)
Modify the code as you need it.
See db<>fiddle for MariaDB (10.2.6) and db-fiddle for MySQL (5.7.17).