I have sort of cache column in mysql table column. Lets call it cacheCol It is structured as json.
cacheCol example
{
"23": {
"variationOption": "23",
"productCode": "322992-015",
"price": "150",
"qnt": ""
},
"25": {
"variationOption": "25",
"productCode": "322992-015",
"price": "150",
"qnt": "0"
},
"26": {
"variationOption": "26",
"productCode": "322992-015",
"price": "150",
"qnt": "7"
}
}
I want to select myslq row if specific json part qnt is > 0. In this example part with key 26 is only one to match.
I have worked out regex to check those values:
https://www.regextester.com/?fam=109762
But when i run my query
SELECT * FROM "tbl" WHERE ("cacheCol" REGEXP ('(?<=\"26":\{)[^\{]*"qnt":"[1-9]\d*"(?=.*\})'))
Error shows up:
Got error 'repetition-operator operand invalid' from regexp
Here is sqlfiddle to play around: http://www.sqlfiddle.com/#!9/30335a/1
(Updated sqlfiddle to cover more variations in DB)
Is there any way to work around my regex to be compatible with mysql.
Found some info about mysql 5.7 having json data type maybe where is way to get my desired result avoiding regex?
For Schema like this :
CREATE TABLE IF NOT EXISTS `tbl` (
`id` int(6) unsigned NOT NULL,
`cacheCol` TEXT NOT NULL,
PRIMARY KEY (`id`)
) DEFAULT CHARSET=utf8;
INSERT INTO `tbl` (`id`, `cacheCol`) VALUES
('1', '{"23":{"variationOption":"23","productCode":"322992-015","price":"150","qnt":""}}'),
('2', '{"25":{"variationOption":"25","productCode":"322992-015","price":"150","qnt":"0"}}'),
('3', '{"26":{"variationOption":"26","productCode":"322992-015","price":"150","qnt":"7"}}');
Use the following query :
SELECT *
FROM `tbl`
where cacheCol REGEXP '.?"qnt":"[1-9]\d*"';
Since the field is always going to be JSON format it is enough to search for the substring alone.
You can play around here
Also it is good idea to explore JSON datatype in Mysql if you would have many updates and reads to this object, if only for simple querying, then you can treat it as string and work with REGEX itself.
Hope it helps
You can use JSON functions like JSON_SEARCH() to find a value that is an exact match, but not a inequality match.
In MySQL 8.0, you can use the JSON_TABLE() function if your JSON were structured as an array, but not with the JSON as you have structured it as an object.
Really, if you need to use inequality expressions to search for specific fields within your data, you should not use JSON at all. You should store the data in normal rows and columns.
The more I see people misusing JSON in complex ways in MySQL, the more I am convinced it was a bad idea for MySQL to implement a JSON data type.
My rule for JSON in MySQL is: references to a JSON column anywhere but the SELECT-list are a code smell. You should use normal columns instead of JSON.
MySQL REGEXP does not support lookaheads, but you can try to achieve the same logic using something like this:
SELECT * FROM "tbl" WHERE ("cacheCol" REGEXP ('(\{[\"\:,\-a-zA-Z0-9]+\"qnt\"\:\"[1-9][0-9]*\"\})'))
Hope it helps.
Related
I was trying MySQL secondary indexing referring to MySQL Documentation, and weird thing happened.
Firstly, I created a table with small modification per the example in the document
create table jemp(
c JSON,
g VARCHAR(20) GENERATED ALWAYS AS (c->"$.name"),
INDEX i (g)
)
Secondly, I inserted values per the example in the document
INSERT INTO jemp (c) VALUES
('{"id": "1", "name": "Fred"}'), ('{"id": "2", "name": "Wilma"}'),
('{"id": "3", "name": "Barney"}'), ('{"id": "4", "name": "Betty"}');
And then, I tried to perform a fuzzy search with "like" and "wildcard". This doesn't work because index doesn't support prefix %, but it can get result.
select c->"$.name" as name from jemp where g like "%F%"
Here is the weird thing, I removed the prefix %, and index did work. However, I didn't get any results. Per my poor understanding of MySQL, this should work.
select c->"$.name" as name from jemp where g like "F%"
I would be so much appreciate if anyone could help me with it.
For your query to work, you want a generated column that extracts the name as text rather than JSON. That is, use ->> instead of ->:
g VARCHAR(20) GENERATED ALWAYS AS (c ->> '$.name')
Then: the index may help for both following conditions:
where g like 'F%'
where g = 'F'
Whether MySQL decides to use it or not is another story; basically the databases assesses whether using the index will be faster than a full scan. If it believes that the condition will match on a large number of rows, it will probably choose to full scan.
Note that I consistently use single quotes for string literals; although MySQL tolerates otherwise, this is what the SQL standard specifies. In some other databases, double quotes stand for identifiers (this also is compliant with the standard).
I have a column from which i want to extract everything before and after a string. I have the following entry:
[{"model": "test.question", "pk": 123456789, "fields": {"status": "graded"}}]
[{"model": "test.question", "pk": 123456789, "fields": {"status": "answered"}}]
I want to extract the substring after "status": {" and before }}]
SQL's LIKE keyword will let you use % as a wildcard. The rest is just straight text matching. So, you should be able to use something like
WHERE columnName LIKE 'status":{"%}}]
(replace columnName with your column, of course).
However, if you have structured data in a table, you might want to reconsider your options. MySQL has a JSON data type (see https://dev.mysql.com/doc/refman/5.7/en/json.html) which may let you query more directly and correctly - for example, the approach I've described above will break if somehow a status exists that includes the string }}].
If you want the substring itself, MySQL has a substring function, detailed at MySQL has a SUBSTRING function, detailed at https://dev.mysql.com/doc/refman/5.7/en/string-functions.html#function_substring , which you would use in your SELECT clause, probably using LOCATE to get the index to use in the substring. Without seeing your current SQL, it's tough to describe how that would have to be put together.
I am using mysql 5.6 and it will not be feasible for me to upgrade it to 5.7. I have a table which stores json as an attribute. Attaching screenshot for reference.
Here, the column policy_status contains status and values of different policies as json for each user.
How can I find the list of users, say, with appVersion' status as success and value = 1437.
I got a few references online but as I am new to stored procedures I am not able to reach a solution. I will appreciate any help. Thanks in advance.
It is not efficient at all but may can help you with ideas:
SELECT *
FROM data
WHERE
(LOCATE('"employmentType":["status":"success"]', policy_status) > 0
AND
LOCATE('"value": 1', policy_status) > 0);
Using the LOCATE function you can see whether the field contains your desired appVersion and value strings. See sqlfiddle demo here.
Where the simple test data:
CREATE TABLE data (
id INT UNSIGNED NOT NULL,
policy_status TEXT
);
INSERT INTO data (id, policy_status) VALUES
(1,'{"employmentType":["status":"success"], "value": 1}'),
(2,'{"employmentType":["status":"no"], "value": 1}'),
(3,'{"employmentType":["status":"no"], "value": 0}'),
(4,'{"employmentType":["status":"success"], "value": 0}'),
(5,'{"employmentType":["status":"no"], "value": 1}');
gives the result:
{"employmentType":["status":"success"], "value": 1}
Where both strings are found.
UPDATE:
Also if you can add FULLTEXT index for your policy_status column than you can use fulltext search in the WHERE clause:
...
WHERE
MATCH (policy_status) AGAINST ('+"employmentType:[status:success]" +"value: 1"' IN BOOLEAN MODE)
Note the + and " characters in AGAINST(...). They are special boolean full-text search operators. See here.
A leading or trailing plus sign indicates that this word must be
present in each row that is returned
and
A phrase that is enclosed within double quote (") characters matches
only rows that contain the phrase literally, as it was typed. The
full-text engine splits the phrase into words and performs a search in
the FULLTEXT index for the words. Nonword characters need not be
matched exactly.
If it is not an option in your case, you can use LIKE for matching the substrings:
...
WHERE
(policy_status LIKE '%"employmentType":["status":"success"]%'
AND
policy_status LIKE '%"value": 1%');
See sqlfiddle demo for both.
With MySQL 5.7 new features involving JSON has emerged. Among these features is the ability to query the fields in the JSON object as it is stored in the database.
My object looks like this.
{
"color": [
{"WHITE" :{ "size": [
{"S": [{"Price" : "31"},
{"discountPrice" : "13" }]}
]}},
{"BLACK" :{ "size": [
{"S": "69"},
{"M": "31"},
{"L": "55.666"}
]}}
]}
I want to query this as if it was regular tabular data, to this end I tried the following query to no avail.
select json_extract(Sku, '$.color[0]') from CRAWL.DAILYDATA;
I want to explode this into a format that looks more like a traditional RDBMS.
Any ideas?
In order to get data out of a json object as values, you need to get all the way down to the values. For instance, if you wanted to pull all of the values like they are regular RDBMS columns:
select json_extract(Sku, '$.color[0].WHITE.size[0].S[0].price') as price,
json_extract(Sku, '$.color[0].WHITE.size[0].S[0].discountPrice') as discountPrice
from CRAWL.DAILYDATA;
Of course, you need to know exactly what you're looking for in the object. This is the price of having a schema-less object like json. In principle, you could define a mysql function that would use combinations of
json_contains_path
and
json_extract
to make sure the path you are looking for exists, and otherwise it returns null. Personally though, if you want the RDBMS quality, why not just force it into a form where you can put the values directly into mysql tables? This is, of course, why RDBMS's exist. If you can't put it into such a form, you're going to be stuck with searching your json as above.
DBMS: MySQL 5.6
I have a table tbl of which column json stores JSON-like text, the type of column is text. The column json looks like
{"id": "123", "name": "foo", "age": "20"}
I tried to select rows with the condition json.id = '123'. The query select * from tbl where json like '%"id": "123"%' failed.
I found MySQL 5.6 not supporting Json functions. So how to use Json in the WHERE clause?
Append
The schema that storing a JSON in a single column is definitely not so reasonable. But I cannot modify the schema since the business has run for a while. The version of MySQL is out of same concern. So I think a workaround is needed.
use LOCATE
Select * from tbl where
LOCATE('"id": "123"','{"id": "123", "name": "foo", "age": "20"}') >0
Try this, common_schema is used for json parsing,
SELECT json
FROM tbl
WHERE common_schema.extract_json_value(json ,'id')
LIKE "123%"
select * from tbl where json like '%\"id\": \"123\"%'
try this. I have escaped " character.