I have a column from which i want to extract everything before and after a string. I have the following entry:
[{"model": "test.question", "pk": 123456789, "fields": {"status": "graded"}}]
[{"model": "test.question", "pk": 123456789, "fields": {"status": "answered"}}]
I want to extract the substring after "status": {" and before }}]
SQL's LIKE keyword will let you use % as a wildcard. The rest is just straight text matching. So, you should be able to use something like
WHERE columnName LIKE 'status":{"%}}]
(replace columnName with your column, of course).
However, if you have structured data in a table, you might want to reconsider your options. MySQL has a JSON data type (see https://dev.mysql.com/doc/refman/5.7/en/json.html) which may let you query more directly and correctly - for example, the approach I've described above will break if somehow a status exists that includes the string }}].
If you want the substring itself, MySQL has a substring function, detailed at MySQL has a SUBSTRING function, detailed at https://dev.mysql.com/doc/refman/5.7/en/string-functions.html#function_substring , which you would use in your SELECT clause, probably using LOCATE to get the index to use in the substring. Without seeing your current SQL, it's tough to describe how that would have to be put together.
Related
I have one column(varchar) containing only json string within one table. I want replace all keys with "" on that column. How can I do that using sql? My database is MySQL.
For example:
|--------------------------------------------------------------------|
| t_column |
|--------------------------------------------------------------------|
| {"name":"mike","email":"xxx#example.com","isManage":false,"age":22}|
|--------------------------------------------------------------------|
SELECT replace(t_column, regexp, "") FROM t_table
I expect:
mikexxx#example.comfalse22
How to write that regexp?
Start from
select t_column->'$.*' from test
This will return a JSON array of attribute values:
[22, "mike", "xxx#example.com", false]
This might be already all you need, and you can try something like
select *
from test
where t_column->'$.*' like '%mike%';
Unfortunately there seems to be no native way to join array values to a single string like JSON_ARRAY_CONCAT(). In MySQL 8.0 you can try REGEXP_REPLACE() and strip all JSON characters:
select regexp_replace(t_column->'$.*', '[" ,\\[\\]]', '') from test
which will return '22mikexxx#example.comfalse'.
If the values can contain one of those characters, they will also be removed.
Note: That isn't very reliable. But it's all I can do in a "simple" way.
See demo on db-fiddle.
I could be making it too simplistic, but this is just a mockup based on your comment. I can formalize it into a query if it fits your requirement.
Let's say you get your JSON string to this format where you replace all the double quotes and curly brackets and then add a comma at the end. After playing with replace and concat_ws, you are now left with:
name:mike,email:xxx#example.com,isManage:false,age:22,
With this format, every value is now preceded by a semicolon and followed by a comma, which is not true for the key. Let's say you now want to see if this JSON string has the value "mike" in it. This, you could achieve using
select * from your_table where json_col like '%:mike,%';
If you really want to solve the problem with your approach then the question becomes
What is the regex that selects all the undesired text from the string {"name":"mike","email":"xxx#example.com","isManage":false,"age":22} ?
Then the answer would be: {\"name\":\"|\"email\":\"|\",\"isManage\":|,\"age\":|}
But as others let you notice I would actually approach the problem parsing JSONs. Look up for functions json_value and json_query
Hope I helped
PS: Keep close attention on how I structured the bolded sentence. Any difference changes the problem.
EDIT:
If you want a more generic expression, something like select all the text that is not a value on a json-formatted string, you can use this one:
{|",|"\w+\":|"|,|}
I have sort of cache column in mysql table column. Lets call it cacheCol It is structured as json.
cacheCol example
{
"23": {
"variationOption": "23",
"productCode": "322992-015",
"price": "150",
"qnt": ""
},
"25": {
"variationOption": "25",
"productCode": "322992-015",
"price": "150",
"qnt": "0"
},
"26": {
"variationOption": "26",
"productCode": "322992-015",
"price": "150",
"qnt": "7"
}
}
I want to select myslq row if specific json part qnt is > 0. In this example part with key 26 is only one to match.
I have worked out regex to check those values:
https://www.regextester.com/?fam=109762
But when i run my query
SELECT * FROM "tbl" WHERE ("cacheCol" REGEXP ('(?<=\"26":\{)[^\{]*"qnt":"[1-9]\d*"(?=.*\})'))
Error shows up:
Got error 'repetition-operator operand invalid' from regexp
Here is sqlfiddle to play around: http://www.sqlfiddle.com/#!9/30335a/1
(Updated sqlfiddle to cover more variations in DB)
Is there any way to work around my regex to be compatible with mysql.
Found some info about mysql 5.7 having json data type maybe where is way to get my desired result avoiding regex?
For Schema like this :
CREATE TABLE IF NOT EXISTS `tbl` (
`id` int(6) unsigned NOT NULL,
`cacheCol` TEXT NOT NULL,
PRIMARY KEY (`id`)
) DEFAULT CHARSET=utf8;
INSERT INTO `tbl` (`id`, `cacheCol`) VALUES
('1', '{"23":{"variationOption":"23","productCode":"322992-015","price":"150","qnt":""}}'),
('2', '{"25":{"variationOption":"25","productCode":"322992-015","price":"150","qnt":"0"}}'),
('3', '{"26":{"variationOption":"26","productCode":"322992-015","price":"150","qnt":"7"}}');
Use the following query :
SELECT *
FROM `tbl`
where cacheCol REGEXP '.?"qnt":"[1-9]\d*"';
Since the field is always going to be JSON format it is enough to search for the substring alone.
You can play around here
Also it is good idea to explore JSON datatype in Mysql if you would have many updates and reads to this object, if only for simple querying, then you can treat it as string and work with REGEX itself.
Hope it helps
You can use JSON functions like JSON_SEARCH() to find a value that is an exact match, but not a inequality match.
In MySQL 8.0, you can use the JSON_TABLE() function if your JSON were structured as an array, but not with the JSON as you have structured it as an object.
Really, if you need to use inequality expressions to search for specific fields within your data, you should not use JSON at all. You should store the data in normal rows and columns.
The more I see people misusing JSON in complex ways in MySQL, the more I am convinced it was a bad idea for MySQL to implement a JSON data type.
My rule for JSON in MySQL is: references to a JSON column anywhere but the SELECT-list are a code smell. You should use normal columns instead of JSON.
MySQL REGEXP does not support lookaheads, but you can try to achieve the same logic using something like this:
SELECT * FROM "tbl" WHERE ("cacheCol" REGEXP ('(\{[\"\:,\-a-zA-Z0-9]+\"qnt\"\:\"[1-9][0-9]*\"\})'))
Hope it helps.
I have a table in MySQL where each row contains JSON returned from another system. The JSON will look something like:
[{"userId": "Dave"},{"userId": "Mary", "errorCode" : "DB Fail"}, {"userId": "Lorenza", "errorCode": "Web Error"}]
and I'm only interested in the members of the array containing an error code. In the future, these will be parsed into seperate rows of their own table, but in the meantime does MySql offer a way to extract only these with an errorCode?
I can use JSON_EXTRACT to extract the errorCodes only
JSON_EXTRACT(jsonData, '$[*].errorCode') AS errorCodes
but I really want the rest of the member (userId in the example above)
You could use the JSON_CONTAINS function to find the records with errorCode and then then use JSON_EXTRACT on those records. Put the JSON_CONTAINS in the where clause
I don't think you could do this with a single query without known boundaries of the number of elements, but you could use a stored procedure to run a loop.
e.g. each iteration runs LOCATE to find the position of "errorCode", and uses that location to run SUBSTR and/or SUBSTRING_INDEX to get the userid value and append it to another variable. The looped variable would just be the offset used in the LOCATE query.
I'am thinking about storing some data in postgres jsonb data type. There would be a structure like
{"name": "Jhon Smith", "emails": ["smith#test.com", "agent#matrix.net"],
"phones": ["123456789", "987654321"]}.
I know, that i can search this structure like
where contact->'emails' #> '"smith#test.com"'::jsonb;
But what I need is to search my data with some LIKE operator, so
where contact->'emails' <SOME_JSON_LIKE_OPERATOR> "smith"';
I can't find if psql have something similar, maybe it does not. So, maybe I can convert contact->'emails' field to Text ('["smith#test.com", "agent#matrix.net"]') and then use simple LIKE.. How would you have solved this problem?
You can expand the json array into a recordset of text and search that in whatever manner you like:
where exists (
select 1 from json_array_elements_text(contact->'emails')
where
value like "%smith%"
)
With MySQL 5.7 new features involving JSON has emerged. Among these features is the ability to query the fields in the JSON object as it is stored in the database.
My object looks like this.
{
"color": [
{"WHITE" :{ "size": [
{"S": [{"Price" : "31"},
{"discountPrice" : "13" }]}
]}},
{"BLACK" :{ "size": [
{"S": "69"},
{"M": "31"},
{"L": "55.666"}
]}}
]}
I want to query this as if it was regular tabular data, to this end I tried the following query to no avail.
select json_extract(Sku, '$.color[0]') from CRAWL.DAILYDATA;
I want to explode this into a format that looks more like a traditional RDBMS.
Any ideas?
In order to get data out of a json object as values, you need to get all the way down to the values. For instance, if you wanted to pull all of the values like they are regular RDBMS columns:
select json_extract(Sku, '$.color[0].WHITE.size[0].S[0].price') as price,
json_extract(Sku, '$.color[0].WHITE.size[0].S[0].discountPrice') as discountPrice
from CRAWL.DAILYDATA;
Of course, you need to know exactly what you're looking for in the object. This is the price of having a schema-less object like json. In principle, you could define a mysql function that would use combinations of
json_contains_path
and
json_extract
to make sure the path you are looking for exists, and otherwise it returns null. Personally though, if you want the RDBMS quality, why not just force it into a form where you can put the values directly into mysql tables? This is, of course, why RDBMS's exist. If you can't put it into such a form, you're going to be stuck with searching your json as above.