Extract data from json inside mysql field - mysql

I've got a a table with rows, and one of the rows has a field with data like this
{"name":"Richard","lastname":null,"city":"Olavarria","cityId":null}
And i want to select all the distinct "city" values i've got. Only using mysql server.
Is it possible? I'm trying with something like this
SELECT id FROM table_name WHERE field_name REGEXP '"key_name":"([^"]*)key_word([^"]*)"';
But i can't make the regexp work
Thanks in advance

MySQL has got support for JSON in version 5.7.7
http://mysqlserverteam.com/json-labs-release-native-json-data-type-and-binary-format/
You will be able to use the jsn_extract function to efficiently parse your JSON string.
If you have an older version and you want to solve it purely in mysql then I am afraid you have to treat it as a string and cut the value out of it (just normal string functions or use regular expressions)
This is not elegant but it will work
http://sqlfiddle.com/#!9/97cfd/14
SELECT
DISTINCT(substring(jsonfield, locate('"city":',jsonfield)+8,
locate('","', jsonfield, locate('"city":',jsonfield))-locate('"city":',jsonfield)-8)
)
FROM
ForgeRock

I have wrapped this into a stored function for those constrained to MySQL <5.7.7:
CREATE FUNCTION `json_extract_string`(
p_json text,
p_key text
) RETURNS varchar(40) CHARSET latin1
BEGIN
SET #pattern = CONCAT('"', p_key, '":"');
SET #start_i = LOCATE(#pattern, p_json) + CHAR_LENGTH(#pattern);
if #start_i = CHAR_LENGTH(#pattern) then
SET #end_i = 0;
else
SET #end_i = LOCATE('"', p_json, #start_i) - #start_i;
end if;
RETURN SUBSTR(p_json, #start_i, #end_i);
END
Note this only works with string values but is a bit more robust than #DmitryK's answer, in that it returns an empty string if the key is not found and the key can be anywhere in the JSON string.

Yes , you can definitely to it using JSON_EXTRACT() function in mysql.
lets take a table that contains JSON (table client_services here) :
+-----+-----------+--------------------------------------+
| id | client_id | service_values |
+-----+-----------+------------+-------------------------+
| 100 | 1000 | { "quota": 1,"data_transfer":160000} |
| 101 | 1000 | { "quota": 2,"data_transfer":800000} |
| 102 | 1000 | { "quota": 3,"data_transfer":70000} |
| 103 | 1001 | { "quota": 1,"data_transfer":97000} |
| 104 | 1001 | { "quota": 2,"data_transfer":1760} |
| 105 | 1002 | { "quota": 2,"data_transfer":1060} |
+-----+-----------+--------------------------------------+
To Select each JSON fields , run this query :
SELECT
id, client_id,
json_extract(service_values, '$.quota') AS quota,
json_extract(service_values, '$.data_transfer') AS data_transfer
FROM client_services;
So the output will be :
+-----+-----------+----------------------+
| id | client_id | quota | data_transfer|
+-----+-----------+----------------------+
| 100 | 1000 | 1 | 160000 |
| 101 | 1000 | 2 | 800000 |
| 102 | 1000 | 3 | 70000 |
| 103 | 1001 | 1 | 97000 |
| 104 | 1001 | 2 | 1760 |
| 105 | 1002 | 2 | 1060 |
+-----+-----------+----------------------+
NOW, if you want lets say DISTINCT quota , then run this query :
SELECT
distinct( JSON_EXTRACT(service_values, '$.quota')) AS quota
FROM client_services;
So this will result into your desired output :
+-------+
| quota |
+-------+
| 1 |
| 2 |
| 3 |
+-------+
hope this helps!

See MariaDB's Dynamic Columns.
Also, search this forum for [mysql] [json]; the topic has been discussed often.

This may be a little late, but the accepted answer didn't work for me. I used SUBSTRING_INDEX to achieve the desired result.
SELECT
ID, SUBSTRING_INDEX(SUBSTRING_INDEX(JSON, '"mykey" : "', -1), '",', 1) MYKEY
FROM MY_TABLE;
Hope this helps.

Related

MySQL - Search JSON data column

I have a MySQL database column that contains JSON array encoded strings. I would like to search the JSON array where the "Elapsed" value is greater than a particular number and return the corresponding TaskID value of the object the value was found. I have been attempting to use combinations of the JSON_SEARCH, JSON_CONTAINS, and JSON_EXTRACT functions but I am not getting the desired results.
[
{
"TaskID": "TAS00000012344",
"Elapsed": "25"
},
{
"TaskID": "TAS00000012345",
"Elapsed": "30"
},
{
"TaskID": "TAS00000012346",
"Elapsed": "35"
},
{
"TaskID": "TAS00000012347",
"Elapsed": "40"
}
]
Referencing the JSON above, if I search for "Elapsed" > "30" then 2 records would return
'TAS00000012346'
'TAS00000012347'
I am using MySQL version 5.7.11 and new to querying json data. Any help would be appreciated. thanks
With MySQL pre-8.0, there is no easy way to turn a JSON array to a recordset (ie, function JSON_TABLE() is not yet available).
So, one way or another, we need to manually iterate through the array to extract the relevant pieces of data (using JSON_EXTRACT()). Here is a solution that uses an inline query to generate a list of numbers ; another classic approchach is to use a number tables.
Assuming a table called mytable with a column called js holding the JSON content:
SELECT
JSON_EXTRACT(js, CONCAT('$[', n.idx, '].TaskID')) TaskID,
JSON_EXTRACT(js, CONCAT('$[', n.idx, '].Elapsed')) Elapsed
FROM mytable t
CROSS JOIN (
SELECT 0 idx
UNION ALL SELECT 1
UNION ALL SELECT 2
UNION ALL SELECT 3
) n
WHERE JSON_EXTRACT(js, CONCAT('$[', n.idx, '].Elapsed')) * 1.0 > 30
NB: in the WHERE clause, the * 1.0 operation is there to force the conversion to a number.
Demo on DB Fiddle with your sample data:
| TaskID | Elapsed |
| -------------- | ------- |
| TAS00000012346 | 35 |
| TAS00000012347 | 40 |
Yes , you can definitely to it using JSON_EXTRACT() function in mysql.
lets take a table that contains JSON (table client_services here) :
+-----+-----------+--------------------------------------+
| id | client_id | service_values |
+-----+-----------+------------+-------------------------+
| 100 | 1000 | { "quota": 1,"data_transfer":160000} |
| 101 | 1000 | { "quota": 2,"data_transfer":800000} |
| 102 | 1000 | { "quota": 3,"data_transfer":70000} |
| 103 | 1001 | { "quota": 1,"data_transfer":97000} |
| 104 | 1001 | { "quota": 2,"data_transfer":1760} |
| 105 | 1002 | { "quota": 2,"data_transfer":1060} |
+-----+-----------+--------------------------------------+
And now lets say we want client_id for all who have quota>1 , then use this query :
SELECT
id,client_id,
JSON_EXTRACT(service_values, '$.quota') AS quota
FROM client_services
WHERE JSON_EXTRACT(service_values, '$.quota') > 1;
And hence it will result into :
+-----+-----------+-------+
| id | client_id | quota |
+-----+-----------+--------
| 101 | 1000 | 2 |
| 102 | 1000 | 3 |
| 104 | 1001 | 2 |
| 105 | 1002 | 2 |
+-----+-----------+-------+
hope this helps!

group_concat does not show all the values mysql

ModelName.all(:having=>"count(receipt_no)>1",:select=>"school_id,group_concat(id SEPARATOR ',') as f_ids,receipt_no,count(distinct id) as id_count,count(receipt_no) as rec_count",:conditions=>"receipt_no is not null",:group=>"receipt_no")
Output is
+------------+-----------+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+
| receipt_no | school_id | id_count | f_ids | rec_count |
+------------+-----------+----------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-----------+
| 1261 | 1783 | 2 | 557660,557661 | 2 |
| 14/15- | 1783 | 1209 | 68352,77056,113664,56320,68353,77057,113665,56321,68354,56322,68355,81923,173571,113667,56323,68356,94980,56324,68357,56325,68358,80390,56326,68359,80391,110599,56327,80392,885... | 1209 |
| 15- | 1783 | 112 | 344067,344068,344069,344070,344075,326923,373261,373262,345882,360218,344091,361755,347685,341542,347689,360233,351530,358705,352829,324674,341576,324684,360018,368469,371541,3... | 112 |
Here group_concat does not show all the values but the count of items as same as the count receipt no. Suppose the items in the f_ids column is more than 200 character then its not showing all the values . In other case it will show correct value
I got the solution
SET SESSION group_concat_max_len = 1000000;
Run this code in MySQL console, then this code will change default group_concat character limit to 1000000 characters.
If you want to use in rails console,you can use in this following way
sql = "SET SESSION group_concat_max_len = 1000000"
ActiveRecord::Base.connection.execute(sql)
Please note:
This configuration will work only in that session

Replacing the last characters of a string value with the id of the table

I have got the columns [id, account_id] in my table. And the values are
+------+----------------+
| id | account_id |
+------+----------------+
| 1 | 01-01-02-0007 |
| 2 | 04-05-06-0001 |
| 3 | 03-07-09-0001 |
| 4 | 03-04-04-0001 |
| 5 | 03-04-08-0101 |
| ... |
| 201 | 03-04-08-0111 |
+------+----------------+
What I want is replace the last part of the each of account_id after - i.e. 0007, 0001 and 0001 etc in this case with respective id (but still padded with the 0s to the left to make it 4 characters). To be more specific, below is what I want to achieve:
+------+----------------+
| id | account_id |
+------+----------------+
| 1 | 01-01-02-0001 |
| 2 | 04-05-06-0002 |
| 3 | 03-07-09-0003 |
| 4 | 03-04-04-0004 |
| 5 | 03-04-08-0005 |
| .... |
| 201 | 03-04-08-0201 |
+------+----------------+
I thought to use REPLACE but unfortunately, that can't be applied to my case, since it is not just the part ( that remains same for each value) that I want to change. I have been searching, but I am unable to achieve this. I think, I would have to use some regular expression and LPAD in some way to achieve this, but not sure how.
Can anyone please show me some light?
For the very simple case with fixed account number lengths you're showing, this will do;
UPDATE accounts
SET account_id=CONCAT(LEFT(account_id,9), LPAD(id, 4, '0'))
An SQLfiddle to test with.
If account_id format would not be changing, you can use
substring_index, lpad, and concat to apply changes as you wanted.
update my_table
set account_id =
concat( substring_index( account_id, '-', 3 ), '-', lpad( id, 4, '0' ) )

mysql search query for 2 columns with single parameter

I am new to databases. In mysql database I have one table course. My question is: how to search all related words in both columns course_name and course_description and i need to get all the matched words in both columns? Can any one tell me the sql query for it? I have tried to write a query, but I am getting some syntax errors.
+----------+-----------+-----------------+------------+------------+
| courseId | cname | cdesc | sdate | edate |
+----------+-----------+-----------------+------------+------------+
| 301 | physics | science | 2013-01-03 | 2013-01-06 |
| 303 | chemistry | science | 2013-01-09 | 2013-01-09 |
| 402 | afm | finanace | 2013-01-18 | 2013-01-25 |
| 403 | English | language | 2013-01-17 | 2013-01-24 |
| 404 | Telugu | spoken language | 2013-01-10 | 2013-01-22 |
+----------+-----------+-----------------+------------+------------+
SELECT * from course WHERE cname='%searchtermhere%' AND cdesc='%searchtermhere%'
Adding the percent % makes the search within each value and not just beginning with.
If you want to search exact word
SELECT * FROM course WHERE cname ='word' AND cdesc = 'word'
OR you can also find each value and not just start from begining.
SELECT * FROM course WHERE cname = '".%searchtermhere%."' AND cdesc = '".%searchtermhere%."'
Since you say single parameter i guess. You will get either 'science' as input or 'physics' as input. Then you could simply use 'OR'.
select * from course where cname = (Input) or cdesc = (Input)

Getting most recent entries when using GROUP BY in RoR

I have a table organized in the following way:
id | userid | action | notes | created_at
-----------------------------------------
1 | 1 | foo | bar | datetime
2 | 33 | foo | bax | datetime
3 | 1 | foo | okay | datetime
4 | 3 | bam | bad | datetime
5 | 33 | foo | bom | datetime
What I would like to be able to do is, in Ruby on Rails, group the rows by userid but grab only the most recent entry for each group.
As it stands, I've gotten this far:
Thing.select("userid, notes").where(:action => "foo").order('`when` DESC')
Which will usually return something like:
userid | notes
--------------
1 | bar
33 | bax
When what I'm looking for is this:
userid | notes
--------------
1 | okay
33 | bom
I think I copy/pasted that all right...heh. Is there a way to achieve what I'm trying to do? Without resorting to searching inside my app itself? Thanks.
Update Attempted the first suggestion, no dice.
Just add .order("id desc") to the end of your line