How to find specific value from MySQL json(or text) column? Row example:
'profile': [{'name':['John', 'Smith'], 'interest':['Sports', 'Games']}]
in above example, I want to search 'John' or 'Sports' without specifying key('name' or interests). MySQL official guide says you have to specify column name, but in my case it's preferred not to choose column name.
Another question is search performance if there are hundreds of thousands of rows in the table.
Use the JSON_SEARCH() function in MySQL 5.7 and later. It returns the path within the JSON document to the value you're searching for:
mysql> set #j = '{"profile": [{"name":["John", "Smith"], "interest":["Sports", "Games"]}]';
mysql> select json_search(#j, 'one', 'John') as path;
+------------------------+
| path |
+------------------------+
| "$.profile[0].name[0]" |
+------------------------+
But I recommend using normal columns instead of JSON if you need to search for values. In other words, you shouldn't ever reference a JSON column in the WHERE clause of a query.
In this case, you should have tables like this:
CREATE TABLE profile (
id SERIAL PRIMARY KEY,
first_name VARCHAR(24) NOT NULL,
last_name VARCHAR(24) NOT NULL
);
CREATE TABLE profile_interests (
id SERIAL PRIMARY KEY,
profile_id BIGINT UNSIGNED NOT NULL,
interest VARCHAR(24) NOT NULL,
FOREIGN KEY (profile_id) REFERENCES profile(id)
);
Then search using queries for the column you want.
P.S.: I fixed the quotes in your JSON document. You used single quotes like ' but you must use double quotes like " for it to be valid JSON format.
I found one solution - use fulltext search in MySQL that json string can be searched with keywords.
Related
I'm new to this website and using Mysql and phpMyAdmin. I need help with one of my table and I would really appreciate it. So, I created a table that has an Integer column I want to be able to limit it to only 7(Seven) digits I'm not quiet sure if this is possible using Mysql or phpMyAdmin.
I haven't tried any query on it. I want to limit the Integer type to only 7(Seven) digits.
This might not be the best possible solution but I think that if you were to store the integer as string in the format char(7) to limit the number of characters able to be entered it would get the job done.
I'm not familiar with Mysql in particular but here's some documentation on it : https://dev.mysql.com/doc/refman/8.0/en/char.html
I hope this helped.
In MySQL <8.0.16 You can't restrict the number of digits for an Integer. That has no meaning.
You can, however, use a DECIMAL type that allows you to specify the number of digits and the number of decimal places.
For example, DECIMAL(7,0) will define what you want.
Your CREATE statement becomes something like
CREATE TABLE IF NOT EXISTS myTable (
id INT AUTO_INCREMENT PRIMARY KEY,
someText VARCHAR(255) NOT NULL,
decimalValue DECIMAL(7,0)
) ;
If you're using MySQL 8.0.16 or later you can use a CHECK constraint to limit the value (as distinct from limiting the number of digits).
The example above becomes
CREATE TABLE IF NOT EXISTS myTable (
id INT AUTO_INCREMENT PRIMARY KEY,
someText VARCHAR(255) NOT NULL,
decimalValue INT,
CONSTRAINT `decValue_chk` CHECK (`decimalValue` <= 9999999))
) ;
The table DDL as flows:
CREATE TABLE `video` (
`short_id` varchar(50) NOT NULL,
`prob` float DEFAULT NULL,
`star_id` varchar(50) NOT NULL,
`qipu_id` int(11) NOT NULL,
`cloud_url` varchar(100) DEFAULT NULL,
`is_identical` tinyint(1) DEFAULT NULL,
`quality` varchar(1) DEFAULT NULL,
PRIMARY KEY (`short_id`),
KEY `ix_video_short_id` (`short_id`),
KEY `sid` (`star_id`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1
The video table has 4.5 million lines.
I execute the same query in mysql shell client as flows. except in where clause the star_id equal to a value with quatation mark, another not as flows.
select * from video where star_id="215343405";
12914 rows in set (0.22 sec)
select * from video where star_id=215343405;
12914 rows in set (3.17 sec)
the one with quatation mark is 10x faster then another(I have create index on star_id).i watch out the slow one does not use the index. I just wonder how mysql process the query?
mysql> explain select * from video where star_id=215343405;
Thanks advance!
This is answered in the manual:
For comparisons of a string column with a number, MySQL cannot use an
index on the column to look up the value quickly. If str_col is an
indexed string column, the index cannot be used when performing the
lookup in the following statement:
SELECT * FROM tbl_name WHERE str_col=1;
The reason for this is that there are many different strings that may convert to the value 1, such as '1', ' 1', or '1a'.
If you do not use Quotation marks mysql uses the value as an int and must convert the value for every record. Therefor the db needs a lot of time.
The quotes define the expression as a string, whereas without the single quote it is evaluated as a number. This means that MySQL is forced to perform a Type Conversion to convert the number to a CHAR to do a proper comparison.
As the doc above says,
For comparisons of a string column with a number, MySQL cannot use an
index on the column to look up the value quickly. If str_col is an
indexed string column, the index cannot be used when performing the
lookup...
However, the inverse of that is not true and while the index can be used, using a string as a value causes a poor execution plan (as illustrated by jkavalik's sqlfiddle) where using where is used instead of the faster using index condition. The main difference between the two is that the former requires a row lookup and the latter can get the data directly from the index.
You should definitely modify the column data type (assuming it truly is only meant to contain numbers) to the appropriate data type ASAP, but make sure that no queries are actually using single quotes, otherwise you'll be back where you started.
In one column of a database, we store the parameters that we used to hit an API, for example if the API call was sample.api/call?foo=1&bar=2&foobar=3 then the field will store foo=1&bar=2&foobar=3
It'd be easy enough to make a query to check 2 or 3 of those values if it was guaranteed that they'd be in that order, but that's not guaranteed. There's a possibility that call could have been made with the parameters as bar=2&foo=1&foobar=3 or any other combination.
Is there a way to make that query without saying:
SELECT * FROM table
WHERE value LIKE "%foo=1%"
AND value LIKE "%bar=2%"
AND value LIKE "%foobar=3%"
I've also tried
SELECT * FROM table
WHERE "foo=1" IN (value)
but that didn't yield any results at all.
Edit: I should have previously mentioned that I won't necessarily be always looking for the same parameters.
But why?
The problem with doing simple LIKE statements is this:
SELECT * FROM table
WHERE value LIKE "%foo=1%"
This will match the value asdffoo=1 and also foo=13. One hacky solution is to do this:
SELECT * FROM `api`
WHERE `params` REGEXP '(^|&)foo=1(&|$)'
AND `params` ...
Be aware, this does not use indexes. If you have a large dataset, this will need to do a row scan and be extremely slow!
Alternatively, if you can store your info in the database differently, you can utilize the FIND_IN_SET() function.
-- Store in DB as foo=1,bar=2,foobar=3
SELECT * FROM `api`
WHERE FIND_IN_SET(`params`, 'foo=1')
AND FIND_IN_SET(`params`, 'bar=2')
...
The only other solution would be to involve either another table, something like the following, and following the solution on this page:
CREATE TABLE `endpoints` (
`id` int(6) unsigned NOT NULL AUTO_INCREMENT,
`url` varchar(200) NOT NULL,
PRIMARY KEY (`id`)
) DEFAULT CHARSET=utf8;
CREATE TABLE IF NOT EXISTS `params` (
`id` int(6) unsigned NOT NULL AUTO_INCREMENT,
`endpoint` int(6) NOT NULL,
`param` varchar(200) NOT NULL,
PRIMARY KEY (`id`),
INDEX `idx_param` (`param`)
) DEFAULT CHARSET=utf8;
The last and final recommendation is to upgrade to 5.7, and utilize JSON functionality. Insert the data as a JSON object, and search it as demonstrated in this question.
This is completely impossible to do properly.
Problem 1. bar and foobar overlap
so if you search for bar=2, you will match on foobar=2. This is not what you want.
This can be fixed by prepending a leading & when storing the get query string.
Problem 2. you don't know how many characters are in the value. SO you must also have an end of string character. Which is the same & character. so you need it at the beginning and end.
You now see the issue.
even if you sort the parameters before storing it all to the database, you still cant do LIKE "%&bar=2&%&foo=1&%&foobar=3&%", because the first match can overlap the second.
even after the corrections, you still have to use three LIKES to match the overlapping strings.
I have around 200,000 rows in database table. When I execute my search query, it's taking around 4-5 seconds to give me results in next page. I want that execution should be fast and results should be loaded under 2 seconds. I have around 16 columns in my table.
Following is my query for creation of table
Create table xml(
PID int not null,
Percentdisc int not null,
name varchar(100) not null,
brand varchar(30) not null,
store varchar(30) not null,
price int not null,
category varchar(20) not null,
url1 varchar(300) not null,
emavail varchar(100) not null,
dtime varchar(100) not null,
stock varchar(30) not null,
description varchar(200) not null,
avail varchar(20) not null,
tags varchar(30) not null,
dprice int not null,
url2 varchar(300),
url3 varchar(300),
sid int primary key auto_increment);
Select query which I'm using
select * from feed where (name like '%Baby%' And NAME like '%Bassinet%')
I dont have much knowledge of indexing the database, to increase performance. Please guide me what index to use.
Indexes aren't going to help. LIKE is a non sargable operator. http://en.wikipedia.org/wiki/Sargable
The wildcard opeartor % used in starting of matching string renders any index created useless .
More are the characters before 1st wildcard operator , faster is the index lookup scan .
Anyways you can add an index to existing table
ALTER TABLE feed ADD INDEX (NAME);
This will have no index usage even after creating index on NAME column becuse it has a leading % character
select * from feed where (name like '%Baby%' And NAME like '%Bassinet%')
This will use indexing as starting % removed
select * from feed where (name like 'Baby%' And NAME like 'Bassinet%')
There's a good read here.
LIKE does not use the full text indexing. If you want to use full text searching you can use MySQL full text search functions, You can read MySQL doc regarding this.
Here's the syntax for adding INDEX in MySQL:
ALTER TABLE `feed`
ADD INDEX (`Name`);
MySQL Match example:
Substring matches: (Matches: Babylonian, Bassineete etc.)
SELECT * FROM `feed` WHERE MATCH (NAME) AGAINST ("+Baby* +Bassinett*" IN BOOLEAN MODE);
Exact matches:
SELECT * FROM `feed` WHERE MATCH (NAME) AGAINST ("+Baby +Bassinett" IN BOOLEAN MODE);
In your case index is not usefull. When we find with like operator it not use index. When we direct search i.e columnname = 'Ajay', at this time it search in index(if apply). The reason is index is searching with the physical data ,not with logical data(for like operator).
You can use Full-text search for this where you can define only those column in which you need to find data. FTS is usefull and get faster data when more data as you have.
How to enable FTS, please check the link.
http://blog.sqlauthority.com/2008/09/05/sql-server-creating-full-text-catalog-and-index/
I've got a mysql database with a table (InnoDB) of Games:
gamerooms
id: bigint(20) unsigned not null auto_increment
PRIMARY KEY (`id`)
I'd like to start generating a UUID value for each row which I can share publicly, something like:
gamerooms
id | id_public |
--------------------
1 | abcde
2 | ghijk
3 | lmnop
...
select * from gamerooms where id_public = ...
How do I add this new column, also keeping in mind that there are already records in the table? I'm confused because the column should be marked NOT NULL, but after adding the column, all records that already exist would have empty values.. Do I have to provide a default value?:
ALTER TABLE `gamerooms` ADD COLUMN `id_public` varchar(36) DEFAULT something AFTER `id`
I want to put an index on id_public of course after it's created, so not sure if null values after the column is first created will mess anything up.
Also, I can use varchar(36) with mysqls UUID() output, right?
Thank you
Your ALTER statement is correct:
ALTER TABLE `gamerooms`
ADD COLUMN `id_public` varchar(36) NOT NULL DEFAULT 'something' AFTER `id`
According to my MySQL Pocket Reference, if you don't provide a default value for a column that is defined as NOT NULL:
MySQL picks a value based on the type of the field
In this case, I'm guessing the default would be empty string. Once your column has been added, simply create a new index for the column, and rebuild the index using a null alteration instruction like so:
CREATE INDEX myIndex ON gamerooms(id_public);
ALTER TABLE gamerooms ENGINE = InnoDB;
You may be able to create the index at the same time you do the insert. My MySQL-fu isn't strong enough to know how to do that.
Should the existing records have a value once you create this new column? If yes, you could do this in multiple steps. First, create the new column without constraint or index and then back populate it with the UUID for all existing records. Once everything is populated, add the not null constraint and your indexes.
As a UUID is a 128-bit number, you don't need a varchar column to store it. a char(16) column would just be ok for saving a UUID binary data.
ALTER TABLE `gamerooms` ADD COLUMN `id_public` char(16) NOT NULL DEFAULT '' AFTER `id`