So, we know this one works when I want to select all ID's that are present in the inner sql statement
Select *
FROM TableA
WHERE Column1 IN (SELECT column2 FROM tableB WHERE = condition)
What kind of syntax do I need to do if Column1 is a long string and I need to check if a certain substring exists.
Ex Column1 = "text text text text 12345" where 12345 is an ID that is present in the list of ID's given by the inner sql statement
Basically I'm trying to detect if an ID is present in one of strings from another table based on my list of ID's from another table.
Should I do this in SQL or let a serverside code do it?
This is usually done using the LIKE operator:
SELECT ... FROM ... WHERE Column1 LIKE "%12345%";
However this is extremely slow, since it is based on substring matching. To improve performance you have to create a search index table storing single words. Such index typically is maintained by trigger definitions: whenever an entry is changed the trigger also changes the set of words extracted into the search index table. Searching in such an index table is obviously fast and can be combined with the original table by means of a JOIN based on the n:1 relationship between words in the index to the original entries in your table.
Instead of using fieldname like '%needle%' search, which is extremely slow because it cannot utilise indexes, create a fulltext index on the given column and use fulltext search to find the matching substring.
Below code excerpt is quoted from the MySQL documentation:
CREATE TABLE articles (
id INT UNSIGNED AUTO_INCREMENT NOT NULL PRIMARY KEY,
title VARCHAR(200),
body TEXT,
FULLTEXT (title,body)
) ENGINE=InnoDB;
SELECT * FROM articles
WHERE MATCH (title,body)
AGAINST ('database' IN NATURAL LANGUAGE MODE);
The catch with syntax is that the list of words being looked for ('database' in the above code example) must be a string literal, it cannot be a subquery. You need to assemble the list of keywords in the application that calls the sql statement.
Related
So I have a SELECT Statement that is comparing the current column content from the table_1 column "table_1_content" with the content of another column (table_2_content) in table_2, whereas content in "table_2_content" can be found anywhere in "table_1_content":
$select = "SELECT * FROM table_1, table_2 WHERE `table_1_content` LIKE CONCAT('%', table_2_content, '%')";
$result = mysqli_query($con, $select);
My problem is that LIKE CONCAT is pretty performance heavy.
Is there another way to search through two columns from different tables, so that no full table scan is performed every time the query is executed?
The LIKE in total free text format (% at the start and at the end of the search string) is the performance heavy part. Is the wildcard at the start of the string necessary? If so: You might have to consider pre-processing the data in a different way so that the search can use a single wildcard or no wildcard at all. This last part (depending on the data) is for example done by splitting the string by a delimiter and storing the data in separate rows, after which a much faster comparison and indexes are possible to be used.
To put data in multiple rows, we would assume a usable separator (can be multiple, the code just gets longer):
CREATE TABLE baseinfo (id INT NOT NULL auto_increment primary key,
some other columns);
CREATE TABLE explodedstring(id INT NOT NULL, str VARCHAR(200),
FOREIGN KEY (id) REFERENCES baseinfo(id));
CREATE PROCEDURE explodestring(id int, fullstr VARCHAR(4000))
BEGIN
{many examples exist already how to do this on SO}
END;
The procedure would take as input your key from the original data (id in this case), and the original string.
The output of the procedure would end up in a secondary table explodedstring against which you now could run a normal select (add some index for performance). The resulting ids would tell you which record would match.
I have some trouble with a MySQL table and a fulltext index. My table structure looks similar to this:
CREATE TABLE example_index (
id int(11) NOT NULL auto_increment,
title tinytext,
content text,
FULLTEXT INDEX title (title),
FULLTEXT INDEX titlecontent (title,content),
PRIMARY KEY (id)
)ENGINE=MyISAM;
As you can see I have created a fulltext index for the fields title and title and content in combination. Furthermore I have stored some data into this table like this:
id | title | content
1 | Teamwork | Example data
2 | CSV-Export | Testdata
If I try to get some data from this table via a match against query then I got one result for the following query:
SELECT *
FROM example_index
WHERE MATCH (title) AGAINST('csv' IN BOOLEAN MODE);
BUT no results for this query:
SELECT *
FROM example_index
WHERE MATCH (title) AGAINST('Team' IN BOOLEAN MODE);
Can someone tell me, why I got no results here?
MySQL fulltext indexes index words. This is a very important limitation. You do not have the word team in your data, but you do have the word csv because - is considered a word delimiter.
If you are looking for words starting with team then you can still use the fulltext index using the * operator within the string to be searched:
SELECT *
FROM example_index
WHERE MATCH (title) AGAINST('Team*' IN BOOLEAN MODE);
If you would like to get records that contain the substring team anywhere within the indexed field, then MySQL fulltext index and fulltext search cannot help you. In this case you either need to revert to the good old title like '%team%' or you need to use a different fulltext search provider that can use MySQL.
Full text search looks for words in the text not partial words. Because your text contains "Teamwork", it does not contain team. This is a limitation of many full text implementations.
If your data is not too big, you can use like:
SELECT *
FROM example_index
WHERE title LIKE '%Team%';
If you are just looking for titles that start with "Team", you can do:
SELECT *
FROM example_index
WHERE title LIKE 'Team%';
The advantage of this approach is that it will take advantage of a regular index on example_index(title).
I notice that MySQL now has an n-gram parser. This parser is designed for non-English (primarily east Asian) languages. However, it might work on English as well. You might be able to try this to do what you want.
I want to search a string 'Trevor DSouza' in table without mentioning the column name which this string belongs. This string might me stored in different different column.so for search this string I wrote my query like:-
SELECT * FROM claim_master
WHERE MATCH (type,cheque) AGAINST('Trevor DSouza' IN NATURAL LANGUAGE MODE);
when I am executing this query it shows error like:-
Error Code: Can't find FULLTEXT index matching the column list
so to resolve this error I have created one FULLTEXT index on cheque as:-
ALTER TABLE claim_master ADD fulltext my_index(cheque);
so then after I am trying to execute my query:-
SELECT * FROM claim_master
WHERE MATCH(type,cheque) AGAINST('Trevor DSouza' IN NATURAL LANGUAGE MODE);
it again show me the same error as:-
Error Code: Can't find FULLTEXT index matching the column list
also at the one time how I can check a search string in all column. is there any way of passing the column name in match function like:- MATCH(full_name,mode,cheque,particular,desrption,final_status)?? and if there is a way of passing the columns at one time,then first we have to create a FULLTEXT index for that columns????? how to do this????
please help me to solve this.i am badly get stuck at this query
You need to have an fulltext index on the same columns as the MATCH(...) you are going to use.
ALTER TABLE claim_master ADD fulltext my_index(type,cheque);
i have a query which is using a like condition and it's killing my server
(this query is what i get frequently in the slow-query-log )
btw my table has about 120k rows - pleas ignor the syntax errors
select * from `images` where `category` like `%,3,%` and ( `tags` like '%,tag1,%' or `tags` like '%,tag2,%' or `tags` like '%,tag3,%' or `tags` like '%,tag4,%')
i don't want to change the query and database design fro now , so i've decided to switch to myisam and use fulltext index for tags column .
afterward server load hasn't change that much , mysql still using up to 90% of cpu (1 out of 8 of curse) from time to time .
so i was wondring , this fulltext indexing ... is it going to index the older data ( before adding this index ) ? cuz it happend very fast and my table is kinda big .
or it's only going to work on the newly stored data ?
Existing data was indexed, but as advised by Pyrce, a query with LIKE %[token]% is unable to leverage a fulltext index.
Rewrite your condition like this, this strictly equivalent to your inital query (ignoring stopwords and the likes):
WHERE MATCH(tags) AGAINST ('tag1 tag2 tag3 tag4' IN BOOLEAN MODE)
However you should rather focus on normalizing your structure. Storing non-scalar values (such as coma-separated values) in a field violates the very first normal form.
Create a new image_tag table and establish a 1-N relationship with images. A regular index will allow instant querying.
CREATE TABLE image_tags (
image_id INT,
tag VARCHAR(50),
PRIMARY KEY (image_id, tag),
FOREIGN KEY (image_id) REFERENCES images(id), -- replace with "images" table's actual primary key
KEY(tag, image_id) -- this index may be superfluous, check EXPLAIN
);
SELECT images.*
FROM images
JOIN image_tags ON image_id = images.id
WHERE tag IN ('tag1', 'tag2', 'tag3', 'tag4');
Repeat with images.category.
Full-text indexing usually only helps with prefix matching on tokens. In other words, all non-alphanumeric plus underscore separated words (anything other than A-Z, 0-9, or _ separates a word -- see here) within each row for the tags column will be indexed for prefix matches. You then have to use MATCH (tags) AGAINST ('tag1') to match the full text index search. You can repeat these matches for each tag to get your full query. Doing an Explain query will tell you if the query builder is using your index once you get the match queries fully configured.
Unfortunately MySQL is rather limited in how you can alter the full text indexing/searching -- so you're mostly stuck with it's default search methods (there's a couple search modes for fulltext -- see docs).
What I would like to do is search for a specificity word in my table like so:
SELECT * FROM my_table WHERE MATCH (description) AGAINST ('test');
And this will return all the rows in my_table that contains the string 'test'.
The problem is this dose not always happen. Meaning it will work sometimes after I have recreated the table.
CREATE TABLE my_table
(date date,
time time,
name varchar(50),
description text,
FULLTEXT INDEX (description)) ENGINE=MYISAM;
Because it only happened sometimes it has led me to believe that there must be some problem with the indexing.
So I tried setting the ft_min_word_len to 1 that did nothing.
My MySQLVersion is 5.1.57-community.
Thank you for your help.
match and against is a natural language search. if your column contains 50% or more of what you pass in the against() parameter, it will be considered common and would not return anything. just use SELECT * FROM my_table WHERE DESCRIPTION LIKE '%test%' where the % wildcard depicts that there is a word or phrase or suffix/prefix before or after.
here read http://dev.mysql.com/doc/refman/5.5/en/fulltext-search.html to learn more