I've looked through several similar topics on stackoverflow that is similar my question but I did not find anything that can help me yet. I have this SQL query:
SELECT * FROM twitter_result
WHERE LOWER(TweetComment) LIKE LOWER('%lebron james%')
AND LOWER(TweetComment) LIKE LOWER('%NBA%')
I want to search a TweetComment that contains the word "LeBron James" and "NBA" at a same time. But these two words need to stand alone by themselves. Like it should not return a tweet that contains #LeBron James and #NBA (or NBATalk)
For instance, it should return a tweet like this
LeBron James Donates $41 Million To Send 1,100 Kids To College, Becomes 6th Most Charitable Athlete NBA In World
where Lebron James and NBA stand alone (no # characters). I have the LOWER there to ignore the case sensitive. Any help is greatly appreciated. Thanks
Sorry I forgot to add, I am just using SQL in PHPMyAdmin
Although there are solutions using regular expressions, it is hard to propose one without knowing the database you are using.
Instead, you can remove the tags you don't want before doing the like:
WHERE REPLACE(LOWER(TweetComment), '#lebron james', '') LIKE LOWER('%lebron james%') AND
REAPLCE(LOWER(TweetComment), '#nba', '') LIKE LOWER('%NBA%')
If you plan to use a regexp use,
select * from twitter_result
where --ignore tweets that contain #lebron james and #nba
TweetComment not regexp '.*#lebron james.*|.*#nba.*'
--select only those tweets that contain lebron james AND nba
and TweetComment regexp '[[:<:]]lebron james[[:>:]]'
and TweetComment regexp '[[:<:]]nba[[:>:]]'
All the patterns being searched for, have to be stated explicitly as MySQL by default doesn't support lookarounds.
The above match is case insensitive by default. Use regexp binary if the search needs to be case sensitive. Add more search words as needed.
Sample fiddle
Related
I'm working on a query which I thought should be quite intuitive, but somehow I'm facing a bit of issues when implementing it. I guess what I'm trying to achieve is to match a string stored in MySQL DB without space and punctuation (other creative approaches are more than welcome). At the same time I would like the query to handle Unicode characters in diacritics insensitive fashion (so options like REGEXP are kinda out of luck). And the last condition is I'm on MySQL 5.5 with InnoDB engine, so full-text indexing is not supported (but I'm open to upgrade to 5.6/5.7 if it helps sorting this out).
Consider the scenario which the string Hello-World from John Doe is stored in DB. I would like to find it when given the search string HelloWorld or JohnDoe. To be more general, the string in DB can contain brackets, understores and any other punctuation (not limited to ASCII but can compromise for now), while the search string can be a combination of words with or without any separators in between. The closest I've gotten so far is to daisy chain the REPLACE function for a list of known punctuation, like below:
SELECT text FROM table WHERE REPLACE(REPLACE(text, '-', ''), ' ', '') LIKE '%JohnDoe%'
My questions are:
Is there a better way instead of using the daisy chain above?
If that's the only solution, how will the performance be impacted when I chain up hundred or more REPLACE functions?
Thanks in advance for your help.
I don't know how restrictive your searches must be, but you could try to strip out all non-alphanumeric characters from it, so that you end up with a string like "HelloWorldfromJohnDoe" that you match with instead.
Have a look at this answer: How to remove all non-alpha numeric characters from a string?
You might have to change it around a bit though to make it fir your purposes. I changed it from CHAR(32) to CHAR(255) to make sure I could get the column, but you might want to look into changing the function altogether to fit your data more precisely.
Then you something like this:
SELECT *
FROM testing
WHERE alphanum(test) LIKE CONCAT('%', alphanum('John Doe'), '%')
which should give you a hit.
Method 1
I would have another column on the schema containing an "hashed" version of the name, for example, let's say you have the user:
John Doe The Great
This name hashes to
johndoethegreat
The hash function is coded in such a way that all of the following strings:
John_Doe_THE_great
John Doe The GREAT
John.Doe.The.Great
johnDOE___theGreat
john Doe the great
___john____DOE____THE____great
hash to the same value
johndoethegreat
It's trivial to write such a function. This way you can get the user input, hash it and then compare it against the hash column in your database
Names like:
Jon Doe
John Doo
will not be found of course
Method 2
Use the FULLTEXT search feature built-in in MySQL, sort the results by score and pick the first non zero entry
http://blog.oneiroi.co.uk/mysql/php/mysql-full-text-search-with-percentage-scoring/
I am totally missing the point of your question. You appear to have the string:
Hello-World from John Doe
If you want to find this when the search string is JohnDoe or John Doe, then you only need to substitute spaces:
where replace(text, ' ') like concat('%', 'JohnDoe', '%')
If you want a string that contains both "John" and "Doe" in that order, then:
where replace(text, ' ') like concat('%', 'John%Doe', '%')
I fail to see why 100 nested replace()s would be needed.
I've got problem - I need to find every single phrase like AbC (small b, between two Capital letters).
For Example a statement:
Little John had a ProBlEm and need to know how to do tHiS.
I need to select ProBlEm and tHiS (you see, BlE and HiS, one small letter in between two capital).
How can I select this?
In MySQL you can use a binary (to ensure case sensitivity) regular expression to filter for those records that contain such a pattern:
WHERE my_column REGEXP BINARY '[[:upper:]][[:lower:]][[:upper:]]'
However, it is not so straightforward to extract the substrings which match such a pattern from within MySQL. One can use a UDF, e.g. lib_mysqludf_preg, but it's probably a task more suited to being performed within your application layer. In either case, regular expressions can again help to simplify this task.
Firstly you have split the String. Please refer this SO Question
and then search each retrive word like
substring(word,2) LIKE '[A-Z]' COLLATE latin1_general_cs
I have a mysql table with a list of keywords such as:
id | keywords
---+--------------------------------
1 | apple, oranges, pears
2 | peaches, pineapples, tangerines
I'm trying to figure out how to query this table using an input string of:
John liked to eat apples
Is there a mysql query type that can query a field with a sentence and return results (in my example, record #1)?
One way to do it could be to convert apple, oranges, pears to apple|oranges|pears and use RLIKE (ie regular expression) to match against it.
For example, 'John liked to eat apples' matches the regex 'apple|orange|pears'.
First, to convert 'apple, oranges, pears' to the regex form, replace all ', ' by '|' using REPLACE. Then use RLIKE to select the keyword entries that match:
SELECT *
FROM keywords_table
WHERE 'John liked to eat apples' RLIKE REPLACE(keywords,', ','|');
However this does depend on your comma-separation being consistent (i.e. if there is one row that looks like apples,oranges this won't work as the REPLACE replaces a comma followed by a space (as per your example rows).
I also don't think it'll scale up very well.
And, if you have a sentence like 'John liked to eat pineapples', it would match both of the rows above (as it does have 'apple' in it). You could then try to add word boundaries to the regex (i.e. WHERE $sentence RLIKE '[[:<:]](apple|oranges|pears)[[:>:]]'), but this would screw up matching when you have plurals ('apples' wouldn't match '[wordboundary]apple[wordboundary]').
Hopefully this isn't more abstract than what you need but maybe good way of doing it.
I haven't tested this but I think it would work. If you can use PHP you can use str_replace to turn the spaces into keyword LIKE '%apple%'
$sentence = "John liked to eat apples";
$sqlversion = str_replace(" ","%' OR Keyword like '%",$sentence );
$finalsql = "%".$sqlversion."%";
the above will echo:
%John%' OR Keyword like '%liked%' OR Keyword like '%to%' OR Keyword like '%eat%' OR Keyword like '%apples%
Then just combine with your SQl statement
SQL ="SELECT *
FROM keywords_table
WHERE Keyword like" . $finalsql;
Storing comma delimited data is... less than ideal.
If you broke up the string "John liked to eat apples" into individual words, you could use the FIND_IN_SET operator:
WHERE FIND_IN_SET('apple', t.keywords) > 0
The performance wouldn't be great - this operation is better suited to Full Text Search.
I'm not aware of any direct solution to that type of query. But Full Text Search is a possibility. If you have a full-text index on the field of interest then a search with OR between each word in the sentence (although I think the OR operator is implied) would find that record ... but it might also find more than you want too.
I really don't think what you are looking for is completely possible but you can look into Full Text Search or SOUNDEX. SOUNDEX, for example, can do something like:
WHERE SOUNDEX(sentence) = SOUNDEX('%'+keywords+'%');
I have never tried it in this context but you should and let me know how it works out.
I'm trying to use a regular expression to match a user entered search string to a title of an entry in my MySQL database.
For example I have the following rows in a table in my databse:
id title
1 IM2 - Article 3 Funky Business
2 IM2 - Article 4 There's no Business That's not Show Business
3 IM2 - There's no Business That's not Show Business
4 CO4 - Life's a business
When a user searches for "IM Article Business", the following query will be executed (spaces are replaced by "(.*)" using str_replace):
SELECT * FROM mytable WHERE title REGEXP 'IM(.*)Article(.*)Business'
This will return the first 2 rows.
Now, I want it to show the same results when a user uses the same words, but in another order, for example: "Business IM Article". The results MUST contain all words entered, only the order of how the words are entered shouldn't matter.
I couldn't figure out how to do it in any way and hoped regular expressions would be the answer. I've never used them before, so does anybody know how to do this?
Thanks,
Pascal
This isn't something regular expressions are great at. Fortunately, it's something SQL is pretty good at. (I'm going to not use mysql's regexp keyword, which I didn't even knew existed, and instead use the SQL standard "%" glob matching.)
select * from mytable where title like '%IM%' and title like '%Article%' and title like '%Business%'
Now title has to contain all three strings, but you haven't specified an order. Exactly what you want.
I am trying to match a list of motorcycle models to a series of ebay codes for listing motorcycles in ebay.
So we get a motorcycle model name that will be something like:
XL883C Sportster where the manufacturer is Harley Davidson
I have a list of ebay codes that look like this
MB-100-0 Other
MB-100-1 883
MB-100-2 1000
MB-100-3 1130
MB-100-4 1200
MB-100-5 1340
MB-100-6 1450
MB-100-7 Dyna
MB-100-8 Electra
MB-100-9 FLHR
MB-100-10 FLHT
MB-100-11 FLSTC
MB-100-12 FLSTR
MB-100-13 FXCW
MB-100-14 FXSTB
MB-100-15 Softail
MB-100-16 Sportster
MB-100-17 Touring
MB-100-18 VRSCAW
MB-100-19 VRSCD
MB-100-20 VRSCR
So I want to match the model name against the list above using a regExp pattern.
I have tried the following code:
SELECT modelID FROM tblEbayModelCodes WHERE
LOWER(makeName) = 'harley-davidson' AND fnmodel REGEXP '[883|1000|1130|1200|1340|1450|Dyna|Electra|FLHR|FLHT|FLSTC|FLSTR|FXCW|FXSTB|Softail|Sportster|Touring|VRSCAW|VRSCD|VRSCR].*' LIMIT 1
however when I run the query I would expect the code to match on either MB-100-1 for 883 or MB-100-16 for Sportster but when I run it the query returns MB-100-0 for Other.
I am guessing that I have the pattern incorrect, so can anybody suggest what I might need to do to correct this?
Many thanks
Graham
[chars] matches any of the characters 'c','h','a','r','s'
So by giving it such a long list, it will inevitably match just the first item (single character)
Try this instead
LOWER(makeName) = 'harley-davidson' AND fnmodel REGEXP '(883|1000|1130|1200|1340|1450|Dyna|Electra|FLHR|FLHT|FLSTC|FLSTR|FXCW|FXSTB|Softail|Sportster|Touring|VRSCAW|VRSCD|VRSCR).*' LIMIT 1
You might also consider not using REGEX and using FIND_IN_SET instead.
Not really fully tested, but it should be something like this:
REGEXP '^MB-[0-9]+-[0-9]+[[:space:]]+(883|1000|1130|1200|1340|1450|Dyna|Electra|FLHR|FLHT|FLSTC|FLSTR|FXCW|FXSTB|Softail|Sportster|Touring|VRSCAW|VRSCD|VRSCR)$'
In detail:
^MB- Starts with MB-
[0-9]+ One or more digits
- Dash
[0-9]+ One or more digits
[[:space:]]+ One or more white space
(883|1000|...)$ Ends with one of these
Here's the reference for the regexp dialect spoken by MySQL:
http://dev.mysql.com/doc/refman/5.1/en/regexp.html
Answer to comment:
If you want to match the Sportster row them remove all other conditions. And you may not even need regular expressions:
WHERE fnmodel LIKE '% Sportster'