The scenario is this: in a table A, I have one column "tags", which is varchar(255).
In this column I store numbers, separated by commas, like this:
2,14,31,33,56
etc. there can be none, one, or several.
and I need to make a SELECT query that will return rows that have a certain number in this field. right now I'm using this method (don't be alarmed, I know its a poor way.. that's why I'm asking for help!). for example, let's assume the number I want to check is 33. the query is:
SELECT * FROM table_a WHERE
tags LIKE "%,33,%" OR tags LIKE "33,%" OR tags LIKE "%,33" OR tags LIKE "33"
I'm no expert but I know this can't be the method. The first question that comes to mind is: is there a command similar to IN() but that works the other way around?
I mean, can I tell it "find rows where 'tags' contains value 33" ?
When asking this question, I can see that there may be another field type other than varchar(255) to contain this type of data (an array of numbers, after all)
Is there a GOOD and efficient way of doing this? my method works for small tables, yes, but if the table grows.. (say, 10k rows, 50k, 300k ... ) this is obviously a problem.
The function that you want is find_in_set():
SELECT *
FROM table_a
WHERE find_in_set(33, tags) > 0;
You can simplify your like statement to be:
SELECT *
FROM table_a
WHERE concat(',', tags, ',') LIKE '%,33,%';
Neither of these can make use of an index. Having a separate table with one row per entity and per tag is the right way to go (but I think you know this already).
Related
I'm trying to write a query to look-up rows which contain one or more of comma-separated values. Please see scenario below:
Table: Events, target row to lookup is called "courses"
The data in the courses column is a comma separated string of course codes.
ABC123,XYZ345,ABC987
ABC123
ABC123,ABC987
XYZ345,ABC123
Now what I want to do is to do a
Select * from events where courses in ("ABC123","XYZ345");
However this returns no results.
What I want is that the where lookup to get me all rows that CONTAINS ABC123 OR XYZ345.
Could someone please help me do this?
You could use like, the % are wildcards.
select *
from events
where courses like "%ABC123%" or courses like "%XYZ345%"
There are other approaches: https://dev.mysql.com/doc/refman/8.0/en/pattern-matching.html
It might also be worth normalising the data so that each value is stored in its own row in another table rather than in a comma separated string, as using like will start to hurt performance as the table grows.
#user3783243 mentions loose matching which is a good point and just one reason why storing data like this isn't the best approach.
One way around it would be to tweak the query above to something like:
select *
from events
where courses like "%ABC123,%" or courses like "%XYZ345,%"
but this poses another problem, are the comma separated values always split by a single comma (ABC123,XYZ345) or is there any whitespace (ABC123, XYZ345).
Another problem pointed out by #GarethD is that the previous approaches won't match the last value of the comma separated string (unless it does have a trailing comma). One way I can think of is to do something like this, but it starts making the query a bit clunky, and also assumes all values are 6 characters in length, at this point, it might be worth using a regular expression.
select *
from events
where courses like "%ABC123,%" or courses like "%XYZ345,%"
or right(courses, 6) = 'ABC123' or right(courses, 6) = 'XYZ345'
If all values are indeed six characters then it might be worth trying other mysql functions such as locate, substring and regexp to try and simplify the query.
Just use FIND_IN_SET()
SELECT columns FROM events WHERE FIND_IN_SET("ABC123", courses) OR FIND_IN_SET("XYZ345", courses)
Better of course would be a normalized table, since you can't use indexes with the current design.
I have a situation where I'm assembling a query based on user provided criteria and want to figure out what the most efficient way is to do this.
If I have a table that looks like this:
int id | varchar phone | varchar email | varchar RFID
and the user will pass in an array which defines the order (and items) with which they'd like to look up a user which *could look like this:
["id","email","phone"]
or it could look like this:
["email"]
or it could look like this:
["phone","rfid"]
or any other possible combination of those 4 fields.
Based on what I receive I need to look the user up in the order in which these fields arrived and if I find a match, I don't want to keep looking.
In other words if the input is
["email","rfid","phone"]
and I look into the db and find a user with the provided email, I don't want to keep looking to see if their rfid also matches, I just want to return said user.
However, if I don't find such an email, then I want to move on to the rfid.
So, in the various tests I've done (mostly playing with a case statement in the where clause) my results have been really terrible. Frequently taking almost a second to return a value, as opposed to taking <50ms when I simplify the where to search for the individual field.
I should note that all these fields are indexed.
So... my question is, should I just bite the bullet and make as many sql calls as there are items in the incoming array, or is there some really efficient way to structure a single query that will not bog down the system as my various attempts have.
I recognize that this may be too abstract a question, but am hoping that there's some mechanism for just such a use that I'm simply overlooking.
I don't think there's any good way to do a short-circuit in SQL. You can construct a WHERE clause that uses OR to combine the critiera, but doing this generally prevents it from using the indexes. You can use a UNION like this:
SELECT * FROM
(SELECT 1 precedence, table.*
FROM table
WHERE field1 = 'value'
UNION
SELECT 2 precedence, table.*
FROM table
WHERE field2 = 'value'
...
) x
ORDER BY precedence
LIMIT 1
where you replace field1, field2, etc. with the field names from the input array. This will produce the desired results in one query, but it will have to perform all the sub-queries, it won't short-circuit.
The best solution is probably to solve it in the application code. Loop through the fields in the input, and perform a query for just that field. When you get a result, break out of the loop and return it.
Please help, I'm very confused about my situation.
I'm beginning to create a search function but I'm not sure the best way to go about it.
On the front-end users will be able to enter words in a text-field, then these will search the MYSQL database, something like below:
so they search for 'Adult' and every item_id (Primary Key) with 'Adult' in column 'name' is listed. Or they enter 'black' and every item_id with 'black' in 'colors' is listed. or they enter 'Adult Blue' and every item with either 'Adult' or Blue would come up. I'm sure you get the idea.
I've read up on multiple methods, but I can't figure out which is best:
Using a MANY TO ONE table: This seems like it would work, but there are over 500 unique items, so that would be thousands and thousands of rows. For item_id 1 I would have to make a row for 'Adult', a row for 'Denim', a row for 'Pants', a row for 'black', a row for 'red', a row for 'blue'. I might as well just hard code everything.
Using FIND_IN_SET: Is this going to work? Would I have to store the values with commas like Adult,Denim,Pants and also EXPLODE to separate the values? I was going to try this method but I keep reading that storing multiple values in a field is very bad practice.
Or are Regular Expressions what I'm looking for?
What is the best way to store the values, and what is the best way to retrieve them? I'm not asking for exact code, just the methods to use. Any advice is appreciated!
So, if we suppose that columns name and colors are the only columns we need to search through, I'd do the following (naive solution but will work fine if your DB doesn't have millions of rows and you don't have thousands of customers searching at once).
First, create a view
CREATE VIEW SearchHere AS
SELECT item_id, CONCAT(name, ' ', colors) AS FullDescription
FROM table
I don't know the name of the table in your screenshot, so I used table as its name.
Now, if a user searches for adult red pants you could issue a query
SELECT item_id
FROM SearchHere
WHERE FullDescription LIKE '%adult%'
AND FullDescription LIKE '%red%'
AND FullDescription LIKE '%pants%'
Of course, you'd need to generate the query on the fly but that's not an issue. You could play with using AND or OR and placing spaces in between the wildcrad symbol % and the search term. Probably you would also want to do the view in a more sophisticated way, e.g., do more tha just CONCAT.
A straightforward solution is to use
name REGEXP 'the|search|terms'
OR colors REGEXP 'the|search|terms'
You should explain what you mean by best, though -- fastest performance? easiest to maintain? other?
I have a query on two columns from two different tables (connected by a left join in my query) and I want to order the search results by the occurence of the term I am looking for. I came up with this as my sorting variable in the statement, which might not be elegant, but works fine:
((LENGTH(table1.column)-LENGTH(REPLACE(lower(table1.column),lower('$term'),'')))/LENGTH('$term') AS sort_frequency
$term is my search term and at the end of the query I do this: ORDER BY sort_frequency DESC.
Now comes the difficulty: the calculation works fine for both tables separately, but when I want to connect the two by addition, the results of table2 always come in front of the results of table1 and nothing is ordered by occurence. My statement looks like this:
(((LENGTH(table1.column)-LENGTH(REPLACE(lower(table1.column),lower('$term'),'')))/LENGTH('$term')) + ((LENGTH(table2.column)-LENGTH(REPLACE(lower(table2.column),lower('$term'),'')))/LENGTH('$term'))) AS sort_frequency
I need this calculation, because the search results come from two different tables, but shall be ordered together on one page (let's say: one table is about images with certain keywords and the second table is about videos with certain keywords, once I searched for a specific keyword I don't care whether it is an image or video, I want the one that fits my keyword query most).
Do you have any idea why the calculation does not work? What is my mistake? I have tried adding/removing brackets, but that does not help.
Any help would be appreciated,
Have you tried using a UNION to combine the two table before searching for occurrences? My Suggestion is to use something along the lines of:
SELECT((LENGTH(newcolumn)-LENGTH(REPLACE(lower(newcolumn),lower('$term'),'')))/LENGTH('$term') AS sort_frequency FROM table1.column UNION table2.column AS newcolumn
I want to query everything, Like:
SELECT * FROM
but I want to exclude two columns because their not necessary, but there's too many columns I need to just type it all one by one. Is there an exclude keyword or except keyword or something in SQL or MYSQL?
No. It's better practice to type out all of the fields as opposed to SELECT * FROM ... anyway.
If you're going to be a programmer, you may as well get used to typing :)
Yes, you must type them all out.
Or, if there really are that many you can do it programatically.
SHOW COLUMNS FROM `tablename`
Will give you the list of columns. Use whatever language you are using to pull them out and build the query with them.
Of course, you have to make sure the column names are escaped (what if you have a column name called select?).