Please help, I'm very confused about my situation.
I'm beginning to create a search function but I'm not sure the best way to go about it.
On the front-end users will be able to enter words in a text-field, then these will search the MYSQL database, something like below:
so they search for 'Adult' and every item_id (Primary Key) with 'Adult' in column 'name' is listed. Or they enter 'black' and every item_id with 'black' in 'colors' is listed. or they enter 'Adult Blue' and every item with either 'Adult' or Blue would come up. I'm sure you get the idea.
I've read up on multiple methods, but I can't figure out which is best:
Using a MANY TO ONE table: This seems like it would work, but there are over 500 unique items, so that would be thousands and thousands of rows. For item_id 1 I would have to make a row for 'Adult', a row for 'Denim', a row for 'Pants', a row for 'black', a row for 'red', a row for 'blue'. I might as well just hard code everything.
Using FIND_IN_SET: Is this going to work? Would I have to store the values with commas like Adult,Denim,Pants and also EXPLODE to separate the values? I was going to try this method but I keep reading that storing multiple values in a field is very bad practice.
Or are Regular Expressions what I'm looking for?
What is the best way to store the values, and what is the best way to retrieve them? I'm not asking for exact code, just the methods to use. Any advice is appreciated!
So, if we suppose that columns name and colors are the only columns we need to search through, I'd do the following (naive solution but will work fine if your DB doesn't have millions of rows and you don't have thousands of customers searching at once).
First, create a view
CREATE VIEW SearchHere AS
SELECT item_id, CONCAT(name, ' ', colors) AS FullDescription
FROM table
I don't know the name of the table in your screenshot, so I used table as its name.
Now, if a user searches for adult red pants you could issue a query
SELECT item_id
FROM SearchHere
WHERE FullDescription LIKE '%adult%'
AND FullDescription LIKE '%red%'
AND FullDescription LIKE '%pants%'
Of course, you'd need to generate the query on the fly but that's not an issue. You could play with using AND or OR and placing spaces in between the wildcrad symbol % and the search term. Probably you would also want to do the view in a more sophisticated way, e.g., do more tha just CONCAT.
A straightforward solution is to use
name REGEXP 'the|search|terms'
OR colors REGEXP 'the|search|terms'
You should explain what you mean by best, though -- fastest performance? easiest to maintain? other?
Related
I'm working on something that shows shops under a specific category, however I have an issue because I store the categories of a shop like this in a record with the id of a category. "1,5,12". Now, the problem is if I want to show shops with category 2, it "mistakens" 12 as category 2. This is the SQL right now.
SELECT * FROM shops WHERE shop_cats LIKE '%".$sqlid."%' LIMIT 8
Is there a way to split the record "shop_cats" by a comma in SQL, so it checks the full number? The only way I can think of is to get all the shops, and do it with PHP, but I don't like that as it will take too many resources.
This is a really, really bad way to store categories, for many reasons:
You are storing numbers as strings.
You cannot declare proper foreign key relationships.
A (normal) column in a table should have only one value.
SQL has poor string functions.
The resulting queries cannot take advantage of indexes.
The proper way to store this information in a database is using a junction table, with one row per shop and per category.
Sometimes, we are stuck with other people's really bad design decisions. If this is your case, then you can use FIND_IN_SET():
WHERE FIND_IN_SET($sqlid, shop_cats) > 0
But you should really fix the data structure.
If you can, the correct solution should be to normalize the table, i.e. have a separate row per category, not with commas.
If you can't, this should do the work:
SELECT * FROM shops WHERE CONCAT(',' , shop_cats , ',') LIKE '%,".$sqlid.",%' LIMIT 8
The table shops does not follow 1NF (1st Normal Form) i.e; every column should exactly one value. To avoid that you need to create another table called pivot table which relates two tables (or entities). But to answer your question, the below SQL query should do the trick.
SELECT * FROM shops WHERE concat(',',shop_cats,',') LIKE '%,".$sqlid.",%' LIMIT 8
The scenario is this: in a table A, I have one column "tags", which is varchar(255).
In this column I store numbers, separated by commas, like this:
2,14,31,33,56
etc. there can be none, one, or several.
and I need to make a SELECT query that will return rows that have a certain number in this field. right now I'm using this method (don't be alarmed, I know its a poor way.. that's why I'm asking for help!). for example, let's assume the number I want to check is 33. the query is:
SELECT * FROM table_a WHERE
tags LIKE "%,33,%" OR tags LIKE "33,%" OR tags LIKE "%,33" OR tags LIKE "33"
I'm no expert but I know this can't be the method. The first question that comes to mind is: is there a command similar to IN() but that works the other way around?
I mean, can I tell it "find rows where 'tags' contains value 33" ?
When asking this question, I can see that there may be another field type other than varchar(255) to contain this type of data (an array of numbers, after all)
Is there a GOOD and efficient way of doing this? my method works for small tables, yes, but if the table grows.. (say, 10k rows, 50k, 300k ... ) this is obviously a problem.
The function that you want is find_in_set():
SELECT *
FROM table_a
WHERE find_in_set(33, tags) > 0;
You can simplify your like statement to be:
SELECT *
FROM table_a
WHERE concat(',', tags, ',') LIKE '%,33,%';
Neither of these can make use of an index. Having a separate table with one row per entity and per tag is the right way to go (but I think you know this already).
I have column in table where i store tag ids as 1|5|10
I want to explode the column using mysql query
What you probably might want to do is not store your tag ids like this. Make a separate table for them and your problem will dissapear naturally.
That's called normalisation.
You can chain SUBSTRING_INDEX to accomplish an explode effect, but let me remind you this is a horrible solution over database normalisation, as suggested by Kos.
If you were trying to select all MySQL entries with a 5 in your middle column (#|5|#) you would use this query:
SELECT * FROM table WHERE SUBSTRING_INDEX(SUBSTRING_INDEX(table.field, '|', 2), '|', -1) = 5
#Everyone: this is a horrible solution so please don't PLAN to use it; however, some times we're in a situation where a quick one liner like this must be used until a finer inspection on the database schema can happen.
I have the following problem:
I have a feed capturer that captures news from different sources every half an hour.
I only insert entries that don't have their URLs already in the database (URL is used to see if the record is already in database).
Even with that, I get some repeated entries, because some sites report the same news (that usually are from a news source like Reuters). I could look for these repeated entries during insertion, but i think this would slow the insertion time even more.
So, I can later find these repeated entries by the title. But I think this search is slow. Then, my idea is to generate a numeric field from the title and then search by this number for repeated titles.
What kind of encoding could I use (I thought in something reverse to base64) to encode the titles?
I'm suposing that searching for repeated numbers is a lot faster than searching for repeated words. Is that true or not?
Do you suggest a better solution for this problem?
Well, I don't care to have the repeated entries in the database, I just don't want to show then to the user. Like google, that filters the repeated results, but shows then if you want.
I hope I explained It well. Thanks in advance.
Fill the MD5 hash of the URL and title and build a UNIQUE index on it:
CREATE UNIQUE INDEX ux_mytable_title_url ON (title_hash, url_hash)
INSERT
INTO mytable (url, title, url_hash, title_hash)
VALUES ('url', 'title', MD5('url'), MD5('title'))
To select like Google (one result per title), use this query:
SELECT *
FROM (
SELECT DISTINCT title_hash
FROM mytable
) md
JOIN mytable mo
ON mo.url_title = md.title_hash
AND mo.url_hash =
(
SELECT url_hash
FROM mytable mi
WHERE mi.title_hash = md.title_hash
ORDER BY
mi.title_hash, mi.url_hash
LIMIT 1
)
so you can use a new table containing only the encoded keys based on title and url, you have then to add a key on it to accelerate search. But i don't think that you can use an effecient algorytm to transform strings to numbers ..
for the encryption use
SELECT MD5(CONCAT('title', 'url'));
and before every insertion you test if the encoded concatenation of title and url exists on this table.
#Quassnoi can explain better than I, but I think there is no visible difference in performance if you use a VARCHAR/CHAR or INT in a index to use it later for GROUPing or other method to find the duplicates. That way you could use the solution proposed by him but use a normal INDEX instead of a UNIQUE index and keep the duplicates in the database, filtering out only when showing to users.
Is there a way that I can do a select as such
select * from attributes where product_id = 500
would return
id name description
1 wheel round and black
2 horn makes loud noise
3 window solid object you can see through
and the query
select * from attributes where product_id = 234
would return the same results as would any query to this table.
Now obviously I could just remove the where clause and go about my day. But this involves editing code that I don't really want to modify so i'm trying to fix this at the database level.
So is there a "magical" way to ignore what is in the where clause and return whatever I want using a view or something ?
Even if it was possible, I doubt it would work. Both of those WHERE clauses expect one thing to be returned, therefore the code would probably just use the first row returned, not all of them.
It would also give the database a behaviour that would make future developers pull their hair out trying to understand.
Do it properly and fix the code.
or you could pass "product_id" instead of an integer, if there's no code checking for that...so the query would become:
select * from attributes where product_id = product_id;
this would give you every row in the table.
If you can't edit the query, maybe you can append to it? You could stick
OR 1=1
on the end.
You may be able to use result set metadata to get what you want, but a result set won't have descriptions of fields. The specific API to get result set metadata from a prepared query varies by programming language, and you haven't said what language you're using.
You can query the INFORMATION_SCHEMA for the products table.
SELECT ordinal_position, column_name, column_comment
FROM INFORMATION_SCHEMA.columns
WHERE table_name = 'products' AND schema_name = 'mydatabase';
You can restructure the database into an Entity-Attribute-Value design, but that's a much more ambitious change than fixing your code.
Or you can abandon SQL databases altogether, and use a semantic data store like RDF, which allows you to query metadata of an entity in the same way you query data.
As far out as this idea seems I'm always interested in crazy ways to do things.
I think the best solution I could come up with is to use a view that uses the products table to get all the products then the attributes table to get the attributes, so every possible product is accounted for and all will get the same result