Alternative to running multiple DB queries - mysql

I’m developing a web app with a search functionality. Generally speaking, the user can search under specific categories or groups of categories. Example:
Mammal (group)
Cat (category)
Dog (category)
Mammal, Cat, and Dog are tables in the DB, and are represented by their own class in the source code. Common fields between Cat and Dog are stored in Mammal; both Cat and Dog have a set of unique fields. I’m trying to figure out the best way to execute a query (or queries) when a user searches under a group (rather than a specific category). For example, the user searches for “all mammals under the age of 4”. As part of the response, I want to return all the fields in the tables belonging to the Mammal category (Cat and Dog, in this case).
Given that the tables Cat and Dog have unique fields, it seems (according to my googling) that I would need to run multiple queries (one for each category). Is this, indeed, the case? If so, what is most efficient way of doing this? And if not, how would I run such a request with a single query?
Essentially, my question is this: What is the most efficient way of executing a query for the situation I’ve described above?
[EDIT]
DB example w/ queries:
https://www.db-fiddle.com/f/na9ctPmi6CjyDB4MNnjycb/3
In the example in the link above, there are two queries which, together, can get all the data for the user's search (described above). I'm wondering if there's a way to do this with a single query, or at least a single call to the DB.
So far, I've tried the naive approach with the multiple query calls. This works fine (insofar as there aren't any errors). My concern is that when the stored data accumulates to the several hundreds/thousands, this approach will become too slow. Furthermore, my current approach requires additional data processing in the source code. For example, if the user wants the top 5 results from the search, then I have to get the top 5 results from each table, create an aggregated collection of the DB results, sort the collection, and pick the top 5 results from the sorted collection to return to the user. I'm wondering if there is a way to accomplish all this on the DB side (assuming that it would be faster).

From your sqlfiddle:
select * from cat left join mammal on (cat.id = mammal.id) where age < 4
union all
select * from dog left join mammal on (dog.id = mammal.id) where age < 4
A few points:
You only want left join if, for example, you'd want to show lines for mammals that don't have corresponding dogs or cats. In your case, it doesn't seem like that's what you want, so it would be better to use an inner join (or simply join, which defaults to an inner one). That way, if there are no dogs, and only 2 cats, the results would only show 2 lines, instead of 2 cats + 1 non-existing dog.
Making one table per mammal type won't scale. What happens if your users now want to add coyotes? Or rabbits? Every single time a new mammal needs to be added to the system, you need to create a new table. The proper way to do this would be to create a join table that would detail:
A table animal_type, with lines something like this:
animal_type_id name
Then you'd have a separate table called attribute_type with something like this:
attribute_id name
A separate table called animals:
animal_id animal_type_id
And then finally you'd have a separate table called animal_attributes:
id animal_id attribute_id value
Now, you can simply add animals by inserting a row in animals, specifying the animal_type_id, which could be cat/dog/whatever. You simply need to have created the animal_type beforehand. And then, you add attributes to your animal by creating rows in animal_attributes, referencing the animal_id you just created, as well as the proper animal_attribute_id, which could be shared attributes like color, length, size, and unique cat/dog/animal fields.

Related

Modelling ownership in MySQL

I have a table Things and I want to add ownership relations to a table Users. I need to be able to quickly query the owners of a thing and the things a user owns. If I know that there will be at most 50 owners, and the pdf for the number of owners will probably look like this, should I rather
add 50 columns to the Things table, like CoOwner1Id, CoOwner2Id, …, CoOwner50Id, or
should I model this with a Ownerships table which has UserId and ThingId columns, or
would it better to create a table for each thing, for example Thing8321Owners with a row for each owner, or
perhaps a combination of these?
The second choice is the correct one; you should create an intermediate table between the table Things and the table Owners (that contains the details of each owner).
This table should have the thing_id and the owner_id as the primary key.
So finally, you well have 3 tables:
Things (the things details and data)
Owner (the owners details and data)
Ownerships (the assignment of each thing_id to an owner_id)
Because in a relational DB you should not have any redundant data.
You should definitely go with option 2 because what you are trying to model is a many to many relationship. (Many owners can relate to a thing. Many things can relate to an owner.) This is commonly accomplished using what I call a bridging table. (Which exactly what option 2 is.) It is a standard technique in a normalized database.
The other two options are going to give you nightmares trying to query or maintain.
With option 1 you'll need to join the User table to the Thing table on 50 columns to get all of your results. And what happens when you have a really popular thing that 51 people want to own?
Option 3 is even worse. The only way to easily query the data is to use dynamic sql or write a new query each time because you don't know which Thing*Owners table to join on until you know the ID value of the thing you're looking for. Or you're going to need to join the User table to every single Thing*Owners table. Adding a new thing means creating a whole new table. But at least a thing doesn't have a limit on the number of owners it could possibly have.
Now isn't this:
SELECT Users.Name, Things.Name
FROM Users
INNER JOIN Ownership ON Users.UserId=Ownership.UserId
INNER JOIN Things ON Things.ThingId=Ownership.ThingId
much easier than any of those other scenarios?

most efficient mysql query construction for multiple 0..N elements

I have the following problem (dumb example)
User -> hasMany(Trousers)
User -> hasMany(Shirts)
Is it possible in one single query to retrieve all User/Trouser User/Shirt couples ? I know it's possible with an union all but fields in Trousers and Shirts table differs.
The database structure seems odd but it cannot be modified as it'll imply too many code refactoring.

Table join--multiple rows to/ from one column (/ cell)

I have searched for a solution for this problem, but haven't found it (yet), probably because I don't quite know how to explain it properly myself. If it is posted somewhere already, please let me know.
What I have is three databases that are related to each other; main, pieces & groups. Basically, the main database contains the most elementary/ most used information from a post and the pieces database contains data that is associated with that post. The groups database contains all of the (long) names of the groups a post in the main database can be 'posted in'. A post can be posted in multiple groups simultaneously. When a new post is added to my site, I check the pieces too see if there are any duplicates (check if the post has been posted already). In order to make the search for duplicates more effective, I only check the pieces that are posted in the same group(s).
Hopefully you're still with me, cause here's where it starts to get really confusing I think (let me know if I need to specify things more clearly): right now, both the main and the pieces database contain the full name of the group(s) (basically I'm not using the groups database at all). What I want to do is replace the names of those groups with their associated IDs from the groups database. For example, I want to change this:
from:
MAIN_table:
id  |  group_posted_in
--------|---------------------------
1   | group_1, group_5
2   | group_15, group_75
3   | group_1, group_215
GROUPS_table:
id  |  group_name
--------|---------------------------
1   | group_1
2   | group_2
3   | group_3
etc...
into:
MAIN_table:
id  |  group_posted_in
--------|---------------------------
1   | 1,5
2   | 15,75
3   | 1,215
Or something similar to this. However, This format specifically causes issues as the following query will return all of the rows (from the example), instead of just the one I need:
SELECT * FROM main_table WHERE group = '5'
I either have to change the query to something like this:
...WHERE group = '5' OR group = '5,%' OR group = '%,5,%' OR group = '%,5'
Or I have to change the database structure from Comma Separated Values to something like this: [15][75]. The accompanying query would be simpler, but it somehow seems like a cumbersome solution to me. Additionally, (simple) joins will not be easy/ possible at all. It will always require me to run a separate query to fetch the names of the groups--whether a user searches for posts in a specific group (in which case, I first have to run a query to fetch the id's, then to search for the associated posts), or whether it is to display them (first the posts, then another query to match the groups).
So, in conclusion: I suppose I know there is a solution to this problem, but my gut tells me that it is not the right/ best way to do it. So, I suppose the question that ties this post together is:
What is the correct method to connect the group database to the others?
For a many-to-many relationship, you need to create a joining table. Rather than storing a list of groups in a single column, you should split that column out into multiple rows in a separate table. This will allow you to perform set based functions on them and will significantly speed up the database, as well as making it more robust and error proof.
Main
MainID ...
Group
GroupID GroupName
GroupsInMain
GroupsInMainID MainID(FK) GroupID(FK)
So, for MainID 1, you would have GroupsInMain records:
1,1,1
2,1,5
This associates groups 1 and 5 with MainID 1
FK in this case means a Foreign Key (i.e. a reference to a primary key in another table). You'd probably also want to add a unique constraint to GroupsInMain on MainID and GroupID, since you'd never want the same values for the pairing to show up more than once.
Your query would then be:
select GroupsInMain.MainID, Group.GroupName
from Group, GroupsInMain
where Group.GroupID=GroupsInMain.GroupID
and Group.GroupID=5

MySQL, per found record join a different parent table

I have the following parent <-> child datamodel:
(almost every line is a table, indented means child-of)
consumerGoods
food
meat
item
fruit
item
vegetable
item
The child-items of meat, fruit and vegetables are in the same table (named items) because they have identical attributes. In the items table I have fields that describes the parent and the parentId.
So an item record could be:
id:1
parentType:meat
parentId:4
price:3.25
expDate:2009-12-31
description:bacon
I'm now building a full text MySQL search for the contents of the description field in "items", but I also want each result to have the information of its parent table, so a "bacon-item" has the data that's in its parent record. I also want each returned result to have data that is in the parent food record and the parent consumerGoods record.
I've got the following query now, but I don't know how to join based on the value of a field in a record, or if that's even possible.
SELECT
*
FROM
item
WHERE MATCH
(description
AGAINST
('searchKey')
One way to do this is is to do multiple queries for each matching "item" record, but if I had a lot of results that would be a lot of queries and would also slow down any filtering I'd want to do for facet-based searching. Another option is to make a new table that contains all the parent item info for each item record and search through that, but then I'd have to constantly update that table if I add item records, which is redundant and quite some work.
I'd like to hear it if I'm thinking in the right direction, or if I'm totally misguided. Any suggestions welcome.
As a general rule of thumb your database structure should contain data, but should not itself be data. A sign that you're breaking this is when you feel that you have to join to a different table based on the data you're reading from some other table. At that point you need to back up and consider your overall data model because odds are very good that you're doing something not quite right.
You could join against a subquery containing the union of all parent types:
select *
from item
left join (
select 'meat' as type, Redness, '' as Ripeness from meat
union all
select 'fruit' as type, -1 as Redness, Ripeness from fruit
union all
select 'vegetable' as type, -1 as Redness, Ripeness from vegetable
) parent on parent.type = item.parentType
But if you can, redesign the database. Instead of the complex model, change it to one table of Items and one table of Categories. The categories should contain one row for meat, one for fruit, and one for vegetables.
Since your example is contrived, it's difficult to know what the actual information requirements are in your case. Damir's diagram shows you the correct way to model PKs and FKs when you have a super-type sub-type relationships.
This situation is one case of a pattern called "generalization-specialization". Almost any treatment of object modeling will deal with generalization-specialization, although it may use different terminology. However, if you want to find articles that help you build a relational database that uses specialization-generalization, search for "generalization specialization relational modeling".
The best of the articles will start by teaching you the same concept that Damir's response illustrated for you. From there, you will learn how to create queries and views that can search for either all kinds of items, or for particular kinds of items, if you know what you are searching for.
A sample view follows:
create view FruitItems as
select
c.ConsumerGoodsID,
Price,
Description,
ConsumerGoodType,
ExpiryDate,
FoodType,
IsTropic
from
ConsumerGoods c
INNER JOIN Food f on f.ConsumerGoodsID = c.ConsumerGoodsID
INNER JOIN Fruit fr on fr.ConsumerGoodsID = c.ConsumerGoodsID
Similarly, you could create views for VegetableItems, MeatItems, and HouseSupplyItems, and even one large view, namely Items, that's the union of each of the specialized views.
In the Items view IsTropic would be true for all tropical fruits, false for all non tropical fruits, and null for Meats, Vegetables, and HouseSupplies. I'm not going to show you the entire Item view for a contrived case, but you get the idea. Especially if you read the best of the articles on relational modeling of this pattern.
The Items view might be a little slow, but it could come in handy when you really don't know any better way to search. And if you search for Istropic = True, you'll automatically exclude all the Meats, Vegetables, and HouseSupplies.
As #Andomar suggested, the design is a bit off; having "multiple parent tables" does not map to DB foreign keys concept. Here is one possible suggestion. This one uses two levels of super-type/subtype relationships. Super-type table contains columns specific to all subtypes (categories), while subtype tables contain columns specific only to the category.

Searching a database of names

I have a MYSQL database containing the names of a large collection of people. Each person in the database could could have one or all of the following name types: first, last, middle, maiden or nick. I want to provide a way for people to search this database to see if a person exists in the database.
Are there any off the shelf products that would be suited to searching a database of peoples names?
With a bit of ingenuity, MySQL will do just what you need... The following gives a few ideas how this could be accomplished.
Your table: (I call it tblPersons)
PersonID (primary key of sorts)
First
Last
Middle
Maiden
Nick
Other columns for extra info (address, whatever...)
By keeping the table as-is, and building an index on each of the name-related columns, the following query provides an inefficient but plausible way of finding all persons whose name matches somehow a particular name. (Jack in the example)
SELECT * from tblPersons
WHERE First = 'Jack' OR Last = 'Jack' OR Middle = 'Jack'
OR Maiden = 'Jack' OR Nick = 'Jack'
Note that the application is not bound to only searching for one name value to be sought in all the various name types. The User can also input a specific set of criteria for example to search for the First Name 'John' and Last Name 'Lennon' and the Profession 'Artist' (if such info is stored in the db) etc.
Also, note that even with this single table approach, one of the features of your application could be to let the user tell the search logic whether this is a "given" name (like Paul, Samantha or Fatima) or a "surname" (like Black, McQueen or Dupont). The main purpose of this is that there are names that can be either (for example Lewis or Hillary), and by being, optionally, a bit more specific in their query, the end users can get SQL to automatically weed-out many irrelevant records. We'll get back to this kind of feature, in the context of alternative, more efficient database layout.
Introducing a "Names" table.
Instead (or in addition...) of storing the various names in the tblPersons table, we can introduce an extra table. and relate it to tblPersons.
tblNames
PersonID (used to relate with tblPersons)
NameType (single letter code, say F, L, M, U, N for First, Last...)
Name
We'd then have ONE record in tblPersons for each individual, but as many records in tblNames as they have names (but when they don't have a particular name, few people for example have a Nickname, there is no need for a corresponding record in tblNames).
Then the query would become
SELECT [DISTINCT] * from tblPersons P
JOIN tblNames N ON N.PersonID = P.PersonID
WHERE N.Name = 'Jack'
Such a layout/structure would be more efficient. Furthermore this query would lend itself to offer the "given" vs. "surname" capability easily, just by adding to the WHERE clause
AND N.NameType IN ('F', 'M', 'N') -- for the "given" names
(or)
AND N.NameType IN ('L', 'U', 'N') -- for the "surname" types. Note that
-- we put Nick name in there, but could just as eaily remove it.
Another interest of this approach is that it would allow storing other kinds of names in there, for example the SOUNDEX form of every name could be added, under their own NameType(s), allowing to easily find names even if the spelling is approximate.
Finaly another improvement could be to introduce a separate lookup table containing the most common abbreviations of given names (Pete for Peter, Jack for John, Bill for William etc), and to use this for search purposes (The name columns used for providing the display values would remain as provided in the source data, but the extra lookup/normalization at the level of the search would increase recall).
You shouldn't need to buy a product to search a database, databases are built to handle queries.
Have you tried running your own queries on it? For example: (I'm imagining what the schema looks like)
SELECT * FROM names WHERE first_name='Matt' AND last_name='Way';
If you've tried running some queries, what problems did you encounter that makes you want to try a different solution?
What does the schema look like?
How many rows are there?
Have you tried indexing the data in any way?
Please provide some more information to help answer your question.