Mysql paginate more than one table - mysql

I'm working on an ECommerce website, in which there are 2 database tables in MySQL, one is products and the other one is taxonomies, products and taxonomies are many to many relationship, and taxonomies have a tree structure, meaning there's a parent_id field in taxonomies table to identify the parent id of a taxonomy.
When user selects one taxonomy, I want to get all the products that belong to this taxonomy and all its offspring taxonomies, I did this by first finding out all the offspring taxonomies of the selected taxonomy, then get paginated products result from there, but in my site there are in total 5000 taxonomies, and my solution makes the site slow like a dog...... Any advice on how I could achieve this for the sake of performance?
products table:
+-------------------+----------------------+------+-----+---------------------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+----------------------+------+-----+---------------------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| code | bigint(20) | NO | UNI | NULL | |
| SKU | varchar(255) | NO | | NULL | |
| name | varchar(100) | NO | | NULL | |
| description | varchar(2000) | NO | | NULL | |
| short_description | varchar(200) | NO | | NULL | |
| price | decimal(8,2) | NO | | 0.00 | |
| discounted_price | decimal(8,2) | NO | | 0.00 | |
| stock | smallint(5) unsigned | NO | | 0 | |
| sales | smallint(5) unsigned | NO | | 0 | |
| num_reviews | smallint(6) | NO | | 0 | |
| weight | decimal(5,2) | NO | | 0.00 | |
| overall_rating | decimal(3,2) | NO | | 5.00 | |
| activity_id | int(10) unsigned | YES | MUL | NULL | |
| created_at | timestamp | NO | | 0000-00-00 00:00:00 | |
| updated_at | timestamp | NO | | 0000-00-00 00:00:00 | |
+-------------------+----------------------+------+-----+---------------------+----------------+
taxonomies table:
+--------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| name | varchar(100) | YES | UNI | NULL | |
| parent_id | int(10) unsigned | YES | MUL | NULL | |
| num_products | smallint(6) | NO | | 0 | |
+--------------+------------------+------+-----+---------+----------------+
product_taxonomy table:
+-------------+------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+------------------+------+-----+---------+----------------+
| id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| product_id | int(10) unsigned | NO | MUL | NULL | |
| taxonomy_id | int(10) unsigned | NO | MUL | NULL | |
+-------------+------------------+------+-----+---------+----------------+

In case depth of single level one can use the following query
SELECT * FROM `product_taxonomy`
INNER JOIN (SELECT * FROM `taxonomies` WHERE `id` = 100 OR `parent_id` = 100) `taxonomies`
ON `product_taxonomy`.`taxonomy_id` = `taxonomies`.`id`
LEFT JOIN `products` ON `product_taxonomy`.`product_id` = `products`.`id`
You can add limit, offset to the above query for pagination.
100 in the above query represents the taxonomy id requested by the user.
Apart from this I would suggest :-
1) id in your product table to renamed if possible to product_id as referenced in your product_taxonomy and I presume in other tables, similarly taxonomy_id.
This way when you join query column name would be the same.
2) I hope product_taxonomy.product_id, product_taxonomy.taxonomy_id are indexed for faster querying.
Update:
What you had mentioned in the comment below is a hierarchical data problem and not what relational database ideally intended for.
Solution 1
IF you know for sure that you will have only 4 levels / generation then you can do 4 join queries.
I can elaborate on this if you need to.
Solution 2
If you are not too deep or committed to the architecture of this project I would recommend restructuring it such a way, where recursion is taken care of by the server side scripting. i.e You change your CMS/taxonomy management in such a way that whenever you add/remove/modify taxonomy the script will update a table called taxonomy_childs with all possible offspring for a given category so that you have a flat data at your disposal when you need it.
Personally I would prefer this. I always like my database to match my business logic requirement.
I can elaborate on this if you need to.
Solution 3
As mentioned earlier hierarchical data is not a strong point of a relational database. Having said that you can implement something called as Nested Set Model.
Please read more at http://mikehillyer.com/articles/managing-hierarchical-data-in-mysql/
You would need to add 3 columns to your taxonomy table :- level_depth, lft, rht.
Please let me know which solution would you want me to elaborate.

Related

Optimising query - user subscriptions in 1 database, 3-level data in another. Finding top level where user has subscriptions

I have an application which stores a hierarchical list of filters which a user can subscribe/unsubscribe from.
There are 2 databases involved:
db1: Stores the hierarchical list
db2: Stores the user's subscription preferences
Both databases are on the same server.
The hierarchical list (in db1) is composed of 3 tables as follows:
mysql> DESCRIBE regulations;
+-------------------+---------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------------+---------------------+------+-----+---------+----------------+
| id | tinyint(3) unsigned | NO | PRI | NULL | auto_increment |
| name | varchar(255) | NO | | NULL | |
+-------------------+---------------------+------+-----+---------+----------------+
mysql> DESCRIBE groups;
+---------------+-----------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+---------------+-----------------+------+-----+---------+----------------+
| id | int(4) unsigned | NO | PRI | NULL | auto_increment |
| name | varchar(255) | NO | | NULL | |
| regulation_id | int(4) unsigned | NO | MUL | NULL | |
+---------------+-----------------+------+-----+---------+----------------+
mysql> DESCRIBE filters;
+----------+----------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+----------+----------------------+------+-----+---------+----------------+
| id | smallint(5) unsigned | NO | PRI | NULL | auto_increment |
| name | varchar(100) | NO | | NULL | |
| group_id | int(4) unsigned | NO | MUL | NULL | |
+----------+----------------------+------+-----+---------+----------------+
So the hierarchy is:
regulations
groups (foreign key: regulation_id)
filters (foreign key: group_id)
The user is subscribed to 1 or more filters.id. These are stored in a separate database (database name: db2) where the f_id field corresponds to filters.id. The table structure is as follows:
mysql> DESCRIBE tbl_alerts;
+--------+-----------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+--------+-----------------------+------+-----+---------+----------------+
| tbl_id | int(10) unsigned | NO | PRI | NULL | auto_increment |
| u_id | mediumint(8) unsigned | NO | | NULL | |
| f_id | smallint(5) unsigned | NO | | NULL | |
+--------+-----------------------+------+-----+---------+----------------+
I need to know which regulations.name the user has subscriptions for.
The way in which I've done this is to select all of the f_id in tbl_alerts (assuming a user ID of 123, represented by u_id), e.g.
SELECT f_id FROM tbl_alerts WHERE u_id = 123;
And then use this in an IN condition as follows:
SELECT
DISTINCT(db1.regulations.`name`)
FROM
db1.groups
JOIN db1.regulations
ON db1.groups.regulation_id = db1.regulations.id
JOIN db1.filters
ON db1.filters.group_id = db1.groups.id
WHERE db1.filters.`id` IN (
SELECT f_id FROM db2.tbl_alerts WHERE u_id = 123
)
Is there a more optimal way to write this?
Using 5.5.60-MariaDB
You can try the following "Cross-Database" Join query:
SELECT
DISTINCT db1.regulations.name
FROM
db1.groups
JOIN db1.regulations
ON db1.groups.regulation_id = db1.regulations.id
JOIN db1.filters
ON db1.filters.group_id = db1.groups.id
JOIN db2.tbl_alerts
ON db2.tbl_alerts.f_id = db1.filters.id
WHERE db2.tbl_alerts.u_id = 123
Note that DISTINCT keyword is not a function, so parentheses are unnecessary. Now, regarding the performance part, you will need to benchmark between your current set of queries, versus this single query.

Having more than one value inserted into the same column

I have a webapp that I'm building. This webapp will take as input some products (cars, motos, boats, houses, etc...) and each product will have one or more photos associated with it. The id of each of photo is generated by the uniqid() function of php.
My problem is:I can't seem to fit more than two id_photos into the same column
+-----------+------------------------------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------------------------------+------+-----+---------+----------------+
| carid | int(11) | NO | MUL | NULL | auto_increment |
| brand | enum('Alfa Romeo','Aston Martin','Audi') | NO | | NULL | |
| color | varchar(20) | NO | | NULL | |
| type | enum('gasoline','diesel','eletric') | YES | | NULL | |
| price | mediumint(8) unsigned | YES | | NULL | |
| mileage | mediumint(8) unsigned | YES | | NULL | |
| model | text | YES | | NULL | |
| year | year(4) | YES | | NULL | |
| id_photos | varchar(30) | YES | | NULL | |
+-----------+------------------------------------------+------+-----+---------+----------------+
What I would like to happen is something like this: INSERT INTO cars(id_photos) values ('id_1st_photo', 'id_2nd_photo')
Ending up having something like this:
| 60 | Audi | Yellow | diesel | 252352 | 1234112 | R8 | 1990 | id_1st_photo id_2nd_photo |
Eventually I would have to grab those photos from the folders they are in which is something like this: /var/www/website/$login/photos/id_of_photo with the query select id_photos from cars where carid=$id.
You may found some data types that is not proprelly good for the data that the server will receive but I'm one week into mysql and I'll worry about data types later on.
First of all I don't know if that is possible, if it's not how can I design something to work like that?
I have found this question that is quite the same of mine but I can't seem to implement something like this: add multiple values in one column
You can insert the concatenated values into a field. But it is not a good practice. You can create another table with foreign key having the id of the parent table.
You can easily adapt the approach in the linked question and even remove one table needed:
You first table stays almost the same, but has the id_photos column removed:
+-----------+------------------------------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------------------------------+------+-----+---------+----------------+
| carid | int(11) | NO | MUL | NULL | auto_increment |
| brand | enum('Alfa Romeo','Aston Martin','Audi') | NO | | NULL | |
| color | varchar(20) | NO | | NULL | |
| type | enum('gasoline','diesel','eletric') | YES | | NULL | |
| price | mediumint(8) unsigned | YES | | NULL | |
| mileage | mediumint(8) unsigned | YES | | NULL | |
| model | text | YES | | NULL | |
| year | year(4) | YES | | NULL | |
+-----------+------------------------------------------+------+-----+---------+----------------+
Then you'll add a second table to store the links to the photo ids:
+-----------+------------------------------------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------------------------------------+------+-----+---------+----------------+
| carid | int(11) | NO | MUL | NULL | |
| id_photos | varchar(30) | NO | | NULL | |
+-----------+------------------------------------------+------+-----+---------+----------------+
Both tables are linked by the field carid (You should even make carid in the second table a foreign key pointing to the one in the first table).
Each id_photos then results in a new row in the second table.
To query the data you probably need a JOIN between both tables and maybe a GROUP BY to reduce the result to one row per carid again, but this depends on the other usecases.
You can insert the string formatted woth multiple photo name
INSERT INTO cars(id_photos) values ('id_1st_photo, id_2nd_photo')
In this way you don'have a well normalized database structure so you have problem when retrive the singole foto name ..
i suggest you of normalize the id_photo column in a separata table with reference to the master table and in this way store each single photo in one row

mysql time field conditions for expiration

I have table with auction lots:
mysql> desc lots;
+------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+----------------+
| id | int(10) | NO | PRI | NULL | auto_increment |
| account_id | int(10) | YES | | NULL | |
| item_id | int(10) | YES | | NULL | |
| bid | int(10) | YES | | NULL | |
| buyout | int(10) | YES | | NULL | |
| leader | int(13) | YES | | NULL | |
| left | int(10) | YES | | NULL | |
+------------+--------------+------+-----+---------+----------------+
left - field in unix timestamp format like '1391143424'. This shows the timestamp after which the lot should be expired.
I need to develop efficient algorithm for:
-when customer click on SEARCH button, the result will show only non-expired lots
-once lot expired it should be automatically put into completed_lots table
I have some ideas, but want to verify if there is any better approaches:
1: add additional search criteria like:
WHERE '$currtime' <= left;
2: setup some asynchronous tasks using crontab that will check lots table for expired lots every 5 minutes and move it to completed_lots table.
Are there any better ways? maybe using another mysql filed types for unix time, or create some mysql functions that will do that job automatically?

Is many to many needed on this DB?

I am designing a DB for a possible PHP MySQL project I may be undertaking.
I am a complete novice at relational DB design, and have only worked with single table DB's before.
This is a diagram of the tables:
So, 'Cars' contains each model of car, and the other 3 tables contains parts that the car can be fitted with.
So each car can have different parts from each of the three tables, and each part can be fitted to different cars from the parts table. In reality, there will be about 10 of these parts tables.
So, what would be the best way to link these together? do I need another table in the middle etc?
and what would I need to do with keys in terms of linking.
There is some inheritance in your parts. The common attributes seem to be:
part_number
price
and there are some specifics for your part types exhaust, software and intake.
There are two strategies:
- have three tables and one view over the three tables
- have one table with a parttype column and may be three views for the tables.
If you'd like to play with your design you might want to look at my companies website http://www.uml2php.com. UML2PHP will automatically convert your UML design to a database design and let you "play" with the result.
At:
http://service.bitplan.com/uml2phpexamples/carparts/
you'll find an example applicaton along your design. The menu does not allow you to access all tables via the menu yet.
via:
http://service.bitplan.com/uml2phpexamples/carparts/index.php?function=dbCheck
the table definitions are accessible:
mysql> describe CP01_car;
+-------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------------+------+-----+---------+-------+
| oid | varchar(32) | NO | | NULL | |
| car_id | varchar(255) | NO | PRI | NULL | |
| model | varchar(255) | YES | | NULL | |
| description | text | YES | | NULL | |
| model_year | decimal(10,0) | YES | | NULL | |
+-------------+---------------+------+-----+---------+-------+
mysql> describe CP01_part;
+-------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+-------+
| oid | varchar(32) | NO | | NULL | |
| part_number | varchar(255) | NO | PRI | NULL | |
| price | varchar(255) | YES | | NULL | |
| car_car_id | varchar(255) | YES | | NULL | |
+-------------+--------------+------+-----+---------+-------+
mysql> describe cp01_exhaust;
+-------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+-------+
| oid | varchar(32) | NO | | NULL | |
| type | varchar(255) | YES | | NULL | |
| part_number | varchar(255) | NO | PRI | NULL | |
| price | varchar(255) | YES | | NULL | |
+-------------+--------------+------+-----+---------+-------+
mysql> describe CP01_intake;
+-------------+--------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+-------+
| oid | varchar(32) | NO | | NULL | |
| part_number | varchar(255) | NO | PRI | NULL | |
| price | varchar(255) | YES | | NULL | |
+-------------+--------------+------+-----+---------+-------+
mysql> describe CP01_software;
+-------------+---------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-------------+---------------+------+-----+---------+-------+
| oid | varchar(32) | NO | | NULL | |
| power_gain | decimal(10,0) | YES | | NULL | |
| part_number | varchar(255) | NO | PRI | NULL | |
| price | varchar(255) | YES | | NULL | |
+-------------+---------------+------+-----+---------+-------+
The above tables have been generated from the UML model and the result does not fit your needs yet.
Especially if you think of having 10 or more table likes this. The field car_car_id that links your parts to the car table should be available in all the tables. And according to the design proposal the base "table" for the parts should be a view like this:
mysql>
create view partview as
select oid,part_number,price from CP01_software
union select oid,part_number,price from CP01_exhaust
union select oid,part_number,price from CP01_intake;
of course the car_car_id column also needs to be selected;
Now you can edit every table by itself and the partview will show all parts together.
To be able to distinguish the parts types you might want to add another column "part_type".
I would do it like this. Instead of having three different tables for car parts:
table - cars table - parts (this would have only an id and a part
number and a type maybe)
table - part_connections (connectin cars with parts)
table - part_options (with all the options which arent in the
parts table like "power gain")
table - part_option_connections (which connects the parts to the
various part options)
In this way it is much easier to add new parts (because you won't need a new table) and its closer to being normalized as well.

MySQL join four tables and get some kind of SUM result

I have four tables like this:
mysql> describe courses;
+-----------------+-------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------------+-------------+------+-----+---------+----------------+
| course_id | int(11) | NO | PRI | NULL | auto_increment |
| course_name | varchar(75) | YES | | NULL | |
| course_price_id | int(11) | YES | MUL | NULL | |
+-----------------+-------------+------+-----+---------+----------------+
mysql> describe pricegroups;
+-------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-------------+--------------+------+-----+---------+----------------+
| price_id | int(11) | NO | PRI | NULL | auto_increment |
| price_name | varchar(255) | YES | | NULL | |
| price_value | int(11) | YES | | NULL | |
+-------------+--------------+------+-----+---------+----------------+
mysql> describe courseplans;
+------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+----------------+
| plan_id | int(11) | NO | PRI | NULL | auto_increment |
| plan_name | varchar(255) | YES | | NULL | |
| plan_time | int(11) | YES | | NULL | |
+------------+--------------+------+-----+---------+----------------+
mysql> describe course_to_plan;
+-----------+---------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+-----------+---------+------+-----+---------+-------+
| course_id | int(11) | NO | PRI | NULL | |
| plan_id | int(11) | NO | PRI | NULL | |
+-----------+---------+------+-----+---------+-------+
Let me try to explain what I have and what I would like to do...
All my courses (course_id) has different steps (plan_id) wich has a value of 1 or more days (plan_time). A course has one or more steps (course_to_plan)A course is connected to a pricegroup (price_id).
I would like to query my MySQL database and get an output off:
The course_name, the plan_id's it has, and based on the value of price_id together with the value in the plan_time get a result who looks something like this:
+------------+--------------+------------+---------+
| course_name| pricegroup | plan_time | RESULT |
+------------+--------------+------------+---------+
| Math | Expensive | 7 | 3500 |
+------------+--------------+------------+---------+
I hope you understand me...
Is it even possible with the structure I have or should I "rebuild-and-redo-correct" something?
SELECT c.course_name, p.price_name, SUM(cp.plan_time), SUM(cp.plan_time * p.price_value)
FROM courses c
INNER JOIN pricegroups p ON p.price_id = c.course_price_id
INNER JOIN course_to_plan cpl ON cpl.course_id = c.course_id
INNER JOIN courseplans cp ON cp.plan_id = cpl.plan_id
GROUP BY c.course_name, p.price_name
Please note that it seems to me that your implementation might be erroneous. The way you want the data makes me think that you could be happier with a plan having a price, so you don't apply the same price for a plan which is "expensive" AND another plan which is "cheap", which is what you are doing at the moment. But I don't really know, this is intuitive :-)
Thanks for accepting the answer, regards.
Let me see if I understand what you need:
SELECT c.course_name, pg.price_name,
COUNT(cp.plan_time), SUM(pg.price_value * cp.plan_time) AS result
FROM courses c
INNER JOIN pricegroups pg ON c.course_price_id = pg.price_id
INNER JOIN course_to_plan ctp ON c.course_id = ctp.course_id
INNER JOIN courseplans cp ON ctp.plan_id = cp.plan_id
GROUP BY c.couse_name, pg.price_name