I have a MySql Table with the following schema.
table_products - "product_id", product_name, product_description, product_image_path, brand_id
table_product_varient - "product_id", "varient_id", product_mrp, product_sellprice, product_imageurl
table_varients - "varient_id", varient_name
table_product_categories - "product_id", "category_id"
and this is the Mysql select query i am using to fetch the data for the category user provided.
select * from table_products, table_product_varients, table_varients, table_product_categories where table_product_categories.category_id = '$cate_id' && table_product_categories.product_id = table_products.product_id && table_products.product_id = table_product_varients.product_id && table_varients.varient_id = table_product_varients.varient_id
The problem is that, as table contains lot of products, and each product contains lot of varients, it is taking too much time to fetch the data. And i doubt, as data will grow, the time will increase to fetch the items. Is there any optimized way to achieve the same.
Your help will be highly appreciated.
Devesh
the query below would be a start, or something similar
SELECT
*
FROM
table_products P
INNER JOIN
table_product_categories PC
ON
PC.product_id = P.product_id
INNER JOIN
table_product_varients PV
ON
P.product_id = PV.product_id
INNER JOIN
table_varients V
ON
V.varient_id = PV.varient_id
where
table_product_categories.category_id = '$cate_id'
and as suggested do you really need to return * as this does mean selecting all columns from all tables within the query, which as we know from the joins themselves there a duplicates.
you should use indexing on tables for faster queries, set relationships between the joining tables this will also ensure referential integrity.
Hope this makes sense and helps :)
You can use the EXPLAIN command to see whats happening in the server. Then you can optimize the request by creating indexes.
Some links:
Some slides about tuning
MYSQL manual: 8.2.1. Optimizing SELECT Statements
yes you are correct, the query you are using above is not efficient:
you can get the same result as above by using ON clause instead of where clause.
the difference between them is, where clause gets all the rows and then filters out based on condidition specified.
Where as in case of ON Clause, the join haappens only on the rows which have met the condidtion specified in the ON clause
so..make your query as below:
So make use of joins instead of using where clause..
Hope this helps..
Related
I am trying to join 5 tables in which i want to get different currency mentioned on different tables against same contract id.
It is giving me results when i join any three tables but when I add one more table in query the server gets unresponsive until I have to kill the process.
Please help me where I am doing a mistake.
SELECT c.department_id,
c.contract_id,
c.seller_id,
c.buyer_id,
c.contract_ratecurrency AS contractcurrency,
b.currency_id AS billcurrency,
s.saleinv_currency AS saleinvcurrency,
cm.currency_id AS commissioncurrency,
sl.currency_id AS cmlogcurrency,
c.contract_iscancel
FROM tbl_contracts C
JOIN tbl_contract_bill b ON c.contract_id=b.contract_id
JOIN tbl_contract_saleinvoice s ON c.contract_id =s.contract_id
JOIN tbl_commission_payment cm ON c.department_id = cm.department_id
JOIN tbl_saleinvoice_commission_log sl ON c.department_id = sl.department_id
WHERE (c.contract_ratecurrency <> s.saleinv_currency
OR c.contract_ratecurrency <> b.currency_id
OR s.saleinv_currency <> b.currency_id
OR cm.currency_id <> sl.currency_id
OR c.contract_ratecurrency <> cm.currency_id
OR s.saleinv_currency <> cm.currency_id
OR b.currency_id <> cm.currency_id)
AND (c.contract_iscancel =0)
requried result should be
ccontractid,csellerid,cbuyerid,ccurrency,bcurrency,scurrency,cmcurrency,slcurrency
101,25,50,1,1,2,3,1
102,28,16,2,3,1,3,2
It looks like you are having performance issues. To optimize your database structure you have multiple options:
Adding indexes on your keys.
Let's take a look to your join statement:
JOIN tbl_saleinvoice_commission_log sl ON c.department_id = sl.department_id
Adding a clustered index on department_id on
tbl_saleinvoice_commission_log table will help you a lot in
performance wise. For more information you can check this link.
Partitioning is another way to increase performance, but you need to check your database structure to see whether it works for you or not. For more information you can check this link.
Also I believe your tables are one to many, so you might need to check how many rows you are trying to retrieve. If your database server is not capable of processing big number of rows you might need to improve your hardware or CPU usage limits of your database daemon.
I am tying to execute this query but it is taking more than 5 hours, but the data base size is just 20mb. this is my code. Here I am joining 11 tables with reg_id. I need all columns with distinct values. Please guide me how to rearrange the query.
SELECT *
FROM degree
JOIN diploma
ON degree.reg_id = diploma.reg_id
JOIN further_studies
ON diploma.reg_id = further_studies.reg_id
JOIN iti
ON further_studies.reg_id = iti.reg_id
JOIN personal_info
ON iti.reg_id = personal_info.reg_id
JOIN postgraduation
ON personal_info.reg_id = postgraduation.reg_id
JOIN puc
ON postgraduation.reg_id = puc.reg_id
JOIN skills
ON puc.reg_id = skills.reg_id
JOIN sslc
ON skills.reg_id = sslc.reg_id
JOIN license
ON sslc.reg_id = license.reg_id
JOIN passport
ON license.reg_id = passport.reg_id
GROUP BY fullname
Please help me if I did any mistake
This is a bit long for a comment.
The first problem with your query is that you are using select * with group by fullname. You have zillions of columns in the select that are not in the group by. Unless you really, really, really know what you are doing (which I doubt), this is the wrong way to write a query.
Your performance problem is undoubtedly due to cartesian products and lack of indexes. You are joining across different dimensions -- such as skills and degrees. The result is a product of all the possibilities. For some people, the data size can grow and grow and grow.
And then, the question is: do you have indexes on the keys used in the joins? For performance, you generally want such indexes.
I thought the problem is in the query.First make sure group by fullname and try to give some column names instead of *.
I did not write this query. I am working on someone else's old code. I am looking into changing what is needed for this query but if I could simply speed up this query that would solve my problem temporarily. I am looking at adding indexes. when I did a show indexes there are so many indexes on the table orders can that also slow down a query?
I am no database expert. I guess I will learn more from this effort. :)
SELECT
orders.ORD_ID,
orders.ORD_TotalAmt,
orders.PAYMETH_ID,
orders.SCHOOL_ID,
orders.ORD_AddedOn,
orders.AMAZON_PurchaseDate,
orders.ORDSTATUS_ID,
orders.ORD_InvoiceNumber,
orders.ORD_CustFirstName,
orders.ORD_CustLastName,
orders.AMAZON_ORD_ID,
orders.ORD_TrackingNumber,
orders.ORD_SHIPPINGCNTRY_ID,
orders.AMAZON_IsExpedited,
orders.ORD_ShippingStreet1,
orders.ORD_ShippingStreet2,
orders.ORD_ShippingCity,
orders.ORD_ShippingStateProv,
orders.ORD_ShippingZipPostalCode,
orders.CUST_ID,
orders.ORD_ShippingName,
orders.AMAZON_ShipOption,
orders.ORD_ShipLabelGenOn,
orders.ORD_SHIPLABELGEN,
orders.ORD_AddressVerified,
orders.ORD_IsResidential,
orderstatuses.ORDSTATUS_Name,
paymentmethods.PAYMETH_Name,
shippingoptions.SHIPOPT_Name,
SUM(orderitems.ORDITEM_Qty) AS ORD_ItemCnt,
SUM(orderitems.ORDITEM_Weight * orderitems.ORDITEM_Qty) AS ORD_ItemTotalWeight
FROM
orders
LEFT JOIN orderstatuses ON
orders.ORDSTATUS_ID = orderstatuses.ORDSTATUS_ID
LEFT JOIN orderitems ON
orders.ORD_ID = orderitems.ORD_ID
LEFT JOIN paymentmethods ON
orders.PAYMETH_ID = paymentmethods.PAYMETH_ID
LEFT JOIN shippingoptions ON
orders.SHIPOPT_ID = shippingoptions.SHIPOPT_ID
WHERE
(orders.AMAZON_ORD_ID IS NOT NULL AND (orders.ORD_SHIPLABELGEN IS NULL OR orders.ORD_SHIPLABELGEN = '') AND orderstatuses.ORDSTATUS_ID <> 101 AND orderstatuses.ORDSTATUS_ID <> 40)
GROUP BY
orders.ORD_ID,
orders.ORD_TotalAmt,
orders.PAYMETH_ID,
orders.SCHOOL_ID,
orders.ORD_AddedOn,
orders.ORDSTATUS_ID,
orders.ORD_InvoiceNumber,
orders.ORD_CustFirstName,
orders.ORD_CustLastName,
orderstatuses.ORDSTATUS_Name,
paymentmethods.PAYMETH_Name,
shippingoptions.SHIPOPT_Name
ORDER BY
orders.ORD_ID
One simple thing you should consider is whether you really need to use left joins or you would be satisfied using inner joins for some of the joins. the new query would not be the same as the original query, so you would need to think carefully about what you really want back. If your foreign key relationships are indexed correctly, this could help substantially, especially between ORDERS and ORDERITEMS, because I would imagine these are your largest tables. The following post has a good explanation: INNER JOIN vs LEFT JOIN performance in SQL Server. There are lots of other things that can be done, but you will need to post the query plan so people can dive deeper.
It looks like just adding the index was all that was needed.
create index orderitems_ORD_ID_index on orderitems(ORD_ID);
I need a query that quickly shows the articles within a particular module (a subset of articles) that a user has NOT uploaded a PDF for. The query I am using below takes about 37 seconds, given there are 300,000 articles in the Article table, and 6,000 articles in the Module.
SELECT *
FROM article a
INNER JOIN article_module_map amm ON amm.article=a.id
WHERE amm.module = 2 AND
a.id NOT IN (
SELECT afm.article
FROM article_file_map afm
INNER JOIN article_module_map amm ON amm.article = afm.article
WHERE afm.organization = 4 AND
amm.module = 2
)
What I am doing in the above query is first truncating the list of articles to the selected module, and then further truncating that list to the articles that are not in the subquery. The subquery is generating a list of the articles that an organization has already uploaded PDF's for. Hence, the end result is a list of articles that an organization has not yet uploaded PDF's for.
Help would be hugely appreciated, thanks in advance!
EDIT 2012/10/25
With #fthiella's help, the below query ran in an astonishing 1.02 seconds, down from 37+ seconds!
SELECT a.* FROM (
SELECT article.* FROM article
INNER JOIN article_module_map
ON article.id = article_module_map.article
WHERE article_module_map.module = 2
) AS a
LEFT JOIN article_file_map
ON a.id = article_file_map.article
AND article_file_map.organization=4
WHERE article_file_map.id IS NULL
I am not sure that i can understand the logic and the structure of the tables correctly. This is my query:
SELECT
article.id
FROM
article
INNER JOIN
article_module_map
ON article.id = article_module_map.article
AND article_module_map.module=2
LEFT JOIN
article_file_map
ON article.id = article_file_map.article
AND article_file_map.organization=4
WHERE
article_file_map.id IS NULL
I extract all of the articles that have a module 2. I then select those that organization 4 didn't provide a file.
I used a LEFT JOIN instead of a subquery. In some circumstances this could be faster.
EDIT Thank you for your comment. I wasn't sure it would run faster, but it surprises me that it is so much slower! Anyway, it was worth a try!
Now, out of curiosity, I would like to try all the combinations of LEFT/INNER JOIN and subquery, to see which one runs faster, eg:
SELECT *
FROM
(SELECT *
FROM
article INNER JOIN article_module_map
ON article.id = article_module_map.article
WHERE
article_module_map.module=2)
LEFT JOIN
etc.
maybe removing *, and I would like to see what changes between the condition on the WHERE clause and on the ON clause... anyway I think it doesn't help much, you should concentrate on indexes now.
Indexes on keys/foreign key should be okay already, but what if you add an index on article_module_map.module and/or article_file_map.organization ?
When optimizing queries I use to check the following points:
First: I would avoid using * in SELECT clause, instead, name the diferent fields you want. This increases crazily the speed (I had one which took 7 seconds with *, and naming the field decreased to 0.1s).
Second: As #Adder says, add indexes to your tables.
Third: Try using INNER JOIN instead of WHERE amm.module = 2 AND a.id NOT IN ( ... ). I think I read (I don't remember it well, so take it carefully) that usually MySQL optimize INNER JOINS, and as your subquery is a filter, maybe using three INNER JOINS plus WHERE would be faster to retrieve.
I have this MySQL query that is very slow, I presume because of all the JOINs (it seems complicated, but it's a matter of lot of tables):
SELECT DISTINCT doctors.doc_id,
doctors.doc_user,
doctors.doc_first,
doctors.doc_last,
doctors.doc_email,
doctors.doc_notes,
titles.tit_name,
specializations.spe_name,
activities.act_name,
users.use_first,
users.use_last,
(SELECT COUNT(*) FROM locations WHERE locations.loc_doctor = doctors.doc_id) AS loc_count,
(SELECT COUNT(*) FROM reception WHERE reception.rec_doctor = doctors.doc_id) AS rec_count,
(SELECT COUNT(*) FROM visits INNER JOIN reports ON visits.vis_report = reports.rep_id WHERE visits.vis_doctor = doctors.doc_id AND reports.rep_user LIKE '%s') AS vis_count
FROM
doctors
INNER JOIN titles ON titles.tit_id = doctors.doc_title
INNER JOIN specializations ON specializations.spe_id = doctors.doc_specialization
INNER JOIN activities ON activities.act_id = doctors.doc_activity
LEFT JOIN locations ON locations.loc_doctor = doctors.doc_id
INNER JOIN users ON doctors.doc_user = users.use_id
WHERE
((doctors.doc_last LIKE %s) OR (doctors.doc_first LIKE %s) OR (doctors.doc_email LIKE %s))
AND doctors.doc_user LIKE %s
AND locations.loc_province LIKE %s
AND doctors.doc_specialization LIKE %s
AND doctors.doc_activity LIKE %s
ORDER BY %s
All the %s are parameters in a sprintf() PHP function
The most important thing to notice is... that I have NO indexes on MySQL! I presume that I can speed up the process adding some indexes... but what and where? There are so many joins and search parameters that I am in confusion about what would be efficient :-)
Please can you help?
Thanks in advance!
You can start with adding indexes on those columns you are using in the where condition.
Further, you should index those fields which are used in join, i.e. primary keys and foreign keys column.
I would suggest that gradually experimenting with these indexes would yield a real performance boost.
Further, I have observed that you are fetching too much of data in a single query. If it is really not required, break it up in different reports and pages (If possible) as even if you do proper indexing, the solution will not be quite scalable and may not handle large amount of data.
Note: You might have to create full text index on fields which you query by '%' qualifier.(i.e. use LIKE operator)
Like operators are pretty slow. Here is a discussion of applying indexes and FULL TEXT
mysql like performance boost