I am having trouble figuring our how I can get results only when products.published, product_types.published, and product_cats.published = 1 but my query isn't working. Please help:
SELECT
`products`.`title`,
`products`.`menu_id`,
`products`.`short_description`,
`products`.`datasheet_icon`,
`products`.`datasheet`,
`products`.`ordering`,
`products`.`product_type_id`,
CASE WHEN CHAR_LENGTH(`products`.`alias`)
THEN CONCAT_WS(':', `products`.`id`, `products`.`alias`)
ELSE `products`.`id`
END AS slug
FROM
`products`,
`product_cats`,
`product_types`
WHERE
`products`.published=1 AND
`product_cats`.published=1 AND
`product_types`.published=1 AND
`products`.`product_cat_id`='42' AND
`product_types`.`id` IN (1,40,48,49,50)
GROUP BY `products`.`id`
ORDER BY `product_types`.`ordering`, `products`.`ordering`
I want to assume tables product_cats and product_types have product ids in them as well. And I call them pid in this:
SELECT
p.title,
p.menu_id,
p.short_description,
p.datasheet_icon,
p.datasheet,
p.ordering,
p.product_type_id,
CASE
WHEN CHAR_LENGTH(p.alias)
THEN CONCAT_WS(':', p.id, p.alias)
ELSE p.id
END AS slug
FROM products p
JOIN product_cats pc ON pc.pid = p.id
JOIN product_types pt ON pt.pid = p.id
WHERE
p.published=1 AND
pc.published=1 AND
pt.published=1
GROUP BY p.id
ORDER BY pt.ordering,p.ordering
You need join tables!
FROM
`products`,
`product_cats`,
`product_types`
Use relational fields to do it and your problem will be gone!
I'm afraid your query is a bit of a mess. Without the table structures we can only guess at what you're trying to do. The critical information is how the three tables are related to each other.
Note the following:
You are using three tables in your SELECT, but are not JOINing them. You will need to explicitly JOIN the tables you use. The lack of explicit JOINs is the reason you're getting too many rows back and are having to use GROUP BY to eliminate duplicates. Your final solution should not use GROUP BY.
If you're only searching for product.cat_id of 42, I presume you know whether than cat_id is published and you don't need to involve the product_cats table. Is that correct?
Presumably there's a column product.type_id or something similar. Since you are searching for a limited number of these, do you know in advance that the ids in that list are published?
Related
I'm really struggling with this query and I hope somebody can help.
I am querying across multiple tables to get the dataset that I require. The following query is an anonymised version:
SELECT main_table.id,
sub_table_1.field_1,
main_table.field_1,
main_table.field_2,
main_table.field_3,
main_table.field_4,
main_table.field_5,
main_table.field_6,
main_table.field_7,
sub_table_2.field_1,
sub_table_2.field_2,
sub_table_2.field_3,
sub_table_3.field_1,
sub_table_4.field_1,
sub_table_4.field_2
FROM main_table
INNER JOIN sub_table_4 ON sub_table_4.id = main_table.id
INNER JOIN sub_table_2 ON sub_table_2.id = main_table.id
INNER JOIN sub_table_3 ON sub_table_3.id = main_table.id
INNER JOIN sub_table_1 ON sub_table_1.id = main_table.id
WHERE sub_table_4.field_1 = '' AND sub_table_4.field_2 = '0' AND sub_table_2.field_1 != ''
The query works, the problem I have is sub_table_1 has a revision number (int 11). Currently I get duplicate records with different revision numbers and different versions of sub_table_1.field_1 which is to be expected, but I want to limit the result set to only include results limited by the latest revision number, giving me only the latest sub_table_1_field_1 and I really can not figure it out!
Can anybody lend me a hand?
Many Thanks.
It's always important to remember that a JOIN can be on a subquery as well as a table. You could build a subquery that returns the results you want to see then, once you've got the data you want, join it in the parent query.
It's hard to 'tailor' an answer that's specific to you problem, as it's too obfuscated (as you admit) to know what the data and tables really look like, but as an example:
Say table1 has four fields: id, revision_no, name and stuff. You want to return a distinct list of name values, with their latest version of stuff (which, we'll pretend varies by revision). You could do this in isolation as:
select t.* from table1 t
inner join
(SELECT name, max(revision_no) maxr
FROM table1
GROUP BY name) mx
on mx.name = t.name
and mx.maxr = t.revision_no;
(Note: see fiddle at the end)
That would return each individual name with the latest revision of stuff.
Once you've got that nailed down, you could then swap out
INNER JOIN sub_table_1 ON sub_table_1.id = main_table.id
....with....
INNER JOIN (select t.* from table1 t
inner join
(SELECT name, max(revision_no) maxr
FROM table1
GROUP BY name) mx
on mx.name = t.name
and mx.maxr = t.revision_no) sub_table_1
ON sub_table_1.id = main_table.id
...which would allow a join with a recordset that is more tailored to that which you want to join (again, don't get hung up on the actual query I've used, it's just there to demonstrate the method).
There may well be more elegant ways to achieve this, but it's sometimes good to start with a simple solution that's easier to replicate, then simplify it once you've got the general understanding of the what and why nailed down.
Hope that helps - as I say, it's as specific as I could offer without having an idea of the real data you're using.
(for the sake of reference, here is a fiddle with a working version of the above example query)
In your case where you only need one column from the table, make this a subquery in your select clause instead of than a join. You get the latest revision by ordering by revision number descending and limiting the result to one row.
SELECT
main_table.id,
(
select sub_table_1.field_1
from sub_table_1
where sub_table_1.id = main_table.id
order by revision_number desc
limit 1
) as sub_table_1_field_1,
main_table.field_1,
...
FROM main_table
INNER JOIN sub_table_4 ON sub_table_4.id = main_table.id
INNER JOIN sub_table_2 ON sub_table_2.id = main_table.id
INNER JOIN sub_table_3 ON sub_table_3.id = main_table.id
WHERE sub_table_4.field_1 = ''
AND sub_table_4.field_2 = '0'
AND sub_table_2.field_1 != '';
Hopefully this is the right forum, my question seems to overlap the stack exchange community so this seemed best.
I have some custom reports for my WooCommerce orders on my wordpress site. I have one query that is just freezing locally, meaning in my localhost my CPU goes to 100% and it never finishes and I don't understand why. To the point here is the query:
SELECT SUM(postmeta.meta_value)
FROM pca_postmeta AS postmeta
LEFT JOIN pca_woocommerce_order_items AS orders ON orders.order_id = postmeta.post_id
WHERE postmeta.meta_key = '_order_total'
AND orders.order_item_id IN (
SELECT item_meta.order_item_id
FROM pca_woocommerce_order_itemmeta AS item_meta
LEFT JOIN pca_woocommerce_order_items AS orders ON item_meta.order_item_id = orders.order_item_id
LEFT JOIN pca_posts AS posts ON posts.ID = orders.order_id
WHERE item_meta.meta_value = '23563'
AND posts.post_status IN ('wc-processing','wc-completed')
GROUP BY orders.order_id
)
As you can hopefully see the goal here is to get the summation of all orders from this specific campaign (23563). The nested query works exactly as expected on its own, returning just a list of IDs like so:
NOTE: little curious if 2.6289 secs is long when it only returned 65 total, although there are 148220 total
The problem is this query doesn't seem to like the nested part. Any suggestions? Completely different approach in mind?
P.S. I use that nested query at other times as well to represent all orders by campaign id in my php reporting class. But for my question PHP has nothing to do with it.
UPDATE/FOLLOW UP:
Is it possible to convert this into a join as described here: Using a SELECT statement within a WHERE clause ? I'm a little light on my SQL so not sure how I would do that but it seems promising
GROUP BY orders.order_id
does not make sense because you are selecting only order_item_id.
pca_woocommerce_order_itemmeta would benefit from
INDEX(meta_value, order_item_id)
An this might be an equivalent query, but avoiding the IN(SELECT...):
SELECT SUM(pm.meta_value)
FROM
( SELECT im.order_item_id
FROM pca_woocommerce_order_itemmeta AS im
LEFT JOIN pca_woocommerce_order_items AS o
ON im.order_item_id = o.order_item_id
LEFT JOIN pca_posts AS posts ON posts.ID = o.order_id
WHERE im.meta_value = '23563'
AND posts.post_status IN ('wc-processing','wc-completed')
GROUP BY o.order_id
) AS w
JOIN pca_woocommerce_order_items AS o ON w.order_item_id = o.order_item_id
JOIN pca_postmeta AS pm ON o.order_id = pm.post_id
WHERE pm.meta_key = '_order_total'
Edit
Some principles behind what I did. Here I am guessing at what the optimizer will do with various possible formulations of the query.
I got rid of LEFT -- This may have changed the output. But I needed to avoid LEFT JOIN ( SELECT ... ) which would not be optimizable.
By having one subquery in the list of "tables" being JOINed, the optimizer will (almost certainly) start with the subquery and do "Nested Loop Joins" to the other tables. NLJ is the common way to perform a query.
A subselect like that has no index, so it needs to be first in the order, else it will be very inefficient.
Without subqueries, the optimizer generally likes to start with whichever table has something in the WHERE clause.
The requirement to start with the subquery "table" is stronger than the desire to pick the table based on WHERE pm.meta_key = '_order_total'.
Inside the subquery, the only "=" test (WHERE im.meta_value = '23563) provides the likely starting point for that set of JOINs. This is further enhanced by it not being 'right' of a LEFT JOIN. Hence, I suggested that index.
I have a mysql query and it works fine when i use where clause, but when i donot use
where clause it gone and never gives the output and finally timeout.
Actually i have used Explain command to check the performance of the query and in both cases the Explain gives the same number of rows used in joining.
I have attached the image of output got with Explain command.
Below is the query.
I couldn't figure whats the problem here.
Any help is highly appreciated.
Thanks.
SELECT
MCI.CLIENT_ID AS CLIENT_ID, MCI.NAME AS CLIENT_NAME, MCI.PRIMARY_CONTACT AS CLIENT_PRIMARY_CONTACT,
MCI.ADDED_BY AS SP_ID, CONCAT(MUD_SP.FIRST_NAME, ' ', MUD_SP.LAST_NAME) AS SP_NAME,
MCI.FK_PROSPECT_ID AS PROSPECT_ID, MCI.DATE_ADDED AS ADDED_ON,
(SELECT GROUP_CONCAT(LT.TAG_TEXT SEPARATOR ', ')
FROM LK_TAG LT
INNER JOIN M_OBJECT_TAG_MAPPING MOTM
ON LT.PK_ID = MOTM.FK_TAG_ID
WHERE MOTM.FK_OBJECT_ID = MCI.FK_PROSPECT_ID
AND MOTM.OBJECT_TYPE = 1
AND MOTM.IS_ACTIVE = 1
) AS TAGS,
IFNULL(SUM(GET_DIGITS(MMR.RCP_AMOUNT)), 0) AS REVENUE_SO_FAR,
IFNULL(SUM(GET_DIGITS(MMR.RCP_RUPEES)), 0) AS REVENUE_INR,
COUNT(DISTINCT PMI_MONTHLY.PROJECT_ID) AS MONTHLY,
COUNT(DISTINCT PMI_FIXED.PROJECT_ID) AS FIXED,
COUNT(DISTINCT PMI_HOURLY.PROJECT_ID) AS HOURLY,
COUNT(DISTINCT PMI_ANNUAL.PROJECT_ID) AS ANNUAL,
COUNT(DISTINCT PMI_CURRENTLY_RUNNING.PROJECT_ID) AS CURRENTLY_RUNNING_PROJECTS,
COUNT(DISTINCT PMI_YET_TO_START.PROJECT_ID) AS YET_TO_START_PROJECTS,
COUNT(DISTINCT PMI_TECH_SALES_CLOSED.PROJECT_ID) AS TECH_SALES_CLOSED_PROJECTS
FROM
M_CLIENT_INFO MCI
INNER JOIN M_USER_DETAILS MUD_SP
ON MCI.ADDED_BY = MUD_SP.PK_ID
LEFT OUTER JOIN M_MONTH_RECEIPT MMR
ON MMR.CLIENT_ID = MCI.CLIENT_ID
LEFT OUTER JOIN M_PROJECT_INFO PMI_FIXED
ON PMI_FIXED.CLIENT_ID = MCI.CLIENT_ID AND PMI_FIXED.PROJECT_TYPE = 1
LEFT OUTER JOIN M_PROJECT_INFO PMI_MONTHLY
ON PMI_MONTHLY.CLIENT_ID = MCI.CLIENT_ID AND PMI_MONTHLY.PROJECT_TYPE = 2
LEFT OUTER JOIN M_PROJECT_INFO PMI_HOURLY
ON PMI_HOURLY.CLIENT_ID = MCI.CLIENT_ID AND PMI_HOURLY.PROJECT_TYPE = 3
LEFT OUTER JOIN M_PROJECT_INFO PMI_ANNUAL
ON PMI_ANNUAL.CLIENT_ID = MCI.CLIENT_ID AND PMI_ANNUAL.PROJECT_TYPE = 4
LEFT OUTER JOIN M_PROJECT_INFO PMI_CURRENTLY_RUNNING
ON PMI_CURRENTLY_RUNNING.CLIENT_ID = MCI.CLIENT_ID AND PMI_CURRENTLY_RUNNING.STATUS = 4
LEFT OUTER JOIN M_PROJECT_INFO PMI_YET_TO_START
ON PMI_YET_TO_START.CLIENT_ID = MCI.CLIENT_ID AND PMI_YET_TO_START.STATUS < 4
LEFT OUTER JOIN M_PROJECT_INFO PMI_TECH_SALES_CLOSED
ON PMI_TECH_SALES_CLOSED.CLIENT_ID = MCI.CLIENT_ID AND PMI_TECH_SALES_CLOSED.STATUS > 4
WHERE YEAR(MCI.DATE_ADDED) = '2012'
GROUP BY MCI.CLIENT_ID ORDER BY CLIENT_NAME ASC
Yes, as many people have said, the key is that when you have the where clause, mysql engine filters the table M_CLIENT_INFO --probably drammatically--.
A similar result as removing the where clause is to to add this where clause:
where 1 = 1
You will see that the performance is degraded also because mysql will try to get all the data.
Remove the where clause and all columns from select and add a count to see how many records you get. If it is reasonable, say up to 10k, then do the following,
put back the select columns related to M_CLIENT_INFO
do not include the nested one "TAGS"
remove all your joins
run your query without where clause and gradually include the joins
this way you'll find out when the timeout is caused.
I would try the following. First, MySQL has a keyword "STRAIGHT_JOIN" which tells the optimizer to do the query in the table order you've specified. Since all you left-joins are child-related (like a lookup table), you don't want MySQL to try and interpret one of those as a primary basis of the query.
SELECT STRAIGHT_JOIN ... rest of query.
Next, your M_PROJECT_INFO table, I dont know how many columns of data are out there, but you appear to be concentrating on just a few columns on your DISTINCT aggregates. I would make sure you have a covering index on these elements to help the query via an index on
( Client_ID, Project_Type, Status, Project_ID )
This way the engine can apply the criteria and get the distinct all out of the index instead of having to go back to the raw data pages for the query.
Third, your M_CLIENT_INFO table. Ensure that has an index on both your criteria, group by AND your Order By, and change your order by from the aliased "CLIENT_NAME" to the actual column of the SQL table so it matches the index
( Date_Added, Client_ID, Name )
I have "name" in ticks as it is also a reserved word and helps clarify the column, not the keyword.
Next, the WHERE clause. Whenever you apply a function to an indexed column name, it doesn't work the greatest, especially on date/time fields... You might want to change your where clause to
WHERE MCI.Date_Added between '2012-01-01' and '2012-12-31 23:59:59'
so the BETWEEN range is showing the entire year and the index can better be utilized.
Finally, if the above do not help, I would consider splitting your query some. The GROUP_CONCACT inline select for the TAGS might be a bit of a killer for you. You might want to have all the distinct elements first for the grouping per client, THEN get those details.... Something like
select
PQ.*,
group_concat(...) tags
from
( the entire primary part of the query ) as PQ
Left join yourGroupConcatTableBasis on key columns
I have a query like this:
SELECT product.id,
GROUP_CONCAT(image.id) AS images_id,
GROUP_CONCAT(image.title) AS images_title,
GROUP_CONCAT(facet.id) AS facets_id
...
GROUP BY product.id
And the query works, but not as expected, because if I have a product with 5 facets and 1 image (suppose an image with id=7), then I get something like this in "images_id":
"7,7,7,7,7"
If I have 2 images (7 and 3) then I get something like:
"7,7,7,7,7,3,3,3,3,3"
and in facets I get something like:
"8,7,6,5,4,8,7,6,5,4"
I think MySQL is making some type of union of the differents rows returned by the query, and then concatenating everything.
My expected result is (for the last example):
images_id = "7,3"
facets_id = "8,7,6,5,4"
I can obtain that using DISTINCT in the GROUP_CONCAT, but then I have another problem:
If I have two images with the same title, one of them is ommited, and then I get something like:
images_id = "7,3,5"
images_title = "Title7and3,Title5"
So I miss the relation between images_id and images_title.
Does somebody know if it's possible to make this query in MySQL?
Maybe I'm complicating everything without any real benefits.
I'm trying to execute only one query because performance, but now I'm not so sure if it's even faster to execute two queries (one for selecting the facets and another for the images for example).
Please explain what do you think is the best solution for this and why.
Thanks !
Just add DISTINCT.
Example:
GROUP_CONCAT(DISTINCT image.id) AS images_id
You'll need to get each group separately:
SELECT
p.id,
images_id,
images_title,
facets_id,
...
FROM PRODUCT p
JOIN (SELECT product.id, GROUP_CONCAT(image.id) AS images_id
FROM PRODUCT GROUP BY product.id) a on a.id = p.id
JOIN (SELECT product.id, GROUP_CONCAT(image.title) AS images_title
FROM PRODUCT GROUP BY product.id) b on b.id = p.id
JOIN (SELECT product.id, GROUP_CONCAT(facet.id) AS facets_id
FROM PRODUCT GROUP BY product.id) b on c.id = p.id
...
You can add just the DISTINCT keyword, you'll get your desire results.
SELECT tb_mod.*, tb_van.*,
GROUP_CONCAT(DISTINCT tb_voil.vt_id) AS voil,
GROUP_CONCAT(DISTINCT tb_other.oa_id) AS other,
GROUP_CONCAT(DISTINCT tb_ref.rp_id) AS referral
FROM cp_modules_record_tbl tb_mod
LEFT JOIN cp_vane_police_tbl tb_van ON tb_van.mr_id= tb_mod.id
LEFT JOIN cp_mod_voilt_tbl tb_voil ON tb_voil.mr_id= tb_mod.id
LEFT JOIN cp_mod_otheraction_tbl tb_other ON tb_other.mr_id= tb_mod.id
LEFT JOIN cp_mod_referral_tbl tb_ref ON tb_ref.mr_id= tb_mod.id
WHERE tb_mod.mod_type = 2 GROUP BY tb_mod.id
If the issue is speed, then it may be a lot faster to simply select all the data you need as separate rows, and do the grouping in the application, i.e.:
SELECT product.id, image.id, image.title, facet.id
Then in the application:
foreach row:
push product_id onto list_of_product_ids
push image_id onto list_of_image_ids
etc.
I am having following database schema, I want to fetch name of all categories with no of quotes related to that category . The query that i wrote giving me one row only can u please tell me the resource efficient query.
SELECT SC.Name, Count(*) AS Quotes
FROM status_categories AS SC
INNER JOIN status_quotes AS SQ ON SC._id = SQ._category_id
GROUP BY SC.Name
SELECT status_categories.NAME, COUNT(status_quotes.category_id)
FROM status_categories JOIN status_quotes ON status_categories._id = status_quotes.category_id
GROUP BY status_categories._id;
Try the following:
SELECT `c`.`name`, COUNT(*) AS `Number of quotes`
FROM `status_categories` AS `c`
INNER JOIN `status_quotes` AS `q`
ON `q`.`category_id` = `c`.`_id`
GROUP BY `c`.`_id`;
EDIT
Feel free to leave out the ` character. But that is the safe way of doing it, even though it looks a bit nasty.