I have a query like this:
SELECT product.id,
GROUP_CONCAT(image.id) AS images_id,
GROUP_CONCAT(image.title) AS images_title,
GROUP_CONCAT(facet.id) AS facets_id
...
GROUP BY product.id
And the query works, but not as expected, because if I have a product with 5 facets and 1 image (suppose an image with id=7), then I get something like this in "images_id":
"7,7,7,7,7"
If I have 2 images (7 and 3) then I get something like:
"7,7,7,7,7,3,3,3,3,3"
and in facets I get something like:
"8,7,6,5,4,8,7,6,5,4"
I think MySQL is making some type of union of the differents rows returned by the query, and then concatenating everything.
My expected result is (for the last example):
images_id = "7,3"
facets_id = "8,7,6,5,4"
I can obtain that using DISTINCT in the GROUP_CONCAT, but then I have another problem:
If I have two images with the same title, one of them is ommited, and then I get something like:
images_id = "7,3,5"
images_title = "Title7and3,Title5"
So I miss the relation between images_id and images_title.
Does somebody know if it's possible to make this query in MySQL?
Maybe I'm complicating everything without any real benefits.
I'm trying to execute only one query because performance, but now I'm not so sure if it's even faster to execute two queries (one for selecting the facets and another for the images for example).
Please explain what do you think is the best solution for this and why.
Thanks !
Just add DISTINCT.
Example:
GROUP_CONCAT(DISTINCT image.id) AS images_id
You'll need to get each group separately:
SELECT
p.id,
images_id,
images_title,
facets_id,
...
FROM PRODUCT p
JOIN (SELECT product.id, GROUP_CONCAT(image.id) AS images_id
FROM PRODUCT GROUP BY product.id) a on a.id = p.id
JOIN (SELECT product.id, GROUP_CONCAT(image.title) AS images_title
FROM PRODUCT GROUP BY product.id) b on b.id = p.id
JOIN (SELECT product.id, GROUP_CONCAT(facet.id) AS facets_id
FROM PRODUCT GROUP BY product.id) b on c.id = p.id
...
You can add just the DISTINCT keyword, you'll get your desire results.
SELECT tb_mod.*, tb_van.*,
GROUP_CONCAT(DISTINCT tb_voil.vt_id) AS voil,
GROUP_CONCAT(DISTINCT tb_other.oa_id) AS other,
GROUP_CONCAT(DISTINCT tb_ref.rp_id) AS referral
FROM cp_modules_record_tbl tb_mod
LEFT JOIN cp_vane_police_tbl tb_van ON tb_van.mr_id= tb_mod.id
LEFT JOIN cp_mod_voilt_tbl tb_voil ON tb_voil.mr_id= tb_mod.id
LEFT JOIN cp_mod_otheraction_tbl tb_other ON tb_other.mr_id= tb_mod.id
LEFT JOIN cp_mod_referral_tbl tb_ref ON tb_ref.mr_id= tb_mod.id
WHERE tb_mod.mod_type = 2 GROUP BY tb_mod.id
If the issue is speed, then it may be a lot faster to simply select all the data you need as separate rows, and do the grouping in the application, i.e.:
SELECT product.id, image.id, image.title, facet.id
Then in the application:
foreach row:
push product_id onto list_of_product_ids
push image_id onto list_of_image_ids
etc.
Related
Short setup
consider the following.
SELECT forum_category.groupid,
forum_category.categoryid,
forum_category.categoryname,
forum_category.categorydescription,
forum_category.category_url,
forum_category.accesslevel ,
COUNT(DISTINCT forum_topic.topicid) AS topics ,
COUNT(DISTINCT forum_post.postid) AS posts
FROM forum_category
INNER JOIN forum_topic ON forum_topic.categoryid=forum_category.categoryid
INNER JOIN forum_post ON forum_post.topicid=forum_topic.topicid
WHERE groupid = 1
result
This gives me actually one result, while i expect multiple rows (in this case 2) to come back. What am I missing here?
I want to get the total numbers of committed crimes when combining the two tables.
But I want to count the numbers for each crime being committed and also display the values of those that has not being committed as 0, How can i achieve this using mysql?
my code:
SELECT count(offense_id)
AS totalnumber,(
select offense_description
from offense
where offense.offense_id = case_crime.offense_id
)as crimeName
from case_crime
group by offense_id
To get a list of all your offenses and the count of registered crimes, you need to use the LEFT JOIN between the two tables .... something like this SQL:
SELECT a.offense_id, a.offense_description, count(b.crime_caseid) as total
FROM offense a LEFT JOIN case_crime b
ON a.offense_id = b.offense_id
group by a.offense_id;
Here is a SQLFiddle to play around with :)
You are looking for a LEFT OUTER JOIN
Something like
SELECT offense.offense_id
, offense_description
, count(case_crime.case_id) as Total_Number
from offense
LEFT OUTER JOIN case_crime ON offense.offense_id = case_crime.offense_id
group by offense.offense_id
I'll admit, I'm a TSQL guy, so I would handle the COUNT(*) returning null when there are no cases with the specified offense using: ISNULL(COUNT(case_crime),0)
Other SQL variants might use COALESCE( COUNT(case_crime), 0)
I have a mysql query and it works fine when i use where clause, but when i donot use
where clause it gone and never gives the output and finally timeout.
Actually i have used Explain command to check the performance of the query and in both cases the Explain gives the same number of rows used in joining.
I have attached the image of output got with Explain command.
Below is the query.
I couldn't figure whats the problem here.
Any help is highly appreciated.
Thanks.
SELECT
MCI.CLIENT_ID AS CLIENT_ID, MCI.NAME AS CLIENT_NAME, MCI.PRIMARY_CONTACT AS CLIENT_PRIMARY_CONTACT,
MCI.ADDED_BY AS SP_ID, CONCAT(MUD_SP.FIRST_NAME, ' ', MUD_SP.LAST_NAME) AS SP_NAME,
MCI.FK_PROSPECT_ID AS PROSPECT_ID, MCI.DATE_ADDED AS ADDED_ON,
(SELECT GROUP_CONCAT(LT.TAG_TEXT SEPARATOR ', ')
FROM LK_TAG LT
INNER JOIN M_OBJECT_TAG_MAPPING MOTM
ON LT.PK_ID = MOTM.FK_TAG_ID
WHERE MOTM.FK_OBJECT_ID = MCI.FK_PROSPECT_ID
AND MOTM.OBJECT_TYPE = 1
AND MOTM.IS_ACTIVE = 1
) AS TAGS,
IFNULL(SUM(GET_DIGITS(MMR.RCP_AMOUNT)), 0) AS REVENUE_SO_FAR,
IFNULL(SUM(GET_DIGITS(MMR.RCP_RUPEES)), 0) AS REVENUE_INR,
COUNT(DISTINCT PMI_MONTHLY.PROJECT_ID) AS MONTHLY,
COUNT(DISTINCT PMI_FIXED.PROJECT_ID) AS FIXED,
COUNT(DISTINCT PMI_HOURLY.PROJECT_ID) AS HOURLY,
COUNT(DISTINCT PMI_ANNUAL.PROJECT_ID) AS ANNUAL,
COUNT(DISTINCT PMI_CURRENTLY_RUNNING.PROJECT_ID) AS CURRENTLY_RUNNING_PROJECTS,
COUNT(DISTINCT PMI_YET_TO_START.PROJECT_ID) AS YET_TO_START_PROJECTS,
COUNT(DISTINCT PMI_TECH_SALES_CLOSED.PROJECT_ID) AS TECH_SALES_CLOSED_PROJECTS
FROM
M_CLIENT_INFO MCI
INNER JOIN M_USER_DETAILS MUD_SP
ON MCI.ADDED_BY = MUD_SP.PK_ID
LEFT OUTER JOIN M_MONTH_RECEIPT MMR
ON MMR.CLIENT_ID = MCI.CLIENT_ID
LEFT OUTER JOIN M_PROJECT_INFO PMI_FIXED
ON PMI_FIXED.CLIENT_ID = MCI.CLIENT_ID AND PMI_FIXED.PROJECT_TYPE = 1
LEFT OUTER JOIN M_PROJECT_INFO PMI_MONTHLY
ON PMI_MONTHLY.CLIENT_ID = MCI.CLIENT_ID AND PMI_MONTHLY.PROJECT_TYPE = 2
LEFT OUTER JOIN M_PROJECT_INFO PMI_HOURLY
ON PMI_HOURLY.CLIENT_ID = MCI.CLIENT_ID AND PMI_HOURLY.PROJECT_TYPE = 3
LEFT OUTER JOIN M_PROJECT_INFO PMI_ANNUAL
ON PMI_ANNUAL.CLIENT_ID = MCI.CLIENT_ID AND PMI_ANNUAL.PROJECT_TYPE = 4
LEFT OUTER JOIN M_PROJECT_INFO PMI_CURRENTLY_RUNNING
ON PMI_CURRENTLY_RUNNING.CLIENT_ID = MCI.CLIENT_ID AND PMI_CURRENTLY_RUNNING.STATUS = 4
LEFT OUTER JOIN M_PROJECT_INFO PMI_YET_TO_START
ON PMI_YET_TO_START.CLIENT_ID = MCI.CLIENT_ID AND PMI_YET_TO_START.STATUS < 4
LEFT OUTER JOIN M_PROJECT_INFO PMI_TECH_SALES_CLOSED
ON PMI_TECH_SALES_CLOSED.CLIENT_ID = MCI.CLIENT_ID AND PMI_TECH_SALES_CLOSED.STATUS > 4
WHERE YEAR(MCI.DATE_ADDED) = '2012'
GROUP BY MCI.CLIENT_ID ORDER BY CLIENT_NAME ASC
Yes, as many people have said, the key is that when you have the where clause, mysql engine filters the table M_CLIENT_INFO --probably drammatically--.
A similar result as removing the where clause is to to add this where clause:
where 1 = 1
You will see that the performance is degraded also because mysql will try to get all the data.
Remove the where clause and all columns from select and add a count to see how many records you get. If it is reasonable, say up to 10k, then do the following,
put back the select columns related to M_CLIENT_INFO
do not include the nested one "TAGS"
remove all your joins
run your query without where clause and gradually include the joins
this way you'll find out when the timeout is caused.
I would try the following. First, MySQL has a keyword "STRAIGHT_JOIN" which tells the optimizer to do the query in the table order you've specified. Since all you left-joins are child-related (like a lookup table), you don't want MySQL to try and interpret one of those as a primary basis of the query.
SELECT STRAIGHT_JOIN ... rest of query.
Next, your M_PROJECT_INFO table, I dont know how many columns of data are out there, but you appear to be concentrating on just a few columns on your DISTINCT aggregates. I would make sure you have a covering index on these elements to help the query via an index on
( Client_ID, Project_Type, Status, Project_ID )
This way the engine can apply the criteria and get the distinct all out of the index instead of having to go back to the raw data pages for the query.
Third, your M_CLIENT_INFO table. Ensure that has an index on both your criteria, group by AND your Order By, and change your order by from the aliased "CLIENT_NAME" to the actual column of the SQL table so it matches the index
( Date_Added, Client_ID, Name )
I have "name" in ticks as it is also a reserved word and helps clarify the column, not the keyword.
Next, the WHERE clause. Whenever you apply a function to an indexed column name, it doesn't work the greatest, especially on date/time fields... You might want to change your where clause to
WHERE MCI.Date_Added between '2012-01-01' and '2012-12-31 23:59:59'
so the BETWEEN range is showing the entire year and the index can better be utilized.
Finally, if the above do not help, I would consider splitting your query some. The GROUP_CONCACT inline select for the TAGS might be a bit of a killer for you. You might want to have all the distinct elements first for the grouping per client, THEN get those details.... Something like
select
PQ.*,
group_concat(...) tags
from
( the entire primary part of the query ) as PQ
Left join yourGroupConcatTableBasis on key columns
I am having trouble figuring our how I can get results only when products.published, product_types.published, and product_cats.published = 1 but my query isn't working. Please help:
SELECT
`products`.`title`,
`products`.`menu_id`,
`products`.`short_description`,
`products`.`datasheet_icon`,
`products`.`datasheet`,
`products`.`ordering`,
`products`.`product_type_id`,
CASE WHEN CHAR_LENGTH(`products`.`alias`)
THEN CONCAT_WS(':', `products`.`id`, `products`.`alias`)
ELSE `products`.`id`
END AS slug
FROM
`products`,
`product_cats`,
`product_types`
WHERE
`products`.published=1 AND
`product_cats`.published=1 AND
`product_types`.published=1 AND
`products`.`product_cat_id`='42' AND
`product_types`.`id` IN (1,40,48,49,50)
GROUP BY `products`.`id`
ORDER BY `product_types`.`ordering`, `products`.`ordering`
I want to assume tables product_cats and product_types have product ids in them as well. And I call them pid in this:
SELECT
p.title,
p.menu_id,
p.short_description,
p.datasheet_icon,
p.datasheet,
p.ordering,
p.product_type_id,
CASE
WHEN CHAR_LENGTH(p.alias)
THEN CONCAT_WS(':', p.id, p.alias)
ELSE p.id
END AS slug
FROM products p
JOIN product_cats pc ON pc.pid = p.id
JOIN product_types pt ON pt.pid = p.id
WHERE
p.published=1 AND
pc.published=1 AND
pt.published=1
GROUP BY p.id
ORDER BY pt.ordering,p.ordering
You need join tables!
FROM
`products`,
`product_cats`,
`product_types`
Use relational fields to do it and your problem will be gone!
I'm afraid your query is a bit of a mess. Without the table structures we can only guess at what you're trying to do. The critical information is how the three tables are related to each other.
Note the following:
You are using three tables in your SELECT, but are not JOINing them. You will need to explicitly JOIN the tables you use. The lack of explicit JOINs is the reason you're getting too many rows back and are having to use GROUP BY to eliminate duplicates. Your final solution should not use GROUP BY.
If you're only searching for product.cat_id of 42, I presume you know whether than cat_id is published and you don't need to involve the product_cats table. Is that correct?
Presumably there's a column product.type_id or something similar. Since you are searching for a limited number of these, do you know in advance that the ids in that list are published?
hi I am doing A query to get some product info, but there is something strange going on, the first query returns resultset fast (.1272s) but the second (note that I just added 1 column) takes forever to complete (28-35s), anyone know what is happening?
query 1
SELECT
p.partnumberp,
p.model,
p.descriptionsmall,
p.brandname,
sum(remainderint) stockint
from
inventario_dbo.inventoryindetails ind
left join purchaseorders.product p on (p.partnumberp = ind.partnumberp)
left join inventario_dbo.inventoryin ins on (ins.inventoryinid= ind.inventoryinid)
group by partnumberp, projectid
query 2
SELECT
p.partnumberp,
p.model,
p.descriptionsmall,
p.brandname,
p.descriptiondetail,
sum(remainderint) stockint
from
inventario_dbo.inventoryindetails inda
left join purchaseorders.product p on (p.partnumberp = inda.partnumberp)
left join inventario_dbo.inventoryin ins on (ins.inventoryinid= inda.inventoryinid)
group by partnumberp, projectid
You shouldn't group by some columns and then select other columns unless you use aggregate functions. Only p.partnumberp and sum(remainderint) make sense here. You're doing a huge join and select and then the results for most rows just end up getting discarded.
You can make the query much faster by doing an inner select first and then joining that to the remaining tables to get your final result for the last few columns.
The inner select should look something like this:
select p.partnumberp, projectid, sum(remainderint) stockint
from inventario_dbo.inventoryindetails ind
left join purchaseorders.product p on (p.partnumberp = ind.partnumberp)
left join inventario_dbo.inventoryin ins on (ins.inventoryinid = ind.inventoryinid)
group by partnumberp, projectid
After the join:
select T1.partnumberp, T1.projectid, p2.model, p2.descriptionsmall, p2.brandname, T1.stockint
from
(select p.partnumberp, projectid, sum(remainderint) stockint
from inventario_dbo.inventoryindetails ind
left join purchaseorders.product p on (p.partnumberp = ind.partnumberp)
left join inventario_dbo.inventoryin ins on (ins.inventoryinid = ind.inventoryinid)
group by partnumberp, projectid) T1
left join purchaseorders.product p2 on (p2.partnumberp = T1.partnumberp)
Is descriptiondetail a really large column? Sounds like it could be a lot of text compared to the other fields based on its name, so maybe it just takes a lot more time to read from disk, but if you could post the schema detail for the purchaseorders.product table or maybe the average length of that column that would help.
Otherswise I would try running the query a few times and see you consistently get the same time results. Could just be load on the database server the time you got the slower result.