I want to do the query directly a Magento database in MySQL but it gives me duplicates. Could you please help me?
SELECT DISTINCT
`catalog_product_entity`.`sku`
, `catalog_product_flat_1`.`name`
, `catalog_product_entity_text`.`value` AS `description`
, `catalog_product_flat_1`.`url_key`
, `catalog_product_flat_1`.`small_image`
, `catalog_product_flat_1`.`price`
, `catalog_product_flat_1`.`special_price`
, `catalog_product_flat_1`.`designer_value`
, `catalog_product_flat_1`.`color_value`
FROM
`ac_magento_gold`.`catalog_product_flat_1`
INNER JOIN `ac_magento_gold`.`catalog_product_entity`
ON (`catalog_product_flat_1`.`entity_id` = `catalog_product_entity`.`entity_id`) AND (`catalog_product_entity`.`sku` = `catalog_product_flat_1`.`sku`) AND (`catalog_product_flat_1`.`sku` NOT REGEXP '(SZ|SIZE|GIFT)')
INNER JOIN `ac_magento_gold`.`catalog_product_entity_text`
ON (`catalog_product_entity_text`.`entity_id` = `catalog_product_entity`.`entity_id`) AND (`catalog_product_entity_text`.`attribute_id`= 61)
INNER JOIN `ac_magento_gold`.`cataloginventory_stock_item`
ON (`cataloginventory_stock_item`.`product_id` = `catalog_product_entity`.`entity_id`) AND (`cataloginventory_stock_item`.`product_id` = `catalog_product_flat_1`.`entity_id`) AND (`catalog_product_entity_text`.`entity_id` = `cataloginventory_stock_item`.`product_id`) AND (`cataloginventory_stock_item`.is_in_stock = 1) LIMIT 6;
If you have multiple websites or stores (or even stock_id), that could be the reason you are getting duplicates. You need to specify the relevant store_id or website_id or stock_id in your joins.
For example,
INNER JOIN `ac_magento_gold`.`catalog_product_entity_text`
ON (`catalog_product_entity_text`.`entity_id` = `catalog_product_entity`.`entity_id`)
AND (`catalog_product_entity_text`.`attribute_id`= 61)
should be
INNER JOIN `ac_magento_gold`.`catalog_product_entity_text`
ON (`catalog_product_entity_text`.`entity_id` = `catalog_product_entity`.`entity_id`)
AND (`catalog_product_entity_text`.`attribute_id`= 61)
AND `catalog_product_entity_text`.`store_id` = 0
or whatever your store_id is. Actually, the query makes even more sense (to me at least) with the parentheses rearranged like so:
INNER JOIN `ac_magento_gold`.`catalog_product_entity_text`
ON (
`catalog_product_entity_text`.`entity_id` = `catalog_product_entity`.`entity_id`
AND `catalog_product_entity_text`.`attribute_id`= 61
AND `catalog_product_entity_text``.store_id` = 0
)
You might also have to rewrite the following:
INNER JOIN `ac_magento_gold`.`cataloginventory_stock_item`
ON (`cataloginventory_stock_item`.`product_id` = `catalog_product_entity`.`entity_id`)
AND (`cataloginventory_stock_item`.`product_id` = `catalog_product_flat_1`.`entity_id`)
AND (`catalog_product_entity_text`.`entity_id` = `cataloginventory_stock_item`.`product_id`)
AND (`cataloginventory_stock_item`.is_in_stock = 1)
to:
INNER JOIN `ac_magento_gold`.`cataloginventory_stock_item`
ON (`cataloginventory_stock_item`.`product_id` = `catalog_product_entity`.`entity_id`
AND `cataloginventory_stock_item`.`product_id` = `catalog_product_flat_1`.`entity_id`
AND `catalog_product_entity_text`.`entity_id` = `cataloginventory_stock_item`.`product_id`
AND `cataloginventory_stock_item`.is_in_stock = 1
AND `cataloginventory_stock_item`.`stock_id` = 1)
again, depending on which stock_id you are interested in. As far as I know, catalog_product_entity only contains one of each entity_id, so no website_id, store_id, or stock_id has to be specified. I also believe that catalog_product_flat_1 is specific to one store_id or website_id or something, but I am not sure.
Related
I have this query which i believe can be optimized:
SELECT floors.id, floors.floor FROM floors
WHERE floors.societies_id = 1
AND floors.status = 'Y'
AND floors.id NOT IN (
SELECT DISTINCT(floors.id) FROM floors
INNER JOIN societies ON societies.id = floors.societies_id
INNER JOIN resident_floors ON resident_floors.floors_id = floors.id
WHERE societies.id = 1
AND floors.status = 'Y'
)
Is this query fine to use or there it can be improved..?
It looks like you want to get all floors that aren't present in resident_floors. For this we can left join RF in and ask for only rows where the join failed resulting in a null in RF:
SELECT floors.* FROM floors
INNER JOIN societies ON societies.id = floors.societies_id
LEFT JOIN resident_floors ON resident_floors.floors_id = floors.id
WHERE societies.id = 1
AND floors.status = 'Y'
AND resident_floors.floors_id IS NULL
I'm not sure how to make the following SQL query more efficient. Right now, the query is taking 8 - 12 seconds on a pretty fast server, but that's not close to fast enough for a Website when users are trying to load a page with this code on it. It's looking through tables with many rows, for instance the "Post" table has 717,873 rows. Basically, the query lists all Posts related to what the user is following (newest to oldest).
Is there a way to make it faster by only getting the last 20 results total based on PostTimeOrder?
Any help would be much appreciated or insight on anything that can be done to improve this situation. Thank you.
Here's the full SQL query (lots of nesting):
SELECT DISTINCT p.Id, UNIX_TIMESTAMP(p.PostCreationTime) AS PostCreationTime, p.Content AS Content, p.Bu AS Bu, p.Se AS Se, UNIX_TIMESTAMP(p.PostCreationTime) AS PostTimeOrder
FROM Post p
WHERE (p.Id IN (SELECT pc.PostId
FROM PostCreator pc
WHERE (pc.UserId IN (SELECT uf.FollowedId
FROM UserFollowing uf
WHERE uf.FollowingId = '100')
OR pc.UserId = '100')
))
OR (p.Id IN (SELECT pum.PostId
FROM PostUserMentions pum
WHERE (pum.UserId IN (SELECT uf.FollowedId
FROM UserFollowing uf
WHERE uf.FollowingId = '100')
OR pum.UserId = '100')
))
OR (p.Id IN (SELECT ssp.PostId
FROM SStreamPost ssp
WHERE (ssp.SStreamId IN (SELECT ssf.SStreamId
FROM SStreamFollowing ssf
WHERE ssf.UserId = '100'))
))
OR (p.Id IN (SELECT psm.PostId
FROM PostSMentions psm
WHERE (psm.StockId IN (SELECT sf.StockId
FROM StockFollowing sf
WHERE sf.UserId = '100' ))
))
UNION ALL
SELECT DISTINCT p.Id AS Id, UNIX_TIMESTAMP(p.PostCreationTime) AS PostCreationTime, p.Content AS Content, p.Bu AS Bu, p.Se AS Se, UNIX_TIMESTAMP(upe.PostEchoTime) AS PostTimeOrder
FROM Post p
INNER JOIN UserPostE upe
on p.Id = upe.PostId
INNER JOIN UserFollowing uf
on (upe.UserId = uf.FollowedId AND (uf.FollowingId = '100' OR upe.UserId = '100'))
ORDER BY PostTimeOrder DESC;
Changing your p.ID in (...) predicates to existence predicates with correlated subqueries may help. Also since both halves of your union all query are pulling from the Post table and possibly returning nearly identical records you might be able to combine the two into one query by left outer joining to UserPostE and adding upe.PostID is not null as an OR condition in the WHERE clause. UserFollowing will still inner join to UPE. If you want the same Post record twice once with upe.PostEchoTime and once with p.PostCreationTime as the PostTimeOrder you'll need keep the UNION ALL
SELECT
DISTINCT -- <<=- May not be needed
p.Id
, UNIX_TIMESTAMP(p.PostCreationTime) AS PostCreationTime
, p.Content AS Content
, p.Bu AS Bu
, p.Se AS Se
, UNIX_TIMESTAMP(coalesce( upe.PostEchoTime
, p.PostCreationTime)) AS PostTimeOrder
FROM Post p
LEFT JOIN UserPostE upe
INNER JOIN UserFollowing uf
on (upe.UserId = uf.FollowedId AND
(uf.FollowingId = '100' OR
upe.UserId = '100'))
on p.Id = upe.PostId
WHERE upe.PostID is not null
or exists (SELECT 1
FROM PostCreator pc
WHERE pc.PostId = p.ID
and pc.UserId = '100'
or exists (SELECT 1
FROM UserFollowing uf
WHERE uf.FollowedId = pc.UserID
and uf.FollowingId = '100')
)
OR exists (SELECT 1
FROM PostUserMentions pum
WHERE pum.PostId = p.ID
and pum.UserId = '100'
or exists (SELECT 1
FROM UserFollowing uf
WHERE uf.FollowedId = pum.UserId
and uf.FollowingId = '100')
)
OR exists (SELECT 1
FROM SStreamPost ssp
WHERE ssp.PostId = p.ID
and exists (SELECT 1
FROM SStreamFollowing ssf
WHERE ssf.SStreamId = ssp.SStreamId
and ssf.UserId = '100')
)
OR exists (SELECT 1
FROM PostSMentions psm
WHERE psm.PostId = p.ID
and exists (SELECT
FROM StockFollowing sf
WHERE sf.StockId = psm.StockId
and sf.UserId = '100' )
)
ORDER BY PostTimeOrder DESC
The from section could alternatively be rewritten to also use an existence clause with a correlated sub query:
FROM Post p
LEFT JOIN UserPostE upe
on p.Id = upe.PostId
and ( upe.UserId = '100'
or exists (select 1
from UserFollowing uf
where uf.FollwedID = upe.UserID
and uf.FollowingId = '100'))
Turn IN ( SELECT ... ) into a JOIN .. ON ... (see below)
Turn OR into UNION (see below)
Some the tables are many:many mappings? Such as SStreamFollowing? Follow the tips in http://mysql.rjweb.org/doc.php/index_cookbook_mysql#many_to_many_mapping_table
Example of IN:
SELECT ssp.PostId
FROM SStreamPost ssp
WHERE (ssp.SStreamId IN (
SELECT ssf.SStreamId
FROM SStreamFollowing ssf
WHERE ssf.UserId = '100' ))
-->
SELECT ssp.PostId
FROM SStreamPost ssp
JOIN SStreamFollowing ssf ON ssp.SStreamId = ssf.SStreamId
WHERE ssf.UserId = '100'
The big WHERE with all the INs becomes something like
JOIN ( ( SELECT pc.PostId AS id ... )
UNION ( SELECT pum.PostId ... )
UNION ( SELECT ssp.PostId ... )
UNION ( SELECT psm.PostId ... ) )
Get what you can done of that those suggestions, then come back for more advice if you still need it. And bring SHOW CREATE TABLE with you.
I have the following query which is actually within a stored procedure, but I removed it as there is too much going on inside the stored procedure. Basically this is the end result which takes ages (more than a minute) to run and I know the reason why - as you will also see from looking at the result of the explain - but I just cannot get it sorted.
Just to quickly explain what this query is doing. It is fetching all products from companies that are "connected" to the company where li.nToObjectID = 37. The result also returns some other information about the other companies like its name, company id, etc.
SELECT DISTINCT
SQL_CALC_FOUND_ROWS
p.id,
p.sTitle,
p.sTeaser,
p.TimeStamp,
p.ExpiryDate,
p.InStoreDate,
p.sCreator,
p.sProductCode,
p.nRetailPrice,
p.nCostPrice,
p.bPublic,
c.id as nCompanyID,
c.sName as sCompany,
m.id as nMID,
m.sFileName as sHighResFileName,
m.nSize,
(
Select sName
FROM tblBrand
WHERE id = p.nBrandID
) as sBrand,
(
Select t.sFileName
FROM tblThumbnail t
where t.nMediaID = m.id AND
t.sType = "thumbnail"
) as sFileName,
(
Select t.nWidth
FROM tblThumbnail t
where t.nMediaID = m.id AND
t.sType = "thumbnail"
) as nWidth,
(
Select t.nHeight
FROM tblThumbnail t
where t.nMediaID = m.id AND
t.sType = "thumbnail"
) as nHeight,
IF (
(
SELECT COUNT(id) FROM tblLink
WHERE
sType = "company"
AND sStatus = "active"
AND nToObjectID = 37
AND nFromObjectID = u.nCompanyID
),
1,
0
) AS bLinked
FROM tblProduct p
INNER JOIN tblMedia m
ON (
m.nTypeID = p.id AND
m.sType = "product"
)
INNER JOIN tblUser u
ON u.id = p.nUserID
INNER JOIN tblCompany c
ON u.nCompanyID = c.id
LEFT JOIN tblLink li
ON (
li.sType = "company"
AND li.sStatus = "active"
AND li.nToObjectID = 37
AND li.nFromObjectID = u.nCompanyID
)
WHERE c.bActive = 1
AND p.bArchive = 0
AND p.bActive = 1
AND NOW() <= p.ExpiryDate
AND (
li.id IS NOT NULL
OR (
li.id IS NULL
AND p.bPublic = 1
)
)
ORDER BY p.TimeStamp DESC
LIMIT 0, 52
Click here to see the output for EXPLAIN. Sorry, just couldn't get the formatting correct.
http://i60.tinypic.com/2hdqjgj.png
And lastly the number of rows for all the tables in this query:
tblProducts
Count: 5392
tblBrand
Count: 194
tblCompany
Count: 368
tblUser
Count: 416
tblMedia
Count: 5724
tblLink
Count: 24800
tblThumbnail
Count: 22207
So I have 2 questions:
1. Is there another way of writing this query which might potentially speed it up?
2. What index combination do I need for tblProducts so that not all the rows are searched through?
UPDATE 1
This is the new query after removing the subqueries and making use of left joins instead:
SELECT DISTINCT DISTINCT
SQL_CALC_FOUND_ROWS
p.id,
p.sTitle,
p.sTeaser,
p.TimeStamp,
p.ExpiryDate,
p.InStoreDate,
p.sCreator,
p.sProductCode,
p.nRetailPrice,
p.nCostPrice,
p.bPublic,
c.id as nCompanyID,
c.sName as sCompany,
m.id as nMID,
m.sFileName as sHighResFileName,
m.nSize,
brand.sName as sBrand,
thumb.sFilename,
thumb.nWidth,
thumb.nHeight,
IF (
(
SELECT COUNT(id) FROM tblLink
WHERE
sType = "company"
AND sStatus = "active"
AND nToObjectID = 37
AND nFromObjectID = u.nCompanyID
),
1,
0
) AS bLinked
FROM tblProduct p
INNER JOIN tblMedia m
ON (
m.nTypeID = p.id AND
m.sType = "product"
)
INNER JOIN tblUser u
ON u.id = p.nUserID
INNER JOIN tblCompany c
ON u.nCompanyID = c.id
LEFT JOIN tblLink li
ON (
li.sType = "company"
AND li.sStatus = "active"
AND li.nToObjectID = 37
AND li.nFromObjectID = u.nCompanyID
)
LEFT JOIN tblBrand AS brand
ON brand.id = p.nBrandID
LEFT JOIN tblThumbnail AS thumb
ON (
thumb.nMediaID = m.id
AND thumb.sType = 'thumbnail'
)
WHERE c.bActive = 1
AND p.bArchive = 0
AND p.bActive = 1
AND NOW() <= p.ExpiryDate
AND (
li.id IS NOT NULL
OR (
li.id IS NULL
AND p.bPublic = 1
)
)
ORDER BY p.TimeStamp DESC
LIMIT 0, 52;
UPDATE 2
ALTER TABLE tblThumbnail ADD INDEX (nMediaID,sType) USING BTREE;
ALTER TABLE tblMedia ADD INDEX (nTypeID,sType) USING BTREE;
ALTER TABLE tblProduct ADD INDEX (bArchive,bActive,ExpiryDate,bPublic,TimeStamp) USING BTREE;
After doing the above changes the explain showed that it is now only searching through 1464 rows on tblProduct instead of 5392.
That's a big query with a lot going on. It's going to take a few steps of work to optimize it. I will take the liberty of just presenting a couple of steps.
First step. Can you get rid of SQL_CALC_FOUND_ROWS and still have your program work correctly? If so, do that. When you specify SQL_CALC_FOUND_ROWS it sometimes means the server has to delay sending you the first row of your resultset until the last row is available.
Second step. Refactor the dependent subqueries to be JOINs instead.
Here's how you might approach that. Part of your query looks like this...
SELECT DISTINCT SQL_CALC_FOUND_ROWS
p.id,
...
c.id as nCompanyID,
...
m.id as nMID,
...
( /* dependent subquery to be removed */
Select sName
FROM tblBrand
WHERE id = p.nBrandID
) as sBrand,
( /* dependent subquery to be removed */
Select t.sFileName
FROM tblThumbnail t
where t.nMediaID = m.id AND
t.sType = "thumbnail"
) as sFileName,
( /* dependent subquery to be removed */
Select t.nWidth
FROM tblThumbnail t
where t.nMediaID = m.id AND
t.sType = "thumbnail"
) as nWidth,
( /* dependent subquery to be removed */
Select t.nHeight
FROM tblThumbnail t
where t.nMediaID = m.id AND
t.sType = "thumbnail"
) as nHeight,
...
Try this instead. Notice how the brand and thumbnail dependent subqueries disappear. You had three dependent subqueries for the thumbnail; they can disappear into a single JOIN.
SELECT DISTINCT SQL_CALC_FOUND_ROWS
p.id,
...
brand.sName,
thumb.sFilename,
thumb.nWidth,
thumb.nHeight,
...
FROM tblProduct p
INNER JOIN tblMedia AS m ON (m.nTypeID = p.id AND m.sType = 'product')
... (other table joins) ...
LEFT JOIN tblBrand AS brand ON p.id = p.nBrandID
LEFT JOIN tblMedia AS thumb ON (t.nMediaID = m.id AND thumb.sType = 'thumbnail')
I used LEFT JOIN rather than INNER JOIN so MySQL will present NULL values if the joined rows are missing.
Edit
You're using a join pattern that looks like this:
JOIN sometable AS s ON (s.someID = m.id AND s.sType = 'string')
You seem to do this for a few tables. You probably can speed up the JOIN operations by creating compound indexes in those tables. For example, try adding the following index to tblThumbnail: (sType, nMediaID). You can do that with this DDL statement.
ALTER TABLE tblThumbnail ADD INDEX (sType, nMediaID) USING BTREE
You can do similar things to other tables with the same join pattern.
i have problem with join table and use multiple conditions...
My code:
SELECT * FROM
(SELECT sid, MAX(info_date_add) AS max_info_date_add FROM skiresort GROUP BY sid) skiresort_max
INNER JOIN skiresort
ON
skiresort_max.sid = skiresort.sid AND
skiresort_max.max_info_date_add = skiresort.info_date_add
JOIN skiresort_theme_value
ON skiresort_theme_value.skiresort_id = skiresort.id
WHERE
skiresort_theme_value.skiresort_theme_id = '1' AND
skiresort_theme_value.skiresort_theme_id = '2' AND
skiresort_theme_value.skiresort_theme_id = '4'
GROUP BY skiresort.sid
ORDER BY skiresort.title_en
In this code, the conditions are in WHERE clausule. I also tried to put in into JOIN ON (...) but it also didn't work.
When i have only one condition it works. I read some articles about using OR instead of AND, it worked but not as i expected. I need to search only rows with certain IDs (multiple).
why not use this instead of many conditions.
WHERE
skiresort_theme_value.skiresort_theme_id in (1, 2,4)
GROUP BY skiresort.sid
HAVING COUNT(DISTINCT skiresort_theme_value.skiresort_theme_id) = 3
ORDER BY skiresort.title_en
when add condition to WHERE, condition must be from FROM tbl
add condition JOIN ON
Try this:
SELECT * FROM
(SELECT sid, MAX(info_date_add) AS max_info_date_add FROM skiresort GROUP BY sid) skiresort_max
INNER JOIN skiresort
ON
skiresort_max.sid = skiresort.sid AND
skiresort_max.max_info_date_add = skiresort.info_date_add
JOIN skiresort_theme_value
ON (skiresort_theme_value.skiresort_id = skiresort.id AND skiresort_theme_value.skiresort_theme_id = '1' AND skiresort_theme_value.skiresort_theme_id = '2' AND skiresort_theme_value.skiresort_theme_id = '4')
GROUP BY skiresort.sid
ORDER BY skiresort.title_en
I have to write an SQL statement which contain a field that contain two different values consecutively but in the way I have wrote it, it return always null because it is interpreted as having the two value in the same time!
My conditions should be : (ci.field = 'Group' and ci.oldString = 'Triage' ) and (ci.field='assignee' and ci.newString is not NULL)
That means calculate time between: when the issue is assigned to group named Triage and when the issue is assigned to a person.
How can I fix it?
My SQL statement:
select TIMEDIFF(a.created,b.created)
from
(select g.created, g.issueid as groupid1
from changegroup g
join changeitem ci on (ci.groupid = g.id)
join jiraissue ji on (ji.id = g.issueid)
join project p on (p.id = ji.project)
join priority pr on (pr.id = ji.priority)
where ci.field = 'Group'
and ci.oldString = 'Triage'
and ci.field='assignee'
and ci.newString is not NULL
and p.pname = 'Test'
and pr.pname='P1'
and ji.created between '2011-08-11 14:01:00' and '2011-08-12 14:11:00'
) a
left join (
select ji.created, ji.id as groupid2
from jiraissue ji
join changegroup g on (g.issueid = ji.id)
join project p on (p.id = ji.project)
where p.pname = 'Test'
and ji.created between '2011-08-11 14:01:00' and '2011-08-12 14:11:00'
) b ON (a.groupid1 = b.groupid2);
This is the table from which I should retrieve data
See my comment about the quality of your question but a hint at how to solve this goes like (assuming you can make sure this doesn't create 1-n joins)
select groupid_orsomething_else, TIMEDIFF(a.created, b.created)
from yourtable
left join
(select groupid_orsomething_else, created
from yourtable
where field = 'Group' and oldstring is 'Triage'
) a
on a.groupid_orsomething_else = yourtable.groupid_orsomething_else
left join
(select groupid_orsomething_else, created
from yourtable
where field = 'assignee' and oldstring is null) b
on b.groupid_orsomething_else = yourtable.groupid_orsomething_else