How can I simplify this hideous query? - mysql

I've gotten to the point with this query where it's become too unwieldy to modify and I'm sure it's also become rather inefficient.
I have a table of items which may have any variable number of properties so these properties are listed in small tables ... item id and property id usually make up the index, and the property tables will have additional information. In essence they're used for filtering a huge number of items to at least the relevant subset.
SELECT * FROM item
INNER JOIN itemwearTable
ON (id=itemwearTable.itemID AND itemwearTable.wearlocID=1)
WHERE item.id IN (
SELECT id FROM item
LEFT JOIN itemalignTable
ON item.id=itemalignTable.itemID
WHERE
itemalignTable.itemID IS NULL OR
(itemalignTable.itemID=item.id AND itemalignTable.alignID=1)
)
AND
item.id IN (
SELECT id FROM item
LEFT JOIN itemgenderTable
ON itemgenderTable.itemID=item.id
WHERE
itemgenderTable.itemID IS NULL OR
(itemgenderTable.itemID=item.id AND itemgenderTable.genderID=1)
)
AND
(item.id IN (
SELECT id FROM item
LEFT JOIN itemgenreTable
ON item.id=itemgenreTable.itemid
WHERE
itemgenreTable.itemid IS NULL OR
(itemgenreTable.itemID=item.id AND itemgenreTable.genreID=1)
)
OR
item.id IN (
SELECT id from item
LEFT JOIN itemclassTable
ON item.id=itemclassTable.itemid
WHERE
itemclassTable.itemid IS NULL OR
(itemclassTable.itemID=item.id AND itemclassTable.classID=1)
))
AND
item.id IN (
SELECT id from item
LEFT JOIN itemlocTable ON
item.id=itemlocTable.itemid
WHERE
itemlocTable.itemid IS NULL OR
(itemlocTable.itemID=item.id AND itemlocTable.locID=1)
)
AND
item.minlvl <= 50
ORDER BY item.sdesc
Note I've used =1 here in many places to replace the php variables that would normally be part of the script generating the query. The problem is that while these items are used to narrow the results, I have another table itemaffTable that has additional data (amount for example) which matches on item.id=itemaffTable.itemID but I can't seem to see how to get that inserted without seriously messing up what's already there.
Any help is appreciated.

I think this should do it:
SELECT DISTINCT i.*, iwt.* FROM item i
INNER JOIN itemwearTable iwt ON i.id=iwt.itemID
LEFT JOIN itemalignTable iat ON i.id = iat.itemID
LEFT JOIN itemgenderTable igt ON i.id = igt.itemID
LEFT JOIN itemgenreTable igrt ON i.id = igrt.itemID
LEFT JOIN itemclassTable ict ON i.id = ict.itemID
LEFT JOIN itemlocTable ilt ON i.id = ilt.itemID
WHERE (iat.itemID IS NULL OR iat.alignID = 1)
AND (igt.itemID IS NULL OR igt.genderID = 1)
AND (igrt.itemID IS NULL OR igrt.genreID = 1)
AND (ict.itemID IS NULL OR ict.classID = 1)
AND (ilt.itemID IS NULL OR ilt.locID = 1)
AND i.minlvl <= 50
AND iwt.wearlocID=1
ORDER BY i.sdesc

Something like this should work.
SELECT item.*, itemwearTable.* FROM item
INNER JOIN itemwearTable
ON (id=itemwearTable.itemID AND itemwearTable.wearlocID=1)
LEFT JOIN itemalignTable
ON item.id=itemalignTable.itemID
LEFT JOIN itemgenderTable
ON itemgenderTable.itemID=item.id
LEFT JOIN itemgenreTable
ON item.id=itemgenreTable.itemid
LEFT JOIN itemclassTable
ON item.id=itemclassTable.itemid
LEFT JOIN itemlocTable
ON item.id=itemlocTable.itemid
WHERE (itemalignTable.itemID IS NULL OR itemalignTable.alignID=1)
AND (itemgenderTable.itemID IS NULL OR itemgenderTable.genderID=1)
AND (itemlocTable.itemid IS NULL OR itemlocTable.locID=1)
AND ((itemgenreTable.itemid IS NULL OR itemgenreTable.genreID=1)
OR (itemclassTable.itemid IS NULL OR itemclassTable.classID=1))
AND item.minlvl <= 50
ORDER BY item.sdesc

Related

SQL Price Scheduling, Select last timestamp thats <= UTC_TIMESTAMP()

I have a table called product_pricing.
In here, I can add multiple prices for each ID. What differentiates them is the product_pricing.timestamp_valid...it is what I use to schedule price changes in advance.
SELECT
`products`.`wo_id`,
`products`.`fty_id`,
`products`.`price` AS price1,
`product_attributes`.`fty_id`,
`product_attributes`.`cat_id`,
`product_attributes`.`design_id`,
`product_attributes`.`season_id`,
`products_u_ids`.`u_id`,
`products_u_ids`.`link_id`,
`product_designs`.`design_id`,
`product_designs`.`brand_id`,
COALESCE(`product_pricing`.`u_id`, NULL) AS price2_u_id,
COALESCE(`product_pricing`.`currency`, NULL) AS price2_currency,
COALESCE(`product_pricing`.`price`, NULL) AS price2,
COALESCE(`product_pricing`.`formula_id`, NULL) price2_formula_id,
COALESCE(`product_pricing`.`vat_calculated`) AS price2_vat_calculated,
COALESCE(`product_pricing`.`vat_id`, NULL) AS price2_vat_id,
COALESCE(`product_pricing`.`timestamp_valid`, NULL) price2_timestamp_valid,
COALESCE(`product_price_formulas`.`formula_id`, NULL) AS price2_formula_id,
COALESCE(`product_price_formulas`.`formula`, NULL) AS price2_formula,
COALESCE(`global_vat_tariffs`.`vat_id`, NULL) AS price2_vat_id,
COALESCE(`global_vat_tariffs`.`percentage`, NULL) AS price2_vat_tariff
FROM `products`
LEFT JOIN `product_attributes`
ON `products`.`fty_id` = `product_attributes`.`fty_id`
LEFT JOIN `products_u_ids`
ON `product_attributes`.`fty_id` = `products_u_ids`.`link_id`
LEFT JOIN `product_designs`
ON `product_attributes`.`design_id` = `product_designs`.`design_id`
LEFT JOIN `product_pricing`
ON `products_u_ids`.`u_id` = `product_pricing`.`u_id`
LEFT JOIN `product_price_formulas`
ON `product_pricing`.`formula_id` = `product_price_formulas`.`formula_id`
LEFT JOIN `global_vat_tariffs`
ON `product_pricing`.`vat_id` = `global_vat_tariffs`.`vat_id`
LEFT OUTER JOIN (
SELECT `product_pricing`.`u_id`, MAX(`timestamp_valid`) AS MaxDate
FROM `product_pricing`
WHERE `product_pricing`.`timestamp_valid` <= UTC_TIMESTAMP
GROUP BY `product_pricing`.`u_id`
) AS temp ON temp.u_id = `product_pricing`.`u_id` AND temp.MaxDate = `product_pricing`.`timestamp_valid`
WHERE `products`.`wo_id` IN ('028284', '018305', '031536')
The SQL I have above, returns all the rows for a given ID, instead of just returning product_pricing.timestamp_valid``<= UTC_TIMESTAMP().
There is a problem in the LEFT OUTER JOIN.
The other thing that this SQL does is, when there isn't anything set for price2, it shouldn't break the script, but just return NULL for price2. I am solving that by using COALESCE.
This is what I get:
This is what I should get:
Assuming the date of the query is 2018-5-7.
Any ideas how to solve this problem with the LEFT OUTER JOIN?
You're using a LEFT JOIN operation on the subquery containing the timestamp filter you mention. LEFT JOINs retain rows that don't match its ON condition. You might try an ordinary INNER join on that subquery; inner JOINs don't retain the rows matching the ON condition.

How to handle SQL joins if missing rows on particular table exist

SELECT
fromData.name as fromname, toData.name as toName, prodData.prodname,
t1.`from_id`, t1.`to_id` , t1.`product_id` , t1.`title`, t1.`message`, t1.`senttime` , t1.`readstatus`, t1.`responded`, t1.`merchanthidden`
FROM `inquiries` as t1
INNER JOIN users as fromData on t1.from_id = fromData.id
INNER JOIN users as toData on t1.to_id = toData.id
INNER JOIN products as prodData on t1.product_id = prodData.id
WHERE t1.id=13
Above query joins 3 tables (inquiries, users, products) together and gets data from each table.
Sometimes it is possible that items in the 'products' table get deleted. Trying to join products table by a deleted product id will fail the query.
Is there a way that I can assign 'prodData.prodname' a default value and execute query without failing in case of a missing item in products table ?
Why don't use left join insted of inner join ,
The LEFT JOIN keyword returns all records from the left table (table1), and the matched records from the right table (table2). The result is NULL from the right side, if there is no match.
SELECT
fromData.name as fromname, toData.name as toName, prodData.prodname,
t1.`from_id`, t1.`to_id` , t1.`product_id` , t1.`title`, t1.`message`, t1.`senttime` , t1.`readstatus`, t1.`responded`, t1.`merchanthidden`
FROM `inquiries` as t1
INNER JOIN users as fromData on t1.from_id = fromData.id
INNER JOIN users as toData on t1.to_id = toData.id
LEFT JOIN products as prodData on t1.product_id = prodData.id
WHERE t1.id=13

Mysql tekes too much time to excute sql query, based on multiple join

My Sql query takes more time to execute from mysql database server . There are number of tables are joined with sb_tblproperty table. sb_tblproperty is main table that contain more than 1,00,000 rows . most of table contain 50,000 rows.
How to optimize my sql query to fast execution. I have also used indexing.
indexing Explain - query - structure
SELECT `t1`.`propertyId`, `t1`.`projectId`,
`t1`.`furnised`, `t1`.`ownerID`, `t1`.`subType`,
`t1`.`fors`, `t1`.`size`, `t1`.`unit`,
`t1`.`bedrooms`, `t1`.`address`, `t1`.`dateConfirm`,
`t1`.`dateAdded`, `t1`.`floor`, `t1`.`priceAmount`,
`t1`.`priceRate`, `t1`.`allInclusive`, `t1`.`booking`,
`t1`.`bookingRate`, `t1`.`paidPercetage`,
`t1`.`paidAmount`, `t1`.`is_sold`, `t1`.`remarks`,
`t1`.`status`, `t1`.`confirmedStatus`, `t1`.`source`,
`t1`.`companyName` as company, `t1`.`monthly_rent`,
`t1`.`per_sqft`, `t1`.`lease_duration`,
`t1`.`lease_commencement`, `t1`.`lock_in_period`,
`t1`.`security_deposit`, `t1`.`security_amount`,
`t1`.`total_area_leased`, `t1`.`lease_escalation_amount`,
`t1`.`lease_escalation_years`, `t2`.`propertyTypeName` as
propertyTypeName, `t3`.`propertySubTypeName` subType,
`t3`.`propertySubTypeId` subTypeId, `Owner`.`ContactName`
ownerName, `Owner`.`companyName`, `Owner`.`mobile1`,
`Owner`.`otherPhoneNo`, `Owner`.`mobile2`,
`Owner`.`email`, `Owner`.`address` as caddress,
`Owner`.`contactType`, `P`.`projectName` as project,
`P`.`developerName` as developer, `c`.`name` as city,
if(t1.projectId="", group_concat( distinct( L.locality)),
group_concat( distinct(L2.locality))) as locality, `U`.`firstname`
addedBy, `U1`.`firstname` confirmedBy
FROM `sb_tblproperty` as t1
JOIN `sb_contact` Owner ON `Owner`.`id` = `t1`.`ownerID`
JOIN `tbl_city` C ON `c`.`id` = `t1`.`city`
JOIN `sb_propertytype` t2 ON `t1`.`propertyType`= `t2`.`propertyTypeId`
JOIN `sb_propertysubtype` t3 ON `t1`.`subType` =`t3`.`propertySubTypeId`
LEFT JOIN `sb_tbluser` U ON `t1`.`addedBy` = `U`.`userId`
LEFT JOIN`sb_tbluser` U1 ON `t1`.`confirmedBy` = `U1`.`userId`
LEFT JOIN `sb_tblproject` P ON `P`.`id` = `t1`.`projectId` LEFT
JOIN `sb_tblpropertylocality` PL ON `t1`.`propertyId` = `PL`.`propertyId`
LEFT JOIN `sa_localitiez` L ON `L`.`id` = `PL`.`localityId`
LEFT JOIN `sb_tblprojectlocality` PROL ON `PROL`.`projectId` = `P`.`id`
LEFT JOIN `sa_localitiez` L2 ON `L2`.`id` = `PROL`.`localityId`
LEFT JOIN `sb_tblfloor` F
ON `F`.`floorName` =`t1`.`floor`
WHERE `t1`.`is_sold` != '1' GROUP BY `t1`.`propertyId`
ORDER BY `t1`.`dateConfirm`
DESC LIMIT 1000
Please provide the EXPLAIN.
Meanwhile, try this:
SELECT ...
FROM (
SELECT propertyId
FROM sb_tblproperty
WHERE `is_sold` = 0
ORDER BY `dateConfirm` DESC
LIMIT 1000
) AS x
JOIN `sb_tblproperty` as t1 ON t1.propertyId = x.propertyId
JOIN `sb_contact` Owner ON `Owner`.`id` = `t1`.`ownerID`
JOIN `tbl_city` C ON `c`.`id` = `t1`.`city`
...
LEFT JOIN `sb_tblfloor` F ON `F`.`floorName` =`t1`.`floor`
ORDER BY `t1`.`dateConfirm` DESC -- yes, again
Together with
INDEX(is_sold, dateConfirm)
How can t1.projectId="" ? Isn't projectId the PRIMARY KEY? (This is one of many reasons for needing the SHOW CREATE TABLE.)
If my suggestion leads to "duplicate" rows (that is, multiple rows with the same propertyId), don't simply add back the GROUP BY propertyId. Instead figure out why, and avoid the need for the GROUP BY. (That is probably the performance issue.)
A likely case is the GROUP_CONCAT. A common workaround is to change from
GROUP_CONCAT( distinct( L.locality)) AS Localities,
...
LEFT JOIN `sa_localitiez` L ON `L`.`id` = `PL`.`localityId`
to
( SELECT GROUP_CONCAT(distinct locality)
FROM sa_localitiez
WHERE id = PL.localityId ) AS Localities
...
# and remove the JOIN

mysql several count and join returns strange value

I'm trying to make a count within several table with JOIN, but when I made several JOINs the COUNTs got wrongly counted.
Basically I've got 4 tables, named:
predective_search
predective_to_product
predective_to_category
predective_to_manufacturer
I want to count the total number of products, categories and manufacturer which has same id in table predective_search.
Here's my code:
SELECT * ,
COUNT(pp.predictive_id) AS total_products,
COUNT(pc.predictive_id) AS total_categories,
COUNT(pm.predictive_id) AS total_manufacturers
FROM predictive_search ps
LEFT JOIN predictive_to_product pp ON (ps.predictive_id = pp.predictive_id)
LEFT JOIN predictive_to_category pu ON (ps.predictive_id = pc.predictive_id)
LEFT JOIN oc_predictive_to_manufacturer pm ON (ps.predictive_id = pm.predictive_id)
GROUP BY ps.predictive_id
Also the GROUP BY is needed I think. I'm stuck at this as I'm not getting any way to do this
SELECT
ps.*,
agg_pp.total_products,
agg_pc.total_categories,
agg_pm.total_manufacturers
FROM predictive_search ps
LEFT JOIN (
SELECT pp.predictive_id, COUNT(*) AS total_products
FROM predictive_to_product pp
GROUP BY pp.predictive_id
) agg_pp ON ps.predictive_id = agg_pp.predictive_id
LEFT JOIN (
SELECT pc.predictive_id, COUNT(*) AS total_categories
FROM predictive_to_category pc
GROUP BY pc.predictive_id
) agg_pc ON ps.predictive_id = agg_pc.predictive_id
LEFT JOIN (
SELECT pm.predictive_id, COUNT(*) AS total_manufacturers
FROM predictive_to_category pm
GROUP BY pm.predictive_id
) agg_pm ON ps.predictive_id = agg_pm.predictive_id

Problems while trying to make a query using left join and group by with having over a field of the left table on MySQL

Im trying to make some queries, first, I do this one (and works):
SELECT RP.id, RP.product_name, RP.price, RP.retailer_id, RP.product_id,
count(G.id) AS duration, G.active, RP.retprod_id, P.pr_id AS video
FROM retailer_products AS RP
LEFT JOIN groups G ON RP.id=G.retailer_product_id
INNER JOIN products P ON P.id=RP.product_id
WHERE (RP.product_id IN (1))
GROUP BY RP.id;
But when I do this one, it gaves me an empty set, the difference is that it has a "HAVING" at the end of the query over a field that may be: 0, 1 or NULL (because of the LEFT JOIN, I have no groups linked to the table retailer_product)
SELECT RP.id, RP.product_name, RP.price, RP.retailer_id, RP.product_id,
count(G.id) AS duration, G.active, RP.retprod_id, P.pr_id AS video
FROM retailer_products AS RP
LEFT JOIN groups G ON RP.id=G.retailer_product_id
INNER JOIN products P ON P.id=RP.product_id
WHERE (RP.product_id IN (1))
GROUP BY RP.id HAVING G.active=1;
So, I tried the following ways but no one works:
-- HAVING G.active=1
-- HAVING G.active=1 OR G.active=NULL
-- HAVING G.active0
What is the right way to handle this on MySQL? Thanks in advance!
The proper way to compare value with NULL is IS NULL. So you need to add HAVING G.active IS NULL OR G.active=1. Your code G.active=NULL is always NULL (false):
1 = NULL // NULL - false in conditions
NULL = NULL // NULL - false in conditions
NULL IS NULL // TRUE
Why you don't add the HAVING condition on the "Group" ON condition?
SELECT RP.id, RP.product_name, RP.price, RP.retailer_id, RP.product_id,
count(G.id) AS duration, G.active, RP.retprod_id, P.pr_id AS video
FROM retailer_products AS RP
LEFT JOIN groups G ON RP.id=G.retailer_product_id AND G.active=1
INNER JOIN products P ON P.id=RP.product_id
WHERE (RP.product_id IN (1))
GROUP BY RP.id;