I need your help to optimize the query ,I am using mysql mariadb.The job of the query is to get some datas for using ui.
I set to null some datas because of example. The query is:
select q.*,
(case when q.numberofcustomizations > 0 then 1 else 0 end) hascustomization,
concat_ws(' - ', q.productname, q.variant) productnamewithvariant
from (
select ca.lastname,
oh.orderid,
oh.orderno,
oh.datecreated,
p.productid, concat_ws(' ', b.name, p.model) as productname,
(case when pv.variantid is not null then concat_ws(' / ', pv.type1, pv.type2) end) variant,
ifnull(concat(' / ',ml.sku), case when pv.variantid is not null then ifnull(concat(' / ',pv.sku),concat(' / ',p.sku)) else concat(' / ',p.sku) end) as sku2,(case when pv.variantid is not null then ifnull(concat(' / ',pv.barcode),concat(' / ',p.barcode)) else concat(' / ',p.barcode) end) as barcode2,
ml.uuid, ml.merchantwarehouseid,
p.sku,
mo.lineid as merchantorderid,
mo.orderlineno,mo.pending,
ol.quantity, ol.comment,
ifnull((select sum(numberofitemshipped) from merchant_order_shipment mos where mos.merchantorderlineid=mo.lineid),0) as numberofitemshipped,
ifnull((select sum(numberofitemshipped) from merchant_order_shipment mos where mos.merchantorderlineid=mo.lineid and mos.`status`='delivered'),0) as numberofitemdelivered,
ol.price,
ol.bundlecode,
concat_ws(' ',bb.name, pb.model) as bundleproductname,
mo.statuscode,
os.isopenorder,
os.name as status, p.barcode,
p.manufactureritemcode,
r.fullsizeurl,
r.thumbnailsizeurl,
sa.postalcode as shipping_postalcode,
sa.countrycode as shipping_countrycode,
(select count(0) from order_line_property olp where olp.orderid=mo.orderid and olp.lineno=mo.orderlineno) numberofcustomizations,
(select count(0) from order_line_property olp where olp.orderid=mo.orderid and olp.lineno=mo.orderlineno and olp.value is null) numberofcustomizationrequests,
case when exists(select null from order_incident oi where oi.orderid=mo.orderid and oi.orderlineno=mo.orderlineno) then 1 else 0 end hasincident,
(ifnull((select sum(ols.price) from order_line ols where ols.orderid=oh.orderid and ols.feetypecode='shipment'),0) / (select count(distinct pm.merchantid) from order_line olm join product pm on pm.productid=olm.productid where olm.orderid=oh.orderid) ) as shippingfee,
(case when olg.giftfrom is not null then 1 else 0 end) hasgiftnote
from merchant_order mo
join merchant_listing ml on ml.merchantlistingid=mo.merchantlistingid
join order_header oh on oh.orderid=mo.orderid
join address sa on sa.addressid=oh.addressid
join order_line ol on ol.orderid=oh.orderid and ol.lineno=mo.orderlineno
join order_status os on os.statuscode=mo.statuscode and os.isvalidorder=1
join product p on p.productid=ol.productid
join customer c on c.customerid=oh.customerid
join address ca on ca.addressid=c.addressid
left outer join product_variant pv on pv.variantid=ml.variantid
left outer join brand b on b.brandid=p.brandid
left outer join product_resource pr on pr.productid=p.productid and pr.isdefault=1
left outer join resource r on r.resourceid=pr.resourceid
left outer join order_line_gift olg on olg.orderid=mo.orderid and olg.lineno=mo.orderlineno
left outer join category cat on cat.categoryid=p.categoryid
left outer join product pb on pb.bundlecode=ol.bundlecode
left outer join brand bb on bb.brandid=pb.brandid
join (select #categoryid=NULL, #brandid=NULL, #merchantwarehouseid=NULL, #hascustomization=NULL, #iscustomizationrequested=NULL, #hassuborder=NULL) params on 1=1
where
mo.statuscode=ifnull(NULL,mo.statuscode)
and oh.orderno=ifnull(NULL, oh.orderno)
and c.customerno=ifnull(NULL, c.customerno)
and ca.firstname=ifnull(NULL, ca.firstname)
and ca.lastname=ifnull(NULL, ca.lastname)
and ca.email=ifnull(NULL, ca.email)
and ml.productid=ifnull(NULL, ml.productid)
and (case when #categoryid is null then 1
when cat.categoryid=#categoryid then 1
when cat.overcategoryid=#categoryid then 1 end)
and (case when #brandid is null then 1
when p.brandid=#brandid and b.isactive=1 then 1
end)
and (case when #merchantwarehouseid is null then 1
when ml.merchantwarehouseid=#merchantwarehouseid then 1 end)
and concat(ifnull(oh.originref,'-'),' / ',ifnull(oh.originsource,'-'))=ifnull(NULL,concat(ifnull(oh.originref,'-'),' / ',ifnull(oh.originsource,'-')))
and mo.pending=ifnull(NULL, mo.pending)
and oh.datecreated between ifnull(NULL, oh.datecreated) and ifnull(NULL, oh.datecreated)
) q
where
(case when #hascustomization is null then 1
when q.numberofcustomizations > 0 and #hascustomization = 1 then 1
when q.numberofcustomizations = 0 and #hascustomization = 0 then 1
end)
and (case when #iscustomizationrequested is null then 1
when q.numberofcustomizationrequests > 0 and #iscustomizationrequested = 1 then 1
when q.numberofcustomizationrequests = 0 and #iscustomizationrequested = 0 then 1
end)
order by 1
Explain cost:
cost
I marked the problematic points in yellow.
Also I added new index for bundlecode,it fixed. But I dont know how to fix first two lines.
Thanks
Some problems!
(1) What is this?? and ca.firstname=ifnull(NULL, ca.firstname). If you don't need the test, build the query without the test.
(2) Avoid #Variables... SET #hascustomization=NULL then WHERE #hascustomization = ... will always fail. That is, NULL is not equal to either 0 or 1.
Don't try to fix that, instead get rid of #variables by constructing the query on the fly. This will [perhaps significantly] help the optimization of the query.
More specifically, get rid of
join
(
SELECT #categoryid=NULL, #brandid=NULL, #merchantwarehouseid=NULL,
#hascustomization=NULL, #iscustomizationrequested=NULL,
#hassuborder=NULL
and then simplify
(case when #hascustomization is null then 1 when q.numberofcustomizations > 0
and #hascustomization = 1 then 1 when q.numberofcustomizations = 0
and #hascustomization = 0 then 1 end
)
(etc)
(3) When you get to putting the pagination back in, rewrite the query. Having all the JOINs before doing the pagination is quite inefficient. Instead, do the minimal effort to find the next, say, 10 IDs, then do the JOINs to get the rest of the info -- this time looking up only 10 items for each JOIN.
(4) After all that, start a new Question showing the revised query; I will make suggestions of indexes (often 'composite') to further improve performance. Be sure to include SHOW CREATE TABLE and EXPLAIN SELECT ... in that Question.
I am trying to figure out how to do an if statement that will essentially calculate the stock of an item based on the dates in the tp_rental table.
I was hoping to do something like :
if Date_Due has an entry but Date_Returned does not
then set stock -1
if Date_Due has an entry and Date_returned has an entry entered
then set stock +1
See below for my attempt, not sure if i need a join here?
update title_platform
set Stock = case
when tp_rental.Date_Returned is not null then stock -1
else stock +1
end
Am I going about this the right way?
You need an UPDATE statement with a join between the 2 tables and a CASE expression:
update title_platform p inner join tp_rental r
on r.platformid = p.platformid
set p.stock = p.stock + case
when r.date_due is not null and r.date_returned is null then -1
when r.date_due is not null and r.date_returned is not null then 1
else ? -- if there is another option
end
Remove the else part if it is not needed.
Edit: if there are multiple rows in tp_rental for each platformid then you must aggregate first and then join:
update title_platform p inner join (
select platformid,
sum(date_due is not null and date_returned is not null) -
sum(date_due is not null and date_returned is null) result
from tp_rental
group by platformid
) r on r.platformid = p.platformid
set p.stock = case
when p.stock + r.result < 0 then 0
else p.stock + r.result
end
I have a table I need to pivot that contains a value I need to join with a field in another table. I'm trying to determine if I can do this in one step, or if I need to pivot the first table and then join them together. GROUP_ID is in field_name in the redcap_data table and needs to be joined with group_id in redcap_data_access_groups.
CREATE VIEW vwGlobalHealthInfants AS
SELECT rd.record as record_id,
MAX(CASE WHEN rd.field_name = '__GROUP_ID__' THEN rd.value ELSE NULL END) as GroupId,
g.group_name as hospno,
MAX(CASE WHEN rd.field_name = 'admission_temperature' THEN rd.value ELSE NULL END) as adtemp,
MAX(CASE WHEN rd.field_name = 'antenatal_care' THEN rd.value ELSE NULL END) as antecare,
MAX(CASE WHEN rd.field_name = 'anti_hypertensive' THEN rd.value ELSE NULL END) as antihyper,
MAX(CASE WHEN rd.field_name = 'anticonvulsants' THEN rd.value ELSE NULL END) as anticonvul
FROM (
redcapVON.redcap_data rd
JOIN redcapVON.redcap_data_access_groups g ON ( (
GroupId = g.group_id
) )
)
WHERE (
rd.project_id = 12
)
GROUP BY rd.record
I don't know if it's possible to get it to recognize the GroupID field in the join before the pivot.
CTE would solve it. We just don't have the most updated version of mysql to run it. 2 steps it is.
While I was preparing an answer to one of our fellows here on SO I've encounter an odd situation, at least to me. The original question is here: Pivot Table Omitting Rows that Have Null values
I've modified the query to use max instead of group_concat in order to show the "problem" in all databases.
SELECT
id,
max(case when colID = 1 then value else '' end) AS fn,
max(case when colID = 2 then value else '' end) AS ln,
max(case when colID = 3 then value else '' end) AS jt
FROM tbl
GROUP BY id
The result of this query is this:
ID FN LN JT
1 Sampo Kallinen Office Manager
2 Jakko Salovaara Vice President
3 (null) Foo No First Name
The user asks to filter the row with id 3 because the field value is null.
When it seems pretty obvious that only it needs to do was to add a WHERE value IS NOT NULL constraint on that query to achieve what the user expect. It won't work.
So I start to test it on the other databases to see what happens (Queries with the WHERE CLAUSE)
SELECT
id,
max(case when colID = 1 then value else '' end) AS fn,
max(case when colID = 2 then value else '' end) AS ln,
max(case when colID = 3 then value else '' end) AS jt
FROM tbl
WHERE value is not null
GROUP BY id
Mysql: http://sqlfiddle.com/#!2/78395/1
Postgres: http://sqlfiddle.com/#!15/78395/1
SQLServer: http://sqlfiddle.com/#!6/78395/1
Oracle: http://sqlfiddle.com/#!4/78395/1
For my surprise the result was the same, none worked.
Then I tried a different version of the same query:
SELECT * FROM (
SELECT
id,
max(case when colID = 1 then value else '' end) AS fn,
max(case when colID = 2 then value else '' end) AS ln,
max(case when colID = 3 then value else '' end) AS jt
FROM tbl
GROUP BY id
) T
WHERE fn IS NOT NULL
AND ln IS NOT NULL
AND jt IS NOT NULL
Oracle: http://sqlfiddle.com/#!4/78395/2 WORKED
MySql: http://sqlfiddle.com/#!2/78395/2
Postgres: http://sqlfiddle.com/#!15/78395/2
SQLServer: http://sqlfiddle.com/#!6/78395/2
The only way I could make it work on all databases was with this query:
SELECT
id,
max(case when colID = 1 then value else '' end) AS fn,
max(case when colID = 2 then value else '' end) AS ln,
max(case when colID = 3 then value else '' end) AS jt
FROM tbl
WHERE NOT EXISTS (SELECT * FROM tbl b WHERE tbl.id=b.id AND value IS NULL)
GROUP BY id
So I ask:
What is happening here that except for that specific case on Oracle all other DBs seem to ignore the IS NOT NULL filter?
To omit the row from the result if any of the source rows for the same id has value IS NULL, a solution in Postgres would be to use the aggregate function every() or (synonym for historical reasons) bool_and() in the HAVING clause:
SELECT id
, max(case when colID = 1 then value else '' end) AS fn
, max(case when colID = 2 then value else '' end) AS ln
, max(case when colID = 3 then value else '' end) AS jt
FROM tbl
GROUP BY id
HAVING every(value IS NOT NULL);
SQL Fiddle.
Explain
Your attempt with a WHERE clause would just eliminate one source row for id = 3 in your example (the one with colID = 1), leaving two more for the same id. So we still get a row for id = 3 in the result after aggregating.
But since we have no row with colID = 1, we get an empty string (note: not a NULL value!) for fn in the result for id = 3.
A faster solution in Postgres would be to use crosstab(). Details:
PostgreSQL Crosstab Query
Other RDBMS
While EVERY is defined in the SQL:2008 standard, many RDBMS do not support it, presumably because some of them have shady implementations of the boolean type. (Not dropping any names like "MySQL" or "Oracle" ...). You can probably substitute everywhere (including Postgres) with:
SELECT id
, max(case when colID = 1 then value else '' end) AS fn
, max(case when colID = 2 then value else '' end) AS ln
, max(case when colID = 3 then value else '' end) AS jt
FROM tbl
GROUP BY id
HAVING count(*) = count(value);
Because count() doesn't count NULL values. In MySQL there is also bit_and().
More under this related question:
Is there any equivalent to Postgresql EVERY aggregate function on other RDBMS?
It works in Oracle because Oracle handles NULL incorrectly in that NULL and '' are the same. The other databases don't do this because it is wrong. NULL is unknown, versus '' which is just a blank, empty string.
So if your where clause said something like WHERE (fn IS NOT NULL or fn <> '') you would probably get further.
I think this is a case where a HAVING clause will do what you need.
SELECT id, max ... (same stuff as before)
FROM tbl
GROUP by id
HAVING fn IS NOT NULL
AND ln IS NOT NULL
AND jt IS NOT NULL
Looking for the way to change row to column. (The comflag is of type bit and not null). Help appreciated
Table1
Id Commflag value
122 0 Ce
125 1 Cf
122 0 Cg
125 1 cs
Here is what I want in result
id ce cf cg cs cp
122 0 null 0 null null
125 null 1 null 1 null
The below query shows error-
SELECT ID , [CE],[CF],[CG],[CS],[CP]
FROM TABLE1
PIVOT ((convert((Commflag)as varchar()) FOR value IN [CE],[CF],[CG],[CS],[CP] as pvt
ORDER BY date
This query does what you want:
select Id, pvt.Ce, pvt.Cf, pvt.CG, pvt.Cs, pvt.Cp
from
(
select Id, cast(Commflag as tinyint) Commflag, value
from Table1
) t
pivot (max(Commflag) for value in ([Ce],[Cf],[CG],[Cs],[Cp])) pvt
SQL Fiddle
Here's another way to do it, without using PIVOT:
select Id,
max(case value when 'Ce' then CAST(Commflag as tinyint) else null end) Ce,
max(case value when 'Cf' then CAST(Commflag as tinyint) else null end) Cf,
max(case value when 'Cg' then CAST(Commflag as tinyint) else null end) Cg,
max(case value when 'Cs' then CAST(Commflag as tinyint) else null end) Cs,
max(case value when 'Cp' then CAST(Commflag as tinyint) else null end) Cp
from Table1
group by Id
order by Id
SQL Fiddle