I have a table called ntr_perf with the following column: data, cos, network, tot_reg, tot_rej.
I need to get the sums of tot_reg and tot_rej for each pair data-cos (I need to take all data-cos pairs and make the sum of the values for all the networks with the same data-cos pair).
I'm using the following MySQL query:
SELECT DISTINCT
data AS d,
cos AS c,
(SELECT SUM(tot_reg) FROM ntr_perf WHERE data=d AND cos=c) AS sumattempts,
(SELECT SUM(tot_rej) FROM ntr_perf WHERE data=d AND cos=c) AS sumrej FROM ntr_perf
It takes a very long time even if the table has only 91.450 rows (the table has a multi-column index data-cos).
Is it possible to speed up the query?
This is exactly what group by is designed for.
Try this:
SELECT data,cos,SUM(tot_reg),SUM(tot_rej) from ntr_perf group by data,cos
You can use this query,
SELECT data AS d, cos AS c,
SUM(tot_reg), SUM(tot_reg) where
data='d' AND cos='c' group by data , cos ;
Hope you got is. Else let me know, will help you
Try this, a group by
SELECT data d,
cos c,
SUM(tot_reg) sumattempts,
SUM(tot_rej) sumrej
FROM ntr_perf
WHERE data = 'd' -- if these are values, put in single quotes
AND cos = 'c' -- if these are values, put in single quotes
GROUP BY data, -- even though aliased, the original name needs to be used on the GROUP BY
cos -- even though aliased, the original name needs to be used on the GROUP BY
this will group your query and filter your sums, as you posted WHERE conditions:
SELECT
data AS d,
cos AS c,
SUM(IIF(data='d' AND cos='c', tot_reg, 0) AS sumattempts,
SUM(IIF(data='d' AND cos='c', tot_rej, 0)) AS sumrej
FROM
ntr_perf
GROUP BY
data,
cos
Example data to sort:
xy3abc
y3bbc
z3bd
Sort order must be abc, bbc, bd regardless of what is before the numeral.
I tried:
SELECT
*,
LEAST(
if (Locate('0',fcccall) >0,Locate('0',fcccall),99),
if (Locate('1',fcccall) >0,Locate('1',fcccall),99),
if (Locate('2',fcccall) >0,Locate('2',fcccall),99),
if (Locate('3',fcccall) >0,Locate('3',fcccall),99),
if (Locate('4',fcccall) >0,Locate('4',fcccall),99),
if (Locate('5',fcccall) >0,Locate('5',fcccall),99),
if (Locate('6',fcccall) >0,Locate('6',fcccall),99),
if (Locate('7',fcccall) >0,Locate('7',fcccall),99),
if (Locate('8',fcccall) >0,Locate('8',fcccall),99),
if (Locate('9',fcccall) >0,Locate('9',fcccall),99)
) as locationPos,
SUBSTRING(fcccall,locationPos,3) as fccsuffix
FROM memberlist
ORDER BY locationPos, fccsuffix
but locationPos gives me an error on the substring function call
It's not possible to reference that expression by its alias locationPos, within another expression in the same SELECT list.
Replicating the entire expression would be the SQL way to do it. (Yes, it is ugly repeating that entire expression.)
Another (less performant) approach is to use your query (minus the fccsuffix expression) as an inline view. The outer query can reference the assigned locationPos alias as a column name.
As a simple example:
SELECT v.locationPos
FROM ( SELECT 'my really big expression' AS locationPos
FROM ...
) v
This approach of using an inline view ("derived table") can have some serious performance implications with large sets.
But for raw performance, repeating the expression is the way to go:
SELECT *
, LEAST(
if (Locate('0',fcccall) >0,Locate('0',fcccall),99),
if (Locate('1',fcccall) >0,Locate('1',fcccall),99),
if (Locate('2',fcccall) >0,Locate('2',fcccall),99),
if (Locate('3',fcccall) >0,Locate('3',fcccall),99),
if (Locate('4',fcccall) >0,Locate('4',fcccall),99),
if (Locate('5',fcccall) >0,Locate('5',fcccall),99),
if (Locate('6',fcccall) >0,Locate('6',fcccall),99),
if (Locate('7',fcccall) >0,Locate('7',fcccall),99),
if (Locate('8',fcccall) >0,Locate('8',fcccall),99),
if (Locate('9',fcccall) >0,Locate('9',fcccall),99)
) AS locationPos
, SUBSTRING(fcccall
, LEAST(
if (Locate('0',fcccall) >0,Locate('0',fcccall),99),
if (Locate('1',fcccall) >0,Locate('1',fcccall),99),
if (Locate('2',fcccall) >0,Locate('2',fcccall),99),
if (Locate('3',fcccall) >0,Locate('3',fcccall),99),
if (Locate('4',fcccall) >0,Locate('4',fcccall),99),
if (Locate('5',fcccall) >0,Locate('5',fcccall),99),
if (Locate('6',fcccall) >0,Locate('6',fcccall),99),
if (Locate('7',fcccall) >0,Locate('7',fcccall),99),
if (Locate('8',fcccall) >0,Locate('8',fcccall),99),
if (Locate('9',fcccall) >0,Locate('9',fcccall),99)
),3
) AS fccsuffix
FROM memberlist
ORDER BY locationPos, fccsuffix
Unfortunately, with MySQL, it's not possible to reference the result of the locationPos column within an expression in the same SELECT list.
For only one numeral I like:
SELECT *
FROM memberlist
ORDER BY SUBSTRING(fcccall,
LOCATE('0',fcccall)+
LOCATE('1',fcccall)+
LOCATE('2',fcccall)+
LOCATE('3',fcccall)+
LOCATE('4',fcccall)+
LOCATE('5',fcccall)+
LOCATE('6',fcccall)+
LOCATE('7',fcccall)+
LOCATE('8',fcccall)+
LOCATE('9',fcccall),3)
But the sensible approach is not to store two separate bits of information in one field.
Can anyone help me with a problem I am having with a CrossTab Query to compare current prices from our suppliers?
The select Query that it works from has a sub query that selects on only the most resent prices for our price comparison and this works perfectly for the data we need, see below:
qryPriceComp:
SELECT tblPriceComp.SupplyerID, tblPriceComp.ProductID,
tblPriceComp.Effdt, tblPriceComp.CostPrice,
tblProduct.Product, tblSupplier.Supplier
FROM tblSupplier INNER JOIN
(tblProduct INNER JOIN tblPriceComp ON tblProduct.ProductID = tblPriceComp.ProductID)
ON tblSupplier.SupplierID = tblPriceComp.SupplyerID
WHERE (((tblPriceComp.Effdt) In
(SELECT MAX(B.EffDt) AS MaxOfDt FROM tblPriceComp AS B
WHERE tblPriceComp.ProductID=B.ProductID
AND tblPriceComp.SupplyerID=B.SupplyerID
AND B.EffDt <= Date()+1)));
This is then used for the crosstab query
qryPriceComp_Crosstab:
TRANSFORM Sum(qryPriceComp.CostPrice) AS SumOfCostPrice
SELECT qryPriceComp.Product
FROM qryPriceComp
GROUP BY qryPriceComp.Product
ORDER BY qryPriceComp.Product, qryPriceComp.Supplier
PIVOT qryPriceComp.Supplier;
But when run it gives an error that both tblPriceComp.ProductID and tblSupplier.SupplierID are invalid. I have tried adding them as perimeters but when run this gives a box to enter the ID numbers which is no good as we want to see all productIDs and SupplyerIDs. If anyone can help it would be greatly appreciated!
Not a real solution, but a usable workaround:
Change qryPriceComp to a INSERT INTO tempTable query, and then base the crosstab query on tempTable.
Before each INSERT run, a DELETE * FROM tempTable must be executed.
pry(main)> Loan.joins(:statistics).where(state: <some states>).where.not(statistics: {state: <some other states>}).order(created_at: "desc").last.statistics.map(&:state)
2015-09-21 20:53:54,423|65310|DEBUG|development| - Loan Load (0.9ms) SELECT `loans`.* FROM `loans` INNER JOIN `statistics` ON `statistics`.`loan_id` = `loans`.`id` WHERE `loans`.`state` IN ('started', 'pending_declined') AND (`statistics`.`state` NOT IN ('prequalified', 'conditionally_approved', '4506t_results_uploaded', 'customer_forms_uploaded', 'ready_for_etran', 'etran_verified', 'forms_to_be_verified', 'forms_verified', 'credit_memo_entered', 'loandoc_generated', 'loandoc_completed', 'loandoc_customer_received_need_signatures', 'signatures_checked_and_uploaded', 'boarded')) ORDER BY `loans`.`created_at` ASC LIMIT 1
2015-09-21 20:53:54,426|65310|DEBUG|development| - Statistic Load (0.3ms) SELECT DISTINCT `statistics`.* FROM `statistics` WHERE `statistics`.`loan_id` = 97
=> ["started", "prequalified", "conditionally_approved", "customer_forms_uploaded", "ready_for_etran", "pending_declined"]
So, maybe I'm not understanding what's going on here... I'm asking SQL to find me some Loans where their Statistics do not contain certain values. In this example, I'm saying to leave out any loans with a Statistic of prequalified, but, as you can see from the print out, the Loan#statistics does have prequalified, along with several other states I'd like to leave out.
Can anyone shed some light on this? I've been fighting with it for hours, and my head is spinning at this point.
With that ActiveRecord query, you've:
First, found a set of loans
next, ordered by created_at
then, used last to find limit to 1 result, finding the oldest of the set
So, you have an instance of Loan.
Since you called #statistics on the method, I can infer that loan has_many :statistics, and you've found all statistics that holds a foreign key value that matches the instance of Loan that you found. Now you have set of statistics.
For the set of statistics, you've mapped them to the map attribute.
Since you've already joined the statistics try removing .last.statistics from your query. User map on the result set to its state. Also, consider using #includes or #select.
It because you use last.statistics. It means the result on loan object will be joined with statistics whereas you have created condition before.
Look at your last result query:
Statistic Load (0.3ms) SELECT DISTINCT `statistics`.* FROM `statistics` WHERE `statistics`.`loan_id` = 97
remove your last.statistics
Loan.joins(:statistics).where(state: <some states>).where.not(statistics: {state: <some other states>}).order(created_at: "desc").map(&:state)
or
if you want to add condition to determine some loans that you need in before map(&state)
Loan.joins(:statistics).where(state: <some states>).where.not(statistics: {state: <some other states>}).where("loans.id IN (97)").order(created_at: "desc")
You query returns product of Loan and Statistic so it still returns Loan records that have some Statistic that does not have state you specified.
If you only want Loan that has no Statistic on those states at all you probably want your SQL to be something along this line:
SELECT loans.*,
FROM loans
LEFT OUTER JOIN (
SELECT statistics.loan_id, COUNT(*) count
FROM quotes
WHERE statistics.state IN ('prequalified', 'conditionally_approved')
GROUP BY statistics.loan_id
) statistics
ON statistics.loan_id = loans.id
WHERE loans.state IN ('started', 'pending_declined')
AND statistics.count IS NULL;
My SQLfu is not what I'd be proud of so this might not be the most optimised query ever but it should get the result you expect.
You could convert that to ActiveRecord query interface but unfortunately subquery and LEFT JOIN are not really supported, at least not in the way that we going to use it will be something like this:
join_query = <<SQL
LEFT OUTER JOIN (
SELECT statistics.loan_id, COUNT(*) AS count
FROM statistics
WHERE statistics.state IN (<<state>>)
) statistics ON loans.id = statistics.loan_id
SQL
Loan
.joins(join_query)
.where(statistics: { count: null })
.where(state: <<somestate>>)
.order(created_at: :desc)
The <<SQL ... SQL is Heredoc by the way if you're not familiar with it.