I have the following table schema:
tbl_portfolio
----------
id (auto number)
name
-
tbl_registration
----------------
id(auto number)
name
portfolio_id (FK to tbl_portfolio.id)
-
tbl_fund
---------
id (auto number)
registration_id (FK to tbl_registration.id)
-
tbl_transaction
---------------
id (auto number)
fund_id (FK to tbl_fund.id)
type
shares
price
I need to create a query that in psuedo-code would do the following:
SELECT port.*, SUM ALL transactions for each portfolio,
FROM tbl_portfolio port
INNER JOIN tbl_registration reg ON reg.portfolio_id = port.id
LEFT JOIN tbl_fund fund on fund.registration_id = reg.id
LEFT JOIN tbl_transaction trans ON trans.fund_id = fund.id
Now of course that query won't work...What I am needing essentially is to sum all the Price * Units for each fund, and then sum those together for each registration, and then sum all of that together for each portfolio.
Each portfolio can have multiple registrations, and each registration can have multiple funds, and each fund can have multiple transactions.
The last item that is throwing a stickler in this, there may be 10's or 100's of portfolios to count so I have no idea how to write the query, much less write it in an effective way that is not relying on subqueries that would cause it to have severely poor performance.
Thank you for the help!
Edit:
PinnyM's answer works and queries the data correctly - however I should expand on the full need.
Besides the tbl_transaction there is also a tbl_distri and tbl_div. Both have fund_id as FK to tbl_fund.id . I need to get the SUM's of tbl_distri.amount and tbl_div.units.
So the full psuedo query would be something to the effect of:
SELECT port.*, SUM ALL transactions for each portfolio, SUM(div.units), SUM(distri.amount)
FROM tbl_portfolio port
INNER JOIN tbl_registration reg ON reg.portfolio_id = port.id
LEFT JOIN tbl_fund fund on fund.registration_id = reg.id
LEFT JOIN tbl_transaction trans ON trans.fund_id = fund.id
LEFT JOIN tbl_distri distri on distri.fund_id = fund.id
LEFT JOIN tbl_div div on div.fund_id = fund.id
Have you tried using SUM()?
SELECT port.*, SUM(trans.shares * trans.price) AS transaction_totals
FROM tbl_portfolio port
INNER JOIN tbl_registration reg ON reg.portfolio_id = port.id
LEFT JOIN tbl_fund fund on fund.registration_id = reg.id
LEFT JOIN tbl_transaction trans ON trans.fund_id = fund.id
GROUP BY port.id
Judging from your question, you are looking for a rolled-up SUM
SELECT port.id AS port_id,
reg.id AS reg_id,
fund.id AS fund_id,
SUM ( trans.shares * trans.price) AS net_asset_value
FROM tbl_portfolio port
INNER JOIN tbl_registration reg ON reg.portfolio_id = port.id
LEFT JOIN tbl_fund fund on fund.registration_id = reg.id
LEFT JOIN tbl_transaction trans ON trans.fund_id = fund.id
GROUP BY port.id, reg.id, fund.id WITH ROLLUP
This will give you the sums id by id. You can use other JOIN operations with this as a subquery to fetch the textual names.
This will give results like this:
port_id reg_id fund_id net_asset_value
1 1 1 150.00
1 1 2 100.00
1 1 NULL 250.00 (rollup of previous two lines)
1 2 1 24.00
1 2 4 80.00
1 2 NULL 104.00 (rollup of previous two lines)
1 NULL NULL 354.00 (rollup at portfolio level)
3 1 1 40.00
3 1 2 50.00
3 1 NULL 90.00 (rollup of previous two lines)
3 2 1 14.00
3 2 4 60.00
3 2 NULL 74.00 (rollup of previous two lines)
3 NULL NULL 164.00 (rollup at portfolio level)
NULL NULL NULL 518.00 (grand total)
The NULLs make it into this resultset because that's what WITH ROLLUP does. This resultset only has the IDs in it; presumably the IDs are unique even if the names aren't. Non-unique names for portfolios, funds, etc, will mess up the GROUP BY pretty badly. Hence my earlier comment about retrieving the names.
Related
Problem:
I'm having trouble finding a solution building a query with QueryBuilder (perhaps getting it done with regular sql query first will help):
Trying to retrieve all customers for a user (has shop credits at one of the shops user is linked to), need the total credits (sum of credits at shops belonging to that user) as virtual column (to be able to order on), using paginate().
Database structure:
Table customers
id email other_fields
1 1#email.com f
2 2#email.com o
3 3#email.com o
Table users
id email other_fields
1 1#user.com b
2 2#user.com a
3 3#user.com r
Table shops
id name other_fields
1 Shop 1 m
2 Shop 1 o
3 Shop 1 o
Table user_shops
user_id shop_id
1 1
1 2
3 3
Table customer_shop_credits
customer_id shop_id credits
1 1 55
1 2 45
2 2 3
3 3 44
Expected result:
When retrieving customers for user 1, I'd expect to get back customer 1 with 100 credits and customer 2 with 3 credits
Closest I got:
$credits_query = CustomerShopCreditQuery::create()
->useShopQuery()
->useUserShopQuery()
->filterByUserId($user->getId())
->endUse()
->endUse()
;
$customers = CustomerQuery::create()
->addSelectQuery($credits_query, 'credits_alias', false)
->useCustomerShopCreditQuery()
->useShopQuery()
->useUserShopQuery()
->filterByUserId($user->getId())
->endUse()
->endUse()
->endUse()
->withColumn('sum(credits_alias.credits)', 'credits')
->groupById()
->orderBy($order_by_column, $direction)
->paginate($page, $page_size);
Which results in the following query:
SELECT customers.id, customers.email, sum(credits_alias.credits) AS credits
FROM customers
CROSS JOIN (
SELECT customer_shop_credits.id, customer_shop_credits.customer_id, customer_shop_credits.shop_id, customer_shop_credits.credits
FROM customer_shop_credits
INNER JOIN shops ON (customer_shop_credits.shop_id=shops.id)
INNER JOIN user_shops ON (shops.id=user_shops.shop_id)
WHERE user_shops.user_id=159
) AS credits_alias
INNER JOIN customer_shop_credits ON (customers.id=customer_shop_credits.customer_id)
INNER JOIN shops ON (customer_shop_credits.shop_id=shops.id)
INNER JOIN user_shops ON (shops.id=user_shops.shop_id)
WHERE user_shops.user_id=159
GROUP BY customers.id
ORDER BY customers.id DESC
LIMIT 25
But gives me results with wrong sum of credits.
Not to sure about the CROSS JOIN. When I edit this query and make it a JOIN and use ON (credits_alias.customer_id = customers.id) as a condition, the sum of credits is better, but seems to have the classic join problem of doubling the sum
MEMBERS_TABLE
member_id
---------------------------------------------
1
ACCOUNTS_TABLE
account_id member_id
---------------------------------------------
1 1
INVESTMENTS_TABLE
investment_id account_id
---------------------------------------------
1 1
2 1
FUNDS_TABLE
fund_id investment_id
---------------------------------------------
1 1
2 2
This is my current query:
SELECT
m.member_id,
a.account_id,
i.investment_id,
f.fund_id,
COUNT(a.account_id) AS member_accounts_total,
COUNT(i.investment_id) AS member_investments_total,
COUNT(f.fund_id) AS member_funds_total
FROM members AS m
LEFT JOIN accounts AS a ON m.member_id = a.member_id
LEFT JOIN investments AS i ON a.account_id = i.account_id
LEFT JOIN funds AS f ON f.fund_id = i.fund_id
I would like to see the following results:
member_accounts_total: 1
member_investments_total: 2
member_funds_total: 2
Instead, I am getting these results:
member_accounts_total: 2
member_investments_total: 2
member_funds_total: 2
I really don't want to write multiple queries for this.
Just need to change
COUNT(a.account_id) AS member_accounts_total,
to
COUNT( distinct a.account_id) AS member_accounts_total,
The reason you're getting 2 is because the left join on accounts to investments results in 2 records. To get a distinct count of members you need to add well... distinct.
Note you may have problems with the other totals as well (Distinct may be needed there as well in the long run...) say if a member had multiple accounts. you may get odd counts as well (if each account had the same investment... would you want to see the count only once or twice?
I have three tables that I'm working with.
AccountingLine - Holds the generic account details
Budget - Holds the budget data for each AccountingLine (Many rows per AccountingLine)
Actual - Holds the actual cost data for each AccountingLine (Many rows per AccountingLine)
I'm trying to get the results in a single query which will return ALL ROWS from the AccountingLine table, and SUM the Amounts for each AccountingLine from the Budget and Actuals table.
Using the SQL below, the SUM isn't working for the Budget or Actual data. If I remove one of the joins and one of the SUM functions then it calculates correctly for the single joined table. Very strange... anyone run across this with multiple SUM functions on three or more tables in MySQL?
SELECT A.*, SUM(B.`amount`) AS BudgetAmount, SUM(ACT.`amount`) as ActualAmount
FROM accounting_line A
LEFT JOIN budget B ON B.accounting_line_id = A.accounting_line_id
LEFT JOIN actual ACT ON ACT.accounting_line_id = A.accounting_line_id
GROUP BY A.`accounting_line_id`
By issuing the statement above, I'd expect to see the accounting_line fields, the SUM of the Budget amounts for each accounting_line and the SUM of the Actual amounts for each accounting_line.
I've searched all over and can't find an instance of multiple SUM functions. Thanks so much for any advice.
Josh
Table Data is below:
Table: AccountingLine
act_line_id department
----------------------------------
1 Sales
2 HumanResources
Table: Budget
budget_id actg_line_id amount
----------------------------------------------
1 1 3500.00
2 2 5000.00
3 2 15000.00
Table: Actual
actual_id actg_line_id amount
----------------------------------------------
1 1 1000.00
2 2 500.00
3 2 9000.00
A join repeats each matching row in the other table. So if you have 3 rows in three tables and join them together, you end up with 9 rows. If you sum, each sum from the second and third table is 3x too high.
One solution is to sum in a subquery, so that the join only finds one row:
SELECT A.*
, B.SumAmount as BudgetAmount
, ACT.SumAmount as ActualAmount
FROM accounting_line A
LEFT JOIN
(
select accounting_line_id
, sum(amount) as SumAmount
from budget
group by
accounting_line_id
) as B
ON B.accounting_line_id = A.accounting_line_id
LEFT JOIN
(
select accounting_line_id
, sum(amount) as SumAmount
from actual
group by
accounting_line_id
) as ACT
ON ACT.accounting_line_id = A.accounting_line_id
try this modified one, calculate it's totals on a subquery
SELECT a.*, b.totalBudget, c.totalActual
FROM AccountingLine a LEFT JOIN
(
SELECT actg_line_id, SUM(amount) totalBudget
FROM Budget
GROUP BY actg_line_id
) b on a.act_line_id = b.actg_line_id
LEFT JOIN
(
SELECT actg_line_id, SUM(amount) totalActual
FROM Actual
GROUP BY actg_line_id
) c on a.act_line_id = c.actg_line_id
SQLFiddle Demo
Try this
Select A.* ,SUM(B.Amount) As BudgetAmount,SUM(Act.Amount) As ActualAmount
from AccountingLine A
INNER JOIN Budget B
ON B.budget_id = A.actg_line_id
INNER JOIN Actual Act
ON Act.actual_id = A.accounting_line_id
Grounp By A.accounting_line_id
I have to select data from two tables with following criteria,
lets say there are two tables as,,
Table one
id | itemName | Quantity | companyName
1 bread 25 the Baker pvt ltd
2 butter 30 green famers
Table two
id | itemName | itemPrice
1 bread 30.50
6 jam 80.25
what I need is,
select items out of two tables which their ids are matching and the quantities of them should be multiplied by the unit price if ids are matching. The rows which don't have matching ids should be selected but their quantities should not multiplied.
SELECT o.id, o.itemName, o.companyName, o.Quantity * IFNULL(t.itemPrice, 1) total
FROM one o
LEFT JOIN two t
ON o.id = t.id
Something like this should work ...
Select a.id, a.itemName, a.companyName, a.Quantity * IFNULL(b.itemPrice,1) As total
From table1 as a
Left Join table2 as b on a.id = b.id
Using the tables below as an example and the listed query as a base query, I want to add a way to select only rows with a max id! Without having to do a second query!
TABLE VEHICLES
id vehicleName
----- --------
1 cool car
2 cool car
3 cool bus
4 cool bus
5 cool bus
6 car
7 truck
8 motorcycle
9 scooter
10 scooter
11 bus
TABLE VEHICLE NAMES
nameId vehicleName
------ -------
1 cool car
2 cool bus
3 car
4 truck
5 motorcycle
6 scooter
7 bus
TABLE VEHICLE ATTRIBUTES
nameId attribute
------ ---------
1 FAST
1 SMALL
1 SHINY
2 BIG
2 SLOW
3 EXPENSIVE
4 SHINY
5 FAST
5 SMALL
6 SHINY
6 SMALL
7 SMALL
And the base query:
select a.*
from vehicle a
join vehicle_names b using(vehicleName)
join vehicle_attribs c using(nameId)
where c.attribute in('SMALL', 'SHINY')
and a.vehicleName like '%coo%'
group
by a.id
having count(distinct c.attribute) = 2;
So what I want to achieve is to select rows with certain attributes, that match a name but only one entry for each name that matches where the id is the highest!
So a working solution in this example would return the below rows:
id vehicleName
----- --------
2 cool car
10 scooter
if it was using some sort of max on the id
at the moment I get all the entries for cool car and scooter.
My real world database follows a similar structure and has 10's of thousands of entries in it so a query like above could easily return 3000+ results. I limit the results to 100 rows to keep execution time low as the results are used in a search on my site. The reason I have repeats of "vehicles" with the same name but only a different ID is that new models are constantly added but I keep the older one around for those that want to dig them up! But on a search by car name I don't want to return the older cards just the newest one which is the one with the highest ID!
The correct answer would adapt the query I provided above that I'm currently using and have it only return rows where the name matches but has the highest id!
If this isn't possible, suggestions on how I can achieve what I want without massively increasing the execution time of a search would be appreciated!
If you want to keep your logic, here what I would do:
select a.*
from vehicle a
left join vehicle a2 on (a.vehicleName = a2.vehicleName and a.id < a2.id)
join vehicle_names b on (a.vehicleName = b.vehicleName)
join vehicle_attribs c using(nameId)
where c.attribute in('SMALL', 'SHINY')
and a.vehicleName like '%coo%'
and a2.id is null
group by a.id
having count(distinct c.attribute) = 2;
Which yield:
+----+-------------+
| id | vehicleName |
+----+-------------+
| 2 | cool car |
| 10 | scooter |
+----+-------------+
2 rows in set (0.00 sec)
As other said, normalization could be done on few levels:
Keeping your current vehicle_names table as the primary lookup table, I would change:
update vehicle a
inner join vehicle_names b using (vehicleName)
set a.vehicleName = b.nameId;
alter table vehicle change column vehicleName nameId int;
create table attribs (
attribId int auto_increment primary key,
attribute varchar(20),
unique key attribute (attribute)
);
insert into attribs (attribute)
select distinct attribute from vehicle_attribs;
update vehicle_attribs a
inner join attribs b using (attribute)
set a.attribute=b.attribId;
alter table vehicle_attribs change column attribute attribId int;
Which led to the following query:
select a.id, b.vehicleName
from vehicle a
left join vehicle a2 on (a.nameId = a2.nameId and a.id < a2.id)
join vehicle_names b on (a.nameId = b.nameId)
join vehicle_attribs c on (a.nameId=c.nameId)
inner join attribs d using (attribId)
where d.attribute in ('SMALL', 'SHINY')
and b.vehicleName like '%coo%'
and a2.id is null
group by a.id
having count(distinct d.attribute) = 2;
The table does not seems normalized, however this facilitate you to do this :
select max(id), vehicleName
from VEHICLES
group by vehicleName
having count(*)>=2;
I'm not sure I completely understand your model, but the following query satisfies your requirements as they stand. The first sub query finds the latest version of the vehicle. The second query satisfies your "and" condition. Then I just join the queries on vehiclename (which is the key?).
select a.id
,a.vehiclename
from (select a.vehicleName, max(id) as id
from vehicle a
where vehicleName like '%coo%'
group by vehicleName
) as a
join (select b.vehiclename
from vehicle_names b
join vehicle_attribs c using(nameId)
where c.attribute in('SMALL', 'SHINY')
group by b.vehiclename
having count(distinct c.attribute) = 2
) as b on (a.vehicleName = b.vehicleName);
If this "latest vehicle" logic is something you will need to do a lot, a small suggestion would be to create a view (see below) which returns the latest version of each vehicle. Then you could use the view instead of the find-max-query. Note that this is purely for ease-of-use, it offers no performance benefits.
select *
from vehicle a
where id = (select max(b.id)
from vehicle b
where a.vehiclename = b.vehiclename);
Without going into proper redesign of you model you could
1) Add a column IsLatest that your application could manage.
This is not perfect but will satisfy you question (until next problem, see not at the end)
All you need is when you add a new entry to issue queries such as
UPDATE a
SET IsLatest = 0
WHERE IsLatest = 1
INSERT new a
UPDATE a
SET IsLatest = 1
WHERE nameId = #last_inserted_id
in a transaction or a trigger
2) Alternatively you can find out the max_id before you issue your query
SELECT MAX(nameId)
FROM a
WHERE vehicleName = #name
3) You can do it in single SQL, and providing indexes on (vehicleName, nameId) it should actually have decent speed with
select a.*
from vehicle a
join vehicle_names b ON a.vehicleName = b.vehicleName
join vehicle_attribs c ON b.nameId = c.nameId AND c.attribute = 'SMALL'
join vehicle_attribs d ON b.nameId = c.nameId AND d.attribute = 'SHINY'
join vehicle notmax ON a.vehicleName = b.vehicleName AND a.nameid < notmax.nameid
where a.vehicleName like '%coo%'
AND notmax.id IS NULL
I have removed your GROUP BY and HAVING and replaced it with another join (assuming that only single attribute per nameId is possible).
I have also used one of the ways to find max per group and that is to join a table on itself and filter out a row for which there are no records that have a bigger id for a same name.
There are other ways, search so for 'max per group sql'. Also see here, though not complete.