I have two main tables that comprise bookings for events.
A Registrants table (Bookings) R and an Events table E.
There are also two connected tables, Field_Values V and Event_Categories C
This diagram shows the relationship
What I am trying to do is create an Invoice query that mirrors the user's shopping cart. Often a user will book multiple events in one transaction, so my invoice should have columns for the common items e.g. User Name, User Email, Booking Date, Transaction ID and aggregated columns for the invoice line item values e.g. Quantity "1,2" Description "Desc1, Desc2" Price "10.00, 20.00" where there are two line items in the shopping cart.
The Transaction ID (dcea4_eb_registrant.transaction_id) is unique per Invoice and repeated per line item in that sale.
I have the following query which produces rows for each line item
SELECT
R.id as ID,
E.event_date as ServiceDate,
E.event_date - INTERVAL 1 DAY as DueDate,
Concat('Ad-Hoc Booking:',E.title) as ItemProductService,
Concat(R.first_name, ' ',R.last_name) as Customer,
R.first_name as FirstName,
R.last_name as LastName,
R.email,
R.register_date as InvoiceDate,
R.amount as ItemAmount,
R.comment,
R.number_registrants as ItemQuantity,
R.transaction_id as InvoiceNo,
R.published as Status,
E.event_date AS SERVICEDATE,
Concat('Ad-Hoc Booking:',E.title) AS DESCRIPTION,
R.number_registrants AS QUANTITY,
FORMAT(R.amount / R.number_registrants,2) AS RATE,
R.amount AS AMOUNT,
C.category_id as CLASS,
Concat(Group_Concat(V.field_value SEPARATOR ', '),'. ',R.comment) as Memo
FROM dcea4_eb_events E
LEFT JOIN dcea4_eb_registrants R ON R.event_id = E.id
LEFT JOIN dcea4_eb_field_values V ON V.registrant_id = R.id
LEFT JOIN dcea4_eb_event_categories C ON C.event_id = R.event_id
WHERE 1=1
AND V.field_id IN(14,26,27,15)
AND R.published <> 2 /*Including this line omits Cancelled Invoices */
AND R.published IS NOT NULL
AND (R.published = 1 OR R.payment_method = "os_offline")
AND (R.register_date >= CURDATE() - INTERVAL 14 DAY)
GROUP BY E.event_date, E.title, R.id, R.first_name, R.last_name, R.email,R.register_date, R.amount, R.comment
ORDER BY R.register_date DESC, R.transaction_id
This produces output like this
I'm using the following query to try to group together the rows with a common transaction_ID (rows two and three in the last picture) - I add group_concat on the columns I want to aggregate and change the Group By to be the transaction_id
SELECT
R.id as ID,
E.event_date as ServiceDate,
E.event_date - INTERVAL 1 DAY as DueDate,
Concat('Ad-Hoc Booking:',E.title) as ItemProductService,
Concat(R.first_name, ' ',R.last_name) as Customer,
R.first_name as FirstName,
R.last_name as LastName,
R.email,
R.register_date as InvoiceDate,
R.amount as ItemAmount,
R.comment,
R.number_registrants as ItemQuantity,
R.transaction_id as InvoiceNo,
R.published as Status,
Group_ConCat( E.event_date) AS SERVICEDATE,
Group_ConCat( Concat('Ad-Hoc Booking:',E.title)) AS DESCRIPTION,
Group_ConCat( R.number_registrants) AS QUANTITY,
Group_ConCat( FORMAT(R.amount / R.number_registrants,2)) AS RATE2,
Group_ConCat( R.amount) AS AMOUNT,
Group_ConCat( C.category_id) as CLASS,
Concat(Group_Concat(V.field_value SEPARATOR ', '),'. ',R.comment) as Memo
FROM dcea4_eb_events E
LEFT JOIN dcea4_eb_registrants R ON R.event_id = E.id
LEFT JOIN dcea4_eb_field_values V ON V.registrant_id = R.id
LEFT JOIN dcea4_eb_event_categories C ON C.event_id = R.event_id
WHERE 1=1
AND V.field_id IN(14,26,27,15)
AND R.published <> 2 /*Including this line omits Cancelled Invoices */
AND R.published IS NOT NULL
AND (R.published = 1 OR R.payment_method = "os_offline")
AND (R.register_date >= CURDATE() - INTERVAL 14 DAY)
GROUP BY R.transaction_id
ORDER BY R.register_date DESC, R.transaction_id
But this produces this output
It seems to be multiplying the rows. The Quantity column in the first row should just be 1 and in the second row it should be 2,1 .
I've tried using Group_Concat with DISTINCT but this doesn't work because often the values being concatenated are the same (e.g. the price for two events being booked are both the same) and the query only returns one value e.g. 10 and not 10, 10. The latter being what I need.
I'm guessing the issue is around the way the tables are joined but I'm struggling to work out how to get what I need.
Pointers in the right direction most appreciated.
You seem determined to go in what seems to me to be the wrong direction, so here's a gentle nudge down that hill...
Consider the following...
CREATE TABLE users
(user_id SERIAL PRIMARY KEY
,username VARCHAR(12) UNIQUE
);
INSERT INTO users VALUES
(101,'John'),(102,'Paul'),(103,'George'),(104,'Ringo');
DROP TABLE IF EXISTS sales;
CREATE TABLE sales
(sale_id SERIAL PRIMARY KEY
,purchaser_id INT NOT NULL
,item_code CHAR(1) NOT NULL
,quantity INT NOT NULL
);
INSERT INTO sales VALUES
( 1,101,'A',1),
( 2,103,'A',2),
( 3,103,'A',3),
( 4,104,'A',1),
( 5,104,'A',2),
( 6,104,'A',3),
( 7,103,'B',2),
( 8,103,'B',2),
( 9,104,'B',3),
(10,103,'B',2),
(11,104,'B',2),
(12,104,'B',1);
SELECT u.*
, x.sale_ids
, x.item_codes
, x.quantities
FROM users u
LEFT
JOIN
( SELECT purchaser_id
, GROUP_CONCAT(sale_id ORDER BY sale_id) sale_ids
, GROUP_CONCAT(item_code ORDER BY sale_id) item_codes
, GROUP_CONCAT(quantity ORDER BY sale_id) quantities
FROM sales
GROUP
BY purchaser_id
) x
ON x.purchaser_id = u.user_id;
+---------+----------+---------------+-------------+-------------+
| user_id | username | sale_ids | item_codes | quantities |
+---------+----------+---------------+-------------+-------------+
| 101 | John | 1 | A | 1 |
| 102 | Paul | NULL | NULL | NULL |
| 103 | George | 2,3,7,8,10 | A,A,B,B,B | 2,3,2,2,2 |
| 104 | Ringo | 4,5,6,9,11,12 | A,A,A,B,B,B | 1,2,3,3,2,1 |
+---------+----------+---------------+-------------+-------------+
See SQLFiddle here: http://sqlfiddle.com/#!9/9bb273
I need to create a report out of 3 queries. It needs to be a single query without subqueries (due to ORM limitations).
The main query is:
SELECT SQL_CALC_FOUND_ROWS
Organization.name as organization_name,
Program.unique_id as program_uuid,
Program.name as program_name,
Program.start_date,
Program.end_date,
Program.grace_period,
'Placeholder A' as 'Participant Count',
'Placeholder B' as 'Total Participant Points',
count(distinct Transaction.id) as 'Transaction Count',
sum(TransactionItem.quantity) as 'Total Redemptions',
sum(((TransactionProduct.retail + IFNULL(TransactionProduct.shipping,0) + IFNULL(TransactionProduct.handling,0)) * TransactionItem.quantity)) as 'Total'
FROM `TransactionItem`
JOIN `Transaction` ON `Transaction`.id = `TransactionItem`.transaction_id
JOIN `TransactionProduct` ON `TransactionItem`.reference_id = `TransactionProduct`.reference_id
JOIN `Participant` ON `Transaction`.participant_id = `Participant`.id
JOIN `Program` ON `Program`.id = `Participant`.program_id
JOIN `Organization` ON `Organization`.id = `Participant`.organization_id
WHERE 1=1
AND `Organization`.`unique_id` = 'demo2'
AND `Program`.`unique_id` = 'demo2'
AND `Transaction`.`created_at` >= '2018-10-01 00:00:00'
AND `Transaction`.`created_at` <= '2018-12-18 00:00:00';
As you can see, this report is for the date range between 10/1 and 12/18. The result set that makes up the report is...
organization_name | program_uuid | program_name | start_date | end_date | grace_period | Participant Count | Total Participant Points | Transaction Count | Total Redemptions | Total
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Demo2 Org demo2 Demo2 2018-10-01 2018-12-27 5 Placeholder A Placeholder B 11 92 2853.13
As you can see, there are 2 data points that I'm unable to get from this query.
(1) The total number of participants belonging to the "demo2" Program. This query gets that data point.
/* Placeholder A */
select program_id, count(*) as 'Participant Count' from participant
where active = 1
group by program_id;
Returns:
program_id | Participant Count
----------------------------------
2 102
(2) The sum of Adjustments.amount for all rows between dates 10/1 and 12/18. This query fulfills that.
/* Placeholder B */
select sum(amount) as 'Total Particpant Points' from adjustment
where participant_id in (select id from participant where program_id =2)
and type = 1
and created_at >= '2018-10-01 00:00:00' and created_at <= '2018-12-18 00:00:00';
Returns:
Total Participant Points
------------------------
10000.50000
Is there a way to gather all of this data in a single query without subqueries?
I have two tables:
booking - records the order detail
id | booking_amount
-------------------
1 | 150
2 | 500
3 | 400
payment - records the payment for order
id | booking_id | amount
------------------------
1 | 1 | 100
2 | 1 | 50
2 | 2 | 100
I want to find all bookings where the payments are not complete. With the above data, we expect the answer to be 2,3, because the sum of payments for booking_id=1 matches the corresponding booking_amount in the booking_table.
To answer your question, you have 2 things you need to think about :
you want the total amount in your table payment by every booking row
you want to join your booking_amount table with payment.
Part 1 is quite simple:
SELECT sum(amount) as TotalP, booking_id FROM payment GROUP BY booking_id
Just a basic query with a simple aggregate function...
For part 2, we want to join booking_amount and payment; the basic JOIN would be:
SELECT * FROM booking b
LEFT JOIN payment p ON b.id = p.booking_id
We do a LEFT JOIN because we may have some booking who are not in the payment table. For those bookings, you will get NULL value. We will use a COALESCE to replace the NULL values by 0.
The final query is this:
SELECT b.id, COALESCE(TotalP, 0), b.booking_amount
FROM
booking b
LEFT JOIN
(SELECT sum(amount) as TotalP, booking_id FROM payment GROUP BY booking_id) as T
ON b.id = T.booking_id
WHERE COALESCE(TotalP, 0) < b.booking_amount
You need to use a outer join to combine your two tables and look for your conditions. Also, you will need to use SUM(..) function to get the sum of the amount for each id in the payment table.
Please try this:
select b.id from booking b
left outer join -- cant be inner join because we lose id:3 in that case.
(
select booking_id, SUM(amount) as Total
from payment group by booking_id
) p on b.id = p.booking_id
where b.booking_amount > Coalesce(Total,0) --Coalesce is required for such values coming NULL, like id:3, they will be assigned as 0.
im having difficulty with the following fairly simple setup:
CREATE TABLE IF NOT EXISTS invoices (
id int(11) NOT NULL auto_increment,
PRIMARY KEY (id)
);
CREATE TABLE IF NOT EXISTS invoices_items (
id int(11) NOT NULL auto_increment,
invoice_id int(11) NOT NULL,
description text NOT NULL,
amount decimal(10,2) NOT NULL default '0.00',
PRIMARY KEY (id)
);
CREATE TABLE IF NOT EXISTS invoices_payments (
id int(11) NOT NULL auto_increment,
invoice_id int(11) NOT NULL,
amount decimal(10,2) NOT NULL default '0.00',
PRIMARY KEY (id)
);
some data:
INSERT INTO invoices (id) VALUES (1);
INSERT INTO invoices_items (id, invoice_id, description, amount) VALUES
(1, 1, 'Item 1', '750.00'),
(2, 1, 'Item 2', '750.00'),
(3, 1, 'Item 3', '50.00'),
(4, 1, 'Item 4', '150.00');
INSERT INTO invoices_payments (id, invoice_id, amount) VALUES
(1, 1, '50.00'),
(2, 1, '1650.00');
and the sql yielding unusual results:
select invoices.id,
ifnull(sum(invoices_payments.amount),0) as payments_total,
ifnull(count(invoices_items.id),0) as item_count
from invoices
left join invoices_items on invoices_items.invoice_id=invoices.id
left join invoices_payments on invoices_payments.invoice_id=invoices.id
group by invoices.id
results in the (erroneous) output
id payments_total item_count
1 6800.00 8
now, as evidenced by there being infact only four 'invoice_item' rows, i dont understand why mysql is not grouping properly.
EDIT
i know i can do something like this:
select x.*, ifnull(sum(invoices_payments.amount),0) as payments_total from (
select invoices.id,
ifnull(count(invoices_items.id),0) as item_count
from invoices
left join invoices_items on invoices_items.invoice_id=invoices.id
group by invoices.id
) as x left join invoices_payments on invoices_payments.invoice_id=x.id
group by x.id
but i want to know if im doing something wrong in the first query - i cant immediately see why the first query is giving incorrect results! :(
Your join logic is incorrect. In your join, you specify invoices_items.invoice_id = invoices.id. You also specify invoices_payments.invoice_id = invoices.id. Because of transitivity, you end up with:
invoices_items.invoice_id = invoices.id
invoices_payments.invoice_id = invoices.id
invoice_items.invoice_id = invoices_payments.invoice_id
The sum of the 2 invoice payments is $1700. For every invoice payment, there are 4 invoice_items that satisfy the above relations. $1700 * 4 = $6800.
For every invoice item, there will be two invoice payments that satisfy the above relations. 4 invoice items * 2 = 8 count.
There are two tables with a many:one relationship with invoices. Your count is the cartesian product.
The payments should be applied to the invoice, not the invoice items. Get the invoice total first, then join the payments to it.
This may be similar to what you are looking for:
SELECT
invoice_total.invoice_id,
invoice_total.amount as invoice_amount,
payments_total.amount as total_paid
FROM
(
SELECT
invoice_id,
SUM(amount) as amount
FROM
invoices_items
GROUP BY
invoice_id
) invoice_total
INNER JOIN
(
SELECT
invoice_id,
SUM(amount) as amount
FROM
invoices_payments
GROUP BY
invoice_id
) payments_total
ON invoice_total.invoice_id = payments_total.invoice_id;
edit:
ah, sorry - see your point now. The reason you're getting unexpected results is that this query:
SELECT *
FROM invoices
LEFT JOIN invoices_items ON invoices_items.invoice_id = invoices.id
LEFT JOIN invoices_payments ON invoices_payments.invoice_id = invoices.id;
results in this:
id id invoice_id description amount id invoice_id amount
1 1 1 Item 1 750.00 1 1 50.00
1 1 1 Item 1 750.00 2 1 1650.00
1 2 1 Item 2 750.00 1 1 50.00
1 2 1 Item 2 750.00 2 1 1650.00
1 3 1 Item 3 50.00 1 1 50.00
1 3 1 Item 3 50.00 2 1 1650.00
1 4 1 Item 4 150.00 1 1 50.00
1 4 1 Item 4 150.00 2 1 1650.00
As you can see you get every invoices_items record once each for every invoices_payments record. You're going to have to grab (i.e. group) them separately.
Note that the GROUP BY clause in your initial query is redundant.
Here's what you need:
SELECT
invoices.id,
payments_total.payments_total,
IFNULL(COUNT(invoices_items.id),0) AS item_count
FROM invoices
LEFT JOIN invoices_items ON invoices.id = invoices_items.invoice_id
LEFT JOIN (
SELECT invoice_id,
IFNULL(SUM(invoices_payments.amount),0) AS payments_total
FROM invoices_payments
GROUP BY invoice_id
) AS payments_total ON invoices.id = payments_total.invoice_id
;