Group_Concat with multiple joined tables - mysql

Question

I have two main tables that comprise bookings for events.
A Registrants table (Bookings) R and an Events table E.
There are also two connected tables, Field_Values V and Event_Categories C
This diagram shows the relationship
What I am trying to do is create an Invoice query that mirrors the user's shopping cart. Often a user will book multiple events in one transaction, so my invoice should have columns for the common items e.g. User Name, User Email, Booking Date, Transaction ID and aggregated columns for the invoice line item values e.g. Quantity "1,2" Description "Desc1, Desc2" Price "10.00, 20.00" where there are two line items in the shopping cart.
The Transaction ID (dcea4_eb_registrant.transaction_id) is unique per Invoice and repeated per line item in that sale.
I have the following query which produces rows for each line item
SELECT
R.id as ID,
E.event_date as ServiceDate,
E.event_date - INTERVAL 1 DAY as DueDate,
Concat('Ad-Hoc Booking:',E.title) as ItemProductService,
Concat(R.first_name, ' ',R.last_name) as Customer,
R.first_name as FirstName,
R.last_name as LastName,
R.email,
R.register_date as InvoiceDate,
R.amount as ItemAmount,
R.comment,
R.number_registrants as ItemQuantity,
R.transaction_id as InvoiceNo,
R.published as Status,
E.event_date AS SERVICEDATE,
Concat('Ad-Hoc Booking:',E.title) AS DESCRIPTION,
R.number_registrants AS QUANTITY,
FORMAT(R.amount / R.number_registrants,2) AS RATE,
R.amount AS AMOUNT,
C.category_id as CLASS,
Concat(Group_Concat(V.field_value SEPARATOR ', '),'. ',R.comment) as Memo
FROM dcea4_eb_events E
LEFT JOIN dcea4_eb_registrants R ON R.event_id = E.id
LEFT JOIN dcea4_eb_field_values V ON V.registrant_id = R.id
LEFT JOIN dcea4_eb_event_categories C ON C.event_id = R.event_id
WHERE 1=1
AND V.field_id IN(14,26,27,15)
AND R.published <> 2 /*Including this line omits Cancelled Invoices */
AND R.published IS NOT NULL
AND (R.published = 1 OR R.payment_method = "os_offline")
AND (R.register_date >= CURDATE() - INTERVAL 14 DAY)
GROUP BY E.event_date, E.title, R.id, R.first_name, R.last_name, R.email,R.register_date, R.amount, R.comment
ORDER BY R.register_date DESC, R.transaction_id
This produces output like this
I'm using the following query to try to group together the rows with a common transaction_ID (rows two and three in the last picture) - I add group_concat on the columns I want to aggregate and change the Group By to be the transaction_id
SELECT
R.id as ID,
E.event_date as ServiceDate,
E.event_date - INTERVAL 1 DAY as DueDate,
Concat('Ad-Hoc Booking:',E.title) as ItemProductService,
Concat(R.first_name, ' ',R.last_name) as Customer,
R.first_name as FirstName,
R.last_name as LastName,
R.email,
R.register_date as InvoiceDate,
R.amount as ItemAmount,
R.comment,
R.number_registrants as ItemQuantity,
R.transaction_id as InvoiceNo,
R.published as Status,
Group_ConCat( E.event_date) AS SERVICEDATE,
Group_ConCat( Concat('Ad-Hoc Booking:',E.title)) AS DESCRIPTION,
Group_ConCat( R.number_registrants) AS QUANTITY,
Group_ConCat( FORMAT(R.amount / R.number_registrants,2)) AS RATE2,
Group_ConCat( R.amount) AS AMOUNT,
Group_ConCat( C.category_id) as CLASS,
Concat(Group_Concat(V.field_value SEPARATOR ', '),'. ',R.comment) as Memo
FROM dcea4_eb_events E
LEFT JOIN dcea4_eb_registrants R ON R.event_id = E.id
LEFT JOIN dcea4_eb_field_values V ON V.registrant_id = R.id
LEFT JOIN dcea4_eb_event_categories C ON C.event_id = R.event_id
WHERE 1=1
AND V.field_id IN(14,26,27,15)
AND R.published <> 2 /*Including this line omits Cancelled Invoices */
AND R.published IS NOT NULL
AND (R.published = 1 OR R.payment_method = "os_offline")
AND (R.register_date >= CURDATE() - INTERVAL 14 DAY)
GROUP BY R.transaction_id
ORDER BY R.register_date DESC, R.transaction_id
But this produces this output
It seems to be multiplying the rows. The Quantity column in the first row should just be 1 and in the second row it should be 2,1 .
I've tried using Group_Concat with DISTINCT but this doesn't work because often the values being concatenated are the same (e.g. the price for two events being booked are both the same) and the query only returns one value e.g. 10 and not 10, 10. The latter being what I need.
I'm guessing the issue is around the way the tables are joined but I'm struggling to work out how to get what I need.
Pointers in the right direction most appreciated.

Answer 1

You seem determined to go in what seems to me to be the wrong direction, so here's a gentle nudge down that hill...
Consider the following...
CREATE TABLE users
(user_id SERIAL PRIMARY KEY
,username VARCHAR(12) UNIQUE
);
INSERT INTO users VALUES
(101,'John'),(102,'Paul'),(103,'George'),(104,'Ringo');
DROP TABLE IF EXISTS sales;
CREATE TABLE sales
(sale_id SERIAL PRIMARY KEY
,purchaser_id INT NOT NULL
,item_code CHAR(1) NOT NULL
,quantity INT NOT NULL
);
INSERT INTO sales VALUES
( 1,101,'A',1),
( 2,103,'A',2),
( 3,103,'A',3),
( 4,104,'A',1),
( 5,104,'A',2),
( 6,104,'A',3),
( 7,103,'B',2),
( 8,103,'B',2),
( 9,104,'B',3),
(10,103,'B',2),
(11,104,'B',2),
(12,104,'B',1);
SELECT u.*
, x.sale_ids
, x.item_codes
, x.quantities
FROM users u
LEFT
JOIN
( SELECT purchaser_id
, GROUP_CONCAT(sale_id ORDER BY sale_id) sale_ids
, GROUP_CONCAT(item_code ORDER BY sale_id) item_codes
, GROUP_CONCAT(quantity ORDER BY sale_id) quantities
FROM sales
GROUP
BY purchaser_id
) x
ON x.purchaser_id = u.user_id;
+---------+----------+---------------+-------------+-------------+
| user_id | username | sale_ids | item_codes | quantities |
+---------+----------+---------------+-------------+-------------+
| 101 | John | 1 | A | 1 |
| 102 | Paul | NULL | NULL | NULL |
| 103 | George | 2,3,7,8,10 | A,A,B,B,B | 2,3,2,2,2 |
| 104 | Ringo | 4,5,6,9,11,12 | A,A,A,B,B,B | 1,2,3,3,2,1 |
+---------+----------+---------------+-------------+-------------+