MySQL GROUP_CONCAT Query Excluding Records - mysql

I'm having a bit of an issue with my query below. I'm pulling data from a few tables and using the GROUP_CONCAT feature for a few rows. The GROUP_CONCAT pulls custom field data based on requests.REQUEST_ID. However, not every REQUEST_ID has an entry in TABLE_fields_data. Therefore, this element in my WHERE clause is causing an issue: data.REQUEST_ID=requests.REQUEST_ID.
I still want to pull the data from TABLE_requests even if there are no corresponding values in TABLE_fields_data. With my current query, the data is being completely omitted.
I've tried IFNULL in my GROUP_CONTACT statements but that didn't seem to make a difference. I've tried figuring out how to do an IF statement in the data.REQUEST_ID=requests.REQUEST_ID WHERE clause but haven't gotten that to work either. Based on my sample data below, the query would not return the request ID 6000002.
Really stuck on this and would be grateful for any help!
TABLE_requests
+------------+----------------+---------+---------------------+
| REQUEST_ID | REQUEST_STATUS | USER_ID | DATE_CREATE |
+------------+----------------+---------+---------------------+
| 6000001 | COMPLETE | 3000001 | 2020-06-01 11:15:16 |
+------------+----------------+---------+---------------------+
| 6000002 | COMPLETE | 3000001 | 2020-06-02 16:14:11 |
+------------+----------------+---------+---------------------+
| 6000003 | PENDING | 3000001 | 2020-06-03 12:11:11 |
+------------+----------------+---------+---------------------+
TABLE_fields
+----------+------+---------------+
| FIELD_ID | TYPE | LABEL_en |
+----------+------+---------------+
| 1 | TEXT | Student Name |
+----------+------+---------------+
| 2 | TEXT | Student Grade |
+----------+------+---------------+
| 3 | TEXT | Course |
+----------+------+---------------+
TABLE_fields_data
+---------+------------+----------+------------+
| DATA_ID | REQUEST_ID | FIELD_ID | DATA |
+---------+------------+----------+------------+
| 1 | 6000001 | 1 | John Smith |
+---------+------------+----------+------------+
| 2 | 6000001 | 2 | 1st Grade |
+---------+------------+----------+------------+
SELECT
requests.REQUEST_ID, requests.REQUEST_STATUS, requests.USER_ID, requests.DATE_CREATE,
GROUP_CONCAT(data.FIELD_ID SEPARATOR '||') AS CUSTOM_FIELD_ID,
GROUP_CONCAT(fields.TYPE SEPARATOR '||') AS CUSTOM_FIELD_TYPE,
GROUP_CONCAT(fields.LABEL_en SEPARATOR '||') AS CUSTOM_FIELD_LABEL,
GROUP_CONCAT(data.DATA SEPARATOR '||') AS CUSTOM_FIELD_DATA
FROM TABLE_requests AS requests, TABLE_fields_data AS data, TABLE_fields AS fields
WHERE requests.REQUEST_STATUS='COMPLETE' AND data.REQUEST_ID=requests.REQUEST_ID AND fields.FIELD_ID=data.FIELD_ID GROUP BY requests.REQUEST_ID;

You are using INNER joins (old style) but what you need is LEFT joins:
SELECT
requests.REQUEST_ID, requests.REQUEST_STATUS, requests.USER_ID, requests.DATE_CREATE,
GROUP_CONCAT(data.FIELD_ID SEPARATOR '||') AS CUSTOM_FIELD_ID,
GROUP_CONCAT(fields.TYPE SEPARATOR '||') AS CUSTOM_FIELD_TYPE,
GROUP_CONCAT(fields.LABEL_en SEPARATOR '||') AS CUSTOM_FIELD_LABEL,
GROUP_CONCAT(data.DATA SEPARATOR '||') AS CUSTOM_FIELD_DATA
FROM TABLE_requests AS requests
LEFT JOIN TABLE_fields_data AS data ON data.REQUEST_ID = requests.REQUEST_ID
LEFT JOIN TABLE_fields AS fields ON fields.FIELD_ID = data.FIELD_ID
WHERE requests.REQUEST_STATUS = 'COMPLETE'
GROUP BY requests.REQUEST_ID;

Related

GroupBy ID but keep multiple values

I have the following MySQL table:
+----------+----------+---------+-------------+------------+----------+----------+-----------+
| queue_id | email_id | user_id | customer_id | send_date | campaign | approved | scheduled |
+----------+----------+---------+-------------+------------+----------+----------+-----------+
| 1 | 1 | 1 | 1 | 2018-10-30 | 1 | 1 | 1 |
| 2 | 1 | 2 | 1 | 2018-10-30 | 1 | 1 | 1 |
| 3 | 2 | 1 | 1 | 2018-11-02 | 1 | 1 | 1 |
| 4 | 2 | 2 | 1 | 2018-11-02 | 1 | 0 | 1 |
| 5 | 2 | 3 | 1 | 2018-11-02 | 1 | 1 | 1 |
+----------+----------+---------+-------------+------------+----------+----------+-----------+
Where the email_id, user_id, and customer_id are all foreign keys.
What I need to do is return the send_date, subject (which is apart of the email table that the email_id references), and name (which is apart of the business table that the user_id references) but only for columns where the approved column is true. The idea is to ultimately display the data to a user in an HTML table where the table would look like the following (using the sample data provided):
+--------------------+--------------------------+---------------+
| October 30th, 2018 | Subject for email_id "1" | View Approved |
| November 2nd, 2018 | Subject for email_id "2" | View Approved |
+--------------------+--------------------------+---------------+
Whenever the user would click on the "View Approved" cell, then it would display all of the business names that approved that particular email.
I tried using the following query, but it is only returning one value in the name column:
SELECT
DATE_FORMAT(q.`send_date`, "%M %D, %Y") AS `date_visited`,
e.`subject`,
b.`name`
FROM
`email_queue` AS q
INNER JOIN
`email` AS e ON q.`email_id` = e.`email_id`
INNER JOIN
`user` AS u ON q.`user_id` = u.`user_id`
INNER JOIN
`business` AS b ON u.`business_id` = b.`business_id`
WHERE
q.`approved` = true
GROUP BY
e.`email_id`
ORDER BY
q.`send_date` DESC
How can I structure my query to where it would return all of the business names in the name column instead of just one?
You can get all the unqiue business names in a Comma separated string, using Group_Concat() function with Distinct clause.
Try:
GROUP_CONCAT(DISTINCT b.`name` SEPARATOR ',') AS name
instead of:
b.`name`
Note:
You can avoid the usage of Distinct clause, if there would not be any duplicate user_id (for a specific email_id), thus ensuring that b.name is also unique.
You can also use any separator, instead of comma. For eg: to use separator as pipe character |, you would write the query as:
GROUP_CONCAT(DISTINCT b.nameSEPARATOR '|') AS name

Merge multiple rows together MySQL

I have a table that looks more or less like this:
user is_su last_login roles_for_groups
+------+---+------------+----------------------------+
| rob | 1 | 2018-02-09 | admin, read |
+------+---+------------+----------------------------+
| gian | 0 | 2018-06-21 | prod_full_access, readOnly |
+------+---+------------+----------------------------+
| gian | 0 | 2018-06-21 | prod_full_access, CCT |
+------+---+------------+----------------------------+
| rob | 1 | 2018-02-09 | admin, write |
+------+---+------------+----------------------------+
and I would like to merge into a single row all the rows with the same user, in a way such that the table will look like this:
+------+---+------------+---------------------------------+--+
| rob | 1 | 2018-02-09 | admin, read, write | |
+------+---+------------+---------------------------------+--+
| gian | 0 | 2018-06-21 | prod_full_access, readOnly, CCT | |
+------+---+------------+---------------------------------+--+
How can I achieve this?
Why not just query out the data the way you want, using GROUP_CONCAT with DISTINCT:
SELECT
user,
is_us,
last_login,
GROUP_CONCAT(DISTINCT roles_for_groups) roles_for_groups
FROM yourTable
GROUP BY
user,
is_us,
last_login;
It isn't clear whether the is_us and last_login columns are always functionally dependent on the user. If not, then you should include some sample data which reveals this behavior.
select a.user,a.is_su,a.last_login,group_concat(distinct a.roles_for_groups separator ',' )
from (select user,is_su,last_login,substring_index(roles_for_groups,',',1) as roles_for_groups from table union select user,is_su,last_login,substring_index(roles_for_groups,',',-1) as roles_for_groups from table) a group by a.user
this query is only for roles_for_groups containing two values

MySQL concat columns and rows from multiple tables

I'm trying to concatenate data from three related tables according to:
orders orderrow orderrow_op
+----+ +----+----------+ +----+-------------+
| id | | id | id_order | | id | id_orderrow |
+----+ +----+----------+ +----+-------------+
| 1 | | 1 | 1 | | 1 | 1 |
| 2 | | 2 | 1 | | 2 | 1 |
| 3 | | 3 | 2 | | 3 | 2 |
+----+ | 4 | 3 | | 4 | 3 |
+----+----------+ | 5 | 3 |
| 6 | 3 |
+----+-------------+
The result i'm looking for is something like:
orderops (Desired Result)
+----------+-----------------+
| id_order | id_row:id_ops |
+----------+-----------------+
| 1 | 1:(1,2); 2:(3); |
| 2 | 3:(4,5,6) |
| 3 | 4:NULL |
+----------+-----------------+
I.e i want the operations and rows all be displayed on one row related to the order. So far i've tried things like:
SELECT
db.orders.id AS orderid,
db.orderrow.id AS rowids,
GROUP_CONCAT(DISTINCT db.orderrow.id) AS a,
GROUP_CONCAT(db.orderrow.id, ':', db.orderrow_op.id) AS b
FROM
db.orders
LEFT JOIN db.orderrow ON db.orders.id = db.orderrow.id_order
LEFT JOIN db.orderrow_op ON db.orderrow.id = db.orderrow_op.id_orderrow
GROUP BY orderid
Where in column 'a' i get the row ids and in column 'b' i get the operation_ids with corresponding row_id prepended. I'd like to combine the two into a single column such that related values in 'b' will start of with id from 'a' and only show once.
I'm fairly new to MySQL so i don't know if this is even possible or if i'ts a good idea at all? The aim is to structure the data into JSON for delivery via REST application so perhaps it's better to deliver the rows directly to the webserver and handle json parsing over there? I just figured that this approach might be faster.
This is not the nicest query but it's working for your example table setup.
SELECT
o.id AS id_order,
group_concat(sub.ops
SEPARATOR ' ') AS id_row_id_ops
FROM
(SELECT
orderrow.id_order,
IF(isnull(l3.ops), concat(orderrow.id, ':', 'NULL'), concat(orderrow.id, ':', l3.ops)) as ops
FROM
orderrow
LEFT JOIN (SELECT
orderrow_op.id_orderrow,
concat('(', group_concat(orderrow_op.id), '); ') as ops
FROM
orderrow_op
GROUP BY orderrow_op.id_orderrow) l3 ON l3.id_orderrow = orderrow.id) sub
LEFT JOIN
orders o ON o.id = sub.id_order
GROUP BY o.id;
One of the things to mind is the LEFT JOIN and that you need to cast a "null" value to a "null" text (otherwise your element 4 will vanish).
The output:

GROUP_CONCAT limiting to 100 values is not working

I am trying to extract a sum of modified history field limited by 100 and grouped by each account, but the query below extracts sum of all rows from CDR_Accounts with charged_quantity = 60 and does not limit to 100.
table1 = table2 and these are the temporary tables. I have simplified the query and filtered all data in temporary tables.
Nevertheless, the query still gives me the total sum of all 'history' fields, however I need the sum for only 100 of them.
table1/table2 format:
+-----------+----------+------------+-------------+----------------------+
| i_account | id | account_id | CLD | history |
+-----------+----------+------------+-------------+----------------------+
| 10272 | 46479968 | 0814781293 | 27815212963 | +1x60#1.32d100%=1.32 |
| 6316 | 46480100 | 0813741427 | 27780233136 | +1x60#1.32d100%=1.32 |
| 6316 | 46480107 | 0813741427 | 27780233136 | +1x60#1.32d100%=1.32 |
| 13830 | 46480435 | 0814346396 | 27781356515 | +1x60#1.32d100%=1.32 |
| 13830 | 46480436 | 0814346396 | 27781356515 | +1x60#1.32d100%=1.32 |
+-----------+----------+------------+-------------+----------------------+
Account
SELECT sum(SUBSTRING_INDEX(history,'=',-1)),
cdr.i_account,
cdr.account_id
FROM table1 cdr
WHERE cdr.id IN
(SELECT SUBSTRING_INDEX(group_concat(s.id SEPARATOR ','), ',', 100) id
FROM table2 s
GROUP BY id
)
GROUP BY i_account;
I'm not clear about what your query does is. However limit applies to the final result which includes the grouping as well. if you need to limit the result of GROUP_CONCAT, first apply the limit and then apply the grouping

Mysql count per distinct user with a join

currently i have two tables with some data. the first table has the following:
+----------------+-----------+
| name | member_id |
+----------------+-----------+
| Juice Box | 49432 |
| Rainsurge | 49631 |
| spiderpigrider | 50482 |
+----------------+-----------+
The second table has the following:
+------------+-----------+
| recruit_id | bin(refs) |
+------------+-----------+
| 49432 | 1 |
| 49631 | 1 |
| 49432 | 1 |
| 49631 | 1 |
| 49432 | 1 |
| 49631 | 1 |
| 49432 | 1 |
| 49631 | 1 |
| 49432 | 1 |
| 49631 | 1 |
+------------+-----------+
I would like to return the name, total refs and member_id/recruit_id like so (listing only users with at least 1 ref)
+------------+-----------+------------+
| recruit_id | name | total_refs |
+------------+-----------+------------+
| 49631 | Rainsurge | 5 |
| 49432 | Juice Box | 5 |
+------------+-----------+------------+
select r.recruit_id,bin(r.refs),ipb.name from refs as r
inner join syndicate_ipb.core_members as ipb on ipb.member_id=r.recruit_id;
this returned my data but obviously without a total count and repeated names/ids
select r.recruit_id,count(bin(r.refs)),ipb.name from refs as r
inner join syndicate_ipb.core_members as ipb on ipb.member_id=r.recruit_id;
this returned data with the total count of everyone but only one id/name
+------------+--------------------+-----------+
| recruit_id | count(bin(r.refs)) | name |
+------------+--------------------+-----------+
| 49432 | 10 | Juice Box |
+------------+--------------------+-----------+
this returns the data but again without a count
select distinct r.recruit_id,bin(r.refs),ipb.name from refs as r
inner join syndicate_ipb.core_members as ipb on ipb.member_id=r.recruit_id;
+------------+-------------+-----------+
| recruit_id | bin(r.refs) | name |
+------------+-------------+-----------+
| 49432 | 1 | Juice Box |
| 49631 | 1 | Rainsurge |
+------------+-------------+-----------+
Any help or guidance is greatly appreciated. I feel like i'm close here but just not competent enough with SQL to get it. thanks!
You were almost there. You just missed the GROUP BY clause at the end.
Query:
SELECT
r.recruit_id,
count(bin(r.refs)),
ipb.name
FROM refs AS r
INNER JOIN syndicate_ipb.core_members AS ipb
ON ipb.member_id = r.recruit_id
GROUP BY r.recruit_id;
Note:
If bin(refs) column always contains value 1 then actually you don't need to keep that column. In that case you can use count(*) or count(r.recruit_id) to get the count.
And if bin(refs) column contains any value then count will not give you the right answer. In that case you need to use sum like Sum( bin(refs)).
You have to use the group by clause:
select r.recruit_id, ipb.name, count(bin(refs)) as total_refs
from refs as r
inner join syndicate_ipb.core_members as ipb
on ipb.member_id=r.recruit_id
group by r.recruit_id, ipb.name
having count(bin(refs)) >= 1
This group by r.recruit_id, ipb.name will group the results and this having count(bin(refs)) >= 1 will garante that it only returns members with at least one ref
Do not only group your columns just by the ones you want. Even though MySql allows it, it is not SQL Ansi pattern and even MySql now is complying with it. Use an aggregation function grouping with your entire columns on the select statement.
SELECT ipb.*, COUNT(`r`.`recruit_id`) AS cid FROM `ipb`
INNER JOIN `r` ON `r`.`join_id` = ipb.`member_id`
GROUP BY ipb.`member_id`