group concat from subquery very slow - mysql

Coming from here sql group_concat and subquery
I managed to "solve" de problem by doing this a "subquery":
SELECT
GROUP_CONCAT(name,',',results separator '#')
as finalresult
FROM
(
select t.name as name, group_concat(distinct r.idResult separator '-') as results
from threshold t
left join threshold_results r on r.idThreshold = t.idThreshold
group by t.idThreshold, t.name
) final
But it is too slow when there are many records, withouth the subquery the initial solution performs very well. Any ideas?
Thank you!

add index on r.idThreshold, t.name

Related

GROUP_CONCAT DISTINCT , DISTINCT not work

SELECT
GROUP_CONCAT( DISTINCT unit.`name` ORDER BY unit.`order_index` ASC SEPARATOR '/' ) AS nameGroup
FROM
pms_spec_unit unit
JOIN pms_spec spec ON spec.id = unit.spec_id
WHERE
spec.`status` = 1
GROUP BY
unit.spec_id
HAVING
nameGroup LIKE '%年%'
duplicate result
The results I got
mysql version 8.0.25
I solved this issue thanks to Akina's comment. Actually unit.spec_id is different. Use SELECT DISTINCT GROUP_CONCAT(....

Why does the same MySQL query take way longer for a different ID?

This is the query I use to fetch and display data for my profiles pages. Each category has its own tables because they are constantly being added to by users. I'm new to MYSQL and must be doing something wrong because the query takes 0.7 seconds for one provider (id=56), but 27 seconds (!!!) for provider with id=164. I thought they both had about the same amount of data tied to them, so I can't figure out why the one is taking so long.
Since I'm new to MySQL, I don't know if my long query is normal, or if there's a shorter, easier way to do it that I'm not realizing. However, it works just fine on all my providers except the one.
I did an EXPLAIN on both and the only difference was that the "Type" listed for the slow query had 4 "ALL"s and the faster one only had 2, same for the column that says "Extra," - the slow query had 4 that said "Using where; Using join buffer (Block Nested Loop)" and the fast had 2. I'm not sure what those mean, but that's what I'm trying to figure out.
Appreciate your help!
Changing/removing Concat and Distinct doesn't help.
SELECT `providers`.`id`,
`providers`.`provider_first_name`,
`providers`.`provider_last_name`,
`providers`.`p_date_added`,
`degrees`.`degree_type`,
GROUP_CONCAT(DISTINCT `degrees`.`degree_type` SEPARATOR ', ') AS
all_degrees,
`specialties`.`specialty`,
GROUP_CONCAT(DISTINCT `specialties`.`specialty` SEPARATOR ', ') AS
all_specialties,
`locations`.`city`, `locations`.`state`,
GROUP_CONCAT(DISTINCT CONCAT(`locations`.`city`, ', ',
`locations`.`state`)
ORDER BY `locations`.`city`
SEPARATOR '<br>'
) AS all_locations,
`practices`.`practice_name`,
GROUP_CONCAT(DISTINCT `practices`.`practice_name` SEPARATOR '<br>')
AS all_practices,
`links`.`link`,
GROUP_CONCAT(DISTINCT `links`.`link` SEPARATOR '<br>') AS
all_links,
`conditions`.`condition_name`,
GROUP_CONCAT(DISTINCT `conditions`.`condition_name` SEPARATOR ', ')
AS all_conditions,
`reviews`.`review`, `reviews`.`star_rating`,
`reviews`.`r_date_added`,
AVG(`reviews`.`star_rating`) AS rating
FROM `providers`
LEFT JOIN `degrees` ON `providers`.`id` = `degrees`.`prov_id`
LEFT JOIN `specialties` ON `providers`.`id` =
`specialties`.`prov_id`
LEFT JOIN `locations` ON `providers`.`id` = `locations`.`prov_id`
LEFT JOIN `practices` ON `providers`.`id` = `practices`.`prov_id`
LEFT JOIN `links` ON `providers`.`id` = `links`.`prov_id`
LEFT JOIN `conditions` ON `providers`.`id` = `conditions`.`prov_id`
LEFT JOIN `reviews` ON `providers`.`id` = `reviews`.`prov_id`
WHERE `id`= {$providerid}
GROUP BY `id`
Explode-Implode -- The JOINs explode the number of rows, then the GROUP BY implodes down to very few (apparently only one).
To avoid that, move the LEFT JOINs into the SELECT. For example:
SELECT ...
GROUP_CONCAT(DISTINCT `degrees`.`degree_type` SEPARATOR ', ') AS all_degrees,
...
LEFT JOIN `degrees` ON `providers`.`id` = `degrees`.`prov_id`
...
-->
SELECT ...
( SELECT GROUP_CONCAT(DISTINCT `degree_type` SEPARATOR ', ')
FROM `degrees`
WHERE `providers`.`id` = `prov_id`
) AS all_degrees,
...
And the final GROUP BY id can be removed.
At this point, the DISTINCT is probably redundant; try removing it.
EXPLAIN query
ALL means it is equivalent to full scan of table. To prevent this is add index on your table. Add index on a column which you normally use on searching or connecting certain data.
ROWS represents the cardinality of records whithin the indexed records. Smaller the number the better.
https://dev.mysql.com/doc/refman/5.7/en/explain-output.html
Reading more about EXPLAIN really helps.

How to get a concrete row from a subquery comparing values from the main query?

I need to get a concrete row from a subquery that returns multiple values from the value that gets the main query in which the sub is going to be called.
So this is my main query:
select ...
from xxxx cons
inner join ...
inner join ...
inner join ...
And I want to apply this subquery into that monster:
(SELECT con.cs_nombre, GROUP_CONCAT(DISTINCT cli_codigo SEPARATOR ' ')
from trcon con
INNER JOIN trcli clin on con.cs_nombre = clin.cli_nombre
group by con.cs_nombre)
This in order to get the group_concat row from the subquery in which the cs_nombre is the same as the one that gets the main query.
I think you can just use a correlated subquery:
SELECT . . .,
(SELECT GROUP_CONCAT(DISTINCT clin.cli_codigoCli SEPARATOR ' ')
FROM trcli clin
WHERE clin.cli_nombre= cons.cs_nombre
) as cli_codigo

MySQL move where clause inside subquery

I have written the following MySQL query. However, I need to move the FIND_IN_SET where clause inside the sub query which joins the actions table.
Is this possible? When I try and move that part inside the subquery it gives me the error, paths column does not exist.
select
users_roles.role_id,
ar.actions,
a.paths
from users_roles
join
(
select
action_id,
role_id,
group_concat(action_id SEPARATOR ',') as actions
from actions_roles
) as ar on ar.role_id = users_roles.role_id
join
(
select
id,
group_concat(path SEPARATOR ',') as paths
from actions
) as a on a.id = ar.action_id
where
users_roles.user_id = 1 and FIND_IN_SET('/admin', paths);
It is possible, try this:
select
users_roles.role_id,
ar.actions,
a.paths
from users_roles
join
(
select
action_id,
role_id,
group_concat(action_id SEPARATOR ',') as actions
from actions_roles
) as ar on ar.role_id = users_roles.role_id
join
(
select
id,
group_concat(path SEPARATOR ',') as paths
from actions
having FIND_IN_SET('/admin', group_concat(path SEPARATOR ',')) > 0
) as a on a.id = ar.action_id
where
users_roles.user_id = 1
Since the paths column you are passing to the function is actually the result of group_concat() function inside your sub-query, all you need to do is add a HAVING clause in the sub-query and change the alias of the column with the actual call of the function, which you also use to generate the data.
Why HAVING and not WHERE?
Because you're verifying the result of group_concat() which is an aggregate function. And the result of aggregate functions can be filtered only through a HAVING clause.

MySQL - Group BY GROUP_CONCAT

I have the following query:
SELECT
issue.`sequence` AS issue_sequence,
issue.`description` AS issue_description,
GROUP_CONCAT(DISTINCT(issue_category.`name`) SEPARATOR ', ') AS issue_category_name,
GROUP_CONCAT(DISTINCT(approach.`name`) SEPARATOR ', ') AS approach_name,
issue_approach.`issue_id` AS issue_approach_issue_id,
issue_approach.`approach_id` AS issue_approach_approach_id
FROM
`approach` approach
INNER JOIN `issue_approach` issue_approach ON approach.`id` = issue_approach.`approach_id`
INNER JOIN `issue` issue ON issue_approach.`issue_id` = issue.`id`
INNER JOIN `project` project ON issue.`project` = project.`id`
INNER JOIN `tenant` tenant ON project.`tenant_id` = tenant.`id`
INNER JOIN `issue_category` issue_category ON project.`id` = issue_category.`project`
INNER JOIN `user` user ON tenant.`id` = user.`tenant_id`
WHERE user.id = 1 AND project.id = 1
GROUP BY issue_category_name
ORDER BY issue_category.`name`, issue.`sequence`
I am having a problem with this line:
GROUP BY issue_category_name
Apparently, MySQL can't seem to Group by the alias for by GROUP_CONCAT result.
I am not really an expert with SQL, but is there a way I can group using the result of GROUP_CONCAT?
Sample data;
Categories: Network, Servers
Issue Id: 1
Description: Some description
Approaches: Some approaches to resolve the issue.
Basically, an issue can belong to one or many categories. I am concatenating categories for each issue. What i want to do is group the results by the result of concatenation of categories. So for example group issues whose categories are Network,Servers.
Thanks.
Im not a MySQL user, but change your group by to
Group By GROUP_CONCAT(DISTINCT(issue_category.`name`) SEPARATOR ', ')
With reference to SQL EXECUTION ORDER, the reason why this will not work is because the select statement is the last statement to be executed so that sql engine will not be knowing your column alias while grouping the records as GROUP BY occurs before SELECT. So that as Charles Bretana's answer suggests, put
Group By GROUP_CONCAT(DISTINCT(issue_category.`name`) SEPARATOR ', ')
in your group by clause. It will work fine then.
Hope this helps you.