Optimize GROUP_CONCAT in SQL on MySQL - mysql

Is there a way to get the following SQL optimized so it would run faster?
SELECT bitValue as bitV, endName as endN, filDuration as filD, filSize as filS, genName as genN, sonAdditionalPrefix AS sonAP, sonDate as sonD, sonID, sonIsRip, sonName, typName, sonPlays as plays,
(SELECT GROUP_CONCAT(a.artName SEPARATOR ';') FROM tblArtists a, tblSoAr sa WHERE a.artID = sa.soarartIDRef AND sa.soarsonIDRef = s.sonID) as artists,
(SELECT GROUP_CONCAT(a.artName SEPARATOR ';') FROM tblArtists a, tblRemixer r WHERE a.artID = r.remartIDRef AND r.remsonIDRef = s.sonID) as remixer,
(SELECT GROUP_CONCAT(a.artName SEPARATOR ';') FROM tblArtists a, tblFeaturing f WHERE a.artID = f.feaartIDRef AND f.feasonIDRef = s.sonID) as featuring,
(SELECT GROUP_CONCAT(ta.tagName SEPARATOR ';') FROM tblTags ta, tblSoTa st WHERE ta.tagID = st.sotatagIDRef AND st.sotasonIDRef = s.sonID) as tags
FROM tblSongs s, tblFiles f, tblGenres g, tblEndings e, tblBitrates b, tblTypes t
WHERE s.sonfilIDRef = f.filID AND s.songenIDRef = g.genID AND f.filendIDRef = e.endID AND f.filbitIDRef = b.bitID AND s.sontypIDRef = t.typID AND sonshaIDRef = 1 AND sonWasEdited = 1 AND sonDeleted = 0 AND sonOnWishlist = 0 ;
I encountered the bottleneck are the GROUP_CONCAT's but I couldn't find a way by using an alternative or optimizing the query so it would run faster.
At the moment the query needs about 3.2s on 2700 entries (mysql database 5.1.73, mysqli). All tables have indices set on the important columns.
So after implementing some LEFT JOINS my query looks the following:
SELECT bitValue as bitV, endName as endN, filDuration as filD, filSize as filS, genName as genN, sonAdditionalPrefix AS sonAP, sonDate as sonD, sonID, sonIsRip, sonName, typName, sonPlays as plays,
conArtists.artists, conRemixer.remixer, conFeaturing.featuring, conTags.tags
FROM (tblSongs s, tblFiles f, tblGenres g, tblEndings e, tblBitrates b, tblTypes t)
LEFT JOIN (SELECT tblSoAr.soarsonIDRef as soarsonIDRef, GROUP_CONCAT(tblArtists.artName SEPARATOR ';') as artists FROM tblSoAr, tblArtists WHERE tblArtists.artID = tblSoAr.soarartIDRef GROUP BY soarsonIDRef) AS conArtists ON conArtists.soarsonIDRef = s.sonID
LEFT JOIN (SELECT tblRemixer.remsonIDRef as remsonIDRef, GROUP_CONCAT(tblArtists.artName SEPARATOR ';') as remixer FROM tblRemixer, tblArtists WHERE tblArtists.artID = tblRemixer.remsonIDRef GROUP BY remsonIDRef) AS conRemixer ON conRemixer.remsonIDRef = s.sonID
LEFT JOIN (SELECT tblFeaturing.feasonIDRef as feasonIDRef, GROUP_CONCAT(tblArtists.artName SEPARATOR ';') as featuring FROM tblFeaturing, tblArtists WHERE tblArtists.artID = tblFeaturing.feasonIDRef GROUP BY feasonIDRef) AS conFeaturing ON conFeaturing.feasonIDRef = s.sonID
LEFT JOIN (SELECT tblSoTa.sotasonIDRef as sotasonIDRef, GROUP_CONCAT(tblTags.tagName SEPARATOR ';') as tags FROM tblSoTa, tblTags WHERE tblTags.tagID = tblSoTa.sotasonIDRef GROUP BY sotasonIDRef) AS conTags ON conTags.sotasonIDRef = s.sonID
WHERE s.sonfilIDRef = f.filID AND s.songenIDRef = g.genID AND f.filendIDRef = e.endID AND f.filbitIDRef = b.bitID AND s.sontypIDRef = t.typID AND sonshaIDRef = 1 AND sonWasEdited = 1 AND sonDeleted = 0 AND sonOnWishlist = 0;

I think you can avoid the subqueries (that are done for every row) by grouping the artName values first and then joining them:
old query:
select
a.id,
(select group_concat(x) as x_concat from table_2 as _b where a.id = _b.id and ...) as b,
(select group_concat(x) as x_concat from table_3 as _c where a.id = _b.id and ...) as c
from table_1 as a
where ...
Should be something like this:
select
a.id,
b.x_concat,
c.x_concat
from
table_1 as a,
left join (select id, group_concat(x) as x_concat from table_2 where ... group by id) as b on a.id = b.id,
left join (select id, group_concat(x) as x_concat from table_3 where ... group by id) as c on a.id = c.id
where ...

Related

Slow MySQL Query When Run As A Procedure

If I run this query directly in PHPMyAdmin, it returns 13420 rows in 0.2091 second, but if I run the exact same query as a stored procedure, it returns the same amount of row but it takes forever and sometimes the SQL server returns an out of memory exception.
I'm at a total loss - any advice would be welcome, because I can't work out why this slows everything down?!
SELECT
el.UID as LUID,
se.UID as DUID,
el.event_title,
el.event_synopsis,
se.behind_the_scenes,
se.sub_event_title,
se.event_eventDateAndTime,
se.event_eventDateAndTimeEnd,
el.event_confirmed,
el.event_active,
(
SELECT GROUP_CONCAT(sp2.color SEPARATOR ',')
FROM setup__spaces sp2
LEFT JOIN events__assigned_spaces eas2 ON ( eas2.space_id = sp2.UID )
LEFT JOIN events__events_list el2 ON (el2.UID = eas2.event_id)
WHERE el2.UID = el.UID
) as spaceColors,
(
SELECT GROUP_CONCAT(sp2.name SEPARATOR ', ')
FROM setup__spaces sp2
LEFT JOIN events__assigned_spaces eas2 ON ( eas2.space_id = sp2.UID )
LEFT JOIN events__events_list el2 ON (el2.UID = eas2.event_id)
WHERE el2.UID = el.UID
) as spaceNames,
(
SELECT GROUP_CONCAT(sp2.UID SEPARATOR ',')
FROM setup__spaces sp2
LEFT JOIN events__assigned_spaces eas2 ON ( eas2.space_id = sp2.UID )
LEFT JOIN events__events_list el2 ON (el2.UID = eas2.event_id)
WHERE el2.UID = el.UID
) as spaceIds,
(
SELECT GROUP_CONCAT(t.UID SEPARATOR ',')
FROM setup__tags t
LEFT JOIN events__assigned_tags eat ON ( eat.tag_id = t.UID )
LEFT JOIN events__events_list el2 ON (el2.UID = eat.event_id)
WHERE el2.UID = el.UID
) as tagIds,
(
SELECT GROUP_CONCAT(t.tag_name SEPARATOR ', ')
FROM setup__tags t
LEFT JOIN events__assigned_tags eat ON ( eat.tag_id = t.UID )
LEFT JOIN events__events_list el2 ON (el2.UID = eat.event_id)
WHERE el2.UID = el.UID
) as tagNames
FROM events__events_list el
INNER JOIN events__sub_events se ON (el.UID = se.event_masterEvent)
WHERE ((el.event_active='1') OR (el.event_active='0' AND el.event_confirmed = '1'))
AND el.company_uid = sp_company_uid
With thanks to #Akina for pointing me in the right direction, I found help from the following places:
stackoverflow.com - Selecting multiple columns/fields in MySQL subquery
geeksengine.com - How to use subquery in JOIN operation in MySQL
Here's the revised code that's now lightning fast!
SELECT
el.UID as LUID,
se.UID as DUID,
el.event_title,
el.event_synopsis,
se.behind_the_scenes,
se.sub_event_title,
se.event_eventDateAndTime,
se.event_eventDateAndTimeEnd,
el.event_confirmed,
el.event_active,
tags.names as tagNames,
tags.ids as tagIds,
spaces.names as spaceNames,
spaces.colors as spaceColors,
spaces.ids as spaceIds
FROM events__events_list el
INNER JOIN events__sub_events se ON (el.UID = se.event_masterEvent)
LEFT JOIN (
SELECT
el3.UID as el2uid,
GROUP_CONCAT(s.name SEPARATOR ', ') as names,
GROUP_CONCAT(s.color SEPARATOR ',') as colors,
GROUP_CONCAT(s.UID SEPARATOR ',') as ids
FROM setup__spaces as s
LEFT JOIN events__assigned_spaces eas ON ( eas.space_id = s.UID )
LEFT JOIN events__events_list el3 ON ( el3.UID = eas.event_id )
GROUP BY el3.UID
) as spaces on spaces.el2uid = el.UID
LEFT JOIN (
SELECT
el2.UID as el2uid,
GROUP_CONCAT(t.tag_name SEPARATOR ', ') as names,
GROUP_CONCAT(t.UID SEPARATOR ',') as ids
FROM setup__tags as t
LEFT JOIN events__assigned_tags eat ON ( eat.tag_id = t.UID )
LEFT JOIN events__events_list el2 ON ( el2.UID = eat.event_id )
GROUP BY el2.UID
) as tags on tags.el2uid = el.UID
WHERE ((el.event_active='1') OR (el.event_active='0' AND el.event_confirmed = '1'))
AND el.company_uid = sp_company_uid

how to use the column in a temporary table for the following update in MSQL

I have a MySQL database.
I want to update a column(in my case title column in bms_title table) in a table using the values from concat columns in other tables.
SELECT * FROM(SELECT distinct t.id, t.title as Textbook,
GROUP_CONCAT(concat(ci.discipline_code, ci.code, " (" , ci.type , ")") SEPARATOR ', ') as CourseCode FROM
tms_local.bms_material m,
tms_local.bms_title t,
tms_local.bms_course c,
tms_local.bms_courseinfo ci
where t.id > 1 AND t.id = m.book_id
and c.id = m.course_id
and ci.id = c.id
and isbn != 'NA'
GROUP BY t.id) AS temporary_table;
UPDATE tms_local.bms_title
SET tms_local.bms_title.thumbnail = temporary_table.CourseCode
WHERE tms_local.bms_title.title=temporary_table.Textbook;
But I got the error: Unknow temporary_table.Textbook in where clause.
How could I update the tms_local.bms_title.thumbnail column using CourseCode column from the selected table?
enter image description here
I have tried
CREATE TEMPORARY TABLE IF NOT EXISTS temporary_table AS (SELECT distinct t.id, t.title as Textbook,
GROUP_CONCAT(concat(ci.discipline_code, ci.code, " (" , ci.type , ")") SEPARATOR ', ') as CourseCode FROM
tms_local.bms_material m,
tms_local.bms_title t,
tms_local.bms_course c,
tms_local.bms_courseinfo ci
where t.id > 1 AND t.id = m.book_id
and c.id = m.course_id
and ci.id = c.id
and isbn != 'NA'
GROUP BY t.id);
UPDATE tms_local.bms_title
SET tms_local.bms_title.thumbnail = temporary_table.CourseCode
WHERE tms_local.bms_title.title=temporary_table.Textbook;
But got the same error.
you need to join the select statement.
As seen below:
UPDATE tms_local.bms_title t0
INNER JOIN
(SELECT
*
FROM
(SELECT DISTINCT
t.id,
t.title AS Textbook,
GROUP_CONCAT(CONCAT(ci.discipline_code, ci.code, ' (', ci.type, ')')
SEPARATOR ', ') AS CourseCode
FROM
tms_local.bms_material m, tms_local.bms_title t, tms_local.bms_course c, tms_local.bms_courseinfo ci
WHERE
t.id > 1 AND t.id = m.book_id
AND c.id = m.course_id
AND ci.id = c.id
AND isbn != 'NA'
GROUP BY t.id) AS temporary_table) t1 ON t0.title = t1.Textbook
SET
t0.thumbnail = t1.ourseCode;
Your temporary_table is being lost between the first statement and the second.
I find the WITH ... AS SQL structure to be helpful to get these together and is far more readable:
WITH temporary_table AS(
SELECT *
FROM(SELECT distinct t.id,
t.title as Textbook,
GROUP_CONCAT(concat(ci.discipline_code,
ci.code,
" (" ,
ci.type ,
")")
SEPARATOR ', '
) as CourseCode
FROM tms_local.bms_material m,
tms_local.bms_title t,
tms_local.bms_course c,
tms_local.bms_courseinfo ci
WHERE t.id > 1 AND t.id = m.book_id
and c.id = m.course_id
and ci.id = c.id
and isbn != 'NA'
GROUP BY t.id)
UPDATE tms_local.bms_title
SET tms_local.bms_title.thumbnail = temporary_table.CourseCode
WHERE tms_local.bms_title.title=temporary_table.Textbook;

GROUP_CONCAT multiple where condition particular data

Query
(SELECT
pid,
visitdate,
GROUP_CONCAT(tooth, ' - ', problem, ' - ', recomendation SEPARATOR ', <br>')
FROM tbl_finds_d) V
WHERE V.pid='1' AND V.visitdate = '16-03-2020'
TABLE
Want to get WHERE pid=1 AND visitdate=16-03-2020
pid visitdate tooth problem recomendation
1 16-03-2020 13 ASX DFFF
1 16-03-2020 12 JHJ HJLP
2 12-03-2020 14 JKB IJLHJ
UPDATE (copied from the comment)
SELECT *
FROM pendingtreatment A
INNER JOIN tbl_finds_m B ON B.pid = '$pid'
AND B.visitdate = '$visitdate'
INNER JOIN treatmentadviced C ON C.pid = '$pid'
AND C.visitdate = '$visitdate'
INNER JOIN treatmentlist D ON D.pid = '$pid'
AND D.visitdate = '$visitdate'
INNER JOIN tbl_appointments E ON E.pid = '$pid'
AND E.visitdate = '$visitdate'
INNER JOIN ( SELECT pid,
visitdate,
GROUP_CONCAT(tooth, ' - ', problem, ' - ', recomendation SEPARATOR ', <br>')
FROM tbl_finds_d V
WHERE V.pid='$pid'
AND V.visitdate = '$visitdate') F
WHERE A.pid = '$pid'
AND A.visitdate = '$visitdate'

how to GROUP_CONCAT with Concat

Product name supplier
A Su1
A Su1
A Su2
B Su1
C Su3
I want like this
A - su1, A-su2, B-su1, C-su3
Query:
SELECT
vtiger_salesorder.salesorder_no,
(Select
group_concat(DISTINCT concat(vtiger_products.productname, '-', vtiger_vendor.vendorname ) SEPARATOR ', ')
FROM
vtiger_salesorder
LEFT Join vtiger_inventoryproductrel ON vtiger_salesorder.salesorderid = vtiger_inventoryproductrel.id
inner Join vtiger_products ON vtiger_products.productid = vtiger_inventoryproductrel.productid
inner Join softMax_SalesOrderVendorInfo ON softMax_SalesOrderVendorInfo.salesorderid = vtiger_salesorder.salesorderid
LEFT JOIN vtiger_vendor ON softMax_SalesOrderVendorInfo.vendorid = vtiger_vendor.vendorid
where (vtiger_salesorder.salesorderid = vtiger_inventoryproductrel.id
AND vtiger_salesorder.salesorderid = vtiger_crmentity.crmid
and (softMax_SalesOrderVendorInfo.status = '0') )Group by vtiger_salesorder.salesorderid Limit 0,1) As SuName1
FROM
vtiger_salesorder
INNER JOIN vtiger_inventoryproductrel ON vtiger_salesorder.salesorderid = vtiger_inventoryproductrel.id
Inner Join vtiger_crmentity ON vtiger_salesorder.salesorderid = vtiger_crmentity.crmid
Order by vtiger_salesorder.salesorder_no
Given your original data, you can simply do this with something like:
select distinct group_concat(ProductName, '-', Supplier separator ', ')
from table t;
I have no idea what the query has to do with the question, because you already seem to have the data in the appropriate format.

Left Join to another select statement

I have an INNER JOIN query that runs perfectly fine by itself, which I want to LEFT JOIN to a another query/SELECT statement which contains WHERE clause.
I'm not able to join both the queries. It should link wt.tkinit = t.tkinit
Can you please suggest what I'm missing.
SELECT c.clnum, m.mmatter, ot.tkinit AS 'otkinit', wt.tkinit AS 'wtikint', t.tkrt01,
SUM(mt.mthrwkdb) AS 'whrs2010',
FROM client c, matter m, timekeep ot, timekeep wt, mattimhs mt, periodt p, timerate t
WHERE c.clnum = m.mclient
AND m.mmatter = mt.mtmatter
GROUP BY c.clnum, m.mmatter, ot.tkinit, wt.tkinit
SELECT t.tkinit, t.tkrt01
FROM timerate t
INNER JOIN (
SELECT tkinit, max(tkeffdate) as max_effdate
FROM timerate WHERE DATEPART(year, tkeffdate) = '2012'
GROUP BY tkinit) mt ON mt.tkinit = t.tkinit AND mt.max_effdate = t.tkeffdate
Did you try the following:
SELECT fields from (SELECT1) s1
LEFT JOIN
(SELECT2) s2
on s1.wt.tkinit = s2.t.tkinit.
Don't forget to get normal name for wt.tkinit and t.tkinit in their selects like wt_tkinit and t_tkinit because you can't do double aliases.
SELECT temp2.tkinit, temp2.tkrt01, temp.clnum, temp.mmatter, temp.otkinit, temp.tkrt01, temp.whrs2010
FROM
( SELECT c.clnum, m.mmatter, ot.tkinit AS 'otkinit', wt.tkinit AS 'wtikint', t.tkrt01,
SUM(mt.mthrwkdb) AS 'whrs2010',
FROM client c, matter m, timekeep ot, timekeep wt, mattimhs mt, periodt p, timerate t
WHERE c.clnum = m.mclient
AND m.mmatter = mt.mtmatter
GROUP BY c.clnum, m.mmatter, ot.tkinit, wt.tkinit ) temp
LEFT JOIN
SELECT t.tkinit, t.tkrt01
FROM timerate t
INNER JOIN (
SELECT tkinit, max(tkeffdate) as max_effdate
FROM timerate WHERE DATEPART(year, tkeffdate) = '2012'
GROUP BY tkinit) mt ON mt.tkinit = t.tkinit AND mt.max_effdate = t.tkeffdate ) temp2
ON temp.wtikint = temp2.tkinit
Be careful of your field names, but this may be a start