Related
I have the following query. I need it to include all records from the 'yahrzeit' and tbldecedent table and the ones that are a match (dupes) should only be listed once.
SELECT *, Count(*)
FROM yahrzeit
WHERE confirmed = 1
INNER JOIN tbldecedent ON CONCAT( yahrzeit.firstName, ' ', yahrzeit.middleName, ' ', yahrzeit.lastName ) = tbldecedent.Name
AND DATE_FORMAT( CONCAT( yahrzeit.gregorianYear, '-', yahrzeit.gregorianMonthNum, '-', yahrzeit.gregorianDay ) , '%Y-%m-%d' ) = tbldecedent.EngDate
GROUP BY tbldecedent.Name, tbldecedent.EngDate
Where clauses belong after the joins.
Since we're using outer joins we have to make sure the where clause criteria for the matching records is moved to the join or the outer join is negated.
in mySQL to simulate a full outer join we simple do a left and right join with a union ALL (union does a distinct which removes the duplicates)
SELECT tbldecedent.Name, tbldecedent.EngDate, Count(*) as Cnt
FROM yahrzeit
LEFT JOIN tbldecedent
ON CONCAT( yahrzeit.firstName, ' ', yahrzeit.middleName, ' ', yahrzeit.lastName ) = tbldecedent.Name
AND DATE_FORMAT( CONCAT( yahrzeit.gregorianYear, '-', yahrzeit.gregorianMonthNum, '-', yahrzeit.gregorianDay ) , '%Y-%m-%d' ) = tbldecedent.EngDate
WHERE yahrzeit.confirmed = 1
AND tblDecedent.name is null -- add this to only show no matches.
GROUP BY tbldecedent.Name, tbldecedent.EngDate
UNION ALL
SELECT tbldecedent.Name, tbldecedent.EngDate, Count(*) as cnt
FROM yahrzeit
RIGHT JOIN tbldecedent
ON CONCAT( yahrzeit.firstName, ' ', yahrzeit.middleName, ' ', yahrzeit.lastName ) = tbldecedent.Name
AND DATE_FORMAT( CONCAT( yahrzeit.gregorianYear, '-', yahrzeit.gregorianMonthNum, '-', yahrzeit.gregorianDay ) , '%Y-%m-%d' ) = tbldecedent.EngDate
AND yhrzeit.confirmed = 1
WHERE CONCAT( yahrzeit.firstName, ' ', yahrzeit.middleName, ' ', yahrzeit.lastName ) is null -- add this to only show no matches.
GROUP BY tbldecedent.Name, tbldecedent.EngDate
To better understand joins I recommend: https://blog.codinghorror.com/a-visual-explanation-of-sql-joins/ the venn diagram approach is a good one.
Lastly, I don't recommend a select *, count() with a group by containing only two fields. Select should only include the values in the group by plus
aggregates or constants. Current version of MySQL wouldn't let you get away with this without changing a global setting, and other engines simply don't support this method. MySQL extends the group by to allow it; but the results can be unexpected for the columns not listed in the group by . More on that in the docs: https://dev.mysql.com/doc/refman/5.7/en/group-by-handling.html
I have been trying to group by the mysql group by in normal column and mysql function aliases column where the mysql function aliases wont work group by
The query i tried and which should work according to the mysql documentation.
SELECT `project`.`project_name`, SUM(cost_allocated) AS cost_allocated,
count(task_id) as countTask,
GROUP_CONCAT(task_registration.members SEPARATOR ';') AS members,
GROUP_CONCAT(task_registration.indicators SEPARATOR ';') AS indicators,
GROUP_CONCAT(task_registration.percent_complete SEPARATOR ';') AS percent_complete,
GROUP_CONCAT(task_registration.status SEPARATOR ';') AS status,
DATE_FORMAT(`task_registration`.`created_at`, '%M') AS month
FROM (`project`)
INNER JOIN `task_registration` ON `task_registration`.`project` = `project`.`project_id`
GROUP BY `task_registration`.`project`,`month`
ORDER BY `project`.`project_id` desc
The result i get
Now after removing the normal column group by and just adding mysql function aliases column it works fine but i need both
SELECT `project`.`project_name`, SUM(cost_allocated) AS cost_allocated,
count(task_id) as countTask,
GROUP_CONCAT(task_registration.members SEPARATOR ';') AS members,
GROUP_CONCAT(task_registration.indicators SEPARATOR ';') AS indicators,
GROUP_CONCAT(task_registration.percent_complete SEPARATOR ';') AS percent_complete,
GROUP_CONCAT(task_registration.status SEPARATOR ';') AS status,
DATE_FORMAT(`task_registration`.`created_at`, '%M') AS month
FROM (`project`)
INNER JOIN `task_registration` ON `task_registration`.`project` = `project`.`project_id`
GROUP BY `month`
ORDER BY `project`.`project_id` desc
after removing task_registration`.`project from group by
GROUP BY `task_registration`.`project`,`month` Doesnt work in this case
http://sqlfiddle.com/#!2/18389/2 i have tried this so far
Very difficult to work out what you want without table declarations, sample input data and sample output data, but from your comment all I can suggest is using the 2nd query as a sub query and joining it to the results of the first query:-
SELECT project.project_name,
SUM(cost_allocated) AS cost_allocated,
count(task_id) as countTask,
GROUP_CONCAT(task_registration.members SEPARATOR ';') AS members,
GROUP_CONCAT(task_registration.indicators SEPARATOR ';') AS indicators,
GROUP_CONCAT(task_registration.percent_complete SEPARATOR ';') AS percent_complete,
GROUP_CONCAT(task_registration.status SEPARATOR ';') AS status,
DATE_FORMAT(task_registration.created_at, '%M') AS month,
sub0.cost_allocated AS months_cost_allocated,
sub0.countTask AS months_countTask,
sub0.members AS months_members,
sub0.indicators AS months_indicators,
sub0.percent_complete AS months_percent_complete,
sub0.status AS months_status
FROM (`project`)
INNER JOIN `task_registration` ON `task_registration`.`project` = `project`.`project_id`
INNER JOIN
(
SELECT DATE_FORMAT(task_registration.created_at, '%M') AS month,
SUM(cost_allocated) AS cost_allocated,
COUNT(task_id) as countTask,
GROUP_CONCAT(task_registration.members SEPARATOR ';') AS members,
GROUP_CONCAT(task_registration.indicators SEPARATOR ';') AS indicators,
GROUP_CONCAT(task_registration.percent_complete SEPARATOR ';') AS percent_complete,
GROUP_CONCAT(task_registration.status SEPARATOR ';') AS status
FROM (project)
INNER JOIN task_registration ON task_registration.project = project.project_id
GROUP BY `month`
) sub0
ON DATE_FORMAT(task_registration.created_at, '%M') = sub0.`month`
GROUP BY task_registration.project,
`month`,
months_cost_allocated,
sub0.countTask AS months_countTask,
sub0.members AS months_members,
sub0.indicators AS months_indicators,
sub0.percent_complete AS months_percent_complete,
sub0.status AS months_status
ORDER BY project.project_id desc
Thank you for the answer i did it on my own
SELECT COUNT( i.project_name ) AS completed_projects,i.month
FROM (
SELECT `project`.`project_name` , SUM( cost_allocated ) AS cost_allocated, COUNT( task_id ) AS countTask, GROUP_CONCAT( task_registration.members
SEPARATOR ';' ) AS members, GROUP_CONCAT( task_registration.indicators
SEPARATOR ';' ) AS indicators, GROUP_CONCAT( task_registration.percent_complete
SEPARATOR ';' ) AS percent_complete, GROUP_CONCAT( task_registration.status
SEPARATOR ';' ) AS
STATUS , DATE_FORMAT( task_registration.created_at, '%M' ) AS
MONTH FROM (
`project`
)
INNER JOIN `task_registration` ON `task_registration`.`project` = `project`.`project_id`
GROUP BY `task_registration`.`project`
ORDER BY `project`.`project_id` DESC
) AS i
GROUP BY i.month
http://sqlfiddle.com/#!2/18389/11
Problem:
I know it's documented in MYSQL DOCUMENTS That in where clause "alias" cannot be used since where clause is not populated yet. But i have to get the data matched from other table values in where clause (IN condition).
I have also read some similar kind of post but this one is quite different & big complex one. I couldn't make it working with my efforts of last 3 days.
It's showing error (excepted as per documentation)
UNKNOWN COLUMN "Ind_ID" in WHERE CLAUSE
I have to match similarly for FA_ID & PREFERRED_LOCATION_ID field
Select a.job_id, a.Employer_ID, a.Sub_user_id,
Date_format(a.creation_on,'%d-%m-%Y') as Created_date, a.Job_type,
a.Designation, a.Open_Positions, a.Job_Description, a.Min_age,
a.Max_age, a.min_exp, a.max_exp, a.Hide_Salary, a.company_name,
a.About_Company, a.Contact_person_name, a.Contact_No, a.Refresh_type,
a.Response_type,
(Select GROUP_CONCAT(DISTINCT g.Education ORDER BY pjedu.Education_ID
SEPARATOR ', ') user_education
from e_pj_edu pjedu
INNER JOIN education g ON FIND_IN_SET(g.Edu_ID, pjedu.Education_ID)
where a.job_id = pjedu.Job_ID
) as Education_ID,
(Select GROUP_CONCAT(DISTINCT h.FA_description ORDER BY uf.FA_ID
SEPARATOR ', ') FA
from e_pj_fa uf
INNER JOIN functional_area h ON FIND_IN_SET(h.FA_ID, uf.FA_ID)
where a.Job_ID = uf.Job_ID
) as FA_ID,
(Select GROUP_CONCAT(DISTINCT i.Industry_description ORDER BY
ui.Industry_ID SEPARATOR ', ') Industry_ID
from e_pj_industry ui
INNER JOIN industry i ON FIND_IN_SET(i.Industry_ID, ui.Industry_ID)
where a.Job_ID = ui.Job_ID
) as Ind_ID,
(Select GROUP_CONCAT(DISTINCT j.location_name ORDER BY
upl.Location_ID SEPARATOR ', ') Location_ID
from e_pj_locations upl
INNER JOIN locations j ON FIND_IN_SET(j.location_id, upl.Location_ID)
where a.Job_ID = upl.Job_ID
) as Preferred_Location_ID,
(Select GROUP_CONCAT(DISTINCT uk.Keyword_Name ORDER BY uk.Keyword_ID
SEPARATOR ', ') keyskills
from e_pj_keywords uk
where a.Job_ID = uk.Job_ID
) as Keyword_Name,
GROUP_CONCAT(DISTINCT cc.salary_description ORDER BY cc.salary_ID
SEPARATOR ', ') Min_salary,
GROUP_CONCAT(DISTINCT dd.salary_description ORDER BY dd.salary_ID
SEPARATOR ', ') Max_salary
from post_jobs a
INNER JOIN user_salary cc ON FIND_IN_SET(cc.salary_ID, a.Min_salary)
INNER JOIN user_salary dd ON FIND_IN_SET(dd.salary_ID, a.Max_salary)
WHERE a.Designation LIKE '%MIS%' or a.company_name LIKE '%MIS%'
And a.max_exp <= 9
And a.Max_salary<=110
And Ind_ID IN (10001,10002,10004)
And FA_ID IN(1001)
group by a.job_id
First thing that comes in my mind is just move your aliased where conditions to outer query, i.e.:
select * from (
Select a.job_id ....
WHERE a.Designation LIKE '%MIS%' or a.company_name LIKE '%MIS%'
And a.max_exp <= 9
And a.Max_salary<=110
group by a.job_id
) inner
where
Ind_ID IN (10001,10002,10004)
And FA_ID IN(1001)
GL!
Posting revised query which worked. May be this helps someone from wasting 3 days like me :-)
select * from (
Select a.job_id, a.Employer_ID, a.Sub_user_id, Date_format(a.creation_on,'%d-%m-%Y') as Created_date, a.Job_type, a.Designation, a.
Open_Positions, a.Job_Description, a.Min_age, a.Max_age, a.min_exp, a.max_exp, a.Hide_Salary, a.company_name, a.About_Company, a.Contact_person_name,
a.Contact_No, a.Refresh_type, a.Response_type,
(Select GROUP_CONCAT(DISTINCT g.Education ORDER BY pjedu.Education_ID SEPARATOR ', ') user_education
from e_pj_edu pjedu
INNER JOIN education g ON FIND_IN_SET(g.Edu_ID, pjedu.Education_ID)
where a.job_id = pjedu.Job_ID
) as Education_ID,
(Select GROUP_CONCAT(DISTINCT h.FA_description ORDER BY uf.FA_ID SEPARATOR ', ') FA
from e_pj_fa uf
INNER JOIN functional_area h ON FIND_IN_SET(h.FA_ID, uf.FA_ID)
where a.Job_ID = uf.Job_ID
) as FA_ID,
(Select GROUP_CONCAT(DISTINCT i.Industry_description ORDER BY ui.Industry_ID SEPARATOR ', ') Industry_ID
from e_pj_industry ui
INNER JOIN industry i ON FIND_IN_SET(i.Industry_ID, ui.Industry_ID)
where a.Job_ID = ui.Job_ID
) as Ind_ID,
(Select GROUP_CONCAT(DISTINCT j.location_name ORDER BY upl.Location_ID SEPARATOR ', ') Location_ID
from e_pj_locations upl
INNER JOIN locations j ON FIND_IN_SET(j.location_id, upl.Location_ID)
where a.Job_ID = upl.Job_ID
) as Preferred_Location_ID,
(Select GROUP_CONCAT(DISTINCT uk.Keyword_Name ORDER BY uk.Keyword_ID SEPARATOR ', ') keyskills
from e_pj_keywords uk
where a.Job_ID = uk.Job_ID
) as Keyword_Name,
GROUP_CONCAT(DISTINCT cc.salary_description ORDER BY cc.salary_ID SEPARATOR ', ') Min_salary,
GROUP_CONCAT(DISTINCT dd.salary_description ORDER BY dd.salary_ID SEPARATOR ', ') Max_salary
from post_jobs a
INNER JOIN user_salary cc ON FIND_IN_SET(cc.salary_ID, a.Min_salary)
INNER JOIN user_salary dd ON FIND_IN_SET(dd.salary_ID, a.Max_salary)
group by a.Job_id
) aa
WHERE Designation LIKE '%op%' or company_name LIKE '%op%'
And max_exp <= 15
And Max_salary<=120
and Ind_ID IN (10001,10002,10004,10003)
And FA_ID IN(1001,1002,1003)
group by Job_id
I'm troubleshooting the performance of a rather complex SQL query I have here. When I run the following command, the results are outlined below...
EXPLAIN SELECT
Import_Values.id,
Import_Values.part_id,
Import_Values.qty,
Import_Values.note,
Parts.partterminologyname,
GROUP_CONCAT(BaseVehicle.YearID, ' ', Make.MakeName, ' ', Model.modelname, ' ', SubModel.SubModelName SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(EngineDesignation.EngineDesignationName) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(EngineVIN.EngineVINName) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(EngineBase.Liter) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(EngineBase.CC) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(EngineBase.CID) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(EngineBase.Cylinders) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(EngineBase.BlockType) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(EngineBase.EngBoreIn) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(EngineBase.EngBoreMetric) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(EngineBase.EngStrokeIn) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(EngineBase.EngStrokeMetric) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(FuelDeliveryType.FuelDeliveryTypeName) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(FuelDeliverySubType.FuelDeliverySubTypeName) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(FuelSystemControlType.FuelSystemControlTypeName) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(FuelSystemDesign.FuelSystemDesignName) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(Aspiration.AspirationName) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(CylinderHeadType.CylinderHeadTypeName) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(FuelType.FuelTypeName) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(IgnitionSystemType.IgnitionSystemTypeName) SEPARATOR ', '),
GROUP_CONCAT(DISTINCT(Mfr.MfrName) SEPARATOR ', ')
FROM
Import_Values
INNER JOIN BaseVehicle
ON Import_Values.base_vehicle_id=BaseVehicle.BaseVehicleID
INNER JOIN Parts
ON Import_Values.part_type_id=Parts.PartTerminologyID
INNER JOIN Make
ON BaseVehicle.MakeID=Make.MakeID
INNER JOIN Model
ON BaseVehicle.ModelID=Model.ModelID
INNER JOIN Vehicle
ON Import_Values.base_vehicle_id=Vehicle.BaseVehicleID
INNER JOIN SubModel
ON Vehicle.SubModelID=SubModel.SubModelID
INNER JOIN VehicleToEngineConfig
ON Vehicle.VehicleID=VehicleToEngineConfig.VehicleID
INNER JOIN EngineConfig
ON VehicleToEngineConfig.EngineConfigID=EngineConfig.EngineConfigID
INNER JOIN EngineDesignation
ON EngineConfig.EngineDesignationID=EngineDesignation.EngineDesignationID
INNER JOIN EngineVIN
ON EngineConfig.EngineVINID=EngineVIN.EngineVINID
INNER JOIN EngineBase
ON EngineConfig.EngineBaseID=EngineBase.EngineBaseID
INNER JOIN FuelDeliveryConfig
ON EngineConfig.FuelDeliveryConfigID=FuelDeliveryConfig.FuelDeliveryConfigID
INNER JOIN FuelDeliveryType
ON FuelDeliveryConfig.FuelDeliveryTypeID=FuelDeliveryType.FuelDeliveryTypeID
INNER JOIN FuelDeliverySubType
ON FuelDeliveryConfig.FuelDeliverySubTypeID=FuelDeliverySubType.FuelDeliverySubTypeID
INNER JOIN FuelSystemControlType
ON FuelDeliveryConfig.FuelSystemControlTypeID=FuelSystemControlType.FuelSystemControlTypeID
INNER JOIN FuelSystemDesign
ON FuelDeliveryConfig.FuelSystemDesignID=FuelSystemDesign.FuelSystemDesignID
INNER JOIN Aspiration
ON EngineConfig.AspirationID=Aspiration.AspirationID
INNER JOIN CylinderHeadType
ON EngineConfig.CylinderHeadTypeID=CylinderHeadType.CylinderHeadTypeID
INNER JOIN FuelType
ON EngineConfig.FuelTypeID=FuelType.FuelTypeID
INNER JOIN IgnitionSystemType
ON EngineConfig.IgnitionSystemTypeID=IgnitionSystemType.IgnitionSystemTypeID
INNER JOIN Mfr
ON EngineConfig.EngineMfrID=Mfr.MfrID
GROUP BY part_id
The results indicate that the table Import_Values has 147377 rows and the type is 'ALL' and FuelDeliveryType has 3 rows and the type is 'ALL'. From what I've read, the 'ALL' type leads to poor performance.
I've added an index but it still says 'ALL' -- is there something else I must do?
Thanks!
Well you definitely figured out the necessity for indexing, but the key is knowing whether or not the indexes are actually fully optimizing your query.
MySQL uses indexes on columns at its maximum efficiency if the columns are the same type and size (i.e. CHAR(10), VARCHAR, etc.). You can alter your tables, ALTER TABLE, to ensure these types are consistent across the columns are you joining on.
Another thing that MySQL does is assume that values in the columns are evenly distributed, which might not be the case. If you run an ANALYZE TABLE on the table in question, you might find that the number of rows it returns are not even close to the actual number in the real table. To fix this you could use a different type of JOIN or even go so far as to reorder your tables in your FROM clause.
If the ALL type still hasn't gone away after these tests, it might be the case that the query is as optimized as it is going to get.
I'm trying to show the boroughs and postcodes a particular town in is.
My database is fairly well structured, with a table such as town, postcode and borough. There are also tables for each of the relationships town_postcode & town_borough.
Ideally I want the data returned as:
"Abbey Wood", "SE2", "Bexley, Greenwich"
"Barbican", "EC1, EC2", "City of London"
I've tried a few different approaches and I'm close but not there yet.
Any help would be appreciated... :)
So far I've tried
SELECT DISTINCT t.town,
GROUP_CONCAT( DISTINCT p.postcode SEPARATOR ', ' ) AS 'postcode',
GROUP_CONCAT( DISTINCT b.borough SEPARATOR ', ' ) AS 'borough'
FROM coverage_towns AS t,
coverage_boroughs AS b,
coverage_postcodes AS p,
coverage_towns_boroughs AS tb,
coverage_towns_postcodes AS tp
WHERE t.id = tp.town_id
AND p.id = tp.postcode_id
AND b.id = tb.borough_id
GROUP BY t.town
ORDER BY t.town ASC
Which returns
"Abbey Wood", "SE2", "Southwark, Hammersmith and Fulham, Tower Hamlets, Wandsworth, Enfield, Newham, LOTS MORE HERE"
"Barbican", "EC1, EC2", "Brent, Greenwich, Kensington and Chelsea, Westminster, Camden, LOTS MORE HERE"
I've also tried
SELECT DISTINCT t.town, (
SELECT SQL_CACHE DISTINCT GROUP_CONCAT( p1.postcode
SEPARATOR ', ' )
FROM coverage_postcodes AS p1
WHERE p1.id = tp.postcode_id
) AS 'postcode', (
SELECT SQL_CACHE DISTINCT GROUP_CONCAT( b1.borough
SEPARATOR ', ' )
FROM coverage_boroughs AS b1
WHERE b1.id = tb.borough_id
) AS 'borough'
FROM coverage_towns AS t, coverage_boroughs AS b, coverage_postcodes AS p, coverage_towns_boroughs AS tb, coverage_towns_postcodes AS tp
WHERE t.id = tp.town_id
AND p.id = tp.postcode_id
AND b.id = tb.borough_id
GROUP BY t.town
ORDER BY t.town ASC
Which returns
"Abbey Wood", "SE2", "Greenwich"
"Acton", "W3", "Greenwich"
"Aldersbrook", "E12", "Greenwich"
First query looks good, just add distinct inside the group_concat, like:
SELECT t.town
, GROUP_CONCAT(DISTINCT p.postcode SEPARATOR ', ' ) AS 'postcode'
, GROUP_CONCAT(DISTINCT b.borough SEPARATOR ', ' ) AS 'borough'
<more code here>
GROUP BY
t.town
SOLUTION
I came back to the question after a good coffee and the answer presented itself.
SELECT DISTINCT t.town,
GROUP_CONCAT( DISTINCT p.postcode SEPARATOR ', ' ) AS 'postcode',
GROUP_CONCAT( DISTINCT b.borough SEPARATOR ', ' ) AS 'borough'
FROM towns AS t, boroughs AS b, postcodes AS p, towns_boroughs AS tb, towns_postcodes AS tp
WHERE (t.id = tp.town_id AND t.id = tb.town_id)
AND (p.id = tp.postcode_id AND b.id = tb.borough_id)
GROUP BY t.town
ORDER BY t.town ASC