MySQL - Optimization select query - mysql

My query is taking 5 seconds to finish but it sounds to much for something so basic. I did an explain query:
https://www.dropbox.com/s/18pedm9n5fssz4e/localhost%20_%20localhost%20_%20logrede%20_%20phpMyAdmin%203.4.11.pdf
It is possible to you to help me optimize it?
Thanks :)

I've restructured the query using JOINs instead of a less readable WHERE clause. I see everything is based on the primary "material" table. So, my first suggestion is to use the "STRAIGHT_JOIN" clause to tell MySQL to do the query in the order you have listed.... otherwise, sometimes it tries to think for you and do some things in what could be an alternate "primary" table driving the query. All your other tables are basically "lookup values" tables.
Secondly... On each of the tables, I would specifically have the following indexes on the tables. I'm sure you already had on the primary keys of each table... but since your GROUP_CONCAT are getting a corresponding descriptive field from the table, I would include that column in the index. Here's the other reason... When a query is run and using the index, if the query does not have to go back to the raw table for the "other" columns, it doesn't have to. So, by having the description as part of the index, it can do the join AND get the description both from the index...
I would have an index on
material ( cdomaterial )
cliente ( idcliente, nome )
classeobra ( idclasse, sigla )
dma ( iddma, nome )
desenho( iddesenho, nome )
fornecedor ( idfornecedor, nome )
modelo ( idmodelo, nome )
fabricante ( idfabri, nome )
fornecido( codmaterial, preco )
unidade( idunidade, sigla )
tipofornecimento( idtipoforn, sigla )
So, the above said, I would run the following query...
SELECT STRAIGHT_JOIN
material.codmaterial,
cliente.nome as cliente,
tipofornecimento.sigla as fornecimento,
unidade.sigla as unidade,
GROUP_CONCAT(DISTINCT classeobra.sigla ORDER BY classeobra.idclasse SEPARATOR ', ') as classeobra,
GROUP_CONCAT(DISTINCT dma.nome ORDER BY dma.iddma SEPARATOR ', ') as dma,
GROUP_CONCAT(DISTINCT desenho.nome ORDER BY desenho.iddesenho SEPARATOR ', ') as desenho,
GROUP_CONCAT(DISTINCT fornecedor.nome ORDER BY fornecedor.idfornecedor SEPARATOR ', ') as fornecedor,
GROUP_CONCAT(DISTINCT modelo.nome ORDER BY modelo.idmodelo SEPARATOR ', ') as modelo,
GROUP_CONCAT(DISTINCT fabricante.nome ORDER BY fabricante.idfabri SEPARATOR ', ') as marca,
GROUP_CONCAT(DISTINCT fornecido.preco ORDER BY fornecido.preco SEPARATOR ', ') as preco
FROM
material
JOIN unidad
ON material.idunidade = unidade.idunidade
JOIN requisitado
ON material.codmaterial = requisitado.codmaterial
JOIN cliente
ON requisitado.idcliente = cliente.idcliente
JOIN tipofornecimento
ON requisitado.idtipoforn = tipofornecimento.idtipoforn
JOIN possuimodelo
ON material.codmaterial = possuimodelo.codmaterial
JOIN modelo
ON possuimodelo.idmodelo = modelo.idmodelo
JOIN pertence,
ON modelo.idmodelo = pertence.idmodelo
JOIN fabricante
ON pertence.idfabri = fabricante.idfabri
JOIN utilizadoclasseobra
ON material.codmaterial = utilizadoclasseobra.codmaterial
JOIN classeobra
ON utilizadoclasseobra.idclasse = classeobra.idclasse
JOIN possuidma
ON material.codmaterial = possuidma.codmaterial
JOIN dma
ON possuidma.iddma = dma.iddma
JOIN fornecido
ON material.codmaterial = fornecido.codmaterial
JOIN fornecedor
ON fornecido.idfornecedor = fornecedor.idfornecedor
JOIN possuidesenho
ON material.codmaterial = possuidesenho.codmaterial
JOIN desenho
ON possuidesenho.iddesenho = desenho.iddesenho
GROUP BY
material.codmaterial,
cliente.nome
ORDER BY
material.codmaterial ASC,
cliente.nome ASC ;

Related

Get 50k rows faster with subqueries - Laravel 5.6

The below query is to get the candidate's details from the table which has 50k rows. Including jobs, regions, and employment types. A candidate has basic details with employment type, jobs, regions are in another table with foreign key relation.
$candidates = DB::table('candidates')
->join('role_users', function($join){
$join->on('candidates.user_id', '=', 'role_users.user_id');
$join->where('role_users.role_id', 6);
})
->join('candidate_statuses', 'candidates.candidate_status_id', '=', 'candidate_statuses.id')
->join('employment_types', 'candidates.employment_types_id', '=', 'employment_types.id')
->select(
'candidates.id',
'candidates.user_id',
'candidates.candidate_code',
'candidates.full_name as name',
'employment_types.title AS employment_type',
DB::raw("(SELECT GROUP_CONCAT(candidate_jobs.job_id SEPARATOR ',') FROM candidate_jobs WHERE candidate_jobs.candidate_id = candidates.id) as job_ids"),
DB::raw("(SELECT GROUP_CONCAT(regions.name SEPARATOR ',') FROM candidate_regions INNER JOIN regions ON regions.id=candidate_regions.region_id WHERE candidate_regions.candidate_id = candidates.id) as regions"),
'role_users.email',
'role_users.login_at',
'candidates.is_deleted')
->where('candidates.candidate_status_id', '!=' , 6)
->where('candidates.is_deleted', $request->is_deleted)
->orderBy('candidates.first_name')
->groupBy('candidates.id')
->paginate(10);
}
select `candidates`.`id`, `candidates`.`user_id`, `candidates`.`candidate_code`,
`candidates`.`first_name`,
`candidates`.`last_name`, `candidates`.`full_name` as `name`,
`candidates`.`profile_img`, `candidates`.`candidate_status_id`,
`candidates`.`employment_types_id`,
`employment_types`.`title` as `employment_type`,
(
SELECT GROUP_CONCAT(candidate_jobs.job_id SEPARATOR ',')
FROM candidate_jobs
WHERE candidate_jobs.candidate_id = candidates.id
) as job_ids,
(
SELECT GROUP_CONCAT(regions.name SEPARATOR ',')
FROM candidate_regions
INNER JOIN regions ON regions.id=candidate_regions.region_id
WHERE candidate_regions.candidate_id = candidates.id
) as regions,
`candidates`.`formatted_mobile_number`, `candidates`.`place`,
`candidates`.`post_code`, `role_users`.`email`, `role_users`.`login_at`,
`role_users`.`email`, `candidates`.`has_access`, `candidates`.`is_deleted`
from `candidates`
inner join `role_users` ON `candidates`.`user_id` = `role_users`.`user_id`
and `role_users`.`role_id` = ?
inner join `candidate_statuses`
ON `candidates`.`candidate_status_id` = `candidate_statuses`.`id`
inner join `employment_types`
ON `candidates`.`employment_types_id` = `employment_types`.`id`
where (`candidates`.`candidate_status_id` in (?))
and `candidates`.`candidate_status_id` != ?
and `candidates`.`is_deleted` = ?
group by `candidates`.`id`
order by `candidates`.`first_name` asc
It takes 2/ 3 seconds to get the result in my local machine but in production takes too long time.
Can anyone please help?
It seems like the second part is unnecessary:
`candidates`.`candidate_status_id` in (?))
and `candidates`.`candidate_status_id` != ?
Making these match avoids an extra pass over the results
group by `first_name`, `id`
order by `first_name` asc, id
Possibly helpful indexes:
candidates: INDEX(candidate_status_id, is_deleted, first_name, id, user_id)
role_users: INDEX(user_id, email, login_at, role_id)
candidate_jobs: INDEX(candidate_id, job_id)
candidate_regions: INDEX(candidate_id, region_id)

MySQL Find Most Recent/Largest Record Per Group by order by

MySQL Find Most Recent/Largest Record Per Group by order by, and how can i minimize / shorten this query and every time it returns the first row value of the group, whereas i like to select the last row value of a group, and sort the values based on ja.id ? I know this is a bad query can anyone suggest or provide me solution to shorten this query . I have used all the necessary Column Indexes in all the tables. How to shorten the query without the use of union all .both the queries in union all are same expect in the where statement.
SELECT
a.previous_status,
a.rejected_status,
a.rejection_reason_text,
a.rejection_reason,
a.rjaId,
a.refer_applied_status,
a.title,
a.playerId,
a.gameId,
a.gamePostDate,
a.game_referal_amount,
a.country,
a.country_name,
a.state,
a.location,
a.state_abb,
a.game_type,
a.appliedId,
a.appliedStatus,
a.admin_review,
a.is_req_referal_check,
a.memberId,
a.appliedEmail,
a.game_id,
a.referred_id,
a.memStateAbb,
a.memState,
a.memZipcode,
a.memCity,
a.memCountryNme,
a.memCountry,
a.appliedMemberName,
a.first_name,
a.primary_contact,
a.last_name,
a.addressbookImage,
a.userImage,
a.last_login,
a.user_experience_year,
a.user_experience_month,
a.current_designation,
a.current_player,
a.appliedDate,
a.addressbook_id,
a.joiningdate,
a.gameStatus,
a.gameReferalAmountType,
a.gameFreezeStatus,
a.gameFreezeMsg,
a.app_assign_back_to_rp_count,
a.applied_source,
a.max_id,
a.gamesApplied,
a.gamesAppliedId,
SUM(a.totalgameApplied) AS totalgameApplied,
a.application_assign_to_rp_status,
a.rpAppliedSource,
a.applied_on
FROM
(
(
SELECT
ja.previous_status,
ja.rejected_status,
ja.rejection_reason_text,
ja.rejection_reason,
rja.id AS rjaId,
rja. STATUS AS refer_applied_status,
jp.title,
jp.user_user_id AS playerId,
jp.id AS gameId,
jp.posted_on AS gamePostDate,
jp.game_referal_amount,
jp.country,
jp.country_name,
jp.state,
jp.location,
jp.state_abb,
jp.game_type,
ja.id AS appliedId,
IFNULL(ja. STATUS, '') AS appliedStatus,
IFNULL(ja.admin_review, '') AS admin_review,
ja.is_req_referal_check,
usr.id AS memberId,
rja.email AS appliedEmail,
rja.game_id,
rja.referred_id,
mem.state_abb AS memStateAbb,
mem.state AS memState,
mem.zipcode AS memZipcode,
mem.city AS memCity,
mem.country_name AS memCountryNme,
mem.country_code AS memCountry,
usc. NAME AS appliedMemberName,
usc.first_name,
IFNULL(
mem.primary_contact,
usc.phone_number
) AS primary_contact,
usc.last_name,
usc.profileimage_path AS addressbookImage,
usr.profile_image AS userImage,
usr.last_login,
mem.user_experience_year,
mem.user_experience_month,
mem.current_designation,
mem.current_player,
rja.create_date AS appliedDate,
rja.addressbook_id,
IFNULL(ja.joining_date, '') AS joiningdate,
jp. STATUS AS gameStatus,
jp.games_referal_amount_type AS gameReferalAmountType,
jp.game_freeze_status AS gameFreezeStatus,
jp.game_freeze_message AS gameFreezeMsg,
ja.app_assign_back_to_rp_count,
ja.applied_source,
MAX(rja.id) AS max_id,
GROUP_CONCAT(
jp.title
ORDER BY
rja.create_date DESC
) AS gamesApplied,
GROUP_CONCAT(DISTINCT(jp.id)) AS gamesAppliedId,
COUNT(DISTINCT(jp.id)) totalgameApplied,
ja.application_assign_to_rp_status,
1 AS rpAppliedSource,
rja.create_date AS applied_on
FROM
(`refer_gameapplied` AS rja)
JOIN `games_post` AS jp ON `jp`.`id` = `rja`.`game_id`
JOIN `user_socialconnections` AS usc ON `rja`.`addressbook_id` = `usc`.`id`
LEFT JOIN `user_user` AS usr ON `usr`.`email` = `rja`.`email`
LEFT JOIN `user_member` AS mem ON `mem`.`user_id` = `usr`.`id`
LEFT JOIN `game_applied` AS ja ON `ja`.`id` = `rja`.`applied_id`
WHERE
`rja`.`referby_id` = '2389'
GROUP BY
`rja`.`email`
)
UNION ALL
(
SELECT
ja.previous_status,
ja.rejected_status,
ja.rejection_reason_text,
ja.rejection_reason,
jr.id AS rjaId,
jrtm. STATUS AS refer_applied_status,
jp.title,
jp.user_user_id AS playerId,
jp.id AS gameId,
jp.posted_on AS gamePostDate,
jp.game_referal_amount,
jp.country,
jp.country_name,
jp.state,
jp.location,
jp.state_abb,
jp.game_type,
ja.id AS appliedId,
IFNULL(ja. STATUS, '') AS appliedStatus,
IFNULL(ja.admin_review, '') AS admin_review,
ja.is_req_referal_check,
usr.id AS memberId,
jrtm.referto_email AS refappliedEmail,
jr.game_id,
jrtm.id,
mem.state_abb AS memStateAbb,
mem.state AS memState,
mem.zipcode AS memZipcode,
mem.city AS memCity,
mem.country_name AS memCountryNme,
mem.country_code AS memCountry,
usc. NAME AS appliedMemberName,
usc.first_name,
IFNULL(
mem.primary_contact,
usc.phone_number
) AS primary_contact,
usc.last_name,
usc.profileimage_path AS addressbookImage,
usr.profile_image AS userImage,
usr.last_login,
mem.user_experience_year,
mem.user_experience_month,
mem.current_designation,
mem.current_player,
jrtm.refer_on AS appliedDate,
jrtm.referto_addressbookid,
IFNULL(ja.joining_date, '') AS joiningdate,
jp. STATUS AS gameStatus,
jp.games_referal_amount_type AS gameReferalAmountType,
jp.game_freeze_status AS gameFreezeStatus,
jp.game_freeze_message AS gameFreezeMsg,
ja.app_assign_back_to_rp_count,
ja.applied_source,
MAX(jrtm.id) AS max_id,
GROUP_CONCAT(
jp.title
ORDER BY
jr.refer_on DESC
) AS gamesApplied,
GROUP_CONCAT(DISTINCT(jp.id)) AS gamesAppliedId,
COUNT(DISTINCT(jp.id)) totalgameApplied,
ja.application_assign_to_rp_status,
2 AS rpAppliedSource,
jrtm.refer_on AS applied_on
FROM
(`game_refer` AS jr)
JOIN `game_refer_to_member` AS jrtm ON `jrtm`.`rid` = `jr`.`id`
JOIN `games_post` AS jp ON `jp`.`id` = `jr`.`game_id`
JOIN `user_socialconnections` AS usc ON `jrtm`.`referto_addressbookid` = `usc`.`id`
LEFT JOIN `user_user` AS usr ON `usr`.`email` = `jrtm`.`referto_email`
LEFT JOIN `user_member` AS mem ON `mem`.`user_id` = `usr`.`id`
LEFT JOIN `game_applied` AS ja ON `ja`.`referred_by` = `jrtm`.`id`
WHERE
`jrtm`.`status` = '1'
AND `jr`.`referby_user_id` = '2389'
AND `jrtm`.`refer_source` NOT IN ('4')
GROUP BY
`jrtm`.`referto_email`
)
) a
GROUP BY
a.appliedEmail
ORDER BY
a.gamesAppliedId DESC
It sounds like a "groupwise-max" problem. I added a tag that you should research.
At least get rid of the columns that are not relevant to the question.
Try tossing the tables other than jrtm and jr to see if the performance problem persists. (I'm thinking that the LEFT JOINs may be red herrings.)
Try with one part of the UNION, then with the other. This may identify which of the two is more of a burden.
Some indexes to add:
rja: (referby_id, create_date)
jrtm: (status, referto_email)
jrtm: (rid, status, referto_email)
jr: (referby_user_id, refer_on)
DISTINCT is not a function. Don't use parents in DISTINCT(jp.id).

mysql | Format the CONCAT function to have small brackets

i have select in query
IFNULL(GROUP_CONCAT(DISTINCT CONCAT(ED.dependent_name, ED.date_of_birth))," No Dependents ") AS Dependents
The Result i get is like this..
Data is fine but problem with this is, it look untidy, i want to have date of birth in brackets
e-g for Employee E-02, i want record something like this, means enclose the date in small braces.
Muhammad Zubair (1998-12-15) ,Amir Khan (2000-12-15)
Is there any way i can update the above select statement and get the result like i want to have or any other better way to achieve a good looking result.?
MY Query:
SELECT
`E`.`employee_code` AS Employee_Code,
E.full_name AS NAME,
E.father_name AS Father_Name,
IFNULL(
GROUP_CONCAT(
DISTINCT CONCAT( ED.dependent_name '(', ED.date_of_birth, ')')), " No Dependents " ) AS Dependents
FROM
(`employee` E)
INNER JOIN `employee_project` EP
ON `EP`.`employee_id` = `E`.`employee_id`
INNER JOIN `permanant_contacts` PC
ON `PC`.`employee_id` = `E`.`employee_id`
INNER JOIN `ml_district` MLD
ON `MLD`.`district_id` = `PC`.`district`
LEFT JOIN `dependents` ED
ON `ED`.`employee_id` = `E`.`employee_id`
AND ED.trashed = 0
WHERE `E`.`trashed` = 0
GROUP BY `E`.`employee_id`
Have youy tried
CONCAT(ED.dependent_name, '(', ED.date_of_birth, ')')
Try
CONCAT(ED.dependent_name,CONCAT('(',CONCAT(ED.date_of_birth, ')')))

MYSQL table pivot manually to automatic

I'm trying to pivotise my columns to get from rows to tables.
It's working like expected with the code I have now, but when my fields get updated I have to manually edit the query in order to update it.
I'm trying to automate the process but I'm not getting it to work. Any ideas?
This is the manual code which works:
SELECT
md.entity_guid AS guid, username, e.time_created, time_updated, e.enabled, banned,e.last_action, last_login,
MAX(IF(msn.string = 'question1', msv.string, NULL)) AS question1
FROM exp_metadata md
JOIN exp_metastrings msn ON md.name_id = msn.id
JOIN exp_metastrings msv ON md.value_id = msv.id
JOIN exp_users_entity u ON u.guid = md.entity_guid
JOIN exp_entities e ON e.guid = md.entity_guid
GROUP BY
guid
And this is the query I'm trying to do to automate it:
SET #sql = NULL;
SELECT
md.entity_guid AS guid, username, e.time_created, time_updated, e.enabled, banned, e.last_action, last_login,
GROUP_CONCAT(DISTINCT
CONCAT(
'MAX(IF(msn.string = ''',
msn.string,
''', msv.string, NULL)) AS ',
msn.string
)
) INTO #sql
FROM exp_metadata md
JOIN exp_metastrings msn ON md.name_id = msn.id
JOIN exp_metastrings msv ON md.value_id = msv.id
JOIN exp_users_entity u ON u.guid = md.entity_guid
JOIN exp_entities e ON e.guid = md.entity_guid
SET #sql = CONCAT('SELECT e.guid, ', #sql, ' FROM exp_entities e GROUP BY e.guid');
I get an error in phpmyadmin about the last line.
When I delete the last line to see if it does anything, I get:
#1222 - The used SELECT statements have a different number of columns
Any help would be appreciated.
Thanks a lot, Dries

group_concat missing on of the group

I have two tables, the first puts puts pairs of people into a group with fldPairNum and a second table which collects scores for each individual person.
The problem I have is that if only one of the pair has submitted a score, then only their name appears in the 'nameOfPair' column, but I really need both names. What can I do to fix this?
SELECT
group_concat(DISTINCT `delegate`.`fldFirstName`,' ',`delegate`.`fldSurname` SEPARATOR ' and ') AS 'nameOfPair',
Sum(`data`.`fldScore`) AS 'totalScore'
FROM
`data`
Inner Join `delegate` ON `data`.`fldDelegateID` = `delegate`.`fldID`
WHERE
`delegate`.`fldCategory` > '0'
AND
`delegate`.`fldPairNum` > '0'
GROUP BY
`delegate`.`fldPairNum`
Many thanks
Dave
SELECT GROUP_CONCAT(DISTINCT
`delegate`.`fldFirstName`, ' ', `delegate`.`fldSurname`
SEPARATOR
' and ') AS 'nameOfPair',
SUM(`data`.`fldScore`) AS 'totalScore'
FROM `delegate`
LEFT JOIN `data`
ON `data`.`fldDelegateID` = `delegate`.`fldID`
WHERE `delegate`.`fldCategory` > '0'
AND `delegate`.`fldPairNum` > '0'
GROUP BY `delegate`.`fldPairNum`