If someone could offer advice on improving the below query this would be most useful. I am unsure how I can make improvement when i have a left join twice in many instance to seperate tables. For example I have a location left join to the user table and a location left join to the image gallery table. I was unsure if i could optimise the sql from this point of view. It is very slow at the moment. I have ensured all columns are indexed on all joins and where statements.
SELECT im.alias_title, im.title,im.guid_id, im.description, im.hits, im.show_comment, im.can_print,
im.can_download, im.can_share, im.created_on, im.date_taken, im.approved, im.visible,
ad.address_line_1, ad.address_line_2, ad.town_village_city, ad.state_province_county, ad.postal_code, ad.other_address_detail, co.country,
geo.latitude, geo.longitude, geo.zoom, geo.yaw, geo.pitch,
c.make, c.model,
us.first_name, us.surname, uf.user_id, uf.real_name, uf.user_name, uf.gender, uf.description, uf.description, uf.buddy_icon_url, uf.first_taken_date, uf.first_date,
uf.time_zone_label, uf.time_zone_offset,
adf.address_line_1 as user_address_line_1, adf.address_line_2 as user_address_line_2, adf.town_village_city as user_town_village_city, adf.state_province_county as user_state_province_county,
adf.postal_code as user_postal_code, adf.other_address_detail as user_other_address_detail, cof.country as user_country,
geof.latitude as user_geolocation_latitude, geof.longitude as user_geolocation_longitude, geof.zoom as user_geolocation_zoom, geof.yaw as user_geolocation_yaw, geof.pitch as user_geolocation_pitch,
im.alias_title = in_image_alias_title AS image_selected -- image selected
FROM image im
LEFT JOIN address ad ON im.address_id = ad.id
LEFT JOIN country co ON ad.country_id = co.id
LEFT JOIN geolocation geo ON im.geolocation_id = geo.id
LEFT JOIN camera c ON im.camera_id = c.id
INNER JOIN user us ON im.user_id = us.id
LEFT JOIN user_flickr uf ON us.id = uf.id
LEFT JOIN address adf ON uf.address_id =adf.id
LEFT JOIN country cof ON ad.country_id = cof.id
LEFT JOIN geolocation geof ON uf.geolocation_id = geof.id
WHERE (im.alias_title = in_image_alias_title OR im.user_id = user_id)
AND im.approved = in_image_approved
AND im.visible = in_image_visible
AND (im.advertise_to <= NOW() OR im.advertise_to IS NULL)
ORDER BY image_selected DESC;
After the discussion / chat room, and learning more of what you were trying to do...
Build a compound index on the components associated with your where clause so all parts can be applied, not just the best of the first key element. Also, by removing the "alias_title" from the where clause (since you were getting the user ID based on the alias title to begin with), it was a redundant clause taking up more consideration in the query.
I would index on (user_id, approved, visible, advertise_to )
The results will come back and be small in the scheme of things, so your ultimate "order by" clause will have no problem with its final sort output.
Related
I am struggling with building mysql query so it returns right results. Idea is that i need to fetch data from main table but some of the fields are only references to other tables where also record exists in various languages.
so example code is:
SELECT cars.model
FROM cars
LEFT JOIN parts ON parts.id = cars.partId AND parts.language IN ('en', 'de')
LEFT JOIN interior ON interior.id = cars.interiorId AND interior.language IN ('en', 'de')
LEFT JOIN exterior ON exterior.id = cars.exteriorId AND exterior.language IN ('en', 'de')
LEFT JOIN wheels ON wheels.id = cars.wheelId AND wheels.language IN ('en', 'de')
LEFT JOIN extra ON extra.id = cars.extraId AND extra.language IN ('en', 'de')
WHERE cars.id IN ('72727272') AND cars.source = 1
What i need is two results from query (one in english and one in german), instead i am getting 24 results. They are in various configuration of languages.
I tried adding:
GROUP BY ... but its not working.
DISTINCT same as above
Maybe someone knows some tricks on how to deal with this kind of situation (at worst case i can execute query twice for each language but its extremely slow).
The problem you are experiencing is due to every match from parts, interior, exterior, and so on are being combined with each other regardless of language. You're getting every combination of the associated de and en data.
SELECT cars.model
FROM cars
CROSS JOIN (SELECT 'en' AS language UNION SELECT 'de') AS l
LEFT JOIN parts ON parts.id = cars.partId AND parts.language = l.language
LEFT JOIN interior ON interior.id = cars.interiorId AND interior.language = l.language
LEFT JOIN exterior ON exterior.id = cars.exteriorId AND exterior.language = l.language
LEFT JOIN wheels ON wheels.id = cars.wheelId AND wheels.language = l.language
LEFT JOIN extra ON extra.id = cars.extraId AND extra.language = l.language
WHERE cars.id IN ('72727272')
AND cars.source = 1
Also, I am assuming the full query you are attempting SELECTs more than just cars.model; otherwise, all this JOINing is pointless.
Could you try putting join condition on language as well like below:
SELECT distinct cars.model
FROM cars
LEFT JOIN parts ON parts.id = cars.partId AND parts.language IN ('en', 'de')
LEFT JOIN interior ON interior.id = cars.interiorId AND interior.language =parts.language
LEFT JOIN exterior ON exterior.id = cars.exteriorId AND exterior.language =parts.language
LEFT JOIN wheels ON wheels.id = cars.wheelId AND wheels.language =parts.language
LEFT JOIN extra ON extra.id = cars.extraId AND extra.language =parts.language
WHERE cars.id IN ('72727272') AND cars.source = 1
As you require result in either english or german, it will make sure only english language or german language results are joined.
My mysql view is really slow in thousands of data how can we improve this functionality?
While fetch this view in 10000s data then it takes more than 30 sec. how could we revise this view table?
SELECT
i.jo_in_id,
j.*,
i.jo_in_week_number,
i.jo_in_client_ref_no,
i.cl_id AS jo_in_cl_id,
c.cl_short_name,
c.cl_business_name,
m.me_first_name,
m.me_last_name,
m.me_mobile,
sk.sk_name,
sk.sk_ticketed,
ti.ti_id,
ta.ta_name,
u.un_id,
u.un_from,
u.un_to,
v.ve_name,
mp.vmp_name,
r.vr_name
FROM
jo_2_no j2n,
jo_in_numbers i,
jobs j
LEFT JOIN venues_new v ON j.ve_id = v.ve_id
LEFT JOIN venues_meeting_place mp ON j.vmp_id = mp.vmp_id
LEFT JOIN venues_rooms r ON j.vr_id = r.vr_id
LEFT JOIN clients c ON j.cl_id = c.cl_id
LEFT JOIN members m ON j.me_id = m.me_id
LEFT JOIN skills sk ON j.sk_id = sk.sk_id
LEFT JOIN tasks ta ON j.ta_id = ta.ta_id
LEFT JOIN crew_tickets ti ON j.sk_id = ti.sk_id AND j.me_id = ti.me_id AND j.jo_time_off < ti.ti_expire
LEFT JOIN unavailability u ON j.me_id = u.me_id AND ((j.jo_time_on BETWEEN u.un_from AND u.un_to) OR (j.jo_time_on BETWEEN u.un_from AND u.un_to))
WHERE
j.jo_id = j2n.jo_id
AND j2n.jo_in_numbers_id = i.jo_in_id
AFTER user EXPLAIN SELECT following is the output
In your EXPLAIN, I see that your joined tables ti and u are read with table-scans (type: ALL). This is probably the biggest problem for your performance.
You should make sure you have the following indexes created:
ALTER TABLE crew_tickets ADD KEY (sk_id, me_id, ti_expire);
ALTER TABLE unavailability ADD KEY (me_id, un_from, un_to);
That should help the joins to those tables work with index lookups instead of table-scans. I think they'll be accessed as covering indexes, too.
Also, please don't use the outdated "comma-joins." Especially do not mix both styles. It will bite you when you get surprised by the order of precedence between comma-joins and JOIN operators. See examples in Can someone help explain why not using a SQL JOIN is bad practice and wrong? or Error on JOIN mysql.
Write your joins this way:
FROM jo_2_no j2n
INNER JOIN jo_in_numbers i ON j2n.jo_in_numbers_id = i.jo_in_id
INNER JOIN jobs j ON j.jo_id = j2n.jo_id
LEFT JOIN venues_new v ON j.ve_id = v.ve_id
LEFT JOIN venues_meeting_place mp ON j.vmp_id = mp.vmp_id
LEFT JOIN venues_rooms r ON j.vr_id = r.vr_id
LEFT JOIN clients c ON j.cl_id = c.cl_id
LEFT JOIN members m ON j.me_id = m.me_id
LEFT JOIN skills sk ON j.sk_id = sk.sk_id
LEFT JOIN tasks ta ON j.ta_id = ta.ta_id
LEFT JOIN crew_tickets ti ON j.sk_id = ti.sk_id
AND j.me_id = ti.me_id AND j.jo_time_off < ti.ti_expire
LEFT JOIN unavailability u ON j.me_id = u.me_id
AND j.jo_time_on BETWEEN u.un_from AND u.un_to
I removed the redundant term in the join condition for u. The optimizer might eliminate that logic, but why make it work so hard?
I have a table for audit trail which saves all actions performed records through out the project like add update and delete.
there I maintain a column which saves primary keys of multiple tables on which action is performed. This is integer column
my query is like this
select * from
user usr1
left join activity_history
on activity_history.userID= usr1.sequenceID
left join candidate can1 on can1.userID = usr1.sequenceID
and can1.userID = activity_history.activity_sequenceID
left join institute ins1 on ins1.userID= usr1.sequenceID
and ins1.userID = activity_history.activity_sequenceID
left join candidate_institutes caninst on caninst.candidateID = can1.candidateID and caninst.instituteID= ins1.instituteID
left join exam exam1 on exam1.instituteID = ins1.instituteID
and exam1.examID = activity_history.activity_sequenceID
left join proctor pro1 on pro1.userID = usr1.sequenceID
and pro1.proctorID = activity_history.activity_sequenceID
left join appointment appt1 on appt1.examID = exam1.examID
and appt1.sequenceID = activity_history.activity_sequenceID
/* COMMENTED CODE-----
on( activity_history.activity_sequenceID=can1.userID
OR activity_history.activity_sequenceID=ins1.userID
OR activity_history.activity_sequenceID=exam1.examID and activity_history.userID= usr1.sequenceID
OR activity_history.activity_sequenceID=pro1.userID
OR activity_history.activity_sequenceID=appt1.sequenceID
)
*/
order by activity_history.sequenceID desc
activity_sequenceID is the column where I am storing keys of other tables
I need to map that. Is this a right way to join these tables or Commented part could be the rite way ?
Or is there any other way to join these tables.
I am confused because I am writing a condition in but activity_histry table may or may not have records of that particular table .
Take a look at these tables
It's simple: Venue contains country_ID which is an FK in Society_Territory where we will find a society_ID which is an FK of Society. I have a Venue_ID during the query and my objective is to get the Society_Name but there is a twist but first lets just get the Society_Name
In the following query only look at JOINS and in there I am gonna add comments with this // prefix
SELECT
uuid()AS `UUID`,
`pc`.`PRSClaimID` AS `prsclaimid`,
`a`.`LoginName` AS `loginname`,
`a`.`BandName` AS `bandname`,
`smartistdetails`.`LoginName` AS `createdbyloginname`,
`Society`.`Society_Name` AS societyName
count(
`smliveclaims`.`LiveclaimsID`
)AS `gigcount`
FROM `smprsliveclaimlink`
JOIN `smliveclaims` ON `smprsliveclaimlink`.`fkLiveClaimID` = `smliveclaims`.`LiveclaimsID`
// Here I have the Venue_ID from smliveclaims so i starting moving towards society name
JOIN Venue ON `smliveclaims`.fk_venueId = Venue.Venue_ID
JOIN Society_Territory ON Venue.Country_ID = Society_Territory.Country_ID
JOIN Society ON Society_Territory.Society_Id = Society.Society_ID
// Now from Society i can select the Society_Name which i am already doing in the query above
JOIN `smartistdetails` `a`
JOIN `smprsclaims` `pc` ON `a`.`ArtistID` = `pc`.`fkArtistID`
JOIN `smcategories` ON `pc`.`FK_CategoryID` = `smcategories`.`Id`
JOIN `smcategoriestype` ON `smcategories`.`fk_CategoryTypeId` = `smcategoriestype`.`Id`
JOIN `smartistdetails` ON `pc`.`CreatedBy` = `smartistdetails`.`ArtistID` AND `smprsliveclaimlink`.`fkPRSClaimID` = `pc`.`PRSClaimID`
GROUP BY
`a`.`LoginName`,
`a`.`BandName`,
`smcategories`.`Id`,
`smcategoriestype`.`CategoryType`,
`smartistdetails`.`LoginName`
All is cool till here. Now here is the TWIST
I will have Country_IDs in Venue which will not be in Society_Territory. And I still want to select them and instead of showing and actual Society_Name want to show a word such as "Other"
use a LEFT OUTER JOIN when you link VENUE with SOCIETY_TERRITORY and so on when you link SOCIETY_TERRITORY with SOCIETY
Pay attention: When you use a LEFT OUTER JOIN all tables depends by its must be linked with other LEFT OUTER JOIN because if you use INNER JOIN you cancel di effects on LEFT.
Edit:
SELECT
uuid()AS `UUID`,
`pc`.`PRSClaimID` AS `prsclaimid`,
`a`.`LoginName` AS `loginname`,
`a`.`BandName` AS `bandname`,
`smartistdetails`.`LoginName` AS `createdbyloginname`,
coalesce(`Society`.`Society_Name`, 'Other') AS societyName
count(`smliveclaims`.`LiveclaimsID`)AS `gigcount`
FROM `smprsliveclaimlink`
JOIN `smliveclaims`
ON `smprsliveclaimlink`.`fkLiveClaimID` = `smliveclaims`.`LiveclaimsID`
// Here I have the Venue_ID from smliveclaims so i starting moving towards society name
JOIN Venue ON `smliveclaims`.fk_venueId = Venue.Venue_ID
LEFT OUTER JOIN Society_Territory ON Venue.Country_ID = Society_Territory.Country_ID
LEFT OUTER JOIN Society ON Society_Territory.Society_Id = Society.Society_ID
// Now from Society i can select the Society_Name which i am already doing in the query above
JOIN `smartistdetails` `a`
JOIN `smprsclaims` `pc` ON `a`.`ArtistID` = `pc`.`fkArtistID`
JOIN `smcategories` ON `pc`.`FK_CategoryID` = `smcategories`.`Id`
JOIN `smcategoriestype` ON `smcategories`.`fk_CategoryTypeId` = `smcategoriestype`.`Id`
JOIN `smartistdetails` ON `pc`.`CreatedBy` = `smartistdetails`.`ArtistID` AND `smprsliveclaimlink`.`fkPRSClaimID` = `pc`.`PRSClaimID`
GROUP BY
`a`.`LoginName`,
`a`.`BandName`,
`smcategories`.`Id`,
`smcategoriestype`.`CategoryType`,
`smartistdetails`.`LoginName`
All your JOINs are INNER JOINs. The INNER keyword is optional in MySQL and frequently omitted (as in your example). Use a LEFT OUTER JOIN where required and amend your SELECT clause to include something like "COALESCE(Society_Name,'Other') Society_Name"
I have this query with many left joins and a inner join with dates.
I need to group by id_art (from articles_art) and date_dat (dates_dat). The problem is that is really slow. it takes 3second for 1000records.
dates_dat is indexed in dates_dat table and id_art is a primary key of articles_art.
What can I do to optimize this query?
SELECT
id_art, image2_art, video_art, website,
text.title_int, text.intro_int, text.text_int, text.extra_int,
dat.date_dat, dat.date2_dat,
group_concat(tim.time_tim),
prd.name_prd,
group_concat(cat.name_cat),
trg.name_trg,
spa.name_spa,
spa2.name_spa
FROM
articles_art AS art
LEFT JOIN internText_int AS text ON text.idart_int = art.id_art
INNER JOIN dates_dat AS dat ON art.id_art = dat.idart_dat
LEFT JOIN spaces_spa As spa ON spa.id_spa = dat.idspa_dat
LEFT JOIN spaces_spa As spa2 ON spa.id_spa = dat.idspa2_dat
LEFT JOIN times_tim AS tim ON tim.iddat_tim = dat.id_dat
LEFT JOIN articles_products_artprd AS artprd ON artprd.idart_artprd = art.id_art
LEFT JOIN products_prd AS prd ON prd.id_prd = artprd.idprd_artprd
LEFT JOIN cater_cev AS cev ON cev.idart_cev = dat.idart_dat
LEFT JOIN categories_cat AS cat ON cat.id_cat = cev.idcat_cev
LEFT JOIN targets_trg AS trg ON trg.id_trg = art.idtrg_art
WHERE
prd.id_prd in (1,2)
AND validated_art = 1
AND text.idlin_int in (1,4)
GROUP BY
id_art, date_dat
Look like you can put an index on these columns
prd.id_prd
validated_art
text.idlin_int
Test this first then if this does not work put indexes on column conditions on the ON clause
If data latency isn't an issue, can you hive the data off (perhaps overnight?) into a single normalised table? That way you query a single table without all those JOINS. You could even apply indexes to help speed things up further.