We have two tables one is properties and another one is property meta when we are getting data from one table "properties" , query only take less then one second in execution but when we are use join to get the data using bellow query from both tables its taking more then 5 second to fetch the data although we have only 12000 record in the tables , i think there is an issue in the sql query any help or suggestion will be appreciated.
SELECT
u.id,
u.property_title,
u.description,
u.city,
u.area,
u.address,
u.slug,
u.latitude,
u.longitude,
u.sync_time,
u.add_date,
u.is_featured,
u.pre_construction,
u.move_in_date,
u.property_status,
u.sale_price,
u.mls_number,
u.bedrooms,
u.bathrooms,
u.kitchens,
u.sub_area,
u.property_type,
u.main_image,
u.area_size as land_area,
pm7.meta_value as company_name,
pm8.meta_value as virtual_tour,
u.year_built,
u.garages
FROM
tbl_properties u
LEFT JOIN tbl_property_meta pm7
ON u.id = pm7.property_id
LEFT JOIN tbl_property_meta pm8
ON u.id = pm8.property_id
WHERE
u.status = 1
AND (pm7.meta_key = 'company_name')
AND (pm8.meta_key = 'virtual_tour')
AND (
(
( u.city = 'Delta'
OR u.post_code LIKE '%Delta%'
OR u.sub_area LIKE '%Delta%'
OR u.state LIKE '%Delta%')
AND country = 'Canada'
)
OR (
( u.city = 'Metro Vancouver Regional District'
OR u.post_code LIKE '%Metro Vancouver Regional District%'
OR u.sub_area LIKE '%Metro Vancouver Regional District%'
OR u.state LIKE '%Metro Vancouver Regional District%' )
AND country = 'Canada'
)
)
AND
u.pre_construction ='0'
GROUP BY
u.id
ORDER BY
u.is_featured DESC,
u.add_date DESC
Try adding this compound index:
ALTER TABLE tbl_property_meta ADD INDEX id_key (property_id, meta_key);
If it doesn't help make things faster, try this one.
ALTER TABLE tbl_property_meta ADD INDEX key_id (meta_key, property_id);
And, you should know that column LIKE '%somevalue' (with a leading %) is a notorious performance antipattern, resistant to optimization via indexes. (There's a way to create indexes for that shape of filter in PostgreSQL, but not in MariaDB / MySQL.)
Add another column with the meta stuff; throw city, post_code, sub_area, and state and probably some other things into it. Then build a FULLTEXT index on that column. Then use MATCH(..) AGAINST("Delta Metro Vancouver Regional District") in the WHERE clause _instead of the LEFT JOINs (which are actually INNER JOINs) and the really messy part of the WHERE clause.
Also, the GROUP BY is probably unnecessary, thereby eliminating extra sort on the intermediate set of rows.
Related
I have a SQL query that links 2 tables to provide the data if a horse is in both the tables:
SELECT ProformSystem.TheDate as racedate,
ProformSystem.Course as course,
ProformSystem.TheTime as thetime,
ProformSystem.Horse as horse,
ATRSpeedRatings.rank as rank
FROM ATRSpeedRatings
INNER JOIN ProformSystem ON (ATRSpeedRatings.Horse = trim(ProformSystem.Horse)) AND (ATRSpeedRatings.TheDate = ProformSystem.TheDate) order by ProformSystem.TheTime;"
Is it possible that if the horse in ProformSystem.Horse is NOT in ATRSpeedRatings.Horse then I just make rank = 0 as a default value, or would I need to run a separate query?
This way I can display all horses from ProformSystem even if they don't have a rank in ATRSpeedRatings.
I think you want a left join and coalesce():
SELECT ps.TheDate as racedate, ps.Course as course, ps.TheTime as thetime,
ps.Horse as horse, COALESCE(sr.rank, 0) as rank
FROM ProformSystem ps LEFT JOIN
ATRSpeedRatings sr
ON sr.Horse = TRIM(ps.Horse) AND sr.TheDate = ps.TheDate)
ORDER BY ps.TheTime;
Note that this query uses table aliases. These make the query easier to write and to read.
Also, the JOIN condition sr.Horse = trim(ps.Horse) is highly suspect. You should fix the data so there are no spaces in ProformSystem. Fixing the data is more efficient and it will prevent problems on future queries.
I am working on project management tool and while generating reports through admin end the loading time for the data from the database is very much > 5 minutes. There are few points which I know would help me to increase the performance but right now I need to have the help in SELECT query
SELECT timesheet_client.organisation as client_name, timesheet_project.title as project_name,
timesheet_task.name as task, CONCAT(timesheet_user.first_name, ' ', timesheet_user.last_name) as resource_name,
timesheet_user.bill_factor, timesheet_client.client_type, sum(spent) as spent, sum(delivered_hours) as delivered_hours,
sum(billable_hours) as billable_hours, comments, color, lock_color, updated_by, updated_by_date, timesheet_user_psdb.grp_id,
timesheet_user_psdb.client_id, timesheet_user_psdb.proj_id, timesheet_user_psdb.task_id, timesheet_user_psdb.uid,
timesheet_user_grp.grp_name
FROM timesheet_user_psdb,timesheet_user, timesheet_client,
timesheet_project,timesheet_task,timesheet_user_grp
WHERE timesheet_user.username=timesheet_user_psdb.uid and
timesheet_client.client_id=timesheet_user_psdb.client_id and timesheet_project.proj_id=timesheet_user_psdb.proj_id and
timesheet_task.task_id = timesheet_user_psdb.task_id and timesheet_user_grp.grp_id=timesheet_user_psdb.grp_id and month =3
AND year = 2017 and month!='' and timesheet_user_psdb.client_id=326
GROUP BY timesheet_user_psdb.task_id,timesheet_user_psdb.uid
ORDER BY timesheet_client.client_type desc,timesheet_client.organisation,timesheet_user_psdb.proj_id,
timesheet_user_psdb.task_id,timesheet_user.uid,timesheet_user_psdb.task_id;
I have already used an index on all the primary keys.
EXPLAIN Output:
Help for this would be highly appreciable.
Give this a try:
SELECT
TC.ORGANISATION AS CLIENT_NAME,
TP.TITLE AS PROJECT_NAME,
TT.NAME AS TASK,
CONCAT(TU.FIRST_NAME, ' ', TU.LAST_NAME) AS RESOURCE_NAME,
TU.BILL_FACTOR,
TC.CLIENT_TYPE, SUM(SPENT) AS SPENT, -- You should specify which table this comes from
SUM(DELIVERED_HOURS) AS DELIVERED_HOURS, -- You should specify which table this comes from
SUM(BILLABLE_HOURS) AS BILLABLE_HOURS, -- You should specify which table this comes from
COMMENTS, -- You should specify which table this comes from
COLOR, -- You should specify which table this comes from
LOCK_COLOR, -- You should specify which table this comes from
UPDATED_BY, -- You should specify which table this comes from
UPDATED_BY_DATE, -- You should specify which table this comes from
TUP.GRP_ID,
TUP.CLIENT_ID,
TUP.PROJ_ID,
TUP.TASK_ID,
TUP.UID,
TUG.GRP_NAME
FROM
TIMESHEET_USER AS TU
LEFT OUTER JOIN
TIMESHEET_USER_PSDB AS TUP
ON TU.USERNAME = TUP.UID
LEFT OUTER JOIN
TIMESHEET_USER_GRP AS TUG
ON TUP.GRP_ID = TUG.GRP_ID
LEFT OUTER JOIN
TIMESHEET_CLIENT AS TC
ON TUP.CLIENT_ID = TC.CLIENT_ID
LEFT OUTER JOIN
TIMESHEET_PROJECT AS TP
ON TUP.PROJ_ID = TP.PROJ_ID
LEFT OUTER JOIN
TIMESHEET_TASK TT
ON TUP.TASK_ID = TT.TASK_ID
WHERE
MONTH = 3 AND -- You should specify which table this comes from
YEAR = 2017 AND -- You should specify which table this comes from
MONTH != '' AND -- You should specify which table this comes from
TUP.CLIENT_ID = 326
GROUP BY
TUP.TASK_ID,
TUP.UID
ORDER BY
TC.CLIENT_TYPE DESC,
TC.ORGANISATION,
TUP.PROJ_ID,
TUP.TASK_ID,
TU.UID,
TUP.TASK_ID;
Doing the EXPLAIN should shed some light on it.
You might see a performance gain by doing the table joins explicitly in the FROM clause instead of inside the WHERE (https://dev.mysql.com/doc/refman/5.7/en/join.html). By doing explicit joins (e.g. LEFT OUTER, etc...) you will not only improve the readability of the query, but may be able to use less expensive joins where needed. This also affects how the query is executed as each clause is executed in a specific order (MySQL query / clause execution order).
I need to write query that joins several tables and I need distinct value from one table based on max count().
These are my tables names and columns:
bands:
db|name|
releases_artists:
release_db|band_db
releases_styles
release_db|style
Relations between tables are (needed for JOINs):
releases_artists.band_db = bands.db
releases_styles.release_db = releases_artists.release_db
And now the query that I need to write:
SELECT b.name, most_common_style
LEFT JOIN releases_artists ra ON ra.band_db = b.db
and here I need to find the most common style from all band releases
JOIN(
SELECT DISTINCT style WHERE releases_styles.release_db = ra.release_db ORDER BY COUNT() DESC LIMIT 1
)
FROM bands b
WHERE b.name LIKE 'something'
This is just a non working example of what I want to accomplish. It would be great if someone could help me build this query.
Thanks in advance.
EDIT 1
Each artist from table bands can have multiple records from releases_artists table based on band_db and each release can have multiple styles from releases_styles based on release_db
So if I search for b.name LIKE '%ray%' it returns something similar to:
`bands`:
o7te|Ray Wilson
9i84|Ray Parkey Jr.
`releases_artists` for Ray Wilson:
tv5c|o7te (for example album `Change`)
78wz|o7te (`The Next Best Thing`)
nz7c|o7te (`Propaganda Man`)
`releases_styles`
tv5c|Pop
tv5c|Rock
tv5c|Alternative Pop/Rock
----
78wz|Rock
78wz|Pop
78wz|Classic Rock
I need style name that repeats mostly from all artist releases as this artist main style.
Ok, this is a bit of a hack. But the only alternatives I could think of involve heaps of nested subqueries. So here goes:
SELECT name
, SUBSTRING_INDEX(GROUP_CONCAT(style ORDER BY release_count DESC SEPARATOR '|'), '|', 1) AS most_common_style
FROM (
SELECT b.db
, b.name
, rs.style
, COUNT(*) AS release_count
FROM bands b
JOIN releases_artists ra ON ra.band_db = b.db
JOIN releases_styles rs ON rs.release_db = ra.release_db
GROUP BY b.db, rs.style
) s
GROUP BY db;
I've (on reflection ridiculously) stored our 'punters' in 2 tables depending on whether they registered or paid through the express checkout form.
My SQL looks like this:
SELECT
DISTINCT(sale_id), sale_punter_type, sale_comment, sale_refund, sale_timestamp,
punter_surname, punter_firstname,
punter_checkout_surname, punter_checkout_firstname,
punter_compo_surname, punter_compo_firstname,
sale_random, sale_scanned, sale_id
FROM
sale
LEFT JOIN
punter ON punter_id = sale_punter_no
LEFT JOIN
punter_checkout ON punter_checkout_id = sale_punter_no
LEFT JOIN
punter_compo ON punter_compo_id = sale_punter_no
WHERE
sale_event_no = :id
ORDER BY
punter_surname, punter_firstname,
punter_checkout_surname, punter_checkout_firstname
It returns results BUT lists the registered users alphabetically first, then the checkout punters alphabetically.
My question is there a way to get all of the users (registered or checkout) all together sorted alphabetically in 1 sorted list instead of 2 joined sorted lists.
I thought maybe I could use something like punter_checkout_surname AS punter_surname but that didn't work.
Any thoughts? I know now that I shouldn't have used 2 separate tables but but I'm stuck with it now.
Thank you.
I think you just want to use coalesce().
ORDER BY COALESCE(punter_surname, punter_checkout_surname)
COALESCE(punter_firstname, punter_checkout_firstname)
Other comments:
I doubt that DISTINCT is necessary. Why would this generate multiple rows for a single sale_id?
When a query has multiple tables, qualify all the column names (that is, include table aliases so you and others know where the table comes from).
Your data has three sets of names. That seems overkill.
You might want to put the COALESCE() in the SELECT so you don't have quite so many names generated by the query.
Here's my answer:
ORDER BY
COALESCE( UCASE( punter_surname) , UCASE( punter_checkout_surname ), UCASE( punter_compo_surname ) ) ,
COALESCE( UCASE( punter_firstname ) , UCASE( punter_checkout_firstname ), UCASE( punter_compo_firstname) )
I'm in over my head with a big mysql query (mysql 5.0), and i'm hoping somebody here can help.
Earlier I asked how to get distinct values from a joined query
mysql count only for distinct values in joined query
The response I got worked (using a subquery with join as)
select *
from media m
inner join
( select uid
from users_tbl
limit 0,30) map
on map.uid = m.uid
inner join users_tbl u
on u.uid = m.uid
unfortunately, my query has grown more unruly, and though I have it running, joining into a derived table is taking too long because there is no indexes available to the derived query.
my query now looks like this
SELECT mdate.bid, mdate.fid, mdate.date, mdate.time, mdate.title, mdate.name,
mdate.address, mdate.rank, mdate.city, mdate.state, mdate.lat, mdate.`long`,
ext.link,
ext.source, ext.pre, meta, mdate.img
FROM ext
RIGHT OUTER JOIN (
SELECT media.bid,
media.date, media.time, media.title, users.name, users.img, users.rank, media.address,
media.city, media.state, media.lat, media.`long`,
GROUP_CONCAT(tags.tagname SEPARATOR ' | ') AS meta
FROM media
JOIN users ON media.bid = users.bid
LEFT JOIN tags ON users.bid=tags.bid
WHERE `long` BETWEEN -122.52224684058 AND -121.79760915942
AND lat BETWEEN 37.07500915942 AND 37.79964684058
AND date = '2009-02-23'
GROUP BY media.bid, media.date
ORDER BY media.date, users.rank DESC
LIMIT 0, 30
) mdate ON (mdate.bid = ext.bid AND mdate.date = ext.date)
phew!
SO, as you can see, if I understand my problem correctly, i have two derivative tables without indexes (and i don't deny that I may have screwed up the Join statements somehow, but I kept messing with different types, is this ended up giving me the result I wanted).
What's the best way to create a query similar to this which will allow me to take advantage of the indexes?
Dare I say, I actually have one more table to add into the mix at a later date.
Currently, my query is taking .8 seconds to complete, but I'm sure if I could take advantage of the indexes, this could be significantly faster.
First, check for indices on ext(bid, date), users(bid) and tags(bid), you should really have them.
It seems, though, that it's LONG and LAT that cause you most problems. You should try keeping your LONG and LAT as a (coordinate POINT), create a SPATIAL INDEX on this column and query like that:
WHERE MBRContains(#MySquare, coordinate)
If you can't change your schema for some reason, you can try creating additional indices that include date as a first field:
CREATE INDEX ix_date_long ON media (date, `long`)
CREATE INDEX ix_date_lat ON media (date, lat)
These indices will be more efficient for you query, as you use exact search on date combined with a ranged search on axes.
Starting fresh:
Question - why are you grouping by both media.bid and media.date? Can a bid have records for more than one date?
Here's a simpler version to try:
SELECT
mdate.bid,
mdate.fid,
mdate.date,
mdate.time,
mdate.title,
mdate.name,
mdate.address,
mdate.rank,
mdate.city,
mdate.state,
mdate.lat,
mdate.`long`,
ext.link,
ext.source,
ext.pre,
meta,
mdate.img,
( SELECT GROUP_CONCAT(tags.tagname SEPARATOR ' | ')
FROM tags
WHERE ext.bid = tags.bid
ORDER BY tags.bid GROUP BY tags.bid
) AS meta
FROM
ext
LEFT JOIN
media ON ext.bid = media.bid AND ext.date = media.date
JOIN
users ON ext.bid = users.bid
WHERE
`long` BETWEEN -122.52224684058 AND -121.79760915942
AND lat BETWEEN 37.07500915942 AND 37.79964684058
AND ext.date = '2009-02-23'
AND users.userid IN
(
SELECT userid FROM users ORDER BY rank DESC LIMIT 30
)
ORDER BY
media.date,
users.rank DESC
LIMIT 0, 30
You might want to compare your perforamnces against using a temp table for each selection, and joining those tables together.
create table #whatever
create table #whatever2
insert into #whatever select...
insert into #whatever2 select...
select from #whatever join #whatever 2
....
drop table #whatever
drop table #whatever2
If your system has enough memory to hold full tables this might work out much faster. It depends on how big your database is.