Hello I have table with 500k records and folowing columns:
id, id_route, id_point, lat, lng, distance, status
I want to select id_routes which are inside radius from my defined point.
Thats no problem
SELECT id_route
FROM route_path
WHERE (((lat < 48.7210 + 2.0869) AND
(lat > 48.7210 - 2.0869)) AND
((lng < 21.2578 + 2.0869) AND
(lng > 21.2578 - 2.0869)))
GROUP BY id_route
But according PHPmyadmin it takes 0.2s. This is pretty to much since I am going to build huge query and this is just beginning.
I have also index on id_route.
Primary key is id, schema is MyISAM
EXPLAIN of SELECT:
id
select_type
table
type
possible_keys
key
key_len
ref
rows
Extra
1
SIMPLE
route_path
ALL
NULL
NULL
NULL
NULL
506902
Using where; Using temporary; Using filesort
How can I reduce time, I think 500K is not su much records to make it so long? Thanks
if queries takes longer time , and you have set the index properly then you need a powerful server to compute the query quickly !
A 2-dimensional search is inherently slow. The tools won't tell you how to improve this particular query.
You seem to have no indexes in your table?? You should at least try INDEX(lat). that will limit the effort to a stripe of about 4 degrees (in your example). This probably includes thousands of rows. Most of them are then eliminated by checking lng, but not until after fetching all of those thousands.
So, you are tempted to try INDEX(lat, lng) only to find that it ignores lng. And perhaps it runs slower because the index is bigger.
INDEX(lat, lng, id) and using a subquery to find the ids, then doing a self-join back to the table to do the rest of the work is perhaps the simplest semi-straightforward solution. This is slightly beneficial because that is a "covering index" for the subquery, and, although you scan thousands of rows in the index, you don't have to fetch many rows in the data.
Can it be made faster? Yes. However, the complexity is beyond the space available here. See Find the nearest 10 pizza parlors. It involves InnoDB (to get index clustering), PARTITIONs (as crude 2D indexing) and modifications to the original data (to turn lat/lng into integers for PARTITION keys).
Click on follwing Link to know how to improve MySQL performance
MySQL Query Analyzer
MySQL performance tools
Related
I'm working with a MariaDB (MySQL) table which contains information about some map points (latitude and longitude) and a quantity.
I'm making a lot of querys that retrieve some of this points and I want to optimize the query using indexes. I don't know how to do it well.
My queries are like this:
SELECT p.id, p.lat, p.lon, p.quantity
FROM Points p
WHERE ((p.lat BETWEEN -10.0 AND 50.5) AND
(p.lon BETWEEN -30.1 AND 20.2) AND
(100 <= p.quantity AND 2000 >= p.quantity))
ORDER BY p.name DESC;
So, the columns involved in the queries are: lat, lon and quantity.
Could anyone help me?
What you want here is a spatial index. You will need to alter the schema of your table (by turning lat and lon into a single POINT or GEOMETRY value) to support this, and use specific functions to query that value. Once you've done this, you can create a spatial index using CREATE SPATIAL INDEX; this index will allow you to perform a variety of highly optimized queries against the value.
There's more information on using spatial types in MySQL in the "Spatial Data Types" section of the MySQL manual.
When you have multiple range conditions, even if you have an standard B-tree index on all the columns, you can only get an index to optimize the first range condition.
WHERE ((p.lat BETWEEN -10.0 AND 50.5) -- index on `lat` helps
AND (p.lon BETWEEN -30.1 AND 20.2) -- no help from index
AND (100 <= p.quantity AND 2000 >= p.quantity)) -- no help from index
You can either index lat or you can index lon or you can index quantity but your query will only be able to use an B-tree index to optimize one of these conditions.
This is why the answer from #achraflakhdhar is wrong, and it's why the answer from #duskwuff suggested using a spatial index.
A spatial index is different from a B-tree index. A spatial index is designed to help exactly this sort of case, where you need range conditions in two dimensions.
Sorry this sounds like it will cause some rework for your project, but if you want it to be optimized, that's what you will have to do.
Toss indexes you have, and add these:
INDEX(lat, lon),
INDEX(lon, lat),
INDEX(quantity)
Some discussion is provided here
I need to do a live search using PHP and jQuery to select cities and countries from the two tables cities (almost 3M rows) and countries (few hundred rows).
For a short moment I was thinking of using a MyISAM table for cities as InnoDB does not support FULLTEXT search, however decided it is not a way to go (frequent table crashes, all other tables are InnoDB etc and with MySQL 5.6+ InnoDB also starts to support FULLTEXT index).
So, right now I still use MySQL 5.1, and as most cities consist of one-word only or max 2-3 Words, but e.g. "New York" - most people will not search for "York" if they mean "New York". So, I just put an index on the city_real column (which is a varchar).
The following query (I tried it in different versions, without any JOIN and without ORDER BY, with USE INDEX and even with FORCE INDEX, I have tried LIKE instead equal (=) but another post said = was faster and if the wildcard is only at the end, it is OK to use it), in EXPLAIN it always says "using where, using filesort". The average time for the query is about 4sec, which you have to admit is a little bit to slow for a live search (user typing in text-box and seeing suggestions of cities and countries)...
Live search (jQuery ajax) searches if the user typed at least 3 characters...
SELECT ci.id, ci.city_real, co.country_name FROM cities ci LEFT JOIN countries co ON(ci.country_id=co.country_id) WHERE city_real='cit%' ORDER BY population DESC LIMIT 5
There is a PRIMARY on ci.id and an INDEX on ci.city_real. Any ideas why MySQL does not use the index? Or how I could speed up the query? Or where else I should/should not set an INDEX?
Thank you very much in advance for your help!
Here's the explain output
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE ci range city_real city_real 768 NULL 1250 Using where; Using filesort
1 SIMPLE co eq_ref PRIMARY PRIMARY 6 fibsi_1.ci.country_id 1
You should use WHERE city_real LIKE 'cit%', not WHERE city_real='cit%'.
I have tried LIKE instead equal (=) but another post said = was faster and if the wildcard is only at the end, it is OK to use it
This is wrong. = doesn't support wildcards so it will give you the wrong results.
Or how I could speed up the query?
Make sure you have an index on country_id in both tables. Post the output of EXPLAIN SELECT ... if you need further help.
The query do use index as seen in the key field of explain output. The reason it uses filesort is the order by, and the reason it uses where is probably one of the fields (city_real, population) allows null values.
I have the following query:
SELECT *
FROM shop_user_member_spots
WHERE delete_flag = 0
ORDER BY date_spotted desc
LIMIT 10
When run, it takes a few minutes. The table is around 2.5 million rows.
Here is the table (not designed by me but I am able to make some changes):
And finally, here are the indexes (again, not made by me!):
I've been attempting to get this query running fast for hours now, to no avail.
Here is the output of EXPLAIN:
Any help / articles are much appreciated.
Based on your query, it seems the index you would want would be on (delete_flag, date_spotted). You have an index that has the two columns, but the id column is in between them, which would make the index unhelpful in sorting based on date_spotted. Now whether mysql will use the index based on Zohaib's answer I can't say (sorry, I work most often with SQL Server).
The problem that I see in the explain plan is that the index on spotted date is not being used, insted filesort mechanism is being used to sort (as far as index on delete flag is concerned, we actually gain performance benefit of index if the column on which index is being created contains unique values)
the mysql documentation says
Index will not used for order by clause if
The key used to fetch the rows is not the same as the one used in the ORDER BY:
SELECT * FROM t1 WHERE key2=constant ORDER BY key1;
http://dev.mysql.com/doc/refman/5.0/en/order-by-optimization.html
I guess same is the case here. Although you can try using Force Index
This query pops up in my slow query logs:
SELECT
COUNT(*) AS ordersCount,
SUM(ItemsPrice + COALESCE(extrasPrice, 0.0)) AS totalValue,
SUM(ItemsPrice) AS totalValue,
SUM(std_delivery_charge) AS totalStdDeliveryCharge,
SUM(extra_delivery_charge) AS totalExtraDeliveryCharge,
this_.type AS y5_,
this_.transmissionMethod AS y6_,
this_.extra_delivery AS y7_
FROM orders this_
WHERE this_.deliveryDate BETWEEN '2010-01-01 00:00:00' AND '2010-09-01 00:00:00'
AND this_.status IN(1, 3, 2, 10, 4, 5, 11)
AND this_.senderShop_id = 10017
GROUP BY this_.type, this_.transmissionMethod, this_.extra_delivery
ORDER BY this_.deliveryDate DESC;
The table is InnoDB and has about 880k rows and takes between 9-12 seconds to execute. I tried adding the following index ALTER TABLE orders ADD INDEX _deliverydate_senderShopId_status ( deliveryDate , senderShop_id , status, type, transmissionMethod, extra_delivery); with no practical gains. Any help and/or suggestion is welcomed
Here is the query execution plan right now:
id select_type table type possible_keys key key_len ref rows filtered Extra
1 SIMPLE this_ ref FKC3DF62E57562BA6F 8 const 139894 100.00 Using where; Using temporary; Using filesort
I took out the possible_keys value out of the text because i think it listed all the indexes in the table. The key used (FKC3DF62E57562BA6F) looks like
Keyname Type Unique Packed Field Cardinality Collation Null Comment
FKC3DF62E57562BA6F BTREE No No senderShop_id 4671 A
I'll tell you one thing that you can look at for increasing the speed.
You only generally have NULL values in the data for either unknown or non-applicable rows. It appears to me that, since you're treating NULL as 0 anyway, you should think about getting rid of them and making sure that all extrasPrice values are 0 where they were previously NULL so that you can get rid of the time penalty of the coalesce.
In fact, you could go one step further and introduce another column called totalPrice which you set with an insert/update trigger to the actual value ItemsPrice + extrasPrice or (ItemsPrice + COALESCE(extrasPrice,0.0) if you still need nullability of extrasPrice).
Then, you can simply use:
SELECT
COUNT(*) AS ordersCount,
SUM(totalPrice) AS totalValue,
SUM(ItemsPrice) AS totalValue2,
:
(I'm not sure you should have two output columns with the same name or whether that was a typo, that's going to be, at worst, an error, at best, confusing).
This moves the cost of the calculation to insert/update time rather than select time and amortises that cost over all the selects - most database tables are read far more often than written. The consistency of the data is maintained due to the trigger and the performance should be better, at the cost of some storage requirements.
But, since the vast majority of database questions are "How can I get more speed?" rather than "How can I use less disk?", that's often a good idea.
Another suggestion is to provide a non-composite index on the column that reduces your result set the fastest (high cardinality). In other words, if you store only two weeks worth of data (14 different dates) in your table but 400 different shops, you should have an index on senderShop_id and make sure your statistics are up to date.
This should cause the DBMS execution engine to whittle down the result set using that key so that subsequent operations are faster.
A composite index on deliveryDate,senderShop_id,... will not be able to use senderShop_id to whittle down the results because the key ordering will be senderShop_id within deliveryDate.
Say I have a simple query like this:
SELECT * FROM topics ORDER BY last_post_id DESC
In accordance to MySQL's Order-By Optimization Docs, an index on last_post_id should be utilized. But EXPLAINing the query tells the contrary:
id select_type table type possible_keys key key_len ref rows Extra
1 SIMPLE topic_info ALL NULL NULL NULL NULL 13 Using filesort
Why is my index not being utilized?
Do you really only have 13 rows? The database may be deciding that simply sorting them is quicker than going via the index.
A basic principle of index optimization is to use representative data for testing. A few rows in a table has no predictive value about how either the index or the optimizer will work in real life.
If you really have only a few records, then the indexing will provide no effective benefit.
EXPLAIN only tells you about the selection process, in which case it's expected that the query would have to examine all 13 rows to see if they meet the WHERE clause (which you don't have, so it's useless information!). It only reports the indexes and keys used for this purpose (evaluating WHERE, JOIN, HAVING). So, regardless of whether the query uses an index to sort or not, EXPLAIN won't tell you about it, so don't get caught up in its results.
Yes, the query uses the index to sort quickly. I've noticed the same results from EXPLAIN (aka reporting all rows despite being sorted by an index and having a limit) and I doubt it's a result of the small number of rows in your table, but rather a limitation of the power of EXPLAIN.
Try selecting the specific columns in order as they are in the table. MySQL indexes don't hold when the order is changed.