I am using MySQL in my application to store a list of cities.
each long/lat represent the center of the city.
I want to be able to pull all cities that are close to a specific city by the distance of X kilometers.
My question is what will be performing faster for that purpose.
Using the Point column, and use "spatial" queries to retrieve the data ?
OR
Using a Float Longitude column And Float Latitude column. and then use java code to generate the long/lat between distance before running the SQL WHERE BETWEEN query on those values .
Another small question I have, does it make sense to request all cities that are 10 Kilometers from New York. When New York range is probably bigger then 10 kilometers?
Spatial extension will always be better in this case, since it's based on R-Tree indexes, which are optimized for range search in N-dimensional space.
Whereas native mysql indexes are B-Tree and in the best case only one field from the index will be used (for the range comparison), or no index at all (in case if you use some advanced geo formulas like in another answer).
You can use the Haversine formula to query the database.
The query below is using PDO
$stmt = $dbh->prepare("SELECT name, lat, lng, ( 6371 * acos( cos( radians(?) ) * cos( radians( lat ) ) * cos( radians( lng ) - radians(?) ) + sin( radians(?) ) * sin( radians( lat ) ) ) ) AS distance FROM mytable HAVING distance < ? ORDER BY distance LIMIT 0 , 20");
// Assign parameters
$stmt->bindParam(1,$center_lat);
$stmt->bindParam(2,$center_lng);
$stmt->bindParam(3,$center_lat);
$stmt->bindParam(4,$radius);
Where
6371 is the radius of Earth in km
$center_lat & $center_lng cordinates of location
$center_lng radius of search
This query took 1.93 secs to run on a 457K row unindexed database.
name Varchar(50)
lat Decimal(9,6)
lng Decimal(9,6)
Related
I have a table called flags that contains a column called coordinates that is full of MySQL 'points'. I need to perform a query where I get all the flags within a circle based on a latitude and longitude position with 100m radius.
From a usage point of view this is based around the user's position. For example, the mobile phone would give the user's latitude and longitude position and then pass it to this part of the API. It's then up to the API to create an invisible circle around the user with a radius of 100 metres and then return the flags that are in this circle.
It's this part of the API I'm not sure how to create as I'm unsure how to use SQL to create this invisible circle and select points only within this radius.
Is this possible? Is there a MySQL spatial function that will help me do this?
I believe the Buffer() function can do this but I can't find any documentation as to how to use it (eg example SQL). Ideally I need an answer that shows me how to use this function or the closest to it. Where I'm storing these coordinates as geospatial points I should be using a geospatial function to do what I'm asking to maximize efficiency.
Flags table:
id
coordinates
name
Example row:
1 | [GEOMETRY - 25B] | Tenacy AB
For the flags table I have latitude, longitude positions and easting and northing (UTM)
The user's location is just standard latitude/longitude but I have a library that can conver this position to UTM
There are no geospatial extension functions in MySQL supporting latitude / longitude distance computations. There is as of MySQL 5.7.
You're asking for proximity circles on the surface of the earth. You mention in your question that you have lat/long values for each row in your flags table, and also universal transverse Mercator (UTM) projected values in one of several different UTM zones. If I remember my UK Ordnance Survey maps correctly, UTM is useful for locating items on those maps.
It's a simple matter to compute the distance between two points in the same zone in UTM: the Cartesian distance does the trick. But, when points are in different zones, that computation doesn't work.
Accordingly, for the application described in your question, it's necessary to use the Great Circle Distance, which is computed using the haversine or another suitable formula.
MySQL, augmented with geospatial extensions, supports a way to represent various planar shapes (points, polylines, polygons, and so forth) as geometrical primitives. MySQL 5.6 implements an undocumented distance function st_distance(p1, p2). However, this function returns Cartesian distances. So it's entirely unsuitable for latitude and longitude based computations. At temperate latitudes a degree of latitude subtends almost twice as much surface distance (north-south) as a degree of longitude(east-west), because the latitude lines grow closer together nearer the poles.
So, a circular proximity formula needs to use genuine latitude and longitude.
In your application, you can find all the flags points within ten statute miles of a given latpoint,longpoint with a query like this:
SELECT id, coordinates, name, r,
units * DEGREES(ACOS(LEAST(1.0, COS(RADIANS(latpoint))
* COS(RADIANS(latitude))
* COS(RADIANS(longpoint) - RADIANS(longitude))
+ SIN(RADIANS(latpoint))
* SIN(RADIANS(latitude))))) AS distance
FROM flags
JOIN (
SELECT 42.81 AS latpoint, -70.81 AS longpoint,
10.0 AS r, 69.0 AS units
) AS p ON (1=1)
WHERE MbrContains(GeomFromText (
CONCAT('LINESTRING(',
latpoint-(r/units),' ',
longpoint-(r /(units* COS(RADIANS(latpoint)))),
',',
latpoint+(r/units) ,' ',
longpoint+(r /(units * COS(RADIANS(latpoint)))),
')')), coordinates)
If you want to search for points within 20 km, change this line of the query
20.0 AS r, 69.0 AS units
to this, for example
20.0 AS r, 111.045 AS units
r is the radius in which you want to search. units are the distance units (miles, km, furlongs, whatever you want) per degree of latitude on the surface of the earth.
This query uses a bounding lat/long along with MbrContains to exclude points that are definitely too far from your starting point, then uses the great circle distance formula to generate the distances for the remaining points. An explanation of all this can be found here. If your table uses the MyISAM access method and has a spatial index, MbrContains will exploit that index to get you fast searching.
Finally, the query above selects all the points within the rectangle. To narrow that down to only the points in the circle, and order them by proximity, wrap the query up like this:
SELECT id, coordinates, name
FROM (
/* the query above, paste it in here */
) AS d
WHERE d.distance <= d.r
ORDER BY d.distance ASC
UPDATE
Use ST_Distance_Sphere() to calculate distances using a lat/long
http://dev.mysql.com/doc/refman/5.7/en/spatial-convenience-functions.html#function_st-distance-sphere
This assumes the coordinates in the table are stored as a POINT() datatype in a column labeled 'point'. The function X(point) and Y(point) extract the latitude and longitude values from the point value respectively.
SET #lat = the latitude of the point
SET #lon = the longitude of the point
SET #rad = radius in Kilometers to search from the point
SET #table = name of your table
SELECT
X(point),Y(point),*, (
6373 * acos (
cos ( radians( #lat ) )
* cos( radians( X(point) ) )
* cos( radians( Y(point) ) - radians( #lon ) )
+ sin ( radians( #lat ) )
* sin( radians( X(point) ) )
)
) AS distance
FROM #table
HAVING distance < #rad
If you want to do it in miles, replace the constant 6373 with 3959
For those wanting to reduce the query syntax, here's a common implementation of a user defined MySQL function for implementing a distance function based on the Haversine formulae.
CREATE FUNCTION HAVERSINE ( coord1 POINT, coord2 POINT )
RETURNS DOUBLE
DETERMINISTIC
BEGIN
DECLARE dist DOUBLE;
SET rlat1 = radians( X( coord1 ) );
SET rlat2 = radians( X( coord2 ) );
SET rlon1 = radians( Y( coord1 ) );
SET rlon2 = radians( Y( coord2 ) );
SET dist = ACOS( COS( rlat1 ) * COS( rlon1 ) * COS( rlat2 ) * COS( rlon2 ) + COS( rlat1 ) * SIN( rlon1 ) * COS( rlat2 ) * SIN( rlon2 ) + SIN( rlat1 ) * SIN( rlat2 ) ) * 6372.8;
RETURN dist;
END
Buffers won't help you much in MySQL < 5.6, since buffer is a polygon, and polygon operations in MySQL < 5.6 are implemented as "Minimal Bounding Rectangles" (MBR), which are pretty useless.
Since MySQL 5.6, the full non-MBR st_* operations were implemented. But the best solution for you, in case of circle, is to use undocumented function st_distance:
select *
from waypoints
where st_distance(point(#center_lon, #center_lat), coordinates) <= radius;
It was hard to find, since it's undocumented :-) But it's mentioned on this blog, whose author also filled the mentioned bugreport. There are caveats though (citing the blog):
The bad news is:
1) All functions still only use the planar system coordinates.
Different SRIDs are not supported.
2) Spatial indexes (RTREE) are only supported for MyISAM tables. One
can use the functions for InnoDB tables, but it will not use spatial
keys.
Point 1) means that the unit of distance will be the same as the unit of coordinates (degrees in case of WGS84). If you need distance in meters, you have to use projected coordination system (e.g. UTM or similar) that has units corresponding to meters.
So, in case you don't want to go with these caveats, or in case of MySQL < 5.6, you will have to write your own custom distance function.
for the sake of completeness, as of MySQL 5.7.6. you can use the ST_Distance_Sphere function which achieves the same result:
SET #pt1 = ST_GeomFromText('POINT(12.3456 34.5678)');
SELECT * from
(SELECT * ,(ST_Distance_Sphere(#pt1, location, 6373)) AS distance FROM mydb.Event ORDER BY distance) x WHERE x.distance <= 30;
In this case, we provide the approximate radius of the Earth in kilometers (6373) and a point (#pt1). This code will calculate the distance (in kilometers) between that point (long 12.3456, lat 34.5678) and all the points contained in the database where the distance is 30km or less.
from:
https://gis.stackexchange.com/questions/31628/find-points-within-a-distance-using-mysql
SELECT
id, (
6371 * acos (
cos ( radians(78.3232) )
* cos( radians( lat ) )
* cos( radians( lng ) - radians(65.3234) )
+ sin ( radians(78.3232) )
* sin( radians( lat ) )
)
) AS distance
FROM markers
HAVING distance < 30
ORDER BY distance
LIMIT 0 , 20;
(remember to replace all constants, this example is for kilometers)
You can use:
SELECT name, lat, lng
FROM vw_mytable
WHERE ST_Contains(ST_Buffer(
ST_GeomFromText('POINT(12.3456 34.5678)'), (0.00001*1000)) , mypoint) = 1
The expression: 0.00001*1000 inside statement above give to you a circle with 1000 Meters of diameter, it's being applied on a view here, name column is just a label to point, mypoint is the name of my point column,
lat was calculated inside view with ST_X(mytable.mypoint) and
lng with ST_Y(mytable.mypoint) and they simply show me the literal values of lat and lng.
It will give to you all coordinates that belongs to circle.
Hope my version helps
SELECT
*
FROM
`locator`
WHERE
SQRT(POW(X(`center`) - 49.843317 , 2) + POW(Y(`center`) - 24.026642, 2)) * 100 < `radius`
details here http://dexxtr.com/post/83498801191/how-to-determine-point-inside-circle-using-mysql
I am trying to optimize my SQL query to display the nearest locations.
Originally I was using the following query
SELECT name,path, ( 6371 * acos( cos( radians(-36.848461) ) * cos( radians( latitude ) ) * cos( radians( longitude ) - radians(174.763336) ) + sin( radians(-36.848461) ) * sin( radians( latitude ) ) ) ) AS distance FROM cityDB HAVING distance < 200 AND path IS NOT NULL
This query uses the longitude / latitude values to calculate the distance and takes 14.5 seconds to complete (too slow!)
I created a spatial index (point) of the longitude / latitude values in an effort to speed up the query.
SELECT DISTINCT name,
(ST_Length(ST_LineStringFromWKB(
LineString(
pt,
ST_PointFromText('POINT(174.763336 -36.848461)', 4326)))))
AS distance
FROM CityDB
ORDER BY distance ASC LIMIT 99
However this is averaging 23seconds even longer than the first query which really surprised me!
Is there something else I should be doing with my query to speed it up?
The only thing that I got to get it below 1 second was to add this (thanks to this post Improving performance of spatial MySQL query)
longitude BETWEEN longpoint - (50.0 / (111.045 * COS(RADIANS(latpoint))))
AND longpoint + (50.0 / (111.045 * COS(RADIANS(latpoint))))
However there are a couple of bugs in the code. If I chose Fiji, it will only display locations < 180 latitude, If I chose Vanuatu it will only choose locations < -180 latitude.
SELECT DISTINCT name, (ST_Length
(ST_LineStringFromWKB(LineString( pt, ST_PointFromText('POINT(-179.276277 -18.378639)', 4326)))))
AS distance FROM CityDB
WHERE longitude BETWEEN -179.276277 - (50.0 / (111.045*COS(RADIANS(-18.378639)))) AND -179.276277 + (50.0 / (111.045 * COS(RADIANS(-18.378639))))
GROUP BY (name) ORDER BY distance ASC LIMIT 99
In addition to this, it will also display locations in Russia which is miles away (I know that we can add a limit to distance but there must be a better way than this)
Any other efficient ways to make such a query?
I have a query which is causing my site to run very slow. Essentially it identifies the nearest location to a longitude / latitude point from a database of around 2m records.
Currently this query takes 7seconds to complete.
I have done the following to speed it up (before it was more than 15 seconds!)
Added index keys to name / longitude / latitude / path
stored the path in the database so that it does not need to run
Stored results into another table so we do not have to run the query again.
Considered splitting the database by country, however this will cause a problem if the nearest location is in a neighboring country.
Any other ideas? Is there a way to possibly limit the longitude / latitude in the query eg + or - 2 degrees?
SELECT name,path, ( 6371 * acos( cos( radians(?) ) * cos( radians( latitude ) ) * cos( radians( longitude ) - radians(?) ) + sin( radians(?) ) * sin( radians( latitude ) ) ) ) AS distance FROM ".$GLOBALS['table']." HAVING distance < 200 AND path IS NOT NULL
Do not use latitude and longitude columns, as this way indices are useless since you need to calculate the distance metric for each record every time you query, with no ability to optimise it.
MySQL now supports geospatial data using POINT datatype and CREATE SPATIAL INDEX, which MySQL knows how to optimise.
Something like this; though MySQL 8.0 should be even better.
I am working with Google Map Fusion Tables and recently faced a tough problem while getting required data.
I am using below query:
SELECT geometry, ZIP, latitude, longitude,( 3959 * acos( cos( radians(37) ) * cos( radians( lat ) ) * cos( radians( lng ) - radians(-122) ) + sin( radians(37) ) * sin( radians( lat ) ) ) ) AS distance FROM 18n-gPzxv_usPqtFJka9ytDArJgi3Hh8tlGnfuwrN WHERE distance <= 25
But the query is returning "could not parse query" error. I also tried below query but i got same error.
SELECT geometry, ZIP, latitude, longitude FROM 18n-gPzxv_usPqtFJka9ytDArJgi3Hh8tlGnfuwrN WHERE ST_DISTANCE(LATLNG(24.547123404292083, -114.32373084375001), LATLNG(37.4,-122.1)) <= 25
I can't calculate the distance after fetching all of the records. it's like millions of records (table size as of now is 10MB). I need a solution like just as we could fetch rows from a MYSQL table using spatial function like ST_DISTANCE or using the distance formula.
If any one could help giving some alternate or some out of the box solution, it would be awesome :)
You can't use functions like acos or sin in a FusionTable-Query, the supported functions are aggregrate-functions:COUNT
SUM
AVERAGE
MAXIMUM
MINIMUM
ST_DISTANCE expects the first argument to be the name of a Location-column
ST_DISTANCE may not be used in a WHERE-clause, it's only supported in ORDER BY
summary: what you are trying to achieve is (currently) not possible with a FusionTable-Query
I have a table called flags that contains a column called coordinates that is full of MySQL 'points'. I need to perform a query where I get all the flags within a circle based on a latitude and longitude position with 100m radius.
From a usage point of view this is based around the user's position. For example, the mobile phone would give the user's latitude and longitude position and then pass it to this part of the API. It's then up to the API to create an invisible circle around the user with a radius of 100 metres and then return the flags that are in this circle.
It's this part of the API I'm not sure how to create as I'm unsure how to use SQL to create this invisible circle and select points only within this radius.
Is this possible? Is there a MySQL spatial function that will help me do this?
I believe the Buffer() function can do this but I can't find any documentation as to how to use it (eg example SQL). Ideally I need an answer that shows me how to use this function or the closest to it. Where I'm storing these coordinates as geospatial points I should be using a geospatial function to do what I'm asking to maximize efficiency.
Flags table:
id
coordinates
name
Example row:
1 | [GEOMETRY - 25B] | Tenacy AB
For the flags table I have latitude, longitude positions and easting and northing (UTM)
The user's location is just standard latitude/longitude but I have a library that can conver this position to UTM
There are no geospatial extension functions in MySQL supporting latitude / longitude distance computations. There is as of MySQL 5.7.
You're asking for proximity circles on the surface of the earth. You mention in your question that you have lat/long values for each row in your flags table, and also universal transverse Mercator (UTM) projected values in one of several different UTM zones. If I remember my UK Ordnance Survey maps correctly, UTM is useful for locating items on those maps.
It's a simple matter to compute the distance between two points in the same zone in UTM: the Cartesian distance does the trick. But, when points are in different zones, that computation doesn't work.
Accordingly, for the application described in your question, it's necessary to use the Great Circle Distance, which is computed using the haversine or another suitable formula.
MySQL, augmented with geospatial extensions, supports a way to represent various planar shapes (points, polylines, polygons, and so forth) as geometrical primitives. MySQL 5.6 implements an undocumented distance function st_distance(p1, p2). However, this function returns Cartesian distances. So it's entirely unsuitable for latitude and longitude based computations. At temperate latitudes a degree of latitude subtends almost twice as much surface distance (north-south) as a degree of longitude(east-west), because the latitude lines grow closer together nearer the poles.
So, a circular proximity formula needs to use genuine latitude and longitude.
In your application, you can find all the flags points within ten statute miles of a given latpoint,longpoint with a query like this:
SELECT id, coordinates, name, r,
units * DEGREES(ACOS(LEAST(1.0, COS(RADIANS(latpoint))
* COS(RADIANS(latitude))
* COS(RADIANS(longpoint) - RADIANS(longitude))
+ SIN(RADIANS(latpoint))
* SIN(RADIANS(latitude))))) AS distance
FROM flags
JOIN (
SELECT 42.81 AS latpoint, -70.81 AS longpoint,
10.0 AS r, 69.0 AS units
) AS p ON (1=1)
WHERE MbrContains(GeomFromText (
CONCAT('LINESTRING(',
latpoint-(r/units),' ',
longpoint-(r /(units* COS(RADIANS(latpoint)))),
',',
latpoint+(r/units) ,' ',
longpoint+(r /(units * COS(RADIANS(latpoint)))),
')')), coordinates)
If you want to search for points within 20 km, change this line of the query
20.0 AS r, 69.0 AS units
to this, for example
20.0 AS r, 111.045 AS units
r is the radius in which you want to search. units are the distance units (miles, km, furlongs, whatever you want) per degree of latitude on the surface of the earth.
This query uses a bounding lat/long along with MbrContains to exclude points that are definitely too far from your starting point, then uses the great circle distance formula to generate the distances for the remaining points. An explanation of all this can be found here. If your table uses the MyISAM access method and has a spatial index, MbrContains will exploit that index to get you fast searching.
Finally, the query above selects all the points within the rectangle. To narrow that down to only the points in the circle, and order them by proximity, wrap the query up like this:
SELECT id, coordinates, name
FROM (
/* the query above, paste it in here */
) AS d
WHERE d.distance <= d.r
ORDER BY d.distance ASC
UPDATE
Use ST_Distance_Sphere() to calculate distances using a lat/long
http://dev.mysql.com/doc/refman/5.7/en/spatial-convenience-functions.html#function_st-distance-sphere
This assumes the coordinates in the table are stored as a POINT() datatype in a column labeled 'point'. The function X(point) and Y(point) extract the latitude and longitude values from the point value respectively.
SET #lat = the latitude of the point
SET #lon = the longitude of the point
SET #rad = radius in Kilometers to search from the point
SET #table = name of your table
SELECT
X(point),Y(point),*, (
6373 * acos (
cos ( radians( #lat ) )
* cos( radians( X(point) ) )
* cos( radians( Y(point) ) - radians( #lon ) )
+ sin ( radians( #lat ) )
* sin( radians( X(point) ) )
)
) AS distance
FROM #table
HAVING distance < #rad
If you want to do it in miles, replace the constant 6373 with 3959
For those wanting to reduce the query syntax, here's a common implementation of a user defined MySQL function for implementing a distance function based on the Haversine formulae.
CREATE FUNCTION HAVERSINE ( coord1 POINT, coord2 POINT )
RETURNS DOUBLE
DETERMINISTIC
BEGIN
DECLARE dist DOUBLE;
SET rlat1 = radians( X( coord1 ) );
SET rlat2 = radians( X( coord2 ) );
SET rlon1 = radians( Y( coord1 ) );
SET rlon2 = radians( Y( coord2 ) );
SET dist = ACOS( COS( rlat1 ) * COS( rlon1 ) * COS( rlat2 ) * COS( rlon2 ) + COS( rlat1 ) * SIN( rlon1 ) * COS( rlat2 ) * SIN( rlon2 ) + SIN( rlat1 ) * SIN( rlat2 ) ) * 6372.8;
RETURN dist;
END
Buffers won't help you much in MySQL < 5.6, since buffer is a polygon, and polygon operations in MySQL < 5.6 are implemented as "Minimal Bounding Rectangles" (MBR), which are pretty useless.
Since MySQL 5.6, the full non-MBR st_* operations were implemented. But the best solution for you, in case of circle, is to use undocumented function st_distance:
select *
from waypoints
where st_distance(point(#center_lon, #center_lat), coordinates) <= radius;
It was hard to find, since it's undocumented :-) But it's mentioned on this blog, whose author also filled the mentioned bugreport. There are caveats though (citing the blog):
The bad news is:
1) All functions still only use the planar system coordinates.
Different SRIDs are not supported.
2) Spatial indexes (RTREE) are only supported for MyISAM tables. One
can use the functions for InnoDB tables, but it will not use spatial
keys.
Point 1) means that the unit of distance will be the same as the unit of coordinates (degrees in case of WGS84). If you need distance in meters, you have to use projected coordination system (e.g. UTM or similar) that has units corresponding to meters.
So, in case you don't want to go with these caveats, or in case of MySQL < 5.6, you will have to write your own custom distance function.
for the sake of completeness, as of MySQL 5.7.6. you can use the ST_Distance_Sphere function which achieves the same result:
SET #pt1 = ST_GeomFromText('POINT(12.3456 34.5678)');
SELECT * from
(SELECT * ,(ST_Distance_Sphere(#pt1, location, 6373)) AS distance FROM mydb.Event ORDER BY distance) x WHERE x.distance <= 30;
In this case, we provide the approximate radius of the Earth in kilometers (6373) and a point (#pt1). This code will calculate the distance (in kilometers) between that point (long 12.3456, lat 34.5678) and all the points contained in the database where the distance is 30km or less.
from:
https://gis.stackexchange.com/questions/31628/find-points-within-a-distance-using-mysql
SELECT
id, (
6371 * acos (
cos ( radians(78.3232) )
* cos( radians( lat ) )
* cos( radians( lng ) - radians(65.3234) )
+ sin ( radians(78.3232) )
* sin( radians( lat ) )
)
) AS distance
FROM markers
HAVING distance < 30
ORDER BY distance
LIMIT 0 , 20;
(remember to replace all constants, this example is for kilometers)
You can use:
SELECT name, lat, lng
FROM vw_mytable
WHERE ST_Contains(ST_Buffer(
ST_GeomFromText('POINT(12.3456 34.5678)'), (0.00001*1000)) , mypoint) = 1
The expression: 0.00001*1000 inside statement above give to you a circle with 1000 Meters of diameter, it's being applied on a view here, name column is just a label to point, mypoint is the name of my point column,
lat was calculated inside view with ST_X(mytable.mypoint) and
lng with ST_Y(mytable.mypoint) and they simply show me the literal values of lat and lng.
It will give to you all coordinates that belongs to circle.
Hope my version helps
SELECT
*
FROM
`locator`
WHERE
SQRT(POW(X(`center`) - 49.843317 , 2) + POW(Y(`center`) - 24.026642, 2)) * 100 < `radius`
details here http://dexxtr.com/post/83498801191/how-to-determine-point-inside-circle-using-mysql