Related
I have these two tables:
Table locations
coords
28.08982880911016,-31.649963296195086
Table places
lat
lon
place
28.08982880911016
-31.649963296195086
Test
I want a MySQL (version 5.7.41 compatible) query which can return all records in table 1; however, I do not want to display the coords value as they stored; instead, I want to use ST_Distance_Sphere or Haversine formula to check each coords with the nearest place respecting these cases:
If the nearest value is less than 300 meter it returns that corresponding place value
If the nearest value is between 300 and 1500 meters it returns: "Near " + the value of the corresponding place + "~" + the nearest value in meters + "m"
If the nearest value is more than 1500 it returns "N/A" 3.
Please note that I have around a million records in the table locations, so I need the simplest query with minimal performance impact.
Thanks in advance.
Performing the distance calculation between all locations and all places would be very expensive so we will create a bounding box, which can use available indices to get a subset of places on which to perform the distance calculations.
We are building a bounding box slightly bigger than the 1500m radius we are interested in (to allow for inaccuracy):
dist in degrees latitude = (dist / rEarth) * (180 / pi)
dist in degrees longitude = (dist / rEarth) * (180 / pi) / cos(latitude * pi/180)
dist = 1.550 km
rEarth ≈ 6371 km
dist in degrees latitude = (dist / rEarth) * (180 / pi)
dist in degrees latitude = (1.55 / 6371) * (180 / pi)
dist in degrees latitude = DEGREES(1.55/6371)
dist in degrees latitude = 0.013939
dist in degrees longitude = (dist / rEarth) * (180 / pi) / cos(latitude * pi/180)
dist in degrees longitude = 0.013939 / COS(RADIANS(latitude))
Start the join with a 3.100 km x 3.100 km bounding box centered on the location.coords
Then do point to point distance calc for the entities found:
SELECT *,
/*
* We are pre-splitting the coords here so we don't have to do
* it repeatedly inside the correlated sub-query
*/
SUBSTRING_INDEX(l.coords, ',', 1) AS loc_lat,
SUBSTRING_INDEX(l.coords, ',', -1) AS loc_lon,
(
SELECT CONCAT(
ROUND(ST_Distance_Sphere(
ST_GeomFromText(CONCAT('POINT(', REPLACE(l.coords, ',', ' '),')')),
ST_GeomFromText(CONCAT('POINT(', lat, ' ', lon,')'))
), -1),
'::',
place
) dist
FROM places
WHERE lat BETWEEN loc_lat - 0.013939 AND loc_lat + 0.013939
AND lon BETWEEN loc_lon - (0.013939 / COS(RADIANS(loc_lat))) AND loc_lon + (0.013939 / COS(RADIANS(loc_lat)))
HAVING dist <= 1500
ORDER BY dist ASC
LIMIT 1
) AS place,
(
SELECT
CASE
WHEN SUBSTRING_INDEX(place, '::', 1) < 300 THEN SUBSTRING_INDEX(place, '::', -1)
WHEN SUBSTRING_INDEX(place, '::', 1) BETWEEN 300 AND 1500 THEN CONCAT('Near ', SUBSTRING_INDEX(place, '::', -1), '~', SUBSTRING_INDEX(place, '::', 1))
ELSE 'N/A'
END
) AS place_text
FROM locations l;
This query will work but how it will perform is another matter entirely. Ideally, you should have two composite indices on places: (lat, lon) and (lon, lat) but having at least one of these is critical, otherwise you will be constantly table scanning.
I am using the gis extension of Netlogo and am trying to calculate distances between agents in km.
With gis:envelope-of I can get agent's positions in degree (!) latitude and longitude, so I want to use the Haversine formula to calculate distances. The projection of my underlying GIS-shapefile is WGS 84.
What I've written for two exemplary points is the following:
to haversine
let lata 31.8930795682288
let longa 36.0898369172224
let latb 31.9964498913126
let longb 36.0318329821497
let rad (pi / 180)
let result acos (cos (rad * lata) * cos(rad * (latb )) + sin ( rad * (lata) )* sin(rad * (latb)) * cos(rad * (longa - longb))) * 6371
print result
end
It yields a distance of 11.5 km, while the same formula applied in excel (using the RADIANS-formula instead of "rad") leaves me with 12.73 km, a number that this website http://www.movable-type.co.uk/scripts/latlong.html confirms.
Any idea what I'm doing wrong here? Thanks in advance.
The formula that computes result is not correct. It looks like you reversed sin and cos. Use
let result acos(sin(rad * lata) * sin(rad * latb) + cos(rad * lata) * cos(rad * latb) * cos(rad * (longa - longb))) * 6371
Also, according to the docs that I've read (e.g. cos and acos), the netlogo trig functions use degrees, so the calculation should be
let result rad * acos(sin(lata) * sin(latb) + cos(lata) * cos(latb) * cos(longa - longb)) * 6371
I found this formula and it works, however what i trying to do is to give ability to filter by distance from his ZIP code.
I found formula that calculates distance between two latitude and longitude coordinates.
(3956 * ACOS(COS(RADIANS(start_lat)) * COS(RADIANS(end_lat)) * COS(RADIANS(end_lon) - RADIANS(start_lon)) + SIN(RADIANS(start_lat)) * SIN(RADIANS(end_lat))))
I have filter on page that sends following info
$_POST["start_latitude"] = 34.023179;
$_POST["start_longitude"] = -118.303965;
$_POST["max_distance"] = 50;
If i do
SELECT (3956 * ACOS(COS(RADIANS({$_POST["start_latitude"]})) * COS(RADIANS(34.018626))
* COS(RADIANS(-118.249978) - RADIANS({$_POST["start_longitude"]}))
+ SIN(RADIANS({$_POST["start_latitude"]})) * SIN(RADIANS(34.018626))))
Will output distance as number of miles 4 miles in this case.
How can i convert this formula for my goal to find places no longer than say 50 miles from coordinates entered? I know all need to be done is change of places in formula, but i am not good with school math.
SELECT place_name FROM places
WHERE place_latitude = ? AND place_longitude = ?
EDIT:
I have places table where i got 1000 records in format
id place_name latitude longitude
1 xxx 432423 -43432
2 yyy 523452 -54353
3 zzz 553453 -53422
etc.
So the formula has to do something like
SELECT place_name FROM places
(CALCULATE each place distance from
$_POST["start_latitude"] and $_POST["start_longitude"]
and select only ones that) < 50
Put the distance formula into the WHERE clause:
SELECT place_name
FROM places
WHERE (3956 * ACOS(COS(RADIANS(:start_latitude)) * COS(RADIANS(latitude)) * COS(RADIANS(-longitude) - RADIANS(:start_longitude)) + SIN(RADIANS(:start_latitude)) * SIN(RADIANS(latitude))))
< :max_distance
This resouce and web service usefull, check it:
http://www.codebump.com/services/PlaceLookup.asmx
function calc_distance($point1, $point2)
{
$radius = 3958; // Earth's radius (miles)
$deg_per_rad = 57.29578; // Number of degrees/radian (for conversion)
$distance = ($radius * pi() * sqrt(
($point1['lat'] - $point2['lat'])
* ($point1['lat'] - $point2['lat'])
+ cos($point1['lat'] / $deg_per_rad) // Convert these to
* cos($point2['lat'] / $deg_per_rad) // radians for cos()
* ($point1['long'] - $point2['long'])
* ($point1['long'] - $point2['long'])
) / 180);
return $distance; // Returned using the units used for $radius.
}
EDIT
Check the page :
http://www.mssqltips.com/sqlservertip/2690/calculate-the-geographical-distance-between-two-cities-in-sql-server/
--
DECLARE #98001 GEOGRAPHY;
DECLARE #Patzip GEOGRAPHY;
SELECT #98001 = Coordinates FROM ZipCodeLKUP INNER JOIN
Facility ON ZipCodeLKUP.ZipCode = Facility.ZipCode
Where Facility.ZipCode=98001
SELECT #Patzip = Coordinates FROM ZipCodeLKUP INNER JOIN
HIDIMV_year ON ZipCodeLKUP.ZipCode = HIDIMV_year .Patzip
where PATZIP in ('98001', '98466','97202')
SELECT #98001.STDistance(#Patzip)/1000 AS [Distance in KM]
I have latitude and longitude and I want to pull the record from the database, which has nearest latitude and longitude by the distance, if that distance gets longer than specified one, then don't retrieve it.
Table structure:
id
latitude
longitude
place name
city
country
state
zip
sealevel
SELECT latitude, longitude, SQRT(
POW(69.1 * (latitude - [startlat]), 2) +
POW(69.1 * ([startlng] - longitude) * COS(latitude / 57.3), 2)) AS distance
FROM TableName HAVING distance < 25 ORDER BY distance;
where [starlat] and [startlng] is the position where to start measuring the distance.
Google's solution:
Creating the Table
When you create the MySQL table, you want to pay particular attention to the lat and lng attributes. With the current zoom capabilities of Google Maps, you should only need 6 digits of precision after the decimal. To keep the storage space required for your table at a minimum, you can specify that the lat and lng attributes are floats of size (10,6). That will let the fields store 6 digits after the decimal, plus up to 4 digits before the decimal, e.g. -123.456789 degrees. Your table should also have an id attribute to serve as the primary key.
CREATE TABLE `markers` (
`id` INT NOT NULL AUTO_INCREMENT PRIMARY KEY ,
`name` VARCHAR( 60 ) NOT NULL ,
`address` VARCHAR( 80 ) NOT NULL ,
`lat` FLOAT( 10, 6 ) NOT NULL ,
`lng` FLOAT( 10, 6 ) NOT NULL
) ENGINE = MYISAM ;
Populating the Table
After creating the table, it's time to populate it with data. The sample data provided below is for about 180 pizzarias scattered across the United States. In phpMyAdmin, you can use the IMPORT tab to import various file formats, including CSV (comma-separated values). Microsoft Excel and Google Spreadsheets both export to CSV format, so you can easily transfer data from spreadsheets to MySQL tables through exporting/importing CSV files.
INSERT INTO `markers` (`name`, `address`, `lat`, `lng`) VALUES ('Frankie Johnnie & Luigo Too','939 W El Camino Real, Mountain View, CA','37.386339','-122.085823');
INSERT INTO `markers` (`name`, `address`, `lat`, `lng`) VALUES ('Amici\'s East Coast Pizzeria','790 Castro St, Mountain View, CA','37.38714','-122.083235');
INSERT INTO `markers` (`name`, `address`, `lat`, `lng`) VALUES ('Kapp\'s Pizza Bar & Grill','191 Castro St, Mountain View, CA','37.393885','-122.078916');
INSERT INTO `markers` (`name`, `address`, `lat`, `lng`) VALUES ('Round Table Pizza: Mountain View','570 N Shoreline Blvd, Mountain View, CA','37.402653','-122.079354');
INSERT INTO `markers` (`name`, `address`, `lat`, `lng`) VALUES ('Tony & Alba\'s Pizza & Pasta','619 Escuela Ave, Mountain View, CA','37.394011','-122.095528');
INSERT INTO `markers` (`name`, `address`, `lat`, `lng`) VALUES ('Oregano\'s Wood-Fired Pizza','4546 El Camino Real, Los Altos, CA','37.401724','-122.114646');
Finding Locations with MySQL
To find locations in your markers table that are within a certain radius distance of a given latitude/longitude, you can use a SELECT statement based on the Haversine formula. The Haversine formula is used generally for computing great-circle distances between two pairs of coordinates on a sphere. An in-depth mathemetical explanation is given by Wikipedia and a good discussion of the formula as it relates to programming is on Movable Type's site.
Here's the SQL statement that will find the closest 20 locations that are within a radius of 25 miles to the 37, -122 coordinate. It calculates the distance based on the latitude/longitude of that row and the target latitude/longitude, and then asks for only rows where the distance value is less than 25, orders the whole query by distance, and limits it to 20 results. To search by kilometers instead of miles, replace 3959 with 6371.
SELECT
id,
(
3959 *
acos(cos(radians(37)) *
cos(radians(lat)) *
cos(radians(lng) -
radians(-122)) +
sin(radians(37)) *
sin(radians(lat )))
) AS distance
FROM markers
HAVING distance < 28
ORDER BY distance LIMIT 0, 20;
This one is to find latitudes and longitudes in a distance less than 28 miles.
Another one is to find them in a distance between 28 and 29 miles:
SELECT
id,
(
3959 *
acos(cos(radians(37)) *
cos(radians(lat)) *
cos(radians(lng) -
radians(-122)) +
sin(radians(37)) *
sin(radians(lat )))
) AS distance
FROM markers
HAVING distance < 29 and distance > 28
ORDER BY distance LIMIT 0, 20;
https://developers.google.com/maps/articles/phpsqlsearch_v3#creating-the-map
The original answers to the question are good, but newer versions of mysql (MySQL 5.7.6 on) support geo queries, so you can now use built in functionality rather than doing complex queries.
You can now do something like:
select *, ST_Distance_Sphere( point ('input_longitude', 'input_latitude'),
point(longitude, latitude)) * .000621371192
as `distance_in_miles`
from `TableName`
having `distance_in_miles` <= 'input_max_distance'
order by `distance_in_miles` asc
The results are returned in meters. So if you want in KM simply use .001 instead of .000621371192 (which is for miles).
MySql docs are here
Here is my full solution implemented in PHP.
This solution uses the Haversine formula as presented in http://www.scribd.com/doc/2569355/Geo-Distance-Search-with-MySQL.
It should be noted that the Haversine formula experiences weaknesses around the poles. This answer shows how to implement the vincenty Great Circle Distance formula to get around this, however I chose to just use Haversine because it's good enough for my purposes.
I'm storing latitude as DECIMAL(10,8) and longitude as DECIMAL(11,8). Hopefully this helps!
showClosest.php
<?PHP
/**
* Use the Haversine Formula to display the 100 closest matches to $origLat, $origLon
* Only search the MySQL table $tableName for matches within a 10 mile ($dist) radius.
*/
include("./assets/db/db.php"); // Include database connection function
$db = new database(); // Initiate a new MySQL connection
$tableName = "db.table";
$origLat = 42.1365;
$origLon = -71.7559;
$dist = 10; // This is the maximum distance (in miles) away from $origLat, $origLon in which to search
$query = "SELECT name, latitude, longitude, 3956 * 2 *
ASIN(SQRT( POWER(SIN(($origLat - latitude)*pi()/180/2),2)
+COS($origLat*pi()/180 )*COS(latitude*pi()/180)
*POWER(SIN(($origLon-longitude)*pi()/180/2),2)))
as distance FROM $tableName WHERE
longitude between ($origLon-$dist/cos(radians($origLat))*69)
and ($origLon+$dist/cos(radians($origLat))*69)
and latitude between ($origLat-($dist/69))
and ($origLat+($dist/69))
having distance < $dist ORDER BY distance limit 100";
$result = mysql_query($query) or die(mysql_error());
while($row = mysql_fetch_assoc($result)) {
echo $row['name']." > ".$row['distance']."<BR>";
}
mysql_close($db);
?>
./assets/db/db.php
<?PHP
/**
* Class to initiate a new MySQL connection based on $dbInfo settings found in dbSettings.php
*
* #example $db = new database(); // Initiate a new database connection
* #example mysql_close($db); // close the connection
*/
class database{
protected $databaseLink;
function __construct(){
include "dbSettings.php";
$this->database = $dbInfo['host'];
$this->mysql_user = $dbInfo['user'];
$this->mysql_pass = $dbInfo['pass'];
$this->openConnection();
return $this->get_link();
}
function openConnection(){
$this->databaseLink = mysql_connect($this->database, $this->mysql_user, $this->mysql_pass);
}
function get_link(){
return $this->databaseLink;
}
}
?>
./assets/db/dbSettings.php
<?php
$dbInfo = array(
'host' => "localhost",
'user' => "root",
'pass' => "password"
);
?>
It may be possible to increase performance by using a MySQL stored procedure as suggested by the "Geo-Distance-Search-with-MySQL" article posted above.
I have a database of ~17,000 places and the query execution time is 0.054 seconds.
Just in case you are lazy like me, here's a solution amalgamated from this and other answers on SO.
set #orig_lat=37.46;
set #orig_long=-122.25;
set #bounding_distance=1;
SELECT
*
,((ACOS(SIN(#orig_lat * PI() / 180) * SIN(`lat` * PI() / 180) + COS(#orig_lat * PI() / 180) * COS(`lat` * PI() / 180) * COS((#orig_long - `long`) * PI() / 180)) * 180 / PI()) * 60 * 1.1515) AS `distance`
FROM `cities`
WHERE
(
`lat` BETWEEN (#orig_lat - #bounding_distance) AND (#orig_lat + #bounding_distance)
AND `long` BETWEEN (#orig_long - #bounding_distance) AND (#orig_long + #bounding_distance)
)
ORDER BY `distance` ASC
limit 25;
Easy one ;)
SELECT * FROM `WAYPOINTS` W ORDER BY
ABS(ABS(W.`LATITUDE`-53.63) +
ABS(W.`LONGITUDE`-9.9)) ASC LIMIT 30;
Just replace the coordinates with your required ones. The values have to be stored as double. This ist a working MySQL 5.x example.
Cheers
Try this, it show the nearest points to provided coordinates (within 50 km). It works perfectly:
SELECT m.name,
m.lat, m.lon,
p.distance_unit
* DEGREES(ACOS(COS(RADIANS(p.latpoint))
* COS(RADIANS(m.lat))
* COS(RADIANS(p.longpoint) - RADIANS(m.lon))
+ SIN(RADIANS(p.latpoint))
* SIN(RADIANS(m.lat)))) AS distance_in_km
FROM <table_name> AS m
JOIN (
SELECT <userLat> AS latpoint, <userLon> AS longpoint,
50.0 AS radius, 111.045 AS distance_unit
) AS p ON 1=1
WHERE m.lat
BETWEEN p.latpoint - (p.radius / p.distance_unit)
AND p.latpoint + (p.radius / p.distance_unit)
AND m.lon BETWEEN p.longpoint - (p.radius / (p.distance_unit * COS(RADIANS(p.latpoint))))
AND p.longpoint + (p.radius / (p.distance_unit * COS(RADIANS(p.latpoint))))
ORDER BY distance_in_km
Just change <table_name>. <userLat> and <userLon>
You can read more about this solution here: http://www.plumislandmedia.net/mysql/haversine-mysql-nearest-loc/
You're looking for things like the haversine formula. See here as well.
There's other ones but this is the most commonly cited.
If you're looking for something even more robust, you might want to look at your databases GIS capabilities. They're capable of some cool things like telling you whether a point (City) appears within a given polygon (Region, Country, Continent).
Check this code based on the article Geo-Distance-Search-with-MySQL:
Example: find the 10 nearest hotels to my current location in a 10 miles radius:
#Please notice that (lat,lng) values mustn't be negatives to perform all calculations
set #my_lat=34.6087674878572;
set #my_lng=58.3783670308302;
set #dist=10; #10 miles radius
SELECT dest.id, dest.lat, dest.lng, 3956 * 2 * ASIN(SQRT(POWER(SIN((#my_lat -abs(dest.lat)) * pi()/180 / 2),2) + COS(#my_lat * pi()/180 ) * COS(abs(dest.lat) * pi()/180) * POWER(SIN((#my_lng - abs(dest.lng)) * pi()/180 / 2), 2))
) as distance
FROM hotel as dest
having distance < #dist
ORDER BY distance limit 10;
#Also notice that distance are expressed in terms of radius.
Find nearest Users to my:
Distance in meters
Based in Vincenty's formula
i have User table:
+----+-----------------------+---------+--------------+---------------+
| id | email | name | location_lat | location_long |
+----+-----------------------+---------+--------------+---------------+
| 13 | xxxxxx#xxxxxxxxxx.com | Isaac | 17.2675625 | -97.6802361 |
| 14 | xxxx#xxxxxxx.com.mx | Monse | 19.392702 | -99.172596 |
+----+-----------------------+---------+--------------+---------------+
sql:
-- my location: lat 19.391124 -99.165660
SELECT
(ATAN(
SQRT(
POW(COS(RADIANS(users.location_lat)) * SIN(RADIANS(users.location_long) - RADIANS(-99.165660)), 2) +
POW(COS(RADIANS(19.391124)) * SIN(RADIANS(users.location_lat)) -
SIN(RADIANS(19.391124)) * cos(RADIANS(users.location_lat)) * cos(RADIANS(users.location_long) - RADIANS(-99.165660)), 2)
)
,
SIN(RADIANS(19.391124)) *
SIN(RADIANS(users.location_lat)) +
COS(RADIANS(19.391124)) *
COS(RADIANS(users.location_lat)) *
COS(RADIANS(users.location_long) - RADIANS(-99.165660))
) * 6371000) as distance,
users.id
FROM users
ORDER BY distance ASC
radius of the earth : 6371000 ( in meters)
simpledb.execSQL("CREATE TABLE IF NOT EXISTS " + tablename + "(id INTEGER PRIMARY KEY AUTOINCREMENT,lat double,lng double,address varchar)");
simpledb.execSQL("insert into '" + tablename + "'(lat,lng,address)values('22.2891001','70.780154','craftbox');");
simpledb.execSQL("insert into '" + tablename + "'(lat,lng,address)values('22.2901396','70.7782428','kotecha');");//22.2904718 //70.7783906
simpledb.execSQL("insert into '" + tablename + "'(lat,lng,address)values('22.2863155','70.772108','kkv Hall');");
simpledb.execSQL("insert into '" + tablename + "'(lat,lng,address)values('22.275993','70.778076','nana mava');");
simpledb.execSQL("insert into '" + tablename + "'(lat,lng,address)values('22.2667148','70.7609386','Govani boys hostal');");
double curentlat=22.2667258; //22.2677258
double curentlong=70.76096826;//70.76096826
double curentlat1=curentlat+0.0010000;
double curentlat2=curentlat-0.0010000;
double curentlong1=curentlong+0.0010000;
double curentlong2=curentlong-0.0010000;
try{
Cursor c=simpledb.rawQuery("select * from '"+tablename+"' where (lat BETWEEN '"+curentlat2+"' and '"+curentlat1+"') or (lng BETWEEN '"+curentlong2+"' and '"+curentlong1+"')",null);
Log.d("SQL ", c.toString());
if(c.getCount()>0)
{
while (c.moveToNext())
{
double d=c.getDouble(1);
double d1=c.getDouble(2);
}
}
}
catch (Exception e)
{
e.printStackTrace();
}
It sounds like you want to do a nearest neighbour search with some bound on the distance. SQL does not support anything like this as far as I am aware and you would need to use an alternative data structure such as an R-tree or kd-tree.
MS SQL Edition here:
DECLARE #SLAT AS FLOAT
DECLARE #SLON AS FLOAT
SET #SLAT = 38.150785
SET #SLON = 27.360249
SELECT TOP 10 [LATITUDE], [LONGITUDE], SQRT(
POWER(69.1 * ([LATITUDE] - #SLAT), 2) +
POWER(69.1 * (#SLON - [LONGITUDE]) * COS([LATITUDE] / 57.3), 2)) AS distance
FROM [TABLE] ORDER BY 3
Sounds like you should just use PostGIS, SpatialLite, SQLServer2008, or Oracle Spatial. They can all answer this question for you with spatial SQL.
+----+-----------------------+---------+--------------+---------------+
| id | email | name | location_lat | location_long |
+----+-----------------------+---------+--------------+---------------+
| 7 | test#gmail.com | rembo | 23.0249256 | 72.5269697 |
| 25 | test1#gmail.com. | Rajnis | 23.0233221 | 72.5342112 |
+----+-----------------------+---------+--------------+---------------+
$lat = 23.02350629;
$long = 72.53230239;
DB::
SELECT
("
SELECT
*
FROM
(
SELECT
,
(
( ( acos( sin(( ". $ lat ." * pi() / 180)) * sin(( lat * pi() / 180)) + cos(( ". $ lat ." pi() / 180 )) * cos(( lat * pi() / 180)) * cos((( ". $ long ." - LONG) * pi() / 180))) ) * 180 / pi() ) * 60 * 1.1515 * 1.609344
)
as distance
FROM
users
)
users
WHERE
distance <= 2");
In extreme cases this approach fails, but for performance, I've skipped the trigonometry and simply calculated the diagonal squared.
Mysql query for search coordinates with distance limit and where condition
SELECT id, ( 3959 * acos( cos( radians('28.5850154') ) * cos( radians(latitude) ) * cos( radians( longitude ) - radians('77.07207489999999') ) + sin( radians('28.5850154') ) * sin( radians( latitude ) ) ) ) AS distance FROM `vendors` HAVING distance < 5;
This problem is not very hard at all, but it gets more complicated if you need to optimize it.
What I mean is, do you have 100 locations in your database or 100 million? It makes a big difference.
If the number of locations is small, get them out of SQL and into code by just doing ->
Select * from Location
Once you get them into code, calculate the distance between each lat/lon and your original with the Haversine formula and sort it.
I currently have just under a million locations in a mysql database all with longitude and latitude information.
I am trying to find the distance between one point and many other points via a query. It's not as fast as I want it to be especially with 100+ hits a second.
Is there a faster query or possibly a faster system other than mysql for this? I'm using this query:
SELECT
name,
( 3959 * acos( cos( radians(42.290763) ) * cos( radians( locations.lat ) )
* cos( radians(locations.lng) - radians(-71.35368)) + sin(radians(42.290763))
* sin( radians(locations.lat)))) AS distance
FROM locations
WHERE active = 1
HAVING distance < 10
ORDER BY distance;
Note: The provided distance is in Miles. If you need Kilometers, use 6371 instead of 3959.
Create your points using Point values of Geometry data types in MyISAM table. As of Mysql 5.7.5, InnoDB tables now also support SPATIAL indices.
Create a SPATIAL index on these points
Use MBRContains() to find the values:
SELECT *
FROM table
WHERE MBRContains(LineFromText(CONCAT(
'('
, #lon + 10 / ( 111.1 / cos(RADIANS(#lat)))
, ' '
, #lat + 10 / 111.1
, ','
, #lon - 10 / ( 111.1 / cos(RADIANS(#lat)))
, ' '
, #lat - 10 / 111.1
, ')' )
,mypoint)
, or, in MySQL 5.1 and above:
SELECT *
FROM table
WHERE MBRContains
(
LineString
(
Point (
#lon + 10 / ( 111.1 / COS(RADIANS(#lat))),
#lat + 10 / 111.1
),
Point (
#lon - 10 / ( 111.1 / COS(RADIANS(#lat))),
#lat - 10 / 111.1
)
),
mypoint
)
This will select all points approximately within the box (#lat +/- 10 km, #lon +/- 10km).
This actually is not a box, but a spherical rectangle: latitude and longitude bound segment of the sphere. This may differ from a plain rectangle on the Franz Joseph Land, but quite close to it on most inhabited places.
Apply additional filtering to select everything inside the circle (not the square)
Possibly apply additional fine filtering to account for the big circle distance (for large distances)
Not a MySql specific answer, but it'll improve the performance of your sql statement.
What you're effectively doing is calculating the distance to every point in the table, to see if it's within 10 units of a given point.
What you can do before you run this sql, is create four points that draw a box 20 units on a side, with your point in the center i.e.. (x1,y1 ) . . . (x4, y4), where (x1,y1) is (givenlong + 10 units, givenLat + 10units) . . . (givenLong - 10units, givenLat -10 units).
Actually, you only need two points, top left and bottom right call them (X1, Y1) and (X2, Y2)
Now your SQL statement use these points to exclude rows that definitely are more than 10u from your given point, it can use indexes on the latitudes & longitudes, so will be orders of magnitude faster than what you currently have.
e.g.
select . . .
where locations.lat between X1 and X2
and locations.Long between y1 and y2;
The box approach can return false positives (you can pick up points in the corners of the box that are > 10u from the given point), so you still need to calculate the distance of each point. However this again will be much faster because you have drastically limited the number of points to test to the points within the box.
I call this technique "Thinking inside the box" :)
EDIT: Can this be put into one SQL statement?
I have no idea what mySql or Php is capable of, sorry.
I don't know where the best place is to build the four points, or how they could be passed to a mySql query in Php. However, once you have the four points, there's nothing stopping you combining your own SQL statement with mine.
select name,
( 3959 * acos( cos( radians(42.290763) )
* cos( radians( locations.lat ) )
* cos( radians( locations.lng ) - radians(-71.35368) )
+ sin( radians(42.290763) )
* sin( radians( locations.lat ) ) ) ) AS distance
from locations
where active = 1
and locations.lat between X1 and X2
and locations.Long between y1 and y2
having distance < 10 ORDER BY distance;
I know with MS SQL I can build a SQL statement that declares four floats (X1, Y1, X2, Y2) and calculates them before the "main" select statement, like I said, I've no idea if this can be done with MySql. However I'd still be inclined to build the four points in C# and pass them as parameters to the SQL query.
Sorry I can't be more help, if anyone can answer the MySQL & Php specific portions of this, feel free to edit this answer to do so.
I needed to solve similar problem (filtering rows by distance from single point) and by combining original question with answers and comments, I came up with solution which perfectly works for me on both MySQL 5.6 and 5.7.
SELECT
*,
(6371 * ACOS(COS(RADIANS(56.946285)) * COS(RADIANS(Y(coordinates)))
* COS(RADIANS(X(coordinates)) - RADIANS(24.105078)) + SIN(RADIANS(56.946285))
* SIN(RADIANS(Y(coordinates))))) AS distance
FROM places
WHERE MBRContains
(
LineString
(
Point (
24.105078 + 15 / (111.320 * COS(RADIANS(56.946285))),
56.946285 + 15 / 111.133
),
Point (
24.105078 - 15 / (111.320 * COS(RADIANS(56.946285))),
56.946285 - 15 / 111.133
)
),
coordinates
)
HAVING distance < 15
ORDER By distance
coordinates is field with type POINT and has SPATIAL index
6371 is for calculating distance in kilometres
56.946285 is latitude for central point
24.105078 is longitude for central point
15 is maximum distance in kilometers
In my tests, MySQL uses SPATIAL index on coordinates field to quickly select all rows which are within rectangle and then calculates actual distance for all filtered places to exclude places from rectangles corners and leave only places inside circle.
This is visualisation of my result:
Gray stars visualise all points on map, yellow stars are ones returned by MySQL query. Gray stars inside corners of rectangle (but outside circle) were selected by MBRContains() and then deselected by HAVING clause.
The following MySQL function was posted on this blog post. I haven't tested it much, but from what I gathered from the post, if your latitude and longitude fields are indexed, this may work well for you:
DELIMITER $$
DROP FUNCTION IF EXISTS `get_distance_in_miles_between_geo_locations` $$
CREATE FUNCTION get_distance_in_miles_between_geo_locations(
geo1_latitude decimal(10,6), geo1_longitude decimal(10,6),
geo2_latitude decimal(10,6), geo2_longitude decimal(10,6))
returns decimal(10,3) DETERMINISTIC
BEGIN
return ((ACOS(SIN(geo1_latitude * PI() / 180) * SIN(geo2_latitude * PI() / 180)
+ COS(geo1_latitude * PI() / 180) * COS(geo2_latitude * PI() / 180)
* COS((geo1_longitude - geo2_longitude) * PI() / 180)) * 180 / PI())
* 60 * 1.1515);
END $$
DELIMITER ;
Sample usage:
Assuming a table called places with fields latitude & longitude:
SELECT get_distance_in_miles_between_geo_locations(-34.017330, 22.809500,
latitude, longitude) AS distance_from_input FROM places;
if you are using MySQL 5.7.*, then you can use st_distance_sphere(POINT, POINT).
Select st_distance_sphere(POINT(-2.997065, 53.404146 ), POINT(58.615349, 23.56676 ))/1000 as distcance
SELECT * FROM (SELECT *,(((acos(sin((43.6980168*pi()/180)) *
sin((latitude*pi()/180))+cos((43.6980168*pi()/180)) *
cos((latitude*pi()/180)) * cos(((7.266903899999988- longitude)*
pi()/180))))*180/pi())*60*1.1515 ) as distance
FROM wp_users WHERE 1 GROUP BY ID limit 0,10) as X
ORDER BY ID DESC
This is the distance calculation query between to points in MySQL, I have used it in a long database, it it working perfect! Note: do the changes (database name, table name, column etc) as per your requirements.
set #latitude=53.754842;
set #longitude=-2.708077;
set #radius=20;
set #lng_min = #longitude - #radius/abs(cos(radians(#latitude))*69);
set #lng_max = #longitude + #radius/abs(cos(radians(#latitude))*69);
set #lat_min = #latitude - (#radius/69);
set #lat_max = #latitude + (#radius/69);
SELECT * FROM postcode
WHERE (longitude BETWEEN #lng_min AND #lng_max)
AND (latitude BETWEEN #lat_min and #lat_max);
source
select
(((acos(sin(('$latitude'*pi()/180)) * sin((`lat`*pi()/180))+cos(('$latitude'*pi()/180))
* cos((`lat`*pi()/180)) * cos((('$longitude'- `lng`)*pi()/180))))*180/pi())*60*1.1515)
AS distance
from table having distance<22;
A MySQL function which returns the number of metres between the two coordinates:
CREATE FUNCTION DISTANCE_BETWEEN (lat1 DOUBLE, lon1 DOUBLE, lat2 DOUBLE, lon2 DOUBLE)
RETURNS DOUBLE DETERMINISTIC
RETURN ACOS( SIN(lat1*PI()/180)*SIN(lat2*PI()/180) + COS(lat1*PI()/180)*COS(lat2*PI()/180)*COS(lon2*PI()/180-lon1*PI()/180) ) * 6371000
To return the value in a different format, replace the 6371000 in the function with the radius of Earth in your choice of unit. For example, kilometres would be 6371 and miles would be 3959.
To use the function, just call it as you would any other function in MySQL. For example, if you had a table city, you could find the distance between every city to every other city:
SELECT
`city1`.`name`,
`city2`.`name`,
ROUND(DISTANCE_BETWEEN(`city1`.`latitude`, `city1`.`longitude`, `city2`.`latitude`, `city2`.`longitude`)) AS `distance`
FROM
`city` AS `city1`
JOIN
`city` AS `city2`
The full code with details about how to install as MySQL plugin are here: https://github.com/lucasepe/lib_mysqludf_haversine
I posted this last year as comment. Since kindly #TylerCollier suggested me to post as answer, here it is.
Another way is to write a custom UDF function that returns the haversine distance from two points. This function can take in input:
lat1 (real), lng1 (real), lat2 (real), lng2 (real), type (string - optinal - 'km', 'ft', 'mi')
So we can write something like this:
SELECT id, name FROM MY_PLACES WHERE haversine_distance(lat1, lng1, lat2, lng2) < 40;
to fetch all records with a distance less then 40 kilometers. Or:
SELECT id, name FROM MY_PLACES WHERE haversine_distance(lat1, lng1, lat2, lng2, 'ft') < 25;
to fetch all records with a distance less then 25 feet.
The core function is:
double
haversine_distance( UDF_INIT* initid, UDF_ARGS* args, char* is_null, char *error ) {
double result = *(double*) initid->ptr;
/*Earth Radius in Kilometers.*/
double R = 6372.797560856;
double DEG_TO_RAD = M_PI/180.0;
double RAD_TO_DEG = 180.0/M_PI;
double lat1 = *(double*) args->args[0];
double lon1 = *(double*) args->args[1];
double lat2 = *(double*) args->args[2];
double lon2 = *(double*) args->args[3];
double dlon = (lon2 - lon1) * DEG_TO_RAD;
double dlat = (lat2 - lat1) * DEG_TO_RAD;
double a = pow(sin(dlat * 0.5),2) +
cos(lat1*DEG_TO_RAD) * cos(lat2*DEG_TO_RAD) * pow(sin(dlon * 0.5),2);
double c = 2.0 * atan2(sqrt(a), sqrt(1-a));
result = ( R * c );
/*
* If we have a 5th distance type argument...
*/
if (args->arg_count == 5) {
str_to_lowercase(args->args[4]);
if (strcmp(args->args[4], "ft") == 0) result *= 3280.8399;
if (strcmp(args->args[4], "mi") == 0) result *= 0.621371192;
}
return result;
}
A fast, simple and accurate (for smaller distances) approximation can be done with a spherical projection. At least in my routing algorithm I get a 20% boost compared to the correct calculation. In Java code it looks like:
public double approxDistKm(double fromLat, double fromLon, double toLat, double toLon) {
double dLat = Math.toRadians(toLat - fromLat);
double dLon = Math.toRadians(toLon - fromLon);
double tmp = Math.cos(Math.toRadians((fromLat + toLat) / 2)) * dLon;
double d = dLat * dLat + tmp * tmp;
return R * Math.sqrt(d);
}
Not sure about MySQL (sorry!).
Be sure you know about the limitation (the third param of assertEquals means the accuracy in kilometers):
float lat = 24.235f;
float lon = 47.234f;
CalcDistance dist = new CalcDistance();
double res = 15.051;
assertEquals(res, dist.calcDistKm(lat, lon, lat - 0.1, lon + 0.1), 1e-3);
assertEquals(res, dist.approxDistKm(lat, lon, lat - 0.1, lon + 0.1), 1e-3);
res = 150.748;
assertEquals(res, dist.calcDistKm(lat, lon, lat - 1, lon + 1), 1e-3);
assertEquals(res, dist.approxDistKm(lat, lon, lat - 1, lon + 1), 1e-2);
res = 1527.919;
assertEquals(res, dist.calcDistKm(lat, lon, lat - 10, lon + 10), 1e-3);
assertEquals(res, dist.approxDistKm(lat, lon, lat - 10, lon + 10), 10);
Here is a very detailed description of Geo Distance Search with MySQL a solution based on implementation of Haversine Formula to mysql. The complete solution description with theory, implementation and further performance optimization. Although the spatial optimization part didn't work correct in my case.
http://www.scribd.com/doc/2569355/Geo-Distance-Search-with-MySQL
Have a read of Geo Distance Search with MySQL, a solution
based on implementation of Haversine Formula to MySQL. This is a complete solution
description with theory, implementation and further performance optimization.
Although the spatial optimization part didn't work correctly in my case.
I noticed two mistakes in this:
the use of abs in the select statement on p8. I just omitted abs and it worked.
the spatial search distance function on p27 does not convert to radians or multiply longitude by cos(latitude), unless his spatial data is loaded with this in consideration (cannot tell from context of article), but his example on p26 indicates that his spatial data POINT is not loaded with radians or degrees.
$objectQuery = "SELECT table_master.*, ((acos(sin((" . $latitude . "*pi()/180)) * sin((`latitude`*pi()/180))+cos((" . $latitude . "*pi()/180)) * cos((`latitude`*pi()/180)) * cos(((" . $longitude . "- `longtude`)* pi()/180))))*180/pi())*60*1.1515 as distance FROM `table_post_broadcasts` JOIN table_master ON table_post_broadcasts.master_id = table_master.id WHERE table_master.type_of_post ='type' HAVING distance <='" . $Radius . "' ORDER BY distance asc";
Using mysql
SET #orig_lon = 1.027125;
SET #dest_lon = 1.027125;
SET #orig_lat = 2.398441;
SET #dest_lat = 2.398441;
SET #kmormiles = 6371;-- for distance in miles set to : 3956
SELECT #kmormiles * ACOS(LEAST(COS(RADIANS(#orig_lat)) *
COS(RADIANS(#dest_lat)) * COS(RADIANS(#orig_lon - #dest_lon)) +
SIN(RADIANS(#orig_lat)) * SIN(RADIANS(#dest_lat)),1.0)) as distance;
See: https://andrew.hedges.name/experiments/haversine/
See: https://stackoverflow.com/a/24372831/5155484
See: http://www.plumislandmedia.net/mysql/haversine-mysql-nearest-loc/
NOTE: LEAST is used to avoid null values as a comment suggested on https://stackoverflow.com/a/24372831/5155484
I really liked #Māris Kiseļovs solution, but I like many others may have the Lat and lng's POINTS reversed from his example. In generalising it I though I would share it. In my case I need to find all the start_points that are within a certain radius of an end_point.
I hope this helps someone.
SELECT #LAT := ST_X(end_point), #LNG := ST_Y(end_point) FROM routes WHERE route_ID = 280;
SELECT
*,
(6371e3 * ACOS(COS(RADIANS(#LAT)) * COS(RADIANS(ST_X(start_point)))
* COS(RADIANS(ST_Y(start_point)) - RADIANS(#LNG)) + SIN(RADIANS(#LAT))
* SIN(RADIANS(ST_X(start_point))))) AS distance
FROM routes
WHERE MBRContains
(
LineString
(
Point (
#LNG + 15 / (111.320 * COS(RADIANS(#LAT))),
#LAT + 15 / 111.133
),
Point (
#LNG - 15 / (111.320 * COS(RADIANS(#LAT))),
#LAT - 15 / 111.133
)
),
POINT(ST_Y(end_point),ST_X(end_point))
)
HAVING distance < 100
ORDER By distance;