I have a MySQL Routine that is getting records within a 50 mile radius when passed Latitude and Longitude via utilizing the Haversin equation.
While this works great, and is pretty speedy (considering it's searching through 82k records), I am thinking that I can get better performance by creating a similar procedure utilizing a POINT column.
So, in my table I created an extra column called Location, gave it a datatype of POINT, updated my data to pass lat & lon to the Location column. Data is valid, and is fine., and added a Spatial Index
The question is, how can I convert the following query to use the Location column, instead of lat and lon columns.
SET #LAT := '37.953';
SET #LON := '-105.688';
SELECT DISTINCT
BPZ.`store_id`,
3956 * 2 * ASIN(SQRT(POWER(SIN((#LAT - abs(Z.`lat`)) * pi()/180 / 2),2) + COS(#LAT * pi()/180 ) * COS(abs(Z.`lat`) * pi()/180) * POWER(SIN((#LON - Z.`lon`) * pi()/180 / 2), 2))) as distance,
c.`name`,c.`address`,c.`city`,c.`state`,c.`phone`,c.`zip`,c.`premise_type`
FROM
`zip_codes` as Z,
`brand_product_zip` as BPZ
LEFT JOIN `customers` c ON c.`store_id` = BPZ.`store_id`
WHERE
BPZ.`zip` = Z.`zip`
AND
3956 * 2 * ASIN(SQRT(POWER(SIN((#LAT - abs(Z.`lat`)) * pi()/180 / 2),2) + COS(#LAT * pi()/180 ) * COS(abs(Z.`lat`) * pi()/180) * POWER(SIN((#LON - Z.`lon`) * pi()/180 / 2), 2))) <= 50
ORDER BY
distance LIMIT 20
I understand that this has been asked before, however, everything I see points to calculations based on lat and lon and not the POINT column
Updated Code:
SET #lat = 41.92;
SET #lon = -72.65;
SET #kmRange = 80.4672; -- = 50 Miles
SELECT *, (3956 * 2 * ASIN(SQRT(POWER(SIN((#lat - abs(`lat`)) * pi()/180 / 2),2) + COS(#lat * pi()/180 ) * COS(abs(`lat`) * pi()/180) * POWER(SIN((lon - `lon`) * pi()/180 / 2), 2)))) as distance
FROM `zip_codes`
WHERE MBRContains(LineString(Point(#lat + #kmRange / 111.1, #lon + #kmRange / (111.1 / COS(RADIANS(#lat)))), Point(#lat - #kmRange / 111.1, #lon - #kmRange / (111.1 / COS(RADIANS(#lat))))), `Location`)
Order By distance
LIMIT 20
Have you looked into hilbert curves solutions? A spatial index doesn't deliver the exact solution? . With a mysql spatial index you can use mbrcontains:
CREATE TABLE lastcrawl (id INT NOT NULL PRIMARY KEY, pnt POINT NOT NULL) ENGINE=MyISAM;
INSERT
INTO lastcrawl
VALUES (1, POINT(40, -100));
SET #lat = 40;
SET #lon = -100;
SELECT *
FROM lastcrawl
WHERE MBRContains
(
LineString
(
Point
(
#lat + 10 / 111.1,
#lon + 10 / ( 111.1 / COS(RADIANS(#lat)))
),
Point (
#lat - 10 / 111.1,
#lon - 10 / ( 111.1 / COS(RADIANS(#lat)))
)
),
pnt
);
Look here: MySQL - selecting near a spatial point.
Here: http://www.drdobbs.com/database/space-filling-curves-in-geospatial-appli/184410998
The article Nearest-location finder for MySQL explains in detail various options, and the best choice for use with the Spatial Extensions starting with MySQL 5.6.
From the article, this sample query lists zip codes within a 50 mile radius from given coordinates (42.81, -70.81):
SELECT zip, primary_city,
latitude, longitude, distance_in_mi
FROM (
SELECT zip, primary_city, latitude, longitude,r,
69.0 * DEGREES(ACOS(COS(RADIANS(latpoint))
* COS(RADIANS(latitude))
* COS(RADIANS(longpoint) - RADIANS(longitude))
+ SIN(RADIANS(latpoint))
* SIN(RADIANS(latitude)))) AS distance_in_mi
FROM zip
JOIN (
SELECT 42.81 AS latpoint, -70.81 AS longpoint, 50.0 AS r
) AS p
WHERE latitude
BETWEEN latpoint - (r / 69)
AND latpoint + (r / 69)
AND longitude
BETWEEN longpoint - (r / (69 * COS(RADIANS(latpoint))))
AND longpoint + (r / (69 * COS(RADIANS(latpoint))))
) d
WHERE distance_in_mi <= r
ORDER BY distance_in_mi;
Related
I really wanna know why place B is more far than place C to place A, when i use Haversine formula, but when i see on the map the place C is more distant, i will show some data:
Place A - lat: -23.598593 lon:-46.644701
Place B - lat: -23.575966 lon:-46.619133 to A distance = 3.6117730949107836
Place C - lat: -23.594992 lon:-46.636688 to A distance = 0.9068309093558656
I'm using this query :
QUERY A to B
set #latdest = -23.575966;
set #londest = -46.619133;
set #lat = -23.598593;
set #lon = -46.644701;
select 6353 * 2 * ASIN(SQRT(POWER(SIN((#lat - #latdest) * pi()/180 / 2), 2)
+ COS(#lat * pi()/180 ) * COS(#latdest * pi()/180)
* POWER(SIN((#lon - #londest) * pi()/180 / 2), 2) )) as distance
QUERY A to C
set #latdest = -23.594992;
set #londest = -46.636688;
set #lat = -23.598593;
set #lon = -46.644701;
select 6353 * 2 * ASIN(SQRT(POWER(SIN((#lat - #latdest) * pi()/180 / 2), 2)
+ COS(#lat * pi()/180 ) * COS(#latdest * pi()/180)
* POWER(SIN((#lon - #londest) * pi()/180 / 2), 2) )) as distance
this is Lat and Lon example in mysql database
ID | Lat | Lon
1 | 3.1412 | 101.6865
2 | 1.2897 | 103.8501
The formula below calculates the distance between two Lat (latitudes) and Lon (longitudes) and gets the result that I want. But this formula doesn't extract my Lat and Lon from the database. I would like to ask, how to get the distance result by using ID and calculate two Lat and Lon? My friend told me using Join table, but I still don't get the answer..
SELECT ROUND(6353 * 2 * ASIN(SQRT( POWER(SIN((3.1412 -
abs(1.2897)) * pi()/180 / 2),2) + COS(3.1412 * pi()/180 ) * COS(
abs(1.2897) * pi()/180) * POWER(SIN((101.6865 - 103.8501) * pi()/180 / 2), 2) )), 2);
Thanks for advance
You can join the same table two time one for Id 1 and one for id 2 assigning different alias name
then use the related column name
SELECT ROUND(6353 * 2 * ASIN(SQRT( POWER(SIN((a.lat -
abs(b.lat)) * pi()/180 / 2),2) + COS(a.lat * pi()/180 ) * COS(
abs(b.lat) * pi()/180) * POWER(SIN((a.lng - b.lng) * pi()/180 / 2), 2) )), 2);
from my_table a
inner join my_table b on a.id = 1 and b.id = 2
It is working for me
SELECT *,(((acos(sin((12.942999*pi()/180)) * sin((`Lat`*pi()/180))+cos((12.942999*pi()/180)) * cos((`Lat`*pi()/180)) * cos(((74.82994- `Lon`)* pi()/180))))*180/pi())*60*1.1515*1.609344 ) as distance FROM ven_vendor HAVING distance <= 10
I have a query that selects all locations within a 30 mile radius of a GPS pt. I want to get the id's of those locations, but it's also returning the distance from the center pt.
Is there anyway to perform the distance calculation without returning it?
Query:
SELECT id, 3956 * 2 * ASIN(SQRT(
POWER(SIN((34.1844709 - abs(dest.lat)) * pi()/180 / 2),
2) + COS(37.7749295 * pi()/180 ) * COS(abs(dest.lat) *
pi()/180) * POWER(SIN((-118.131809 - dest.lng) *
pi()/180 / 2), 2) )) as distance
FROM location dest
having distance < 30
ORDER by distance
LIMIT 30
Output:
---------------------------
id | distance
---------------------------
1 | 2.310
2 | 2.356
17 | 4.298
Query based off:
http://www.notaires.fr/sites/default/files/geo_searchjkkjkj_0.pdf
Can you just do another select on this?
Select id
From (SELECT id, 3956 * 2 * ASIN(SQRT(
POWER(SIN((34.1844709 - abs(dest.lat)) * pi()/180 / 2),
2) + COS(37.7749295 * pi()/180 ) * COS(abs(dest.lat) *
pi()/180) * POWER(SIN((-118.131809 - dest.lng) *
pi()/180 / 2), 2) )) as distance
FROM location dest
having distance < 30
ORDER by distance
LIMIT 30) dst
I'm using a slightly-modified version of the geocoded gem which returns this query when I call near on my model (calling Deal.near(southwest), where southwest is an array of geo coordinates):
SELECT
deals.*,
3958.755864232 * 2 * ASIN(SQRT(POWER(SIN((37.772476604436974 - addresses.lat) * PI() / 180 / 2), 2) + COS(37.772476604436974 * PI() / 180) * COS(addresses.lat * PI() / 180) * POWER(SIN((-122.42336332798004 - addresses.lng) * PI() / 180 / 2), 2) )) AS distance,
CAST(DEGREES(ATAN2( RADIANS(addresses.lng - -122.42336332798004), RADIANS(addresses.lat - 37.772476604436974))) + 360 AS decimal) % 360 AS bearing
FROM "deals"
INNER JOIN "companies" ON "companies"."id" = "deals"."company_id"
INNER JOIN "addresses" ON "addresses"."addressable_id" = "companies"."id" AND "addresses"."addressable_type" = 'Company'
WHERE (
addresses.lat BETWEEN 37.483013038215276 AND 38.06194017065867
AND addresses.lng BETWEEN -122.78956461309022 AND -122.05716204286986
)
GROUP BY
deals.id,
deals.created_at,
deals.updated_at,
deals.active,
deals.company_id,
deals.title,
deals.limitations,
deals.redemption_count,
addresses.lat,
addresses.lng
HAVING 3958.755864232 * 2 * ASIN(SQRT(POWER(SIN((37.772476604436974 - addresses.lat) * PI() / 180 / 2), 2) + COS(37.772476604436974 * PI() / 180) * COS(addresses.lat * PI() / 180) * POWER(SIN((-122.42336332798004 - addresses.lng) * PI() / 180 / 2), 2) )) <= 20
ORDER BY 3958.755864232 * 2 * ASIN(SQRT(POWER(SIN((37.772476604436974 - addresses.lat) * PI() / 180 / 2), 2) + COS(37.772476604436974 * PI() / 180) * COS(addresses.lat * PI() / 180) * POWER(SIN((-122.42336332798004 - addresses.lng) * PI() / 180 / 2), 2) )) ASC
My issue is that this will return multiple Deal records if that Deal's company has multiple Addresses, which I don't want.
In MySQL, I could just omit address.lat, address.lng in the GROUP_BY clause and it will properly group the records, but I can't do this in PostgreSQL.
I know I could wrap the whole query above in another SELECT and GROUP_BY, like this:
SELECT
id, created_at, updated_at, active, title, punches_to_complete, company_id, description, lat, lng, MIN(distance), bearing
FROM ( ... ) AS t
GROUP BY company_id
... where the ellipsis is the query from above. That (I believe) should get me the desired result in both MySQL and PostgreSQL.
The only problem is that I have no idea how to write this in ARel!
I had tried the following, a la this tip from the ARel guru, but I couldn't really make it work quite right (calling to_sql as the OP had said fixed his issue escapes the quotes, which freaks PostgreSQL out).
Can anyone help me with this???
UPDATE:
I've managed to get this done with an additional scope, like so:
scope :nearest, lambda { |coords|
subquery = "(#{Deal.near(coords).to_sql}) AS t1"
columns = Deal.columns.map{ |c| c.name }.join(',')
Deal.select(columns)
.select('MIN(distance) AS distance')
.from(subquery)
.group(columns)
.order('distance ASC')
}
However, this totally breaks chainability, as now I cannot call something like current_user.deals.nearest(coords), since that tags on an additional WHERE deals.user_id = 1 to the query outside of the subselect. I tried compensating for this by moving this logic into a class method and blanking the wheres clause on the SelectManager manually, like this:
def self.nearest(coords)
subquery = "(#{Deal.near(coords).to_sql}) AS t1"
columns = Deal.columns.map{ |c| c.name }.join(',')
query = Deal.select(columns)
.select('MIN(distance) AS distance')
.from(subquery)
.group(columns)
.order('distance ASC')
query.arel.ast.cores[0].wheres = []
query
end
... but that doesn't seem to work either: the additional WHERE clause is still appended:
Failure/Error:
#user.deals.nearest(southwest).first.distance.to_f.round(2).should ==
ActiveRecord::StatementInvalid:
Mysql2::Error: Unknown column 'deals.user_id' in 'where
clause': SELECT id,created_at,updated_at,user_id,company_id,
MIN(distance) AS distance FROM (SELECT deals.*, 3958.755864232 * 2 *
ASIN(SQRT(POWER(SIN((37.772476604436974 - addresses.lat) * PI() / 180
/ 2), 2) + COS(37.772476604436974 * PI() / 180) * COS(addresses.lat *
PI() / 180) * POWER(SIN((-122.42336332798004 - addresses.lng) * PI() /
180 / 2), 2) )) AS distance, CAST(DEGREES(ATAN2( RADIANS(addresses.lng
- -122.42336332798004), RADIANS(addresses.lat - 37.772476604436974)))
+ 360 AS decimal) % 360 AS bearing FROM deals INNER JOIN companies
ON companies.id = deals.company_id INNER JOIN addresses ON
addresses.addressable_id = companies.id AND
addresses.addressable_type = 'Company' WHERE deals.user_id =
26 AND (addresses.lat BETWEEN 37.483013038215276 AND 38.06194017065867
AND addresses.lng BETWEEN -122.78956461309022 AND -122.05716204286986)
GROUP BY
deals.id,deals.created_at,deals.updated_at,deals.user_id,deals.company_id,
addresses.lat, addresses.lng HAVING 3958.755864232 * 2 *
ASIN(SQRT(POWER(SIN((37.772476604436974 - addresses.lat) * PI() / 180
/ 2), 2) + COS(37.772476604436974 * PI() / 180) * COS(addresses.lat *
PI() / 180) * POWER(SIN((-122.42336332798004 - addresses.lng) * PI() /
180 / 2), 2) )) <= 20 ORDER BY 3958.755864232 * 2 *
ASIN(SQRT(POWER(SIN((37.772476604436974 - addresses.lat) * PI() / 180
/ 2), 2) + COS(37.772476604436974 * PI() / 180) * COS(addresses.lat *
PI() / 180) * POWER(SIN((-122.42336332798004 - addresses.lng) * PI() /
180 / 2), 2) )) ASC) AS t1 WHERE deals.user_id = 26 GROUP BY
id,created_at,updated_at,user_id,company_id ORDER BY distance ASC
LIMIT 1
Is what I'm trying to do even possible with ARel? The additional scopes above feel really dirty to me (parsing the subquery to raw SQL? I thought ARel was supposed to make it so I never did that!)
Related question: Can ARel formulate cross-db queries for CTEs (Common Table Expressions)?
I have the next query for getting addresses within a given distance and given postal code. Distance is calculated, based upon longitude and latitude data.
In this example i have replaced the user-input for just values (lat=52.64, long=6.88 en desired distance=10km)
the query:
SELECT *,
ROUND( SQRT( POW( ( (69.1/1.61) * ('52.64' - latitude)), 2) + POW(( (53/1.61) * ('6.88' - longitude)), 2)), 1) AS distance
FROM lp_relations_addresses distance
WHERE distance < 10
ORDER BY `distance` DESC
gives unknown column distance as error message.
when leaving out the where clausule i get every record of the table including their calculated distance. In this case i have to fetch the whole table.
How do i get only the desired records to fetch??
Thanks in advance for any comment!
You can't reference an alias in the select clause from another part of the sql statement. You need to put the whole expression in your where clause:
WHERE
ROUND( SQRT( POW( ( (69.1/1.61) * ('52.64' - latitude)), 2)
+ POW(( (53/1.61) * ('6.88' - longitude)), 2)), 1) < 10
A cleaner solution would be to use a sub-query to generate the calculated data:
SELECT *, distance
FROM (
SELECT *,
ROUND( SQRT( POW( ( (69.1/1.61) * ('52.64' - latitude)), 2)
+ POW(( (53/1.61) * ('6.88' - longitude)), 2)), 1) AS distance
FROM lp_relations_addresses
) d
WHERE d.distance < 10
ORDER BY d.distance DESC
Demo: http://www.sqlize.com/q96p2mCwnJ
As mellamokb notes, you can't reference column aliases in the WHERE clause. You can, however, do it in a HAVING clause:
SELECT *,
ROUND( SQRT( POW( ( (69.1/1.61) * ('52.64' - latitude)), 2) +
POW(( (53/1.61) * ('6.88' - longitude)), 2)), 1) AS distance
FROM lp_relations_addresses
HAVING distance < 10
ORDER BY distance DESC
Ps. If you have lots of addresses, you might want to consider optimizing the query by ruling out some of them early. For example, with suitable indexes, the following version might be considerably faster:
SELECT *,
ROUND( SQRT( POW( ( (69.1/1.61) * ('52.64' - latitude)), 2) +
POW(( (53/1.61) * ('6.88' - longitude)), 2)), 1) AS distance
FROM lp_relations_addresses
WHERE latitude > '52.64' - 10 / (69.1/1.61)
AND latitude < '52.64' + 10 / (69.1/1.61)
AND longitude > '6.88' - 10 / (53/1.61)
AND longitude < '6.88' + 10 / (53/1.61)
HAVING distance < 10
ORDER BY distance DESC
You are aliasing the calculation as 'distance', but you are also aliasing table 'lp_relations_addresses' as 'distance'. Try giving them a different name like this:
SELECT *,
ROUND( SQRT( POW( ( (69.1/1.61) * ('52.64' - latitude)), 2) + POW(( (53/1.61) * ('6.88' - longitude)), 2)), 1) AS distance
FROM lp_relations_addresses addr
WHERE distance < 10
ORDER BY `distance` DESC