I am trying to find the closest addresses to each other in my database. I can easily find the closest addresses to a specific coordinate ie -1.1337599754333496, 53.52299880981445 ...
SELECT st_distance_sphere(
address.coordinates,
POINT(-1.1337599754333496, 53.52299880981445 )
) AS distance
FROM address
ORDER BY distance ASC;
Is there a way to order the entire database by distance or extract a subset of the X nearest neighbours ? After wracking my brains all day I am starting to think that there isn't :-(
Related
I'm using CodeIgniter 2 and in my database model, I have a query that joins two tables and filters row based upon distance from a given geolocation.
SELECT users.id,
(3959 * acos(cos(radians(42.327612)) *
cos(radians(last_seen.lat)) * cos(radians(last_seen.lon) -
radians(-77.661591)) + sin(radians(42.327612)) *
sin(radians(last_seen.lat)))) AS distance
FROM users
JOIN last_seen ON users.id = last_seen.seen_id
WHERE users.age >= 18 AND users.age <= 30
HAVING distance < 50
I'm not sure if it's the distance that is making this query take especially long. I do have over 300,000 rows in my users table. The same amount in my last_seen table. I'm sure that plays a role.
But, the age column in the users table is indexed along with the id column.
The lat and lon columns in the last_seen table are also indexed.
Does anyone have ideas as to why this query takes so long and how I can improve it?
UPDATE
It turns out that this query actually runs pretty quickly. When I execute this query in PHPMyAdmin, it takes 0.56 seconds. Not too bad. But, when I try to execute this query with a third party SQL client like SequelPro, it takes at least 20 seconds and all of the other apps on my mac slow down. When the query is executed by loading the script via jQuery's load() method, it takes around the same amount of time.
Upon viewing my network tab in Google Chrome's developer tools, it seems that the reason it's taking so long to load is because of what's called TTFB or Time To First Byte. It's taking forever.
To make this query faster you need to limit the count of rows using an index before actually calculating the distance on every and each of them. To do so you can limit the rows from last_seen based on their lat/lon and a rough formula for desired distance.
The idea is that the positions with the same latitude as the reference latitude would be in 50 miles distance if their longitude falls in a certain distance from the reference longitude and vice versa.
For 50 miles distance, RefLat+-1 and RefLon+-1 would be a good start to limit the rows before actually calculating the precise distance.
last_seen.lat BETWEEN 42.327612 - 1 AND 42.327612 + 1
AND last_seen.lon BETWEEN -77.661591 - 1 AND -77.661591 + 1
For this query:
SELECT users.id, (3959 * acos(cos(radians(42.327612)) * cos(radians(last_seen.lat)) * cos(radians(last_seen.lon) - radians(-77.661591)) + sin(radians(42.327612)) * sin(radians(last_seen.lat)))) AS distance
FROM users JOIN
last_seen
ON users.id = last_seen.seen_id
WHERE users.age >= 18 AND users.age <= 30
HAVING distance < 50;
The best index is users(age, id) and last_seen(seen_id). Unfortunately, the distance calculations are going to take a while, because they have to be calculated for every row. You might want to consider a GIS extension to MySQL to help with this type of query.
I am trying to get the nearest location to a users input from within a database, (nearest store based on latitude and longitude), so based on the users postcode I am converting that to latitude and longitude and from these results I need to search my database to find the store that is the nearest to these values. I have the latitude and longitude of all stores saved and so far (from looking at previous questions) I have tried something like:
SELECT *
FROM mystore_table
WHERE `latitude` >=(51.5263472 * .9) AND `longitude` <=(-0.3830181 * 1.1)
ORDER BY abs(latitude - 51.5263472 AND longitude - -0.3830181) limit 1;
When I run this query, it does display a result, but it is not the nearest store, not sure if it could be something to do with the negative numbers, both my columns latitude + longitude are saved as decimal data types?
You have a logic operation in the order by rather than an arithmetic one. Try this:
SELECT *
FROM mystore_table
WHERE `latitude` >=(51.5263472 * .9) AND `longitude` <=(-0.3830181 * 1.1)
ORDER BY abs(latitude - 51.5263472) + abs(longitude - -0.3830181)
limit 1;
The AND in your original version would be producing a boolean value, either 0 or 1 -- and it would only be 1 when the values match exactly to the last decimal point. Not very interesting.
There are many reasons why this is not the nearest distance, but it might be close enough for your purposes. Here are some reasons:
Euclidean distance would take the square of the differences
Distance between two latitudes depends on the longitude (varying from about 70 miles on the equator to 0 at the poles).
I have a table A which has a column 'template_phash'. I store the phash generated from 400K images.
Now I take a random image and generate a phash from that image.
Now how do I query so that I can get the record from table A which hamming distance difference is less than a threshold value, say 20.
I have seen Hamming distance on binary strings in SQL, but couldn't figure it out.
I think I figured out that I need to make a function to achieve this but how?
Both of my phash are in BigInt eg: 7641692061273169067
Please help me make the function so that I could query like
SELECT product_id, HAMMING_DISTANCE(phash1, phash2) as hd
FROM A
WHERE hd < 20 ORDER BY hd ASC;
I figured out that the hamming distance is just the count of different bits between the two hashes. First xor the two hashes then get the count of binary ones:
SELECT product_id, BIT_COUNT(phash1 ^ phash2) as hd from A ORDER BY hd ASC;
I'm trying to execute this query:
SELECT `venues` . *
FROM `venues`
WHERE `lat` >= 39.847991180331
AND `lng` >= -86.332592054637
AND `lat` <= 40.137846252794
AND `lng` <= -85.954901904578
LIMIT 0 , 30
Here is the table structure of the lat and lng fields and a sample row, it should be returning AT LEAST 1 row
lng decimal(65,15)
lat decimal(65,15)
Name Lng Lat
Ambre Blends Studio -86.143746979607530 39.875314973898870
However when I execure the above query it returns no rows. What's the problem here?
I'm always circumspect about floating point equality and inequality operations and MySQL has had a history of problems in the past with this sort of thing (I'm not sure which version you're using but there were some changes made around 503/505).
The first thing I would do is debug the actual statement. Start with:
select venues.* from venues
where name like 'Ambre%'
and ensure you only get one row (you may have to tweak the where clause if there's lots of rows starting with Ambre).
Then add the other subclauses one at a time until the row doesn't appear. The first would be:
select venues.* from venues
where name like 'Ambre%'
and lat >= 39.847991180331
Once you establish which condition is causing the select to fail, you can start looking into why.
I'm trying to get 100 points from my table with a lowest distance to a given point.
I'm using
SELECT *, GLENGTH(
LINESTRINGFROMWKB(
LINESTRING(
ASBINARY(
POINTFROMTEXT("POINT(40.4495 -79.988)")
),
ASBINARY(pt)
)
)
)
AS `distance` FROM `ip_group_city` ORDER BY distance LIMIT 100
(Yeah, that's painful. I've just googled it. I have no idea how to measure distance in MySQL correctly)
It takes very long time for execute. EXPLAIN says that there are no possible_keys.
I created a SPATIAL index on the pt column:
CREATE SPATIAL INDEX sp_index ON ip_group_city (pt);
Though I don't really know how to use it correctly. Can you please help me?
Because you don't have WHERE clause therefore no affected index. I think you should improve this query by add using MBR_ (MySQL 5.0 or later) or ST_ functions (MySQL 5.6 or later).
Something like:
SELECT *, GLENGTH(
LINESTRINGFROMWKB(
LINESTRING(
ASBINARY(
POINTFROMTEXT("POINT(40.4495 -79.988)")
),
ASBINARY(pt)
)
)
)
AS `distance`
FROM `ip_group_city`
WHERE
MBRWithin(
pt, -- your point
GeomFromText('Polygon( -- your line (in polygon format) from pt to target point
(
#{bound.ne.lat} #{bound.ne.lng}, --North East Lat - North East Long
#{bound.ne.lat} #{bound.sw.lng}, --North East Lat - South West Long
#{bound.sw.lat} #{bound.sw.lng}, --
#{bound.sw.lat} #{bound.ne.lng},
#{bound.ne.lat} #{bound.ne.lng}
)
)')
)
ORDER BY distance LIMIT 100
I've used the great circle equation to do these types of calculations in the past. I'm not sure how the performance compares but it might be worth trying it and comparing.
Here is a good SO post that goes over how to do it in MySQL.
Have a look at these questions:
Finding Cities within 'X' Kilometers (or Miles)