In postgis, is the ST_GeomFromText call very expensive? I ask mostly because I have a frequently called query that attempts to find the point that is nearest another point that matches some criteria, and which is also within a certain distance of that other point, and the way I currently wrote it, it's doing the same ST_GeomFromText twice:
$findNearIDMatchStmt = $postconn->prepare(
"SELECT internalid " .
"FROM waypoint " .
"WHERE id = ? AND " .
" category = ? AND ".
" (b.category in (1, 3) OR type like ?) AND ".
" ST_DWithin(point, ST_GeomFromText(?," . SRID .
" ),". SMALL_EPSILON . ") " .
" ORDER BY ST_Distance(point, ST_GeomFromText(?,", SRID .
" )) " .
" LIMIT 1");
Is there a better way to re-write this?
Slightly OT: In the preview screen, all my underscores are being rendered as & # 9 5 ; - I hope that's not going to show up that way in the post.
I don't believe ST_GeomFromText() is particularly expensive, although in the past I've optimized PostGIS queries by creating a function, declaring a variable and then assigning the result of ST_GeomFromText to the variable.
Have you tried checking the execution plan for you query with a variety of different parameters because that should give you a definite idea of which bits of the query are taking the time?
I'm guessing most of the execution time will be in the calls to ST_DWithin() and ST_Distance(), although if the id and category columns aren't indexed then it might be doing some interesting table scanning.
#Ubiguch
It appears that ST_DWithin uses the spatial index, so that seems to cut down on the number of points to be queried pretty quickly.
navaid=> explain select internalid from waypoint where id != 'KROC' AND ST_DWithin(point, ST_GeomFromText('POINT(-77.6723888888889 43.1188611111111)',4326), 0.05) order by st_distance(point, st_geomfromtext('POINT(-77.6723888888889 43.1188611111111)',4326)) limit 1;
QUERY PLAN
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Limit (cost=8.37..8.38 rows=1 width=104)
-> Sort (cost=8.37..8.38 rows=1 width=104)
Sort Key: (st_distance(point, '0101000020E61000002FFE676B086B53C0847E44D7368F4540'::geometry))
-> Index Scan using waypoint_point_idx on waypoint (cost=0.00..8.36 rows=1 width=104)
Index Cond: (point && '0103000020E61000000100000005000000000000C03B6E53C000000060D0884540000000C03B6E53C0000000409D95454000000020D56753C0000000409D95454000000020D56753C000000060D0884540000000C03B6E53C000000060D0884540'::geometry)
Filter: (((id)::text <> 'KROC'::text) AND (point && '0103000020E61000000100000005000000000000C03B6E53C000000060D0884540000000C03B6E53C0000000409D95454000000020D56753C0000000409D95454000000020D56753C000000060D0884540000000C03B6E53C000000060D0884540'::geometry) AND ('0101000020E61000002FFE676B086B53C0847E44D7368F4540'::geometry && st_expand(point, 0.05::double precision)) AND (st_distance(point, '0101000020E61000002FFE676B086B53C0847E44D7368F4540'::geometry) < 0.05::double precision))
(6 rows)
Without the order by and the limit, it looks like a typical query is only returning 5-10 waypoints max. So I probably shouldn't worry about the additional cost of the filter that's applied to the points returned.
Related
I am implementing a web application where I need to check if a given point is within a polygon in MySQL table?
I am using ASP.net with MySQL. I am trying to use the following SQL statement
SELECT REGION_USER_ID FROM region WHERE (ST_Within(point(-23, 1), geom));
with my table and get the following error.
(FUNCTION ST_Within DOES NOT exist)
What is the issue here?
Also, is geom is a keyword ? (I got this from a site but can not remember where)
My polygon coordinates are in the table written with the following statement : (It's working and I can read and see)
INSERT INTO region (REGION_POLYGON) VALUES (PolygonFromText(#Parameter1))
Any help is greatly appreciated.
Here it is if someone is looking for the solution :
string Query = #"SELECT " +
"A," +
"B,"+
"C,"+
"D,"+
"E,"+
"F"+
"FROM user " +
"INNER JOIN State ON " +
"A=B AND C=#Parameter1 " +
"INNER JOIN Country ON " +
"CONTAINS(REGION_POLYGON, point(#Parameter2, #Parameter3)=1)";
Parameter2 : Logitude
Parameter3 : Latitude
Thanks.
What could be wrong with this select statement? I get an error called incorrect syntax near 'A'.
Since I am more used to write queries in postgreSQL, my guess is that MySQL has a bit different syntax.
cmd.CommandText = "WITH CurrentStop AS (SELECT[Stop Id] FROM Stops WHERE[Route Id] = " +
routeId + "AND Serial = " + stopsDriven + ")" +
"SELECT A.Firstname, A.Lastname, B.Make, B.Capacity, B.Route, D.Name" +
"FROM Driver A, Bus B, CurrentStop C, Stop D" +
"WHERE A.Id = " + row[0] + "AND B.[Bus Id] = " + row[1] + "AND C.[Stop Id] = D.[Stop Id]";
By the way, all inputs are in system only so no SQL injection could possibly happen.
Print out the query before trying to execute it and examine it closely. This solves a large chunk of the "what's wrong with my dynamically generated query?" style questions we see here.
For example, I'd be wary of the lack of space between (not necessarily an exhaustive list, these are just the ones I noticed):
routeId and the following AND;
row[0] and the following AND;
row[1] and the following AND;
D.Name and the following FROM; and
Stop D and the following WHERE.
Those last two are definitely problematic since, while it is possible the variables may end in a space (though unusual), the fixed strings certainly don't. And both may be causing the specific error you see since they would come out as:
D.NameFROM Driver A
Stop DWHERE A.
Ok, here is the code i have for my pagination:
$SQL = "SELECT
cpc.product_id,
cp.product_internal_ref,
cp.product_name,
cpa.product_sale_price,
cpa.is_product_service,
cpa.product_service_price,
cpi.image_name,
cpi.image_ext
FROM catalog_products_categories cpc
JOIN catalog_products cp ON cp.product_id = cpc.product_id
JOIN catalog_products_attributes cpa ON cpa.product_id = cpc.product_id
LEFT JOIN catalog_products_images cpi ON cpi.product_id = cpc.product_id
WHERE cpc.category_id = ".$catID;
// PAGINATOR SECTION
if($paginatorVARS['paginatorACTION'] == "next") {
$SQL .= " AND cpc.product_id > ".$paginatorVARS['paginatorGOID']." ";
$SQL .= "GROUP BY cpc.product_id ORDER BY cpc.product_id ASC LIMIT ".$paginatorVARS['catalogPaginatorPAGEROWS'];
}
elseif($paginatorVARS['paginatorACTION'] == "prev") {
$SQL .= " AND cpc.product_id < ".$paginatorVARS['paginatorGOID']." ";
$SQL .= "GROUP BY cpc.product_id ORDER BY cpc.product_id DESC LIMIT ".$paginatorVARS['catalogPaginatorPAGEROWS'];
}
// END PAGINATOR SECTION
I used the method described here: http://www.slideshare.net/Eweaver/efficient-pagination-using-mysql but i can't seam to find a way to also sort by other columns like for example by cpa.product_sale_price. If i do : ORDER BY cpc.product_id ASC, cpa.product_sale_price DESC/ASC it will break the paginator next results i dont know what happens... Please help!!
not saying their stuff is perfect, but it is often a place to start, to consider if you really want to reinvent the wheel, or take their concepts and incorporate it into your new rounder wheel. take a look at
http://dev.sencha.com/deploy/ext-4.0.0/examples/grid/paging.html
and use fiddler to examine the http gets that are occuring during pagination, as well as changes in the sort context. when it starts out it brings back the http response header, json data with jsonp callback, and top node value at the end of the json data specifying record count=6679 / 50 per page meaning 134 pages total as calculated by the front-end. the record count can vary from call to call (from page to page) as rows are inserted and deleted.
the sort column gets passed in the query string just like the pages impact the limit start,total that goes into sql. this can be seen in fiddler output showing changes in pages or sort column requests (column name, and asc or desc):
GET /forum/topics-browse-remote.php?_dc=1369401925806&page=1&start=0&limit=50&sort=lastpost&dir=DESC&callback=Ext.data.JsonP.callback1 HTTP/1.1
GET /forum/topics-browse-remote.php?_dc=1369401977137&page=1&start=0&limit=50&sort=replycount&dir=ASC&callback=Ext.data.JsonP.callback2 HTTP/1.1
GET /forum/topics-browse-remote.php?_dc=1369401978355&page=1&start=0&limit=50&sort=replycount&dir=DESC&callback=Ext.data.JsonP.callback3 HTTP/1.1
Are there pre-existing libraries that will take a user input and transform it into a SQL WHERE clause?
For example given a database that has columns first_name, last_name, and address the user could input something like:
John State St
and the library would build a query such that it would return rows that match a guy named John that lived on State St (or a guy named State that lived on John St, for that matter).
It could also support things like specifying the column:
first_name:John address:State
I have some simple code to handle some of these cases already but it's getting a little unwieldily. I would think there are some pre-existing solutions to this problem but I'm having a hard time finding them. Generally, the problem is how to enable the user to easily search a structured database with a single input field.
In this matter you can break down string as multiple values using string manipulation, and then search for each word in each column using "or" conditions.
additionally you can define index on columns so as to achieve faster search.
You might have tried this technique since you mentioned, but you have to look up each word in each column
Jquery UI autocomplete could be what you are looking for. the css along with it is also necessary please see this link http://api.jqueryui.com/autocomplete/ for the .js ans css needed.
$( "#myInputBoxId" ).autocomplete({
source: "search.php",
minLength: 2,
select: function( event, ui ) {
log( ui.item ?
"Selected: " + ui.item.value + " aka " + ui.item.id :
"Nothing selected, input was " + this.value );
}
});
and in search.php
<?php
include 'config.php';
$results =array();
$req = "SELECT product_name "
."FROM table4 "
."WHERE product_name LIKE '%".$_REQUEST['term']."%' LIMIT 5";
$query = mysqli_query($con,$req);
while($row = mysqli_fetch_array($query))
{
array_push($results,$row['product_name']);
}
echo json_encode($results);
}
I have found many calculations here and some php examples and most are just over my head.
I found this example:
SELECT b.zip_code, b.state,
(3956 * (2 * ASIN(SQRT(
POWER(SIN(((a.lat-b.lat)*0.017453293)/2),2) +
COS(a.lat*0.017453293) *
COS(b.lat*0.017453293) *
POWER(SIN(((a.lng-b.lng)*0.017453293)/2),2))))) AS distance
FROM zips a, zips b
WHERE
a.zip_code = '90210' ## I would use the users submitted value
GROUP BY distance
having distance <= 5; ## I would use the users submitted value
But, I am having trouble understanding how to implement the query with my database.
It looks like that query has all I need.
However, I cannot even find/understand what b.zip_code actually is! (whats the b. and zips a, zips b?)
I also do not need the state in the query.
My mySQL db structure is like this:
ZIP | LAT | LONG
33416 | 26.6654 | -80.0929
I wrote this in attempt to return some kind of results (not based on above query) but, it only kicks out one zip code.
## Just for a test BUT, in reality I desire to SELECT a zip code WHERE ZIP = the users submitted zip code
## not by a submitted lat lon. I left off the $connect var, assume it's there.
my $set1 = (26.6654 - 0.20);
my $set2 = (26.6654 + 0.20);
my $set3 = (-80.0929 - 0.143);
my $set4 = (-80.0929 + 0.143);
my $test123 = $connect->prepare(qq{SELECT `ZIP` FROM `POSTAL`
WHERE `LAT` >= ? AND `LAT` <= ?
AND `LONG` >= ? AND `LONG` <= ?}) or die "$DBI::errstr";
$test123->execute("$set1","$set2","$set3","$set4") or die "$DBI::errstr";
my $cntr;
while(#zip = $test123->fetchrow_array()) {
print qq~$zip[$cntr]~;
push(#zips,$zip[$cntr]);
$cntr++;
}
As you can see, I am quite the novice so, I need some hand holding here with verbose explanation.
So, in Perl, how can I push zip codes into an array from a USER SUBMITTED ZIP CODE and user submitted DISTANCE in miles. Can be a square instead of a circle, not really that critical of a feature. Faster is better.
I'll tackle the small but crucial part of the question:
However, I cannot even find/understand what b.zip_code actually is! (whats the "b." and "zips a, zips b"?)
Basically, the query joins two tables. BUT, both tables being joined are in fact the same table - "zips" (in other words, it joins "zips" table to itself"). Of course, since the rest of the query needs to understand when you are referring to the first copy of the "zips" table and when to the second copy of the "zips" table, you are giving a table alias to each copy - to wit, "a" and "b"'.
So, "b.xxx" means "column xxx from table zips, from the SECOND instance of that table being joined".
I don't see what's wrong with your first query. You have latitude and longitude in your database (if I'm understanding, you're comparing a single entry to all others). You don't need to submit or return the state that's just part of the example. Make the first query work like this:
my $query = "SELECT b.zip_code,
(3956 * (2 * ASIN(SQRT(
POWER(SIN(((a.lat-b.lat)*0.017453293)/2),2) +
COS(a.lat*0.017453293) *
COS(b.lat*0.017453293) *
POWER(SIN(((a.lng-b.lng)*0.017453293)/2),2))))) AS distance
FROM zips a, zips b WHERE
a.zip_code = ?
GROUP BY distance having distance <= ?";
my $sth = $dbh->prepare($query);
$sth->execute( $user_submitted_zip, $user_submitted_distance );
while( my ($zip, $distance) = $sth->fetchrow() ) ) {
# do something
}
This won't be that fast, but if you have a small record set ( less than 30k rows ) it should be fine. If you really want to go faster you should look into a search engine such as Sphinx which will do this for you.
fetchrow_array returns a list of list references, essentially a two-dimensional array, where each row represents a different result from the database query and each column represents a field from the query (in your case, there is only one field, or column, per row).
Calling while ($test123->fetchrow_array()) will cause an infinite loop as your program executes the query over and over again. If the query returns results, then the while condition will be satisfied and the loop will repeat. The usual idiom would be to say something more like for my $row ($test123->fetchrow_array()) { ..., which will only execute the query once and then iterate over the results.
Each result is a list reference, and the zip code you are interested in is in the first (and only) column, so you could accumulate the results in an array like this:
my #zips = (); # for final results
for my $row ($test123->fetchrow_array()) {
push #zips, $row->[0];
}
or even more concisely with Perl's map statement:
my #zips = map { $_->[0] } $test123->fetchrow_array()
which does the same thing.