I have a MultiPolygon field, preferences.locations, and a Point field, rental.location. When querying to check whether a rental's location is contained by preferences.locations, the query is only successful if the rental.location is contained by the first polygon in the preferences.locations MultiPolygon.
For example, with these geometries:
point1 = (81.20141954209073, -129.891357421875)
point2 = (40.70875101828792, -73.93179774284363)
preferences.locations = MultiPolygon(
Polygon(((81.14748070499664, -163.289794921875),
point1, # contains the first point
(81.14748070499664, -163.289794921875),
(81.14748070499664, -163.289794921875),)),
Polygon(((40.70718949655447, -73.98123621940613),
point2, # contains the second point
(40.683762276904055, -73.99702906608582),
(40.70718949655447, -73.98123621940613),)),
)
rental1.location = Point(*point1)
rental2.location = Point(*point2)
When querying to check which rentals' locations are contained by preferences.locations, while both rentals should be returned, only the first rental is returned.
>>> Rental.objects.filter(location__contained=preferences.locations)
[<Rental: Rental object>] # rental1
How can I successfully check which rentals' locations are contained by preferences.locations (no matter which Polygon they're contained by).
The proper way to check whether a Point is contained by a MultiPolygon is to use point.intersects(multipolygon).
>>> Rental.objects.filter(location__intersects=preferences.locations)
[<Rental: Rental object>, <Rental: Rental object>]
Related
I am trying to calculate SAVI vegetation index using MODIS data. But I am getting an error showing:
Image.select: Pattern 'B2' did not match any bands.
Code:
countries = ee.FeatureCollection("USDOS/LSIB_SIMPLE/2017")
canada = countries.filter(ee.Filter.eq("country_na", "Canada"))
image = ee.ImageCollection("MODIS/061/MOD09A1")\
.filterDate('2017-01-01','2017-12-31')\
.filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE',10))\
.filterBounds(canada)\
.median()\
.clip(canada)
savi = image.expression(
'1.5*((NIR-RED)/(NIR+RED+0.5))',{
'NIR':image.select('B2'),
'RED':image.select('B1')
}).rename('savi')
saviVis = {'min':0.0, 'max':1, 'palette':['yellow', 'green']}
Map = geemap.Map()
Map.addLayer(savi, saviVis, 'SAVI')
Map
Why am I getting this error? Isn't B1 designated to Red and B2 to NIR?
The general thing to do when you hit this type of problem is to start examining the dataset for what is actually there — how many images are you matching, what properties and bands those images have, etc. I found two problems:
Your filter criteria matched zero images. Therefore the collection is empty, and therefore the median() image from that collection has no bands at all. (You can check this by putting the collection in a variable and printing the size() of it.) You will need to adjust the criteria.
It seems that the main reason they didn't match is that the images in MODIS/061/MOD09A1 do not have a CLOUDY_PIXEL_PERCENTAGE property.
The band names for MODIS/061/MOD09A1 are not B1, B2, ... but sur_refl_b01, sur_refl_b02 and so on. You can see this with the Inspector in the Earth Engine Code Editor, or on the dataset description page.
Perhaps you were working from information about a different dataset?
With the two problems above fixed, your code produces some results. This is the (JS) version I produced while testing (Code Editor link):
var countries = ee.FeatureCollection("USDOS/LSIB_SIMPLE/2017");
var canada = countries.filter(ee.Filter.eq("country_na", "Canada"));
var images = ee.ImageCollection("MODIS/061/MOD09A1")
.filterDate('2017-01-01','2017-12-31')
// .filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE',10))
.filterBounds(canada);
// print(images);
var image = images.median().clip(canada);
Map.addLayer(canada);
Map.addLayer(image);
var savi = image.expression(
'1.5*((NIR-RED)/(NIR+RED+0.5))',{
'NIR':image.select('sur_refl_b02'),
'RED':image.select('sur_refl_b01')
}).rename('savi');
var saviVis = {'min':0.0, 'max':1, 'palette':['yellow', 'green']};
Map.addLayer(savi, saviVis, 'SAVI')
I want to find an algorithm for the following problem:
Given a directed graph G with starting point s and N sets of nodes, I need to find a path that starts in the start node and then goes to at least one node of every set.
Not all of the nodes are in the sets- only some of them.
There are no nodes that are in more than one set.
For example:
starting point: [n1]
First set: [n5,n6,n8,n9]
Seconed set: [n2,n10]
Third set: [n4,n7]
Forth set: [n3]
Other nodes: [m1..m7]
Showing the graph edges:
In this case a shortest path will be:
[n1,m3,m4,m5,n3,m6,n2,m7,n9]
[n1,m1,n7,m5,n3,m6,n2,m7,n9]
This problem is similar to the generalization of the Traveling salesman problem (wikipedia), but in this case, the graph is directional and unweighted. Also, we have a known start point, we have more than just the nodes in the sets and we can walk on the same node more than once.
I tried starting with a lazy fast way of searching for the closest node from the start to all of the nodes in all of the sets with Dijkstra and then removing the set from the calculation and continue searching for the next closest node from all sets from that point.
It is easy to notice that this way doesn't give the shortest path, but the real problem is that sometimes it falls to a dead-end and returns "no path" even when there is a path.
A naive way I can see that can be used is to create a function to calculate for every specific permutation of sets and use an algorithm like the one suggested here for a close but easier problem.
It will take O(|N|! * 2^(the number of all nodes in all the N sets))
Adding the code of the lazy way (sorry for the mess).
from Queue import Queue
from my_utils import RGBtoHex
from igraph import *
from random import choice
class path_info:
def __init__(self, way, length_of_way, satisfied_goal):
self.way = way
self.length_of_way = length_of_way
self.satisfied_goal = satisfied_goal
class graph_actions():
# Goal point has a "start" node and all other goal points are saved in a dictionary
def __init__(self, graph, goal_points):
self.graph = graph
self.goal_points = goal_points
self.best_path = list()
def calculate_best_path(self):
self.best_path = list()
queued_best_path = scatter(self.graph, self.goal_points)
while not queued_best_path.empty():
self.best_path.append(queued_best_path.get())
return self.best_path
def scatter(graph, goal_points):
best_path = Queue()
starting_point = goal_points["start"]
del goal_points["start"]
paths_dict = dict()
return find_best_way(best_path, goal_points, graph, paths_dict, starting_point)
def find_best_way(best_path, goal_points, graph, paths_dict, starting_point):
last = -1
while len(goal_points) > 0:
vertex_color = known_colors[choice(known_colors.keys())]
for vertex in goal_points.keys():
# Find shortest paths from vertex
# Color vertex
graph.vs.find(str(vertex).encode('ascii', 'replace'))["Fill Color"] = RGBtoHex(vertex_color)
# Find shortest path if exist with igraph function
way = graph.get_shortest_paths(starting_point, vertex)
if len(way[0]) != 0:
if (last != -1):
# Taking out the first node that is the end of the last route taken
way[0].pop(0)
paths_dict[vertex] = path_info(way, len(way[0]), goal_points[vertex])
# Find the closest node
stage = min(paths_dict, key=lambda x: paths_dict[x].length_of_way)
# Transfer from list to queue
for step in paths_dict[stage].way[0]:
best_path.put(step)
last = step
# Delete the other nodes with of the same group from the goal list
for key, value in goal_points.items():
if paths_dict.has_key(stage) and value == paths_dict[stage].satisfied_goal:
del goal_points[key]
if key != stage:
del paths_dict[key]
del paths_dict[stage]
# Saving the last node of this route
starting_point = last
graph.save("states_graph_color.graphml", "graphml")
return best_path
Is there a way that is better than the naive? If not, can I use a heuristic way to do this?
I also wonder if igraph has a trick to this in a more clean way.
Even if only the naive way is possible, I'm not sure how to implement it.
I have a FeatureCollection made up of many (100-200) polygons ('ftr_polygons'). I also have an ImageCollection made up of monthly median Landsat8 bands and indices ('byMonth'). I want to ReduceRegions and save a median (or mean) spatial average from each polygon in the FeatureCollection. End goal is to export to csv a timeseries of monthly mean bands/indices within each polygons over multiple years (2013-2019).
With the code below, I am able to do this for ~1 year, but any more than that, and I get an error: 'FeatureCollection (Error) Computation timed out’. Is there a better way to do this?
// define the function that will grab median (or mean) spatial reductions for each polygon, for each month
var extractdata = function(medianImage,ftr_polygons) {
var date_start = ee.Date(medianImage.get('system:time_start')).format("YYYY-MM"); // get date as string to append to each property
// spatial MEDIAN
ftr_polygons = medianImage.reduceRegions({ // create feature collection with new properties, bands for each month, uniquely named
collection: ftr_polygons,
reducer: ee.Reducer.median(),
scale: 30,
tileScale: 1}); // tile scale
var ftr_polygons_propnames = ftr_polygons.first().propertyNames(); // get property names first
var ftr_polygons_newnames = ftr_polygons_propnames.replace('NDVI_median',
ee.String('NDVI_median_').cat(date_start)); //replace property names with band+date
ftr_polygons_newnames = ftr_polygons_newnames.replace('EVI_median',
ee.String('EVI_median_').cat(date_start)); //replace property names with band+date
ftr_polygons_newnames = ftr_polygons_newnames.replace('NIRv_median',
ee.String('NIRv_median_').cat(date_start)) ; //replace property names with band+date
ftr_polygons = ftr_polygons.map(function(f) {return f.select(ftr_polygons_propnames,ftr_polygons_newnames)});
return ftr_polygons;
};
// apply the function over ImageCollection byMonth, beginning with feature collection ftr_polygons
var ftr_polygons = ee.FeatureCollection(byMonth.iterate(extractdata,ftr_polygons));
// remove geometry on each feature before printing or exporting
var myproperties=function(feature){
feature=ee.Feature(feature).setGeometry(null);
return feature;
};
var ftr_polygon_export = ftr_polygon.map(myproperties)
print(ftr_polygon_export.limit(1), 'For export w monthly properties');
Maybe this answer: https://stackoverflow.com/a/48412324/12393507 alludes to a better way:
The same approach can be used with reduceRegions() as well, mapping over images and then over regions. However, you will have to map over the resulting features to set dates.
I would appreciate more info on this approach.
Thanks.
For computationally intensive operations that will run for a long time you should always export your results instead of visualizing/printing them.
For more info read through this section of the debugging page in the Earth Engine manual.
I am working on a small MeteorJS app that plots points on a map based on popular areas for work.
I have this:
Template.list.jobs = function() {
if(Session.get('currentIndustryOnet')) {
jobs = Jobs.find({onet: Session.get('currentIndustryOnet')}).fetch();
// Session.set('jobCount', jobs.count());
var cnt = _.groupBy(jobs, 'address');
console.log(cnt);
return Pagination.collection(jobs);
} else {
jobs = Jobs.find()
Session.set('jobCount', jobs.count());
return Pagination.collection(jobs.fetch());
}
}
The cnt variable returns a properly grouped array (the key of the array is an address like Allentown, PA). I have a collection of Cities which have ever city in the USA along with their LAT/LONGs to plot on a Google Map. So I will take the top 100 from the grouped array, find the lat/long in the Cities collection and plot those points on a map.
I am not familiar with working with a groupedBy method to sort the list based on the length and then pull out the key to use as my search.
I'm not 100% certain about the data structure... but assuming jobs have an address field, and you want them sorted by frequency of occurrence and capped, you could do something like this:
var addresses = _.chain(jobs)
.countBy('address')
.pairs()
.sortBy(function(j) {return -j[1];})
.map(function(j) {return j[0];})
.first(100)
.value();
Note there may be a more clever way to use underscore to arrive at this result. Once you have the capped, sorted list of addresses, you can probably get the lat/long values via a find like:
Cities.find({address: {$in: addresses}}).fetch();
I have found many calculations here and some php examples and most are just over my head.
I found this example:
SELECT b.zip_code, b.state,
(3956 * (2 * ASIN(SQRT(
POWER(SIN(((a.lat-b.lat)*0.017453293)/2),2) +
COS(a.lat*0.017453293) *
COS(b.lat*0.017453293) *
POWER(SIN(((a.lng-b.lng)*0.017453293)/2),2))))) AS distance
FROM zips a, zips b
WHERE
a.zip_code = '90210' ## I would use the users submitted value
GROUP BY distance
having distance <= 5; ## I would use the users submitted value
But, I am having trouble understanding how to implement the query with my database.
It looks like that query has all I need.
However, I cannot even find/understand what b.zip_code actually is! (whats the b. and zips a, zips b?)
I also do not need the state in the query.
My mySQL db structure is like this:
ZIP | LAT | LONG
33416 | 26.6654 | -80.0929
I wrote this in attempt to return some kind of results (not based on above query) but, it only kicks out one zip code.
## Just for a test BUT, in reality I desire to SELECT a zip code WHERE ZIP = the users submitted zip code
## not by a submitted lat lon. I left off the $connect var, assume it's there.
my $set1 = (26.6654 - 0.20);
my $set2 = (26.6654 + 0.20);
my $set3 = (-80.0929 - 0.143);
my $set4 = (-80.0929 + 0.143);
my $test123 = $connect->prepare(qq{SELECT `ZIP` FROM `POSTAL`
WHERE `LAT` >= ? AND `LAT` <= ?
AND `LONG` >= ? AND `LONG` <= ?}) or die "$DBI::errstr";
$test123->execute("$set1","$set2","$set3","$set4") or die "$DBI::errstr";
my $cntr;
while(#zip = $test123->fetchrow_array()) {
print qq~$zip[$cntr]~;
push(#zips,$zip[$cntr]);
$cntr++;
}
As you can see, I am quite the novice so, I need some hand holding here with verbose explanation.
So, in Perl, how can I push zip codes into an array from a USER SUBMITTED ZIP CODE and user submitted DISTANCE in miles. Can be a square instead of a circle, not really that critical of a feature. Faster is better.
I'll tackle the small but crucial part of the question:
However, I cannot even find/understand what b.zip_code actually is! (whats the "b." and "zips a, zips b"?)
Basically, the query joins two tables. BUT, both tables being joined are in fact the same table - "zips" (in other words, it joins "zips" table to itself"). Of course, since the rest of the query needs to understand when you are referring to the first copy of the "zips" table and when to the second copy of the "zips" table, you are giving a table alias to each copy - to wit, "a" and "b"'.
So, "b.xxx" means "column xxx from table zips, from the SECOND instance of that table being joined".
I don't see what's wrong with your first query. You have latitude and longitude in your database (if I'm understanding, you're comparing a single entry to all others). You don't need to submit or return the state that's just part of the example. Make the first query work like this:
my $query = "SELECT b.zip_code,
(3956 * (2 * ASIN(SQRT(
POWER(SIN(((a.lat-b.lat)*0.017453293)/2),2) +
COS(a.lat*0.017453293) *
COS(b.lat*0.017453293) *
POWER(SIN(((a.lng-b.lng)*0.017453293)/2),2))))) AS distance
FROM zips a, zips b WHERE
a.zip_code = ?
GROUP BY distance having distance <= ?";
my $sth = $dbh->prepare($query);
$sth->execute( $user_submitted_zip, $user_submitted_distance );
while( my ($zip, $distance) = $sth->fetchrow() ) ) {
# do something
}
This won't be that fast, but if you have a small record set ( less than 30k rows ) it should be fine. If you really want to go faster you should look into a search engine such as Sphinx which will do this for you.
fetchrow_array returns a list of list references, essentially a two-dimensional array, where each row represents a different result from the database query and each column represents a field from the query (in your case, there is only one field, or column, per row).
Calling while ($test123->fetchrow_array()) will cause an infinite loop as your program executes the query over and over again. If the query returns results, then the while condition will be satisfied and the loop will repeat. The usual idiom would be to say something more like for my $row ($test123->fetchrow_array()) { ..., which will only execute the query once and then iterate over the results.
Each result is a list reference, and the zip code you are interested in is in the first (and only) column, so you could accumulate the results in an array like this:
my #zips = (); # for final results
for my $row ($test123->fetchrow_array()) {
push #zips, $row->[0];
}
or even more concisely with Perl's map statement:
my #zips = map { $_->[0] } $test123->fetchrow_array()
which does the same thing.