I am trying to speed the spatial distance calculation between polygons and points/polygons by calling the GEOS library directly. However I couldn't find any help how to call this function correctly. Can anyone please point me to the location where I can find the reference for this function or point out where I have done incorrectly?
working example:
from shapely.geos import lgeos
points_geom = np.array([x._geom for x in points])
polygons_geom = np.array([x._geom for x in polygons])
lgeos._lgeos.GEOSContains_r(lgeos.geos_handle,polygons_geom[0],points_geom[0])
Not working:
lgeos._lgeos.GEOSDistance_r(lgeos.geos_handle,polygons_geom[0],points_geom[0])
TypeError Traceback (most recent call last)
<ipython-input-138-392cb700cfbc> in <module>()
----> 1 lgeos._lgeos.GEOSDistance_r(lgeos.geos_handle,polygons_geom[0],points_geom[0])
TypeError: this function takes at least 4 arguments (3 given)
GEOSDistance_r takes 4 arguments, you're only passing three:
extern int GEOS_DLL GEOSDistance_r(GEOSContextHandle_t handle,
const GEOSGeometry* g1,
const GEOSGeometry* g2, double *dist);
(from https://github.com/OSGeo/geos/blob/5a730fc50dab2610a9e6c037b521accc66b7777b/capi/geos_c.h.in#L1100)
You're using shapely's private interface to GEOS, which it looks like uses ctypes, so you'll need to use the ctypes invocation to pass a double by reference:
import ctypes
dist = ctypes.c_double()
lgeos._lgeos.GEOSContains_r(
lgeos.geos_handle, polygons_geom[0], points_geom[0], ctypes.byref(dist))
dist = dist.value
Related
I am using ORTools for path optimization. When I try my code in localhost it returns the expected result. However, when I try it on my DigitalOcean App it gives an error message:" returned a result with an exception set". Can anyone help me to find a solution ?
Here is the TraceBack code from Postman if it is useful:
The above exception ('numpy.float64' object cannot be interpreted as an integer) was the direct cause of the following exception:
Local vars
/workspace/.heroku/python/lib/python3.10/site-packages/ortools/constraint_solver/pywrapcp.py, line 5730, in GetArcCostForVehicle
return _pywrapcp.RoutingModel_CostVar(self)
def GetArcCostForVehicle(self, from_index: "int64_t", to_index: "int64_t", vehicle: "int64_t") -> "int64_t":
r"""
Returns the cost of the transit arc between two nodes for a given vehicle.
Input are variable indices of node. This returns 0 if vehicle < 0.
"""
return _pywrapcp.RoutingModel_GetArcCostForVehicle(self, from_index, to_index, vehicle) …
Please check that all your transit callbacks are returning an int type.
In your code you must have registered a transit callback:
# Create and register a transit callback.
def distance_callback(from_index, to_index):
"""Returns the distance between the two nodes."""
# Convert from routing variable Index to distance matrix NodeIndex.
from_node = manager.IndexToNode(from_index)
to_node = manager.IndexToNode(to_index)
return data['distance_matrix'][from_node][to_node]
transit_callback_index = routing.RegisterTransitCallback(distance_callback)
then you have passed this transit_callback_index to setup the arc cost evaluator function.
# Define cost of each arc.
routing.SetArcCostEvaluatorOfAllVehicles(transit_callback_index)
The point is, the function distance_callback() will be call internally by the C++ library (python is just a wrapper on top of the C++ library). The C++ expect a C++ int64_t returned by the distance_callback() wrapper function (i.e. the python-C++ bridge need an int from the python method to be able to convert it to a C++ int64_t).
If your method return a numpy.float64 (e.g. because you r distance matrix data are an numpy array full of floating point values) then everything blow up thus the error message...
This is a minimally reproducible version of my Cython error. The code runs in C++.
The compiler is telling me error C2088 that "+= is illegal for struct". However, it is being passed an array.
The pyx file:
from libc.stdint cimport uint32_t as Card
from cpython cimport array
import array
cdef extern from "ace_eval.h":
void ACE_addcard(h, Card c)
def create_hand():
cdef array.array h = array.array('I',[0, 0, 0, 0, 0])
ACE_addcard(h, 257)
return h
The function imported from the header is:
#define ACE_addcard(h,c) h[c&7]+=c,h[3]|=c
I have also tried declaring my arrays using
cdef Card h[5]
array.array is a Python object that is ultimately compiled into a struct (so this is what C++ sees). Element access to it is controlled at a Python level by __getitem__ and __setitem__, which are compiled by Cython into C API function calls. When Cython sees code for an array being manipulated it'll generate the appropriate C API function calls. You code using C++ #define statements attempts to manipulate it at C++ compile time and prevents Cython from knowing what's going on.
Ideally you should be using "typed memoryviews" which give Cython quicker access to the array (but will still not work with the C++ #define since this is applied after Cython has processed the file):
cdef int[::1] h = array.array('I',[0, 0, 0, 0, 0]) # you may have to change the type long... I haven't tested it
h[257&7]+=257
h[3]|=257
If you absolutely insist on using macros instead then they need to take something with a C++ array interface. A pointer is probably the easiest option and can be got from:
cdef int* h_ptr = &h[0]
#DavidW 's second way of
cdef Card h[5]
h[:] = [0, 0, 0, 0, 0]
cdef Card* h_ptr = &h[0]
also worked once I also adjusted my cdef like so to accept the pointer. Note that the function in the #define macro is not changed and does not have return type specified.
cdef extern from "ace_eval.h":
void ACE_addcard(Card* h, Card c)
This allowed me to pass any of my arrays over flawlessly.
This is actually what it says in the docs, but it was a bit obtuse to me - hopefully my explanation helps someone else.
https://cython.readthedocs.io/en/latest/src/userguide/external_C_code.html
If the header file defines a function using a macro, declare it as though it were an ordinary function, with appropriate argument and result types.
I've been playing with Cython recently for the speed ups, but when I was trying to use copy.deepcopy() some error occurred.Here is the code:
from copy import deepcopy
cdef class cy_child:
cdef public:
int move[2]
int Q
int N
def __init__(self, move):
self.move = move
self.Q = 0
self.N = 0
a = cy_child((1,2))
b = deepcopy(a)
This is the error:
can't pickle _cython_magic_001970156a2636e3189b2b84ebe80443.cy_child objects
How can I solve the problem for this code?
As hpaulj says in the comments, deepcopy looks to use pickle by default to do its work. Cython cdef classes didn't used to be pickleable. In recent versions of Cython they are where possible (see also http://blog.behnel.de/posts/whats-new-in-cython-026.html) but pickling the array looks to be a problem (and even without the array I didn't get it to work).
The solution is to implement the relevant functions yourself. I've done __deepcopy__ since it's simple but alternatively you could implement the pickle protocol
def __deepcopy__(self,memo_dictionary):
res = cy_child(self.move)
res.Q = self.Q
res.N = self.N
return res
I suspect that you won't need to do that in the future as Cython improves their pickle implementation.
A note on memo_dictionary: Suppose you have
a=[None]
b=[A]
a[0]=B
# i.e. A contains a link to B and B contains a link to A
c = deepcopy(a)
memo_dictionary is used by deepcopy to keep a note of what it's already copied so that it doesn't loop forever. You don't need to do much with it yourself. However, if your cdef class contains a Python object (including another cdef class) you should copy it like this:
cdef class C:
cdef object o
def __deepcopy__(self,memo_dictionary):
# ...
res.o = deepcopy(self.o,memo_dictionary)
# ...
(i.e. make sure it gets passed on to further calls of deepcopy)
I have a cython module that uses memoryview arrays, that is...
double[:,:] foo
I want to run this module in parallel using multiprocessing. However I get the error:
PicklingError: Can't pickle <type 'tile_class._memoryviewslice'>: attribute lookup tile_class._memoryviewslice failed
Why can't I pickle a memory view and what can I do about it.
Maybe passing the actual array instead of the memory view can solve your problem.
If you want to execute a function in parallel, all of it parameters have to be picklable if i recall correctly. At least that is the case with python multiprocessing. So you could pass the array to the function and create the memoryview inside your function.
def some_function(matrix_as_array):
cdef double[:,:] matrix = matrix_as_array
...
I don't know if this helps you, but I encountered a similar problem. I use a memoryview as an attribute in a cdef class. I had to write my own __reduce__ and __setstate__ methods to correctly unpickle instances of my class. Pickling the memory view as an array by using numpy.asarray and restoring it in __setstate__ worked for me. A reduced version of my code:
import numpy as np
cdef class Foo:
cdef double[:,:] matrix
def __init__(self, matrix):
'''Assign a passed array to the typed memory view.'''
self.matrix = matrix
def __reduce__(self):
'''Define how instances of Foo are pickled.'''
d=dict()
d['matrix'] = np.asarray(self.matrix)
return (Foo, (d['matrix'],), d)
def __setstate__(self, d):
'''Define how instances of Foo are restored.'''
self.matrix = d['matrix']
Note that __reduce__ returns a tuple consisting of a callable (Foo), a tuple of parameters for that callable (i.e. what is needed to create a 'new' Foo instance, in this case the saved matrix) and the dictionary with all values needed to restore the instance.
The problem involved a JAVA call to a C-function (API) which returned a pointer-to-pointer as an argout argument. I was trying to call the C API from JAVA and I had no way to modify the API.
Using SWIG typemap to pass pointer-to-pointer:
Here is another approach using typemaps. It is targetting Perl, not Java, but the concepts are the same. And I finally managed to get it working using typemaps and no helper functions:
For this function:
typedef void * MyType;
int getblock( int a, int b, MyType *block );
I have 2 typemaps:
%typemap(perl5, in, numinputs=0) void ** data( void * scrap )
{
$1 = &scrap;
}
%typemap(perl5, argout) void ** data
{
SV* tempsv = sv_newmortal();
if ( argvi >= items ) EXTEND(sp,1);
SWIG_MakePtr( tempsv, (void *)*$1, $descriptor(void *), 0);
$result = tempsv;
argvi++;
}
And the function is defined as:
int getblock( int a, int b, void ** data );
In my swig .i file. Now, this passes back an opaque pointer in the argout typemap, becaust that's what useful for this particular situation, however, you could replace the SWIG_MakePtr line with stuff to actually do stuff with the data in the pointer if you wanted to. Also, when I want to pass the pointer into a function, I have a typemap that looks like this:
%typemap(perl5, in) void * data
{
if ( !(SvROK($input)) croak( "Not a reference...\n" );
if ( SWIG_ConvertPtr($input, (void **) &$1, $1_descriptor, 0 ) == -1 )
croak( "Couldn't convert $1 to $1_descriptor\n");
}
And the function is defined as:
int useblock( void * data );
In my swig .i file.
Obviously, this is all perl, but should map pretty directly to Java as far as the typemap architecture goes. Hope it helps...
[Swig] Java: Using C helper function to pass pointer-to-pointer
The problem involved a JAVA call to a C-function (API) which returned a pointer-to-pointer as an argout argument. I was trying to call the C API from JAVA and I had no way to modify the API.
The API.h header file contained:
extern int ReadMessage(HEADER **hdr);
The original C-call looked like:
HEADER *hdr;
int status;
status = ReadMessage(&hdr);
The function of the API was to store data at the memory location specified by the pointer-to-pointer.
I tried to use SWIG to create the appropriate interface file. SWIG.i created the file SWIGTYPE_p_p_header.java from API.h. The problem is the SWIGTYPE_p_p_header constructor initialized swigCPtr to 0.
The JAVA call looked like:
SWIGTYPE_p_p_header hdr = new SWIGTYPE_p_p_header();
status = SWIG.ReadMessage(hdr);
But when I called the API from JAVA the ptr was always 0.
I finally gave up passing the pointer-to-pointer as an input argument. Instead I defined another C-function in SWIG.i to return the pointer-to-pointer in a return value. I thought it was a Kludge ... but it worked!
You may want to try this:
SWIG.i looks like:
// return pointer-to-pointer
%inline %{
HEADER *ReadMessageHelper() {
HEADER *hdr;
int returnValue;
returnValue = ReadMessage(&hdr);
if (returnValue!= 1) hdr = NULL;
return hdr;
}%}
The inline function above could leak memory as Java won't take ownership of the memory created by ReadMessageHelper, since the HEADER instance iscreated on the heap.
The fix for the memory leak is to define ReadMessageHelper as a newobject in order for Java to take control of the memory.
%newobject ReadMessageHelper();
JAVA call now would look like:
HEADER hdr;
hdr = SWIG.ReadMessageHelper();
If you are lucky, as I was, you may have another API available to release the message buffer. In which case, you wouldn’t have to do the previous step.
William Fulton, the SWIG guru, had this to say about the approach above:
“I wouldn't see the helper function as a kludge, more the simplest solution to a tricky problem. Consider what the equivalent pure 100% Java code would be for ReadMessage(). I don't think there is an equivalent as Java classes are passed by reference and there is no such thing as a reference to a reference, or pointer to a pointer in Java. In the C function you have, a HEADER instances is created by ReadMessage and passed back to the caller. I don't see how one can do the equivalent in Java without providing some wrapper class around HEADER and passing the wrapper to the ReadMessage function. At the end of the day, ReadMessage returns a newly created HEADER and the Java way of returning newly created objects is to return it in the return value, not via a parameter.”