fipy - level set 1D - numerical-methods

I am trying to run the code below for a level set 1D problem (example in fipy webpage). I am getting this error:
Traceback (most recent call last):
File "C:/Users/sgowda/Documents/pde solver code/level set 1D.py", line 20, in
var.calcDistanceFunction()
File "C:\Users\sgowda\AppData\Local\Continuum\Anaconda\lib\site-packages\fipy\variables\distanceVariable.py", line 335, in calcDistanceFunction
raise Exception, "Neither lsmlib nor skfmm can be found on the $PATH"
Exception: Neither lsmlib nor skfmm can be found on the $PATH
Could you please let me know how to fix this. i tried looking into the distancefucntion() but im not sure what this error means?
from fipy import Grid1D, CellVariable, TransientTerm, DiffusionTerm, Viewer, DistanceVariable
import matplotlib.pyplot as plt
velocity = 1.
dx = 1.
nx = 10
timeStepDuration = 1.
steps = 2
L = nx * dx
interfacePosition = L / 5.
from fipy.tools import serialComm
mesh = Grid1D(dx=dx, nx=nx, communicator=serialComm)
var = DistanceVariable(name='level set variable',
mesh=mesh,
value=-1.,
hasOld=1)
var.setValue(1., where=mesh.cellCenters[0] > interfacePosition)
var.calcDistanceFunction()
advEqn = TransientTerm() + FirstOrderAdvectionTerm(velocity)
viewer = Viewer(vars=var, datamin=-10., datamax=10.)
viewer.plot()
for step in range(steps):
var.updateOld()
advEqn.solve(var, dt=timeStepDuration)
viewer.plot()
plt.show()

FiPy doesn't have a native level set implementation so uses either LSMLIB or Scikit-fmm to provide the level set / fast marching method functionality.
To see whether you have them installed correctly, use either
$ python -c “import pylsmlib; pylsmlib.test()”
or
$ python -c “import skfmm; skfmm.test()”
to test.
The requirement is outlined in the FiPy documentation, see http://www.ctcms.nist.gov/fipy/INSTALLATION.html#level-set-packages
It is probably easier to install Scikit-fmm initially, see https://pythonhosted.org/scikit-fmm/, but
$ pip install scikit-fmm
should work.

Related

convert pb file to tflite file for running on Coral Dev Board (Segmentation fault (core dumped) )

How to convert a pb file into tflite file using python3 or in terminal.
Don't know any of the details of the model.Here is the link of the pb file.
(Edited)
I have done converting pb file to tflite file by the following code :
import tensorflow.compat.v1 as tf
import numpy as np
graph_def_file = "./models/20170512-110547.pb"
def representative_dataset_gen():
for _ in range(num_calibration_steps):
# Get sample input data as a numpy array in a method of your choosing.
yield [input]
converter = tf.lite.TFLiteConverter.from_frozen_graph(graph_def_file,
input_arrays=["input","phase_train"],
output_arrays=["embeddings"],
input_shapes={"input":[1,160,160,3],"phase_train":False})
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = representative_dataset_gen
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
tflite_model = converter.convert()
print("converting")
open("./models/converted_model.tflite", "wb").write(tflite_model)
print("Done")
Error :Getting Segmentation fault(core dumped)
2020-01-20 11:42:18.153263: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer.so.6'; dlerror: libnvinfer.so.6: cannot open shared object file: No such file or directory
2020-01-20 11:42:18.153363: W tensorflow/stream_executor/platform/default/dso_loader.cc:55] Could not load dynamic library 'libnvinfer_plugin.so.6'; dlerror: libnvinfer_plugin.so.6: cannot open shared object file: No such file or directory
2020-01-20 11:42:18.153385: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:30] Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
2020-01-20 11:42:18.905028: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-01-20 11:42:18.906845: E tensorflow/stream_executor/cuda/cuda_driver.cc:351] failed call to cuInit: CUDA_ERROR_NO_DEVICE: no CUDA-capable device is detected
2020-01-20 11:42:18.906874: I tensorflow/stream_executor/cuda/cuda_diagnostics.cc:156] kernel driver does not appear to be running on this host (kalgudi-GA-78LMT-USB3-6-0): /proc/driver/nvidia/version does not exist
2020-01-20 11:42:18.934144: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3616020000 Hz
2020-01-20 11:42:18.934849: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x39aa0f0 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-01-20 11:42:18.934910: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version
Segmentation fault (core dumped)
with out any details of the model, you cannot convert it to a .tflite model. I suggest that you go through this document for Post Training Quantization again. As there are too many details that is redundant to show here.
Here is an example for post training quantization of a frozen graph. The model is taken from here (you can see that it's a full tarball of infomation about the model)
import sys, os, glob
import tensorflow as tf
import pathlib
import numpy as np
if len(sys.argv) != 2:
print('Usage: <' + sys.argv[0] + '> <frozen_graph_file> <representative_image_dir>')
exit()
tf.compat.v1.enable_eager_execution()
tf.compat.v1.logging.set_verbosity(tf.compat.v1.logging.DEBUG)
def fake_representative_data_gen():
for _ in range(100):
fake_image = np.random.random((1,192,192,3)).astype(np.float32)
yield [fake_image]
frozen_graph = sys.argv[1]
input_array = ['input']
output_array = ['MobilenetV1/Predictions/Reshape_1']
converter = tf.compat.v1.lite.TFLiteConverter.from_frozen_graph(frozen_graph, input_array, output_array)
converter.optimizations = [tf.lite.Optimize.DEFAULT]
converter.representative_dataset = fake_representative_data_gen
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS_INT8]
converter.inference_input_type = tf.uint8
converter.inference_output_type = tf.uint8
tflite_model = converter.convert()
quant_dir = pathlib.Path(os.getcwd(), 'output')
quant_dir.mkdir(exist_ok=True, parents=True)
tflite_model_file = quant_dir/'mobilenet_v1_0.25_192_quant.tflite'
tflite_model_file.write_bytes(tflite_model)

FiPy Simple Convection

I am trying to understand how FiPy works by working an example, in particular I would like to solve the following simple convection equation with periodic boundary:
$$\partial_t u + \partial_x u = 0$$
If initial data is given by $u(x, 0) = F(x)$, then the analytical solution is $u(x, t) = F(x - t)$. I do get a solution, but it is not correct.
What am I missing? Is there a better resource for understanding FiPy than the documentation? It is very sparse...
Here is my attempt
from fipy import *
import numpy as np
# Generate mesh
nx = 20
dx = 2*np.pi/nx
mesh = PeriodicGrid1D(nx=nx, dx=dx)
# Generate solution object with initial discontinuity
phi = CellVariable(name="solution variable", mesh=mesh)
phiAnalytical = CellVariable(name="analytical value", mesh=mesh)
phi.setValue(1.)
phi.setValue(0., where=x > 1.)
# Define the pde
D = [[-1.]]
eq = TransientTerm() == ConvectionTerm(coeff=D)
# Set discretization so analytical solution is exactly one cell translation
dt = 0.01*dx
steps = 2*int(dx/dt)
# Set the analytical value at the end of simulation
phiAnalytical.setValue(np.roll(phi.value, 1))
for step in range(steps):
eq.solve(var=phi, dt=dt)
print(phi.allclose(phiAnalytical, atol=1e-1))
As addressed on the FiPy mailing list, FiPy is not great at handling convection only PDEs (absent diffusion, pure hyperbolic) as it's missing higher order convection schemes. It is better to use CLAWPACK for this class of problem.
FiPy does have one second order scheme that might help with this problem, the VanLeerConvectionTerm, see an example.
If the VanLeerConvectionTerm is used in the above problem, it does do a better job of preserving the shock.
import numpy as np
import fipy
# Generate mesh
nx = 20
dx = 2*np.pi/nx
mesh = fipy.PeriodicGrid1D(nx=nx, dx=dx)
# Generate solution object with initial discontinuity
phi = fipy.CellVariable(name="solution variable", mesh=mesh)
phiAnalytical = fipy.CellVariable(name="analytical value", mesh=mesh)
phi.setValue(1.)
phi.setValue(0., where=mesh.x > 1.)
# Define the pde
D = [[-1.]]
eq = fipy.TransientTerm() == fipy.VanLeerConvectionTerm(coeff=D)
# Set discretization so analytical solution is exactly one cell translation
dt = 0.01*dx
steps = 2*int(dx/dt)
# Set the analytical value at the end of simulation
phiAnalytical.setValue(np.roll(phi.value, 1))
viewer = fipy.Viewer(phi)
for step in range(steps):
eq.solve(var=phi, dt=dt)
viewer.plot()
raw_input('stopped')
print(phi.allclose(phiAnalytical, atol=1e-1))

How to calculate the the number and area of habitat patches in Arcview 10

I'm currently working on my masters thesis and having real trouble with GIS. I've downloaded the arc gis grid data set from http://www.kew.org/gis/projects/mad_veg/datasets_gis.html
Ive sucessfully plotted it in arcmap 10. The map consists of various different habitats. I want to know how I could take one of those habitats types, say "humid forest", and calculate how many patches of that habitat there are, and how big each patch is.
I've been been at this for weeks and haven't made much headway. someone suggested I look at zonal geometry as a table http://help.arcgis.com/en/arcgisdesktop/10.0/help/index.html#//009z000000w5000000.htm which look promising but I gave the coding a try and I couldnt get it to work. I posted some of my attempts below.
>>> import arcpy
>>> from arcpy import env
>>> from arcpy.sa import *
>>> env.workspace = "Q:/MADGIS"
>>> outZonalGeometryAsTable = ZonalGeometryAsTable("zones.shp", "Classes "zonalgeomout", 0.2)
Runtime error <class 'arcgisscripting.ExecuteError'>: ERROR 000626: Tool ZonalGeometryAsTable is not licensed.
>>> arcpy.CheckOutExtension("Spatial")
u'CheckedOut'
>>> outZonalGeometryAsTable = ZonalGeometryAsTable(inZoneData, zoneField, "AREA", cellSize)
Runtime error <type 'exceptions.NameError'>: name 'inZoneData' is not defined
The problem is some of the things ive copied in the example are specific to the example, but i'm not sure. If someone could even point me in the right direction it would be a big help
It seems that you didn’t set some parameters.
According to the link above, you must set this parameters:
# Set local variables
inZoneData = "YourShapefileName.shp"
zoneField = "Classes"
outTable = "zonalgeomout02.dbf"
processingCellSize = 0.2
# Check out the ArcGIS Spatial Analyst extension license
arcpy.CheckOutExtension("Spatial")
Update:
You must use this code for your raster data:
import arcpy
from arcpy import env
from arcpy.sa import *
env.workspace = "C:/Users/Puya/Downloads/Documents/StackOverflow/veg_grid"
inZoneData = "vegetation"
zoneField = "Value"
outTable = "zonalgeomout02.dbf"
processingCellSize = 29
arcpy.CheckOutExtension("Spatial")
outZonalGeometryAsTable = ZonalGeometryAsTable(inZoneData, zoneField, "AREA", processingCellSize)
Also, in ArcMap you can use ArcToolbox -> Spatial Analyst -> Zonal -> ZonalGeometryAsTable and select above parameters and run ZonalGeometryAsTable.

Loading CPP functions into Python using Ctypes

I have a question to Ctypes and do not know what I do wrong. And yes, I am a newbie for Python and searched the other posts in here. So any advice is appreciated.
What do I want to do :
I simply want to load the FXCM C++ APP fundtions into Python 3.3 so I can call them for connecting to their server.
As it seems Ctypes seems to be the best tool. So a simple code in Python :
import os
dirlist = os.listdir('ForexConnectAPIx64/bin')
from pprint import pprint
pprint(dirlist)
from ctypes import *
myDll = cdll.LoadLibrary ("ForexConnectAPIx64/bin/ForexConnect.dll")
gives a result :
Traceback (most recent call File "C:\Users\scaberia3\Python_Projects \FTC\ListDir_Test.py", line 20, in <module>
myDll = cdll.LoadLibrary ("ForexConnectAPIx64/bin/ForexConnect.dll")
File "C:\Python33\lib\ctypes\__init__.py", line 431, in LoadLibrary
return self._dlltype(name)
File "C:\Python33\lib\ctypes\__init__.py", line 353, in __init__
self._handle = _dlopen(self._name, mode)
OSError: [WinError 126] Das angegebene Modul wurde nicht gefunden (Module not found)
['ForexConnect.dll',
'fxmsg.dll',
'gsexpat.dll',
'gslibeay32.dll',
'gsssleay32.dll',
'gstool2.dll',
'gszlib.dll',
'java',
'log4cplus.dll',
'msvcp80.dll',
'msvcr80.dll',
'net',
'pdas.dll']
Means the path is correct ForextConnect.dll is present and I might do some very simple wrong, but have no clue what.
You can either use Dependency Walker to figure out the correct sequence to manually load the DLLs, or simply add the directory to the system search path:
dllpath = os.path.abspath('ForexConnectAPIx64/bin')
os.environ['PATH'] += os.pathsep + dllpath
myDLL = CDLL('ForexConnect.dll')

How do you read a file inside a zip file as text, not bytes?

A simple program for reading a CSV file inside a ZIP archive:
import csv, sys, zipfile
zip_file = zipfile.ZipFile(sys.argv[1])
items_file = zip_file.open('items.csv', 'rU')
for row in csv.DictReader(items_file):
pass
works in Python 2.7:
$ python2.7 test_zip_file_py3k.py ~/data.zip
$
but not in Python 3.2:
$ python3.2 test_zip_file_py3k.py ~/data.zip
Traceback (most recent call last):
File "test_zip_file_py3k.py", line 8, in <module>
for row in csv.DictReader(items_file):
File "/somedir/python3.2/csv.py", line 109, in __next__
self.fieldnames
File "/somedir/python3.2/csv.py", line 96, in fieldnames
self._fieldnames = next(self.reader)
_csv.Error: iterator should return strings, not bytes (did you open the file
in text mode?)
The csv module in Python 3 wants to see a text file, but zipfile.ZipFile.open returns a zipfile.ZipExtFile that is always treated as binary data.
How does one make this work in Python 3?
I just noticed that Lennart's answer didn't work with Python 3.1, but it does work with Python 3.2. They've enhanced zipfile.ZipExtFile in Python 3.2 (see release notes). These changes appear to make zipfile.ZipExtFile work nicely with io.TextWrapper.
Incidentally, it works in Python 3.1, if you uncomment the hacky lines below to monkey-patch zipfile.ZipExtFile, not that I would recommend this sort of hackery. I include it only to illustrate the essence of what was done in Python 3.2 to make things work nicely.
$ cat test_zip_file_py3k.py
import csv, io, sys, zipfile
zip_file = zipfile.ZipFile(sys.argv[1])
items_file = zip_file.open('items.csv', 'rU')
# items_file.readable = lambda: True
# items_file.writable = lambda: False
# items_file.seekable = lambda: False
# items_file.read1 = items_file.read
items_file = io.TextIOWrapper(items_file)
for idx, row in enumerate(csv.DictReader(items_file)):
print('Processing row {0} -- row = {1}'.format(idx, row))
If I had to support py3k < 3.2, then I would go with the solution in my other answer.
Update for 3.6+
Starting w/3.6, support for mode='U' was removed^1:
Changed in version 3.6: Removed support of mode='U'. Use io.TextIOWrapper for reading compressed text files in universal newlines mode.
Starting w/3.8, a Path object was added which gives us an open() method that we can call like the built-in open() function (passing newline='' in the case of our CSV) and we get back an io.TextIOWrapper object the csv readers accept. See Yuri's answer, here.
You can wrap it in a io.TextIOWrapper.
items_file = io.TextIOWrapper(items_file, encoding='your-encoding', newline='')
Should work.
And if you just like to read a file into a string:
with ZipFile('spam.zip') as myzip:
with myzip.open('eggs.txt') as myfile:
eggs = myfile.read().decode('UTF-8'))
Lennart's answer is on the right track (Thanks, Lennart, I voted up your answer) and it almost works:
$ cat test_zip_file_py3k.py
import csv, io, sys, zipfile
zip_file = zipfile.ZipFile(sys.argv[1])
items_file = zip_file.open('items.csv', 'rU')
items_file = io.TextIOWrapper(items_file, encoding='iso-8859-1', newline='')
for idx, row in enumerate(csv.DictReader(items_file)):
print('Processing row {0}'.format(idx))
$ python3.1 test_zip_file_py3k.py ~/data.zip
Traceback (most recent call last):
File "test_zip_file_py3k.py", line 7, in <module>
items_file = io.TextIOWrapper(items_file,
encoding='iso-8859-1',
newline='')
AttributeError: readable
The problem appears to be that io.TextWrapper's first required parameter is a buffer; not a file object.
This appears to work:
items_file = io.TextIOWrapper(io.BytesIO(items_file.read()))
This seems a little complex and also it seems annoying to have to read in a whole (perhaps huge) zip file into memory. Any better way?
Here it is in action:
$ cat test_zip_file_py3k.py
import csv, io, sys, zipfile
zip_file = zipfile.ZipFile(sys.argv[1])
items_file = zip_file.open('items.csv', 'rU')
items_file = io.TextIOWrapper(io.BytesIO(items_file.read()))
for idx, row in enumerate(csv.DictReader(items_file)):
print('Processing row {0}'.format(idx))
$ python3.1 test_zip_file_py3k.py ~/data.zip
Processing row 0
Processing row 1
Processing row 2
...
Processing row 250
Starting with Python 3.8, the zipfile module has the Path object, which we can use with its open() method to get an io.TextIOWrapper object, which can be passed to the csv readers:
import csv, sys, zipfile
# Give a string path to the ZIP archive, and
# the archived file to read from
items_zipf = zipfile.Path(sys.argv[1], at='items.csv')
# Then use the open method, like you'd usually
# use the built-in open()
items_f = items_zipf.open(newline='')
# Pass the TextIO-like file to your reader as normal
for row in csv.DictReader(items_f):
print(row)
Here's a minimal recipe to open a zip file and read a text file inside that zip. I found the trick to be the TextIOWrapper read() method, not mentioned in any answers above (BytesIO.read() was mentioned above, but Python docs recommend TextIOWrapper).
import zipfile
import io
# Create the ZipFile object
zf = zipfile.ZipFile('my_zip_file.zip')
# Read a file that is inside the zip...reads it as a binary file-like object
my_file_binary = zf.open('my_text_file_inside_zip.txt')
# Convert the binary file-like object directly to text using TextIOWrapper and it's read() method
my_file_text = io.TextIOWrapper(my_file_binary, encoding='utf-8', newline='').read()
I wish they kept the mode='U' parameter in the ZipFile open() method to do this same thing since that was so succinct but, alas, that is obsolete.