I have a geotiff file which overlaps with a shapefile. To clip for the overlapping part of the tif file, I can do this:
gdalwarp -co compress=deflate -dstnodata 255 -cutline shapefile.shp original.tif overlap.tif
But how can I clip for the non-intersecting part? That is, I want to create the complement of "overlap.tif" w.r.t. "original.tif".
You can use gdal_rasterize to burn a value where the shapefile overlaps the file. It works on an existing file, so make sure you use a copy.
gdal_rasterize -burn 255 shapefile.shp copy_of_original.tif
This burns a value of 255, setting -a_nodata 255 doesnt work on my version of GDAL. If you need it to be a real nodata value using gdal_translate with -a_nodata 255 afterwards would do the trick.
Gdal_rasterize also has a convenient -i flag which inverts the shapefile.
Related
I have been trying to convert a set of GeoTIFF files into MBTiles using gdal_translate (GDAL 3.0.4). My command looks as follows:
gdal_translate -of MBTiles -ot Byte -strict -scale 0 255 bogota.tif bogota.mbtiles
The GeoTIFF image is successfully converted to MBTiles, and I am able to render it using QGIS. However, it appears that the result is somewhat compressed, or that the new image has lost some resolution. I have been experimenting with the -outsize option and trying to force it to 100% of the original size of the image, but with no success.
Is there a way to make sure that the result maintains the full resolution in the output?
Here are some screenshots to compare the results:
Before
After
Note: GeoTIFF image is taken from the following link:
https://download.osgeo.org/geotiff/samples/made_up/bogota.tif
I'm trying to import a raster grid to NetLogo but am encountering many issues. My raster file is only 57x41 pixels (I want each pixel here to represent a patch) and the world envelope is [-382875 -381135 700185 701445]. I am also trying to match my raster-dataset value to the patch variable fuel-code in a .csv file. However when I run the code (below) I encounter errors. I'm not using a set coordinate projection in netlogo since my original raster is not in an acceptable projection type for NetLogo (I removed the .prj file associated with the raster when importing the .asc file). Below is my code (with included error messages to the code I tried to edit):
extensions [ csv table gis]
globals [ fuel-type-40 fuel-code setrial1]
to dictionary-file ;put in the setup procedure
ca
;load the ascii file
set setrial1 gis:load-dataset "setrial_ascii.asc"
;match dimensions of raster to the dimensions of the Netlogo world
;I've tried each of the below codes independently, not together
resize-world 0 gis:width-of setrial1 0 gis:height-of setrial1 ;ERROR: Java Heap space error
gis:set-world-envelope gis:envelope-of setrial1 ;ERROR: can't modify a patch's coordinates
;below is visuals of width and height of setrial1
print gis:height-of setrial1 ;41
print gis:width-of setrial1 ;57
print envelope-of setrial1 ;[-382875 -381135 700185 701445]
; Load the csv
set fuel-type-40 but-first csv:from-file "fuel-type-40.csv"
;print fuel-type-40
; Pull first value (Fuel-code)
set fuel-code map first fuel-type-40
;print fuel-code
ask patches [
; Randomly set patch 'land cover' for this example. change for raster
gis:apply-raster setrial1 fuel-code
]
end
You might report which errors you are having. Remember to always read the stackoverflow guide to asking questions.
I'd suggest you to focus on using a rasterfile with all your data on it (ESRI files have a .dbf file which supports data), and thus avoid using both extensions. By having a raster file with a defined resolution (e.g. 100 x 100 m), the resize-world function should work smoothly. Try following my answer to this question.
I am relatively new to machine learning/python/ubuntu.
I have a set of images in .jpg format where half contain a feature I want caffe to learn and half don't. I'm having trouble in finding a way to convert them to the required lmdb format.
I have the necessary text input files.
My question is can anyone provide a step by step guide on how to use convert_imageset.cpp in the ubuntu terminal?
Thanks
A quick guide to Caffe's convert_imageset
Build
First thing you must do is build caffe and caffe's tools (convert_imageset is one of these tools).
After installing caffe and makeing it make sure you ran make tools as well.
Verify that a binary file convert_imageset is created in $CAFFE_ROOT/build/tools.
Prepare your data
Images: put all images in a folder (I'll call it here /path/to/jpegs/).
Labels: create a text file (e.g., /path/to/labels/train.txt) with a line per input image . For example:
img_0000.jpeg 1
img_0001.jpeg 0
img_0002.jpeg 0
In this example the first image is labeled 1 while the other two are labeled 0.
Convert the dataset
Run the binary in shell
~$ GLOG_logtostderr=1 $CAFFE_ROOT/build/tools/convert_imageset \
--resize_height=200 --resize_width=200 --shuffle \
/path/to/jpegs/ \
/path/to/labels/train.txt \
/path/to/lmdb/train_lmdb
Command line explained:
GLOG_logtostderr flag is set to 1 before calling convert_imageset indicates the logging mechanism to redirect log messages to stderr.
--resize_height and --resize_width resize all input images to same size 200x200.
--shuffle randomly change the order of images and does not preserve the order in the /path/to/labels/train.txt file.
Following are the path to the images folder, the labels text file and the output name. Note that the output name should not exist prior to calling convert_imageset otherwise you'll get a scary error message.
Other flags that might be useful:
--backend - allows you to choose between an lmdb dataset or levelDB.
--gray - convert all images to gray scale.
--encoded and --encoded_type - keep image data in encoded (jpg/png) compressed form in the database.
--help - shows some help, see all relevant flags under Flags from tools/convert_imageset.cpp
You can check out $CAFFE_ROOT/examples/imagenet/convert_imagenet.sh
for an example how to use convert_imageset.
I've been using ogr2ogr to do most of what I need with shapefiles (including dissolving them). However, I find that for big ones, it takes a REALLY long time.
Here's an example of what I'm doing:
ogr2ogr new.shp old.shp -dialect sqlite -sql "SELECT ST_Union(geometry) FROM old"
In certain instances, one might want to dissolve common neighboring shapes (which is what I think is going on here in the above command). However, in my case I simply want to flatten the entire file and every shape in it regardless of the values (I've already isolated the shapes I need).
Is there a faster way to do this when you don't need to care about the values and just want a shape that outlines the array of shapes in the file?
If you have isolated the shapes, and they don't have any shared boundaries, they can be easily collected into a single MULTIPOLYGON using ST_Collect. This should be really fast and simple to do:
ogr2ogr gcol.shp old.shp -dialect sqlite -sql "SELECT ST_Collect(geometry) FROM old"
If the geometries overlap and the boundaries need to be "dissolved", then ST_Union must be used. Faster spatial unions are done with a cascaded union technique, described here for PostGIS. It is supported by OGR, but it doesn't seem to be done elegantly.
Here is a two step SQL query. First make a MULTIPOLYGON of everything with ST_Collect (this is fast), then do a self-union which should trigger a UnionCascaded() call.
ogr2ogr new.shp old.shp -dialect sqlite -sql "SELECT ST_Union(gcol, gcol) FROM (SELECT ST_Collect(geometry) AS gcol FROM old) AS f"
Or to better view the actual SQL statement:
SELECT ST_Union(gcol, gcol)
FROM (
SELECT ST_Collect(geometry) AS gcol
FROM old
) AS f
I've had better success (i.e. faster) by converting it to raster then back to vector. For example:
# convert the vector file old.shp to a raster file new.tif using a pixel size of XRES/YRES
gdal_rasterize -tr XRES YRES -burn 255 -ot Byte -co COMPRESS=DEFLATE old.shp new.tif
# convert the raster file new.tif to a vector file new.shp, using the same raster as a -mask speeds up the processing
gdal_polygonize.py -f 'ESRI Shapefile' -mask new.tif new.tif new.shp
# removes the DN attribute created by gdal_polygonize.py
ogrinfo new.shp -sql "ALTER TABLE new DROP COLUMN DN"
I want to use gdal_rasterize to generate a TIFF from a .shp shapefile. Usually the result is big, so I want to compress it using the LZW compress option.
I tried to do so with the command
gdal_rasterize.exe -burn 255 -burn 255 -burn 0 -burn 255 -ot Byte -tr 0.0332147 0.0332147 shp.shp shp0.tif --config COMPRESS LZW
but it seems the --config COMPRESS LZW option doesn't have any effect. (The result is exactly the same size as without the option.)
Maybe I have some misunderstanding of how to use this option.
You should add an = symbol between the option and the value. Without your data i cant test your specific example, but for me this fails:
gdal_translate --config COMPRESS LZW infile.tif outfile.tif
and this works fine:
gdal_translate --config COMPRESS=LZW infile.tif outfile.tif
You can also write the --config as -co, and wrapping it with quotes also works, which is how i usually do it.
gdal_translate -co "COMPRESS=LZW" infile.tif outfile.tif