I would like to transform some coordinates from EPSG 21781 to EPSG 2056. If I run projinfo on these to projections, i get the information that a certain grid is needed but not found on the system (see below).
$ projinfo -o PROJ -s EPSG:21781 -t EPSG:2056
Candidate operations found: 1
-------------------------------------
Operation No. 1:
unknown id, Inverse of Swiss Oblique Mercator 1903M + CH1903 to CH1903+ (1) + Swiss Oblique Mercator 1995, 0.2 m, Europe - Liechtenstein and Switzerland, at least one grid missing
PROJ string:
+proj=pipeline +step +inv +proj=somerc +lat_0=46.9524055555556 +lon_0=7.43958333333333 +k_0=1 +x_0=600000 +y_0=200000 +ellps=bessel +step +proj=hgridshift +grids=CHENyx06a.gsb +step +proj=somerc +lat_0=46.9524055555556 +lon_0=7.43958333333333 +k_0=1 +x_0=2600000 +y_0=1200000 +ellps=bessel
Grid CHENyx06a.gsb needed but not found on the system. Can be obtained from the proj-datumgrid-europe package at https://download.osgeo.org/proj/proj-datumgrid-europe-1.5.zip
After I have downloaded the specified file (proj-datumgrid-europe-1.5.zip), what do I need to do with it? This does not seem to be described in the docs.
I'm working on Ubuntu 20.04 and proj 6.3.1
$ pkg-config --modversion proj
6.3.1
For the PROJ version you are using (6.3.1), the proj-datumgrid docs say that you just need to unzip the file into the PROJ data directory, which is either /usr/local/share/proj or /usr/share/proj
Related
I have QGIS 2.18 (latest version) installed for windows (new users). Along came OSGeo4W Shell. Now using this shell, I want to convert a specific value in one CRS into another. For example, if I know coordinates in WGS84 (say, 91.7362, 26.1445 just to give an example), I would like to know how to convert it to Indian 1954/UTM Zone 46N (which are in meters) using OSGeoShell.
PS: I know there is a way because I once successfully found the way. I had copied the syntax of the command but I deleted the file by mistake and I can't find the way in net again even after long time searches. It was barely a 2 line and simple command.
I think, the command is:
osgeo4w
gdaltransform -s_srs EPSG:4326 -t_srs EPSG:XXXX < input.csv > output.txt
Where the EPSG codes are the codes for the CRS (4326 is for WGS84). You have to find out the epsg code for your target crs and then you can perform the transformation.
I want to get a full control flow graph of a binary (malware) using radare2.
I followed this post from another question on SO. I wanted to ask if instead of ag there is another command that gives the control flow graph of the whole binary and not only the graph of one function.
First of all, make sure to install radare2 from git repository and use the newest version:
$ git clone https://github.com/radare/radare2.git
$ cd radare2
$ ./sys/install.sh
After you've downloaded and installed radare2, open your binary and perform analysis on it using the aaa command:
$ r2 /bin/ls
-- We fix bugs while you sleep.
[0x004049a0]> aaa
[x] Analyze all flags starting with sym. and entry0 (aa)
[x] Analyze function calls (aac)
[x] Analyze len bytes of instructions for references (aar)
[x] Check for objc references
[x] Check for vtables
[x] Type matching analysis for all functions (aaft)
[x] Propagate noreturn information
[x] Use -AA or aaaa to perform additional experimental analysis.
Adding ? after almost every command in radare will output the subcommands. For example, you know that the ag command and its subcommands can help you to output the visual graphs so by adding ? to ag you can discover its subcommands:
[0x00000000]> ag?
Usage: ag<graphtype><format> [addr]
Graph commands:
| aga[format] Data references graph
| agA[format] Global data references graph
| agc[format] Function callgraph
| agC[format] Global callgraph
| agd[format] [fcn addr] Diff graph
... <truncated> ...
Output formats:
| <blank> Ascii art
| * r2 commands
| d Graphviz dot
| g Graph Modelling Language (gml)
| j json ('J' for formatted disassembly)
| k SDB key-value
| t Tiny ascii art
| v Interactive ascii art
| w [path] Write to path or display graph image (see graph.gv.format and graph.web)
You're searching for the agCd command which will output a full call-graph of the program in dot format.
[0x004049a0]> agCd > output.dot
The dot utility is part of the Graphviz software which can be installed using sudo apt-get install graphviz.
You can view your output in any offline dot viewer, paste the output into an online Graphviz viewer and even convert the dot file to PNG:
$ r2 /bin/ls
[0x004049a0]> aa
[x] Analyze all flags starting with sym. and entry0 (aa)
[0x004049a0]> agCd > output.dot
[0x004049a0]> !!dot -Tpng -o callgraph.png output.dot
I would like to import data from a CSV file to use in scikit-learn. It has a mix of numerical data categorical data, e.g.
someValue,color,someOtherValue
1.2,red,55.6
1.9,blue,20.5
3.2,red,16.5
I need to convert this representation into a purely numerical one where categorical data points get converted into multiple binary columns, e.g.
someValue,colorIsRed,colorIsBlue,someOtherValue
1.2,1,0,55.6
1.9,0,1,20.5
3.2,1,0,16.5
Is there any utility that does this for me, or an easy way to iterate through the data and get this representation?
scikit-learn doesn't offer data-loading functions as far as I know, but it does prefer Numpy arrays as input. Numpy's loadtxt function together with its converters parameter can be used to load your csv and specify the types of each column. It does not binarize your second column though.
In this answer, I'm assuming that you're trying to convert your CSV into a file that LibSVM, LIBLINEAR, or scikit-learn can load.
You can use csv2libsvm, which is provided as part of the Ruby gem vector_embed:
$ gem install vector_embed
Successfully installed vector_embed-0.1.0
1 gem installed
You need Ruby 1.9+...
$ ruby -v
ruby 1.9.3p374 (2013-01-15 revision 38858) [x86_64-darwin12.2.0]
If you don't have Ruby 1.9, it's easy to install with rvm, which does not require (or recommend using) root:
$ curl -#L https://get.rvm.io | bash -s stable
$ rvm install 1.9.3
Once you have successfully run gem install vector_embed, make sure your first column is called "label":
$ cat example.csv
label,color,someOtherValue
1.2,red,55.6
1.9,blue,20.5
3.2,red,16.5
$ csv2libsvm example.csv > example.libsvm
$ cat example.libsvm
1.2 1139043:55.6 1997960:1
1.9 1089740:1 1139043:20.5
3.2 1139043:16.5 1997960:1
Note that it handles both categorical and continuous data, and that it uses MurmurHash version 3 to generate the feature names ("colorIsBlue" corresponds to 1089740, "colorIsRed" is 1997960... though the Ruby code is really hashing something like "color\0red").
If you're using svm, be sure to scale your data like they recommend in "A practical guide to SVM classification".
Finally, let's say you're using scikit-learn's svmlight/libsvm loader:
>>> from sklearn.datasets import load_svmlight_file
>>> X_train, y_train = load_svmlight_file("/path/to/example.libsvm")
I have an ascii text file containing location data (column 9-lat and 10-long) and intensity(column 20)
200010 207 020311 40658.5 406593 52 344927.31 7100203.50 -26.2078720 127.4491855 345060.64 7100369.14 26.4 650.3 628.0 55471.293 20.168 55648.817 55637.523 -146.062
the text file has many lines 10k+
I am trying to visualize this using GDAL, but not sure how to proceed.
Ideas?
Try QGis. It is free software for making maps with data.
GDAL is for doing sophisticated data transformations.
If your file is named viz.txt, then you can extract
and plot the data using the following commands:
$ awk '{print $9, $10, $20}' < viz.txt > viz2.txt
$ gnuplot
...
gnuplot> plot "viz2.txt" with points palette
This will give you a chart, nicely coloured by intensity.
If you want a more interactive solution, or to overlay the
data on a map, then you will have to use GIS software such
as ArcView, MapInfo or the free tools Generic Mapping Tools (GMT) or QGIS.
Can anyone see anything wrong with the following set of commands? Every time I run these image.png is a image of the UK and not the JOSM map I exported. I'm guessing there's something awry with the db import however the output mentions that it's processing my coords and data.
Steps:
1 - Exported a .osm file from JOSM or Merkaator.
2 - Imported into psql using the following command:
osm2pgsql -m -d gis -S ~/mapnik/default.style -b 103,1.3,104,1.4 ion.osm -v -c
The output for this looks like:
marshall#ubuntu:~/mapnik$ osm2pgsql -m -d gis -S ~/mapnik/default.style -b 103,1.3,104,1.4 ion.osm -v -c
osm2pgsql SVN version 0.66-
Using projection SRS 900913 (Spherical Mercator)
Applying Bounding box: 103.000000,1.300000 to 104.000000,1.400000
Setting up table: planet_osm_point
Setting up table: planet_osm_line
Setting up table: planet_osm_polygon
Setting up table: planet_osm_roads
Mid: Ram, scale=100
Reading in file: ion.osm
Processing: Node(25k) Way(3k) Relation(0k)
Node stats: total(25760), max(844548651)
Way stats: total(3783), max(69993379)
Relation stats: total(27), max(536780)
Writing way(3k)
Writing rel(0k)
Committing transaction for planet_osm_point
Sorting data and creating indexes for planet_osm_point
Committing transaction for planet_osm_line
Committing transaction for planet_osm_roads
Sorting data and creating indexes for planet_osm_line
Committing transaction for planet_osm_polygon
Sorting data and creating indexes for planet_osm_roads
Sorting data and creating indexes for planet_osm_polygon
Completed planet_osm_polygon
Completed planet_osm_roads
Completed planet_osm_point
Completed planet_osm_line
I can see the correct lat/lon coords being passed in, I'm not sure how to verify this within the database
3 - ./generate_xml.py --accept-none --dbname gis --symbols ./symbols/ --world_boundaries ../world_boundaries/
4 - ./generate_image.py
At this point image.png is a map of the UK, not Singapore which I have specified.
Can anyone see anything wrong with this? This is with mapnik 0.71 on ubuntu
Found the solution.
The issue is that the generate_image.py script does not read the data from the database but rather has it hardcoded inside. I'm not sure the reasoning behind this.
The solution is to edit generate_image.py manually and change the relevant line:
ll = (103,1.3,104,1.4)