Is there the equivalent of the "Hello World" program for GIS applications?
I am looking to become more familiar with the development of GIS applications. What are the popular (and free/low cost) tutorials and/or sample applications that would help someone get started? Are there any books that you would consider essential for beginner GIS developers?
You could start with some basic desktop mapping software like uDig or Quantum GIS. And download some Shape files.
From there you might want to take a look at PostGIS. For web development start with MapServer and OpenLayers.
Would also be worth taking a look at the book Mapping Hacks.
Perhaps these books from Pragmatic Programmers could help you:
Desktop GIS: Mapping the Planet with Open Source Tools
(source: pragprog.com)
GIS for Web Developers: Adding Where to Your Web Applications
(source: pragprog.com)
Let's give it a try:
Polylines:
( 0, 100), ( 0, 0), ( 0, 50), ( 80, 50), ( 80, 0), ( 80, 100)
( 180, 100), ( 100, 100), ( 100, 50), ( 140, 50), ( 100, 50), ( 100, 0), ( 180, 0)
( 200, 100), ( 200, 0), ( 280, 0)
( 300, 100), ( 300, 0), ( 380, 0)
( 400, 50), ( 440, 100), ( 480, 50), ( 440, 0), ( 400, 50)
( 600, 100), ( 620, 0), ( 640, 50), ( 660, 0), ( 680, 100)
( 700, 50), ( 740, 100), ( 780, 50), ( 740, 0), ( 700, 50)
( 800, 0), ( 800, 100), ( 880, 75), ( 800, 50), ( 880, 0)
( 900, 100), ( 900, 0), ( 980, 0)
(1000, 0), (1000, 100), (1080, 50), (1000, 0)
If your company has ESRI software I'd recommend getting ArcMap, opening up the VBA editor and copying and pasting a few simple code samples from the help files.
If you don't have desktop software try out http://openlayers.org/ the open source equivalent of Google Maps. It has a nice API and samples.
I'd say the equivalent of "Hello World" in GIS is clicking on a map and popping up a message box with the latitude and longtitude of the click.
Are you interesting in desktop based GIS development or web based?
ESRI is the leader in this space. www.esri.com poke around on their site.
The Resources section of the website has lots of information. You can play around with the JavaScript API on the site and use data that they host to play with. There are several getting started sections and walk throughs here.
http://resources.esri.com/gateway/index.cfm
Here is a link to the JavaScript API resource site
http://resources.esri.com/arcgisserver/apis/javascript/arcgis/index.cfm?fa=home
You can download ArcExplorer and work with that. I think it is customizable. You can view shapefiles and geodatabases with it.
http://resources.esri.com/arcgisexplorer/900/index.cfm?fa=home
Rectangular <-> geographic coordinate transformation
I feel like sooner or later every generalist GIS developer has to do project coordinates on their own, for whatever reason. Given a latitude, longitude pair and a bitmap image representing the surface of the Earth (or a part of it), locate the geographic coordinates in the image - or vice-versa, get the geographic coordinates for a particular pixel in the image. The Lambert cylindrical projection is probably the easiest place to start, since the math is comparatively simple and equal-area projections are all the rage these days, but the spherical Mercator projection is not much harder and still in common use, especially on the web.
After this you can go in many different directions - some projections involve more complex math, or you can keep using simpler mathematical models but complicate the shape of the Earth (It's an ellipsoid! No, it's a potato!), affine and perspective transformations, or all sorts of other great stuff. But ultimately, GIS is about coordinates, and you often have to make different coordinates talk to each other, so I think that's a good place to start.
ESRI Their tools are definitely not cheap, but they are widely used. They do have good learning resources and tutorial materials, although many do assume you already have their products.
You should surely start with QuantumGis (QGIS). It's open source, relatively fast and simple to use. Download any shapefile from web, make sure you can interpret it yourself (for example shapefile with boundaries of European countries). Think what can you do with the data, or what you want to do. Then I suggest you take some basic Python (GDAL, Shapely, Fiona) tutorials and in the end conclude it with POSTGis and MapServer. I'm not saying you should learn it all in one day, one month or one year, take your time and make sure you understand everything. I'm suggesting the stuff like that cause that's how I started and it worked.
Try out http://code.google.com/p/tf-net/ (Topology Framework .NET) and browse through the introductory topics. Though it's .NET oriented, it may give you a glimpse into basic GIS tasks:
Spatial predicates (based on the DE-9IM model)
Overlay functions (intersection, difference, union, symmetric difference)
Buffer
Convex hull
Area and distance functions
Topological validity checking
Coordinate systems manipulation (transformations)
Topological graphs manipulation
Generic geometry I/O support: WKT, WKB, GML, SHP
...
Related
I'm adding Multi-Headed attention at the input of my CNN to improve interpretability and explainability of my model. The data is formed as time-series 3D input of shape (125, 5, 6) where 5x6 part represents the data in a single sample and 125 represents 125 ordered samples (basically 1 second long recording of 5x6 time-series data).
While reading about transformers for time-series and video data I've come across positional embedding. From my understanding, positional embedding is used to connect the data to its order so that learning is not completely random which would be wrong in the case of data where order is crucial. Now, one thing I don't understand is if my model needs positional embedding considering I input it in order as (125, 5, 6) and separate inputs don't have any connection, but the order of these 125 samples must be preserved.
Essentially, I'm not sure if the addition to my model should be:
Multi-headed attention -> Dropout -> Normalization -> the developed CNN
or
Positional-embedding -> Multi-headed attention -> Dropout -> Normalization -> the developed CNN
or
Multi-headed attention -> the developed CNN
or
Positional-embedding -> Multi-headed attention -> the developed CNN
or something completely different.
Another question is: Do I need more than a single layer of attention considering that the addition of it is solely with purpose of improving interpretability and explainability?
Ps. I'm aware the data needs to be reshaped into (125, 5, 6, 1) to be able to use Transformers.
I am interested on generating GeoTiff images given a set of scattered points, the dataset is normally between 500K points up to 2M points. I have been using datashader and geoviews to generate static images of the data and that works fine. However, when I want to generate the geotiff, there are two options:
To generate an RGB geotiff with the coloring as provided by datashader Image color map
To use the values inside the geotiff to create a colormap on the geotiff server.
We are interested on going for the second solution but I do not understand the values which I extract from the shade function:
import pandas as pd
import datashader as ds
import datashader.transfer_functions as tf
df = pd.DataFrame({"x": [0, 0.6, 1, 1.2, 2, 5], "y":[0, 0.6, 0.8, 1, 5, 6], "value":[1, 1, 1, 1, 0.5, 0.5]})
canvas = ds.Canvas(plot_width=100, plot_height=100, x_range=(0, 5), y_range=(0, 5), x_axis_type="linear", y_axis_type="linear")
shaded = tf.shade(canvas.points(df, "x", "y", ds.mean("value")))
As far as I know, I extract the aggregated values from shade as numpy and can use this array to save a GeoTiff with one band but the values into that array make no sense for me.
shaded.to_numpy().max()
4293318829
It is always huge numbers which have no relationship with the aggregation process.
So the questions are, what do those values actually mean?
Is it possible to get the actual aggregated values ?
Is it possible to extract the color mapping from an Image object so I can pass it to the server and they color the geotiff as I expect it to be colored?
So datashader uses a couple of steps to produce the results we get. The more important, there is a step in which the values are mapped to RGB using the color_map input. After that the colourize function inside datashader stack the 4 RGBA uint8 arrays of size (100, 100) into one uint32 (100, 100) array using this:
From datashader:
/transfer_functions/__init__.py
values = module.dstack([r, g, b, a]).view(module.uint32).reshape(a.shape)
That is why the values are so strange. To get the RGBA, one can access the data like this:
shaded.to_numpy().base.shape
(310, 400, 4)
All your questions can be found in the datashader documentation.
The actual aggregated values are stored in canvas.points(df, "x", "y", ds.mean("value")). This is a xarray object. You can write it to netcdf or geotiff easily. But in this way you will lose the geographic references/coordinate system information.
I suppose your dataset use epsg:3857 as your projection coordinate system. You can use the rioxarray package to add the geographic references/coordinate system information and write to geotiff (see https://corteva.github.io/rioxarray/latest/rioxarray.html#rioxarray.rioxarray.XRasterBase.write_crs). like this:
ag = canvas.points(df, "x", "y", ds.mean("value"))
ag.rio.write_crs("epsg:3857", inplace=True)
ag.rio.to_raster("../data/interim/test2.tif")
I think the color mapping should be the function of your server. To know what information can geotiff store, you can check here:https://www.bluemarblegeo.com/about-geotiff-format/.
I want to calculate fuel consumption of a car quite accurately with a raspberry pi.
The car I use is a Citroen and has not many available PID for OBD II.
Here are the available PIDs (in decimal) : 1, 3-7, 11-15, 17, 19-21, 18, 31-33, 35, 46, 48, 49, 51, 60, 64, 66, 68, 71, 73, 74, 76-80
So I use a method based on MAF (Mass air flow) estimation described here : http://www.lightner.net/obd2guru/IMAP_AFcalc.
I have ~20% error for the average consumption. But I'm more interested in the immediate consumption, especially during accelerations and decelerations, and this method doesn't work well : when I use the engine brake, I have a >2L/100km consumption instead of 0 (probably because there is an air flow but no fuel flow), and consumption increases with speed at a slower rate than estimated on the dashboard.
I think I need a live estimation of air/fuel ratio.
I would like to find values similar to those displayed on the dashboard, but can't find how to do it. Does the OBC use sensors that are not available via OBD-II ?
2 questions :
Can I get a better consumption estimation with the data I have on OBD-II ?
Is there a way to get other data via OBD-II, or via another mean ? (after all the OBC can do it...)
Thanks !
All vehicles are continually trimming fuel rate based on many factors (mostly the #1 O2 sensor) to try to hit 0.5% lean of stoichiometric while operating to meet a power demand. So you're right, that method is correct in principle, but naiive.
I bet you'd go a long way by towards accuracy by logging the Commanded Air-Fuel Equivalence Ratio as well. Please ping me and let me know how it goes.
I've a very simple table (CSV) that I've imported into mycrostrategy to visualize my data.
The data represents the points of certain teams (e.g. football) for each played round (px).
csv:
#teams, p1, p2, p3, p4
Team1, 3, 6, 6, 7
Team2, 0, 0, 3, 4
Team3, 3, 6, 9, 12
Team4, 1, 4, 7, 8
The expected graph ( I did this one in Excel) is attached.
This seems very simple to me to do, but so far, I could not figure out how to organize the data (attributes/metrics, etc) to create this visualization.
Any idea? Is there any metric that I've to create (like max number Points)..
EDIT:
The best I could do was a 'discrete' representation (see attach) of the data with Dots, I'd like to have a line/continuous.
Solution with dots instead of a line connecting the dots
Thanks
You need to import your data using the Crosstab option.
You should create two attributes:
Team (team1, team2, team3, team4)
Player Rounds (p1, p2, p3, p4)
And a Points metric.
With these three elements you should be able to create your graph easily.
Let's say I have a sequence of values (e.g., 3, 5, 8, 12, 15) and I want to occasionally decrease all of them by a certain value.
If I store them as the sequence (0, 2, 3, 4, 3) and keep a variable as a base of 3, I now only have to change the base (and check the first items) whenever I want to decrease them instead of actually going over all the values.
I know there's an official term for this, but when I literally translate from my native language to English it doesn't come out right.
Differential Coding / Delta Encoding?
I don't know a name for the data structure, but it's basically just base+offset :-)
An offset?
If I understand your question right, you're rebasing. That's normally used in reference to patching up addresses in DLLs from a load address.
I'm not sure that's what you're doing, because your example seems to be incorrect. In order to come out with { 3, 5, 8, 12, 15 }, with a base of 3, you'd need { 0, 2, 5, 9, 12 }.
I'm not sure. If you imagine your first array as providing the results of some function of an index value f(i) where f(0) is 3, f(1) is 5, and so forth, then your second array is describing the function f`(i) where f(i+1) = f(i) + f'(i) given f(0) = 3.
I'd call it something like a derivative function, where the process of retrieving your original data is simply the summation function.
What will happen more often, will you be changing f(0) or retrieving values from f(i)? Is this technique rooted in a desire to optimize?
Perhaps you're looking for a term like "Inductive Sequence" or "Induction Sequence." (I just made that up.)