Table structure for Geo Spatial Data - mysql

What is the suggested way to structure a table in a MYSQL Database containing Geo Spatial data. As a POC i am working to insert the data as in the image as below into a database table.
This below is the snapshot of the table creation of Phpmyadmin table creation
looking for suggestions to create table for geospatial data as in screenshots.
EDIT: I am working on XAMPP V1.8.3 Windows8 running MYSQL version 5.6.16.
Created Table geomduplicate and columns and inserted data as in screenshot with the below sql
CREATE TABLE geomduplicate1(
zip INTEGER(3) NOT NULL PRIMARY KEY,
latitude NUMERIC(9,6),
longitude NUMERIC(10,6),
city VARCHAR(10),
state VARCHAR(2),
county VARCHAR(9)
);
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(501,40.922326,-72.637078,'Holtsville','NY','Suffolk');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(544,40.922326,-72.637078,'Holtsville','NY','Suffolk');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(601,18.165273,-66.722583,'Adjuntas','PR','Adjuntas');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(602,18.393103,-67.180953,'Aguada','PR','Aguada');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(603,18.455913,-67.14578,'Aguadilla','PR','Aguadilla');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(604,18.49352,-67.135883,'Aguadilla','PR','Aguadilla');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(605,18.465162,-67.141486,'Aguadilla','PR','Aguadilla');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(606,18.172947,-66.944111,'Maricao','PR','Maricao');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(610,18.288685,-67.139696,'Anasco','PR','Anasco');
and the data was successfully inserted in mysql database.


Store it is as a geometry data type. MySQL supports Geometry (generic), as well as Point, Linestring and Polygon data types, see creating spatial data types. A single longitude or latitude value can not be a geometry on its own, as you have it in your screen shot.
If you go the route of using geometry types, it gives you two advantages over having separate latitude and longitude fields: you can add a spatial index and you will be able to use some of MySQL's spatial operator functions such as ST_Buffer, ST_Intersects, ST_Distance to do further analysis. Spatial indexes are based on R-trees and will perform far better than two B-tree indexes on non spatial columns, latitude and longitude -- and this performance difference will grow as your table size grows.
You can still get the latitude and longitude values back by using the X and Y point functions so you will not lose anything by storing your data as a Point.
If you already have your data in two separate lat/lon columns, and you want to go the geometry/point datatype route, you can use the Point function to create the Point datatype:
alter table mytable add column pt POINT;
update mytable set pt=Point(longitude, latitude);
alter table mytable modify pt POINT NOT NULL;
create spatial index ix_spatial_mytable_pt ON mytable(pt);
Note that the Point function was only introduced in MySQL 5.1.x (it isn't too well documented, so I'm not sure of exact version), and before that you had to use concat with the GeomFromText function, see Moving lat/lon text columns into a 'point' type column for some more information on this, although note that Quassnoi's answer has lon and lat the wrong way round -- it is Point(lon, lat) not the other way, though this is a very common mistake.
NOTE: Until recently, you could only index a spatial column if using the MyISAM engine.
EDIT: In the upcoming release, MySQL 5.7.5, InnoDB will finally support indexes on spatial data types (and not just store spatial types without an index, which is considerably less useful). This means you can have foreign keys, ACID guarantees, spatial indexes all in one engine, which has been a long time in coming.


Since version 5.6 MySQL's innodb engine supports spatial data.
Refer the following links -
http://dev.mysql.com/doc/refman/5.6/en/creating-spatial-columns.html
https://dev.mysql.com/doc/refman/5.5/en/gis-data-formats.html
https://dev.mysql.com/doc/refman/5.5/en/populating-spatial-columns.html

Related

Spatial index on generated (virtual/stored) column?

I have a generated (storedAs) column in my Laravel database that takes two decimal columns lat and lng and returns a point geospatial type. I'm using a generated column so as to avoid a scenario where the decimal lat/lng columns somehow get updated and the spatial latlng column doesn't (or vice versa).
I'd like to also index this column so as to perform speedy lookups, but running into a slight hitch.
I'm using the Laravel migration builder to generate the table schema. Although the source columns are not nullable and default to 0, it won't run if I also add a spatial index, citing all parts must be not null. I'm assuming that "all parts" refers to just the latlng column.
The migration below uses the spatialIndex() method from Grimzy's Laravel MySQL Spatial extension, but I don't think that's the issue, as the standard Laravel Blueprint class also returns the same result.
public function up()
{
Schema::table('locations', function (Blueprint $table) {
$table->decimal('lat', 8, 6)->default(0);
$table->decimal('lng', 9, 6)->default(0);
$table->point('latlng')->storedAs('POINT(lat, lng)');
$table->spatialIndex('latlng');
});
}
Upon migration:
Illuminate\Database\QueryException : SQLSTATE[42000]: Syntax error or access violation: 1252 All parts of a SPATIAL index must be NOT NULL (SQL: alter table `locations` add spatial `locations_latlng_spatial`(`latlng`))
My only guess is that MySQL doesn't support spatial indexes on generated columns. Or am I doing something wrong?
UPDATE: seems I'm using MariaDB and not MySQL, which doesn't currently support them.

Creating spatial index (PostGIS vs MySQL)

Is it possible to create index in MySQL similar to this one (PostGIS):
CREATE INDEX spatial_indx_test ON spatial_test USING gist(st_makeline(st_makepoint(x1,y1), st_makepoint(x2,y2)));
where x1,x2,y1,y2 are double-typed numbers stored in the table: spatial_test. Basically is there a way to make index on data that is not directly a column in a table? I've tried smth like this:
CREATE SPATIAL INDEX spatial_indx_test ON spatial_test (LineString(Point(x1,y1), Point(x2,y2));
however it doesn't work in MySQL. Is there any workaround for that?

You can create spation index on the geometry eg:
CREATE TABLE geom (g GEOMETRY NOT NULL, SPATIAL INDEX(g))
ENGINE=MyISAM;
you can find more here http://dev.mysql.com/doc/refman/5.7/en/creating-spatial-indexes.html and here https://dev.mysql.com/doc/refman/5.7/en/using-spatial-indexes.html

Getting all values of an index in mysql

I'm working on a heatmap application which retrieves longitude and latitude points from a mysql database. Since the table I'm using is innodb I cannot use spatial index. So instead I divided the region where the points are located into a 1000*1000 block grid and assign each point an index based on the block it falls into using this formula:
set blockindex=FLOOR((lon-minlon)/londifference*1000)*1000+FLOOR(lat-minlat)/latdifference*1000);
Is there a syntax in mysql to get all the values in this index that's been assigned to a point?

MariaDB 5.2 (etc) has "Virtual Columns" , which can be indexed.
5.7.6 will have "functional indexes" by using "Functional Expressions" .

Is there a convenient way to create spatial index with Sqlalchemy or Geoalchemy?

Yesterday someone told me that Geoalchemy was a good choice to support spatial extensions if I want to use Sqlalchemy. It really did good. Thanks for him a lot.
But I have not found a convenient way to create spatial index with Geoalchemy. Must I run this SQL directly?
CREATE SPATIAL INDEX sp_index ON my_class (
geom_col
);
Is there a convenient way to do it without directly writing the SQL?
Thanks!

When creating a table with table.create() GeoAlchemy will automatically create an index for the geometry column.
Also, SQLAlchemy Index instances have a create method. See the "Indexes" section in http://docs.sqlalchemy.org/en/rel_0_7/core/schema.html#metadata-constraints. I'm not sure if you'll be able to create a spatial index in MySQL with that but it's worth a try.

With GeoAlchemy, there are 2 steps you should take to create Spatial Index on the geometry column
add attr 'spatial_index' to the geometry column
Call the GeometryDDL extension
Example:
# note that spatial_index will set the column to not null
class Building(Base):
geom = GeometryColumn(Point(2, spatial_index=True))
GeometryDDL(Building.__table__)

MySql Insert Select uuid()

Say you have a table:
`item`
With fields:
`id` VARCHAR( 36 ) NOT NULL
,`order` BIGINT UNSIGNED NOT NULL
And:
Unique(`id`)
And you call:
INSERT INTO `item` (
`item`.`id`,`item`.`order`
) SELECT uuid(), `item`.`order`+1
MySql will insert the same uuid into all of the newly created rows.
So if you start with:
aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa, 0
bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb, 1
You'll end up with:
aaaaaaaa-aaaa-aaaa-aaaa-aaaaaaaaaaaa, 0
bbbbbbbb-bbbb-bbbb-bbbb-bbbbbbbbbbbb, 1
cccccccc-cccc-cccc-cccc-cccccccccccc, 1
cccccccc-cccc-cccc-cccc-cccccccccccc, 2
How do I command MySql to create a different uuid for each row?
I know that the following works as expected in MSSQL:
INSERT INTO item (
id,[order]
) SELECT newid(), [order]+1
n.b. I know I could SELECT the results, loop through them and issue a separate INSERT command for each row from my PHP code but I don't want to do that. I want the work to be done on the database server where it's supposed to be done.

Turns out uuid() is generating a different uuid per row.
But instead of generating all the chunks randomly, as I would normally expect, MySql appears to only be generating the 2nd chunk randomly. Presumably to be more efficient.
So at a glance the uuids appear identical when in fact MySql has altered the 2nd chunk. e.g.
cccccccc-cccc-cccc-cccc-cccccccccccc
ccccdddd-cccc-cccc-cccc-cccccccccccc
cccceeee-cccc-cccc-cccc-cccccccccccc
ccccffff-cccc-cccc-cccc-cccccccccccc
I assume if there is a collision it would try again.
My bad.

How do I command MySql to create a different uuid foreach row?
MySQL won't allow expressions as a default value. You can work around this by allowing the field to be null. Then add insert/update triggers which, when null, set the field to uuid().

Please try with MID(UUID(),1,36) instead of uuid().

MySQL's UUID() function generates V1 UUIDs, which are split into time, sequence and node fields. If you call it on a single node, only a few bits in the time field will be different; this is referred to as temporal uniqueness. If you call it on different nodes at the exact same time, the node fields will be different; this is referred to as spatial uniqueness. Combining the two is very powerful and gives a guarantee of universal uniqueness, but it also leaks information about the when and where each V1 UUID was created, which can be a security issue. Oops.
V4 UUIDs are generally more popular now because they hash that data (and more) together and thus don't leak anything, but you'll need a different function to get them--and beware what they'll do to performance if you have high INSERT volume; MySQL (at least for now) isn't very good at indexing (pseudo)random values, which is why V1 is what they give you.

First generate an uniq string using the php uniqid() function
and insert to the ID field.