I have a generated (storedAs) column in my Laravel database that takes two decimal columns lat and lng and returns a point geospatial type. I'm using a generated column so as to avoid a scenario where the decimal lat/lng columns somehow get updated and the spatial latlng column doesn't (or vice versa).
I'd like to also index this column so as to perform speedy lookups, but running into a slight hitch.
I'm using the Laravel migration builder to generate the table schema. Although the source columns are not nullable and default to 0, it won't run if I also add a spatial index, citing all parts must be not null. I'm assuming that "all parts" refers to just the latlng column.
The migration below uses the spatialIndex() method from Grimzy's Laravel MySQL Spatial extension, but I don't think that's the issue, as the standard Laravel Blueprint class also returns the same result.
public function up()
{
Schema::table('locations', function (Blueprint $table) {
$table->decimal('lat', 8, 6)->default(0);
$table->decimal('lng', 9, 6)->default(0);
$table->point('latlng')->storedAs('POINT(lat, lng)');
$table->spatialIndex('latlng');
});
}
Upon migration:
Illuminate\Database\QueryException : SQLSTATE[42000]: Syntax error or access violation: 1252 All parts of a SPATIAL index must be NOT NULL (SQL: alter table `locations` add spatial `locations_latlng_spatial`(`latlng`))
My only guess is that MySQL doesn't support spatial indexes on generated columns. Or am I doing something wrong?
UPDATE: seems I'm using MariaDB and not MySQL, which doesn't currently support them.
I'm using php and MySQL with a rather large MySQL database and I am trying to use idexes for the first time. I get the concept that the server will look through the index first but I'm moving trouble getting the server to use the index. So my questions are:
Does having a primary key (thus primary index?) in the table get used over the index I'm trying to use from another column? Do I have to explicitly specify the index in the select query? (I'm using several table joins, btw)
Does anyone know of a good beginners guide to using MySQL indexes? I haven't found a good one!
An important rule for mysql indexes, say you have the following index:
KEY(A,B,C)
Then in your code you have a where clause or join such as:
WHERE B = 'mydata' AND C = 'moredata'
Mysql will not be able to make use of the index for the query because the indexes work in the order given and the column A has not been included.
The fix is to either use A in the query or re-order the index (or add a second index) as so:
KEY(C,B,A)
or (if you don`t need A at all in the WHERE/JOIN):
KEY(C,B)
Also check out explain which should help you find why your indexes are not being used.
What is the suggested way to structure a table in a MYSQL Database containing Geo Spatial data. As a POC i am working to insert the data as in the image as below into a database table.
This below is the snapshot of the table creation of Phpmyadmin table creation
looking for suggestions to create table for geospatial data as in screenshots.
EDIT: I am working on XAMPP V1.8.3 Windows8 running MYSQL version 5.6.16.
Created Table geomduplicate and columns and inserted data as in screenshot with the below sql
CREATE TABLE geomduplicate1(
zip INTEGER(3) NOT NULL PRIMARY KEY,
latitude NUMERIC(9,6),
longitude NUMERIC(10,6),
city VARCHAR(10),
state VARCHAR(2),
county VARCHAR(9)
);
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(501,40.922326,-72.637078,'Holtsville','NY','Suffolk');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(544,40.922326,-72.637078,'Holtsville','NY','Suffolk');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(601,18.165273,-66.722583,'Adjuntas','PR','Adjuntas');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(602,18.393103,-67.180953,'Aguada','PR','Aguada');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(603,18.455913,-67.14578,'Aguadilla','PR','Aguadilla');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(604,18.49352,-67.135883,'Aguadilla','PR','Aguadilla');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(605,18.465162,-67.141486,'Aguadilla','PR','Aguadilla');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(606,18.172947,-66.944111,'Maricao','PR','Maricao');
REPLACE INTO geomduplicate1(zip,latitude,longitude,city,state,county) VALUES
(610,18.288685,-67.139696,'Anasco','PR','Anasco');
and the data was successfully inserted in mysql database.
Store it is as a geometry data type. MySQL supports Geometry (generic), as well as Point, Linestring and Polygon data types, see creating spatial data types. A single longitude or latitude value can not be a geometry on its own, as you have it in your screen shot.
If you go the route of using geometry types, it gives you two advantages over having separate latitude and longitude fields: you can add a spatial index and you will be able to use some of MySQL's spatial operator functions such as ST_Buffer, ST_Intersects, ST_Distance to do further analysis. Spatial indexes are based on R-trees and will perform far better than two B-tree indexes on non spatial columns, latitude and longitude -- and this performance difference will grow as your table size grows.
You can still get the latitude and longitude values back by using the X and Y point functions so you will not lose anything by storing your data as a Point.
If you already have your data in two separate lat/lon columns, and you want to go the geometry/point datatype route, you can use the Point function to create the Point datatype:
alter table mytable add column pt POINT;
update mytable set pt=Point(longitude, latitude);
alter table mytable modify pt POINT NOT NULL;
create spatial index ix_spatial_mytable_pt ON mytable(pt);
Note that the Point function was only introduced in MySQL 5.1.x (it isn't too well documented, so I'm not sure of exact version), and before that you had to use concat with the GeomFromText function, see Moving lat/lon text columns into a 'point' type column for some more information on this, although note that Quassnoi's answer has lon and lat the wrong way round -- it is Point(lon, lat) not the other way, though this is a very common mistake.
NOTE: Until recently, you could only index a spatial column if using the MyISAM engine.
EDIT: In the upcoming release, MySQL 5.7.5, InnoDB will finally support indexes on spatial data types (and not just store spatial types without an index, which is considerably less useful). This means you can have foreign keys, ACID guarantees, spatial indexes all in one engine, which has been a long time in coming.
Since version 5.6 MySQL's innodb engine supports spatial data.
Refer the following links -
http://dev.mysql.com/doc/refman/5.6/en/creating-spatial-columns.html
https://dev.mysql.com/doc/refman/5.5/en/gis-data-formats.html
https://dev.mysql.com/doc/refman/5.5/en/populating-spatial-columns.html
This question follows up the following previous question.
is it possible to make a field in Mysql table to to have length max=700, to be unicode (3 bytes) and to be unique?
I have a table MyText which have ID column, text column & many other columns
Id - text - type ..& other fileds
1 - this is my text
2 - xxxx
I want the text column support unicode with max length can hold 700 Unicode characters. I can't set Unique (text) because MYSQL only supports 765 bytes max length for unique column while Unicode takes 3 bytes so I need 2100 bytes (700*3) unique column.
So, the solution is to crate a trigger that prevents the user to insert the duplicate. For example, if user inserts "THIS is My Text" (We won't care case sensitive) into MyText table, then Mysql will abort completely ALL Queries that contain that Inserting Statement and will generate an SQLException to prevent the system to do other query.
Ok, suppose you have to run a series of sql statements in your Java code
insert into MyText('THIS is My Text',1);
insert into OtherTable ('some text');
update othetTable...
Then when the system doing the insert into MyText('THIS is My Text',1);, it should stop doing other queries below it.
Also, some people suggests to do the prefix index to help Nysql to do the select quicker, but I am not sure it is too necessary since I got ID colum which was indexed.
Note: MYSQL 5.027 is 2006 version which is pretty old, but I love it
SO how to create trigger that meets my requirement or how to fix my problem?
I would recommend not using a trigger for this because of performance reasons.
Instead, create an additional column to store an MD5 or SHA1 hash of your value, and make that column unique using a constraint.
As per the above links, both hashing functions exist in your version of MySQL. Alternatively, if it's easier to integrate this in your Java code, you could do the hashing in Java using the MessageDigest class.
The part in your question where you indicate that no further queries should be executed if the insert statement fails because of a duplicate, is best handled using transactions. These are also supported in Java using plain JDBC or most ORM frameworks.
Yesterday someone told me that Geoalchemy was a good choice to support spatial extensions if I want to use Sqlalchemy. It really did good. Thanks for him a lot.
But I have not found a convenient way to create spatial index with Geoalchemy. Must I run this SQL directly?
CREATE SPATIAL INDEX sp_index ON my_class (
geom_col
);
Is there a convenient way to do it without directly writing the SQL?
Thanks!
When creating a table with table.create() GeoAlchemy will automatically create an index for the geometry column.
Also, SQLAlchemy Index instances have a create method. See the "Indexes" section in http://docs.sqlalchemy.org/en/rel_0_7/core/schema.html#metadata-constraints. I'm not sure if you'll be able to create a spatial index in MySQL with that but it's worth a try.
With GeoAlchemy, there are 2 steps you should take to create Spatial Index on the geometry column
add attr 'spatial_index' to the geometry column
Call the GeometryDDL extension
Example:
# note that spatial_index will set the column to not null
class Building(Base):
geom = GeometryColumn(Point(2, spatial_index=True))
GeometryDDL(Building.__table__)