Original Question
MySQL workbench allows one to define "inserts": rows to be inserted into the database on creation. It does this by adding lines such as
START TRANSACTION;
USE `someDB`;
INSERT INTO `someDB`.`countries` (`name`) VALUES ('South Africa');
COMMIT;
However, if the database, table and entry exists, this throws an error. Creation of tables does not, as workbench uses CREATE IF NOT EXISTS for those. Is there a way to get workbench to insert using INSERT...ON DUPLICATE KEY UPDATE?
Half Solution
Running the script with the force argument:
mysql user=xx password=xx --force < script.sql
Ignores such errors, and is thus a solution in my particular case. However, the actual question of modifying the type of INSERTS still stands (for interest)
See here
Related
I am trying to import a SQL dump from a live WordPress site into my local MAMP dev environment using PhpMyAdmin so I can make edits to the site locally. I keep getting this error:
Error
SQL query: INSERT INTO `wp_options` VALUES (259568, '_transient_timeout_geoip_98.80.4.79', '1440122500', 'no');
MySQL said: Documentation
#1062 - Duplicate entry '259568' for key 'PRIMARY'
My knowledge of SQL is minimal. What could be causing this and what do I need to do in order to fix the problem so that I can successfully import the database and get the site up and running locally?
You can replace INSERT statement with INSERT IGNORE. That helps import entries even if they have duplicates.
If you use unix-like OS, you can use sed command to replace insert:
cat dump.sql | sed s/"^INSERT"/"INSERT IGNORE"/g > dump-new.sql
Or you can add option --insert-ignore for mysqldump to write INSERT IGNORE statements rather than INSERT statements.
For export via phpMyAdmin it's possible to set an option:
Settings -> Export -> SQL -> Use ignore inserts
Your table already has a record with a primary key value of 259568, and primary keys are required to be unique. Deleting the existing record would allow you to insert this one, but deleting the existing record may cause problems as well.
Is there a SQL statement (or atomic sequence of statements) supported by both MySQL and HSQLDB to insert values if they aren't already there?
I'm working on an app that uses MySQL as its production database and HSQLDB for unit tests; I'd like to have a single "initial data import when the tables are empty" script.
MySQL supports INSERT IGNORE, REPLACE INTO and INSERT INTO ... ON DUPLICATE KEY UPDATE ..., but HSQLDB doesn't; conversely, HSQLDB supports MERGE but MySQL doesn't.
HSQLDb from version 2.3.4 adds support for insert ignore.
http://hsqldb.org/
Version 2.3.4 added the UUID type for columns, SYNONYM for tables and
functions, PERIOD predicates, and auto-updated TIMESTAMP columns on
row updates. Other new features included the ability to cancel
long-running statements from JDBC as well as from admin sessions, and
UTF-16 file support for text table sources, in addition to 8-bit text
files. MySQL compatibility for REPLACE, INSERT IGNORE and ON
DUPLICATE KEY UPDATE statements.
And
http://hsqldb.org/doc/guide/guide.pdf (page 260).
HyperSQL supports and translates INSERT IGNORE, REPLACE and ON
DUPLICATE KEY UPDATE variations of INSERT into predictable and
error-free operations. When INSERT IGNORE is used, if any of the
inserted rows would violate a PRIMARY KEY or UNIQUE constraint, that
row is not inserted. With multi-row inserts, the rest of the rows are
then inserted only if there is no other violation such as long strings
or type mismatch, otherwise the appropriate error is returned. When
REPLACE or ON DUPLICATE KEY UPDATE is used, the rows that need
replacing or updating are updated with the given values. This works
exactly like an UPDATE statement for those rows. Referential
constraints and other integrity checks are enforced and update
triggers are activated. The row count returned is simply the total
number of rows inserted and updated.
If someone still has this problem you can enable syntax support for MySQl by adding the following to your script
SET DATABASE SQL SYNTAX MYS TRUE
I was once told that it is faster to just run an insert and let the insert fail than to check if a database entry exists and then inserting if it is missing.
I was also told that that most databases are heavily optimized for reading reading rather than writing, so wouldn't a quick check be faster than a slow insert?
Is this a question of the expected number of collisions? (IE it's faster to insert only if there is a low chance of the entry already existing.) Does it depend on the database type I am running? And for that matter, is it bad practice to have a method that is going to be constantly adding insert errors to my error log?
Thanks.
If the insert is going to fail because of an index violation, it will be at most marginally slower than a check that the record exists. (Both require checking whether the index contains the value.) If the insert is going to succeed, then issuing two queries is significantly slower than issuing one.
You can use INSERT IGNORE so that if the key already exist, the insert command would just be ignored, else the new row will be inserted. This way you need to issue a single query, which checks the duplicate values as well inserts new values too.
still Be careful with INSERT IGNORE as it turns EVERY error into a warning. Read this post for insert ignore
On duplicate key ignore?
I think INSERT IGNORE INTO .... can be used here, either it will insert or ignore it.
If you use the IGNORE keyword, errors that occur while executing the INSERT statement are treated as warnings instead. For example, without IGNORE, a row that duplicates an existing UNIQUE index or PRIMARY KEY value in the table causes a duplicate-key error and the statement is aborted. With IGNORE, the row still is not inserted, but no error is issued.
If you want to delete the old value and insert a new value you can use REPLACE You can use REPLACE instead of INSERT to overwrite old rows.
REPLACE works exactly like INSERT, except that if an old row in the table has the same value as a new row for a PRIMARY KEY or a UNIQUE index, the old row is deleted before the new row is inserted.
Else use the INSERT IGNORE as it will either inserts or ignores.
a row that duplicates an existing UNIQUE index or PRIMARY KEY value in the table causes a duplicate-key error and the statement is aborted. With IGNORE, the row still is not inserted, but no error is issued.
If your intension is to Insert if its a new record OR Update the record if it already exists then how about doing an UPSERT?
Check out - http://vadivel.blogspot.com/2011/09/upsert-insert-and-update-in-sql-server.html
Instead of checking whether the record exists or not we can try to Update it directly. If there is no matching record then ##RowCount would be 0. Based on that we can Insert it as a new record. [In SQL Server 2008 you can use MERGE concept for this]
EDIT: Please note, I know this works for MS SQL Server and I don't know about MySQL or ORACLE
I am trying to restore a DB using an SQL script, but things foreign key constraints get in the way
I am taking a MySQL DB and bringing it over to PostgreSQL.
Since the MySQL create table syntax ended up being quite different, I took another PostgreSQL DB with the same schema, but different data and restored the schema only, from that.
In other words, I now have a database with tables, constraints, sequences and all that shnaz but no data inside.
So, it's is time to restore data.
I take a backup of the MySQL DB with phpMyAdmin (data only) as an SQL script (pgAdmin does not seem to accept zip or gzip files for some reason) and run the SQL script.
Now, this is where the problems start to happen, it's only natural, I am going from MySQL to PostgreSQL, so syntax errors are bound to happen.
But, there are other non syntax related problems to, like this one:
ERROR: insert or update on table "_account" violates foreign key constraint "fk_1_account"
DETAIL: Key (accountid)=(2) is not present in table "_entity".
So, yeah, basically, a foreign constraint exists, the query is trying to insert data into the _account table, but the corresponding data has not been inserted into the _entity table yet.
How do I get around that? Is there a way to make pgAdmin3/PostgreSQL disable ALL OF the constraints, insert the data, and then re-enable the constraints?
A syntax related error I encountered, was this one:
INSERT INTO _accounttype_seq (id) VALUES (11);
The PostgreSQL equivalent of that statement (if I am correct) is
ALTER SEQUENCE _accounttype_seq INCREMENT BY 11;
But, it's a bit of a pain to run through the whole script and change all 200+ Sequence insert statements. So, I am being lazy here, but is there an easier way to deal with the sequences as well?
Or, do you guys have any suggestions for a different set of tools to make this easier?
Thanks for your time, have a good day.
Do not try to get around the foreign key constraints. That is the way to make sure the data is bad.
First look at the constraints and make sure you are inserting to the tables in the correct order. If _entity is parent of "_account, then it should be populated first.
Next you need to have the script move any failing records to an exception table. Then you can look at them and see what the data integrity issues is and if you need to throw the records away permanently or try to figure out what the missing parent value should be. If it is critical data such as orders where the customer no longer exists (possible in any system that didn't have correct fks to begin with) and you must keep the record and cannot determine what the parent value should have been, you can create an 'Unknown" record in the customer table and assign all bad orders to that customer id.
And manually changing the alter sequences shouldn't take long even if it is boring. There wil be plently of other things you need to handle manually in a conversion of this type.
I would try to find a data import tool for PostgreSQL - I live in SQL server world where I would use SSIS but you need the equivalent of SSIS for the PostgreSQL world.
Aparently the foreign keys weren't actually enforced in MySQL (maybe because of using MyISAM) or the generated SQL just does it in the wrong order.
If it's "only" the wrong order, I see two possible solutions:
edit the generated script and either move all FK definitions to the end of the script
Edit the definition of each FK constraint and set them all to initially deferred. Then run the script as one single transaction with only on commit at the very end.
Edit (because this is too much to be put as a comment)
Using SET CONSTRAINTS ALL DEFERRED will only work if the constraints have been created with the option DEFERRABLE.
To run everything in one single transaction, you have to make sure you have turned autocommit off. Then simply run the INSERTs and at the very end issue a COMMIT. A ; will only commit if you have autocommit on.
If you want to be independent of the autocommit setting, then start your script with [BEGIN][1] and make sure there is only a single COMMIT at the very end.
BEGIN DEFERRABLE
INSERT INTO table_one ... ;
INSERT INTO table_two ... ;
.....
COMMIT;
I'm fetching data from a text file or log periodically and it gets inserted in the database every time fetched. Is there a way in MySQL that the insert is only done when the log files are updated or I have to do it using the programming language ? I mean Is there a type of insert that when It sees a duplicate primary key, It doesn't give an error of "Duplicate Entry" .. It just ignore.
Put the fetch in a logrotate postrotate script, and fetch from the just rotated log.
Ignoring duplicates can be done with either INSERT IGNORE OR INSERT .... ON DUPLICATE KEY UPDATE syntax (which will either ignore the lines causing a duplcate unique key, or give you the possibility to alter some values in the existing row.)