We are creating an app wich uses local and remote DB, the information comes from the remote server and if necessary gets stored in the local DB.
The problem is, some records, will come again to the local DB, and i dont want to duplicate the entry, just update it, and if does not exists, insert a new one.
In Mysql i probably use :
INSERT INTO 'table'
('key', 'name', 'time')
VALUES
(4815162342, user, NOW())
ON DUPLICATE KEY UPDATE
'name' = newname
Is there a way to use it in SQLite, more specifically with FMDB?
I think what you want is more generally referred to as an 'upsert'. See this answer for some recommendations (or search further as there's a bunch more around).
https://stackoverflow.com/a/15277374/297472
Related
I have an app that has to import TONS of data from a remote source. From 500 to 1500 entries per call.
Sometimes some of the data coming in will need to replace data already stored in the dB. If I had to guess, I would say once in 300 or 400 entries would one need to be replaced.
Each incoming entry has a unique ID. So I am trying to figure out if it is more efficient to always issue a delete command based on this ID or to check if there is already an entry THEN delete.
I found this SO post where it talks about the heavy work a dB has to do to delete something. But it is discussing a different issue so I'm not sure if it applies here.
Each incoming entry has a unique ID. So I am trying to figure out if it is more efficient to always issue a delete command based on this ID or to check if there is already an entry THEN delete.
Neither. Use INSERT ... ON DUPLICATE KEY UPDATE ....
Since you are using MySQL and you have a unique key then let MySQL do the work.
You can use
INSERT INTO..... ON DUPLICATE KEY UPDATE......
MySQL will try to insert a new record in the table, is the unique value exists in the table then MySQL will update all the field that you have set after the update
You can read more about the INSERT INTO..... ON DUPLICATE KEY UPDATE...... syntax on
http://dev.mysql.com/doc/refman/5.0/en/insert-on-duplicate.html
i wanna migrate my origin database to new database(mariaDB)
but new database has some information
so i have to check
if some datas are in new database origin should not migrate to new one
i wanna seperate each situation, i wanna avoid
how can i check? just using sql query
take a look at
INSERT ............ ON DUPLICATE KEY UPDATE <---if you set UPDATE clause to do something useless, it'll do what you want.
INSERT IGNORE <---- skip if duplicate.
I personally suggest the on duplicate key approach as insert ignore will also ignore some other errors and doesn't give you error message.
Is it possible to use any sort of logic in MySQL without using any procedures? My web hosting does not let me create any procedures so I'm looking for a workaround.
The type of thing I want to do is only add an item to a table if it doesn't already exist. Or add a column to a table if it's not already there. There are some operations that can be done such as CREATE TABLE IF NOT EXISTS and so on, but some operations I require do not have such luxuries :(
I realised late on that my lovely procs won't work and so I tried writing IF/ELSE logic as top-level queries, but for MySQL, IF ELSE blocks only seem to work inside functions/procs and not at the global scope.
Any workarounds greatfully received - I've already asked the hosting to grant me privileges to create procedures but no reply as yet...
I suppose you don't have access to the INFORMATION_SCHEMA either. You can possibly find solutions but it would be better, in my oninion, to:
Change your hosting provider. Seriously. Pay more - if needed - for a MySQL instance that you can configure to your needs. You only have a crippled DBMS if you are not allowed to create procedures and functions.
Posible workarounds for the specific task: You want to add a column if it doesn't exist.
1) Just ALTER TABLE and add the column. If it already exists, you'll get an error. You can catch that error, in your application.
2) (If you have no access to the INFORMATION_SCHEMA) maintain a version of the schema, for your database.
The best solution that I can think of would be to use an additional language with SQL. For example, you can run a query for a specific record, and based on the response that you get, you can conditionally run an INSERT statement.
For inserting a table if it doesn't exist, try using the SHOW TABLES statement and testing whether or not a name exists in the result set.
MySQL supports INSERT IGNORE. and INSERT ... ON DUPLICATE KEY UPDATE.
The following will insert a new row, but only if there is no existing row with id=10. (This assumes that id is defined as a unique or primary key).
INSERT IGNORE INTO my_table (id, col1, col2) values (10, "abc", "def");
The following will insert a new row, but if there is an existing row with id=10 (again, assuming id is unique or primary), the existing row will be updated to hold the new values, instead of inserting a new row.
INSERT INTO my_table (id, col1, col2) values (10, "abc", "def")
ON DUPLICATE KEY UPDATE col1=VALUES(col1), col2=VALUES(col2)
Also, CREATE TABLE supports the IF NOT EXISTS modifier. So you can do something like:
CREATE TABLE IF NOT EXISTS my_table ...
There are many other similar options and modifiers available in MySQL. Check the docs for more.
Originally I created a big script to create or update the database schema, to make it easier to deploy database changes from my local machine to the server.
My script was doing a lot of "if table 'abc' exists and it doesn't have a FK constraint called 'blah'" then create an FK constraint called 'blah' on table 'abc'... and so on.
I now realise it's not actually necessary to check whether a table has a certain column or constraint etc, because I can just maintain a schema-versioning system, and query the DB schema-version when my app starts, or when I navigate to a certain page.
e.g. let's say I want to add a new column to a table. It works like this:
Add a new migration script to the app code, containing the SQL required to add the column to the existing table
Increment the app's schema-version by 1
On app startup, the app queries the DB for the DB's schema-version
If DB schema-version < app schema-version, execute the SQL migration scripts between the two schema-versions, and then update the DB schema-version to be the same as the app
e.g. if the DB's schema-version is 5 and the app version is 8, the app will apply migration scripts 5-6, 6-7 and 7-8 to the DB. These can just be run without having to check anything on the DB side.
The app is therefore solely responsible for updating the DB schema and there's no need for me to ever have to execute schema change scripts on the local or remote DB.
I think it's a better system than the one I was trying to implement for my question.
I am trying to restore a DB using an SQL script, but things foreign key constraints get in the way
I am taking a MySQL DB and bringing it over to PostgreSQL.
Since the MySQL create table syntax ended up being quite different, I took another PostgreSQL DB with the same schema, but different data and restored the schema only, from that.
In other words, I now have a database with tables, constraints, sequences and all that shnaz but no data inside.
So, it's is time to restore data.
I take a backup of the MySQL DB with phpMyAdmin (data only) as an SQL script (pgAdmin does not seem to accept zip or gzip files for some reason) and run the SQL script.
Now, this is where the problems start to happen, it's only natural, I am going from MySQL to PostgreSQL, so syntax errors are bound to happen.
But, there are other non syntax related problems to, like this one:
ERROR: insert or update on table "_account" violates foreign key constraint "fk_1_account"
DETAIL: Key (accountid)=(2) is not present in table "_entity".
So, yeah, basically, a foreign constraint exists, the query is trying to insert data into the _account table, but the corresponding data has not been inserted into the _entity table yet.
How do I get around that? Is there a way to make pgAdmin3/PostgreSQL disable ALL OF the constraints, insert the data, and then re-enable the constraints?
A syntax related error I encountered, was this one:
INSERT INTO _accounttype_seq (id) VALUES (11);
The PostgreSQL equivalent of that statement (if I am correct) is
ALTER SEQUENCE _accounttype_seq INCREMENT BY 11;
But, it's a bit of a pain to run through the whole script and change all 200+ Sequence insert statements. So, I am being lazy here, but is there an easier way to deal with the sequences as well?
Or, do you guys have any suggestions for a different set of tools to make this easier?
Thanks for your time, have a good day.
Do not try to get around the foreign key constraints. That is the way to make sure the data is bad.
First look at the constraints and make sure you are inserting to the tables in the correct order. If _entity is parent of "_account, then it should be populated first.
Next you need to have the script move any failing records to an exception table. Then you can look at them and see what the data integrity issues is and if you need to throw the records away permanently or try to figure out what the missing parent value should be. If it is critical data such as orders where the customer no longer exists (possible in any system that didn't have correct fks to begin with) and you must keep the record and cannot determine what the parent value should have been, you can create an 'Unknown" record in the customer table and assign all bad orders to that customer id.
And manually changing the alter sequences shouldn't take long even if it is boring. There wil be plently of other things you need to handle manually in a conversion of this type.
I would try to find a data import tool for PostgreSQL - I live in SQL server world where I would use SSIS but you need the equivalent of SSIS for the PostgreSQL world.
Aparently the foreign keys weren't actually enforced in MySQL (maybe because of using MyISAM) or the generated SQL just does it in the wrong order.
If it's "only" the wrong order, I see two possible solutions:
edit the generated script and either move all FK definitions to the end of the script
Edit the definition of each FK constraint and set them all to initially deferred. Then run the script as one single transaction with only on commit at the very end.
Edit (because this is too much to be put as a comment)
Using SET CONSTRAINTS ALL DEFERRED will only work if the constraints have been created with the option DEFERRABLE.
To run everything in one single transaction, you have to make sure you have turned autocommit off. Then simply run the INSERTs and at the very end issue a COMMIT. A ; will only commit if you have autocommit on.
If you want to be independent of the autocommit setting, then start your script with [BEGIN][1] and make sure there is only a single COMMIT at the very end.
BEGIN DEFERRABLE
INSERT INTO table_one ... ;
INSERT INTO table_two ... ;
.....
COMMIT;
With the query below, I am trying to overwrite the 10th field in a MySQL table called "login" with the value NEW_VALUE. It's not working. Is the code below the correct method for overwriting existing data in a MySQL table?
Thanks in advance,
John
INSERT INTO login VALUES (NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, NULL, 'NEW_VALUE', NULL, NULL, NULL)
Just as an addition if anyone is still looking for an actual overwrite and not just an update. If you want to OVERWRITE always, (not update, just overwrite) you can use REPLACE instead of INSERT.
REPLACE works exactly like INSERT, except that if an old row in the table has the same value as a new row for a PRIMARY KEY or a UNIQUE index, the old row is deleted before the new row is inserted. See Section 13.2.5, “INSERT Syntax”
http://dev.mysql.com/doc/refman/5.5/en/replace.html
No your code is not correct. You are adding a new row to your table not updating existing values. To update existing values, you want to use an update statement:
Upate a specific record
mysql_query("Update login SET nameOfYourColumn = '$cleanURL' WHERE primaryKey = idOfRowToUpdate")
To update the entire table
mysql_query("Update login SET nameOfYourColumn = '$cleanURL'")
If I've understood your question then the answer is "no". This isn't a mysql specific issue either, it's a generic SQL question. I'd strongly recommend going through an SQL tutorial, the best one I know if is here:
http://philip.greenspun.com/sql/
To answer your question, you should be able to do:
mysql_query("UPDATE login SET foo = '$cleanurl'");
where "foo" is the name of the tenth field.
A few other comments though:
Firstly, don't rely on the position of your fields, always explicitly list the field names. For example, it's better to go
INSERT INTO login (id, name) VALUES (1, 'Fred')
instead of
INSERT INTO login VALUES (1, 'Fred')
Point 2: You have directly embedded the value of $cleanurl into your query. Of course, you have to learn one thing at a time but be aware that this is very dangerous. If $cleanurl contains something like "'); DROP TABLE login;" then you might be in trouble. This is called SQL injection and is the source of constant security problems. Without going into too much detail, you should learn how to use prepared statements.
Point 3: PHP comes with a library called PDO which supports prepared statements. It also provides a common API for interacting with your database so if you find that you need to move from Mysql to another DBMS, PDO will abstract away most of the differences. By using the mysql_query function you lock yourself into using mysql.
You don't have to address all of these issues simultaneously but don't forget about them either, once you get familiar with PHP and SQL come back to the points about PDO and prepared statements.
First off: INSERT adds a new record to a table, UPDATE updates (overwrites) one or more existing records.
Second: UPDATE needs to know the name of the column to update, and which rows to update
UPDATE <tableName>
SET <columnName> = '$cleanurl'
WHERE <some condition to identify which record should be updated>
Thirdly: it's probably worth your while reading a few basic tutorials on MySQL/SQL