We're having a problem on a wiki site (using mysql 5.5.9). There is a text table, and a revision table where revision.rev_text_id is a foreign key to text.old_id (there isn't a new one, ask wikimedia).
There is an INSERT INTO text..., then INSERT INTO revision... which uses the new old_id/rev_text_id received from the previous query.
The second query works, the first - I'm not sure. The thing is after the whole thing is gone through (with a few dozen more queries) the revision row is there, with a fresh new value in it's rev_text_id column.
However, the text row isn't there. Funny thing - the text table auto increment advances, as the next action skips a value for it's old_id, the missing value matching what we have in revision table.
Is there a chance the first query makes the auto increment index advance though the row isn't actually inserted? What causes this?
ADDTITION
When I copied the INSERT INTO text query from the logs and ran it on the server - it executed fine (row was added to table)
EDIT
Full queries are:
INSERT INTO text (old_id,old_text,old_flags) VALUES (NULL,'{text input by user}','utf8');
INSERT INTO `revision` (rev_id,rev_page,rev_text_id,rev_comment,rev_minor_edit,rev_user,rev_user_text,rev_timestamp,rev_deleted,rev_len,rev_parent_id,rev_sha1) VALUES (NULL,'{pageId}','{textId}','{comment}','{isMinor}','{userId}','{userName}','{TS}','{isDeleted}','{length}','{parentRevision}','{HASH}')"
Also, if relevant, text table is running InnoDB and revision is MyISAM.
EDIT
More information from logs:
Bad request
BEGIN
INSERT INTO `text`
INSERT INTO `revision`
UPDATE `page` SET page_latest
INSERT INTO `recentchanges`
INSERT INTO `cu_changes`
SELECT wl_user FROM `watchlist`
SELECT user_id FROM `user`
SELECT user_id,user_name,user_real_name,user_password,user_newpassword,user_newpass_time,user_email,user_touched,use
SELECT ug_group FROM `user_groups`
SELECT up_property,up_value FROM `user_properties`
SELECT user_id,user_name,user_real_name,user_password,user_newpassword,user_newpass_time,user_email,user_touched,use
SELECT ug_group FROM `user_groups`
SELECT up_property,up_value FROM `user_properties`
SELECT lc_value FROM `l10n_cache`
SELECT lc_value FROM `l10n_cache`
...
A few more SELECTs. Pause of activity for 2 sec, then the log has commands from a new user, no more queries from this user (on same thread).
An edit by same user to a different page, which went well:
BEGIN
INSERT INTO `text`
INSERT INTO `revision`
UPDATE `page` SET page_latest
INSERT INTO `recentchanges`
INSERT INTO `cu_changes`
SELECT wl_user FROM `watchlist`
COMMIT
BEGIN
UPDATE `watchlist` SET wl_notificationtimestamp
COMMIT
BEGIN
SELECT user_id FROM `user`
SELECT user_id,user_name,user_real_name,user_password,user_newpasswo
SELECT ug_group FROM `user_groups`
SELECT up_property,up_value FROM `user_properties`
SELECT * FROM `user`
SELECT up_property,up_value FROM `user_properties`
INSERT INTO `logging`
UPDATE `user` SET user_editcount=user_editcount+1
SELECT 1 FROM `user`
UPDATE `user` SET user_touched = '20121227211743'
COMMIT
etc.
Is there a chance the first query makes the auto increment index advance though the row isn't actually inserted? What causes this?
Yes, when you try to insert a row in a table with an auto_increment column, the insertion will fail, but the auto_increment value will be incremented by 1.
If you decide to convert the tables to InnoDB, I suggest you to use transactions. This way if one the queries fails none of them will be inserted.
Related
Hi i am trying to insert data into another table and i would like to skip duplicate record in the target table. I have used the following mysql query.
insert into adggtnz.`reg02_maininfo`(farmermobile,farmername,farmergender,origin)
select * from (SELECT mobile_no,name,sex,'EADD' FROM EADD.farmer)
as tmp where not exists (select farmermobile from adggeth.`reg02_maininfo` where farmermobile = tmp.mobile_no)
The problem is that when there is a duplicate the query does not completely run how can i avoid the following error
16:09:03 insert into adggtnz.`reg02_maininfo`(farmermobile,farmername,farmergender,origin) select * from (SELECT mobile_no,name,sex,'EADD' FROM EADD.farmer) as tmp where not exists (select farmermobile from adggeth.`reg02_maininfo` where farmermobile = tmp.mobile_no) Error Code: 1062. Duplicate entry '0724961552' for key 'PRIMARY' 0.828 sec
Please help me modify my query
If you want to avoid duplicate entries, you never EVER query first to see if a record exists. You place a unique constraint and use INSERT IGNORE or INSERT INTO ... ON DUPLICATE KEY UPDATE.
The problem with first approach is that you can (and will) get false positives.
In your particular case, the fix is quite easy. You need to add IGNORE after INSERT. That will skip the record if duplicate and continue onto the next one.
INSERT IGNORE INTO adggtnz.`reg02_maininfo`(farmermobile,farmername,farmergender,origin)
SELECT mobile_no, name, sex, 'EADD' FROM EADD.farmer
Get the select query which initially checks the farmer mobile number in reg02_maininfo and then insert into reg02_maininfo.
insert into adggtnz.`reg02_maininfo`(farmermobile,farmername,farmergender,origin)
SELECT mobile_no,name,sex,'EADD' FROM EADD.farmer where mobile_no not in
(select farmermobile from adggeth.`reg02_maininfo`)
I have a table with primary key (its name is "id") defined as auto_increment. I use NULL in INSERT statements to "fill" the id value. It works, of course. However now I need to "move" an existing record to a new primary key value (the next available, the value is not so much important, but it must be a new one, and the last one if ordered by id). How can I do it in an "elegant" way? Since the "use NULL at INSERT" does not work too much with UPDATE:
update idtest set id=NULL where id=1;
This simply makes the id of the record zero. I would expect to do the same thing as with INSERT, but it seems my idea was incorrect.
Of course I can use "INSERT ... SELECT" statement, then a DELETE on the old one, or I can use something like MAX(id) + 1 to UPDATE the id of the old record in one step, etc, but I am curious if there is a finer solution.
Also, the MAX(id) solution doesn't seem to work either by the way:
mysql> update idtest set id=max(id)+1 where id=3;
ERROR 1111 (HY000): Invalid use of group function
mysql> update idtest set id=(select max(id)+1 from idtest) where id=3;
ERROR 1093 (HY000): You can't specify target table 'idtest' for update in FROM clause
This is the way I believe:
UPDATE users SET id = (SELECT `AUTO_INCREMENT`
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = 'test'
AND TABLE_NAME = 'users') WHERE id = 2;
select * from users;
I used by own tables substitute yours.
test is database name, users is table name and id is AUTO_INCREMENT in my case.
EDIT: My Query above works perfect but its side effects are somewhat 'dangerous', upon next insert as AUTO_INCREMENT value will collide with this recently updated record so just next single insert will fail. To avoid that case I've modified above query to a transaction:
START transaction;
UPDATE users SET id = (SELECT `AUTO_INCREMENT`
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = 'test'
AND TABLE_NAME = 'users') WHERE id = 2;
#renew auto increment to avoid duplicate warning on next insert
INSERT IGNORE INTO users(username) values ('');
COMMIT
Hope this will help someone if not OP.
The way you are trying to update same table is wrong but you can use join on same table
update idtest t
join (select id +1 as id
from idtest order by id desc
limit 1) t1
set t.id=t1.id
where t.id=3;
or
update idtest t
join (select max(id) +1 as id
from idtest ) t1
set t.id=t1.id
where t.id=3;
You can use the REPLACE INTO clause to do the trick.
From the manual:
REPLACE works exactly like INSERT, except that if an old row in the table has the same value as a new row for a PRIMARY KEY or a UNIQUE index, the old row is deleted before the new row is inserted. See Section 13.2.5, "INSERT Syntax".
EDIT
My mistake (in the comments) that you have to have two unique constraint to achieve this:
When you use the auto_increment value to REPLACE the record, the record will be replaced with the give ID and will not change (however the AI value will increment).
You have to exclude the AI column from the query. You can do that if you have one more UQ constraint.
Check this SQLFiddle demo: http://sqlfiddle.com/#!2/1a702e
The first query will replace all the records (but the id's value will not change).
The second one will replace it too, and the new AI value will be used. (Please note, that the second query does not contain the id column, and there is a UQ constraint on the some column).
You can notice, that the second query uses higher AI values than it is excepted: this is because the first replace incremented the AI value.
If you do not have two unique keys (one for the AI and one for another columns), the REPLACE statement will work as a normal INSERT statement!
(Ofcourse you can change one of the UNIQUE KEYs with a PRIMARY KEY)
I have a table with 3 fields: Id(PK,AI), Name(varchar(36)), LName(varchar(36)).
I have to insert name and last name, Id inserts automatically because of it's constraints,
Is There a way to Jump id auto increment value when it reaches 6?
for instance do this 7 times:
Insert Into table(Name, LName) Values ('name1', 'lname1') "And jump id to 7 if it is going to be 6"
It may sound stupid to do this but I have the doubt.
Also Jump and do not record id 6.
record only, 1-5, 7,8,9 and so on
What I want to achieve starts from a Union:
Select * From TableNames
Union All
Select * From TableNames_general
In the TableNames_general I assign it's first value so that when the user sees the table for the first time it will be displayed the record I inserted.
The problem comes when the user inserts a new record, if the Id of the inserted record is the same as the one I have inserted it will be duplicated, that is why I want to achieve when the users inserts one record and if the last insert id already exists just jump that record. this is because I must have different ids due to its relationship among child tables.
Identity column generate values for you, And its best left this way, You have the ability to insert specific values in Identity column but its best left alone and let it generate values for you.
Imagine you have inserted a value explicitly in an identity column and then later on Identity column generates the same value for you, you will end up with duplicates.
If you want to have your input in that column then why bother with identity column anyway ??
Well this is not the best practice but you can jump to a specific number by doing as follows:
MS SQL SERVER 2005 and Later
-- Create test table
CREATE TABLE ID_TEST(ID INT IDENTITY(1,1), VALUE INT)
GO
-- Insert values
INSERT INTO ID_TEST (VALUE) VALUES
(1),(2),(3)
GO
-- Set idnentity insert on to insert values explicitly in identity column
SET IDENTITY_INSERT ID_TEST ON;
INSERT INTO ID_TEST (ID, VALUE) VALUES
(6, 6),(8,8),(9,9)
GO
-- Set identity insert off
SET IDENTITY_INSERT ID_TEST OFF;
GO
-- 1st reseed the value of identity column to any smallest value in your table
-- below I reseeded it to 0
DBCC CHECKIDENT ('ID_TEST', RESEED, 0);
-- execute the same commad without any seed value it will reset it to the
-- next highest idnetity value
DBCC CHECKIDENT ('ID_TEST', RESEED);
GO
-- final insert
INSERT INTO ID_TEST (VALUE) VALUES
(10)
GO
-- now select data from table and see the gap
SELECT * FROM ID_TEST
If you query the database to get the last inserted ID, then you can check if you need to increment it, by using a parameter in the query to set the correct ID.
If you use MSSQL, you can do the following:
Before you insert check for the current ID, if it's 5, then do the following:
Set IDENTITY_INSERT to ON
Insert your data with ID = 7
Set IDENTITY_INSERT to OFF
Also you might get away with the following scenario:
check for current ID
if it's 5, run DBCC CHECKIDENT (Table, reseed, 6), it will reseed the table and in this case your next identity will be 7
If you're checking for current identity just after INSERT, you can use SELECT ##IDENTITY or SELECT SCOPE_IDENTITY() for better results (as rcdmk pointed out in comments)
Otherwise you can just use select: SELECT MAX(Id) FROM Table
There's no direct way to influence the AUTO_INCREMENT to "skip" a particular value, or values on a particular condition.
I think you'd have to handle this in an AFTER INSERT trigger. An AFTER INSERT trigger can't update the values of the row that was just inserted, and I don't think it can make any modifications to the table affected by the statement that fired the trigger.
A BEFORE INSERT trigger won't work either, because the value assigned to an AUTO_INCREMENT column is not available in a BEFORE INSERT trigger.
I don't believe there's a way to get SQL Server IDENTITY to "skip" a particular value either.
UPDATE
If you need "unique" id values between two tables, there's a rather ugly workaround with MySQL: roll your own auto_increment behavior using triggers and a separate table. Rather than defining your tables with AUTO_INCREMENT attribute, use a BEFORE INSERT trigger to obtain a value.
If an id value is supplied, and it's larger than the current maximum value from the auto_increment column in the dummy auto_increment_seq table, we'd need to either update that row, or insert a new one.
As a rough outline:
CREATE TABLE auto_increment_seq
(id INT NOT NULL PRIMARY KEY AUTO_INCREMENT) ENGINE=MyISAM;
DELIMITER $$
CREATE TRIGGER TableNames_bi
BEFORE INSERT ON TableNames
FOR EACH ROW
BEGIN
DECLARE li_new_id INT UNSIGNED;
IF ( NEW.id = 0 OR NEW.id IS NULL ) THEN
INSERT INTO auto_increment_seq (id) VALUES (NULL);
SELECT LAST_INSERT_ID() INTO li_new_id;
SET NEW.id = li_new_id;
ELSE
SELECT MAX(id) INTO li_max_seq FROM auto_increment_seq;
IF ( NEW.id > li_max_seq ) THEN
INSERT INTO auto_increment_seq (id) VALUES (NEW.id);
END IF;
END IF;
END$$
CREATE TRIGGER TableNames_ai
AFTER INSERT ON TableNames
FOR EACH ROW BEGIN
DECLARE li_max_seq INT UNSIGNED;
SELECT MAX(id) INTO li_max_seq FROM auto_increment_seq;
IF ( NEW.id > li_max_seq ) THEN
INSERT INTO auto_increment_seq (id) VALUES (NEW.id);
END IF;
END;
DELIMITER ;
The id column in the table could be defined something like this:
TableNames
( id INT UNSIGNED NOT NULL DEFAULT 0 PRIMARY KEY
COMMENT 'populated from auto_increment_seq.id'
, ...
You could create an identical trigger for the other table as well, so the two tables are effectively sharing the same auto_increment sequence. (With less efficiency and concurrency than an Oracle SEQUENCE object would provide.)
IMPORTANT NOTES
This doesn't really insure that the id values between the tables are actually kept unique. That would really require a query of the other table to see if the id value exists or not; and if running with InnoDB engine, in the context of some transaction isolation levels, we might be querying a stale (as in, consistent from the point in time at the start of the transaction) version of the other table.
And absent some additional (concurrency killing) locking, the approach outline above is subject to a small window of opportunity for a "race" condition with concurrent inserts... the SELECT MAX() from the dummy seq table, followed by the INSERT, allows a small window for another transaction to also run a SELECT MAX(), and return the same value. The best we can hope for (I think) is for an error to be thrown due to a duplicate key exception.
This approach requires the dummy "seq" table to use the MyISAM engine, so we can get an Oracle-like AUTONOMOUS TRANSACTION behavior; if inserts to the real tables are performed in the context of a REPEATABLE READ or SERIALIZABLE transaction isolation level, reads of the MAX(id) from the seq table would be consistent from the snapshot at the beginning of the transaction, we wouldn't get the newly inserted (or updated) values.
We'd also really need to consider the edge case of an UPDATE of row changing the id value; to handle that case, we'd need BEFORE/AFTER UPDATE triggers as well.
I've a java program to update a SQL table (id, name, status). Entire table is updated with same data or some changed data. How can I track if a row is same like it was before update or it has some modified data? id will be always same, only small typo on name. I just want to check on next update if name is modified. In this case the status field should be changed from 'same' to 'modified'. Will timestamp solve my issue? Please help.
1 - If you are looking to audit the table (inserts, updates, deletes), look at my how to prevent unwanted transactions slide deck w/code - http://craftydba.com/?page_id=880.
SEE CODE AT END!
The trigger that fills the audit table can hold information from multiple tables since the data is saved as XML. Therefore, you can un-delete if necessary. It tracks who and what made the change.
2 - If you are never going to purge the data from the audit table, why not mark the row as deleted but keep it for ever?
Many systems like people soft use effective dating to show if a record is no longer active. In the BI world this is called a type 2 dimensional table (slowly changing dimensions).
See the data warehouse institute article. http://www.bidw.org/datawarehousing/scd-type-2/
Each record has a begin and end date. All active records have a end date of null.
3 - Micorsoft SQL Server introduced the change data capture feature. While this tracks data change using a LOG reader after the fact, it lacks things like who and what made the change.
Again, all the above solutions work. I am partial to my solution!
Sincerely
John
The Crafty DBA
--
-- 7 - Auditing data changes (table for DML trigger)
--
-- Delete existing table
IF OBJECT_ID('[AUDIT].[LOG_TABLE_CHANGES]') IS NOT NULL
DROP TABLE [AUDIT].[LOG_TABLE_CHANGES]
GO
-- Add the table
CREATE TABLE [AUDIT].[LOG_TABLE_CHANGES]
(
[CHG_ID] [numeric](18, 0) IDENTITY(1,1) NOT NULL,
[CHG_DATE] [datetime] NOT NULL,
[CHG_TYPE] [varchar](20) NOT NULL,
[CHG_BY] [nvarchar](256) NOT NULL,
[APP_NAME] [nvarchar](128) NOT NULL,
[HOST_NAME] [nvarchar](128) NOT NULL,
[SCHEMA_NAME] [sysname] NOT NULL,
[OBJECT_NAME] [sysname] NOT NULL,
[XML_RECSET] [xml] NULL,
CONSTRAINT [PK_LTC_CHG_ID] PRIMARY KEY CLUSTERED ([CHG_ID] ASC)
) ON [PRIMARY]
GO
-- Add defaults for key information
ALTER TABLE [AUDIT].[LOG_TABLE_CHANGES] ADD CONSTRAINT [DF_LTC_CHG_DATE] DEFAULT (getdate()) FOR [CHG_DATE];
ALTER TABLE [AUDIT].[LOG_TABLE_CHANGES] ADD CONSTRAINT [DF_LTC_CHG_TYPE] DEFAULT ('') FOR [CHG_TYPE];
ALTER TABLE [AUDIT].[LOG_TABLE_CHANGES] ADD CONSTRAINT [DF_LTC_CHG_BY] DEFAULT (coalesce(suser_sname(),'?')) FOR [CHG_BY];
ALTER TABLE [AUDIT].[LOG_TABLE_CHANGES] ADD CONSTRAINT [DF_LTC_APP_NAME] DEFAULT (coalesce(app_name(),'?')) FOR [APP_NAME];
ALTER TABLE [AUDIT].[LOG_TABLE_CHANGES] ADD CONSTRAINT [DF_LTC_HOST_NAME] DEFAULT (coalesce(host_name(),'?')) FOR [HOST_NAME];
GO
--
-- 8 - Make DML trigger to capture changes
--
-- Delete existing trigger
IF OBJECT_ID('[ACTIVE].[TRG_FLUID_DATA]') IS NOT NULL
DROP TRIGGER [ACTIVE].[TRG_FLUID_DATA]
GO
-- Add trigger to log all changes
CREATE TRIGGER [ACTIVE].[TRG_FLUID_DATA] ON [ACTIVE].[CARS_BY_COUNTRY]
FOR INSERT, UPDATE, DELETE AS
BEGIN
-- Detect inserts
IF EXISTS (select * from inserted) AND NOT EXISTS (select * from deleted)
BEGIN
INSERT [AUDIT].[LOG_TABLE_CHANGES] ([CHG_TYPE], [SCHEMA_NAME], [OBJECT_NAME], [XML_RECSET])
SELECT 'INSERT', '[ACTIVE]', '[CARS_BY_COUNTRY]', (SELECT * FROM inserted as Record for xml auto, elements , root('RecordSet'), type)
RETURN;
END
-- Detect deletes
IF EXISTS (select * from deleted) AND NOT EXISTS (select * from inserted)
BEGIN
INSERT [AUDIT].[LOG_TABLE_CHANGES] ([CHG_TYPE], [SCHEMA_NAME], [OBJECT_NAME], [XML_RECSET])
SELECT 'DELETE', '[ACTIVE]', '[CARS_BY_COUNTRY]', (SELECT * FROM deleted as Record for xml auto, elements , root('RecordSet'), type)
RETURN;
END
-- Update inserts
IF EXISTS (select * from inserted) AND EXISTS (select * from deleted)
BEGIN
INSERT [AUDIT].[LOG_TABLE_CHANGES] ([CHG_TYPE], [SCHEMA_NAME], [OBJECT_NAME], [XML_RECSET])
SELECT 'UPDATE', '[ACTIVE]', '[CARS_BY_COUNTRY]', (SELECT * FROM deleted as Record for xml auto, elements , root('RecordSet'), type)
RETURN;
END
END;
GO
SQL Server doesn't have versioning on the table level. If you want to track difference between two field you have at least two options:
Control from your Java application - do pre-update check in your update method.
Control from SQL Server - write trigger again for pre-update check
You can also create suplimentary field where Version number will be kept
Yes you can use time stamp.
By using time stamp, you can find out the latest entry into the table and by using Order by timestamp in the query,you can get the latest and set the status based on corresponding value
It depends on how much information you need. If all you care about is whether the record has ever been modified, you can use created_when and updated_when fields. If the latter is greater than the former, it's been updated.
If you want to know what fields have been updated, you have to log the changes. The details depend on your requirements. If you need to log changes, a trigger is the best way to do it.
Is it possible to perform update like insert?
UPDATE `table` SET `value` ('N','N','N','N','Y','Y','Y','N', 'N') WHERE `my_id` = '1'
The problem is that the number of values to be inserted i dont now. It can be a 5 or 10.
replace is just like insert, it just checks if there is duplicate key and if it is it deletes the row, and inserts the new one, otherwise it just inserts
you can do this if there is for example unique index of (Name,Type) and if you type the following command
REPLACE INTO table1 (Name,Type,InitialValue,FinalValue) VALUES ('A',3,50,90 )
and there already exists a row with Name = 'A' and Type = 3 it will be replaced
CREATE UNIQUE INDEX idx_name_type ON table1(Name,Type)
EDIT: a quick note - REPLACE always DELETES and then INSERTs, so it is never a very good idea to use it in heavy load because it needs exclusive lock when it deletes, and then when it inserts
some of the database engines have
INSERT ... ON DUPLICATE KEY UPDATE ...
You have to specify the column-name.
UPDATE `table` SET `Col1`='y',`Col2`='n' ... WHERE `ID`='1'
Or want to update one or more columns/rows:
UPDATE `table` Set Col1='Y' WHERE `ID` IN ('1','11','13')