I am getting values of a business by the API and inserting it into a database.
SNO ID category
1 aaa Machine learning
2 aaa AI
3 bbb mobile
Where SNO is the primary key which is set to auto increment.
Now after 2 days, for keeping my database up to date I need to get the new data from the API.
So suppose that I come to know that ID ‘aaa’ now has one more category as “data structures”
Question:
How can I update my table to reflect this new category?
I am expecting something like
SNO ID category
1 aaa Machine learning
2 aaa AI
3 bbb mobile
4 aaa data structures
I don’t know the SNO as they are Auto Incremented.
Deleting all the rows which has ID = “aaa” and then Inserting it again is one option but I am trying to avoid that as it might increase the overheads.
I am getting new values from the API,
So I am getting Machine learning, AI and Data structures ( all 3 )
If I use , so in my code when I iterate over the category variable 3 SQL Will be generated
INSERT INTO tablename (ID, category) VALUES ('aaa', ‘Machine learning’);
INSERT INTO tablename (ID, category) VALUES ('aaa', ‘AI’);
INSERT INTO tablename (ID, category) VALUES ('aaa', 'data structures');
So in this case 1st and 2nd insert statement will be duplicated, my table will have duplicate rows
.
.
I basically need to check if the ID and Category exists in the BD if they do not then INSERT ?( The Primary Key SNO is set to auto Increment)
+ Options
Field Type Null Key Default Extra
uuid varchar(50) NO NULL
categoryName varchar(50) NO NULL
sno int(5) NO PRI NULL
If you just insert the new "data structures" row, it will have the next auto-incremented number, which in your case would likely be 4. The only time it wouldn't be 4 is if you inserted some rows and deleted them, which would cause the auto increment counter to be higher. You could just insert the row like this:
INSERT INTO tablename (ID, category) VALUES ('aaa', 'data
structures');
Or maybe I'm not understanding the question correctly. If not reply back and I'll try to help.
You can simply INSER the new row by background checking or INSERT ... ON DUPLICATE KEY UPDATE Syntax or UPDATE if exist.
Related
While demonstrating the INSERT statement to the students of my SQL course, we've come up on some odd behavior in MySQL 8.0. Please help us learn what is happenning. (No need for workarounds as we're aware of a few and this is for learning, not for production. Thank you)
We are creating a new database and copying some rows from the well-known Sakila sample DB, like so:
CREATE DATABASE simpsons;
USE simpsons;
CREATE TABLE `character` (
character_id smallint unsigned NOT NULL AUTO_INCREMENT,
first_name VARCHAR(20) NOT NULL,
last_name VARCHAR(20),
shoe_size INT,
PRIMARY KEY (character_id));
INSERT INTO `character`
(first_name, last_name)
SELECT
first_name, last_name
FROM
sakila.actor;
When we do this and SELECT * FROM ``character`` we see that all 200 records from sakila.actor have been copied correctly over to the new character table.
The last row gets the value 200 for its character_id auto-incremented PK. The output window shows no errors in any of the above commands.
Then, when we immediately add one more record manually:
INSERT INTO `character`
(first_name, last_name, shoe_size)
VALUES
('Bart', 'Simpson', 35);
Quite oddly, we find that this record gets the value 256 as its character_id and not 201.
This is despite the fact that running SHOW VARIABLES LIKE 'auto_inc%'; shows that both auto_increment_increment and auto_increment_offset are set to 1.
We would like to learn why does MySQL skip 56 numbers?
Please note, this question is different from MySQL InnoDB auto_increment value increases by 2 instead of 1. Virus? and MySQL autoincrement column jumps by 10- why? because auto_incerement_increment is 1, there are no DELETE operations in our (easily reproducible) scenario and we each are the only users of our prospective DBs. Plus none of the answers to that question are conclusive as to what actually happened. Finally, please see #Postman's wonderful answer which references a root cause not mentioned in any of the answers to the above questions. Thank you
This behavior has something to do with "bulk inserts" and the innodb_autoinc_lock_mode setting.
As far as I understand it (the documentation isn't quite clear about this), when you use a INSERT INTO ... SELECT statement, MySQL cannot know how many rows are actually being inserted before running the query, but the IDs for the new AUTO_INCREMENT values have to be reserved when using innodb_autoinc_lock_mode=1 (consecutive) or 2 (interleaved). From my observation it reserves a set of AUTO_INCREMENT numbers where the count is a power of 2 (cannot confirm this, only a guess). See the following example:
CREATE TABLE sourceTable(
id INT AUTO_INCREMENT PRIMARY KEY,
name VARCHAR(20)
);
CREATE TABLE targetTable(
id INT AUTO_INCREMENT PRIMARY KEY,
original VARCHAR(30)
);
INSERT INTO sourceTable(name) VALUES ('one');
INSERT INTO sourceTable(name) VALUES ('two');
INSERT INTO sourceTable(name) VALUES ('three');
INSERT INTO sourceTable(name) VALUES ('four');
INSERT INTO sourceTable(name) VALUES ('five');
INSERT INTO targetTable(original) SELECT name FROM sourceTable;
INSERT INTO targetTable(original) VALUES ('manual');
SELECT * FROM targetTable;
This will generate the following output:
+----+----------+
| id | original |
+----+----------+
| 1 | one |
| 2 | two |
| 3 | three |
| 4 | four |
| 5 | five |
| 8 | manual |
+----+----------+
When inserting the 5 rows from the source table, it reserves the next 8 possible AUTO_INCREMENT values because that is the closest power of 2 number greater than 5. However, it will use only 5 of them since you insert only 5 rows.
In your case, you are inserting 200 rows, so the closest power of 2 number greater than 200 would be 256. So you have a "gap" of 56 missing AUTO_INCREMENT values and the next entry gets the ID 256.
Consider following tables:
product
+----+------+-------------+
| id | name | category_id |
+----+------+-------------+
| 1 | foo | 1 |
+----+------+-------------+
categories
+----+------+
| id | name |
+----+------+
| 1 | food |
+----+------+
Now lets assume I someone POSTs a new product:
{
"name": "bar",
"category": "drink"
}
In this case we need to create the new category automatically:
INSERT IGNORE INTO categories (name) VALUES ('drink')
Then we finally could insert the actual product row:
INSERT INTO products (name, category_id) VALUES ('bar', SELECT id FROM categories WHERE name = 'drink')
However though this works it would require a transaction setup to be safe and as this does not seem to be a super complicated query I would like to know if it would be possible to merge both queries together (e.g. put the insert query of the categories into the select subquery of the product insertion)?
I would advise against INSERT IGNORE, for the reasons that #Bill Karwin so eloquently explains. Furthermore, as documented under INSERT ... ON DUPLICATE KEY UPDATE Syntax:
If a table contains an AUTO_INCREMENT column and INSERT ... ON DUPLICATE KEY UPDATE inserts or updates a row, the LAST_INSERT_ID() function returns the AUTO_INCREMENT value. Exception: For updates, LAST_INSERT_ID() is not meaningful prior to MySQL 5.1.12. However, you can work around this by using LAST_INSERT_ID(expr). Suppose that id is the AUTO_INCREMENT column. To make LAST_INSERT_ID() meaningful for updates, insert rows as follows:
INSERT INTO table (a,b,c) VALUES (1,2,3)
ON DUPLICATE KEY UPDATE id=LAST_INSERT_ID(id), c=3;
One can then obtain the id (whether preexisting or newly inserted) as documented under How to Get the Unique ID for the Last Inserted Row:
If you insert a record into a table that contains an AUTO_INCREMENT column, you can obtain the value stored into that column by calling the mysql_insert_id() function.
[ deletia ]
When a new AUTO_INCREMENT value has been generated, you can also obtain it by executing a SELECT LAST_INSERT_ID() statement with mysql_query() and retrieving the value from the result set returned by the statement.
So, for example, one could simply do:
INSERT INTO categories (name) VALUES ('drink')
ON DUPLICATE KEY UPDATE id=LAST_INSERT_ID(id);
INSERT INTO products (name, category_id) VALUES ('bar', LAST_INSERT_ID());
Of course, these statements would still need to be executed within a transaction if you require atomicity.
No, that is not possible. What you can do however, is creating a BEFORE INSERT trigger on the product table and check inside the trigger, if the category already exists. In case it doesn't exist, you can create it there.
The trigger will be automatically executed in the same transaction as your INSERT-statement, so no additional transactions necessary there.
The only problem I see is, when you forget about the Trigger, you might start wondering why categories appear automatically for each new product with a new category (something similar happened to me).
Update:
You can see example of simple triggers in the MySQL Documentation
You can access the new to be inserted value in a BEFORE INSERT trigger with NEW.column_name so for you it would be NEW.category
I have a table that have two fields.
table test
{
fname char(20),
id int not null auto_increment,
primary key(id)
}
now I add 3 records to the table like below:
insert into test(fname) values
('a'),('b'),('c');
and the table looks like
fname id
a 1
b 2
c 3
now I delete b from table so I have:
fname id
a 1
b 3
now again I insert a new record into the table
insert into test(fname) values('d);
and get:
fname id
a 1
b 3
d 4
but I want last record's id to be "2"
how can I do this?
An auto increment column would be used to identify your rows as unique if you have no other candidate for a primary key. If you are relying on their being no gaps in your sequence then you have trouble with the logic of how you are approching the problem, your queries should not rely on anything other than them being unique.
Also find a piece of MySQL Cookbook chapter that says the same
I don't think you can change that. This is houw auto_increment works in mysql.
You can't do that with autoincrement. It only keeps the id of the last inserted record and increments it when you insert. It doesn't keep track of delete operations.
Anyway, why do you want to do it?
is there a way in SQL to create the constraint that a column has to be unique, if a specific column has a specific value?
Example: the rows are not really deleted, but marked as 'deleted' in the database. And within the 'not-deleted' rows, ValueA has to be unique:
ID ValueA ValueB Deleted
-----------------------------------------------------
1 'foo' 10 0
2 'bar' 20 0
3 'bar' 30 1
4 'bar' 40 1
5 'foo' 50 0 --NOT ALLOWED
I thought of something like a CHECK constraint, however I don't know how to do this.
with SQL92 this is not possible, may be you could implement something with a trigger
Can you change the design a little bit?
It seems to me that you have a list of "thingies". For each ValueA, there's a single active "thingy" at any one time. This can best be modeled as follows:
Remove ValueA and Deleted from your main Thingies table.
Create a new table ActiveThingies with columns ValueA and ID. Protect this table by making ValueA a unique or primary key. (You may also need to make ID unique as well depending on whether a single ID can represent more than 1 ValueA).
Now, use the ActiveThingies table to control which record is current at any time. To change the active (non-deleted) record for "foo", update it's ID column in ActiveThingies.
To get your list of non-deleted items join the two tables.
With this design, however, you will lose the ability to remember the ValueA for "deleted" "thingies". If you need to remember those values, you will also need to include the ValueA column in Thingies.
There is workaround this problem - create another column deleted_on
deleted_on timestamp NOT NULL DEFAULT '0000-00-00 00:00:00'
and make unique key on both ValueA and deleted_on
UNIQUE KEY not_deleted (ValueA, deleted_on)
When soft deleting a record insert NOW() for value of deleted_on
MySQL ignores CHECK constraints, so you cannot do this in MySQL as you might in another database.
Here is a hack. Unique constraint on valueA + deleted. When deleting rows you cannot use just 1, they must be 1, 2, 3...
This at least lets you do it server-side in MySQL, but introduces a step. When marking a row for deletion, you have to first go find the max(deleted), add 1, and plug that value in when marking for deletion.
Split your table into two tables: One which has a UNIQUE constraint on ValueA and one that doesn't. Use a view+triggers to combine the two tables. Something like:
CREATE TABLE _Active (
ID INTEGER,
ValueA VARCHAR(255) UNIQUE,
ValueB INTEGER
);
CREATE TABLE _Deleted (
ID INTEGER,
ValueA VARCHAR(255), /* NOT unique! */
ValueB INTEGER
);
CREATE VIEW Thingies AS
SELECT ID, ValueA, ValueB, 0 AS Deleted FROM _Active
UNION ALL
SELECT ID, ValueA, ValueB, 1 AS Deleted FROM _Deleted;
CREATE TRIGGER _trg_ii_Thingies_Active
INSTEAD OF INSERT ON Thingies
FOR EACH ROW WHEN NOT NEW.Deleted
BEGIN
INSERT INTO _Active(ID, ValueA, ValueB)
VALUES (NEW.ID, NEW.ValueA, NEW.ValueB);
END;
CREATE TRIGGER _trg_ii_Thingies_Deleted
INSTEAD OF INSERT ON Thingies
FOR EACH ROW WHEN NEW.Deleted
BEGIN
INSERT INTO _Deleted(ID, ValueA, ValueB)
VALUES (NEW.ID, NEW.ValueA, NEW.ValueB);
END;
/* Add triggers for DELETE and UPDATE as appropriate */
(I'm not sure about the CREATE TRIGGER syntax, but you know what I mean.)
For the sake of simplicity lets say I have a table with 3 columns; id, parent_id and name. In this table id is my auto-incrementing primary key. I want to group multiple names together in this table, to do this all names in a group will share the same parent_id. If I am inserting the first name in the group I want the id=parent_id, if i am inserting another name I want to specify a specific parent_id to place that name into a specific group. It would be nice if I could define a default for that column to be the same as the id, if I specify a value for parent_id in the insert query then I would like it to use that value. I know you can set a default to be a specific static value, but can you specify the default to be the same as that row's auto-incrementing primary key? Perhaps this is a job for a trigger or stored procedure?
(I know I could obtain the primary key generated by the last insert and then update the table, but that's 2 quires I'd rather not burn.)
Thanks!
This is a job of a trigger!
CREATE TRIGGER NAME1 AFTER INSERT ON TABLE1
BEGIN
UPDATE TABLE1 SET parent_id = id WHERE (parent_id IS NULL OR parent_id = '');
END;
INSERT INTO TABLE1 (id,parent_id) VALUES (null,null); -- parent_id will be equal to id
INSERT INTO TABLE1 (id,parent_id) VALUES (null,'1'); -- parent_id will be 1
INSERT INTO TABLE1 (id,parent_id) VALUES (null,'2'); -- parent_id will be 2