Order of INSERT or UPDATE in a transaction in binlog - mysql

I'm using debezium connector to stream my Mysql binlog change records to Kafka Topic. My transaction is like so:
Tx begin
Insert INTO Project(Name, InProgress) VALUES ('New Project', 0);
-- We get the created id (say PID)
Insert INTO Tasks(TaskName, ProjectID) VALUES ('Task A', PID);
-- We get the taskID (say TID1)
Insert INTO Tasks(TaskName, ProjectID) VALUES ('Task B', PID);
-- We get the taskID (say TID2)
UPDATE Tasks SET TaskName = 'Task C' where ID = TID2;
Update Project SET InProgress = 1 where ID = PID;
Tx end
My question is in what order will I get the bin log records? Will I get the records in the order in which the records are inserted? So like this?
NEW Project with ID = PID
NEW Task with ID = TID1
NEW Task with ID = TID2
Is this documented anyplace?

Related

rollback transaction based on condition

I want to abort batch sql query transactions if column value is less than or equal to user's input value and alert user.
Here's what happens:
available_qty value is subtracted from user's input qty value and the value is updated in food_menu table.
user's order records are inserted into food_order table.
Here's my query:
foreach($menuData as $item=>$qty)
{
if(empty($qty) || $qty == '0')
continue;
$sqlArray[] = "INSERT INTO food_order
(user_id, item_id, quantity, order_type, updated_on, updated_by, created_by, created_on)
VALUES ($uid, $item, $qty, '$menu_type', NOW(), $uid, $uid, NOW())";
$sqlArray[] = "UPDATE food_menu SET available_qty = IF($qty >= available_qty, available_qty - $qty, available_qty) WHERE item_id = $item AND type = '$menu_type' AND DATE(updated_on) = CURRENT_DATE";
}
return (boolean) $this->db->exec($sqlArray);
if $qty value is greater than available_qty field value, I want to abort the whole series of transactions in $sqlArray[].
Can I do that in the query itself?
I think a much better approach is to check the available quantity with remaining quantity before the start of your insert and update statement. But for the sake of information you can try the following:
Create a before update trigger on food_menu table. In the trigger check the available quantity of it is negative meaning the order quantity is more than available quantity then use the SIGNAL statement to raise an error which you will have to catch in your code and do the rollback and show appropriate message to the user.

Mysql integrity error 1452 on insert

select * from memory_games_game; gives me following table:
select * from memory_games_game_state;gives me following table:
I have a stored proc as bellow:
delimiter //
CREATE PROCEDURE get_game_by_user_id(
p_user_id int, p_game_id int
)
BEGIN
insert into memory_games_game_state(user_id, game_id, level, progress)
SELECT 24 as user_id,
game.game_type as game_id,
1 as level,
0 as progress
FROM memory_games_game game
left outer join memory_games_game_state gameState on
game.game_type=gameState.game_id and
gameState.user_id=24
where game.level=1 and gameState.user_id is null;
if p_game_id = -1 then
SELECT gameState.level, game_type, `current_date`
FROM memory_games_game game join memory_games_game_state gameState on
game.game_type=gameState.game_id and
gameState.user_id=24 and
game.level=gameState.level;
else
SELECT gameState.level, game_type, `current_date`
FROM memory_games_game game join memory_games_game_state gameState on
game.game_type=gameState.game_id and
gameState.user_id=p_user_id and
game.level=gameState.level
WHERE game_type=12;
end if;
END
//
The first insert inserts the records into memory_games_game_statetable.
This insert is successful for game_id from 8 to 11 however, it fails for 12 with following error:
I am able to insert records in table memory_games_gamefor game_type 12 which is nothing but game_id in the other table i.e. memory_games_game_state
What's going wrong?
UPDATE:
My django models:
class Game(models.Model):
#Field for storing game type
GAME_TYPE_CHOICES = (
(8, 'Simon Game'),
(9, 'Pinpoint reaction'),
(10, 'Loslassen'),
(11, 'Word pair'),
(12, 'Wortschatz')
)
game_type = models.IntegerField(choices=GAME_TYPE_CHOICES)
level = models.IntegerField(default='1')
#This helps to print in admin interface
def __str__(self):
return u"%s level %s" % (self.get_game_type_display(), self.level)
class Game_state(models.Model):
game = models.ForeignKey(Game, blank=True, null=True)
user = models.ForeignKey(User, blank=True, null=True)
level = models.IntegerField(default='1')
progress = models.IntegerField(default='0')
current_date = models.DateField(auto_now=True)
class Game_state_ui_model(models.Model):
GAME_TYPE_CHOICES = (
(8, 'Simon Game'),
(9, 'Pinpoint reaction'),
(10, 'Loslassen'),
(11, 'Word pair'),
(12, 'Wortschatz')
)
game_type = models.IntegerField(choices=GAME_TYPE_CHOICES)
level = models.IntegerField()
user_id = models.IntegerField(default='0')
current_date = models.DateField(auto_now=True)
# static method to fetch games for a paricular user
#staticmethod
def fetch_games(user_id, game_id):
print("Fetching games in model")
# create a cursor
cur = connection.cursor()
# execute the stored procedure passing in
# search_string as a parameter
cur.callproc('get_game_by_user_id', [user_id, game_id,])
# grab the results
results = cur.fetchall()
cur.close()
Game_state_list=[]
for row in results:
print("Get game", row)
Gs = Game_state_ui_model()
Gs.level=row[0]
Gs.game_type=row[1]
Gs.current_date=row[2]
Game_state_list.append(Gs)
return Game_state_list
As the error states, game_id references memory_games_game.id; NOT memory_games_game.game_type.
The thinking that "game_type 12 which is nothing but game_id in the other table i.e. memory_games_game_state" is incorrect.
You need a row in memory_games_game with id = 12.
I dropped all the tables and did migration again, which somehow solved the problem. I didn't change anything else.

MySQL INSERT multiple rows with same WHERE clause

I am currently able to insert rows into a table using the following SQL:
INSERT INTO ots (score_id,tab_id) VALUES ((SELECT id FROM score WHERE score_id = 11),(SELECT tab_id FROM tab WHERE layover_state = 7))
INSERT INTO ots (score_id,tab_id) VALUES ((SELECT id FROM score WHERE score_id = 12),(SELECT tab_id FROM tab WHERE layover_state = 7))
INSERT INTO ots (score_id,tab_id) VALUES ((SELECT id FROM score WHERE score_id = 19),(SELECT tab_id FROM tab WHERE layover_state = 9))
INSERT INTO ots (score_id,tab_id) VALUES ((SELECT id FROM score WHERE score_id = 27),(SELECT tab_id FROM tab WHERE layover_state = 10))
INSERT INTO ots (score_id,tab_id) VALUES ((SELECT id FROM score WHERE score_id = 45),(SELECT tab_id FROM tab WHERE layover_state = 10))
INSERT INTO ots (score_id,tab_id) VALUES ((SELECT id FROM score WHERE score_id = 51),(SELECT tab_id FROM tab WHERE layover_state = 10))
INSERT INTO ots (score_id,tab_id) VALUES ((SELECT id FROM score WHERE score_id = 52),(SELECT tab_id FROM tab WHERE layover_state = 11))
The above INSERT INTO statements work correctly. However, they seem very inefficient since there should be some way to do this with lists that capture all the integers I need to specify as part of the nested SELECT statements. However, I am not sure how to put score_id into a SQL list while also preserving the order of layover_state in a separate list.
Question
Is there a way to replace the above lines of code with one line that uses 2 SQL lists for the numbers?
Additional Information:
I am using MySQL with MySQL Workbench.

SQL - Update multiple records in one query

I have table - config.
Schema:
config_name | config_value
And I would like to update multiple records in one query. I try like that:
UPDATE config
SET t1.config_value = 'value'
, t2.config_value = 'value2'
WHERE t1.config_name = 'name1'
AND t2.config_name = 'name2';
but that query is wrong :(
Can you help me?
Try either multi-table update syntax
UPDATE config t1 JOIN config t2
ON t1.config_name = 'name1' AND t2.config_name = 'name2'
SET t1.config_value = 'value',
t2.config_value = 'value2';
Here is a SQLFiddle demo
or conditional update
UPDATE config
SET config_value = CASE config_name
WHEN 'name1' THEN 'value'
WHEN 'name2' THEN 'value2'
ELSE config_value
END
WHERE config_name IN('name1', 'name2');
Here is a SQLFiddle demo
You can accomplish it with INSERT as below:
INSERT INTO mytable (id, a, b, c)
VALUES (1, 'a1', 'b1', 'c1'),
(2, 'a2', 'b2', 'c2'),
(3, 'a3', 'b3', 'c3'),
(4, 'a4', 'b4', 'c4'),
(5, 'a5', 'b5', 'c5'),
(6, 'a6', 'b6', 'c6')
ON DUPLICATE KEY UPDATE id=VALUES(id),
a=VALUES(a),
b=VALUES(b),
c=VALUES(c);
This insert new values into table, but if primary key is duplicated (already inserted into table) that values you specify would be updated and same record would not be inserted second time.
in my case I have to update the records which are more than 1000, for this instead of hitting the update query each time I preferred this,
UPDATE mst_users
SET base_id = CASE user_id
WHEN 78 THEN 999
WHEN 77 THEN 88
ELSE base_id END WHERE user_id IN(78, 77)
78,77 are the user Ids and for those user id I need to update the base_id 999 and 88 respectively.This works for me.
instead of this
UPDATE staff SET salary = 1200 WHERE name = 'Bob';
UPDATE staff SET salary = 1200 WHERE name = 'Jane';
UPDATE staff SET salary = 1200 WHERE name = 'Frank';
UPDATE staff SET salary = 1200 WHERE name = 'Susan';
UPDATE staff SET salary = 1200 WHERE name = 'John';
you can use
UPDATE staff SET salary = 1200 WHERE name IN ('Bob', 'Frank', 'John');
maybe for someone it will be useful
for Postgresql 9.5 works as a charm
INSERT INTO tabelname(id, col2, col3, col4)
VALUES
(1, 1, 1, 'text for col4'),
(DEFAULT,1,4,'another text for col4')
ON CONFLICT (id) DO UPDATE SET
col2 = EXCLUDED.col2,
col3 = EXCLUDED.col3,
col4 = EXCLUDED.col4
this SQL updates existing record and inserts if new one (2 in 1)
Camille's solution worked. Turned it into a basic PHP function, which writes up the SQL statement. Hope this helps someone else.
function _bulk_sql_update_query($table, $array)
{
/*
* Example:
INSERT INTO mytable (id, a, b, c)
VALUES (1, 'a1', 'b1', 'c1'),
(2, 'a2', 'b2', 'c2'),
(3, 'a3', 'b3', 'c3'),
(4, 'a4', 'b4', 'c4'),
(5, 'a5', 'b5', 'c5'),
(6, 'a6', 'b6', 'c6')
ON DUPLICATE KEY UPDATE id=VALUES(id),
a=VALUES(a),
b=VALUES(b),
c=VALUES(c);
*/
$sql = "";
$columns = array_keys($array[0]);
$columns_as_string = implode(', ', $columns);
$sql .= "
INSERT INTO $table
(" . $columns_as_string . ")
VALUES ";
$len = count($array);
foreach ($array as $index => $values) {
$sql .= '("';
$sql .= implode('", "', $array[$index]) . "\"";
$sql .= ')';
$sql .= ($index == $len - 1) ? "" : ", \n";
}
$sql .= "\nON DUPLICATE KEY UPDATE \n";
$len = count($columns);
foreach ($columns as $index => $column) {
$sql .= "$column=VALUES($column)";
$sql .= ($index == $len - 1) ? "" : ", \n";
}
$sql .= ";";
return $sql;
}
Execute the code below to update n number of rows, where Parent ID is the id you want to get the data from and Child ids are the ids u need to be updated so it's just u need to add the parent id and child ids to update all the rows u need using a small script.
UPDATE [Table]
SET column1 = (SELECT column1 FROM Table WHERE IDColumn = [PArent ID]),
column2 = (SELECT column2 FROM Table WHERE IDColumn = [PArent ID]),
column3 = (SELECT column3 FROM Table WHERE IDColumn = [PArent ID]),
column4 = (SELECT column4 FROM Table WHERE IDColumn = [PArent ID]),
WHERE IDColumn IN ([List of child Ids])
Execute the below code if you want to update all record in all columns:
update config set column1='value',column2='value'...columnN='value';
and if you want to update all columns of a particular row then execute below code:
update config set column1='value',column2='value'...columnN='value' where column1='value'
Assuming you have the list of values to update in an Excel spreadsheet with config_value in column A1 and config_name in B1 you can easily write up the query there using an Excel formula like
=CONCAT("UPDATE config SET config_value = ","'",A1,"'", " WHERE config_name = ","'",B1,"'")
INSERT INTO tablename
(name, salary)
VALUES
('Bob', 1125),
('Jane', 1200),
('Frank', 1100),
('Susan', 1175),
('John', 1150)
ON DUPLICATE KEY UPDATE salary = VALUES(salary);
UPDATE 2021 / MySql v8.0.20 and later
The most upvoted answer advises to use the VALUES function which is now DEPRECATED for the ON DUPLICATE KEY UPDATE syntax. With v8.0.20 you get a deprecation warning with the VALUES function:
INSERT INTO chart (id, flag)
VALUES (1, 'FLAG_1'),(2, 'FLAG_2')
ON DUPLICATE KEY UPDATE id = VALUES(id), flag = VALUES(flag);
[HY000][1287] 'VALUES function' is deprecated and will be removed in a future release. Please use an alias (INSERT INTO ... VALUES (...) AS alias) and replace VALUES(col) in the ON DUPLICATE KEY UPDATE clause with alias.col instead
Use the new alias syntax instead:
official MySQL worklog
Docs
INSERT INTO chart (id, flag)
VALUES (1, 'FLAG_1'),(2, 'FLAG_2') AS aliased
ON DUPLICATE KEY UPDATE flag=aliased.flag;
just make a transaction statement, with multiple update statement and commit. In error case, you can just rollback modification handle by starting transaction.
START TRANSACTION;
/*Multiple update statement*/
COMMIT;
(This syntax is for MySQL, for PostgreSQL, replace 'START TRANSACTION' by 'BEGIN')
Try either multi-table update syntax
Try it copy and SQL query:
CREATE TABLE #temp (id int, name varchar(50))
CREATE TABLE #temp2 (id int, name varchar(50))
INSERT INTO #temp (id, name)
VALUES (1,'abc'), (2,'xyz'), (3,'mno'), (4,'abc')
INSERT INTO #temp2 (id, name)
VALUES (2,'def'), (1,'mno1')
SELECT * FROM #temp
SELECT * FROM #temp2
UPDATE t
SET name = CASE WHEN t.id = t1.id THEN t1.name ELSE t.name END
FROM #temp t
INNER JOIN #temp2 t1 on t.id = t1.id
select * from #temp
select * from #temp2
drop table #temp
drop table #temp2
UPDATE table name SET field name = 'value' WHERE table name.primary key
If you need to update several rows at a time, the alternative is prepared statement:
database complies a query pattern you provide the first time, keep the compiled result for current connection (depends on implementation).
then you updates all the rows, by sending shortened label of the prepared function with different parameters in SQL syntax, instead of sending entire UPDATE statement several times for several updates
the database parse the shortened label of the prepared function , which is linked to the pre-compiled result, then perform the updates.
next time when you perform row updates, the database may still use the pre-compiled result and quickly complete the operations (so the first step above can be omitted since it may take time to compile).
Here is PostgreSQL example of prepare statement, many of SQL databases (e.g. MariaDB,MySQL, Oracle) also support it.

Resue from Mysql unique index duplicate entry

I am trying to fix a double request problem: when browser spawn two or more identical requests to server and they got processed on different app servers.
In this case one of two requests hits:
Mysql::Error: Duplicate entry '...'
for key 'index_purchases_on_site_id_and_order_number_and_email
There is rescue code afterwards that selects existing record instead that uses the same parameters as insert request:
select * from purchases where site_id = ? and order_number = ? and email = ?
But it doesn't find anything in database.
A fragment from DB query log:
SQL (1.3ms) BEGIN
......
Purchase Create (0.0ms) Mysql::Error: Duplicate entry '1887-100264587-9z1CIIDsH2a21+AEEH2OR9LsndO3oIS4D4Am1U5XJ04=
' for key 'index_purchases_on_site_id_and_order_number_and_email': INSERT INTO `purchases` (`order_date`, `referrer`, `created_at`, `updated_at`, `encrypted_email`, `visitor_id`, `order_number`, `coupon_code`, `subtotal`, `customer_id`, `site_id`, `ip_address`) VALUES('2013-01-14 20:57:47', 'https://www.bonobos.com/b/checkout', '2013-01-14 20:57:47', '2013-01-14 20:57:47', '9z1CIIDsH2a21+AEEH2OR9LsndO3oIS4D4Am1U5XJ04=\n', 15813843, '100264587', '', 218.0, NULL, 1887, '12.106.186.6')
Purchase Load (0.5ms) SELECT * FROM `purchases` WHERE (order_number = '100264587' AND site_id = 1887 AND encrypted_email = '9z1CIIDsH2a21+AEEH2OR9LsndO3oIS4D4Am1U5XJ04=\n') LIMIT 1
SQL (18.0ms) ROLLBACK
How is this possible?