Related
I came across a method of fast bulk update in mySQL recently:
The slow way:
UPDATE `example` SET `name` = 'Mary', `age` = '20' WHERE id = 2;
UPDATE `example` SET `name` = 'Nancy', `age` = '30' WHERE id = 4;
UPDATE `example` SET `name` = 'Oliver', `age` = '40' WHERE id = 5;
The fast way:
UPDATE `example` SET
`name` = ELT(FIELD(id,2,4,5),'Mary','Nancy','Oliver') ,
`age` = ELT(FIELD(id,2,4,5),'20','30','40')
WHERE id IN (2,4,5);
According to my google searching, the fast way is 50 times faster at large scale than the slow way, so I am wondering whether Laravel support this type of bulk update without the need to execute raw sql generated by myself.
Also if anyone is familiar with mysql and can tell me is this really that much faster or not, and if there is any scenario this method is actually worse I will be very graceful.
Edit(as requested in comment):
My current Laravel code would work like this for update:
foreach ($modelList as $model) {
$model->setConnection($connection);
$model->save();
}
What I would like for bulk update is
DB::connection($connection)->table($table)->bulkUpdate($models);
As I know currently Laravel already supports bulk insert as follows:
foreach ($models as $model) {
$attributes = $model->getAttributes();
$params[] = $attributes;
}
DB::connection($connection)->table($table)->insert($params);
which will generate one insert statement with multiple records instead of multiple insert statements.
If there's no solution, you can still do it raw with prepared statements:
DB::update("
UPDATE `example` SET
`name` = ELT(FIELD(id,2,4,5),?,?,?) ,
`age` = ELT(FIELD(id,2,4,5),?,?,?)
WHERE id IN (?,?,?)
", [
'Mary', 'Nancy', 'Oliver',
'20', '30', '40',
2, 4, 5
]);
Technically, this should also work. Haven't checked it yet:
$sql = DB::table('users')->update([
['name' => DB::raw("ELT(FIELD(id,2,4,5),'Mary','Nancy','Oliver')")],
['age' => DB::raw("ELT(FIELD(id,2,4,5),'20','30','40')")]
])
->whereIn('id', [ 2, 4, 5 ]);
Another approach:
INSERT INTO t
(name, age, id)
VALUES
('Mary', 20, 2),
('Nancy', 30, 4),
('Oliver', 40, 5)
ON DUPLICATE KEY UPDATE name = VALUES(name);
Assuming the ids are already in the table and it has PRIMARY KEY(id), there would be any INSERTs, only UPDATEs -- which is what you want.
The name = VALUES(name) is a dummy no-op. It needs to change to name = NEW.name in MySQL 8.0.
From Comment...
If you must not INSERT "new" rows, then consider this:
INSERT INTO tmp
(name, age, id)
VALUES
('Mary', 20, 2),
('Nancy', 30, 4),
('Oliver', 40, 5);
UPDATE t
JOIN tmp USING(name);
(I am not sure of the syntax; play around with the "multi-table update".)
This second 'solution' requires creating the tmp table and/or having it already available (plus possibly a TRUNCATE).
UPDATE student set roll='12',fee=fee-10 where roll='15'
Use this example for your code. This is easier and fastest way to support any platform for mysql.
select * from memory_games_game; gives me following table:
select * from memory_games_game_state;gives me following table:
I have a stored proc as bellow:
delimiter //
CREATE PROCEDURE get_game_by_user_id(
p_user_id int, p_game_id int
)
BEGIN
insert into memory_games_game_state(user_id, game_id, level, progress)
SELECT 24 as user_id,
game.game_type as game_id,
1 as level,
0 as progress
FROM memory_games_game game
left outer join memory_games_game_state gameState on
game.game_type=gameState.game_id and
gameState.user_id=24
where game.level=1 and gameState.user_id is null;
if p_game_id = -1 then
SELECT gameState.level, game_type, `current_date`
FROM memory_games_game game join memory_games_game_state gameState on
game.game_type=gameState.game_id and
gameState.user_id=24 and
game.level=gameState.level;
else
SELECT gameState.level, game_type, `current_date`
FROM memory_games_game game join memory_games_game_state gameState on
game.game_type=gameState.game_id and
gameState.user_id=p_user_id and
game.level=gameState.level
WHERE game_type=12;
end if;
END
//
The first insert inserts the records into memory_games_game_statetable.
This insert is successful for game_id from 8 to 11 however, it fails for 12 with following error:
I am able to insert records in table memory_games_gamefor game_type 12 which is nothing but game_id in the other table i.e. memory_games_game_state
What's going wrong?
UPDATE:
My django models:
class Game(models.Model):
#Field for storing game type
GAME_TYPE_CHOICES = (
(8, 'Simon Game'),
(9, 'Pinpoint reaction'),
(10, 'Loslassen'),
(11, 'Word pair'),
(12, 'Wortschatz')
)
game_type = models.IntegerField(choices=GAME_TYPE_CHOICES)
level = models.IntegerField(default='1')
#This helps to print in admin interface
def __str__(self):
return u"%s level %s" % (self.get_game_type_display(), self.level)
class Game_state(models.Model):
game = models.ForeignKey(Game, blank=True, null=True)
user = models.ForeignKey(User, blank=True, null=True)
level = models.IntegerField(default='1')
progress = models.IntegerField(default='0')
current_date = models.DateField(auto_now=True)
class Game_state_ui_model(models.Model):
GAME_TYPE_CHOICES = (
(8, 'Simon Game'),
(9, 'Pinpoint reaction'),
(10, 'Loslassen'),
(11, 'Word pair'),
(12, 'Wortschatz')
)
game_type = models.IntegerField(choices=GAME_TYPE_CHOICES)
level = models.IntegerField()
user_id = models.IntegerField(default='0')
current_date = models.DateField(auto_now=True)
# static method to fetch games for a paricular user
#staticmethod
def fetch_games(user_id, game_id):
print("Fetching games in model")
# create a cursor
cur = connection.cursor()
# execute the stored procedure passing in
# search_string as a parameter
cur.callproc('get_game_by_user_id', [user_id, game_id,])
# grab the results
results = cur.fetchall()
cur.close()
Game_state_list=[]
for row in results:
print("Get game", row)
Gs = Game_state_ui_model()
Gs.level=row[0]
Gs.game_type=row[1]
Gs.current_date=row[2]
Game_state_list.append(Gs)
return Game_state_list
As the error states, game_id references memory_games_game.id; NOT memory_games_game.game_type.
The thinking that "game_type 12 which is nothing but game_id in the other table i.e. memory_games_game_state" is incorrect.
You need a row in memory_games_game with id = 12.
I dropped all the tables and did migration again, which somehow solved the problem. I didn't change anything else.
How to update mysql data in bulk ?
How to define something like this :
UPDATE `table`
WHERE `column1` = somevalues
SET `column2` = othervalues
with somevalues like :
VALUES
('160009'),
('160010'),
('160011');
and othervalues :
VALUES
('val1'),
('val2'),
('val3');
maybe it's impossible with mysql ?
a php script ?
The easiest solution in your case is to use ON DUPLICATE KEY UPDATE construction. It works really fast, and does the job in easy way.
INSERT into `table` (id, fruit)
VALUES (1, 'apple'), (2, 'orange'), (3, 'peach')
ON DUPLICATE KEY UPDATE fruit = VALUES(fruit);
or to use CASE construction
UPDATE table
SET column2 = (CASE column1 WHEN 1 THEN 'val1'
WHEN 2 THEN 'val2'
WHEN 3 THEN 'val3'
END)
WHERE column1 IN(1, 2 ,3);
If the "bulk" data you have is dynamic and is coming from PHP (you did tag it, after all), then the query would look something like this:
INSERT INTO `foo` (id, bar)
VALUES
(1, 'pineapple'),
(2, 'asian pear'),
(5, 'peach')
ON DUPLICATE KEY UPDATE bar = VALUES(bar);
and the PHP to generate this from an existing array (assuming the array is of a format like:
$array = (
somevalues_key => othervalues_value
);
) would look something like this (by no means the best (doesn't address escaping or sanitizing the values, for instance), just an quick example):
$pairs = array();
foreach ($array as $key => $value) {
$pairs[] = "($key, '$value')";
}
$query = "INSERT INTO `foo` (id, bar) VALUES " . implode(', ', $pairs) . " ON DUPLICATE KEY UPDATE bar = VALUES(bar)";
You could try an UPDATE with JOIN as below:
UPDATE table
INNER JOIN (
SELECT 1 column1, 2 column2, 10 new_v1, 20 new_v2, 30 new_v3
UNION ALL SELECT 4 column1, 5 column2, 40 new_v1, 50 new_v2, 60 new_v3
) updates
ON table.column1 = updates.column1
AND table.column2 = updates.column2
SET
table.column1 = updates.new_v1,
table.column2 = updates.new_v2,
table.column3 = updates.new_v3;
As long as you can craft the inner SELECT statements from the updates subquery you would get the benefit of running all these updates in a single statement (which should give you some performance boost on InnoDB depending on your table size).
If you are using a drag & drop tableView or collectionView to sort datas in your app, like allowing users to arrange their photos by drag and drop functionality, send a comma seperated list of ordered ids to the backend after user edits finish.
In your backend, explode ids to the an array like
$new_ranks = array();
$supplied_orders = explode(",", $_POST["supplied_new_order"]); //52,11,6,54,2 etc
$start_order = 99999;
foreach ($supplied_orders as $supplied_row_id) {
//your all validations... make sure supplied_row_id belongs to that user or not etc..
$new_ranks[intval($supplied_row_id)] = $start_order--;
}
now, you can update all new ranks like #Farside recommendation 2.
if (count($new_ranks) > 0) {
$case_sqls = array();
foreach ($new_ranks as $id => $rank) {
$case_sqls[] = "WHEN ".intval($id)." THEN ".intval($rank)."";
}
$case_sql = implode(" ", $case_sqls);
$this->db->query("
UPDATE
service_user_medias
SET
rank = (CASE id ".$case_sql." END)
WHERE
id IN(".implode(",", array_keys($new_ranks)).");
");
}
If you have data in array format then try this
and your query is like "UPDATE table WHERE column1 = ? SET column2 = ?"
then set it like below
foreach($data as $key => $value) {
$query->bind_param('ss', $key, $value);
$query->execute();
}
hope it'll work.
Reference from this.
I have table - config.
Schema:
config_name | config_value
And I would like to update multiple records in one query. I try like that:
UPDATE config
SET t1.config_value = 'value'
, t2.config_value = 'value2'
WHERE t1.config_name = 'name1'
AND t2.config_name = 'name2';
but that query is wrong :(
Can you help me?
Try either multi-table update syntax
UPDATE config t1 JOIN config t2
ON t1.config_name = 'name1' AND t2.config_name = 'name2'
SET t1.config_value = 'value',
t2.config_value = 'value2';
Here is a SQLFiddle demo
or conditional update
UPDATE config
SET config_value = CASE config_name
WHEN 'name1' THEN 'value'
WHEN 'name2' THEN 'value2'
ELSE config_value
END
WHERE config_name IN('name1', 'name2');
Here is a SQLFiddle demo
You can accomplish it with INSERT as below:
INSERT INTO mytable (id, a, b, c)
VALUES (1, 'a1', 'b1', 'c1'),
(2, 'a2', 'b2', 'c2'),
(3, 'a3', 'b3', 'c3'),
(4, 'a4', 'b4', 'c4'),
(5, 'a5', 'b5', 'c5'),
(6, 'a6', 'b6', 'c6')
ON DUPLICATE KEY UPDATE id=VALUES(id),
a=VALUES(a),
b=VALUES(b),
c=VALUES(c);
This insert new values into table, but if primary key is duplicated (already inserted into table) that values you specify would be updated and same record would not be inserted second time.
in my case I have to update the records which are more than 1000, for this instead of hitting the update query each time I preferred this,
UPDATE mst_users
SET base_id = CASE user_id
WHEN 78 THEN 999
WHEN 77 THEN 88
ELSE base_id END WHERE user_id IN(78, 77)
78,77 are the user Ids and for those user id I need to update the base_id 999 and 88 respectively.This works for me.
instead of this
UPDATE staff SET salary = 1200 WHERE name = 'Bob';
UPDATE staff SET salary = 1200 WHERE name = 'Jane';
UPDATE staff SET salary = 1200 WHERE name = 'Frank';
UPDATE staff SET salary = 1200 WHERE name = 'Susan';
UPDATE staff SET salary = 1200 WHERE name = 'John';
you can use
UPDATE staff SET salary = 1200 WHERE name IN ('Bob', 'Frank', 'John');
maybe for someone it will be useful
for Postgresql 9.5 works as a charm
INSERT INTO tabelname(id, col2, col3, col4)
VALUES
(1, 1, 1, 'text for col4'),
(DEFAULT,1,4,'another text for col4')
ON CONFLICT (id) DO UPDATE SET
col2 = EXCLUDED.col2,
col3 = EXCLUDED.col3,
col4 = EXCLUDED.col4
this SQL updates existing record and inserts if new one (2 in 1)
Camille's solution worked. Turned it into a basic PHP function, which writes up the SQL statement. Hope this helps someone else.
function _bulk_sql_update_query($table, $array)
{
/*
* Example:
INSERT INTO mytable (id, a, b, c)
VALUES (1, 'a1', 'b1', 'c1'),
(2, 'a2', 'b2', 'c2'),
(3, 'a3', 'b3', 'c3'),
(4, 'a4', 'b4', 'c4'),
(5, 'a5', 'b5', 'c5'),
(6, 'a6', 'b6', 'c6')
ON DUPLICATE KEY UPDATE id=VALUES(id),
a=VALUES(a),
b=VALUES(b),
c=VALUES(c);
*/
$sql = "";
$columns = array_keys($array[0]);
$columns_as_string = implode(', ', $columns);
$sql .= "
INSERT INTO $table
(" . $columns_as_string . ")
VALUES ";
$len = count($array);
foreach ($array as $index => $values) {
$sql .= '("';
$sql .= implode('", "', $array[$index]) . "\"";
$sql .= ')';
$sql .= ($index == $len - 1) ? "" : ", \n";
}
$sql .= "\nON DUPLICATE KEY UPDATE \n";
$len = count($columns);
foreach ($columns as $index => $column) {
$sql .= "$column=VALUES($column)";
$sql .= ($index == $len - 1) ? "" : ", \n";
}
$sql .= ";";
return $sql;
}
Execute the code below to update n number of rows, where Parent ID is the id you want to get the data from and Child ids are the ids u need to be updated so it's just u need to add the parent id and child ids to update all the rows u need using a small script.
UPDATE [Table]
SET column1 = (SELECT column1 FROM Table WHERE IDColumn = [PArent ID]),
column2 = (SELECT column2 FROM Table WHERE IDColumn = [PArent ID]),
column3 = (SELECT column3 FROM Table WHERE IDColumn = [PArent ID]),
column4 = (SELECT column4 FROM Table WHERE IDColumn = [PArent ID]),
WHERE IDColumn IN ([List of child Ids])
Execute the below code if you want to update all record in all columns:
update config set column1='value',column2='value'...columnN='value';
and if you want to update all columns of a particular row then execute below code:
update config set column1='value',column2='value'...columnN='value' where column1='value'
Assuming you have the list of values to update in an Excel spreadsheet with config_value in column A1 and config_name in B1 you can easily write up the query there using an Excel formula like
=CONCAT("UPDATE config SET config_value = ","'",A1,"'", " WHERE config_name = ","'",B1,"'")
INSERT INTO tablename
(name, salary)
VALUES
('Bob', 1125),
('Jane', 1200),
('Frank', 1100),
('Susan', 1175),
('John', 1150)
ON DUPLICATE KEY UPDATE salary = VALUES(salary);
UPDATE 2021 / MySql v8.0.20 and later
The most upvoted answer advises to use the VALUES function which is now DEPRECATED for the ON DUPLICATE KEY UPDATE syntax. With v8.0.20 you get a deprecation warning with the VALUES function:
INSERT INTO chart (id, flag)
VALUES (1, 'FLAG_1'),(2, 'FLAG_2')
ON DUPLICATE KEY UPDATE id = VALUES(id), flag = VALUES(flag);
[HY000][1287] 'VALUES function' is deprecated and will be removed in a future release. Please use an alias (INSERT INTO ... VALUES (...) AS alias) and replace VALUES(col) in the ON DUPLICATE KEY UPDATE clause with alias.col instead
Use the new alias syntax instead:
official MySQL worklog
Docs
INSERT INTO chart (id, flag)
VALUES (1, 'FLAG_1'),(2, 'FLAG_2') AS aliased
ON DUPLICATE KEY UPDATE flag=aliased.flag;
just make a transaction statement, with multiple update statement and commit. In error case, you can just rollback modification handle by starting transaction.
START TRANSACTION;
/*Multiple update statement*/
COMMIT;
(This syntax is for MySQL, for PostgreSQL, replace 'START TRANSACTION' by 'BEGIN')
Try either multi-table update syntax
Try it copy and SQL query:
CREATE TABLE #temp (id int, name varchar(50))
CREATE TABLE #temp2 (id int, name varchar(50))
INSERT INTO #temp (id, name)
VALUES (1,'abc'), (2,'xyz'), (3,'mno'), (4,'abc')
INSERT INTO #temp2 (id, name)
VALUES (2,'def'), (1,'mno1')
SELECT * FROM #temp
SELECT * FROM #temp2
UPDATE t
SET name = CASE WHEN t.id = t1.id THEN t1.name ELSE t.name END
FROM #temp t
INNER JOIN #temp2 t1 on t.id = t1.id
select * from #temp
select * from #temp2
drop table #temp
drop table #temp2
UPDATE table name SET field name = 'value' WHERE table name.primary key
If you need to update several rows at a time, the alternative is prepared statement:
database complies a query pattern you provide the first time, keep the compiled result for current connection (depends on implementation).
then you updates all the rows, by sending shortened label of the prepared function with different parameters in SQL syntax, instead of sending entire UPDATE statement several times for several updates
the database parse the shortened label of the prepared function , which is linked to the pre-compiled result, then perform the updates.
next time when you perform row updates, the database may still use the pre-compiled result and quickly complete the operations (so the first step above can be omitted since it may take time to compile).
Here is PostgreSQL example of prepare statement, many of SQL databases (e.g. MariaDB,MySQL, Oracle) also support it.
It is pretty straight forward, but I could not find an answer. If the combination does exist, then UPDATE, else INSERT.
IF EXISTS(SELECT customer_id FROM payment_conditions WHERE customer_id = 2 AND shipping_company_id = 1)
THEN
UPDATE payment_conditions SET payment_condition = "pay in full" WHERE customer_id = 2 AND shipping_company_id = 1;
ELSE
INSERT INTO payment_conditions customer_id, shipping_company_id, payment_condition, active VALUES (2,1,"some value",1);
END IF
You should set customer_id and shipping_company_id to be foreign keys, so you can use
INSERT INTO payment_conditions
customer_id, shipping_company_id, payment_condition, active
VALUES (2, 1, "some value", 1)
ON DUPLICATE KEY UPDATE payment_condition = "pay in full"
if you have a unique key on (shipping_company_id, customer_id) you can use:
INSERT INTO payment_conditions
SET customer_id = 2,
shipping_company_id = 1,
payment_condition = 'some value',
active = 1
ON DUPLICATE KEY UPDATE payment_condition = 'pay in full'
INSERT INTO payment_conditions (customer_id, shipping_company_id, payment_condition, active)
VALUES (2,1,"some value",1)
ON DUPLICATE KEY UPDATE payment_condition = "pay in full"
Or issue two statements (N.B. with UPDATE first), and do away with the conditional logic. The UPDATE will have no effect if the row does not exist. This should work if your key happens to be different to the customer_id + shipping_company.
UPDATE payment_conditions SET payment_condition = "pay in full"
WHERE customer_id = 2 AND shipping_company_id = 1;
then
INSERT INTO payment_conditions
(customer_id
, shipping_company_id
, payment_condition
, active)
VALUES (2,1,"some value",1)
where not exists
(select * FROM payment_conditions
WHERE customer_id = 2 AND shipping_company_id = 1);
Use the merge statement in SQL Server - it looks like these answers imply MySQL syntax. You don't state the database flavor, but Merge is the recommended way to do this on SQL Server.