If you will consider the following table
table_A (id (PK), value1, value2)
If I want to insert a set of data, for example: (1,5), (1,3), (3,5)
I could perform a query such as :
INSERT INTO table_A (value1, value2) VALUES (1,5), (1,3), (3,5)
which would work. However, I am told prepared statements would be better. Looking into prepared statements it seems I would have to do something like this
$stmt = $dbh->prepare("INSERT INTO table_A (value1, value2) VALUES (?, ?)");
$stmt->bindParam(1, $value1);
$stmt->bindParam(2, $value2);
//for each set of values
$value1 = 1;
$value2 = 5;
$stmt->execute();
my question is, how can a prepared statement be better (performance wise) than the first method? One is a single query, the other involves several executions of the same query. Does the first query get compiled into three separate queries or something?
The prepared statement by itself is not going to be faster when you insert only once. However, if you need to run the same inserts multiple times, you will save on the time it takes to parse the query and prepare the query plan. The prepared statement insert will be parsed once, the plan for it will be cached, and then reused for all subsequent insertions. The statement with multiple embedded values, on the other hand, will need to be re-processed every time you run a new one, slowing the process down.
On the other hand, network roundtrips are slow as well. It may be slower to do an extra roundtrip than to parse and prepare a query plan, so you should profile before making a decision one way or the other.
Related
I have 1-many number of records that need to be entered into a table. What is the best way to do this in a query? Should I just make a loop and insert one record per iteration? Or is there a better way?
From the MySQL manual
INSERT statements that use VALUES
syntax can insert multiple rows. To do
this, include multiple lists of column
values, each enclosed within
parentheses and separated by commas.
Example:
INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9);
Most of the time, you are not working in a MySQL client and you should batch inserts together using the appropriate API.
E.g. in JDBC:
connection con.setAutoCommit(false);
PreparedStatement prepStmt = con.prepareStatement("UPDATE DEPT SET MGRNO=? WHERE DEPTNO=?");
prepStmt.setString(1,mgrnum1);
prepStmt.setString(2,deptnum1);
prepStmt.addBatch();
prepStmt.setString(1,mgrnum2);
prepStmt.setString(2,deptnum2);
prepStmt.addBatch();
int [] numUpdates=prepStmt.executeBatch();
http://publib.boulder.ibm.com/infocenter/db2luw/v8/index.jsp?topic=/com.ibm.db2.udb.doc/ad/tjvbtupd.htm
Load data infile query is much better option but some servers like godaddy restrict this option on shared hosting so , only two options left then one is insert record on every iteration or batch insert , but batch insert has its limitaion of characters if your query exceeds this number of characters set in mysql then your query will crash , So I suggest insert data in chunks withs batch insert , this will minimize number of connections established with database.best of luck guys
Insert into table(col1,col2) select col1,col2 from table_2;
Please refer to MySQL documentation on INSERT Statement
mysql allows you to insert multiple rows at once INSERT manual
INSERT INTO test_1 VALUES(24, 'B', '1990-12-07'), (25, 'C', '1990-12-08');
I've got a table with two columns that links a row id from one table to a row id of another, basically I need to be able to insert multiple rows on this table where one column is a fixed value for the insert and the other changes. The insert is done in php using prepared statements and there is an unknown number of rows to be inserted (I've solved this part using call_user_func_array).
Here is an example of what I need to do:
example_table (column_A, column_B)
The insert:
INSERT INTO example_table (column_A, column_B) VALUES(a, b), VALUES(a, c), VALUES(a, d);
Translated to prepared statement:
INSERT INTO example_table (column_A, column_B) VALUES(?, ?), VALUES(?, ?), VALUES(?, ?);
values on bind_param:
('ssssss', 'a', 'b', 'a', 'c', 'a', 'd')
As you can see the repetition of 'a'. is there a way to store 'a' and default it to column_A for the current insert only?
example of what i would like to do on bind_param:
('ssss', 'a', 'b', 'c', 'd')
Where 'a' above is used for column_A on all rows inserted (3 rows in the example). Any help would be appreciated
No, it's not possible to "alias" a value multiple times. Each placeholder is unique and must be bound to exactly one value. (It is not even possible to do this reliably with named parameters in PDO.)
If automatically building SQL dynamically (with placeholders) and associated data array, then there is no issue with such "duplication" as it's already handled by the generator.
If transactions are correctly used - which they should be anyway! - then simply calling INSERT in a loop, once for each pair, will likely have similar performance. This avoids needing to build the dynamic SQL (with placeholders) itself, but can increase latency as each command needs to round-trip to the SQL server.
If finding a bunch of such repetition, it might be time to use a Data Access Layer to hind such details; write it, test it, move onto something interesting.
In response to the comment about using SQL variables:
While the following will not improve performance, excepting possibly for ridiculously large values of #v, and makes the code more difficult to understand - it ought to be possible to use user-defined variables in a single insert statement. For instance, consider:
INSERT INTO t VALUES (#v := ?, ?), (#v, ?), (#v, ?)
This is "valid" in MySQL (and is also MySQL-specific), where the placeholders are valid expressions; if it works will depend on how/if such a binding is allowed in a prepared statement.
You may try making a temporary table:
$query='CREATE TEMPORARY TABLE tempTable(column_B VARCHAR(50));';
$mysqli->query($query);
then insert unique values of column_B into this table
and finally
$query="INSERT INTO example_table (column_A, column_B) select $a , column_B from tempTable";
$mysqli->query($query);
$stmt2 = $db->prepare("INSERT INTO
usertabbrige(`tabId`,`uId`)
VALUES
((LAST_INSERT_ID()),$userId)");
anything wrong with this query? It's wrap within my first stmt, which will insert a value into uId (PK) in other table. usertabbrige table contain a field uId which is a FK.
Do not use LAST_INSERT_ID() in your query. You dont know which insert statement was last in current session. You can insert to one table, and if you use LAST_INSERT_ID() in another query, you dont actually know where LAST_INSERT_ID() came from.
As I can see you are using PDO. After you executed an insert query, save id:
$db->query("INSERT INTO ...");
$lastInsertedTabId = $db->lastInsertId;
Use it in your next prepared statement
$stmt2 = $db->prepare("INSERT INTO
usertabbrige(`tabId`,`uId`)
VALUES
($lastInsertedTabId ,$userId)");
In my table I insert around 20,000 rows on each load. Right now I am doing it one-by-one. From mysql website I came to know inserting multiple rows with single insert query is faster.
Can I insert all 20000 in single query?
What will happen if there are errors within this 20000 rows? how will mysql handle that?
If you are inserting the rows from some other table then you can use the INSERT ... SELECT pattern to insert the rows.
However if you are inserting the values using INSERT ... VALUES pattern then you have the limit of max_allowed_packet.
Also from the docs:-
To optimize insert speed, combine many small operations into a single
large operation. Ideally, you make a single connection, send the data
for many new rows at once, and delay all index updates and consistency
checking until the very end.
Example:-
INSERT INTO `table1` (`column1`, `column2`) VALUES ("d1", "d2"),
("d1", "d2"),
("d1", "d2"),
("d1", "d2"),
("d1", "d2");
What will happen if there are errors within this 20000 rows?
If there are errors while inserting the records then the operation will be aborted.
http://dev.mysql.com/doc/refman/5.5/en/insert.html
INSERT statements that use VALUES syntax can insert multiple rows. To
do this, include multiple lists of column values, each enclosed within
parentheses and separated by commas.
Example:
INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9);
You can use code to generate the insert VALUES section based on your data source.
Errors: if there are errors in the INSERT statement (including in any of the rows) the operation will be aborted.
Generating the query - this will be based on your data source, for example, if you are getting data from an associative array in PHP, you'll do something like this:
$sql = "INSERT INTO tbl_name (a, b, c) VALUES ";
foreach($dataset as $row)
{
$sql .= "(" + $row['a'] + ", " + $row['a'] + ", " + $row['a'] + ")";
// OR
$sql .= "($row[a], $row[b], $row[c])";
}
Some more resources:
Optimize MySQL Queries – Fast Inserts With Multiple Rows
The fastest way to insert 100K records
batch insert with SQL: insert into table (col...coln) values (col... coln),(col...coln)... but the SQL length is limited by 1M default, you can change max_allowed_packet parameter to support more bigger single insert
If I insert multiple records with a loop that executes a single record insert, the last insert id returned is, as expected, the last one. But if I do a multiple records insert statement:
INSERT INTO people (name,age)
VALUES ('William',25), ('Bart',15), ('Mary',12);
Let's say the three above are the first records inserted in the table. After the insert statement I expected the last insert id to return 3, but it returned 1. The first insert id for the statement in question.
So can someone please confirm if this is the normal behavior of LAST_INSERT_ID() in the context of multiple records INSERT statements. So I can base my code on it.
Yes. This behavior of last_insert_id() is documented in the MySQL docs:
Important
If you insert multiple rows using a single INSERT statement, LAST_INSERT_ID() returns the value generated for the first inserted row only. The reason for this is to make it possible to reproduce easily the same INSERT statement against some other server.
This behavior is mentioned on the man page for MySQL. It's in the comments but is not challenged, so I'm guessing it's the expected behavior.
I think it's possible if your table has unique autoincrement column (ID) and you don't require them to be returned by mysql itself. I would cost you 3 more DB requests and some processing. It would require these steps:
Get "Before MAX(ID)" right before your insert:
SELECT MAX(id) AS before_max_id FROM table_name`
Make multiple INSERT ... VALUES () query with your data and keep them:
INSERT INTO table_name
(col1, col2)
VALUES
("value1-1" , "value1-2"),
("value2-1" , "value2-2"),
("value3-1" , "value3-2"),
ON DUPLICATE KEY UPDATE
Get "After MAX(ID)" right after your insert:
SELECT MAX(id) AS after_max_id FROM table_name`
Get records with IDs between "Before MAX(ID)" and "After MAX(ID)" including:
SELECT * FROM table_name WHERE id>$before_max_id AND id<=$after_max_id`
Do a check of retrieved data with data you inserted to match them and remove any records that were not inserted by you. The remaining records have your IDs:
foreach ($after_collection as $after_item) {
foreach ($input_collection as $input_item) {
if ( $after_item->compare_content($input_item) ) {
$intersection_array[] = $after_item;
}
}
}
This is just how a common person would solve it in a real world, with parts of code. Thanks to autoincrement it should get smallest possible amount of records to check against, so they will not take lot of processing. This is not the final "copy & paste" code - eg. you have to create your own function compare_content() according you your needs.