I have 1-many number of records that need to be entered into a table. What is the best way to do this in a query? Should I just make a loop and insert one record per iteration? Or is there a better way?
From the MySQL manual
INSERT statements that use VALUES
syntax can insert multiple rows. To do
this, include multiple lists of column
values, each enclosed within
parentheses and separated by commas.
Example:
INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9);
Most of the time, you are not working in a MySQL client and you should batch inserts together using the appropriate API.
E.g. in JDBC:
connection con.setAutoCommit(false);
PreparedStatement prepStmt = con.prepareStatement("UPDATE DEPT SET MGRNO=? WHERE DEPTNO=?");
prepStmt.setString(1,mgrnum1);
prepStmt.setString(2,deptnum1);
prepStmt.addBatch();
prepStmt.setString(1,mgrnum2);
prepStmt.setString(2,deptnum2);
prepStmt.addBatch();
int [] numUpdates=prepStmt.executeBatch();
http://publib.boulder.ibm.com/infocenter/db2luw/v8/index.jsp?topic=/com.ibm.db2.udb.doc/ad/tjvbtupd.htm
Load data infile query is much better option but some servers like godaddy restrict this option on shared hosting so , only two options left then one is insert record on every iteration or batch insert , but batch insert has its limitaion of characters if your query exceeds this number of characters set in mysql then your query will crash , So I suggest insert data in chunks withs batch insert , this will minimize number of connections established with database.best of luck guys
Insert into table(col1,col2) select col1,col2 from table_2;
Please refer to MySQL documentation on INSERT Statement
mysql allows you to insert multiple rows at once INSERT manual
INSERT INTO test_1 VALUES(24, 'B', '1990-12-07'), (25, 'C', '1990-12-08');
Related
I have large sql insert query with multiple inserts seperated by commas something like below.
INSERT INTO `table1` VALUES (396,3,90,'collar','admin','2015-05-27 10:10:52','2015-05-27 10:10:52',0),(397,3,90,'across shoulder','admin','2015-05-27 10:10:52','2015-05-27 10:10:52',0);
How can I split this large sql insert into multiple inserts?
Some thing like this:
INSERT INTO `table1` VALUES(396,3,90,'collar','admin','2015-05-27 10:10:52','2015-05-27 10:10:52',0);
INSERT INTO `table1` VALUES (397,3,90,'across shoulder','admin','2015-05-27 10:10:52','2015-05-27 10:10:52',0);
There are two option are avaialble.(This are Only alternatives)
1) Create CSV file of Inputs and import it into table by using Workbench.
or
2) I will suggest you alternative to this use Mysql WorkBench for insert values.
I've got a table with two columns that links a row id from one table to a row id of another, basically I need to be able to insert multiple rows on this table where one column is a fixed value for the insert and the other changes. The insert is done in php using prepared statements and there is an unknown number of rows to be inserted (I've solved this part using call_user_func_array).
Here is an example of what I need to do:
example_table (column_A, column_B)
The insert:
INSERT INTO example_table (column_A, column_B) VALUES(a, b), VALUES(a, c), VALUES(a, d);
Translated to prepared statement:
INSERT INTO example_table (column_A, column_B) VALUES(?, ?), VALUES(?, ?), VALUES(?, ?);
values on bind_param:
('ssssss', 'a', 'b', 'a', 'c', 'a', 'd')
As you can see the repetition of 'a'. is there a way to store 'a' and default it to column_A for the current insert only?
example of what i would like to do on bind_param:
('ssss', 'a', 'b', 'c', 'd')
Where 'a' above is used for column_A on all rows inserted (3 rows in the example). Any help would be appreciated
No, it's not possible to "alias" a value multiple times. Each placeholder is unique and must be bound to exactly one value. (It is not even possible to do this reliably with named parameters in PDO.)
If automatically building SQL dynamically (with placeholders) and associated data array, then there is no issue with such "duplication" as it's already handled by the generator.
If transactions are correctly used - which they should be anyway! - then simply calling INSERT in a loop, once for each pair, will likely have similar performance. This avoids needing to build the dynamic SQL (with placeholders) itself, but can increase latency as each command needs to round-trip to the SQL server.
If finding a bunch of such repetition, it might be time to use a Data Access Layer to hind such details; write it, test it, move onto something interesting.
In response to the comment about using SQL variables:
While the following will not improve performance, excepting possibly for ridiculously large values of #v, and makes the code more difficult to understand - it ought to be possible to use user-defined variables in a single insert statement. For instance, consider:
INSERT INTO t VALUES (#v := ?, ?), (#v, ?), (#v, ?)
This is "valid" in MySQL (and is also MySQL-specific), where the placeholders are valid expressions; if it works will depend on how/if such a binding is allowed in a prepared statement.
You may try making a temporary table:
$query='CREATE TEMPORARY TABLE tempTable(column_B VARCHAR(50));';
$mysqli->query($query);
then insert unique values of column_B into this table
and finally
$query="INSERT INTO example_table (column_A, column_B) select $a , column_B from tempTable";
$mysqli->query($query);
I have rails application and its using mysql.
I have a piles table with two columns that I care for.
The columns are name_he_il and name_en_us
I don't have a problem doing these
select name_he_il from piles;
select name_en_us from piles;
I need to insert data into the name_he_il column into the piles table where name_en_us = "a specific value"
I tried something like this
insert into piles (name_he_il) values 'לא מאפיין כלל' where name_en_us = "Extremely Uncharacteristic";
I am getting syntax error.
I was googling and I figured the sql should be
insert into table (column 1) values (blah) where conditions;
but its not working.
Basically that hebrew text means extremely uncharacteristic.
You want to use UPDATE ... WHERE
INSERT is for creating new records only.
Do UPDATE and not INSERT:
UPDATE piles SET name_he_il = 'לא מאפיין כלל' WHERE name_en_us = "Extremely Uncharacteristic";
Insert query is to insert new rom into table. If you already have row with value of "name_he_il" column then you need to use update.
UPDATE piles SET name_he_il='new value' WHERE name_he_es = 'something'
In my table I insert around 20,000 rows on each load. Right now I am doing it one-by-one. From mysql website I came to know inserting multiple rows with single insert query is faster.
Can I insert all 20000 in single query?
What will happen if there are errors within this 20000 rows? how will mysql handle that?
If you are inserting the rows from some other table then you can use the INSERT ... SELECT pattern to insert the rows.
However if you are inserting the values using INSERT ... VALUES pattern then you have the limit of max_allowed_packet.
Also from the docs:-
To optimize insert speed, combine many small operations into a single
large operation. Ideally, you make a single connection, send the data
for many new rows at once, and delay all index updates and consistency
checking until the very end.
Example:-
INSERT INTO `table1` (`column1`, `column2`) VALUES ("d1", "d2"),
("d1", "d2"),
("d1", "d2"),
("d1", "d2"),
("d1", "d2");
What will happen if there are errors within this 20000 rows?
If there are errors while inserting the records then the operation will be aborted.
http://dev.mysql.com/doc/refman/5.5/en/insert.html
INSERT statements that use VALUES syntax can insert multiple rows. To
do this, include multiple lists of column values, each enclosed within
parentheses and separated by commas.
Example:
INSERT INTO tbl_name (a,b,c) VALUES(1,2,3),(4,5,6),(7,8,9);
You can use code to generate the insert VALUES section based on your data source.
Errors: if there are errors in the INSERT statement (including in any of the rows) the operation will be aborted.
Generating the query - this will be based on your data source, for example, if you are getting data from an associative array in PHP, you'll do something like this:
$sql = "INSERT INTO tbl_name (a, b, c) VALUES ";
foreach($dataset as $row)
{
$sql .= "(" + $row['a'] + ", " + $row['a'] + ", " + $row['a'] + ")";
// OR
$sql .= "($row[a], $row[b], $row[c])";
}
Some more resources:
Optimize MySQL Queries – Fast Inserts With Multiple Rows
The fastest way to insert 100K records
batch insert with SQL: insert into table (col...coln) values (col... coln),(col...coln)... but the SQL length is limited by 1M default, you can change max_allowed_packet parameter to support more bigger single insert
If I insert multiple records with a loop that executes a single record insert, the last insert id returned is, as expected, the last one. But if I do a multiple records insert statement:
INSERT INTO people (name,age)
VALUES ('William',25), ('Bart',15), ('Mary',12);
Let's say the three above are the first records inserted in the table. After the insert statement I expected the last insert id to return 3, but it returned 1. The first insert id for the statement in question.
So can someone please confirm if this is the normal behavior of LAST_INSERT_ID() in the context of multiple records INSERT statements. So I can base my code on it.
Yes. This behavior of last_insert_id() is documented in the MySQL docs:
Important
If you insert multiple rows using a single INSERT statement, LAST_INSERT_ID() returns the value generated for the first inserted row only. The reason for this is to make it possible to reproduce easily the same INSERT statement against some other server.
This behavior is mentioned on the man page for MySQL. It's in the comments but is not challenged, so I'm guessing it's the expected behavior.
I think it's possible if your table has unique autoincrement column (ID) and you don't require them to be returned by mysql itself. I would cost you 3 more DB requests and some processing. It would require these steps:
Get "Before MAX(ID)" right before your insert:
SELECT MAX(id) AS before_max_id FROM table_name`
Make multiple INSERT ... VALUES () query with your data and keep them:
INSERT INTO table_name
(col1, col2)
VALUES
("value1-1" , "value1-2"),
("value2-1" , "value2-2"),
("value3-1" , "value3-2"),
ON DUPLICATE KEY UPDATE
Get "After MAX(ID)" right after your insert:
SELECT MAX(id) AS after_max_id FROM table_name`
Get records with IDs between "Before MAX(ID)" and "After MAX(ID)" including:
SELECT * FROM table_name WHERE id>$before_max_id AND id<=$after_max_id`
Do a check of retrieved data with data you inserted to match them and remove any records that were not inserted by you. The remaining records have your IDs:
foreach ($after_collection as $after_item) {
foreach ($input_collection as $input_item) {
if ( $after_item->compare_content($input_item) ) {
$intersection_array[] = $after_item;
}
}
}
This is just how a common person would solve it in a real world, with parts of code. Thanks to autoincrement it should get smallest possible amount of records to check against, so they will not take lot of processing. This is not the final "copy & paste" code - eg. you have to create your own function compare_content() according you your needs.