Is there a way to dynamically specify a file name in the LOAD DATA INFILE? Can it be parameterized like for instance (syntax may be incorrect) LOAD DATA INFILE '$filename'?
A citation from MySQL documentation:
The LOAD DATA INFILE statement reads rows from a text file into a table at a very high speed. The file name must be given as a literal string.
That means that it can not be a parameter of a prepared statement. But no one forbids to make the string interpolation while the statement is just a string in your PHP code.
Unfortunately, this feature is not yet supported in MySQL and is currently listed as bug/feature request #39115 http://bugs.mysql.com/bug.php?id=39115
Or you can make a temporary copy of the file (BATCH example):
LOAD_DATA.bat
COPY %1 TempFileToLoad.csv
mysql --user=myuser --password=mypass MyDatabase < ScriptLoadMyDatabase.sql
DEL TempFileToLoad.csv
the SQL (for info) :
ScriptLoadMyDatabase.sql
load data infile 'TempFileToLoad.csv' IGNORE
into table tLoad
FIELDS TERMINATED BY ';' OPTIONALLY ENCLOSED BY '"'
lines terminated by '\r\n'
IGNORE 1 LINES
(#DateCrea, NomClient, PrenomClient, TypeMvt, #Montant, NumeroClient)
set DateCrea = str_to_date(#DateCrea, '%Y-%m-%d'), Montant = (round(#Montant / 1000)*2) ;
And finished to put a link to the BAT file in SendTo windows folder.
If you're asking if it can be used in a script; you can do some thing like this with php:
<?php
$mysqli = new mysqli("host", "user", "pwd", "db");
/* check connection */
if (mysqli_connect_errno()) {
printf("Connect failed: %s\n", mysqli_connect_error());
exit();
}
$sql = "CREATE TABLE number1 (id INT PRIMARY KEY auto_increment,data TEXT)";
if ($result = $mysqli->query($sql)) {
} else {
printf("<br>%s",$mysqli->error);
}
$host = $_SERVER['HTTP_HOST'];
$uri = rtrim(dirname($_SERVER['PHP_SELF']), '/\\');
$filename = "data.csv";
$sql = "LOAD DATA LOCAL INFILE '$host$uri$filename' INTO TABLE number1";
if ($result = $mysqli->query($sql)) {
} else {
printf("<br>%s",$mysqli->error);
}
// Close the DB connection
$mysqli->close();
exit;
%>
If the file is in the same folder as the script just use $filename a instead of $host$uri$filename. I put this together quick from a couple scripts I'm using, sorry if it doesn't work without debug, but it should be pretty close. It requires mysqli.
Related
The Code i tried is
$source = fopen('filename.csv', 'r') or die("Problem open file");
while (($data = fgetcsv($source, 1000, ",")) !== FALSE)
{
$name = $data[0];
`$mobile = $data[1];
mysql_query("INSERT INTO `table` (`name`,`mobile`) VALUES ('".$name."','".$mobile ."') ");
}
fclose($source);
but it takes only last record...
Why not just add the whole file using MySQL rather than breaking it up by line? Something like
LOAD DATA INFILE 'filename.csv' INTO TABLE table FIELDS TERMINATED BY ',';
or if you've got specific columns that need to go into specific fields,
LOAD DATA INFILE 'filename.csv' INTO TABLE table FIELDS TERMINATED BY ',' (#c1, #c2)
set name = #c1, mobile = #c2;
Here is an easy way to do it, using PHP fgetcsv() and mysqli():
1. Connect to mysqli:
$mysqli = new mysqli("localhost", "my_user", "my_password", "my_db");
2. Open your CSV file:
$source = fopen('filename.csv', 'r') or die("Problem open file");
3. With a loop read the CSV and put your inserts queries in a variable:
$query="";
do {
if ($fields[0]) {
$query.="INSERT INTO `table_name` (`name`, `mobile`) VALUES
(
'".addslashes($fields[0])."',
'".addslashes($fields[1])."'
)
;";
}
} while ($fields = fgetcsv($source,1000,",","'"));
4. Excute your query and close mysqli:
if($mysqli->multi_query($query)) echo "Data successfuly added";
$mysqli->close();
fgetcsv documentation.
Myqli multiquery documentation
I have one problem, I am totally beginner and I would like Insert or convert data, which I have on my PC like txt file to my tables created by Mysql language. Is it possible?
Table has same columns like txt file.
Thank you so much
You can write direct sql query Like this
LOAD DATA INFILE 'C://path/to/yourfilename.txt'
INTO TABLE 'database_name'.'table_name'
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n\r'
(column1,column2)
Here i assumed that your fields are terminated by semi column.
for line termination Character and Escape sequence you can take reference from this thread
http://dev.mysql.com/doc/refman/5.1/en/load-data.html
You can write a simple program that loops through the content of the file and inserts the data into the database. Here is something to get you started.
This simply inserts the data you have put in the PHP code.
What you will want to do is to open a file and insert the data from it
mysql_connect('localhost', 'user', 'pass') or die(mysql_error());
mysql_select_db('db_name') or die(mysql_error());
$lines = file('company.txt');
$company_names = ""; $insert_string
= "INSERT INTO company(company_name) VALUES"; $counter = 0; $maxsize = count($lines);
foreach($lines as $line => $company) {
$insert_string .= "('".$company."')"; $counter++; if($counter <
$maxsize) { $insert_string .= ","; }//if }//foreach
mysql_query($insert_string) or die(mysql_error());
Below is my perl code
I referred to the following link http://www.tol.it/doc/MySQL/chapter6.html. If there is a better please post.
use Mysql;
my $db = Mysql->connect($mysqlhost, $mysqlDB, $mysqlUser, $mysqlPassword);
$db->selectdb("$mysqlDB");
my $loadQuery="LOAD DATA INFILE '$filename' INTO TABLE $pageURLTable FIELDS TERMINATED BY '\\t' LINES TERMINATED BY '\\n'";
print "Executing $loadQuery";
my $loadresult=$db->query($loadQuery);
if(!$loadresult){
print "Error: MySQl Load failed.System error message:$!.";
return -1;
}
print "Info:".$loadresult->info"; ## this raises error MySQL::Statement info not loadable;
What is wrong ?
Can you suggest a better way of coding this so that Load data file errors are better captured?
Thanks,
Neetesh
Try to use ->do method instead of ->query.
Make sure that Load Data Inline is enabled in your database.
If above don't work, you can use system(mysql -e "LOAD DATA INFILE...")
I generally use DBI recipes. "executing sql statement" I like to do something like $rows = $sth->execute(); Gives affected rows by last statement, and if rows >0 after loading csv files I'know that I've successfully loaded data in database.
my $sth = $dbh->prepare($sql)
or die "Can't prepare SQL statement: ", $dbh->errstr(), "\n";
$rows = $sth->execute();
if ($rows>0)
{
print "Loaded $rows rows\n" if ($Debug);
}
else {
print "Can't execute SQL statement: ", $sth->errstr(), "\n";
}
I have a .txt file that has a bunch of formatted data in it that looks like the following:
...
1 75175.18 95128.46
1 790890.89 795829.16
1 875975.98 880914.25
8 2137704.37 2162195.53
8 2167267.27 2375275.28
10 2375408.74 2763997.33
14 2764264.26 2804437.77
15 2804504.50 2881981.98
16 2882048.72 2887921.25
16 2993093.09 2998031.36
19 3004104.10 3008041.37
...
I am trying to load each row as an entry into a table in my database, where each column is a different field. I am having trouble getting mySQL to separate all of the data properly. I think the issue is coming from the fact that not all of the numbers are separated with an equidistant white-space amount.
Here are two queries I have tried so far (I have also tried several variations of these queries):
LOAD DATA LOCAL INFILE
'/some/Path/segmentation.txt'
INTO TABLE clip (slideNum, startTime, endTime)
SET presID = 1;
LOAD DATA LOCAL INFILE
'/some/Path/segmentation.txt'
INTO TABLE clip
FIELDS TERMINATED BY ' '
LINES TERMINATED BY '\n'
(slideNum, startTime, endTime)
SET presID = 1;
Any ideas how to get this to work?
These are what we call "fixed-width" records and LOAD DATA doesn't play well with them. Options:
Clean up data in Excel first, or
Load up the data to a temp table with only 1 column, shoving an entire text row into that column. Then you can use SUBSTR() and TRIM() to slice out the columns you need into the final table.
Or with user variables (#row) you can do it all within the LOAD DATA statement.
LOAD DATA LOCAL INFILE
'/some/Path/segmentation.txt'
INTO TABLE clip
(#row)
SET slideNum = TRIM(SUBSTR(#row,1,4)),
startTime = TRIM(SUBSTR(#row,5,13)),
endTime = TRIM(SUBSTR(#row,18,13))
;
LOAD DATA
CHARACTERSET AL32UTF8
INFILE 'DCF Master 14APR2013 VSPCFM_reduced size.txt'
INTO TABLE EMPLOYEE3
(
a = TRIM(SUBSTR(#row,1,11)),
b = TRIM(SUBSTR(#row,33,38)),
c = TRIM(SUBSTR(#row,70,86))
)
If you're on unix/linux then you can put it through sed to strip out spaces. The solution is here
You can programmatically replace spaces with a different delimiter. I decided to use PHP, you can also safely do it in Python
<?php
$mysqli = new mysqli(
"***",
"***",
"***",
"***",
3306
);
mysqli_options($mysqli, MYSQLI_OPT_LOCAL_INFILE, true);
if (mysqli_connect_errno()) {
printf("Connect failed: %s\n", mysqli_connect_error());
exit();
}
function createTempFileWithDelimiter($filename, $path){
$content = file_get_contents($filename);
$replaceContent = preg_replace('/\ +/', ',', $content); // NOT \s+
$onlyFileName = explode('\\',$filename);
$newFileName = $path.end($onlyFileName);
file_put_contents($newFileName, $replaceContent);
return $newFileName;
}
$pathTemp = 'C:\\TempDir\\';
$pathToFile = 'C:\\some\\Path\\segmentation.txt';
$file = createFileWithDelimiter($pathToFile, $pathTemp);
$file = str_replace(DIRECTORY_SEPARATOR, '/', $file);
$sql = "LOAD DATA LOCAL INFILE '".$file."' INTO TABLE `clip`
COLUMNS TERMINATED BY ','
LINES TERMINATED BY '\n' // or '\r\n'
(slideNum, startTime, endTime)
SET presID = 1;";
if (!($stmt = $mysqli->query($sql))) {
echo "\nQuery execute failed: ERRNO: (" . $mysqli->errno . ") " . $mysqli->error;
};
unlink($file);
?>
Don't use '/\s+/' in preg_replace because \s matches any whitespace character (equivalent to [\r\n\t\f\v ]) and the formatting will change, columns and line breaks will disappear.
sample
user id User Name
U456 Mathew
U457 Leon
U458 Cris
U459 Yancy
U460 Jane
and so on up to 500k.
I need to read this text file and insert to MySQL in two columns say User ID and User Name. How do I do this in PHP?
LOAD DATA INFILE
Example:
NOTE: if you run this from Windows you need to escape the forward slashes in the file path.
EXAMPLE:
C:\\path\to\file.txt
Looks like:
C:\\\\path\\to\\file.txt
Here is the query:
LOAD DATA INFILE '/path/to/sample.txt'
INTO TABLE `database_name`.`table_name`
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
(
user_id, user_name
)
Delete the first line and use this command
LOAD DATA INFILE 'C:\\sample.txt'
INTO TABLE Users
FIELDS TERMINATED BY ' '
LINES TERMINATED BY '\r\n';
For more information visit
http://tech-gupshup.blogspot.com/2010/04/loading-data-in-mysql-table-from-text.html
Using PHP, Possibly Something similar to this:
$file = "/path/to/text/file.txt";
$fp = fopen($file, "r");
$data = fread($fp, filesize($file));
fclose($fp);
The above reads the text file into a variable
$output = explode("\n", $output);
foreach($output as $var) {
$tmp = explode("|", $var);
$userId = $tmp[0];
$userName = $tmp[1];
Tell it to explode at each Endline and then store the data in temp variables
$sql = "INSERT INTO table SET userId='$userId', userName='$userName'";
mysql_query($sql);
Execute the query for each line
Depends. If the two fields are separated using a TAB character, then fgetcsv($f,1000,"\t") would work to read in each line. Otherwise use substr() if it's a fixed width column text file to split up the fields (apply trim() eventually).
Loop over the rows and fields, and use your database interface of choice:
db("INSERT INTO tbl (user_id, user_name) VALUES (?,?)", $row);
Use the fopen function of PHP to access and read the file.. From there on the rest is pretty much simple. Read the file line by line and insert the values into the database.
http://us3.php.net/manual/en/function.fopen.php
The above link gives a very good description of how the fopen function works.. Using a loop will be easy in this task.