I have one problem, I am totally beginner and I would like Insert or convert data, which I have on my PC like txt file to my tables created by Mysql language. Is it possible?
Table has same columns like txt file.
Thank you so much
You can write direct sql query Like this
LOAD DATA INFILE 'C://path/to/yourfilename.txt'
INTO TABLE 'database_name'.'table_name'
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\n\r'
(column1,column2)
Here i assumed that your fields are terminated by semi column.
for line termination Character and Escape sequence you can take reference from this thread
http://dev.mysql.com/doc/refman/5.1/en/load-data.html
You can write a simple program that loops through the content of the file and inserts the data into the database. Here is something to get you started.
This simply inserts the data you have put in the PHP code.
What you will want to do is to open a file and insert the data from it
mysql_connect('localhost', 'user', 'pass') or die(mysql_error());
mysql_select_db('db_name') or die(mysql_error());
$lines = file('company.txt');
$company_names = ""; $insert_string
= "INSERT INTO company(company_name) VALUES"; $counter = 0; $maxsize = count($lines);
foreach($lines as $line => $company) {
$insert_string .= "('".$company."')"; $counter++; if($counter <
$maxsize) { $insert_string .= ","; }//if }//foreach
mysql_query($insert_string) or die(mysql_error());
Related
I am not so into database and I have the following situation: I am working using MySQL and I have to import some data moacked into a Microsoft Excel file.
In this file the first line cells represents a table fields (each cell is a field), the below rows cells contains the value related to these fields.
At the beginning I had thought to develop a Java program that access to this Excel file, parse it and populate my DB. But this solution is unsustainable because I had a lot of Excel files (each file contains the mocked data for a specific table).
Can I directly use an Excel file (or the converted version in .csv) to populate a MySql table? Exist an easy way to do it?
If so what are the precautions to be taken into account?
Mysql Query (if you have admin rights)
SELECT * INTO OUTFILE "/mydata.csv"
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
LINES TERMINATED BY "\n"
FROM test;
Or an easy PHP script
$result = $db_con->query('SELECT * FROM `some_table`');
if (!$result) die('Couldn\'t fetch records');
$num_fields = mysql_num_fields($result);
$headers = array();
for ($i = 0; $i < $num_fields; $i++) {
$headers[] = mysql_field_name($result , $i);
}
$fp = fopen('php://output', 'w');
if ($fp && $result) {
header('Content-Type: text/csv');
header('Content-Disposition: attachment; filename="export.csv"');
header('Pragma: no-cache');
header('Expires: 0');
fputcsv($fp, $headers);
while ($row = $result->fetch_array(MYSQLI_NUM)) {
fputcsv($fp, array_values($row));
}
die;
}
The Code i tried is
$source = fopen('filename.csv', 'r') or die("Problem open file");
while (($data = fgetcsv($source, 1000, ",")) !== FALSE)
{
$name = $data[0];
`$mobile = $data[1];
mysql_query("INSERT INTO `table` (`name`,`mobile`) VALUES ('".$name."','".$mobile ."') ");
}
fclose($source);
but it takes only last record...
Why not just add the whole file using MySQL rather than breaking it up by line? Something like
LOAD DATA INFILE 'filename.csv' INTO TABLE table FIELDS TERMINATED BY ',';
or if you've got specific columns that need to go into specific fields,
LOAD DATA INFILE 'filename.csv' INTO TABLE table FIELDS TERMINATED BY ',' (#c1, #c2)
set name = #c1, mobile = #c2;
Here is an easy way to do it, using PHP fgetcsv() and mysqli():
1. Connect to mysqli:
$mysqli = new mysqli("localhost", "my_user", "my_password", "my_db");
2. Open your CSV file:
$source = fopen('filename.csv', 'r') or die("Problem open file");
3. With a loop read the CSV and put your inserts queries in a variable:
$query="";
do {
if ($fields[0]) {
$query.="INSERT INTO `table_name` (`name`, `mobile`) VALUES
(
'".addslashes($fields[0])."',
'".addslashes($fields[1])."'
)
;";
}
} while ($fields = fgetcsv($source,1000,",","'"));
4. Excute your query and close mysqli:
if($mysqli->multi_query($query)) echo "Data successfuly added";
$mysqli->close();
fgetcsv documentation.
Myqli multiquery documentation
I have a .txt file that has a bunch of formatted data in it that looks like the following:
...
1 75175.18 95128.46
1 790890.89 795829.16
1 875975.98 880914.25
8 2137704.37 2162195.53
8 2167267.27 2375275.28
10 2375408.74 2763997.33
14 2764264.26 2804437.77
15 2804504.50 2881981.98
16 2882048.72 2887921.25
16 2993093.09 2998031.36
19 3004104.10 3008041.37
...
I am trying to load each row as an entry into a table in my database, where each column is a different field. I am having trouble getting mySQL to separate all of the data properly. I think the issue is coming from the fact that not all of the numbers are separated with an equidistant white-space amount.
Here are two queries I have tried so far (I have also tried several variations of these queries):
LOAD DATA LOCAL INFILE
'/some/Path/segmentation.txt'
INTO TABLE clip (slideNum, startTime, endTime)
SET presID = 1;
LOAD DATA LOCAL INFILE
'/some/Path/segmentation.txt'
INTO TABLE clip
FIELDS TERMINATED BY ' '
LINES TERMINATED BY '\n'
(slideNum, startTime, endTime)
SET presID = 1;
Any ideas how to get this to work?
These are what we call "fixed-width" records and LOAD DATA doesn't play well with them. Options:
Clean up data in Excel first, or
Load up the data to a temp table with only 1 column, shoving an entire text row into that column. Then you can use SUBSTR() and TRIM() to slice out the columns you need into the final table.
Or with user variables (#row) you can do it all within the LOAD DATA statement.
LOAD DATA LOCAL INFILE
'/some/Path/segmentation.txt'
INTO TABLE clip
(#row)
SET slideNum = TRIM(SUBSTR(#row,1,4)),
startTime = TRIM(SUBSTR(#row,5,13)),
endTime = TRIM(SUBSTR(#row,18,13))
;
LOAD DATA
CHARACTERSET AL32UTF8
INFILE 'DCF Master 14APR2013 VSPCFM_reduced size.txt'
INTO TABLE EMPLOYEE3
(
a = TRIM(SUBSTR(#row,1,11)),
b = TRIM(SUBSTR(#row,33,38)),
c = TRIM(SUBSTR(#row,70,86))
)
If you're on unix/linux then you can put it through sed to strip out spaces. The solution is here
You can programmatically replace spaces with a different delimiter. I decided to use PHP, you can also safely do it in Python
<?php
$mysqli = new mysqli(
"***",
"***",
"***",
"***",
3306
);
mysqli_options($mysqli, MYSQLI_OPT_LOCAL_INFILE, true);
if (mysqli_connect_errno()) {
printf("Connect failed: %s\n", mysqli_connect_error());
exit();
}
function createTempFileWithDelimiter($filename, $path){
$content = file_get_contents($filename);
$replaceContent = preg_replace('/\ +/', ',', $content); // NOT \s+
$onlyFileName = explode('\\',$filename);
$newFileName = $path.end($onlyFileName);
file_put_contents($newFileName, $replaceContent);
return $newFileName;
}
$pathTemp = 'C:\\TempDir\\';
$pathToFile = 'C:\\some\\Path\\segmentation.txt';
$file = createFileWithDelimiter($pathToFile, $pathTemp);
$file = str_replace(DIRECTORY_SEPARATOR, '/', $file);
$sql = "LOAD DATA LOCAL INFILE '".$file."' INTO TABLE `clip`
COLUMNS TERMINATED BY ','
LINES TERMINATED BY '\n' // or '\r\n'
(slideNum, startTime, endTime)
SET presID = 1;";
if (!($stmt = $mysqli->query($sql))) {
echo "\nQuery execute failed: ERRNO: (" . $mysqli->errno . ") " . $mysqli->error;
};
unlink($file);
?>
Don't use '/\s+/' in preg_replace because \s matches any whitespace character (equivalent to [\r\n\t\f\v ]) and the formatting will change, columns and line breaks will disappear.
sample
user id User Name
U456 Mathew
U457 Leon
U458 Cris
U459 Yancy
U460 Jane
and so on up to 500k.
I need to read this text file and insert to MySQL in two columns say User ID and User Name. How do I do this in PHP?
LOAD DATA INFILE
Example:
NOTE: if you run this from Windows you need to escape the forward slashes in the file path.
EXAMPLE:
C:\\path\to\file.txt
Looks like:
C:\\\\path\\to\\file.txt
Here is the query:
LOAD DATA INFILE '/path/to/sample.txt'
INTO TABLE `database_name`.`table_name`
FIELDS TERMINATED BY ','
LINES TERMINATED BY '\r\n'
IGNORE 1 LINES
(
user_id, user_name
)
Delete the first line and use this command
LOAD DATA INFILE 'C:\\sample.txt'
INTO TABLE Users
FIELDS TERMINATED BY ' '
LINES TERMINATED BY '\r\n';
For more information visit
http://tech-gupshup.blogspot.com/2010/04/loading-data-in-mysql-table-from-text.html
Using PHP, Possibly Something similar to this:
$file = "/path/to/text/file.txt";
$fp = fopen($file, "r");
$data = fread($fp, filesize($file));
fclose($fp);
The above reads the text file into a variable
$output = explode("\n", $output);
foreach($output as $var) {
$tmp = explode("|", $var);
$userId = $tmp[0];
$userName = $tmp[1];
Tell it to explode at each Endline and then store the data in temp variables
$sql = "INSERT INTO table SET userId='$userId', userName='$userName'";
mysql_query($sql);
Execute the query for each line
Depends. If the two fields are separated using a TAB character, then fgetcsv($f,1000,"\t") would work to read in each line. Otherwise use substr() if it's a fixed width column text file to split up the fields (apply trim() eventually).
Loop over the rows and fields, and use your database interface of choice:
db("INSERT INTO tbl (user_id, user_name) VALUES (?,?)", $row);
Use the fopen function of PHP to access and read the file.. From there on the rest is pretty much simple. Read the file line by line and insert the values into the database.
http://us3.php.net/manual/en/function.fopen.php
The above link gives a very good description of how the fopen function works.. Using a loop will be easy in this task.
Is there a way to dynamically specify a file name in the LOAD DATA INFILE? Can it be parameterized like for instance (syntax may be incorrect) LOAD DATA INFILE '$filename'?
A citation from MySQL documentation:
The LOAD DATA INFILE statement reads rows from a text file into a table at a very high speed. The file name must be given as a literal string.
That means that it can not be a parameter of a prepared statement. But no one forbids to make the string interpolation while the statement is just a string in your PHP code.
Unfortunately, this feature is not yet supported in MySQL and is currently listed as bug/feature request #39115 http://bugs.mysql.com/bug.php?id=39115
Or you can make a temporary copy of the file (BATCH example):
LOAD_DATA.bat
COPY %1 TempFileToLoad.csv
mysql --user=myuser --password=mypass MyDatabase < ScriptLoadMyDatabase.sql
DEL TempFileToLoad.csv
the SQL (for info) :
ScriptLoadMyDatabase.sql
load data infile 'TempFileToLoad.csv' IGNORE
into table tLoad
FIELDS TERMINATED BY ';' OPTIONALLY ENCLOSED BY '"'
lines terminated by '\r\n'
IGNORE 1 LINES
(#DateCrea, NomClient, PrenomClient, TypeMvt, #Montant, NumeroClient)
set DateCrea = str_to_date(#DateCrea, '%Y-%m-%d'), Montant = (round(#Montant / 1000)*2) ;
And finished to put a link to the BAT file in SendTo windows folder.
If you're asking if it can be used in a script; you can do some thing like this with php:
<?php
$mysqli = new mysqli("host", "user", "pwd", "db");
/* check connection */
if (mysqli_connect_errno()) {
printf("Connect failed: %s\n", mysqli_connect_error());
exit();
}
$sql = "CREATE TABLE number1 (id INT PRIMARY KEY auto_increment,data TEXT)";
if ($result = $mysqli->query($sql)) {
} else {
printf("<br>%s",$mysqli->error);
}
$host = $_SERVER['HTTP_HOST'];
$uri = rtrim(dirname($_SERVER['PHP_SELF']), '/\\');
$filename = "data.csv";
$sql = "LOAD DATA LOCAL INFILE '$host$uri$filename' INTO TABLE number1";
if ($result = $mysqli->query($sql)) {
} else {
printf("<br>%s",$mysqli->error);
}
// Close the DB connection
$mysqli->close();
exit;
%>
If the file is in the same folder as the script just use $filename a instead of $host$uri$filename. I put this together quick from a couple scripts I'm using, sorry if it doesn't work without debug, but it should be pretty close. It requires mysqli.