I am relatively new to perl and this forum. I am trying to use perl DBI for mySql LOAD DATA INFILE statement to upload a csv file to a mySQL database. However, the execute statement returns an undef value. However if i use a SELECT or DESC statement, it works fine. I suspect that the single quotes and braces in the query is causing the error but don't know how to debug.
#!/usr/bin/perl -w
use strict;
use warnings;
use File::Basename;
use DBI;
use DBD::mySQL;
my $data_path="D:\\NickD\\Project\\StockData\\";
my $db = "TestMMDB";
my $user = "user";
my $pass ="pass";
my $host = "localhost";
my $query ="";
my #row;
my #files = glob("$data_path*.csv");
DBI->trace(1);
my $dbh = DBI->connect("dbi:mysql:$db:$host",$user,$pass);
foreach my $file(#files){
my $filename = basename($file);
my ($db_table,$date) = split("_",$filename);
$query = q{LOAD DATA INFILE ? INTO TABLE ? FIELDS TERMINATED BY ',' (Date,Symbol,Open,High,Low,Close,Volume)};
my $sqlQuery = $dbh->prepare($query);
my $rv = $sqlQuery->execute($file,$db.".".$db_table) or die "Oops!: Can't execute the query :".$sqlQuery->errstr;
while (#row = $sqlQuery->fetchrow_array()) {
print "#row\n";
}
}
my $rc = $dbh->disconnect();
exit(0);
All help will be greatly appreciated.
Related
I am trying to import several .csv files into a mysql database, the script below works except that it only imports the first row of my csv data into the database. Both my tables are populated with exactly one data entry.
Any help would be appreciated.
Thank you
#!/usr/bin/perl
use DBI;
use DBD::mysql;
use strict;
use warnings;
# MySQL CONFIG VARIABLES
my $host = "localhost";
my $user = "someuser";
my $pw = "somepassword";
my $database = "test";
my $dsn = "DBI:mysql:database=" . $database . ";host=" . $host;
my $dbh = DBI->connect($dsn, $user, $pw)
or die "Can't connect to the DB: $DBI::errstr\n";
print "Connected to DB!\n";
# enter the file name that you want import
my $filename = "/home/jonathan/dep/csv/linux_datetime_test_4.26.13_.csv";
open FILE, "<", $filename or die $!;
$_ = <FILE>;
$_ = <FILE>;
while (<FILE>) {
my #f = split(/,/,$_);
if (length($f[4]) < 10) {
print "No Weight\n";
}
else {
#insert the data into the db
print "insert into datetime_stamp\n";
}
my $sql = "INSERT INTO datetime_stamp (subject, date, time, weight)
VALUES('$f[1]', '$f[2]', '$f[3]', '$f[4]')";
print "$sql\n";
my $query = $dbh->do($sql);
my $sql = "INSERT INTO subj_weight (subject, weight) VALUES('$f[1]', '$f[2]')";
my $query = $dbh->do($sql);
close(FILE);
}
As has been commented, you close the input file after reading the first data entry, and so only populate your database with a single record.
However there are a few problems with your code you may want to consider:
You should set autoflush on the STDOUT file handle if you are printing diagnostics as the program runs. Otherwise perl won't print the output until either it has a buffer full of text to print or the file handle is closed when the program exits. That means you may not see the messages you have coded until long after the event
You should use Text::CSV to parse CSV data instead of relying on split
You can interpolate variables into a double-quoted string. That avoids the use of several concatenation operators and makes the intention clearer
Your open is near-perfect - an unusual thing - because you correctly use the three-parameter form of open as well as testing whether it succeeded and putting $! in the die string. However you should also always use a lexical file handle as well instead of the old-fashioned global ones
You don't chomp the lines you read from the input, so the last field will have a trailing newline. Using Text::CSV avoids the need for this
You use indices 1 through 4 of the data split from the input record. Perl indices start at zero, so that means you are droppping the first field. Is that correct?
Similarly you are inserting fields 1 and 2, which appear to be subject and date, into fields called subject and weight. It seems unlikely that this can be right
You should prepare your SQL statements, use placeholders, and provide the actual data in an execute call
You seem to diagnose the data read from the file ("No Weight") but insert the data into the database anyway. This may be correct but it seems unlikely
Here is a version of your program that includes these amendments. I hope it is of use to you.
#!/usr/bin/perl
use strict;
use warnings;
use DBI;
use Text::CSV;
use IO::Handle;
STDOUT->autoflush;
# MySQL config variables
my $host = "localhost";
my $user = "someuser";
my $pw = "somepassword";
my $database = "test";
my $dsn = "DBI:mysql:database=$database;host=$host";
my $dbh = DBI->connect($dsn, $user, $pw)
or die "Can't connect to the DB: $DBI::errstr\n";
print "Connected to DB!\n";
my $filename = "/home/jonathan/dep/csv/linux_datetime_test_4.26.13_.csv";
open my $fh, '<', $filename
or die qq{Unable to open "$filename" for input: $!};
my $csv = Text::CSV->new;
$csv->getline($fh) for 1, 2; # Drop header lines
my $insert_datetime_stamp = $dbh->prepare( 'INSERT INTO datetime_stamp (subject, date, time, weight) VALUES(?, ?, ?, ?)' );
my $insert_subj_weight = $dbh->prepare( 'INSERT INTO subj_weight (subject, weight) VALUES(?, ?)' );
while (my $row = $csv->getline($fh)) {
if (length($row->[4]) < 10) {
print qq{Invalid weight: "$row->[4]"\n};
}
else {
#insert the data into the db
print "insert into datetime_stamp\n";
$insert_datetime_stamp->execute(#$row[1..4]);
$insert_subj_weight->execute(#$row[1,4]);
}
}
J have my inventory data inside a csv file.
I get an error message because I have a special character on my insert values, I don't want to deal with those values because they were tagged with special characters $" to not use them anymore for insert. I use the code bellow:
#!/usr/bin/perl
# PERL MODULES WE WILL BE USING
use DBI;
use DBD::mysql;
# HTTP HEADER
print "Content-type: text/html \n\n";
# CONFIG VARIABLES
$platform = "mysql";
$database = "store";
$host = "localhost";
$port = "3306";
$tablename = "inventory";
$user = "username";
$pw = "password";
# DATA SOURCE NAME
$dsn = "dbi:$platform:$database:$host:$port";
# PERL DBI CONNECT
$connect = DBI->connect($dsn, $user, $pw);
# PREPARE THE QUERY
$query = "INSERT INTO inventory (id, product, quantity) VALUES (DEFAULT, 'tomatoes$"', '4')";
$query_handle = $connect->prepare($query);
# EXECUTE THE QUERY
$query_handle->execute();
How can I skipe the error message and move to the next insert.
You'll have to escape the $ in your query, because right now it's a perl syntax error:
$sql = "INSERT .... 'tomatoes$"', '4')";
^----
that quote TERMINATES the sql string, as it's not a valid variable. Try
$sql = "INSERT .... 'tomatoes\$"', '4')";
^--
instead.
Handle your insert sentence avoiding sql-injection:
my $query = qq!INSERT INTO inventory (id, product, quantity) VALUES (?,?,?)!;
my $query_handle = $connect->prepare($query);
$query_handle->execute('DEFAULT', 'tomatoes$', '4');
Below, is the code to connect mysql database and retrieving results using perl.
In the below example, samples table has 10 columns. I just want the second and third columns of record 99 into variables.
Can some one help me in this?
use strict;
use warnings;
use DBI;
my $dbh = DBI->connect('dbi:mysql:perltest','root','password') or die "Connection Error: $DBI::errstr\n";
my $sql = "select * from samples where record='99'";
my $sth = $dbh->prepare($sql);
$sth->execute or die "SQL Error: $DBI::errstr\n";
while (my #row = $sth->fetchrow_array) {
print "#row\n";
}
Thanks in advance
This code will get you the second and third columns of the row into variables:
while (my #row = $sth->fetchrow_array) {
$var1 = #row[1];
$var2 = #row[2];
}
I have written the following code in Perl. I have ActivePerl 5.14 for Windows 7.
#!C:\perl64\bin\perl.exe -wT
use strict;
use warnings;
use DBI;
print "Content-type: text/html \n\n";
# MYSQL CONFIG VARIABLES
my $driver = "mysql";
my $database = "test555";
my $tablename3 = "test77";
my $user = "root";
my $pw = "root";
# PERL MYSQL CONNECT()
my $dbh = DBI->connect("DBI:$driver:$database", $user, $pw,);
my $sth = $dbh->prepare("
SELECT *
FROM t6
WHERE paragraph='PWE1234'
");
$sth->execute();
#$dbh->disconnect;
#exit 0;
When the program reaches $dbh->disconnect, the system is throwing an error; hence commented it out. When I comment that out, the system is not throwing any error, but neither do I get output.
There is a result for this query, I checked with MySQL once separately.
There is no output because you have no code to create any output.
After calling execute you need to call one of the fetchsomething methods and do something with the data structure you get back.
How do I call MySQL stored procedures from Perl? Stored procedure functionality is fairly new to MySQL and the MySQL modules for Perl don't seem to have caught up yet.
MySQL stored procedures that produce datasets need you to use Perl DBD::mysql 4.001 or later. (http://www.perlmonks.org/?node_id=609098)
Below is a test program that will work in the newer version:
mysql> delimiter //
mysql> create procedure Foo(x int)
-> begin
-> select x*2;
-> end
-> //
perl -e 'use DBI; DBI->connect("dbi:mysql:database=bonk", "root", "")->prepare("call Foo(?)")->execute(21)'
But if you have too old a version of DBD::mysql, you get results like this:
DBD::mysql::st execute failed: PROCEDURE bonk.Foo can't return a result set in the given context at -e line 1.
You can install the newest DBD using CPAN.
There's an example in the section on Multiple result sets in the DBD::mysql docs.
#!/usr/bin/perl
# Stored Proc - Multiple Values In, Multiple Out
use strict;
use Data::Dumper;
use DBI;
my $dbh = DBI->connect('DBI:mysql:RTPC;host=db.server.com',
'user','password',{ RaiseError => 1 }) || die "$!\n";
my $sth = $dbh->prepare('CALL storedProcedure(?,?,?,?,#a,#b);');
$sth->bind_param(1, 2);
$sth->bind_param(2, 1003);
$sth->bind_param(3, 5000);
$sth->bind_param(4, 100);
$sth->execute();
my $response = $sth->fetchrow_hashref();
print Dumper $response . "\n";
It took me a while to figure it out, but I was able to get what I needed with the above. if you need to get multiple return "lines" I'm guessing you just...
while(my $response = $sth->fetchrow_hashref()) {
print Dumper $response . "\n";
}
I hope it helps.
First of all you should be probably connect through the DBI library and then you should use bind variables. E.g. something like:
#!/usr/bin/perl
#
use strict;
use DBI qw(:sql_types);
my $dbh = DBI->connect(
$ConnStr,
$User,
$Password,
{RaiseError => 1, AutoCommit => 0}
) || die "Database connection not made: $DBI::errstr";
my $sql = qq {CALL someProcedure(1);} }
my $sth = $dbh->prepare($sql);
eval {
$sth->bind_param(1, $argument, SQL_VARCHAR);
};
if ($#) {
warn "Database error: $DBI::errstr\n";
$dbh->rollback(); #just die if rollback is failing
}
$dbh->commit();
Mind you i haven't tested this, you'll have to lookup the exact syntax on CPAN.
Hi, similar to above but using SQL exec. I could not get the CALL command to work. You will need to fill in anything that is within square brackets and remove the square brackets.
use DBI;
#START: SET UP DATABASE AND CONNECT
my $host = '*[server]*\\*[database]*';
my $database = '*[table]*';
my $user = '*[user]*';
my $auth = '*[password]*';
my $dsn = "dbi:ODBC:Driver={SQL Server};Server=$host;Database=$database";
my $dbh = DBI->connect($dsn, $user, $auth, { RaiseError => 1 });
#END : SET UP DATABASE AND CONNECT
$sql = "exec *[stored procedure name]* *[param1]*,*[param2]*,*[param3]*;";
$sth = $dbh->prepare($sql);
$sth->execute or die "SQL Error: $DBI::errstr\n";