I'm working on a Perl script which executes few queries on SQL server.
For example:
my $sth = $dbh->prepare ( "select x y z .. );
$sth->execute();
# do some code
my $sth = $dbh->prepare ( "select a b c .. );
$sth->execute();
# do some code
# etc..
It is recommended to disconnect from the server explicitly $dbh->disconnect.
Do I need to disconnect after each execution ? Or it is enough to disconnect only once when script is finished ?
From the documentation:
Disconnects the database from the database handle. disconnect is typically only used before exiting the program. The handle is of little use after disconnecting.
So no.
Disconnecting just means you have to reconnect. It would be a waste of resources.
No, you do not need to disconnect every time. That would be very contra-productive.
You can just fire as many queries as you want. But take into account that you are redeclaring $sth if you use my every time.
Typically if there are several tables that you need to fill, it would look something like this.
use strict;
use warnings;
use DBI;
my $dsn = '...';
my $dbh = DBI->connect($dsn);
my $sth_employee =
$dbh->prepare('select name, title, department_id from employee');
my $sth_department =
$dbh->prepare('select name from department where department_id=?');
$sth_employee->execute;
while (my $res_employee = $sth_employee->fetchrow_hashref) {
$sth_department->execute($res_employee->{department_id});
( my $department ) = $sth_department->fetchrow_array;
say $res_employee->{name};
say "$res_employee->{title}, $department";
say '=' x 10;
}
# no need to explicitly finnish handles or disconnect, DBI does that for us
Related
I have text file(employeedata.txt) with values like this :
ABC#VVV#JHY#ABC#VVV#JHY#ABC#VVV#JHY#ABC#VVV
BBN#NJU#NULL#ABC#VVV#JHY#ABC#VVV#JHY#ABC#OLJ
ABC#BYR#MPL#ABC#VVV#JHY#ABC#TGB#JHY#ABC#NULL
NMP#JDC#NULL#ABC#VVV#JHY#ABC#XCD#JHY#ABC#NULL
UJK#SAB#NULL#ABC#VVV#JHY#ABC#NBG#JHY#ABC#MPL
my text file contains 5,000 lines and I have a Table called Employee with values like this:
id|EmployeLastName|EmployeFirstName|EmployeeAddress
In my file text, in each line, i have EmployeLastName in the first position, EmployeFirstName in the fourth position, EmployeeAddress in the last position
Example :
EmployeLastName#VVV#JHY#EmployeFirstName#VVV#JHY#ABC#VVV#JHY#ABC#EmployeeAddress
Now I want to read text file line by line and insert into table Employee by using perl 5.10.
I am a novice in perl. How can do it?
Well you need to do some reading on DBI driver to get a grip on the code provided bellow -- it is best invested time to work with DB.
NOTE: in this piece of code I read data from internal block __DATA__
use strict;
use warnings;
use Data::Dumper;
use DBI;
my $debug = 0;
my #fields = qw(id last first address); # Field names in Database
my(%record,$rv);
my $hostname = 'db_server_1'; # Database server name
my $database = 'db_employees'; # Database name
my $table = 'db_table'; # Table name
my $port = '3306'; # Database port [default]
# Define DSN
my $dsn = "DBI:mysql:database=$database;host=$hostname;port=$port";
# Connect to Database
my $dbh = DBI->connect($dsn, $user, $password, {RaiseError => 1});
# Define query
my $stq = qq(INSERT INTO $table (id,last,first,address) VALUES(?,?,?,?););
# Prepare query
my $sth = $dbh->prepare($stq);
$dbh->begin_work(); # Ok, we will do insert in one transaction
my $skip = <DATA>; # We skip header in data block
while( <DATA> ) {
#record{#fields} = split /#/; # Fill the hash with record data
print Dumper(\%row) if $debug; # Look at hash in debug mode
$rv = $sth->execute(#record{#fields}); # Execute query with data
print $DBI::errstr if $rv < 0; # If error lets see ERROR message
}
$dbh->commit(); # Commit the transaction
$dbh->disconnect(); # Disconnect from DataBase
__DATA__
id#EmployeLastName#EmployeFirstName#EmployeeAddress
1#Alexander#Makedonsky#267 Mozarella st., Pizza, Italy
2#Vladimir#Lenin#12 Glinka st., Moscow, Italy
3#Donald#Trump#765 Tower ave., Florida, USA
4#Angela#Merkel#789 Schulstrafe st., Berlin, Germany
You can read data from a file with following code
use strict;
use warnings;
my $debug = 1;
my #fields = qw(id last first address); # Field names in Database
my(%record);
my $filename = shift
or die "Provide filename on command line";
open DATA, "< $filename"
or die "Could not open $filename";
while( <DATA> ) {
#record{#fields} = split /#/;
print Dumper(\%record) if $debug;
}
close DATA;
As you very fresh in Perl programming then you probably should start from Learning Perl, then move on Programming Perl, when you get in trouble visit Perl Cookbook and if you decided to dive into database programming Programming the Perl DBI
Well it is nice to know Perl and DBI driver programming. But in your particular case you could load data from a file directly by utilizing MySQL commands.
Loading Data into a Table
Sometimes it is much much easier than it looks at first sight.
I have already connected MySQL to the database but the problem is that
I have some problems with inserting Data from Perl to Mysql.
The error that pops out is:
Use of uninitialized value in concatenation (.) or string at ./etl_server_info.pl line 204, (Which is the connection of the database )
DBD::mysql::st execute failed: called with 1 bind variables when 0 are needed at ($stmt->execute($sql);)
sub insert_record(){
my($data,$db_config)=#_;
my $result = -1; # -1 fail; 0 - succ
# connect to db
# connect to MySQL database
my $dsn = "DBI:mysql:database=".$db_config->{'Top_Data'}.";host=".$db_config->{'127.0.0.1'}.";port=".$db_config->{'3306'};
my $username = $db_config->{'username'};
my $password = $db_config->{'password'};
my %attr = (PrintError=>0,RaiseError=>1 );
my $dbh = DBI->connect($dsn,$username,$password,\%attr) or die $DBI::errstr;
print "We Have Successfully Connected To The Database \n";
# prepare sql statement
# execute insert
my $sql = 'insert into Top(Load_Average, CPU_us, CPU_id, CPU-wa, CPU_hi, CPU_si, CPU_st, Swap_Total, Swap_Free, Swap_Used, Memory_Total, Memeory_Free, Memory_Used, Memory_Buff, Date) values(float,float,float,float,float,float,float,float,varchar,varchar,varchar,varchar,varchar,varchar,date)';
my $stmt =$dbh->prepare($sql) or die "Its A Fail" . $dbh->errstr();
$stmt->execute($sql);
$stmt->finish();
$dbh->disconnect();
$result = 0;
return($result);
Your use of the $db_config variable looks suspicious to me. Either your config hash is strange, or you're using values instead of keys.
You haven't shown us where $db_config is set up, but I'd guess it looks something like this:
$db_config = {
name => 'Top_Data',
host => '127.0.0.1',
port => 3306,
username => 'someone',
password => 'a secret',
};
And then you would use it like this:
my $dsn = "DBI:mysql:database=".$db_config->{name}.";host=".$db_config->{host}.";port=".$db_config->{port};
Notice that I've used the key names (name, host and port) instead of the values (Top_Data, 127.0.0.1 and 3306).
I'll also point out that you can simplify this slightly by using Perl's ability to expand variables inside a double-quoted string.
my $dsn = "DBI:mysql:database=$db_config->{name};host=$db_config->{host};port=$db_config->{port}";
There's another problem later on, with your SQL statement.
my $sql = 'insert into Top(Load_Average, CPU_us, CPU_id, CPU-wa, CPU_hi,
CPU_si, CPU_st, Swap_Total, Swap_Free, Swap_Used, Memory_Total,
Memeory_Free, Memory_Used, Memory_Buff, Date)
values(float,float,float,float,float,float,float,float,
varchar,varchar,varchar,varchar,varchar,varchar,date)';
The values that you should be inserting are the actual data items. So where you have the strings "float", "varchar" or "date", you should actually have data items (a floating-point number, a string or a date).
Finally, having prepared your statement, you don't need to pass it to the execute() method. You should, however, look at using bind points in your SQL and passing your actual data items to the execute() call
I am trying to connect to 2 databases on the same instance of MySQL from 1 Perl script.
I am using this in a migration script where I am grabbing data from the original database and inserting it into the new one.
Connecting to 1 database and then trying to initiate a second connection with the same user just changes the current database to the new one.
#!/usr/bin/perl
use DBI;
use strict;
my $driver = "mysql";
my $database1 = "db1";
my $dsn1 = "DBI:$driver:database=$database1";
my $userid = "userhead";
my $password = "pwdhead";
my $database2 = "db2";
my $dsn2 = "DBI:$driver:database=$database2";
my $dbh1 = DBI->connect($dsn1, $userid, $password ) or die $DBI::errstr;
my $dbh2 = DBI->connect($dsn2, $userid, $password ) or die $DBI::errstr;
my $sth = $dbh2->prepare("INSERT INTO Persons") $dbh1->prepare("SELECT *FROM Persons");
$sth->execute() or die $DBI::errstr;
print "Number of rows found :" + $sth->rows;
In the above example i am trying to copy from one database table to another datbase table. but i am getting error while running the script. Please help me out
At a guess, you're trying to use the same database handle to connect to both databases. If you need to operate two separate connections then you need two separate handles
This program uses the data_sources class method to discover all of the available MySQL databases and creates a connection to each of them, putting the handles in the array #dbh. You can use each element of that array as normal, for instance
my $stmt = $dbh[0]->prepare('SELECT * FROM table)
It may be that you prefer to set up the #databases array manually, or the username and password may be different for the two data sources, so some variation on this may be necessary
use strict;
use warnings 'all';
use DBI;
my $user = 'username';
my $pass = 'password';
my #databases = DBI->data_sources('mysql');
my #dbh = map { DBI->connect($_, $user, $pass) } #databases;
Update
You need to select data from the source table, fetch it one row at a time, and insert each row into the destination table
Here's an idea how that might work, but you need to adjust the number of question marks in the VALUES of the INSERT statement to match the number of columns
Note that, if you're just intending to copy the whole dataset, there aree better ways to go about this. In particular, if you have any foreign key constraints then you won't be able to add data until the table it it is dependent on is populated
#!/usr/bin/perl
use strict;
use warnings 'all';
use DBI;
my $userid = "userhead";
my $password = "pwdhead";
my ($dbase1, $dbase2) = qw/ db1 db2 /;
my $dsn1 = "DBI:mysql:database=$dbase1";
my $dsn2 = "DBI:mysql:database=$dbase2";
my $dbh1 = DBI->connect($dsn1, $userid, $password ) or die $DBI::errstr;
my $dbh2 = DBI->connect($dsn2, $userid, $password ) or die $DBI::errstr;
my $select = $dbh1->prepare("SELECT * FROM Persons");
my $insert = $dbh2->prepare("INSERT INTO Persons VALUES (?, ?, ?, ?, ?)");
$select->execute;
while ( my #row = $select->fetchrow_array ) {
$insert->execute(#row);
}
If you need to handle the columns from the source data separately then you can use named scalars instead of the array #row. Like this
while ( my ($id, $name) = $select->fetchrow_array ) {
my $lastname = '';
$insert->execute($id, $name, $lastname);
}
Plan A (especially if one-time task):
Run mysqldump on the source machine; feed the output to mysql on the target machine. This will be much faster and simpler. If you are on a Unix machine, do it with an exec() from Perl (if you like).
Plan B (especially if repeated task):
If the table is not "too big", do one SELECT to fetch all the rows into an array in Perl. Then INSERT the rows into the target machine. This can be sped up (with some effort) if you build a multi-row INSERT or create a CSV file and use LOAD DATA instead of INSERT.
I've created a Perl script which is meant to loop through an array (a shortlist of customers who meet certain criteria), execute an external command using system() , then update a field within each row once the operation has completed.
It works on the first record (ie external command executes, customer record updates), however when it gets to the second record I receive this error:
DBD::mysql::st fetchrow_array failed: fetch() without execute() at customer_update.pl
Through some googling I added the $sth->finish(); command, however whether I include it or not (either inside the loop as shown, or straight afterward) I still get the same error.
Can anyone shed any light for me as to what I am doing wrong here?
Here's an extract:
# PERL MYSQL CONNECT()
$dbh = DBI->connect('dbi:mysql:signups', $user, $pw)
or die "Connection Error: $DBI::errstr\n";
# DEFINE A MySQL QUERY
$myquery = "SELECT * FROM accounts WHERE field3 = false";
$sth = $dbh->prepare($myquery);
# EXECUTE THE QUERY
$sth->execute
or die "SQL Error: $DBI::errstr\n";
#records = $sth->rows;
print "Amount of new customers: #records\n\n";
while ( my ($field1, $field2, $field3) = $sth->fetchrow_array() ) {
#execute external command via system();
$update_customer_status = "UPDATE accounts SET field3=true WHERE id=$id";
$sth = $dbh->prepare($update_customer_status);
$sth->execute
or die "SQL Error: $DBI::errstr\n";
print "Customer record modified & MySQL updated accordingly\n\n";
$sth->finish();
}
Building a SQL statement with variables and then prepare()ing it defeats the purpose of the prepare. You should build the SQL statement with a placeholder ? instead of $id, prepare() it, and then execute($id) it. As it is, you are leaving yourself open to SQL injection attacks.
Also, it seems that you are not using the warnings and strict pragmas. These two lines should be at the top of every program you write:
use warnings;
use strict;
They will save you much heartache and frustration in the future.
In your loop, you overwrite the handle over from which you are fetching. Use a different variable. (Changing $sth = ...; to my $sth = ...; will do.) While we're at it, let's move the prepare out of the loop.
my $sth_get = $dbh->prepare("SELECT * FROM accounts WHERE field3 = false");
my $sth_upd = $dbh->prepare("UPDATE accounts SET field3=true WHERE id = ?");
$sth_get->execute();
while ( my ($field1, $field2, $field3) = $sth_get->fetchrow_array() ) {
...
$sth_upd->execute($id);
}
You are stomping on your $sth variable when you execute this line ...
$sth = $dbh->prepare($update_customer_status);
Why not save off the result of $sth->fetchrow_array() to an array variable.
Something like ...
my #select_results_AoA = $sth->fetchrow_array();
... and then iterate over the array ...
for my #row ( #select_resilts_AoA ) {
... instead of ...
while ( my ($field1, $field2, $field3) = $sth->fetchrow_array() ) {
I have tried with Perl fork manager and DBI . But i got the error DBD::mysql::st execute failed: Lost connection to MySQL server during query .
Here the sample code: I want make query between low to high value (i have spitted int 10k records)
use Parallel::ForkManager;
my $pm = new Parallel::ForkManager(50);
my $db = krish::DB->new or die $!; # its has all connection details
while ( $low < $high ) {
# Some value manipulation
my $pid = $pm->start and next;
#db_execution returns execution
while ( my $sth = db_execution ( $db, $low , $high ) ) {
...
#fetch row operation
...
}
$pm->finish;
}
sub db_execution {
...
my $dbh = $db->connect( 'students' ) or die $!;
my $sth = $dbh->prepare( $sql ) or die "$!:" . $dbh->errstr;
$sth->execute or die "$!:" . $sth->errstr;
...
}
The same code is executing with out parallel processing. What is the issue?
How to resolve is this?
When you share database connections between processes (which is what you're doing with a fork) you need to make sure that one process doesn't close it out from under the other one. Because the connections are also variables, when the Perl interpreter shuts down it will call the DESTROY method of that object which in this case will close the connection.
So if any of the children close the db connection (which will happen when they finish and shutdown) it will kill it out from under the parent process. The way to prevent this is to set InactiveDestroy to true in the parent process before the fork and then to close the connection explicitly in the parent when done.
https://metacpan.org/pod/DBI#InactiveDestroy
You are asking for trouble by using the same db handle simultaneously in all of the child processes. You should be creating a new connection in each child.
Never mind...I read the rest of the code.