Perl module to log in database and do a simple query? - mysql

I can't seem to figure out the issue with my .pm file and script. I am fairly new to Perl.
I have a database with name "project" and there is table with name "mailing".
mailing table has 7 entries, which I want to display using module.
So, I have this custom module to log in to database and do a query. This module is names as DB.pm
DB.pm is stored on my FEDORA 20 at /root/mysql/GUI/DB.pm.
DB.pm is defined as follows:
package GUI::DB;
use strict;
use DBI;
use vars qw(#ISA #EXPORT);
use Exporter;
#ISA = qw(Exporter);
#EXPORT = qw(dbConnect query);
#
# dbConnect - connect to the database, get the database handle
#
sub dbConnect {
# Read database settings from config file:
print "Works";
my $dsn = "DBI:mysql:project";
my $dbh = DBI->connect( $dsn,
'root',
'mydatabasepassword',
{ RaiseError => 1 }
);
return $dbh;
}
#
# query - execute a query with parameters
# query($dbh, $sql, #bindValues)
#
sub query {
my $dbh = shift;
my $sql ="SELECT * FROM mailing";
my #bindValues = #_; # 0 or serveral parameters
my #returnData = ();
# issue query
my $sth = $dbh->prepare($sql); //**line number 39 that is giving** error
if ( #bindValues ) {
$sth->execute(#bindValues);
} else {
$sth->execute();
}
if ( $sql =~ m/^select/i ) {
while ( my $row = $sth->fetchrow_hashref ) {
push #returnData, $row;
}
}
# finish the sql statement
$sth->finish();
return #returnData;
}
1;
Now I want to use this module inside my per script. This is what I tried:
#!/usr/bin/perl
use warnings;
use strict;
use lib '/root/mysql/';
use GUI::DB qw(dbConnect query);
dbConnect();
query();
This is the error I'm getting -->
Can't call method "prepare" on an undefined value at /root/mysql/GUI/DB.pm line 39.
Please help me with this. I am not sure how to proceed. I am guessing it has something to do with argument passing. Nothing is wrong with database. It works fine from CLI.
Thanks :)
_x_X_X_X_X_X_X_X_X_X_X_X_X_X_X_X__X_X_X_X_X__X
TILL HERE IT IS RESOLVED
_X_X_X_X_X_X_X_X_X_X_X_X_X_X_X_X_X__X_X_X
FURTHER ISSUE is SQL command is not working.
In the mailing table of my database I have email id with different domains.
For example, some id's are xyz#gmail.com, 12343#gmail.com , bae#yahoo.com as so on and I am assuming new email ids will be added to mailing tables every day with different domains.
I am trying to write a scripts that updates another table which holds a daily count of email addresses by their domain name. This is what I tried:
#!/usr/bin/perl
use warnings;
use strict;
use lib '/root/mysql/';
use 5.016;
use Data::Dumper;
use GUI::DB qw(dbConnect query);
my $data = dbConnect();
my #domain = query($data, "SELECT substr(addr,locate('\#',addr)+1) as maildomain, count (*) as mailcount FROM mailing GROUP BY maildomain ORDER BY mailcount DESC");
for my $key (#domain){
say Dumper ($key);
}
But I am getting an error,
You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '*) as mailcount FROM mailing GROUP BY maildomain ORDER BY mailcount DESC' at line 1 at /root/mysql/GUI/DB.pm line 44.
Same SQL statement works from CLI with no issues.
Any help would be appreciated. :)

1) Your error is saying that $dbh is undefined here:
sub query {
my $dbh = shift;
...
# issue query
my $sth = $dbh->prepare($sql); #<***LOOK HERE***
...which means $dbh must be undefined here:
sub query {
my $dbh = shift; #<***LOOK HERE***
...
# issue query
my $sth = $dbh->prepare($sql);
2) Let's see why. Your dbConnect() method returns $dbh:
sub dbConnect {
# Read database settings from config file:
print "Works";
my $dsn = "DBI:mysql:project";
my $dbh = DBI->connect(
$dsn,
'root',
'mydatabasepassword',
{ RaiseError => 1 }
);
return $dbh; #<***LOOK HERE*****
}
3) But, you call dbConnect() like this:
dbConnect();
Because you never save the return value anywhere, $dbh is discarded.
4) Furthermore, you call query() like this:
query();
Yet, you defined query() like this:
sub query {
my $dbh = shift;
The query() sub believes that the first argument will be the database handle--but you didn't call query() with any arguments.
You need to do this:
my $data_base_handle = dbConnect();
my #results = query($data_base_handle);
#do something with #results
Response to comment:
I printed #results, this is what I see HASH(0x1d05be8)
HASH(0x1d05ba0) HASH(0x1d05b58) HASH(0x1d05b10) HASH(0x1d05ac8)
HASH(0x1d05a80) HASH(0x1d05a38)
You wrote:
my $row = $sth->fetchrow_hashref;
...which asks DBI to return each row as a reference to a hash. Then you wrote:
push #returnData, $row;
...which pushed each hash reference into an array. So query() returns an array of hash references. The notation HASH(0x1d05be8) is what perl outputs when you print a hash reference.
If you want to see what's in those hashes, then do this:
use 5.016; #enable say()
use Data::Dumper;
...
...
for my $href (#results) {
say Dumper($href);
}
To access the data in a hash reference, you can do this:
use strict;
use warnings;
use 5.016;
use Data::Dumper;
my $href = {
c => 3,
a => 1,
b => 2,
};
my %hash = %{$href}; #dereference, {}, the reference into a hash, %
for my $key ( keys %hash ) {
say "$key $hash{$key}";
}
--output:--
c 3
a 1
b 2
Response to next comment:
(Answer posted in comments under op.)
By the way, perl is pretty good at text processing, so if you couldn't figure out the problem with your query, you could process the email addresses with perl:
use strict;
use warnings;
use 5.012;
use Data::Dumper;
use DBI;
use DBD::mysql;
# CONFIG VARIABLES
my $db_type = "mysql";
my $database = "my_db";
my $host = "localhost";
my $port = "3306";
my $user = "root";
my $pword = "";
# DATA SOURCE NAME
my $dsn = "dbi:$db_type:$database:$host:$port";
# PERL DBI CONNECT
my $dbh = DBI->connect($dsn, $user, $pword);
# PREPARE THE QUERY
my $tablename = "mailing";
my $select =<<"END_OF_SELECT";
select addr from $tablename
END_OF_SELECT
my $addr_aref = $dbh->selectcol_arrayref($select); #Returns a reference to a flat array containing all the email addresses
$dbh->disconnect;
my %count_for;
for my $addr (#{$addr_aref}) {
$addr =~ s/.*#//;
$count_for{$addr}++;
}
say Dumper(\%count_for);
--output:--
$VAR1 = {
'google.com' => 2,
'gorilla.com' => 1,
'yahoo.com' => 3
};

Related

Separate query in single var and execute it using perl script

I have currently working on perl script.
Select Column1,Column2,Column3.. from table.
This query contain some part in $cmd="Select Column1 ";
and other $cmd1=",Column2,Column3 from table"; // This is dynamic part, so split query in two different variable.
After this execute whole query.
How to do this query splitting part.?
use DBI;
use strict;
use warnings;
# Your input !
my $cmd = "Select Column1 ";
my $cmd1 = ",Column2,Column3 from table";
# I am wondering why you have your query like this ...
# but anyway, lets assume there's a reason behind this!
my $dbh =
DBI->connect(
'DBI:mysql:databasename;host=db.example.com', # TODO Change this
'username', # TODO change this
'password', # TODO change this
{ RaiseError => 1 }
) or die "Could not connect to database: $DBI::errstr";
my $sth = $dbh->prepare( $cmd . $cmd1 );
$sth->execute();
my #row;
while ( #row = $sth->fetchrow_array ) {
print "#row\n";
}

Perl DB interaction

I just started using Perl. I am able to connect to my MySQL database, create tables and get query results using my Perl Script. I came across a task that involves "You MUST use the provided DB.pm for all database interaction, and you must use it as it is (DB.pm cannot be modified except for the connection settings)."
What does that mean? Any one can guide me in the right direction ?
DB.pm file contains the following code
package GUI::DB;
use strict;
use DBI;
use vars qw(#ISA #EXPORT);
use Exporter;
#ISA = qw(Exporter);
#EXPORT = qw(dbConnect query);
#
# dbConnect - connect to the database, get the database handle
#
sub dbConnect {
# Read database settings from config file:
my $dsn = "DBI:mysql:database=test";
my $dbh = DBI->connect( $dsn,
'',
'',
{ RaiseError => 1 }
);
return $dbh;
}
#
# query - execute a query with parameters
# query($dbh, $sql, #bindValues)
#
sub query {
my $dbh = shift;
my $sql = shift;
my #bindValues = #_; # 0 or several parameters
my #returnData = ();
# issue query
my $sth = $dbh->prepare($sql);
if ( #bindValues ) {
$sth->execute(#bindValues);
} else {
$sth->execute();
}
if ( $sql =~ m/^select/i ) {
while ( my $row = $sth->fetchrow_hashref ) {
push #returnData, $row;
}
}
# finish the sql statement
$sth->finish();
return #returnData;
}
__END__
Probably, it means, that in your code you must use something like this:
use GUI::DB;
my $dbh = dbConnect();
my $sql = qq{SELECT * FROM my_table};
my #data = query($sql, $dbh);
You interact with the database through the provided module.

Unable to insert a record into MySQL database using DBI

I am trying to insert a record into a MySQL database using Perl DBI. I am not getting any errors but the insert is not working. However, I am able to successfully fetch records from the database using DBI.
Here is the code that does the insert:
#!"C:\xampp\perl\bin\perl.exe"
use diagnostics;
use DBI;
use strict;
use warnings;
my $driver = "mysql";
my $database = "mysql";
my $dsn = "DBI:$driver:database=$database";
my $userid = "root";
my $password = "password";
my $buffer;
my #pairs;
my $pair;
my $name;
my $value;
my %FORM;
# Read in text
my $ENV;
$ENV{'REQUEST_METHOD'} =~ tr/a-z/A-Z/;
if ($ENV{'REQUEST_METHOD'} eq "GET")
{
$buffer = $ENV{'QUERY_STRING'};
}
# Split information into name/value pairs
#pairs = split(/&/, $buffer);
foreach $pair (#pairs)
{
($name, $value) = split(/=/, $pair);
$value =~ tr/+/ /;
$value =~ s/%(..)/pack("C", hex($1))/eg;
$FORM{$name} = $value;
}
my $first_name= $FORM{name};
my $address = $FORM{address};
my $city = $FORM{city};
my $occupation = $FORM{occupation};
my $age = $FORM{age};
my $dbh = DBI->connect("dbi:mysql:dbname=mysql", "root", "password",{ AutoCommit => 0,RaiseError => 1}, ) or die ("Couldn't connect to database: ") , $DBI::errstr;
# my $sth = $dbh->prepare("INSERT INTO persons
# (FirstName, LastName,Address,City)
# values
# ($first_name, $last_name,$address,$city)");
my $query = "insert into userrecords(Address,Age,City,Name,Occupation)
values (?, ?, ?, ?, ?) ";
my $statement = $dbh->prepare($query) or die ("Couldn't connect to database: "), $DBI::errstr;
$statement->execute($address,$age,$city,$name,$occupation) or die ("Couldn't connect to database: "), $DBI::errstr;
$dbh->disconnect();
my $URL = "http://.....:81/cgi-bin/showdata.cgi";
print "Location: $URL\n\n";
exit(0);
When I run my code in the Padre IDE, I get the following errors:
****Error*********
Useless use of a variable in void context at InsertRecord.cgi line 50 (#1)
(W void) You did something without a side effect in a context that does
nothing with the return value, such as a statement that doesn't return a
value from a block, or the left side of a scalar comma operator. Very
often this points not to stupidity on your part, but a failure of Perl
to parse your program the way you thought it would. For example, you'd
get this if you mixed up your C precedence with Python precedence and
said
$one, $two = 1, 2;
when you meant to say
($one, $two) = (1, 2);
Another common error is to use ordinary parentheses to construct a list
reference when you should be using square or curly brackets, for
example, if you say
$array = (1,2);
when you should have said
$array = [1,2];
The square brackets explicitly turn a list value into a scalar value,
while parentheses do not. So when a parenthesized list is evaluated in
a scalar context, the comma is treated like C's comma operator, which
throws away the left argument, which is not what you want. See
perlref for more on this.
This warning will not be issued for numerical constants equal to 0 or 1
since they are often used in statements like
1 while sub_with_side_effects();
String constants that would normally evaluate to 0 or 1 are warned
about.
Useless use of a variable in void context at InsertRecord.cgi line 59 (#1)
Useless use of a variable in void context at InsertRecord.cgi line 60 (#1)
Use of uninitialized value in transliteration (tr///) at InsertRecord.cgi line
23 (#2)
(W uninitialized) An undefined value was used as if it were already
defined. It was interpreted as a "" or a 0, but maybe it was a mistake.
To suppress this warning assign a defined value to your variables.
To help you figure out what was undefined, perl will try to tell you the
name of the variable (if any) that was undefined. In some cases it cannot
do this, so it also tells you what operation you used the undefined value
in. Note, however, that perl optimizes your program and the operation
displayed in the warning may not necessarily appear literally in your
program. For example, "that $foo" is usually optimized into "that "
. $foo, and the warning will refer to the concatenation (.) operator,
even though there is no . in your program.
Use of uninitialized value $ENV{"REQUEST_METHOD"} in string eq at
InsertRecord.cgi line 24 (#2)
Use of uninitialized value $buffer in split at InsertRecord.cgi line 29 (#2)
Location: http://.......:81/cgi-bin/showdata.cgi
Press any key to continue . . .
***********END***********************
What is the issue?
When I was editing your code so that it was more readable, I stumbled upon what I assume is the solution:
You are using $name when inserting into the database, but you use $first_name when getting the value $FORM{name}. So since you used $name above, it has the value of the last name used, whatever that might be. The relevant code snippets:
($name, $value) = split(/=/, $pair);
...
$FORM{$name} = $value;
...
my $first_name = $FORM{name};
...
$statement->execute($address,$age,$city,$name,$occupation)
# ^^^^^--- should be $first_name
Your problem would have been solved if you had used proper scope on your variables, namely something like this:
foreach my $pair (#pairs) {
my ($name, $value) = split(/=/, $pair);
$value =~ tr/+/ /;
$value =~ s/%(..)/pack("C", hex($1))/eg;
$FORM{$name} = $value;
}
Then when you later would try to use $name, you would get the error
Global variable "$name" requires explicit package name ...
Which would alert you to your mistake and save you hours in debugging time. When you declare variables at the top of the script, instead of in the smallest possible scope, you effectively disable the protection that use strict 'vars' offers. So don't do that.
Also, you should probably use the CGI module instead of trying to handle it manually. It will make things easier, and safer. Don't forget to perform sanity checks on your data to prevent database injection attacks.
Your script when cleaned up and properly formatted looks like this.
What happens when you replace your code with this:
#!"C:\xampp\perl\bin\perl.exe"
use strict;
use warnings;
use diagnostics;
use DBI;
use CGI qw[param redirect];
my $driver = "mysql";
my $database = "mysql";
my $dsn = "DBI:$driver:database=$database";
my $userid = "root";
my $password = "password";
my $dbh = DBI->connect("dbi:mysql:dbname=mysql", "root", "password",
{ AutoCommit => 0,RaiseError => 1}, )
or die "Couldn't connect to database: ", $DBI::errstr;
my $query = "insert into userrecords(Address,Age,City,Name,Occupation)
values (?, ?, ?, ?, ?) ";
my $statement = $dbh->prepare($query)
or die "Couldn't connect to database: " , $DBI::errstr;
$statement->execute(param('address'), param('age'), param('city'),
param('name'), param('occupation'))
or die "Couldn't connect to database: " , $DBI::errstr;
$dbh->disconnect();
my $URL = "http://.....:81/cgi-bin/showdata.cgi";
print redirect($URL);
I've basically made two changes:
Use the CGI.pm module to handle the CGI interaction (getting the parameters and printing the redirection header).
Fixed your "void context" errors by removing the misplaced parentheses in all of your calls to die.
I'm made no substantive changes to the code, but at least we now have a clean version to go with.
Update: D'oh. It's obvious now the code is cleaned up a bit. If you have "Autocommit" turned off, then you need to commit your changes. Add $dbh->commit between the calls to execute() and disconnect().
The warning comes from this:
or die ("Couldn't connect to database: ") , $DBI::errstr;
The , $DBI::errstr is outside of the die and nothing is done with it, thus being in void context. You want something like this:
or die ("Couldn't connect to database: $DBI::errstr");
Also, your form handling code has some issues. If you're writing CGI scripts, you may as well use the CGI module. Here's a quick cleanup of your code:
#!"C:\xampp\perl\bin\perl.exe"
use diagnostics;
use CGI ':standard';
use DBI;
use strict;
use warnings;
my $driver = "mysql";
my $database = "mysql";
my $dsn = "DBI:$driver:database=$database";
my $userid = "root";
my $password = "password";
my $name = param('name');
my $address = param('address');
my $city = param('city');
my $occupation = param('occupation');
my $age = param('age');
my $dbh = DBI->connect( $dsn, $userid, $password,
{ AutoCommit => 1, RaiseError => 1 },
) or die("Couldn't connect to database: $DBI::errstr");
my $query = <<'END';
INSERT INTO userrecords(Address,Age,City,Name,Occupation)
VALUES ( ?, ?, ?, ?, ?)
END
my $statement = $dbh->prepare($query);
$statement->execute( $address, $age, $city, $name, $occupation );
$dbh->disconnect();
my $URL = "http://.....:81/cgi-bin/showdata.cgi";
print "Location: $URL\n\n";
Note that I've removed many or die statements because you already have RaiseError set to a true value.
For simplicity's sake, I've also (reluctantly) turned on AutoCommit.

Saving the output of a Perl SQL query to a hash instead of an array

I'm trying to add an argument to the end of a command line, run that search through a MySQL database, and then list the results or say that nothing was found. I'm trying to do it by saving the query data as both hashes and arrays (these are exercises, I'm extremely new at PERL and scripting and trying to learn). However, I can't figure out how to do the same thing with a hash. I do want the SQL query to complete, and then write the output to a hash, so as not to invoke the While function. Any guidance would be appreciated.
#!/usr/bin/perl -w
use warnings;
use DBI;
use Getopt::Std;
&function1;
&function2;
if ($arrayvalue != 0) {
print "No values found for '$search'"."\n"};
sub function1 {
getopt('s:');
$dbh = DBI->connect("dbi:mysql:dbname=database", "root", "password")
or die $DBI::errstr;
$search = $opt_s;
$sql = $dbh->selectall_arrayref(SELECT Player from Players_Sport where Sport like '$search'")
or die $DBI::errstr;
#array = map { $_->[0] } #$sql;
$dbh->disconnect
or warn "Disconnection failed": $DBI::errstr\n";
}
sub function2 {
#array;
$arrayvalue=();
print join("\n", #array, "\n");
if(scalar (#array) == 0) {
$arrayvalue = -1
}
else {$arrayvalue = 0;
};
}
Please see and read the DBI documentation on selectall_hashref. It returns a reference to a hash of reference to hashes.
Use Syntax:
$dbh->selectall_hashref($statement, $key_field[, \%attri][, #bind_values])
So here is an example of what/how it would be returned:
my $dbh = DBI->connect($dsn, $user, $pw) or die $DBI::errstr;
my $href = $dbh->selectall_hashref(q/SELECT col1, col2, col3
FROM table/, q/col1/);
Your returned structure would look like:
{
value1 => {
col1 => 'value1',
col2 => 'value2',
col3 => 'value3'
}
}
So you could do something as follows for accessing your hash references:
my $href = $dbh->selectall_hashref( q/SELECT Player FROM
Players_Sport/, q/Player/ );
# $_ is the value of Player
print "$_\n" for (keys %$href);
You can access each hash record individually by simply doing as so:
$href->{$_}->{Player}
Cribbing from the documentation:
$sql = $dbh->selectall_hashef("SELECT Player from Players_Sport where Sport like ?", 'Players_Sport_pkey', $sport_like_value);
my %hash_of_sql = %{$sql};

How do I call MySQL stored procedures from Perl?

How do I call MySQL stored procedures from Perl? Stored procedure functionality is fairly new to MySQL and the MySQL modules for Perl don't seem to have caught up yet.
MySQL stored procedures that produce datasets need you to use Perl DBD::mysql 4.001 or later. (http://www.perlmonks.org/?node_id=609098)
Below is a test program that will work in the newer version:
mysql> delimiter //
mysql> create procedure Foo(x int)
-> begin
-> select x*2;
-> end
-> //
perl -e 'use DBI; DBI->connect("dbi:mysql:database=bonk", "root", "")->prepare("call Foo(?)")->execute(21)'
But if you have too old a version of DBD::mysql, you get results like this:
DBD::mysql::st execute failed: PROCEDURE bonk.Foo can't return a result set in the given context at -e line 1.
You can install the newest DBD using CPAN.
There's an example in the section on Multiple result sets in the DBD::mysql docs.
#!/usr/bin/perl
# Stored Proc - Multiple Values In, Multiple Out
use strict;
use Data::Dumper;
use DBI;
my $dbh = DBI->connect('DBI:mysql:RTPC;host=db.server.com',
'user','password',{ RaiseError => 1 }) || die "$!\n";
my $sth = $dbh->prepare('CALL storedProcedure(?,?,?,?,#a,#b);');
$sth->bind_param(1, 2);
$sth->bind_param(2, 1003);
$sth->bind_param(3, 5000);
$sth->bind_param(4, 100);
$sth->execute();
my $response = $sth->fetchrow_hashref();
print Dumper $response . "\n";
It took me a while to figure it out, but I was able to get what I needed with the above. if you need to get multiple return "lines" I'm guessing you just...
while(my $response = $sth->fetchrow_hashref()) {
print Dumper $response . "\n";
}
I hope it helps.
First of all you should be probably connect through the DBI library and then you should use bind variables. E.g. something like:
#!/usr/bin/perl
#
use strict;
use DBI qw(:sql_types);
my $dbh = DBI->connect(
$ConnStr,
$User,
$Password,
{RaiseError => 1, AutoCommit => 0}
) || die "Database connection not made: $DBI::errstr";
my $sql = qq {CALL someProcedure(1);} }
my $sth = $dbh->prepare($sql);
eval {
$sth->bind_param(1, $argument, SQL_VARCHAR);
};
if ($#) {
warn "Database error: $DBI::errstr\n";
$dbh->rollback(); #just die if rollback is failing
}
$dbh->commit();
Mind you i haven't tested this, you'll have to lookup the exact syntax on CPAN.
Hi, similar to above but using SQL exec. I could not get the CALL command to work. You will need to fill in anything that is within square brackets and remove the square brackets.
use DBI;
#START: SET UP DATABASE AND CONNECT
my $host = '*[server]*\\*[database]*';
my $database = '*[table]*';
my $user = '*[user]*';
my $auth = '*[password]*';
my $dsn = "dbi:ODBC:Driver={SQL Server};Server=$host;Database=$database";
my $dbh = DBI->connect($dsn, $user, $auth, { RaiseError => 1 });
#END : SET UP DATABASE AND CONNECT
$sql = "exec *[stored procedure name]* *[param1]*,*[param2]*,*[param3]*;";
$sth = $dbh->prepare($sql);
$sth->execute or die "SQL Error: $DBI::errstr\n";