MySQL query 2 tables, 1 insert - mysql

I'm trying to pull data from 2 tables and then insert result into a third table. My code follows but only does 1 correct entry, and the rest are blank. There are 348 entries total. What am I missing here?
$dbh = DBI->connect(
"DBI:mysql:$mysqldatabase:$mysqlhostname",
"$mysqlusername",
"$mysqlpassword"
);
if(!$dbh) { die("Error: could not get DBI handle\n"); }
$sqlConnect = 1;
$SQL =<<SQL;
SELECT * FROM oscmax2.info2
SQL
$sth = $dbh->prepare($SQL);
if(!$sth) { die("Error: " . $dbh->errstr . "\n"); }
if(!$sth->execute) { die("Error4: " . $sth->errstr . "\n"); }
while (my #row = $sth->fetchrow_array) {
$products_id = $FORM{'product_id'};
$affiliate_id = $FORM{'affiliate_id'};
$demo = $FORM{'demo'};
}
if($sth->rows != 0) {
$total_rows = $sth->rows;
for ($counter = 0; $counter < $total_rows; $counter++) {
$SQL =<<SQL;
SELECT products_attributes_id FROM oscmax2.products_attributes
WHERE products_id = '$products_id'
SQL
$sth = $dbh->prepare($SQL);
if(!$sth) { die("Error: " . $dbh->errstr . "\n"); }
if(!$sth->execute) { die("Error: " . $sth->errstr . "\n");}
while (my #row = $sth->fetchrow_array) {
$products_attributes_id = $FORM2{'products_attributes_id'};
}
$SQL =<<SQL;
INSERT INTO oscmax2.products_attributes_download(
products_attributes_id, products_attributes_filename,
products_attributes_maxdays, products_attributes_maxcount
)
VALUES
('$products_attributes_id', '$affiliate_id/$demo', '7', '1')
SQL
$dbh->do($SQL) || die("Error5: " . $dbh->errstr . "\n");
}
}
$sth->finish();
if($sqlConnect) { $dbh->disconnect();

The blocks
while (my #row = $sth->fetchrow_array) {
$products_id = $FORM{'product_id'};
$affiliate_id = $FORM{'affiliate_id'};
$demo = $FORM{'demo'};
}
and
while (my #row = $sth->fetchrow_array) {
$products_attributes_id = $FORM2{'products_attributes_id'};
}
are wrong. You use #row to accept the data from each row of the result, but never use it. The database fetch won't affect %FORM and %FORM2 so you are just collecting the same data from them several times over
Update
At a guess you want something like this. Please study it and use the techniques rather than copying and testing it as it is, as I have no way of knowing what the structure of your database is and I have made several guesses
You should note the following points
There is no need to test the status of each DBI operation and die if it failed. By default the PrintError option is enabled and DBI will raise a warning if there are any errors. If you want your program to die instead, which is wise, then you can enable RaiseError and disable PrintError and DBI will do it all for you
There is no need to fetch all of the data from a table into memory (which I think is what you are trying to do with your while loops. You should fetch each row into an array and process the data row by row unless you have a reason to do otherwise
You should always prepare your statement and use placeholders. Then you can pass the actual parameters to the execute call and DBI will correctly quote them for you. Furthermore you can move all the prepare calls top the top of the program, making your logic much clearer to read
There is almost never a reason to call finish or disconnect. Perl will do the right thing for you when your database or statement handles go out of scope or your program ends
I have named the statement handles $select1 and $select2. These are very poor names, but I don't know the structure of your database so I couldn't write anything better. That shouyldn't stop you from improving them
I have had to guess at the columns returned by the first SELECT statement. If the three variables don't correspond to the first three elements of #row then you need to correct that
You should avoid using capital letters in Perl lexical identifiers. They are reserved for globals like package names, and nasty clashes of purpose can be caused if you don't abide by this rule
use strict;
use warnings;
my ($mysqldatabase, $mysqlhostname, $mysqlusername, $mysqlpassword) = qw/ dbase host user pass /;
my $dbh = DBI->connect(
"DBI:mysql:$mysqldatabase:$mysqlhostname",
"$mysqlusername",
"$mysqlpassword",
{RaiseError => 1, PrintError => 0}
) or die "Unable to connect to database: $DBI::errstr";
my $select1 = $dbh->prepare('SELECT * FROM oscmax2.info2');
my $select2 = $dbh->prepare(<<__END_SQL__);
SELECT products_attributes_id FROM oscmax2.products_attributes\
WHERE products_id = ?
__END_SQL__
my $insert = $dbh->prepare(<<__END_SQL__);
INSERT INTO oscmax2.products_attributes_download (
products_attributes_id,
products_attributes_filename,
products_attributes_maxdays,
products_attributes_maxcount
)
VALUES ( ?, ?, ?, ? )
__END_SQL__
$select1->execute;
while ( my #row = $select1->fetchrow_array ) {
my ($products_id, $affiliate_id, $demo) = #row;
$select2->execute($products_id);
while ( my #row = $select2->fetchrow_array ) {
my ($products_attributes_id) = #row;
$insert->execute($products_attributes_id, "$affiliate_id/$demo", 7, 1 );
}
}

Related

Separate query in single var and execute it using perl script

I have currently working on perl script.
Select Column1,Column2,Column3.. from table.
This query contain some part in $cmd="Select Column1 ";
and other $cmd1=",Column2,Column3 from table"; // This is dynamic part, so split query in two different variable.
After this execute whole query.
How to do this query splitting part.?
use DBI;
use strict;
use warnings;
# Your input !
my $cmd = "Select Column1 ";
my $cmd1 = ",Column2,Column3 from table";
# I am wondering why you have your query like this ...
# but anyway, lets assume there's a reason behind this!
my $dbh =
DBI->connect(
'DBI:mysql:databasename;host=db.example.com', # TODO Change this
'username', # TODO change this
'password', # TODO change this
{ RaiseError => 1 }
) or die "Could not connect to database: $DBI::errstr";
my $sth = $dbh->prepare( $cmd . $cmd1 );
$sth->execute();
my #row;
while ( #row = $sth->fetchrow_array ) {
print "#row\n";
}

Perl: Breaking out of foreach loop when last array element is encountered

Perl noob here. I have a small script (see below) that I'm using to build a MySQL INSERT statement.
use strict;
my #records = qw/Record1 Record2 Record3/;
my $insert = "
INSERT INTO table
VALUES
";
foreach my $record (#records) {
$insert .= "('" . $record . "'),\n ";
}
print "$insert\n";
Current Output
INSERT INTO table
VALUES
('Record1'),
('Record2'),
('Record3'),
I need to know how to break at the last element of the #records array and append a ; instead of ,
Desired Output
INSERT INTO table
VALUES
('Record1'),
('Record2'),
('Record3');
You can do that with map and join.
my #records = qw/Record1 Record2 Record3/;
my $insert = "
INSERT INTO table
VALUES
";
$insert .= join ',', map { '(' . $dbh->quote($_) . ')' } #records;
$insert .= ';'; # this line is not needed
The quote method of $dbh is better than just putting the stuff into quotes because it handles bad stuff for you. The map is not much different from a foreach loop, and the join will take care of putting the commas in between the elements, and not the last one.
On a related matter, I always try to avoid putting data and sql statements on the same line, thus minimize the risk of sql injection. In perl you have a prepare/execute mechanism available for this:
my #records = qw/Record1 Record2 Record3/;
$sth = $dbh->prepare("INSERT INTO table VALUES ?");
foreach my $record (#records) {
$sth->execute($record);
}
http://bobby-tables.com/perl.html

Update MySQL within Perl loop failing (fetchrow_array)

I've created a Perl script which is meant to loop through an array (a shortlist of customers who meet certain criteria), execute an external command using system() , then update a field within each row once the operation has completed.
It works on the first record (ie external command executes, customer record updates), however when it gets to the second record I receive this error:
DBD::mysql::st fetchrow_array failed: fetch() without execute() at customer_update.pl
Through some googling I added the $sth->finish(); command, however whether I include it or not (either inside the loop as shown, or straight afterward) I still get the same error.
Can anyone shed any light for me as to what I am doing wrong here?
Here's an extract:
# PERL MYSQL CONNECT()
$dbh = DBI->connect('dbi:mysql:signups', $user, $pw)
or die "Connection Error: $DBI::errstr\n";
# DEFINE A MySQL QUERY
$myquery = "SELECT * FROM accounts WHERE field3 = false";
$sth = $dbh->prepare($myquery);
# EXECUTE THE QUERY
$sth->execute
or die "SQL Error: $DBI::errstr\n";
#records = $sth->rows;
print "Amount of new customers: #records\n\n";
while ( my ($field1, $field2, $field3) = $sth->fetchrow_array() ) {
#execute external command via system();
$update_customer_status = "UPDATE accounts SET field3=true WHERE id=$id";
$sth = $dbh->prepare($update_customer_status);
$sth->execute
or die "SQL Error: $DBI::errstr\n";
print "Customer record modified & MySQL updated accordingly\n\n";
$sth->finish();
}
Building a SQL statement with variables and then prepare()ing it defeats the purpose of the prepare. You should build the SQL statement with a placeholder ? instead of $id, prepare() it, and then execute($id) it. As it is, you are leaving yourself open to SQL injection attacks.
Also, it seems that you are not using the warnings and strict pragmas. These two lines should be at the top of every program you write:
use warnings;
use strict;
They will save you much heartache and frustration in the future.
In your loop, you overwrite the handle over from which you are fetching. Use a different variable. (Changing $sth = ...; to my $sth = ...; will do.) While we're at it, let's move the prepare out of the loop.
my $sth_get = $dbh->prepare("SELECT * FROM accounts WHERE field3 = false");
my $sth_upd = $dbh->prepare("UPDATE accounts SET field3=true WHERE id = ?");
$sth_get->execute();
while ( my ($field1, $field2, $field3) = $sth_get->fetchrow_array() ) {
...
$sth_upd->execute($id);
}
You are stomping on your $sth variable when you execute this line ...
$sth = $dbh->prepare($update_customer_status);
Why not save off the result of $sth->fetchrow_array() to an array variable.
Something like ...
my #select_results_AoA = $sth->fetchrow_array();
... and then iterate over the array ...
for my #row ( #select_resilts_AoA ) {
... instead of ...
while ( my ($field1, $field2, $field3) = $sth->fetchrow_array() ) {

Saving the output of a Perl SQL query to a hash instead of an array

I'm trying to add an argument to the end of a command line, run that search through a MySQL database, and then list the results or say that nothing was found. I'm trying to do it by saving the query data as both hashes and arrays (these are exercises, I'm extremely new at PERL and scripting and trying to learn). However, I can't figure out how to do the same thing with a hash. I do want the SQL query to complete, and then write the output to a hash, so as not to invoke the While function. Any guidance would be appreciated.
#!/usr/bin/perl -w
use warnings;
use DBI;
use Getopt::Std;
&function1;
&function2;
if ($arrayvalue != 0) {
print "No values found for '$search'"."\n"};
sub function1 {
getopt('s:');
$dbh = DBI->connect("dbi:mysql:dbname=database", "root", "password")
or die $DBI::errstr;
$search = $opt_s;
$sql = $dbh->selectall_arrayref(SELECT Player from Players_Sport where Sport like '$search'")
or die $DBI::errstr;
#array = map { $_->[0] } #$sql;
$dbh->disconnect
or warn "Disconnection failed": $DBI::errstr\n";
}
sub function2 {
#array;
$arrayvalue=();
print join("\n", #array, "\n");
if(scalar (#array) == 0) {
$arrayvalue = -1
}
else {$arrayvalue = 0;
};
}
Please see and read the DBI documentation on selectall_hashref. It returns a reference to a hash of reference to hashes.
Use Syntax:
$dbh->selectall_hashref($statement, $key_field[, \%attri][, #bind_values])
So here is an example of what/how it would be returned:
my $dbh = DBI->connect($dsn, $user, $pw) or die $DBI::errstr;
my $href = $dbh->selectall_hashref(q/SELECT col1, col2, col3
FROM table/, q/col1/);
Your returned structure would look like:
{
value1 => {
col1 => 'value1',
col2 => 'value2',
col3 => 'value3'
}
}
So you could do something as follows for accessing your hash references:
my $href = $dbh->selectall_hashref( q/SELECT Player FROM
Players_Sport/, q/Player/ );
# $_ is the value of Player
print "$_\n" for (keys %$href);
You can access each hash record individually by simply doing as so:
$href->{$_}->{Player}
Cribbing from the documentation:
$sql = $dbh->selectall_hashef("SELECT Player from Players_Sport where Sport like ?", 'Players_Sport_pkey', $sport_like_value);
my %hash_of_sql = %{$sql};

How can I re-use WHERE clause logic with DBI?

Disclaimer: first time I've used DBI.
I have a MySQL table with a lot of indexed fields (f1, f2, f3, etc) that are used to generate WHERE clauses by long-running processes that iterate over chunks of the database performing various cleaning and testing operations.
The current version of this code works something like this:
sub get_list_of_ids() {
my ($value1, $value2, $value3...) = #_;
my $stmt = 'SELECT * FROM files WHERE 1';
my #args;
if (defined($value1)) {
$stmt .= ' AND f1 = ?';
push(#args, $value1);
}
# Repeat for all the different fields and values
my $select_sth = $dbh->prepare($stmt) or die $dbh->errstr;
$select_sth->execute(#args) or die $select_sth->errstr;
my #result;
while (my $array = $select_sth->fetch) {
push(#result, $$array[0]);
}
return \#result;
}
sub function_A() {
my ($value1, $value2, $value3...) = #_;
my $id_aref = get_list_of_ids($value1, $value2, $value3...);
foreach my $id (#$id_aref) {
# Do something with $id
# And something else with $id
}
}
sub function_B() {
my ($value1, $value2, $value3...) = #_;
my $id_aref = get_list_of_ids($value1, $value2, $value3...);
foreach my $id (#$id_aref) {
# Do something different with $id
# Maybe even delete the row
}
}
Anyway, I'm about to dump an awful lot more rows in the database, and am well aware that the code above wont scale up. I can think of several ways to fix it based on other languages. What is the best way to handle it in Perl?
Key points to note are that the logic in get_list_of_ids() is too long to replicate in each function; and that the operations on the selected rows are very varied.
Thanks in advance.
I presume by "scale up" you mean in maintenance terms rather than performance.
The key change to your code is to pass in your arguments as column/value pairs rather than a list of values with an assumed set of columns. This will allow your code to handle any new columns you might add.
DBI->selectcol_arrayref is both convenient and a bit faster, being written in C.
If you turn on RaiseError in your connect call, DBI will throw an exception on errors rather than having to write or die ... all the time. You should do that.
Finally, since we're writing SQL from possibly untrusted user input, I've taken care to escape the column name.
The rest is explained in this Etherpad, you can watch your code be transformed step by step.
sub get_ids {
my %search = #_;
my $sql = 'SELECT id FROM files';
if( keys %search ) {
$sql .= " WHERE ";
$sql .= join " AND ", map { "$_ = ?" }
map { $dbh->quote_identifier($_) }
keys %search;
}
return $dbh->selectcol_arrayref($sql, undef, values %search);
}
my $ids = get_ids( foo => 42, bar => 23 );
If you expect get_ids to return a huge list, too much to keep in memory, then instead of pulling out the whole array and storing it in memory you can return the statement handle and iterate with that.
sub get_ids {
my %search = #_;
my $sql = 'SELECT id FROM files';
if( keys %search ) {
$sql .= " WHERE ";
$sql .= join " AND ", map { "$_ = ?" }
map { $dbh->quote_identifier($_) }
keys %search;
}
my $sth = $dbh->prepare($sql);
$sth->execute(values %search);
return $sth;
}
my $sth = get_ids( foo => 42, bar => 23 );
while( my $id = $sth->fetch ) {
...
}
You can combine both approaches by returning a list of IDs in array context, or a statement handle in scalar.
sub get_ids {
my %search = #_;
my $sql = 'SELECT id FROM files';
if( keys %search ) {
$sql .= " WHERE ";
$sql .= join " AND ", map { "$_ = ?" }
map { $dbh->quote_identifier($_) }
keys %search;
}
# Convenient for small lists.
if( wantarray ) {
my $ids = $dbh->selectcol_arrayref($sql, undef, values %search);
return #$ids;
}
# Efficient for large ones.
else {
my $sth = $dbh->prepare($sql);
$sth->execute(values %search);
return $sth;
}
}
my $sth = get_ids( foo => 42, bar => 23 );
while( my $id = $sth->fetch ) {
...
}
my #ids = get_ids( baz => 99 );
Eventually you will want to stop hand coding SQL and use an Object Relation Mapper (ORM) such as DBIx::Class. One of the major advantages of an ORM is it is very flexible and can do the above for you. DBIx::Class can return a simple list of results, or very powerful iterator. The iterator is lazy, it will not perform the query until you start fetching rows, allowing you to change the query as needed without having to complicate your fetch routine.
my $ids = get_ids( foo => 23, bar => 42 );
$ids->rows(20)->all; # equivalent to adding LIMIT 20