perl DBI, fastest way to get a single scalar value - mysql

I have this code to get a value count.
Short way:
my $count = $dbh->selectrow_array("SELECT COUNT(name) AS RESCOUNT FROM users");
Long way
my $sth = $dbh->prepare("SELECT COUNT(name) AS RESCOUNT FROM users");
$sth->execute() or die "$DBI::errstr";
my $count = $sth->fetchrow_array();
$sth->finish;
selectrow_array, fetchrow_array --> but I don't need an array. I checked the docs, but found nothing for scalars. Just methods for arrays and hashes.
The method I use is fast enough, but I was just curious if there is a better, fastest way to get a single value from the call. Or this is the fastest possible way?

The fastest way is to use fetchrow_arrayref or selectrow_arrayref, depending on how many executes you have. This only really makes a difference if executed in a loop and you have thousands (or rather hundreds of thousands) of rows.
When using fetchrow_array, it will make a copy every time, which slows you down. Also keep in mind that the behaviour for scalar context is only partly defined.
If called in a scalar context for a statement handle that has more than one column, it is undefined whether the driver will return the value of the first column or the last. So don't do that.
You can also do bind_col, which works with references.
There used to be a good presentation on DBI speeds from about 10 or more years ago that I can't find right now. Also take a look at this very old Perlmonks post, that explains quite a bit about performance.
Keep in mind that you should only do optimisation when you really know you need it. Most of the time you won't.

If "modern" means "I only heard of it recently", I'm feeling all modern with DBI's bind_col and bind_columns. Cribbing from a post by DBI hero Tim Bunce...
For your case:
my $sth = $dbh->prepare("SELECT COUNT(name) AS RESCOUNT FROM users");
my $count = 0;
$sth->bind_col(1,\$count); # bind to a reference to the variable
$sth->execute() or die "$DBI::errstr";
$sth->fetch;
print $count;
In a loop for a SELECT statement returning multiple records:
my $sth = $dbh->prepare(qq{SELECT name FROM users WHERE zip_code == '20500'});
my $name = '';
$sth->bind_col(1,\$name); # bind to a reference to the variable
$sth->execute() or die "$DBI::errstr";
while ($sth->fetch) {
print $name, "\n";
}
And with bind_columns this works:
my $sth = $dbh->prepare(qq{SELECT name,phone,address FROM users WHERE zip_code == '20500'});
my #fields = qw/name phone address/;
# With a 'SELECT All * ...', get all columns with #{$sth->{NAME_lc}}
my %data;
$sth->bind_columns( \( #data{#fields} ) ); # \(...) gives references to its elements
$sth->execute() or die "$DBI::errstr";
while ($sth->fetch) {
print "$data{name} lives at $data{address}, with phone $data{phone}.", "\n";
}
Once the setup is handled, the looping is simple to write and fast to run. (But, benchmark).
HTH, apologize if this diverges too much from the OP's problem statement. But it's the simplest and most direct way to get your returned data into the form of variable(s) you want, so you can move on to doing something with it...

Related

How can I grab multiple records from a MySQL query in Perl using array pointers?

I can do this all as one function, but in trying to port it over to my packages of functions (library) I am missing something.
Here's what I want to do from my main Perl script
my #rows;
$result = Funx::dbcdata($myConnection,
"SELECT * FROM Inv where name like \"%DOG%\";", \#rows);
Then in my library package I am attempting this
sub dbcdata
{
my ($connection, $command, $array) = #_;
my $query = $connection->prepare($command);
my $result = $query->execute();
my $i =0;
while(my $row = $query->fetchrow_arrayref() )
{
#{$array}[$i] = $row;
$i++;
}
$query->finish;
return $result;
}
I was hoping to get back pointers or references to each row (which was 4in this case) but am not. Every element in #rows is the same:
ARRAY(0x5577a0f77ec0) ARRAY(0x5577a0f77ec0) ARRAY(0x5577a0f77ec0)
ARRAY(0x5577a0f77ec0)
Nor do I know how to turn each one into the original separate row. Any help would be appreciated, thanks.
From the documentation for fetchrow_arrayref:
Note that the same array reference is returned for each fetch, so don't store the reference and then use it after a later fetch. Also, the elements of the array are also reused for each row, so take care if you want to take a reference to an element.
Sounds like you want fetchall_arrayref:
The fetchall_arrayref method can be used to fetch all the data to be returned from a prepared and executed statement handle. It returns a reference to an array that contains one reference per row.
After executing the statement, you can do something like
#{$array} = $query->fetchall_arrayref->#*;
instead of that ugly loop.
But selectall_array might be even better. Your whole function can be replaced by a call to it:
my #rows =
$myConnection->selectall_array(q/SELECT * FROM Inv WHERE name LIKE '%DOG%'/);

Simple perl mysql query not working

I've been out of the mysql and perl game for quite a few years and can't seem to get this right. I have a table with just 3 columns. 'cnt' is one of them. All I want to do is query the table on 'name' and see if name exists. If it does, I want to capture the value of 'cnt'. The table has a record of testName with a value of 2 I added manually. When this script is run it returns empty.
my $count;
my $pop = qq(SELECT cnt FROM popular WHERE name="testName");
my $sth = $dbh->prepare($pop);
$sth->execute() or die $dbh->errstr;
my #return;
while (#return = $sth->fetchrow_array()) {
$count = $return[1];
}
print "our return count is $count";
Is it obvious to anyone what I did wrong?
You probably mean
$count = $return[0];
According to perl doc on mysql
An alternative to fetchrow_arrayref. Fetches the next row of data and returns it as a list containing the field values.
Since you select cnt as the return value ,so , the size of #return is 1,but you misunderstand it as the number of results which meets your query condition.No, it is not so!Please have a more careful reading of perl doc.

Loop through query results without loading them all in array in Codeigniter [duplicate]

The normal result() method described in the documentation appears to load all records immediately. My application needs to load about 30,000 rows, and one at a time, submit them to a third-party search index API. Obviously loading everything into memory at once doesn't work well (errors out because of too much memory).
So my question is, how can I achieve the effect of the conventional MySQLi API method, in which you load one row at a time in a loop?
Here is something you can do.
while ($row = $result->_fetch_object()) {
$data = array(
'id' => $row->id
'some_value' => $row->some_field_name
);
// send row data to whatever api
$this->send_data_to_api($data);
}
This will get one row at the time. Check the CodeIgniter source code, and you will see that they will do this when you execute the result() method.
For those who want to save memory on large result-set:
Since CodeIgniter 3.0.0,
There is a unbuffered_row function,
All the methods above will load the whole result into memory (prefetching). Use unbuffered_row() for processing large result sets.
This method returns a single result row without prefetching the whole result in memory as row() does. If your query has more than one row, it returns the current row and moves the internal data pointer ahead.
$query = $this->db->query("YOUR QUERY");
while ($row = $query->unbuffered_row())
{
echo $row->title;
echo $row->name;
echo $row->body;
}
You can optionally pass ‘object’ (default) or ‘array’ in order to specify the returned value’s type:
$query->unbuffered_row(); // object
$query->unbuffered_row('object'); // object
$query->unbuffered_row('array'); // associative array
Official Document: https://www.codeigniter.com/userguide3/database/results.html#id2
Well, the thing is that result() gives away the entire reply of the query. row() simply fetches the first case and dumps the rest. However the query can still fetched 30 000 rows regardles of which function you use.
One design that would fit your cause would be:
$offset = (int)#$_GET['offset'];
$query = $this-db->query("SELECT * FROM table LIMIT ?, 1", array($offset));
$row = $query->row();
if ($row) {
/* Run api with values */
redirect(current_url().'?offset'.($offset + 1));
}
This would take one row, send it to api, update the page and use the next row. It will alos prevent the page from having a timeout. However it would most likely take a while with 30 000 records and refreshes, so you may wanna adjust your LIMIT ?, 1 to a higher number than 1 and go result() and foreach() multiple apis per pageload.
Well, there'se the row() method, which returns just one row as an object, or the row_array() method, which does the same but returns an array (of course).
So you could do something like
$sql = "SELECT * FROM yourtable";
$resultSet = $this->db->query($sql);
$total = $resultSet->num_rows();
for($i=0;$i<$total;$i++) {
$row = $resultSet->row_array($i);
}
This fetches in a loop each row from the whole result set.
Which is about the same as fetching everyting and looping over the $this->db->query($sql)->result() method calls I believe.
If you want a row at a time either you make 30.000 calls, or you select all the results and fetch them one at a time or you fetch all and walk over the array. I can't see any way out now.

number of parameters requested in a select statement

This is probably a somewhat simple question but I am trying to make sure that a query statement (specifically a select statement) contains a specific number of parameters only:
$result = mysql_query("select type,some_other_column from my_table");
$row = mysql_fetch_array($result);
$number = count($row);
print $number;
This returns twice the number I think it should return (4) - as I believe it must also be returning the key and the value as separate parameters.
The select statement above is just an example and it could be any number of statements. They could be a lot more complicated and the tests I have run do not seem to have any problems. I want to make sure that there are only ever two parameters (it can be any two) and they could be from different tables too.
I just want to make sure that it that what I am doing above is both the fastest way to check that the number of parameters is correct and that it won't get upset if there is a much more complicated statement given to it.
I am sure there is a really easy answer to this. Thanks in advance for any help.
Try mysql_fetch_assoc or mysql_fetch_row. Both functions available on php.net
mysql_fetch_array -- Fetch a result row as an associative array, a numeric array, or both. You end up having
$row["type"] = "somevalue"; // AND
$row[0] = "somevalue";
hence double the number
Whatever you SELECT would be in the $row variable, so in your code:
$result = mysql_query("select type,some_other_column from my_table");
$row = mysql_fetch_array($result);
/*
$row = array(
'type' => 'type_value',
'0' => 'type_value',
'some_other_column' => 'col_value',
'1' => 'col_value'
)
*/
$number = count($row);
print $number; // prints 4
I am not sure i understood your question right.
Do you just want to limit your number of returned values to one row?
If this is your point, you can add LIMIT 1 to your SQL-Query. This would, as it says, limit the number of results to one row.

How can I insert strings with quotes into Perl DBI queries?

What is the preferred way to insert strings that can contain both single and double quotes (",') into MySql using DBI? For example, $val1 and $val2 can contain quotes:
my $dbh = DBI->connect( ... );
my $sql = "insert into tbl_name(col_one,col_two) values($val1, $val2)";
my $sth = $dbh->prepare($sql);
$sth->execute();
Use a bound query using
$sth = $dbh->prepare("insert into tbl_name(col_one,col_two) values(?,?)");
$sth->execute($val1, $val2);
If you use bound variables, everything is escaped for you.
Update: Changed my example to correspond with the example edited into the question.
Update: I don't know why Adam deleted his answer, but if for some reason you can't use bound variables (aka "placeholders"), you can also use $dbh->quote($var) on the variable. For example:
$sql = sprintf "SELECT foo FROM bar WHERE baz = %s",
$dbh->quote(q("Don't"));
Use the quote() method. It will intelligently handle the quoting for you. Example from the docs:
$sql = sprintf "SELECT foo FROM bar WHERE baz = %s",
$dbh->quote("Don't");
Slightly modified to have both types of quotes:
$sql = sprintf "SELECT foo FROM bar WHERE baz = %s",
$dbh->quote(q("Don't"));
One small caveat on the bound placeholders, I build a rather large database-loading script that initially used bound placeholders in an older version of Perl/DBI and found what appears to be a memory leak in the placeholder implementation, so if you're looking at using them in a persistent process/daemon or in a high-volume context you may want to make sure process size doesn't become an issue. Switching over to building the query strings using the quote() method eliminated the issue for me.
DBI placeholders are awesome. They shine when you need to execute the same query in a loop. Consider this:
my $dbh = DBI->connect(...);
my $name_pairs = get_csv_data("data.csv");
my $sth = $dbh->prepare("INSERT INTO t1 (first_name, last_name) VALUES (?,?)");
for my $pair (#$name_pairs) {
unless ($sth->execute(#$pair)) {
warn($sth->errstr);
}
}
In this case, having the prepared statement handle is, er, handy.
However, barring this sort of tight-loop cases, I like to see the actual statement that was sent to the server. This is where I lean heavily on quote and frankly sprintf.
# Here, I am confident about the hash keys, less so about the values
$sql = sprintf("INSERT INTO t1 (%s) VALUES (%s)",
join(",", keys(%hash)),
join("," map { $dbh->quote($_) } values(%hash))
);
$sth = $dbh->prepare($sql);
unless ($sth->execute) {
warn($sth->{Statement});
}
Note that you do have to set RaiseError => 0 on $dbh so that you can see the SQL that failed, but this has helped me a great deal in the past.
Cheers.