Perl DBI / MySQL - can't for specific float values - mysql

I've got a problem with my perl and mysql code.
I have a database with a table "STORAGEDATA". In this table I have the column "CLOSETIME" with the datatype "double", which stores an epoch timestamp as a float value (the floating part represents milliseconds).
I now have the problem, that I can't find some of those values with a SELECT-Statement.
MySQL-Example:
SELECT * FROM STORAGEDATA WHERE CLOSETIME = 1360021730.666;
-- This will find the data I am looking for
SELECT * FROM STORAGEDATA WHERE CLOSETIME = '1360021730.666';
-- This works, too
SELECT * FROM STORAGEDATA WHERE CLOSETIME = 1360209405.574;
-- This will find my data, too
SELECT * FROM STORAGEDATA WHERE CLOSETIME = '1360209405.574';
-- But this does not find anything.
The searched values definitly exists in the table and are exactly saved as the values shown above (no trailing zeros or something).
Now I've got the problem that I don't know how to fetch the data using perl dbi.
Perl-Example:
my $sql = 'SELECT * FROM STORAGEDATA WHERE CLOSETIME = ?';
my #args = (1360209405.574);
# $dbh is a database-handler, providing the DBI-functionality
my $sth = $dbh->prepare($sql);
my #data;
if ($sth) {
$sth->execute(#args);
while ( my $dataset = $sth->fetchrow_hashref() ) { push #data, $dataset; }
}
This won't find anything, it seems like DBI inserts the arguments as strings. When I put my quittime directly into the code, it works:
my $sql = 'SELECT * FROM STORAGEDATA WHERE CLOSETIME = 1360209405.574';
my #args = ();
# $dbh is a database-handler, providing the DBI-functionality
my $sth = $dbh->prepare($sql);
my #data;
if ($sth) {
$sth->execute(#args);
while ( my $dataset = $sth->fetchrow_hashref() ) { push #data, $dataset; }
}
This will work, but that's not what I want to do. When I use DBIx, the problem is still the same. I could use "LIKE" instead of "=" to compare, but this would extremely slow down the query.
I tried to change the datatype for CLOSETIME to "Decimal(13,3)", but this didn't change anything.
I run MySQL in version 5.1.73.
Do you have any suggestions how I can solve this problem?

Solution
Either upgrade MySQL, or explicitly specify your predicate's type in the SQL itself:
SELECT * FROM STORAGEDATA WHERE CLOSETIME = CAST(? AS DECIMAL(13,3))
^^^^^^^^^^^^^^^^^^^^^^^^
Rationale
To my thinking, you show us the heart of the matter right away:
SELECT * FROM STORAGEDATA WHERE CLOSETIME = 1360209405.574;
-- This will find my data, too
SELECT * FROM STORAGEDATA WHERE CLOSETIME = '1360209405.574';
-- But this does not find anything.
Huh? That's not what I'd expect. This appears to be a bug in MySQL 5.1, remedied in at least 5.5 (possibly earlier). Smallest demonstration I could think of:
SELECT VERSION(), '1360209405.574' = 1360209405.574 AS "str-num comparison";
-- Under 5.5, TRUE: http://sqlfiddle.com/#!2/e705e6/1
SELECT VERSION(), '1360209405.574' = 1360209405.574 AS "str-num comparison";
-- Under 5.1, FALSE: http://sqlfiddle.com/#!8/e705e/1
Under 5.1, explicitly CASTing to DECIMAL worked for me.

try turning on bind type guessing
my $dbh= DBI->connect('DBI:mysql:test', 'username', 'pass',
{ mysql_bind_type_guessing => 1})
- OR after handle creation
$dbh->{mysql_bind_type_guessing} = 1;

You can use bind_param: https://metacpan.org/pod/DBI#bind_param
$sth->bind_param(1, $args[0], SQL_DOUBLE);
$sth->execute;

Related

how to get latest date from two conditions?

I have drop down where the drop down is list of namaJabatan
my table - infojawatan
ID - PK of the table
namaJabatan - where the condition appear ($search - its up to where the user select from Dropdown)
tarikhKemaskini - where i want to get the latest date of row
my query
$sql = "SELECT * FROM infojawatan WHERE namaJabatan = '$search' && tarikh Kemaskini IN (SELECT MAX(tarikhKemaskini) FROM infojawatan GROUP BY ID)";
$sql_rs = mysql_query($sql);
while($row_Sql = mysql_fetch_array($sql_rs)) {
$tarikhKemaskini = $row_Sql['tarikhKemaskini'];
}
echo "Current Date :" .$tarikhKemaskini;
You have a few syntax errors in your SQL.
SQL spells out AND, not &&.
tarikhKemaskini is one word.
SELECT *
FROM infojawatan
WHERE namaJabatan = :namaJabatan AND
tarikhKemaskini IN (
SELECT MAX(tarikhKemaskini)
FROM infojawatan
GROUP BY ID
)
Note carefully that I used :namaJabatan there instead of hard coding $search. Hard coding variables into SQL leaves you open to a SQL Injection Attack where a malicious attacker can craft a search query that lets them get more information than they're allowed to, or even run arbitrary SQL queries.
Instead, use parameters, the :namaJabatan there, and pass your variables in when you execute the query.
Unfortunately the mysql_query interface doesn't support this. Fortunately it was deprecated and there are now better interfaces. Here's a breakdown. I'd recommend using PDO as it is a generic interface applicable to any SQL database. Then you can use the more secure and efficient prepared statements with bind parameters.
$stmt = $dbh->prepare("
SELECT *
FROM infojawatan
WHERE namaJabatan = :namaJabatan AND
tarikhKemaskini IN (
SELECT MAX(tarikhKemaskini)
FROM infojawatan
GROUP BY ID
)
")
$stmt->execute(array( ':namaJabatan' => $search));
while( $row = $stmt->fetch() ) {
echo $row['tarikhKemaskini'];
}

SQL: Binding an undef (NULL) to the placeholder will not select rows which have a NULL value

I'm using perl 5.20 and MySQL 5.7, but I think the question is about SQL in general:
perldoc DBI says:
Binding an undef (NULL) to the placeholder will not select rows which have a NULL age! At least for database engines that conform to the SQL standard. Refer to the SQL manual for your database engine or any SQL book for the reasons for this. To explicitly select NULLs you have to say "WHERE age IS NULL".
I don't even know what to google for... My question is: What are the reasons behind = ? not matching a binding to NULL/undef? (Beyond "that is how it is defined and documented".)
I've discovered that MySQL has an operator <=> that allows comparisons with NULL and so:
my $sth = $dbh->prepare('select count(*) from table where field <=> ?');
$sth->execute(345);
$sth->execute(undef);
both work as expected. Unfortunately, the doc says:
The <=> operator is equivalent to the standard SQL IS NOT DISTINCT FROM operator.
And MySQL doesn't support the IS NOT DISTINCT FROM operator :-(. So there seems to be no portable way to do this. Except for the very hackish:
my $sth = $dbh->prepare('
select count(*) from table
where field = ? OR ( ? IS NULL AND field IS NULL )
');
$sth->execute(345, 345);
$sth->execute(undef, undef);
or the even more hackish snippet from perldoc DBI
$sql_clause = defined $age? "age = ?" : "age IS NULL";
$sth = $dbh->prepare(qq{
SELECT fullname FROM people WHERE $sql_clause
});
$sth->execute(defined $age ? $age : ());
Is there a portable way to do WHERE FIELD = ? and have it do what I mean also with NULLs/undefs? What did I miss?
EDIT: I also came up with this workaround, which especially works great if field is a numeric type such as an INT, so we're sure the string "NULL" is not a possible non-NULL value.
my $sth = $dbh->prepare('
select count(*) from table
where COALESCE(field, "NULL") = COALESCE(?,"NULL")
');
$sth->execute(345);
$sth->execute(undef);
But performance goes out the window, as I don't think any indexes can be used....
I understand that this is not exactly what you asked for but if you use DBIx::Class, the ORM will do that lifting for you.
my $res = $schema->resultset('table')->search({ field => [345, undef] });
print $res->count;
It will be translated to this SELECT COUNT( * ) FROM table me WHERE ( ( field = ? OR field IS NULL ) ): '345'

MySql auto_increment plus a certain number

I want my the id field in my table to be a bit more " random" then consecutive numbers.
Is there a way to insert something into the id field, like a +9, which will tell the db to take the current auto_increment value and add 9 to it?
Though this is generally used to solve replication issues, you can set an increment value for auto_increment:
auto_increment_increment
Since that is both a session and a global setting, you could simply set the session variable just prior to the insert.
Besides that, you can manually do it by getting the current value with MAX() then add any number you want and insert that value. MySQL will let you know if you try to insert a duplicate value.
You have a design flaw. Leave the auto increment alone and shuffle your query result (when you fetch your data)
As far as i know, it's not possible to 'shuffle' your current IDs. If you wanted though, you could pursue non-linear IDs in the future.
The following is written in PDO, there are mysqli equivalents.
This is just an arbitrary INSERT statement
$name = "Jack";
$conn = new PDO("mysql:host=$dbhost;dbname=$dbname",$dbuser,$dbpass);
$sql = "INSERT INTO tableName (name) VALUES(:name)";
$q = $conn->prepare($sql);
$q->execute(':name' => $name);
Next, we use lastInsertId() to return the ID of the last inserted row, then we concatenate the result to rand()
$lastID = $conn->lastInsertId();
$randomizer = $lastID.rand();
Finally, we use our 'shuffled' ID and UPDATE the previously inserted record.
$sql = "UPDATE tableName SET ID = :randomizer WHERE ID=:lastID ";
$q = $conn->prepare($sql);
$q->execute(array(':lastID' => $lastID , ':randomizer' => $randomizer));
An idea.. (Not tested)
CREATE TRIGGER 'updateMyAutoIncrement'
BEFORE INSERT
ON 'DatabaseName'.'TableName'
FOR EACH ROW
BEGIN
DECLARE aTmpValueHolder INT DEFAULT 0;
SELECT AUTO_INCREMENT INTO aTmpValueHolder
FROM INFORMATION_SCHEMA.TABLES
WHERE TABLE_SCHEMA = 'DatabaseName'
AND TABLE_NAME = 'TableName';
SET NEW.idColumnName =aTmpValueHolder + 9;
END;
Edit : If the above trigger doesn't work try to update AUTO_INCREMENT value directly into the system's schema. But as noted by Eric, your design seems to be flawed. I don't see the point of having an auto-increment here.
Edit 2 : For a more 'random' and less linear number.
SET NEW.idColumnName =aTmpValueHolder + RAND(10);
Edit 3 : As pointed out by Jack Williams, Rand() produces a float value between 0 and 1.
So instead, to produce an integer, we need to use a floor function to transform the 'random' float into an integer.
SET NEW.idColumnName =aTmpValueHolder + FLOOR(a + RAND() * (b - a));
where a and b are the range of the random number.

MySQL: How can fetch SUM() of all fields in one Query?

I just want somthing like this:
select SUM(*) from `mytable` group by `year`
any suggestion?
(I am using Zend Framework; if you have a suggestion using ZF rather than pure query would be great!)
Update: I have a mass of columns in table and i do not want to write their name down one by one.
No Idea??
SELECT SUM(column1) + SUM(column2) + SUM(columnN)
FROM mytable
GROUP BY year
Using the Zend Framework's Zend_Db_Select, your query might look like
$db = Zend_Db::factory( ...options... );
$select = $db->select()
->from('mytable', array('sum1' => 'SUM(`col1`)', 'sum2' => 'SUM(col2)')
->group('year');
$stmt = $select->query();
$result = $stmt->fetchAll();
Refer to the Zend_Db_Select documentation in the ZF manual for more.
EDIT: My bad, I think I misunderstood your question. The query above will return each colum summed, but not the sum of all of the columns. Rewriting Maxem's query so that you can use it with a Zend Framework DB adapter, it might look like
$sql = '<insert Maxem's query here>';
$result = $db->fetchAll($sql);
You might choose to use fetchCol() to retrieve the single result.
It sounds like you don't want to explicitly enumerate the columnn and that you want to sum all the columns (probably excluding the year column) over all the rows, with grouping by year.
Note that the method Zend_Db_Table::info(Zend_Db_Table_Abstract::COLS) will return an array containing the columns names for the underlying table. You could build your query using that array, something like the following:
Zend_Db_Table::setDefaultAdapter($db);
$table = new Zend_Db_Table('mytable');
$fields = $table->info(Zend_Db_Table_Abstract::COLS);
unset($fields['year']);
$select = $table->select();
$cols = array();
foreach ($fields as $field){
$cols[] = sprintf('SUM(%s)', $field);
}
$select->cols(implode(' + ', $cols));
$select->group('year');
I have not tested the specific syntax, but the core of the idea is the call to info() to get the fields dynamically.
Done in ZF rather than pure query and you don't have to write the name of the columns one by one.
(I assume you are extending Zend_Db_Table_Abstract)
If you're asking how to write
select SUM(*) from `mytable` group by `year`
This is how it is done:
public function sumOfAllFields(){
return $this->fetchAll( $this->select()->from('mytable','SUM(*)')->group('year') )->toArray();
}
Or not using Zend...
function mysql_cols($table){
$sql="SHOW COLUMNS FROM `".$table."`";
$res=mysql_query($sql);
$cols=array();
while($row=mysql_fetch_assoc($res))$cols[]=$row['Field'];
return $cols;
}
$cols=mysql_cols("mytable");
$select_sql=array();
foreach($cols as $col){
$select_sql[]="SUM(`".$col."`)";
}
$select_sql=implode('+',$select_sql);
$sql="select (".$select_sql.") from `mytable` group by `year`";

MYSQL: How to copy an entire row from one table to another in mysql with the second table having one extra column?

I have two tables with identical structure except for one column... Table 2 has an additional column in which I would insert the CURRENT_DATE()
I would like to copy all the values from table1 to table2.
If I use
INSERT INTO dues_storage SELECT * FROM dues WHERE id=5;
it throws an error pointing to the difference in the number of columns.
I have two questions:
How do I get around this?
How do I add the value for the additional date column (CURRENT_DATE()) in table2 within this same statement?
To refine the answer from Zed, and to answer your comment:
INSERT INTO dues_storage
SELECT d.*, CURRENT_DATE()
FROM dues d
WHERE id = 5;
See T.J. Crowder's comment
The safest way to do it is to fully specify the columns both for insertion and extraction. There's no guarantee (to the application) that either of these will be the order you think they may be.
insert into dues_storage (f1, f2, f3, cd)
select f1, f2, f3, current_date() from dues where id = 5;
If you're worried about having to change many multiple PHP pages that do this (as you seem to indicate in the comment to another answer), this is ripe for a stored procedure. That way, all your PHP pages simply call the stored procedure with (for example) just the ID to copy and it controls the actual copy process. That way, there's only one place where you need to maintain the code, and, in my opinion, the DBMS is the right place to do it.
INSERT INTO dues_storage
SELECT field1, field2, ..., fieldN, CURRENT_DATE()
FROM dues
WHERE id = 5;
Hope this will help someone... Here's a little PHP script I wrote in case you need to copy some columns but not others, and/or the columns are not in the same order on both tables. As long as the columns are named the same, this will work. So if table A has [userid, handle, something] and tableB has [userID, handle, timestamp], then you'd "SELECT userID, handle, NOW() as timestamp FROM tableA", then get the result of that, and pass the result as the first parameter to this function ($z). $toTable is a string name for the table you're copying to, and $link_identifier is the db you're copying to. This is relatively fast for small sets of data. Not suggested that you try to move more than a few thousand rows at a time this way in a production setting. I use this primarily to back up data collected during a session when a user logs out, and then immediately clear the data from the live db to keep it slim.
function mysql_multirow_copy($z,$toTable,$link_identifier) {
$fields = "";
for ($i=0;$i<mysql_num_fields($z);$i++) {
if ($i>0) {
$fields .= ",";
}
$fields .= mysql_field_name($z,$i);
}
$q = "INSERT INTO $toTable ($fields) VALUES";
$c = 0;
mysql_data_seek($z,0); //critical reset in case $z has been parsed beforehand. !
while ($a = mysql_fetch_assoc($z)) {
foreach ($a as $key=>$as) {
$a[$key] = addslashes($as);
next ($a);
}
if ($c>0) {
$q .= ",";
}
$q .= "('".implode(array_values($a),"','")."')";
$c++;
}
$q .= ";";
$z = mysql_query($q,$link_identifier);
return ($q);
}
Alternatively, you can use Inner Queries to do so.
SQL> INSERT INTO <NEW_TABLE> SELECT * FROM CUSTOMERS WHERE ID IN (SELECT ID FROM <OLD_TABLE>);
Hope this helps!
SET #sql =
CONCAT( 'INSERT INTO <table_name> (',
(
SELECT GROUP_CONCAT( CONCAT('`',COLUMN_NAME,'`') )
FROM information_schema.columns
WHERE table_schema = <database_name>
AND table_name = <table_name>
AND column_name NOT IN ('id')
), ') SELECT ',
(
SELECT GROUP_CONCAT(CONCAT('`',COLUMN_NAME,'`'))
FROM information_schema.columns
WHERE table_schema = <database_name>
AND table_name = <table_source_name>
AND column_name NOT IN ('id')
),' from <table_source_name> WHERE <testcolumn> = <testvalue>' );
PREPARE stmt1 FROM #sql;
execute stmt1;
Of course replace <> values with real values, and watch your quotes.
Just wanted to add this little snippet which works beautifully for me.
INSERT INTO your_target_table
SELECT *
FROM your_rescource_table
WHERE id = 18;
And while I'm at it give a big shout out to Sequel Pro, if you're not using it I highly recommend downloading it...makes life so much easier