I have a report that should return something along the lines of
SELECT brand, ROUND(SUM(count * price) / SUM(count), 2)
WHERE ... GROUP BY brand, ...;
The problem is, I sometimes get 9990.32999999999992345 in my perl code instead of 9990.33 which direct SQL request returns.
The number starts looking that way right after fetchrow_hashref, if it ever does. The same number can come in 'good' or 'bad' form in different queries, but always the same way in any specific query.
How can I track this down?
Read all about floating point accuracy problems here: http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems
As mellamokb said, you have to round your floating-point numbers. More importantly, count and price probably means that you are calculating the price of something. As this page explains for the FLOAT and DOUBLE datatype, calculations are approximate while for DECIMAL they are exact. For your particular example, the chance is low that will give problems but not if you do a lot of calculations with your price. The usual rule is to always use exact datatypes for calculating prices.
Always round floating point numbers when displaying them on the screen. And do it as the final step as it is displayed. Any intermediate operation has the potential to cause problems like this.
I can think of a couple of causes of this, but first:
Does it make any difference to put a CONCAT( '', ... ) around your ROUND? What version of perl are you using? What does perl -V:nvtype report?
33/100 is a periodic number in binary just like 1/3 is a periodic number in decimal.
$ perl -e'printf "%.20f\n", 0.33'
0.33000000000000001554
Therefore, it would take infinite storage to store it as a floating point number. To avoid the problem, you'll need to store the number as a string, either early (in the query before it's a float) or late (by rounding).
It's an issue inherent with floating point numbers. It's a design feature, not a flaw.
Make sure the value returned from the database is not a floating point value, but a string or decimal. (If the data types of `price` and `count` are both DECIMAL, then the resulting expression should be DECIMAL.
If either of those is a floating point, then you can convert to DECIMAL...
SELECT brand, CONVERT( SUM(count * price) / SUM(count), DECIMAL(18,2) )
WHERE ... GROUP BY brand, ...;
Or convert to a string
SELECT brand, CONVERT(CONVERT( SUM(count * price) / SUM(count), DECIMAL(18,2)),CHAR)
WHERE ... GROUP BY brand, ...;
You can let the conversion to DECIMAL do the rounding for you. If you return a DECIMAL or VARHCAR to Perl, that should avoid floating point issues.
More generally, to handle representation (rounding) of floating point in Perl, you can format using the sprintf function, e.g.
my $rounded_val = sprintf(%.2f, $float_val);
Related
I'm trying to get the average number, and then remove the trailing, pointless zeros afterwards, (new to SQL) but I can't understand why it wont remove them, do I have the wrong idea??
So far I have;
SELECT total,
AVG(total(TRUNCATE(total/1,2))
I think you are looking for cast as below.
select cast(17.800000 as dec(3,1))
Result:
val
----
17.8
so you query will be
SELECT total, cast(AVG(total) as dec(3,1))
considering you just need 2 digit before . If you need more digits, you can adjust it accordingly.
DEMO
Assuming you are using SQL Server then you can cast the answer to a decimal with one decimal point:
select cast(avg(total) as decimal(9,1))
This SQLFiddle shows it: link
SELECT
TRUNCATE(AVG(myFloat), 2),
AVG(myFloat),
ROUND(AVG(myFloat), 2)
FROM docs
You should probably use ROUND instead of TRUNCATE.
The stuff after the decimal is odd because of floating point math, and there are occasions where floating point math is internally calculated as .009999999 instead of .01000000000
I believe these answers that use a CAST may have the same truncation problem.
You simply want to avoid casting or truncation when you are removing the decimal places beyond what you're interested in. Be explicit in what you are doing and less mistakes will pop up later.
I have a simple float column, that doesn't yield the correct value when selected via a CASE:
SELECT my_column FROM my_table LIMIT 1; yields 815.35
But SELECT (CASE WHEN true THEN my_column ELSE 0 END) AS my_column FROM my_table LIMIT 1; yields 815.3499755859375
Problem is obviously coming from the case and from the ELSE value (using 'test' rather than 0 works as intended, but using an other float does not)
I could solve it by using ROUND(my_column,2), or using a decimal column instead of a float one, but I'd actually want to understand what's happening here
I think #dasblinkenlight has explained the underlying issue with the representation. Your question is also about the CASE.
A CASE expression returns a single type. MySQL has to decide on the type when the query is compiled.
Your CASE is combining two different types, a float and an int. I believe that this should be returning a float value.
The rules leave me a bit confused as to why anything is happening; after all, a float to a float sounds like a no-op. But, there are two representations for floats, 4-byte and 8-byte. My guess is that your column is stored as a 4-byte float. The SQL engine decides that the CASE expression should return an 8-byte double. The conversion to the double is the cause of your issue.
In fact, this little SQLFiddle confirms this guess. The issue is a conversion to double.
The value 815.35 has no exact representation as IEEE-754 float. The value that is actually stored in a float field is an approximation that depends on the number of bits used for the representation.
When you use single-precision float, the value becomes 815.3499755859375, which is what you see printed when you run the query. You can compute the representation that you get with an IEEE-754 calculator (for example, this one).
In order to avoid representation differences like this, use decimal data ty[e to represent values that need precise decimal representation, such as amounts of money.
I spotted some rounding bug in MySQL. Here is my query:
SELECT /*debugz*/ ROUND((SUM(grade)/2),0) AS grade, SUM(grade) FROM entry.computed_grade a WHERE a.stud_id='7901159' AND a.sy='2014' AND a.term=01 AND a.terms=01 AND a.catalog_no='Christian Life Formation';
and the result is this:
grade sum(grade)
------ ------------
92 185
The grade result should be 93, not 92 because 185/2 = 92.5
Try this
SELECT CEIL((SUM(grade)/2),0) AS grade, SUM(grade) FROM entry.computed_grade a WHERE ((a.stud_id='7901159') AND (a.sy='2014') AND (a.term=01) AND (a.terms=01) AND (a.catalog_no='Christian Life Formation'));
Try to use ceil instead of round.
e.g ceil(1.45) = 2
You should check rounding behavior artickle for mysql. I believe here is the reason of your problem:
For approximate-value numbers, the result depends on the C library. On
many systems, this means that ROUND() uses the “round to nearest even”
rule: A value with any fractional part is rounded to the nearest even
integer.
By the way it's IEEE standard for float point rounding, so you might want stay with it
Do not "patch" this problem by tweaking the query. Actually fix your database. If you are not storing the "grade" column as the DECIMAL data type, and are instead using FLOAT or DOUBLE, your design is inherently broken.
Because floating-point values are approximate and not stored as exact values, attempts to treat them as exact in comparisons may lead to problems.
http://dev.mysql.com/doc/refman/5.6/en/floating-point-types.html
This is not a bug in MySQL. It is an inherent limitation in industry-standard floating point number storage. Use DECIMAL columns to store meaningful, precise numbers, and the other two types only when low storage space or a wide range of allowable values are more important than precision.
Currently using MySQL version 5.1.6
This is my first real world build and so far I have actually been enjoying it; however, I now am stuck on a decision regarding a field datatype and hoping someone could sort it out for me.
I essentially have 10 fields that are all different test results. The numbers range from -100 to 100 and can have a decimal with one spot after the actual point.
For example, -5.1, 0, 1, 16.3, 99.2, and 100 are all possible data. From what I have read, one should use DECIMAL for those things that we usually measure and are exact (which these are), whereas FLOAT and DOUBLE are approximations, which I do not really want (though I am sure at this level, the approximation is very small if existent at all).
If I use DECIMAL, do I have to include a space for the '-' at the beginning if used? I.E, would I use DECIMAL(4,1) or DECIMAL(5,1) or am I way off here? I might be overthinking this a bit.
DECIMAL(4,1) will be enough, the sign digit does not need to be included.
More info: http://dev.mysql.com/doc/refman/5.1/en/precision-math-decimal-changes.html
For example, a DECIMAL(3,0) column supports a range of -999 to 999
Decimal will be indeed the best option for your needs. Float and Double can give you ugly numbers (e.g. 0.2 cannot be represented as float, you'd get 0.19999999)
The negative symbol does NOT count as a digit in your calculation, so DECIMAL(4,1) should be fine.
Edit: That also seems like the right field to use for your purposes. Try it out!
Look at this query please
SELECT max( val_amd ) FROM `best_deposits`
I have the max value in the table equal to 14.6(the fields has type float),
But it returns 14.3599996566772
why does it happen, and how can i get the exact value?
Thanks much
floats are evil!
NEVER use floats for storing amounts or prices. instead of that, use an int and store the amount in cents. thats the only way to get around those problems forever.
why this happens: because floats can't be saved exactly in many cases (such as 0.6 in your case)
PS: we had those questions a hundret times for different languages till now:
Use Float or Decimal for Accounting Application Dollar Amount?
PHP rounding problem (5.2.3)?
Rounding problem with double type
Javascript rounding v c# rounding
Python rounding problem
... and a lot more
EDIT: to your comment: as i said:
use an int and store the amount in
cents
(alternatively you could use an DECIMAL(10,2) (or how big/how much decimal places you need)... not sure about how this works)
Or you better use "decimal" with length 10,2 or something like that for storing prices.