I have a simple float column, that doesn't yield the correct value when selected via a CASE:
SELECT my_column FROM my_table LIMIT 1; yields 815.35
But SELECT (CASE WHEN true THEN my_column ELSE 0 END) AS my_column FROM my_table LIMIT 1; yields 815.3499755859375
Problem is obviously coming from the case and from the ELSE value (using 'test' rather than 0 works as intended, but using an other float does not)
I could solve it by using ROUND(my_column,2), or using a decimal column instead of a float one, but I'd actually want to understand what's happening here
I think #dasblinkenlight has explained the underlying issue with the representation. Your question is also about the CASE.
A CASE expression returns a single type. MySQL has to decide on the type when the query is compiled.
Your CASE is combining two different types, a float and an int. I believe that this should be returning a float value.
The rules leave me a bit confused as to why anything is happening; after all, a float to a float sounds like a no-op. But, there are two representations for floats, 4-byte and 8-byte. My guess is that your column is stored as a 4-byte float. The SQL engine decides that the CASE expression should return an 8-byte double. The conversion to the double is the cause of your issue.
In fact, this little SQLFiddle confirms this guess. The issue is a conversion to double.
The value 815.35 has no exact representation as IEEE-754 float. The value that is actually stored in a float field is an approximation that depends on the number of bits used for the representation.
When you use single-precision float, the value becomes 815.3499755859375, which is what you see printed when you run the query. You can compute the representation that you get with an IEEE-754 calculator (for example, this one).
In order to avoid representation differences like this, use decimal data ty[e to represent values that need precise decimal representation, such as amounts of money.
Related
I am inserting data from one table into another in a MariaDB database, where the column in the first table is FLOAT, and in the second it's DOUBLE. The data can have values of any size, precision and decimal places.
Here is what happens to the values when I do a straight-forward copy:
INSERT INTO data2 (value) SELECT value FROM data1
The values are given random extra significant figures:
FLOAT in data1 DOUBLE in data2
-0.000000000000454747 -0.0000000000004547473508864641
-122.319 -122.31932830810547
14864199700 14864220160
CAST(value AS DECIMAL(65,30)) generates exactly the same values as col 2 above, except I see trailing zeroes.
Yet when I just do
UPDATE data2 SET value = 14867199700 WHERE id = 133025046;
the DOUBLE value is accepted.
Do I have to export all the value to an SQL script and re-import them? Isn't there a better way?
Despite hours trying to experimenting with the issue, I'm not much closer to a solution, despite its limited nature. I can see this is problem that besets all technologies, not just MariaDB or databases, so I have probably just missed the answer somewhere. Stackoverflow is desperately trying to guide to a solution with new suggestion features I hadn't seen before, but unfortunately they are no help, like the other suggested answers.
Your test case is flawed. You are feeding in decimal digits, and not testing just the transfer of FLOAT to DOUBLE.
UPDATE tbl SET double_col = float_col will always copy exactly the same value. This because the DOUBLE representation is a superset of the FLOAT representation (53 vs 24 bits of precision; etc).
Literal, with decimal places: UPDATE tbl SET double_col = 123.456 will mangle the number because of rounding from decimal to DOUBLE. Ditto for float_col. Furthermore, the mangled results will be different!
Hole number literal: UPDATE tbl SET double_col = 14867199700 will be stored exactly. But if you put that same literal into a FLOAT, it will be rounded to 24 bits, so it cannot be stored exactly. You lose exactness at about 7 significant digits for FLOAT and about 16 for DOUBLE. The literal in this example has 9 significant digits (after ignoring trailing zeros).
That's just a sampling of the nightmares you can get into.
You must consider FLOAT and DOUBLE to be approximate. You should never compare for equality; you don't know what might have messed with the last bit of the value.
Also, you should not try to guess when MySQL will perform expressions in DECIMAL instead of DOUBLE.
And, keep in mind that division is usually imprecise due to rounding to some number of bits or decimals.
The "mantissa" of 14864199700 is
1.10111010111111001101100 (binary of FLOAT : 24 bits including 'hidden' leading bit)
1.1011101011111100110110000000101000000000000000000000 (binary of DOUBLE)
^ ^ (lost in FLOAT)
Each of those is multiplied by the same power of 2. The DOUBLE gets exactly 14864199700. The FLOAT lost the bits pointed to.
You can play around with such at https://gregstoll.dyndns.org/~gregstoll/floattohex/
Believe it or not, things used to be worse. People would be billed for $0.00 -- due to rounding errors. Or results of what should have been 1+1 showed as 1.99999999.
Database field name with datatype value float(5,2)
Inserted value
7.80
78.00
My query in modal
$checkValue = static::find()->where(['value' => $this->value])->one();
If i passed $this->valueequal to 78.00 or 78.000 then it returns proper result.
But if I pass 7.80 or 7.8 then 0 rows are returned. Why?
i suspect internally mysql treats 7.8 as something like 7.800000000001 so you cannot get a result if you compare with a fixed value.
you may have come across mysql's reference manuals on datatypes.
please note the following:
MySQL permits a nonstandard syntax: FLOAT(M,D) or REAL(M,D) or DOUBLE PRECISION(M,D). Here, (M,D) means than values can be stored with up to M digits in total, of which D digits may be after the decimal point. For example, a column defined as FLOAT(7,4) will look like -999.9999 when displayed. MySQL performs rounding when storing values, so if you insert 999.00009 into a FLOAT(7,4) column, the approximate result is 999.0001.
Because floating-point values are approximate and not stored as exact values, attempts to treat them as exact in comparisons may lead to problems. They are also subject to platform or implementation dependencies. For more information, see Section B.5.4.8, “Problems with Floating-Point Values”
For maximum portability, code requiring storage of approximate numeric data values should use FLOAT or DOUBLE PRECISION with no specification of precision or number of digits.
for most applications you can safely use a fixed point type.
essentially using decimal(5,2) instead of float(5,2) ensuring that any value displayed is the exact value stored internally.
when applicable rounding happens on insert with the "round half up" rule to the precision you specifed and is somewhat more intuitive and easy to manage
I resolve this issue with double datatype
value double(5,2)
I am creating for fun, but I still want to approach it seriously, a site which hosts various tests. With these tests I hope to collect statistical data.
Some of the data will include the percentage of the completeness of the tests as they are timed. I can easily compute the percentage of the tests but I would like true data to be returned as I store the various different values concerning the tests on completion.
Most of the values are, in PHP floats, so my question is, if I want true statistical data should I store them in MYSQL as FLOAT, DOUBLE or DECIMAL.
I would like to utilize MYSQL'S functions such as AVG() and LOG10() as well as TRUNCATE(). For MYSQL to return true data based off of my values that I insert, what should I use as the database column choice.
I ask because some numbers may or may not be floats such as, 10, 10.89, 99.09, or simply 0.
But I would like true and valid statistical data to be returned.
Can I rely on floating point math for this?
EDIT
I know this is a generic question, and I apologise extensively, but for non mathematicians like myself, also I am not a MYSQL expert, I would like an opinion of an expert in this field.
I have done my research but I still feel I have a clouded judgement on the matter. Again I apologise if my question is off topic or not suitable for this site.
This link does a good job of explaining what you are looking for. Here is what is says:
All these three Types, can be specified by the following Parameters (size, d). Where size is the total size of the String, and d represents precision. E.g To store a Number like 1234.567, you will set the Datatype to DOUBLE(7, 3) where 7 is the total number of digits and 3 is the number of digits to follow the decimal point.
FLOAT and DOUBLE, both represent floating point numbers. A FLOAT is for single-precision, while a DOUBLE is for double-precision numbers. A precision from 0 to 23 results in a 4-byte single-precision FLOAT column. A precision from 24 to 53 results in an 8-byte double-precision DOUBLE column. FLOAT is accurate to approximately 7 decimal places, and DOUBLE upto 14.
Decimal’s declaration and functioning is similar to Double. But there is one big difference between floating point values and decimal (numeric) values. We use DECIMAL data type to store exact numeric values, where we do not want precision but exact and accurate values. A Decimal type can store a Maximum of 65 Digits, with 30 digits after decimal point.
So, for the most accurate and precise value, Decimal would be the best option.
Unless you are storing decimal data (i.e. currency), you should use a standard floating point type (FLOAT or DOUBLE). DECIMAL is a fixed point type, so can overflow when computing things like SUM, and will be ridiculously inaccurate for LOG10.
There is nothing "less precise" about binary floating point types, in fact, they will be much more accurate (and faster) for your needs. Go with DOUBLE.
Decimal : Fixed-Point Types (Exact Value). Use it when you care about exact precision like money.
Example: salary DECIMAL(8,2), 8 is the total number of digits, 2 is the number of decimal places. salary will be in the range of -999999.99 to 999999.99
Float, Double : Floating-Point Types (Approximate Value). Float uses 4 bytes to represent value, Double uses 8 bytes to represent value.
Example: percentage FLOAT(5,2), same as the type decimal, 5 is total digits and 2 is the decimal places. percentage will store values between -999.99 to 999.99.
Note that they are approximate value, in this case:
Value like 1 / 3.0 = 0.3333333... will be stored as 0.33 (2 decimal place)
Value like 33.009 will be stored as 33.01 (rounding to 2 decimal place)
Put it simply, Float and double are not as precise as decimal. decimal is recommended for money related number input.(currency and salary).
Another point need to point out is: Do NOT compare float number using "=","<>", because float numbers are not precise.
Linger: The website you mention and quote has IMO some imprecise info that made me confused. In the docs I read that when you declare a float or a double, the decimal point is in fact NOT included in the number. So it is not the number of chars in a string but all digits used.
Compare the docs:
"DOUBLE PRECISION(M,D).. Here, “(M,D)” means than values can be stored with up to M digits in total, of which D digits may be after the decimal point. For example, a column defined as FLOAT(7,4) will look like -999.9999 when displayed"
http://dev.mysql.com/doc/refman/5.1/en/floating-point-types.html
Also the nomenclature in misleading - acc to docs: M is 'precision' and D is 'scale', whereas the website takes 'scale' for 'precision'.
Thought it would be useful in case sb like me was trying to get a picture.
Correct me if I'm wrong, hope I haven't read some outdated docs:)
Float and Double are Floating point data types, which means that the numbers they store can be precise up to a certain number of digits only.
For example for a table with a column of float type if you store 7.6543219 it will be stored as 7.65432.
Similarly the Double data type approximates values but it has more precision than Float.
When creating a table with a column of Decimal data type, you specify the total number of digits and number of digits after decimal to store, and if the number you store is within the range you specified it will be stored exactly.
When you want to store exact values, Decimal is the way to go, it is what is known as a fixed data type.
Simply use FLOAT. And do not tack on '(m,n)'. Do display numbers to a suitable precision with formatting options. Do not expect to get correct answers with "="; for example, float_col = 0.12 will always return FALSE.
For display purposes, use formatting to round the results as needed.
Percentages, averages, etc are all rounded (at least in some cases). That any choice you make will sometimes have issues.
Use DECIMAL(m,n) for currency; use ...INT for whole numbers; use DOUBLE for scientific stuff that needs more than 7 digits of precision; use FLOAT` for everything else.
Transcendentals (such as the LOG10 that you mentioned) will do their work in DOUBLE; they will essentially never be exact. It is OK to feed it a FLOAT arg and store the result in FLOAT.
This Answer applies not just to MySQL, but to essentially any database or programming language. (The details may vary.)
PS: (m,n) has been removed from FLOAT and DOUBLE. It only added extra rounding and other things that were essentially no benefit.
I have the following sql query in mysql:
SELECT *
FROM _t_test
WHERE pret NOT
IN ( 2.6700, 2.6560, 1.8200 )
I would expect the rows with the value 1.8200 not to be shown, yet I still get them.
Am I missing something?
The field "pret" is double(16,4).
This is a rounding error. A double is not an exact value, so 1.8200 isn't represented exactly, so the values are not exactly the same.
For MYSQL floating points, see http://dev.mysql.com/doc/refman/5.0/en/problems-with-float.html
The correct way to do floating-point number comparison is to first
decide on an acceptable tolerance for differences between the numbers
and then do the comparison against the tolerance value. For example,
if we agree that floating-point numbers should be regarded the same if
they are same within a precision of one in ten thousand (0.0001), the
comparison should be written to find differences larger than the
tolerance value
See http://en.wikipedia.org/wiki/Double_precision_floating-point_format
I have a report that should return something along the lines of
SELECT brand, ROUND(SUM(count * price) / SUM(count), 2)
WHERE ... GROUP BY brand, ...;
The problem is, I sometimes get 9990.32999999999992345 in my perl code instead of 9990.33 which direct SQL request returns.
The number starts looking that way right after fetchrow_hashref, if it ever does. The same number can come in 'good' or 'bad' form in different queries, but always the same way in any specific query.
How can I track this down?
Read all about floating point accuracy problems here: http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems
As mellamokb said, you have to round your floating-point numbers. More importantly, count and price probably means that you are calculating the price of something. As this page explains for the FLOAT and DOUBLE datatype, calculations are approximate while for DECIMAL they are exact. For your particular example, the chance is low that will give problems but not if you do a lot of calculations with your price. The usual rule is to always use exact datatypes for calculating prices.
Always round floating point numbers when displaying them on the screen. And do it as the final step as it is displayed. Any intermediate operation has the potential to cause problems like this.
I can think of a couple of causes of this, but first:
Does it make any difference to put a CONCAT( '', ... ) around your ROUND? What version of perl are you using? What does perl -V:nvtype report?
33/100 is a periodic number in binary just like 1/3 is a periodic number in decimal.
$ perl -e'printf "%.20f\n", 0.33'
0.33000000000000001554
Therefore, it would take infinite storage to store it as a floating point number. To avoid the problem, you'll need to store the number as a string, either early (in the query before it's a float) or late (by rounding).
It's an issue inherent with floating point numbers. It's a design feature, not a flaw.
Make sure the value returned from the database is not a floating point value, but a string or decimal. (If the data types of `price` and `count` are both DECIMAL, then the resulting expression should be DECIMAL.
If either of those is a floating point, then you can convert to DECIMAL...
SELECT brand, CONVERT( SUM(count * price) / SUM(count), DECIMAL(18,2) )
WHERE ... GROUP BY brand, ...;
Or convert to a string
SELECT brand, CONVERT(CONVERT( SUM(count * price) / SUM(count), DECIMAL(18,2)),CHAR)
WHERE ... GROUP BY brand, ...;
You can let the conversion to DECIMAL do the rounding for you. If you return a DECIMAL or VARHCAR to Perl, that should avoid floating point issues.
More generally, to handle representation (rounding) of floating point in Perl, you can format using the sprintf function, e.g.
my $rounded_val = sprintf(%.2f, $float_val);