We are working with Amounts of which value are higher. We are displaying the formatted amount in the respective spark TextInput. We are using the simple mx CurrencyFormatter for formatting the amount values. We dont have any problems till 16 digits . But after crossing 16 digits , the numbers are automtically rounded off. We are using the CurrencyFormatter with the following configurations,
<mx:CurrencyFormatter id="formateer" thousandsSeparatorTo="," decimalSeparatorTo="."
precision="2" currencySymbol="" rounding="none" />
My output:
We dont have any problem upto 16 digits
original-->1234567890123456
Number(txtInput.text)-->1234567890123456
formatted-->1,234,567,890,123,456.00
Erroneous output:
original-->12345678901234567
Number(txtInput.text)-->12345678901234568
formatted-->12,345,678,901,234,568.00
Here the last digit 7 is rounded to 8.
Erroneous output:
original-->12345678901234567890
Number(txtInput.text)-->12345678901234567000
formatted-->12,345,678,901,234,567,000.00
I have debugged the code and had gone into the format() method CurrencyFormatter . There actually the problem occurs from the Number conversion. I am wondering since the Number.MAX_VALUE is 1.79769313486231e+308 .
Also I found one more weird behavior of the Number. I described below,
var a:Number = 2.03;
var b:Number = 0.03
var c:Number = a- b;
trace("c --> "+c);
Output : c --> 1.9999999999999998
This kind of output is obtaining for this numbers only.
Please suggest me how to solve this issue or suggest me a workaround method.
Thanks in advance.
Vengatesh s
It's a common problem with big numbers in languages that use 64-bit floating point arithmetic (Actionscript and Javascript are the same in this, to make an example).
It has nothing to do with the CurrencyFormatter, if you try to trace(12345678901234566+1) you'll get 12345678901234568. That's because that number has so many digits that fills the 64-bit storage space and so it gets rounded off. I realise the explanation is quite simplistic, the argument is in fact quite complex.
There are a few BigInt libraries already available (i think as3crypt has one) that can be used if you have to do some arithmetic ... for the formatting i think you'll have to roll your own
EDIT:
out of curiosity, you can use this to see how your number is being represented in the IEEE754 binary format
Related
I have two fields, both have the size set to double in the table properties. When I subtract one field from the other some of the results are displayed as scientific notation when I click in the cell and others just show regular standard format to decimal places.
The data in both fields was updated with Round([Field01],2) and Round([Filed2],2) so the numbers in the fields should not be any longer than 2 decimal places.
Here's an example:
Field1 = 7.01
Field2 = 7.00
But when I subtract Field1 from Field2 the access display shows 0.01 but when I click on the result it displays, -9.99999999999979E-03. So of course, when I try to filter on all results that have 0.01 the query comes back empty because it thinks the result is -9.99999999999979E-03.
Even stranger is if Field1 = 1.02 and Field2 = 1.00, the result is 0.02 and when I click on the result the display still shows 0.02 and I can filter on all results that equal 0.02.
Why would MS Access treat numbers in the same query differently? Why is it displaying in Scientific Notation and not filtering?
Thanks for any support.
Take this simple code in Access (or even Excel) and run it!
Public Sub TestAdd()
Dim MyNumber As Single
Dim I As Integer
For I = 1 To 10
MyNumber = MyNumber + 1.01
Debug.Print MyNumber
Next I
End Sub
Here is the output of the above:
1.01
2.02
3.03
4.04
5.05
6.06
7.070001
8.080001
9.090001
10.1
You can see that after just 7 additions rounding is occurring!
Note how after JUST 7 simple little additions Access is now spitting out wrong numbers and has rounding errors!
More amazing? The above code runs the SAME in Excel!
Ok, I am sure I have your attention now!
If I recall, the FIRST day and first class in computing science? Computers don't store exact numbers when using floating point numbers.
So, then how is it possible that the WHOLE business community using Excel, or Access, or in fact your desktop calculator not come crashing down?
You mean Access cannot add up 7 simple little numbers without having errors?
How can I even do payroll then?
The basic concept and ALL you need to know here is that computers store real (floating) numbers only as approximate.
And integer values are stored exact.
so, there are several approaches here, and in fact if you writing ANY business software that needs to work with money values? And not suffer rounding errors?
Then you better off to choose what we called some kind of "scaled" integer. Behind the scenes, the computer does NOT use floating numbers, but uses a integer value, and the also has a "decimal" position.
In fact, in a lot of older business BASIC languages, or others? We often had to do the scaling on our own. (so, we would choose a large integer format). In fact, this "scaling" feature still exists in Access!!! (and you see it in the format options).
So, two choices here. If you don't want "tiny" rounding errors, then use "currency" data type. This may, or may not be sufficient for you, since it only allows a max of 4 decimal places. But in most cases, it should suffice. And if you need "more" decimal places, then you can multiply the values by 1000, and then divide by 1000 when done the calculations.
however, try changing the column type to currency and that should work. (this type of data is how your desktop calculator also works - and thus you not see funny rounding errors as a result (in most cases).
but, the FIRST rule of the day? First computer course?
Computers do not store exact numbers for floating point numbers - they are approximations, and are subject to rounding errors. Now, if you really are using double for the table, then I don't think these rounding errors should show up - since you have "so many decimal places" available.
But, I would try using currency data type - it is a scaled integer, or so called packed decimal.
You can ALSO choose to use a packed decimal in Access, and it supports out to 28 digits, and you can set the "scale" (the decimal point location). However, since you can't declare a decimal type in VBA, then I would suggest that in the table (and in VBA code, use currency data types).
If you need more then 4 decimal points, then consider scaling the currency in your code, or perhaps at that point, you consider using a packed decimal type in the table, but values in VBA will have to use the "variant" type, and they will correctly take on the data column setting if used in code and assigned a value from the table(s) in question.
Needless to say, the first day you start dealing with computers, and that first day ANYTHING beyond being a "end user"? Well, this is your first lesson of the day!
"The data in both fields was updated with Round([Field01],2) and Round([Filed2],2) so the numbers in the fields should not be any longer than 2 decimal places." instead of rounding up(which i think is the reason for the scientific notation) you can use number field as data type , then under field size choose double, then under decimal places choose 2.
Hi after calling this code (Octave) I get an answer with 7 digits of precision, I need only 6. It is worth mentioning that on different data-set the output is normal(with 6 digits);
output_precision(6);
Prev
output:
Prev =
0.1855318
0.2181108
0.1796457
I know this is a little late but I wanted to add an answer for anyone with the same question in the future.
According to the function reference for output_precision(), the argument passed to the function (in this case, 6) specifies the minimum number of significant figures, which only guarantees that future numeric output won't have less than that number of significant figures.
From what I've seen, if you use output_precision(new_val) before displaying an array (e.g., Prev in the question), then octave will round the element with the least digits before the decimal place to have new_val significant figures and then all other elements will be rounded to have the same number of digits after the decimal place as that initial rounded result. If you use a statement to output a single value instead of an array, then the output is just rounded to new_val significant figures. However, I don't know if this behavior is guaranteed .
Here's a short example of what I mean:
% array defined with values having 5 digits after decimal
F = [401.51670 313.70753 -88.55225 188.50067 280.21988 354.51821 54.51821 350];
output_precision(4)
F
output_precision(6)
F
Output:
F =
401.52 313.71 -88.55 188.50 280.22 354.52 54.52 350.00
F =
401.5167 313.7075 -88.5523 188.5007 280.2199 354.5182 54.5182 350.0000
It can be a little quirky if you try to round the values too much. When I used output_precision(3) and then output F, the numbers were actually rounded as if my system's default precision, 5, was still active. However, when I used elements with only 2 or 3 digits after the decimal to define another array, it displayed as expected with output_precision(3).
Check out Octave Forge if you ever need docs for octave features. It's not perfect but it's something. Hope this was helpful.
code is here :
RstJson = rfc4627:encode({obj, [{"age", 45.99}]}),
{ok, Req3} = cowboy_req:reply(200, [{<<"Content-Type">>, <<"application/json;charset=UTF-8">>}], RstJson, Req2)
then I get this wrong data from front client:
{
"age": 45.990000000000002
}
the float number precision is changed !
how can I solved this problem?
Let's have a look at the JSON that rfc4627 generates:
> io:format("~s~n", [rfc4627:encode({obj, [{"age", 45.99}]})]).
{"age":4.59900000000000019895e+01}
It turns out that rfc4627 encodes floating-point values by calling float_to_list/1, which uses "scientific" notation with 20 digits of precision. As Per Hedeland noted on the erlang-questions mailing list in November 2007, that's an odd choice:
A reasonable question could be why float_to_list/1 generates 20 digits
when a 64-bit float (a.k.a. C double), which is what is used internally,
only can hold 15-16 worth of them - I don't know off-hand what a 128-bit
float would have, but presumably significantly more than 20, so it's not
that either. I guess way back in the dark ages, someone thought that 20
was a nice and even number (I hope it wasn't me:-). The 6.30000 form is
of course just the ~p/~w formatting.
However, it turns out this is actually not the problem! In fact, 45.990000000000002 is equal to 45.99, so you do get the correct value in the front end:
> 45.990000000000002 =:= 45.99.
true
As noted above, a 64-bit float can hold 15 or 16 significant digits, but 45.990000000000002 contains 17 digits (count them!). It looks like your front end tries to print the number with more precision than it actually contains, thereby making the number look different even though it is in fact the same number.
The answers to the question Is floating point math broken? go into much more detail about why this actually makes sense, given how computers handle floating point values.
the encode float number function in rfc4627 is :
encode_number(Num, Acc) when is_float(Num) ->
lists:reverse(float_to_list(Num), Acc).
I changed it like this :
encode_number(Num, Acc) when is_float(Num) ->
lists:reverse(io_lib:format("~p",[Num]), Acc).
Problem Solved.
I'm trying to the the mass of the black hole at the center of this galaxy, I have the mass in solar masses, but need it in kg. However when I try to convert (1Msolar = 1.989*10^30kg) idl just gives me 0.0000. I have no idea what I'm doing wrong and I tried just telling idl to print both 1.989*10^30 and 1989000000000000000000000000000 and the outputs are 0.00000 and -1 respectively. Can someone please explain why this is happening?
This is a type conversion error/overflow issue. When you use large numbers you either need to explicitly define them as long or long64 (i.e., 64-bit long integer) for integer numbers. For real numbers, you can use float or double and to do this, the easiest way is the following:
msun = 1.989d30
which is equivalent to 1.989 x 1030 as a double-precision floating point number. If you want single precision, then just do the following:
msun = 1.989e30
To make a 32- or 64-bit long integer, just use:
msun = 1989L * 10L^(27)
or for 64-bit
msun = 1989LL * 10LL^(27)
I agree with #honeste_vivere's answer about overflow and data types, but I would add that I often change units to avoid this. I frequently have densities that are order 1e19/m^3, so I cast density in units of 1e19/m^3 and then deal with numbers that are order 1. This prevents math errors during least squares fits and other operations that might do things like squaring my data.
I have this issue in converting a HEX string to Number in as3
I have a value
str = "24421bff100317"; decimal value = 10205787172373271
but when I parseInt it I get
parseInt(str, 16) = 10205787172373272
Can anyone please tell me what am I doing wrong here
Looks like adding one ("24421bff100318") works fine. I have to assume that means this is a case of precision error.
Because there are only a finite amount of numbers that can be represented with the memory available, there will be times that the computer is estimating. This is common when working with decimals and very large numbers. It's visible, for example, in this snippet where apparently the computer can't add basic decimals:
for(var i=0;i<3;i+=0.2){
trace(i);
}
There are a few workarounds if accuracy at this level is critical, namely using datatypes that store more information ("long" instead of "int" in Java - I believe "Number" might work in AS3 but I have not tested it for your scenario) or if that fails, breaking the numbers down into smaller parts and adding them together.
For further reading to understand this topic (since I do think it's fascinating), look up "precision errors" and "data types".