I have a calculated field in my table called C. its the result of A-B=C. A & B are number fields (single, fixed). I have having trouble setting up C as a calculated (Decimal Field).
The precision / decimal places seem to work perfectly, I can modify them freely. But no matter what I do to "SCALE". It always seems to return to "0". I need it to be 2 since all my data in my reports are rounding off at the wrong locations giving me hole numbers.
As you can see "scale = 0", no matter what I do to this number. it will always revert to "0". Why is that?
You can’t change the scale in a calculated field, because it takes the values and settings from the calculation.
So the fact of a scale of 0 should not matter. The resulting number if it needs decimal places will (should) have the decimal value. The setting is IGNORED
I mean, if the calculation is:
2 x 3 = 6
Then you get 6.
If you have 4 / 3 = 1.3333
Then, in your case you get:
1.33333333333333
And you WILL get the above EVEN if the scale = 0. So the scale setting is NOT used nor available in a calculated field.
You are certainly free to round, or format the above result. And in fact you could (should) consider using the round() function in the actual calculation. So use something like:
Round([Field1] / [Field2],4)
And you thus get:
1.3333
Related
I want to store decimal value into my field when you divide
1120/90 = 12.444444444444444444444444444444
This long but I am losing right side digits, I am only getting 13 right side digits - like that:
12.4444444444444
But when I multiply this back again:
12.4444444444444 x 90 = 1119.999999999996
This is not the correct answer; the correct answer is
12.444444444444444444444444444444 x 90 = 1120
Please help me here.
Thanks
You can use Decimal for this:
? CDec(1120) / CDec(90)
12.444444444444444444444444444
? (CDec(1120) / CDec(90)) * 90
1120
I banged on this for a long time but couldn't find a way to prove the following. assuming any operands can be cast to the integer type without loss of information (1120 & 90 can) then you usually must do the calculation simultaneously: The floating point division operator / handles integer like operands correctly but returns the type double. chained operations are also handled correctly if done all at once. Hence (1120/90) * 90 is calculated correctly as 1120 but also, typename(1120/90*90) returns the double datatype. if you store the intermediate value 1120/90 you get a double of 12.44444 repeating which isn't quite 1120/90.
My suggestion is to store both the operands rather than reducing them to one number. Then do the calculation all at once.
isinteger(sqrt(3))
0
isinteger(sqrt(4))
0
Both answers give zero. The answers must be:
isinteger(sqrt(3))
0
isinteger(sqrt(4))
1
isinteger checks for type. integer is a type of variable, not a property of a number. e.g. isinteger(2.0) returns 0.
Try:
mod(sqrt(x),1) == 0
However, you may still have issues with this due to numerical precision.
You may do as well
y = sqrt(4);
y==round(y)
or to take round-off error into account with a (2*eps) relative tolerance
abs(y-round(y)) <= 2*eps*y
Others have touched on this, but you need to be careful that floating point effects are taken into account for your application. Limited precision issues can give unexpected results. E.g., take this example:
Here you start with a non-integer value that is very close to 4 (x). The square root of this number in double precision is exactly 2 (y), but squaring this number does not equal the original x. So the calculated square root y is exactly an integer, but it really isn't indicative of the situation since the original x isn't an integer. The actual square root of x isn't an integer even though the floating point calculation of sqrt(x) is exactly an integer.
What if we also checked to see if the original x is an integer? Well, take this example:
Here the original x is so large that every floating point number near x is an integer, so the x+eps(x) is an integer. The calculated square root in double precision is also an integer (y). But even though both are integers, y*y does not equal x. Again we have the situation where the actual square root of x isn't an integer, but the floating point calculated value of sqrt(x) is exactly an integer.
So, bottom line is this can be a bit trickier than you might have anticipated. We don't know your application, but you might want to check that both x and y are integers and that y*y == x is true before convincing yourself that the square root of x is an integer. And even then, there might be cases where all these checks pass but still there is a discrepancy that floating point effects simply didn't uncover.
I have two fields, both have the size set to double in the table properties. When I subtract one field from the other some of the results are displayed as scientific notation when I click in the cell and others just show regular standard format to decimal places.
The data in both fields was updated with Round([Field01],2) and Round([Filed2],2) so the numbers in the fields should not be any longer than 2 decimal places.
Here's an example:
Field1 = 7.01
Field2 = 7.00
But when I subtract Field1 from Field2 the access display shows 0.01 but when I click on the result it displays, -9.99999999999979E-03. So of course, when I try to filter on all results that have 0.01 the query comes back empty because it thinks the result is -9.99999999999979E-03.
Even stranger is if Field1 = 1.02 and Field2 = 1.00, the result is 0.02 and when I click on the result the display still shows 0.02 and I can filter on all results that equal 0.02.
Why would MS Access treat numbers in the same query differently? Why is it displaying in Scientific Notation and not filtering?
Thanks for any support.
Take this simple code in Access (or even Excel) and run it!
Public Sub TestAdd()
Dim MyNumber As Single
Dim I As Integer
For I = 1 To 10
MyNumber = MyNumber + 1.01
Debug.Print MyNumber
Next I
End Sub
Here is the output of the above:
1.01
2.02
3.03
4.04
5.05
6.06
7.070001
8.080001
9.090001
10.1
You can see that after just 7 additions rounding is occurring!
Note how after JUST 7 simple little additions Access is now spitting out wrong numbers and has rounding errors!
More amazing? The above code runs the SAME in Excel!
Ok, I am sure I have your attention now!
If I recall, the FIRST day and first class in computing science? Computers don't store exact numbers when using floating point numbers.
So, then how is it possible that the WHOLE business community using Excel, or Access, or in fact your desktop calculator not come crashing down?
You mean Access cannot add up 7 simple little numbers without having errors?
How can I even do payroll then?
The basic concept and ALL you need to know here is that computers store real (floating) numbers only as approximate.
And integer values are stored exact.
so, there are several approaches here, and in fact if you writing ANY business software that needs to work with money values? And not suffer rounding errors?
Then you better off to choose what we called some kind of "scaled" integer. Behind the scenes, the computer does NOT use floating numbers, but uses a integer value, and the also has a "decimal" position.
In fact, in a lot of older business BASIC languages, or others? We often had to do the scaling on our own. (so, we would choose a large integer format). In fact, this "scaling" feature still exists in Access!!! (and you see it in the format options).
So, two choices here. If you don't want "tiny" rounding errors, then use "currency" data type. This may, or may not be sufficient for you, since it only allows a max of 4 decimal places. But in most cases, it should suffice. And if you need "more" decimal places, then you can multiply the values by 1000, and then divide by 1000 when done the calculations.
however, try changing the column type to currency and that should work. (this type of data is how your desktop calculator also works - and thus you not see funny rounding errors as a result (in most cases).
but, the FIRST rule of the day? First computer course?
Computers do not store exact numbers for floating point numbers - they are approximations, and are subject to rounding errors. Now, if you really are using double for the table, then I don't think these rounding errors should show up - since you have "so many decimal places" available.
But, I would try using currency data type - it is a scaled integer, or so called packed decimal.
You can ALSO choose to use a packed decimal in Access, and it supports out to 28 digits, and you can set the "scale" (the decimal point location). However, since you can't declare a decimal type in VBA, then I would suggest that in the table (and in VBA code, use currency data types).
If you need more then 4 decimal points, then consider scaling the currency in your code, or perhaps at that point, you consider using a packed decimal type in the table, but values in VBA will have to use the "variant" type, and they will correctly take on the data column setting if used in code and assigned a value from the table(s) in question.
Needless to say, the first day you start dealing with computers, and that first day ANYTHING beyond being a "end user"? Well, this is your first lesson of the day!
"The data in both fields was updated with Round([Field01],2) and Round([Filed2],2) so the numbers in the fields should not be any longer than 2 decimal places." instead of rounding up(which i think is the reason for the scientific notation) you can use number field as data type , then under field size choose double, then under decimal places choose 2.
With Microsoft Access 2010, I have two Single fields:
A = 1.1
B = 2.1
I create a query where I have defined C=A*B
Microsoft Access says that C = 2.30999994277954
but, in reality, C =2.31
How can I get the right result (2.31)?
Slightly off results from operations performed on decimal values can happen if your numeric field size is single or double rather than decimal. Single and double (or floating point) numbers are very close approximations of the "true" numbers, but should not be relied upon if accuracy in operations is required. A related stackoverflow question has more information about this issue: Access comparing floating-point numbers "incorrectly"
If it's possible to modify the underlying table's design, you should change the field size property for the "A" and "B" fields from single to decimal. After changing the field size BUT BEFORE saving the table, you will also need to adjust the Scale property for "A" and "B" from 0 to whatever number of places to the right of the decimal point you might require. You will likely still have a notice about losing data, but if you adjust the field properties correctly before saving the table, this shouldn't be a problem. You should probably make a copy of the table before doing this so that you can verify that there was no data loss. After saving your table and verifying the changes did not result in data loss, your query should represent A * B accurately.
why is this so?
when i try out:
Math.pow(2,58)=288230376151711740
while in fact, it is 288230376151711744
or
Math.pow(2,57)=144115188075855870
while it really equals 144115188075855872
it just throws that number without any warning.
i would understand if it stopped going above some number in case of maximum value reached. however, this seems to calculate the first n digits correctly and then go wrong at the very end of the digits only
You've ran out of Number type display precision. The trick is that with powers of 2 the actual value stored in the variable will be precise, while when you'll trace it the engine will truncate the displayed value by 16 digits, as it divides by 10 in process, and leftovers will eventually hit "machine zero" if compared to original value taken without exponential part. This is made to prevent white noise generated by imprecise floating-point division to be displayed. You can work around this issue if you'll advance to big integers/floating point numbers, that store more bits than a double precision number.