In Access 2002 VBA, what datatype can be used to limit the precision of a double result? - ms-access

I'm performing a calulation that results in a floating point number. When I try and write it to a Access field, I get this error:
Run-time error '3759': Scaling of decimal value resulted in data truncation
I've intentionally limited the access field to a fixed precision. I was hoping that the value would be automatically truncated, but instead it's throwing this error - how can I explicitly change the precision of the value in VBA to avoid this error?
Code:
value = X / Y
With myTAble
.AddNew
!calc = value
.Update
End With

You could try using VBA's Round function, as in:
!calc = Round(value, 2)
Replace the 2 with however many decimal places you want.
Depending on the scale and precision of your numbers, you might still encounter the 3759 error if Round returns a value that cannot be expressed exactly as a Double floating point number (and so ends up having more decimal places than you asked for). A more robust approach might be to use something like:
!calc = CDec(FormatNumber(value, 2))
Replace the 2 with however many decimal places you want.
FormatNumber will round the number and convert it to a string (with only the number of decimal places that you specify), and CDec will convert the string into a Decimal.

How about this:
value = X / Y
With myTAble
.AddNew
!calc = Fix(value*10^decimalPlaces)/10^decimalPlaces
.Update
End With
Where decimalPlaces is the number of decimal places that you wish to work with.
Fix truncates the number. If you wish some kind of rounding, try Clng.

Related

Ms Access Database Field losing decimal right side digits

I want to store decimal value into my field when you divide
1120/90 = 12.444444444444444444444444444444
This long but I am losing right side digits, I am only getting 13 right side digits - like that:
12.4444444444444
But when I multiply this back again:
12.4444444444444 x 90 = 1119.999999999996
This is not the correct answer; the correct answer is
12.444444444444444444444444444444 x 90 = 1120
Please help me here.
Thanks
You can use Decimal for this:
? CDec(1120) / CDec(90)
12.444444444444444444444444444
? (CDec(1120) / CDec(90)) * 90
1120
I banged on this for a long time but couldn't find a way to prove the following. assuming any operands can be cast to the integer type without loss of information (1120 & 90 can) then you usually must do the calculation simultaneously: The floating point division operator / handles integer like operands correctly but returns the type double. chained operations are also handled correctly if done all at once. Hence (1120/90) * 90 is calculated correctly as 1120 but also, typename(1120/90*90) returns the double datatype. if you store the intermediate value 1120/90 you get a double of 12.44444 repeating which isn't quite 1120/90.
My suggestion is to store both the operands rather than reducing them to one number. Then do the calculation all at once.

problems handling significant digits in mysql converting float to double

I am inserting data from one table into another in a MariaDB database, where the column in the first table is FLOAT, and in the second it's DOUBLE. The data can have values of any size, precision and decimal places.
Here is what happens to the values when I do a straight-forward copy:
INSERT INTO data2 (value) SELECT value FROM data1
The values are given random extra significant figures:
FLOAT in data1 DOUBLE in data2
-0.000000000000454747 -0.0000000000004547473508864641
-122.319 -122.31932830810547
14864199700 14864220160
CAST(value AS DECIMAL(65,30)) generates exactly the same values as col 2 above, except I see trailing zeroes.
Yet when I just do
UPDATE data2 SET value = 14867199700 WHERE id = 133025046;
the DOUBLE value is accepted.
Do I have to export all the value to an SQL script and re-import them? Isn't there a better way?
Despite hours trying to experimenting with the issue, I'm not much closer to a solution, despite its limited nature. I can see this is problem that besets all technologies, not just MariaDB or databases, so I have probably just missed the answer somewhere. Stackoverflow is desperately trying to guide to a solution with new suggestion features I hadn't seen before, but unfortunately they are no help, like the other suggested answers.
Your test case is flawed. You are feeding in decimal digits, and not testing just the transfer of FLOAT to DOUBLE.
UPDATE tbl SET double_col = float_col will always copy exactly the same value. This because the DOUBLE representation is a superset of the FLOAT representation (53 vs 24 bits of precision; etc).
Literal, with decimal places: UPDATE tbl SET double_col = 123.456 will mangle the number because of rounding from decimal to DOUBLE. Ditto for float_col. Furthermore, the mangled results will be different!
Hole number literal: UPDATE tbl SET double_col = 14867199700 will be stored exactly. But if you put that same literal into a FLOAT, it will be rounded to 24 bits, so it cannot be stored exactly. You lose exactness at about 7 significant digits for FLOAT and about 16 for DOUBLE. The literal in this example has 9 significant digits (after ignoring trailing zeros).
That's just a sampling of the nightmares you can get into.
You must consider FLOAT and DOUBLE to be approximate. You should never compare for equality; you don't know what might have messed with the last bit of the value.
Also, you should not try to guess when MySQL will perform expressions in DECIMAL instead of DOUBLE.
And, keep in mind that division is usually imprecise due to rounding to some number of bits or decimals.
The "mantissa" of 14864199700 is
1.10111010111111001101100 (binary of FLOAT : 24 bits including 'hidden' leading bit)
1.1011101011111100110110000000101000000000000000000000 (binary of DOUBLE)
^ ^ (lost in FLOAT)
Each of those is multiplied by the same power of 2. The DOUBLE gets exactly 14864199700. The FLOAT lost the bits pointed to.
You can play around with such at https://gregstoll.dyndns.org/~gregstoll/floattohex/
Believe it or not, things used to be worse. People would be billed for $0.00 -- due to rounding errors. Or results of what should have been 1+1 showed as 1.99999999.

How to convert exponential number to decimal numbers in action script 3?

I am facing the problem on multiplying two decimal number in flex.
When i am multiplying two decimal number the result is like a exponential number so i don't know how to get the decimal number as a result instead of getting an exponential number.
I am using the following codes to make a multiplication:
var num1:Number = 0.00005;
var num2:Number = 0.000007;
var result:Number = num1 * num2;
And in result variable i am getting the value as 3.5000000000000003E-10.
So i don't know how to get decimal number as result instead of getting an exponential number as above.
If anybody have knowledge how to resolve this problem please help me to solve.
You need to use the .toPrecision(precision:uint) method available in the Number class. This method takes one parameter which is :
precision:uint — An integer between 1 and 21, inclusive, that
represents the desired number of digits to represent in the resulting
string.
So simply do :
trace(result.toPrecision(2));
And you should get an output of 0.00000000035
Official documentation here :
http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/Number.html#toPrecision()
Hope this helps. Cheers.

Data truncated for column when multiplying numbers

I am using the below query on the below data. However, when I do this I am getting the error "Data truncated for column 'strength' at row 1" for every column and row. I researched it a little, and as far as I can tell, most people are getting this error because they are trying to use text or char. I have never seen this warning before and I am getting the expected results, but with 4,700 warnings.
UPDATE userstats
SET strength = (strength * .999),
agility = (agility * .999),
guard = (guard * .999),
labour = (labour * .999),
IQ = (IQ * .999)
WHERE gym_train_since_cron = 0
Any help would be greatly appreciated!
strength * .999 is resulting in a number with more than 4 decimal places. That goes for all of these calculations.
To avoid the warnings, you can either ROUND or TRUNCATE the result of the calculation. For example: SET strength = ROUND(strength * .999, 4) or SET strength = TRUNCATE(strength * .999, 4).
You may wonder what the difference between the two functions are; it's the rounding behavior. For TRUNCATE it will round a number towards 0, whereas ROUND, depending on the numeric data type (exact or approximate) which in your case is decimal (an exact type), the following occurs (taken from Rounding Behavior):
For exact-value numbers, ROUND() uses the “round half up” rule: A value with a fractional part of .5 or greater is rounded up to the next integer if positive or down to the next integer if negative. (In other words, it is rounded away from zero.) A value with a fractional part less than .5 is rounded down to the next integer if positive or up to the next integer if negative.
To demonstrate, here's an example using the strength value for user 4898 from your sample data:
strength * .999 = 16331143.7521566 -- Over 4 decimal places, hence the warning
ROUND(strength * .999, 4) = 16331143.7522 -- Rounds up
TRUNCATE(strength * .999, 4) = 16331143.7521 -- Rounds down

Microsoft Access - Decimal Scale stuck at 0

I have a calculated field in my table called C. its the result of A-B=C. A & B are number fields (single, fixed). I have having trouble setting up C as a calculated (Decimal Field).
The precision / decimal places seem to work perfectly, I can modify them freely. But no matter what I do to "SCALE". It always seems to return to "0". I need it to be 2 since all my data in my reports are rounding off at the wrong locations giving me hole numbers.
As you can see "scale = 0", no matter what I do to this number. it will always revert to "0". Why is that?
You can’t change the scale in a calculated field, because it takes the values and settings from the calculation.
So the fact of a scale of 0 should not matter. The resulting number if it needs decimal places will (should) have the decimal value. The setting is IGNORED
I mean, if the calculation is:
2 x 3 = 6
Then you get 6.
If you have 4 / 3 = 1.3333
Then, in your case you get:
1.33333333333333
And you WILL get the above EVEN if the scale = 0. So the scale setting is NOT used nor available in a calculated field.
You are certainly free to round, or format the above result. And in fact you could (should) consider using the round() function in the actual calculation. So use something like:
Round([Field1] / [Field2],4)
And you thus get:
1.3333