I am using the below query on the below data. However, when I do this I am getting the error "Data truncated for column 'strength' at row 1" for every column and row. I researched it a little, and as far as I can tell, most people are getting this error because they are trying to use text or char. I have never seen this warning before and I am getting the expected results, but with 4,700 warnings.
UPDATE userstats
SET strength = (strength * .999),
agility = (agility * .999),
guard = (guard * .999),
labour = (labour * .999),
IQ = (IQ * .999)
WHERE gym_train_since_cron = 0
Any help would be greatly appreciated!
strength * .999 is resulting in a number with more than 4 decimal places. That goes for all of these calculations.
To avoid the warnings, you can either ROUND or TRUNCATE the result of the calculation. For example: SET strength = ROUND(strength * .999, 4) or SET strength = TRUNCATE(strength * .999, 4).
You may wonder what the difference between the two functions are; it's the rounding behavior. For TRUNCATE it will round a number towards 0, whereas ROUND, depending on the numeric data type (exact or approximate) which in your case is decimal (an exact type), the following occurs (taken from Rounding Behavior):
For exact-value numbers, ROUND() uses the “round half up” rule: A value with a fractional part of .5 or greater is rounded up to the next integer if positive or down to the next integer if negative. (In other words, it is rounded away from zero.) A value with a fractional part less than .5 is rounded down to the next integer if positive or up to the next integer if negative.
To demonstrate, here's an example using the strength value for user 4898 from your sample data:
strength * .999 = 16331143.7521566 -- Over 4 decimal places, hence the warning
ROUND(strength * .999, 4) = 16331143.7522 -- Rounds up
TRUNCATE(strength * .999, 4) = 16331143.7521 -- Rounds down
Related
I've searched the existing threads and can't figure out where I'm doing it wrong...
I'm trying to get the average of two values and then round the average to the nearest TEN (10).
My expression is:
=ROUND((Fields!Cyl1Stress.Value + Fields!Cyl2Stress.Value) / 2, -1)
The above returns an error.
Rounding to the nearest TENTH works fine, but as soon as I change the 1 to -1, I get the error.
Where Cyl1Stress.Value = 7600 and Cyl2Stress.Value = 7490.
=ROUND((Fields!Cyl1Stress.Value + Fields!Cyl2Stress.Value) / 2, **1**)
The above code returns 7545.
But I need the result to be 7550, so I change my formula to:
=ROUND((Fields!Cyl1Stress.Value + Fields!Cyl2Stress.Value) / 2, **-1**).
This one returns an error.
Can't figure out why this isn't working!
Text Box Properties are Numeric, 0 decimal places.
I don't believe you can use a negative value for the Digits argument of Round. The documentation indicates:
The number of fractional digits in the return value. For Decimal values, it can range from 0 to 28. For Double values, it can range from 0 to 15.
You can accomplish what you are looking to do by dividing the value by 10, round to the nearest whole number, and then multiply by 10. Also note that if you want a value ending in 5 to round up to the next 10, you will need to add a third argument to Round to indicate this, otherwise I believe the default behavior rounds to the nearest even number. This would cause your example to round to 7540 instead of 7550.
This expression should be what you are looking for:
=ROUND(((Fields!Cyl1Stress.Value + Fields!Cyl2Stress.Value) / 2) / 10, 0, MidpointRounding.AwayFromZero) * 10
For your example values, this turns 7545 into 754.5, rounds to the nearest whole number and rounds it away from zero instead of to the nearest even number (755), then multiplies by 10 (7550).
I have a float column and I'm trying to save the value 1000000. It automatically turns it to 1e+06. How can I fix it?
To have the value returned formatted as 1000000, you can simply add integer zero to the column in the SELECT list.
SELECT mycol+0 AS mycol FROM mytable
MySQL is storing the value IEEE floating point format. (One bit for sign, a certain number of bits for the exponent, and a certain number of bits for the mantissa. This isn't really a MySQL thing, it's the standard representation for floating point values.)
As far as what's being returned, that's an issue with converting that value into string representation.
A floating point number has a large range of values. To represent the maximum value of a float (3.402823e+38) as a decimal value, that would require 38 decimal digits. The seven left most digits of the value are significant, but we'd need to add another 32 zeros/digits to indicate the position of the decimal point.
So, returning a string representation of scientific notation is a reasonable approach to returning a representation of the value.
Those two things are equivalent:
1e+06
= 1 * 10^6
= 1 * 1,000,000
= 1,000,000
It's called scientific notation (see here). mySQL uses it to display huge/tiny values, especially approximate values (see here).
You can use DOUBLE(8, 3) where 8 is the total no. of digits excluding the decimal point, and 3 is the no. of digits to follow the decimal.
I have a calculated field in my table called C. its the result of A-B=C. A & B are number fields (single, fixed). I have having trouble setting up C as a calculated (Decimal Field).
The precision / decimal places seem to work perfectly, I can modify them freely. But no matter what I do to "SCALE". It always seems to return to "0". I need it to be 2 since all my data in my reports are rounding off at the wrong locations giving me hole numbers.
As you can see "scale = 0", no matter what I do to this number. it will always revert to "0". Why is that?
You can’t change the scale in a calculated field, because it takes the values and settings from the calculation.
So the fact of a scale of 0 should not matter. The resulting number if it needs decimal places will (should) have the decimal value. The setting is IGNORED
I mean, if the calculation is:
2 x 3 = 6
Then you get 6.
If you have 4 / 3 = 1.3333
Then, in your case you get:
1.33333333333333
And you WILL get the above EVEN if the scale = 0. So the scale setting is NOT used nor available in a calculated field.
You are certainly free to round, or format the above result. And in fact you could (should) consider using the round() function in the actual calculation. So use something like:
Round([Field1] / [Field2],4)
And you thus get:
1.3333
I've looked all over and can't find this answer.
How many actual digits are there for a MySQL FLOAT?
I know (think?) that it truncates what's in excess of the FLOAT's 4 byte limit, but what exactly is that?
From the manual (emphasis mine):
For FLOAT, the SQL standard permits an optional specification of the
precision (but not the range of the exponent) in bits following the
keyword FLOAT in parentheses. MySQL also supports this optional
precision specification, but the precision value is used only to
determine storage size. A precision from 0 to 23 results in a 4-byte
single-precision FLOAT column. A precision from 24 to 53 results in an
8-byte double-precision DOUBLE column.
So up to 23 bits of precision for the mantissa can be stored in a FLOAT, which is equivalent to about 7 decimal digits because 2^23 ~ 10^7 (8,388,608 vs 10,000,000). I tested it here. You can see that 12 decimal digits are returned, of which only the first 7 are really accurate.
for those of you who think that MySQL treats floats the same as, for example JAVA, I got some SHOCKING news: MySQL degrades the available accuracy which is possible to a float, in order to hide from you decimal places which might be incorrect! Check this out:
JAVA:
public static void main(String[] args) {
long i = 16777225;
DecimalFormat myFormatter = new DecimalFormat("##,###,###");
float iAsFloat = Float.parseFloat("" + i);
System.out.println("long i = " + i + " becomes " + myFormatter.format(iAsFloat));
}
the output is
long i = 16777225 becomes 16,777,224
So far, so normal. Our example integer is just above 2^24 = 16777216. Due to the 23 bit mantissa, between 2^23 and 2^24, a float can hold every integer. Then from 2^24 to 2^25, it can hold only even numbers, from 2^25 to 2^26 only numbers divisible by 4 and so on (also in the other direction: from 2^22 to 2^23, it can hold all multiples of 0.5). As long as the exponent isn't out of range, that's the rule of what a float can store.
16777225 is odd, so the "float version" is one off, because in that range (from 2^24 to 2^25) the "step size" of the float is 2.
And now, what does MySQL make of it.
Here is the fiddle, in case you don't believe me (I wouldn't)
http://www.sqlfiddle.com/#!2/a42e79/1
CREATE TABLE IF NOT EXISTS `test` (
`test` float NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
INSERT INTO `test`(`test`) VALUES (16777225)
SELECT * FROM `test`
result:
16777200
the result is off by 25 rather than 1, but has the "advantage" of being divisible by 100. Thanks a lot.
I think I understand the "philosophy" behind this utter nonsense, but I can't say I approve. Here is the "reason":
They don't want you to see the decimal places which could be wrong, which they accomplish by rounding the "actual" float value (as it is in JAVA and according to the industry standard) to some suitable power of ten.
In the example, if we leave it as it is, the last digit is wrong, without being a zero, and we can't have that.
Then, if we round to multiples of ten, the correct value would be 16777230, while what the "actual" float would be rounded to 16777220. Now, the 7th digit is wrong (it wasn't wrong before, but now it is.) And it's not zero. We can't have that. Better round to multiples of 100. Now both the correct value and the "actual" float value round to 16777200. So you see only the 6 correct digits. You don't want to know the "24" at the end, telling you (since the step size is 2 in that range) that your original number must have been between 1677723 and 1677725. No, you don't want to know that; those 2 numbers differ in the 7th digit after rounding to the 7th digit, so you can't know the "proper" 7th digit, and hence you want to stop at the 6th digit. Anyway, that's what they think you want at MySQL.
So their goal is to round to some number of decimal digits (namely, 6), such those 6 digits are always "correct", in that you'd have gotten the same 6 digits if you'd rounded the original exact number (before converting it to a float) to 6 digits. And since log_base10(2^23) = 6.92, rounded down 6, I can see why they think that this will always work. Tragically, not even that is true.
example:
i = 33554450
the number is between 2^25 and 2^26, so the "float version" (that is the JAVA float version, not the MySQL float version) of it is the closest multiple of 4 (the smaller one, if it's right in the middle), so that is
i_as_float = 33554448
i rounded to 6 decimals (i.e. to multiples of 100, since it's an 8 digit number) gives 33554500.
i_as_float rounded to 6 decimals gives 33554400
Oops! those differ at the 6th digit! But don't tell the MySQL people. They might just start "improving" 16777200 towards 16777000.
UPDATE
other databases don't do it like that.
fiddle: http://www.sqlfiddle.com/#!15/d9144/1
I'm performing a calulation that results in a floating point number. When I try and write it to a Access field, I get this error:
Run-time error '3759': Scaling of decimal value resulted in data truncation
I've intentionally limited the access field to a fixed precision. I was hoping that the value would be automatically truncated, but instead it's throwing this error - how can I explicitly change the precision of the value in VBA to avoid this error?
Code:
value = X / Y
With myTAble
.AddNew
!calc = value
.Update
End With
You could try using VBA's Round function, as in:
!calc = Round(value, 2)
Replace the 2 with however many decimal places you want.
Depending on the scale and precision of your numbers, you might still encounter the 3759 error if Round returns a value that cannot be expressed exactly as a Double floating point number (and so ends up having more decimal places than you asked for). A more robust approach might be to use something like:
!calc = CDec(FormatNumber(value, 2))
Replace the 2 with however many decimal places you want.
FormatNumber will round the number and convert it to a string (with only the number of decimal places that you specify), and CDec will convert the string into a Decimal.
How about this:
value = X / Y
With myTAble
.AddNew
!calc = Fix(value*10^decimalPlaces)/10^decimalPlaces
.Update
End With
Where decimalPlaces is the number of decimal places that you wish to work with.
Fix truncates the number. If you wish some kind of rounding, try Clng.