I've searched the existing threads and can't figure out where I'm doing it wrong...
I'm trying to get the average of two values and then round the average to the nearest TEN (10).
My expression is:
=ROUND((Fields!Cyl1Stress.Value + Fields!Cyl2Stress.Value) / 2, -1)
The above returns an error.
Rounding to the nearest TENTH works fine, but as soon as I change the 1 to -1, I get the error.
Where Cyl1Stress.Value = 7600 and Cyl2Stress.Value = 7490.
=ROUND((Fields!Cyl1Stress.Value + Fields!Cyl2Stress.Value) / 2, **1**)
The above code returns 7545.
But I need the result to be 7550, so I change my formula to:
=ROUND((Fields!Cyl1Stress.Value + Fields!Cyl2Stress.Value) / 2, **-1**).
This one returns an error.
Can't figure out why this isn't working!
Text Box Properties are Numeric, 0 decimal places.
I don't believe you can use a negative value for the Digits argument of Round. The documentation indicates:
The number of fractional digits in the return value. For Decimal values, it can range from 0 to 28. For Double values, it can range from 0 to 15.
You can accomplish what you are looking to do by dividing the value by 10, round to the nearest whole number, and then multiply by 10. Also note that if you want a value ending in 5 to round up to the next 10, you will need to add a third argument to Round to indicate this, otherwise I believe the default behavior rounds to the nearest even number. This would cause your example to round to 7540 instead of 7550.
This expression should be what you are looking for:
=ROUND(((Fields!Cyl1Stress.Value + Fields!Cyl2Stress.Value) / 2) / 10, 0, MidpointRounding.AwayFromZero) * 10
For your example values, this turns 7545 into 754.5, rounds to the nearest whole number and rounds it away from zero instead of to the nearest even number (755), then multiplies by 10 (7550).
Related
I am using the below query on the below data. However, when I do this I am getting the error "Data truncated for column 'strength' at row 1" for every column and row. I researched it a little, and as far as I can tell, most people are getting this error because they are trying to use text or char. I have never seen this warning before and I am getting the expected results, but with 4,700 warnings.
UPDATE userstats
SET strength = (strength * .999),
agility = (agility * .999),
guard = (guard * .999),
labour = (labour * .999),
IQ = (IQ * .999)
WHERE gym_train_since_cron = 0
Any help would be greatly appreciated!
strength * .999 is resulting in a number with more than 4 decimal places. That goes for all of these calculations.
To avoid the warnings, you can either ROUND or TRUNCATE the result of the calculation. For example: SET strength = ROUND(strength * .999, 4) or SET strength = TRUNCATE(strength * .999, 4).
You may wonder what the difference between the two functions are; it's the rounding behavior. For TRUNCATE it will round a number towards 0, whereas ROUND, depending on the numeric data type (exact or approximate) which in your case is decimal (an exact type), the following occurs (taken from Rounding Behavior):
For exact-value numbers, ROUND() uses the “round half up” rule: A value with a fractional part of .5 or greater is rounded up to the next integer if positive or down to the next integer if negative. (In other words, it is rounded away from zero.) A value with a fractional part less than .5 is rounded down to the next integer if positive or up to the next integer if negative.
To demonstrate, here's an example using the strength value for user 4898 from your sample data:
strength * .999 = 16331143.7521566 -- Over 4 decimal places, hence the warning
ROUND(strength * .999, 4) = 16331143.7522 -- Rounds up
TRUNCATE(strength * .999, 4) = 16331143.7521 -- Rounds down
I found at least two numbers for which ActionScript's Number.toExponential(20) returns incorrect results.
The most obvious value is 0.
trace(Number(0).toExponential(20)); // 0.00000000000000000000e-16
trace(Number(0).toExponential(2)); // 0.00e-16
trace(Number(0).toExponential(1)); // 0.0e-16
trace(Number(0).toExponential(0)); // 1e-15 - even worse!
I've made a class Double for playing around with Numbers (which are IEEE 754 double-precision binary floating point format numbers). It takes a Number and extracts all the bits into sign, bits and significand (the latter is printed including the implicit leading bit), or can generate a Number provided sign, bits and significand. There are methods to get the next and prior nearest representable numbers.
Here's what 0 looks like internally:
[Double number=0.00000000000000000000e-16 sign=1 exponent=-1023 isZero
significand=00000000000000000000000000000000000000000000000000000]
The other incorrect results are the nearest representable Number greater than Number.MAX_VALUE / 2 and Number.MAX_VALUE / 4.
// Number.MAX_VALUE / 2.
trace(Number(8.98846567431157854072e+307).toExponential(20)); // 8.98846567431157854072e+307
// Nearest representable Number greater than Number.MAX_VALUE / 2. Printed incorrectly!
trace(Number(8.98846567431157953864e+307).toExponential(20)); // 0.e+327
// A Number two ULPs greater than Number.MAX_VALUE / 2.
trace(Number(8.98846567431158153448e+307).toExponential(20)); // 8.98846567431158153448e+307
// Nearest representable Number greater than Number.MAX_VALUE / 4. Printed incorrectly!
trace(Number(4.49423283715578976932e+307).toExponential(20)); // 0.e+327
Here's what these 4 numbers look like internally:
[Double number=8.98846567431157854072e+307 sign=1 exponent=1022
significand=11111111111111111111111111111111111111111111111111111]
[Double number=0.e+327 sign=1 exponent=1023
significand=10000000000000000000000000000000000000000000000000000]
[Double number=8.98846567431158153448e+307 sign=1 exponent=1023
significand=10000000000000000000000000000000000000000000000000001]
[Double number=0.e+327 sign=1 exponent=1022
significand=10000000000000000000000000000000000000000000000000000]
So far all three numbers that are printed incorrectly have a 0 significand (with implicit leading 1), and the exponent values of 1023 (Emax for normal numbers), 1022 (Emax - 1), or -1023 (special exponent value for ±zero and subnormal numbers, Emin - 1).
Looks like toExponent doesn't correctly handle at least these three cases
. Are there any other numbers that would print incorrectly?
Trying to check if toExponential would produce correct results with other precision values.
trace(Number(8.98846567431157953864e+307).toExponential(2)); // 0.e+309
trace(Number(8.98846567431157953864e+307).toExponential(1)); // 0.e+308
trace(Number(8.98846567431157953864e+307).toExponential(0)); // 1e+308 - less bad!
If anyone interested in my Double class I will post it. I've also written faster implementations next(x:Number) and prior(x:Number) functions which work pretty fast without messing with internal bits of the numbers.
References:
https://en.wikipedia.org/wiki/Double-precision_floating-point_format
I have a calculated field in my table called C. its the result of A-B=C. A & B are number fields (single, fixed). I have having trouble setting up C as a calculated (Decimal Field).
The precision / decimal places seem to work perfectly, I can modify them freely. But no matter what I do to "SCALE". It always seems to return to "0". I need it to be 2 since all my data in my reports are rounding off at the wrong locations giving me hole numbers.
As you can see "scale = 0", no matter what I do to this number. it will always revert to "0". Why is that?
You can’t change the scale in a calculated field, because it takes the values and settings from the calculation.
So the fact of a scale of 0 should not matter. The resulting number if it needs decimal places will (should) have the decimal value. The setting is IGNORED
I mean, if the calculation is:
2 x 3 = 6
Then you get 6.
If you have 4 / 3 = 1.3333
Then, in your case you get:
1.33333333333333
And you WILL get the above EVEN if the scale = 0. So the scale setting is NOT used nor available in a calculated field.
You are certainly free to round, or format the above result. And in fact you could (should) consider using the round() function in the actual calculation. So use something like:
Round([Field1] / [Field2],4)
And you thus get:
1.3333
If you have a binary number say 1010 (which is 10 in base 10), is saying that dividing by two will remove the first digit (making it end up as 010), true?
Basically how do you remove the first digit (i.e. if the binary number is 0 or 1, then it will end up as nothing)? I don't want code or anything, I just want to know like something like you divide or multiply by two.
Also do not consider any of the left most zeroes, of a binary number.
It works the same way as it does in base ten. The number 401, without its first digit, is 1. You've subtracted 400, no? Now, to divide by ten, you would SHIFT the digits to the right. 401 shifted right is 040. 401/10 = 40. Note that the 1 is discarded because we're working with integer division.
So in binary, it's exactly the same, but with powers of 2. Removing the first bit does not DIVIDE by two. It SUBTRACTS the value of its position. So 101b (which is 4+1 = 5), without its largest bit, is 001b, or 1 decimal. It's subtraction: 5 - 4 = 1.
To divide by two, you shift the bits to the right, just like in base 10. So 101b would become 010b, which is 2 decimal. 5/2 == 2 (we're dropping the fractional part since it's integer division)
Make sense? If you're ever confused about binary, just think of how the digits & positions work in base ten, and instead of powers of ten, use powers of two.
If by "first digit" you mean "first significant digit", then what you're looking for is something like number and not (1 shl (int(log number / log 2))), where and and not are the bitwise operations, shl means shift left, and int is rounding down (never up) to integer. log is just a logarithm, in any base (same base for both cases).
If by "first digit" you mean the digit in some nth position (let the rightmost position be 0, counting to the left), then you just do number and not (1 shl position).
Removing a digit is like changing it to 0. Changing 1010 to 0010 is accomplished by subtracting 1000: 1010 - 1000 = 0010.
I've looked all over and can't find this answer.
How many actual digits are there for a MySQL FLOAT?
I know (think?) that it truncates what's in excess of the FLOAT's 4 byte limit, but what exactly is that?
From the manual (emphasis mine):
For FLOAT, the SQL standard permits an optional specification of the
precision (but not the range of the exponent) in bits following the
keyword FLOAT in parentheses. MySQL also supports this optional
precision specification, but the precision value is used only to
determine storage size. A precision from 0 to 23 results in a 4-byte
single-precision FLOAT column. A precision from 24 to 53 results in an
8-byte double-precision DOUBLE column.
So up to 23 bits of precision for the mantissa can be stored in a FLOAT, which is equivalent to about 7 decimal digits because 2^23 ~ 10^7 (8,388,608 vs 10,000,000). I tested it here. You can see that 12 decimal digits are returned, of which only the first 7 are really accurate.
for those of you who think that MySQL treats floats the same as, for example JAVA, I got some SHOCKING news: MySQL degrades the available accuracy which is possible to a float, in order to hide from you decimal places which might be incorrect! Check this out:
JAVA:
public static void main(String[] args) {
long i = 16777225;
DecimalFormat myFormatter = new DecimalFormat("##,###,###");
float iAsFloat = Float.parseFloat("" + i);
System.out.println("long i = " + i + " becomes " + myFormatter.format(iAsFloat));
}
the output is
long i = 16777225 becomes 16,777,224
So far, so normal. Our example integer is just above 2^24 = 16777216. Due to the 23 bit mantissa, between 2^23 and 2^24, a float can hold every integer. Then from 2^24 to 2^25, it can hold only even numbers, from 2^25 to 2^26 only numbers divisible by 4 and so on (also in the other direction: from 2^22 to 2^23, it can hold all multiples of 0.5). As long as the exponent isn't out of range, that's the rule of what a float can store.
16777225 is odd, so the "float version" is one off, because in that range (from 2^24 to 2^25) the "step size" of the float is 2.
And now, what does MySQL make of it.
Here is the fiddle, in case you don't believe me (I wouldn't)
http://www.sqlfiddle.com/#!2/a42e79/1
CREATE TABLE IF NOT EXISTS `test` (
`test` float NOT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
INSERT INTO `test`(`test`) VALUES (16777225)
SELECT * FROM `test`
result:
16777200
the result is off by 25 rather than 1, but has the "advantage" of being divisible by 100. Thanks a lot.
I think I understand the "philosophy" behind this utter nonsense, but I can't say I approve. Here is the "reason":
They don't want you to see the decimal places which could be wrong, which they accomplish by rounding the "actual" float value (as it is in JAVA and according to the industry standard) to some suitable power of ten.
In the example, if we leave it as it is, the last digit is wrong, without being a zero, and we can't have that.
Then, if we round to multiples of ten, the correct value would be 16777230, while what the "actual" float would be rounded to 16777220. Now, the 7th digit is wrong (it wasn't wrong before, but now it is.) And it's not zero. We can't have that. Better round to multiples of 100. Now both the correct value and the "actual" float value round to 16777200. So you see only the 6 correct digits. You don't want to know the "24" at the end, telling you (since the step size is 2 in that range) that your original number must have been between 1677723 and 1677725. No, you don't want to know that; those 2 numbers differ in the 7th digit after rounding to the 7th digit, so you can't know the "proper" 7th digit, and hence you want to stop at the 6th digit. Anyway, that's what they think you want at MySQL.
So their goal is to round to some number of decimal digits (namely, 6), such those 6 digits are always "correct", in that you'd have gotten the same 6 digits if you'd rounded the original exact number (before converting it to a float) to 6 digits. And since log_base10(2^23) = 6.92, rounded down 6, I can see why they think that this will always work. Tragically, not even that is true.
example:
i = 33554450
the number is between 2^25 and 2^26, so the "float version" (that is the JAVA float version, not the MySQL float version) of it is the closest multiple of 4 (the smaller one, if it's right in the middle), so that is
i_as_float = 33554448
i rounded to 6 decimals (i.e. to multiples of 100, since it's an 8 digit number) gives 33554500.
i_as_float rounded to 6 decimals gives 33554400
Oops! those differ at the 6th digit! But don't tell the MySQL people. They might just start "improving" 16777200 towards 16777000.
UPDATE
other databases don't do it like that.
fiddle: http://www.sqlfiddle.com/#!15/d9144/1