I have some time series data and I would like to weight my data, so that recent observations are weighted higher than older observations. Therefor I'm looking for a parametric weighting function, which satisfy a few properties. It should look like this:
weighting(time, minTime, maxTime, minWeight, slope) = ?
whereby
time is obviously the time of the observation to weight and should be between minTime and maxTime (time >= minTime, time <= maxTime),
minTime is the oldest observation time,
maxTime is the newest observation time,
minWeight is the minimum weight to return (and also the intercept with the weight axis; interval: [0,1]),
slope adjusts the shape of the curve.
Output:
The output should be in the interval [minWeight, 1.0].
Does anyone have an idea, how this weighting function might look like, or some hints or code examples / pseudocode?
Some functions I have looked at:
A root or power functions like
f(x) = x^(n/m), if n < m -- root function
f(x) = x^(n/m), if n = m -- linear function
f(x) = x^(n/m), if n > m -- power function
Exponential, logarithmic or sigmoid functions...
Rescaling (min-max normalization) also satisfies a few of those properties:
rescaling(time, minTime, maxTime) = (time - minTime) / (maxTime -
minTime)
This weighting function provides a weight in the interval [0, 1]. But the curve shape is always linear (and cannot be adjusted) and the minimum value is always 0 (I also would like to adjust that).
I guess I'm too stupid to get all the parts together. Can someone help?
The problem can be simplified by introducing new axes, t' and w', as illustrated here
With these coordinates the equations are simple:
w'^2 = t' - high-score
w' = t' - normal-score
w' = t'^2 - low-score
so, it only remains to replace w' with: (w - w0)/(1 - w0) and t' with: (t - t0)/(t1 - t0) to get:
(w - w0)^2/(1 - w0)^2 = (t - t0)/(t1 - t0) - high-score
(w - w0)/(1 - w0) = (t - t0)/(t1 - t0) - normal-score
(w - w0)/(1 - w0) = (t - t0)^2 / (t1 - t0)^2 - low-score
Now we have to solve for w:
w = w0 + (1 - w0)sqrt((t - t0)/(t1 - t0)) - high-slope
w = w0 + (1 - w0)(t - t0)/(t1 - t0) - normal-slope
w = w0 + (1 - w0)(t - t0)^2 / (t1 - t0)^2 - low-slope
The very same technique can be used if you choose another functions instead of sqrt() and ^2.
I found this really well explained calculation but I really don't know if it's correct or not. I've seen a lot of posts on the subject I haven't seen one calculate for 5.
The most important part which is the following code, but I cannot understand the math can't tell if its correct or not.
CREATE
TRIGGER `update_rating` BEFORE UPDATE ON `ratings`
FOR EACH ROW
SET new.total = new.positive + new.negative,
new.stars = ROUND( (((new.positive / new.total) * 4) + 1) * 2, 0) / 2,
new.lower_bound = ((new.positive + 1.9208) / (new.positive + new.negative) - 1.96 * SQRT((new.positive * new.negative) / (new.positive + new.negative) + 0.9604) / (new.positive + new.negative)) / (1 + 3.8416 / (new.positive + new.negative))
I need to know if it's correct or not. And it be really useful some explanation for how that formula works.
ps: Please do not mark this as a repost, I wrote a similar question but that one was more for sql conversion to c# and did not get the response for sql verification.
Theorem 7
When β = 2, if m and n are integers with |m| < 2^(p - 1) and n has the special form n = 2^i + 2^j, then (m n) n = m, provided floating-point operations are exactly rounded.
If I post the whole proof here, it will be long, unreadable and ugly. So please click the link on the right, press ctrl + F, and find Theorem 7. There it is! Goldberg91
OK, I have to say, at least from my own perspective, the proof of theorem 7 is too weird to comprehend, although the author claims that it's ingenious.
What I can only understand is that m has at most 1 bit right of the binary point. Yes, I know that, but why so n*qbar will round to m consequently? I can't also understand the so-called "halfway case" and almost everything from that line on.
Any help is welcome, thank you in advance.
EDIT:
Interestingly, the first comment below has solved all my questions in a roll, and the second comment suggests me to narrow my post.
That's right. It's inhuman to ask a person to explain the whole proof. So now my question becomes this:
Why the initial unscaled m's low-order bit was 0(found in the paragraph below formula 9)? Shouldn't the most significant digit be zero instead of the least significant one? Does this have something to do with 'Big-endian' or 'Little-endian'?
First off, let's scale n by 2. n should be made greater than or equal to 2^p - 1, and less than 2^p. The scaled n will be donated by n'. Scaling won't make any difference. It is exponent that is modified, since numbers are in binary, and we just have to focus on the significand / mantissa.
Next, m is scaled, producing m', so that q' = m'/n', less than 1 and greater than 1/2 (In fact, I think there should be 1/2 < q' ≤ 1). Scaling like these are possible because the upper bounds are exactly double the lower bounds, and β = 2.
As we can see, 2^p - 1 ≤ n' < 2^p and 1/2 < q' < 1, while m' = q' * n'. Since q' and n' are positive, the maximum of m' is the product of n' and q''s maximums, that is 2^p * 1 = 2^p. Similarly, the minimum of m' is 2^p - 1 * 1/2 = 2^p - 2.
Now that 2^p - 2 < m' < 2^p, we can say p - 2 < log2(m') < p, so the number of digits to the right of the binary point will be either p-1 (this happens when log2(m') is between p-2 and p-1) or p (when log2(m') is greater than or equal to p-1). Thus, m' has at most one bit to the right of the binary point.
As Mark Dickinson has said, as a result, the difference between m' and the next precision-p float up/down is at least 1/2. So to show that a quantity will round to m', it's enough to show that it's within 1/4 of m'.
Besides that, "the halfway case", namely the case where that quantity is exactly 1/4 from m', worth a separate discussion: Since the initial unscaled m had |m| < 2*p - 1, it has at least one bit to the right of the binary point, due to the same reason as mentioned above. m is an integer, so all digits to the right of the binary point are zero. Certainly, its low-order bit is 0 for that reason. Because scaling doesn't have effect on the significand / mantissa, the low-order bit of m' is also 0.
Consequently, using round to even, which is adopted by the original author by writing "Throughout the rest of this paper, round to even will be used." above (You can use ctrl + F again to find it), m' + 1/4 (0.01 in binary) will round to m', since 0 is even.
That is, if q̄ = m n, to prove the theorem requires showing that
|n' * q̄ - m'| ≤ 1/4.
Update 1
q' is a rational number, and q' < 1, so we can suppose that q' = 0.q1 q2... in binary, where qi, where i= 1,2,3... are single digits that contains 0 or 1. Let q̂ = 0.q1 q2 ... qp 1. Be careful, this token is "q-hat", not "q-bar", and it's a new variable I've just introduced in.
Now, if we shift q̂ left by p + 1 digits, we'll get an integer, namely q1 q2 ... qp 1, because all bits are to the left of the binary point. I'll use N to donate this integer hereafter. As a result, |q̂ - q'| = |N / 2^(p + 1) - m' / n'|.
The lower-bit of N is 1, so we know N = 1 + (qp * 2^1 + qp-1 * 2^2 + ... + q1 * 2^p). Obviously, N is an odd integer. Originally, n = 2^i + 2^j. Since scaling a number is just multiplying or dividing it by 2, n' is still a sum of two exponents of 2. Let them be n' = 2^i' + 2^j'. For convenience, it's assumed that i' ≥ j'.
2^p - 1 ≤ n' < 2^p, so 2^p - 1 ≤ 2^i' + 2^j' < 2^p. As a result, i', which accounts for a higher proportion of n', must be p - 1. To make n' less than 2^p, j' mustn't equal to i'. For readability, let k = j', so k ≤ p - 2. Thus,
. (Replace all n by n', m by m')
I suggest you to use some scratch papers to verify this formula on your own.
Update 2
Take a look at the numerator |(2^(p - 1 - k) + 1) * N - 2^(p + 1 - k) * m'|. As we have proved, k ≤ p - 2, so p - 1 - k ≥ 1, ensuring 2^(p - 1 - k) and 2^(p + 1 - k) to be even. Both (2^(p - 1 - k) + 1) and N are odd, so (2^(p - 1 - k) + 1) * N is odd, while 2^(p + 1 - k) * m' is even, and an odd integer can't equal to an even one. Therefore, (2^(p - 1 - k) + 1) * N - 2^(p + 1 - k) * m' is a non-zero integer. It's absolute value, namely the numerator, is consequently guaranteed to be equal to or greater than 1. Hence,
|q̂ - q'| ≥ 1 / (n' * 2^(p + 1 - k)).
q' < 1, and q̄ < 1, so q' * q̄ < 1. As a result, (m' * q') * q̄ < m', that is, n' * q̄ < m. Consequently,
|n' * q̄ - m'|
= m' - n' * q̄
= n' * (q' - q̄)
/* Since q̄ only has a precision of p, it will be 0.q1 q2 ... qp. So q̄ = q̂ - 2^(- p - 1). */
= n' * {q' - [q̂ - 2^(- p - 1)]}
= n' * [q' - q̂ + 2^(- p - 1)]
/* Assume that q' < q̂. The case q > q̂ is not discussed. */
= n' * [- |q' - q̂| + 2^(- p - 1)]
/* Since |q̂ - q'| ≥ 1 / (n' * 2^(p + 1 - k)), - |q̂ - q'| ≤ - 1 / (n' * 2^(p + 1 - k)). So */
≤ n' * {- 1 / [n' * 2^(p + 1 - k)] + 2^(- p - 1)}
= n' * {2^(- p - 1) - 1 / [n' * 2^(p + 1 - k)]}
/* We know n' = 2^i' + 2^j' = 2^(p - 1) + 2^k */
= [2^(p - 1) + 2^k] * {2^(- p - 1) - 1 / {[2^(p - 1) + 2^k] * 2^(p + 1 - k)}}
/* The equation is becoming less as less readable. For brevity, insignificant algebraic steps are omitted. */
= 2^-2 + 2^(- p - 1 + k) - 1 / 2^(p + 1 - k)
= 1/4
By now, |n' * q̄ - m'| ≤ 1/4 have been established. As mentioned above, this proves the theorem.
Q.E.D.
Questions that still remain:
Why "the case q > q̂ is similar" ? I think things will be totally different without that minus!
I don't understand why the result of:
4 / 3 * Math.PI * Math.pow(radio,3)
is different of:
(4 * Math.PI * Math.pow(radio,3)) / 3
I use this in a program to calculate the sphere's volume.
First, lets insert some extra brackets ... for illustrative purposes:
This code:
4 / 3 * Math.PI * Math.pow(radio,3)
is equivalent to
((4 / 3) * Math.PI) * Math.pow(radio,3)
In words ... first we divide 4 by 3, then we multiply it by Pi, then we multiply by radio to the power of 3.
Now lets look at the first sub-expression (4 / 3). Since the operands are both integers, this is an integer division, and it produces an integer answer. And that answer is 1. The answer you really need there is 1.33333... but that isn't an integer.
How to fix it? Changing either or both of the numbers to double literals will cause the division to be treated as a floating point (double) division, and that will you the best possible approximation to 4 thirds.
4.0 / 3.0 * Math.PI * Math.pow(radio,3)
First of all check for precedence of operator.
In your case precedence order is :
() > * > / (Note : this precedence is for C operators)
So for above operations result must be same for both cases.
Let's take
Case 1 :
4 / 3 * Math.PI * Math.pow(radio,3)
Here Math.PI = 3.14
Suppose Math.pow(radio , 3) = 10
In this scenario first multiplication will happen
result = 4*3.14*10 = 125.60000000000001
125.600/3 = 41.86666666666667
Case 2 :
'(4 * Math.PI * Math.pow(radio,3)) / 3'
Here first it will calculate () block then / operator
i.e (4 * Math.PI * Math.pow(radio,3))
Here again result will be same as before
Why does the following return .500000 rather than .500?
SELECT 50 / CAST(100 AS decimal(3))
Because you are casting the divisor not the result. And you have not specified precision.
See this for examples:
SELECT 50 / CAST(100 AS decimal(3))
,50 / CAST(100 AS decimal(6,3))
,CAST (50 / CAST(100 AS decimal(3)) AS decimal(3))
,CAST (50 / CAST(100 AS decimal(3)) AS decimal(6,3))
Precision, Scale, and Length (Transact-SQL)
Precision is the number of digits in a number. Scale is the number of
digits to the right of the decimal point in a number. For example, the
number 123.45 has a precision of 5 and a scale of 2.
DECIMAL uses a default precision of 18 and scale of 3.
Therefore when you case to DECIMAL(3) you get a DECIMAL(3,0). Then you use it as a divisor and SQL Server follows its rules for the resultant scale.
Operation Result precision Result scale
e1 / e2 p1 - s1 + s2 + max(6, s1 + p2 + 1) max(6, s1 + p2 + 1)