I have the following code in Octave for implementing the composite trapezoid rule and for some reason the function only stalls whenever I execute it in Octave on f = #(x) x^2, a = 0, b = 4, TOL = 10^-6. Whenever I call trapezoid(f, a, b, TOL), nothing happens and I have to exit the Terminal in order to do anything else in Octave. Here is the code:
% INPUTS
%
% f : a function
% a : starting point
% b : endpoint
% TOL : tolerance
function root = trapezoid(f, a, b, TOL)
disp('test');
max_iterations = 10000;
disp(max_iterations);
count = 1;
disp(count);
initial = (b-a)*(f(b) + f(a))/2;
while count < max_iterations
disp(initial);
trap_0 = initial;
trap_1 = 0;
trap_1_midpoints = a:(0.5^count):b;
for i = 1:(length(trap_1_midpoints)-1)
trap_1 = trap_1 + (trap_1_midpoints(i+1) - trap_1_midpoints(i))*(f(trap_1_midpoints(i+1) + f(trap_1_midpoints(i))))/2;
endfor
if abs(trap_0 - trap_1) < TOL
root = trap_1;
return;
endif
intial = trap_1;
count = count + 1;
disp(count);
endwhile
disp(['Process ended after ' num2str(max_iterations), ' iterations.']);
I have tried your function in Matlab.
Your code is not stalling. It is rather that the size of trap_1_midpoints increases exponentionaly. With that the computation time of trap_1 increases also exponentionaly. This is what you experience as stalling.
I also found a possible bug in your code. I guess the line after the if clause should be initial = trap_1. Check the missing 'i'.
With that, your code still takes forever, but if you increase the tolerance (e.g. to a value of 1) your code return.
You could try to vectorize the for loop for speed up.
Edit: I think inside your for loop, a ) is missing after f(trap_1_midpoints(i+1).
After count=52 or so, the arithmetic sequence trap_1_midpoints is no longer representable in any meaningful fashion in floating point numbers. After count=1075 or similar, the step size is no longer representable as a positive floating point double number. That all is to say, the bound max_iterations = 10000 is ludicrous. As explained below, all computations after count=20 are meaningless.
The theoretical error for stepsize h is O(T·h^2). There is a numerical error accumulation in the summation of O(T/h) numbers that is of that size, i.e., O(mu/h) with mu=1ulp=2^(-52). Which in total means that the lowest error of the numerical integration can be expected around h=mu^(1/3), for double numbers thus h=1e-5 or in the algorithm count=17. This may vary with interval length and how smooth or wavy the function is.
One can expect the behavior that the error divides by four while halving the step size only for step sizes above this boundary 1e-5. This also means that abs(trap_0 - trap_1) is a reliable measure for the error of trap_0 (and abs(trap_0 - trap_1)/3 for trap_1) only inside this range of step sizes.
The error bound TOL=1e-6 should be met for about h=1e-3, which corresponds to count=10. If the recursion does not stop for count = 14 (which should give an error smaller than 1e-8) then the method is not accurately implemented.
Could someone tell me the logic behind exponenting binary numbers? For example, I want to take 110^10, but I don't know the logic behind it. If someone could supply me with that, it'd be a great help.. (And I want it to be done in pure binary with no conversions and no looping multiplication. Just logic...)
peenut is correct in that exponentiation doesn't care what base you're representing your numbers in, and I don't know what you mean by "just logic," but here's a stab at it.
A quick search over at Wikipedia reveals this algorithm. The basic ideas is to square your base, store the result, and then square the result and repeat. This will give you the factors of your answer, which you can then multiply together. I think of it as a "binary search"-flavored exponentiation algorithm since you can skip a lot of intermediate steps by squaring and storing.
Binary exponents are very easy. They are simply additions and shifts only.
the number 110 is where you start.
Working backwards from the number 10 - (i.e. 0) - it's a zero, so this means "do not add it in."
Now you shift left - so 110 becomes 1100
Now you work on the next bit of the 10 (i.e. 1) - it's a one, so this means "add this to the result" - it's 0 so far, because we didn't already add it, so the result is now 1100
there are no more bits to do - so the answer is 1100
If you were doing 110^110 - you would have one more to do - so - you again shift and get 11000 now.
The last bit is again a one, so now you add:
1100 +
11000 =
100100
110^10=1100 i.e. 6^2=12
110^110=100100 i.e. 6^6=36
Exponentiation is operation that is independent of actual textual representation of number (e.g. in base 2 - binary, base 10 - decimal).
Maybe you want to ask about binary XOR (eXclusive OR) operation?
Unfortunately the easiest way for your computer to handle simple exponents is your "looping multiplication" (or the naïve approach), which is the most rudimentary (and literal) way of handling it. As #user1561358 commented, it is NOT just binary adds and shifts. That is multiplication. To raise 66 (110110) the naïve approach has you multiplying the base n times (as below):
110
x 110
--------------
100100 = 36
x 110
--------------
11011000 = 216
x 110
--------------
10100010000 = 1296
x 110
--------------
1111001100000 = 7776
x 110
--------------
01011011001000000 = 46656
The simple code for a naïve multiplication is elegant for most applications:
long long binpow(long long a, long long b) {
if (b == 0)
return 1;
long long res = binpow(a, b / 2);
if (b % 2)
return res * res * a;
else
return res * res;
}
For larger or arbitrary exponents you can dramatically reduce the number of calculations by applying Horner's Method, explained in great detail in this video specifically calculating binary exponents.
In essence, you are just multiplying the bits with non-zero exponents. Let's look at 11021102, (or 66):
11021102 breaks down into the following exponents:
There is no "1" bit set so 61 won't be multiplied, but we do have the two and four bits to calculate:
6102 = 36
61002 = 1296
So, 66 = 36 x 1296 = 46656
The above code can be modified only slightly to check for non-zero exponents with a while {.. test:
long long binpow(long long a, long long b) {
long long res = 1;
while (b > 0) {
if (b & 1)
res = res * a;
a = a * a;
b >>= 1;
}
return res;
}
To really see the advantage of this let's try the binary exponentiation of
11121000000002, which is 7256.
The naïve approach would require us to make 256 multiplication iterations!
Instead, all the exponents except 2256 are zero, so they are skipped in the while loop. There is one single iterative calculation where a * a happens 256 times:
11121000000002 = (a 718 digit binary beginning with 11001101011....)
728 = 2213595400046048155450188615474945937162517050260073069916366390524704974007989996848003433837940380782794455262312607598867363425940560014856027866381946458951205837379116473663246733509680721264246243189632348313601
Within my daily work, I have got to maximize a particular function making use of fminsearch; the code is:
clc
clear all
close all
f = #(x,c,k) -(x(2)/c)^3*(((exp(-(x(1)/c)^k)-exp(-(x(2)/c)^k))/((x(2)/c)^k-(x(1)/c)^k))-exp(-(x(3)/c)^k))^2;
c = 10.1;
k = 2.3;
X = fminsearch(#(x) f(x,c,k),[4,10,20]);
It works fine, as I expect, but not the issue is coming up: I need to bound x within certain limits, as:
4 < x(1) < 5
10 < x(2) < 15
20 < x(3) < 30
To achieve the proper results, I should use the optimization toolbox, that I unfortunately cannot hand.
Is there any way to get the same analysis by making use of only fminsearch?
Well, not using fminsearch directly, but if you are willing to download fminsearchbnd from the file exchange, then yes. fminsearchbnd does a bound constrained minimization of a general objective function, as an overlay on fminsearch. It calls fminsearch for you, applying bounds to the problem.
Essentially the idea is to transform your problem for you, in a way that your objective function sees as if it is solving a constrained problem. It is totally transparent. You call fminsearchbnd with a function, a starting point in the parameter space, and a set of lower and upper bounds.
For example, minimizing the rosenbrock function returns a minimum at [1,1] by fminsearch. But if we apply purely lower bounds on the problem of 2 for each variable, then fminsearchbnd finds the bound constrained solution at [2,4].
rosen = #(x) (1-x(1)).^2 + 105*(x(2)-x(1).^2).^2;
fminsearch(rosen,[3 3]) % unconstrained
ans =
1.0000 1.0000
fminsearchbnd(rosen,[3 3],[2 2],[]) % constrained
ans =
2.0000 4.0000
If you have no constraints on a variable, then supply -inf or inf as the corresponding bound.
fminsearchbnd(rosen,[3 3],[-inf 2],[])
ans =
1.4137 2
Andrey has the right idea, and the smoother way of providing a penalty isn't hard: just add the distance to the equation.
To keep using the anonymous function:
f = #(x,c,k, Xmin, Xmax) -(x(2)/c)^3*(((exp(-(x(1)/c)^k)-exp(-(x(2)/c)^k))/((x(2)/c)^k-(x(1)/c)^k))-exp(-(x(3)/c)^k))^2 ...
+ (x< Xmin)*(Xmin' - x' + 10000) + (x>Xmax)*(x' - Xmax' + 10000) ;
The most naive way to bound x, would be giving a huge penalty for any x that is not in the range.
For example:
function res = f(x,c,k)
if x(1)>5 || x(1)<4
penalty = 1000000000000;
else
penalty = 0;
end
res = penalty - (x(2)/c)^3*(((exp(-(x(1)/c)^k)-exp(-(x(2)/c)^k))/((x(2)/c)^k-(x(1)/c)^k))-exp(-(x(3)/c)^k))^2;
end
You can improve this approach, by giving the penalty in a smoother way.
One of my former students sent me a message about this interview question he got while applying for a job as a Junior Developer.
There are two candidates running for president in a mock classroom election. Given the two percentages of voters, find out the least amount of possible voters in the classroom.
Examples:
Input: 50.00,50.00
Output: 2
Input: 25.00,75.00
Output: 4
Input: 53.23, 46.77
Output: 124 // The first value, 1138 was wrong. Thanks to Loïc for the correct value
Note: The sum of the input percentages are always 100.00%, two decimal places
The last example got me scratching my head. It was the first time I heard about this problem, and I'm kindof stumped on how to solve this.
EDIT: I called my student about the problem, and told me that he was not sure about the last value. He said, and I quote, "It was an absurdly large number output" :( sorry! I should've researched more before posting it online~ I'm guessing 9,797 is the output on the last example though..
You can compute these values by using the best rational approximations of the voter percentages. Wikipedia describes how to obtain these values from the continued fraction (which can be computed these using the euclidean algorithm). The desired result is the first approximation which is within 0.005% of the expected value.
Here's an example with 53.23%:
10000 = 1 * 5323 + 4677
5323 = 1 * 4677 + 646
4677 = 7 * 646 + 155
646 = 4 * 155 + 26
155 = 5 * 26 + 25
26 = 1 * 25 + 1
25 = 25* 1 + 0
Approximations:
1: 1 / 1
-> 1 = 100%
2: 1 / (1 + 1/1)
-> 1/2 = 50%
2.5: 1 / (1 + 1 / (1 + 1/6))
-> 7/1 = 53.75%
3: 1 / (1 + 1 / (1 + 1/7))
-> 8/15 = 53.33%
3.5: 1 / (1 + 1 / (1 + 1 / (7 + 1/3)))
-> 25/47 = 53.19%
4: 1 / (1 + 1 / (1 + 1 / (7 + 1/4)))
-> 33/62 = 53.23%
The reason we have extra values before the 3rd and 4th convergents is that their last terms (7 and 4 respectively) are greater than 1, so we must test the approximation with the last term decremented.
The desired result is the denominator of the first value which rounds to the desired value, which in this vase is 62.
Sample Ruby implementation available here (using the formulae from the Wikipedia page here, so it looks slightly different to the above example).
First you can notice that a trivial solution is to have 10.000 voters. Now let's try to find something lower than that.
For each value of N starting à 1
For Each value of i starting à 1
If i/N = 46.77
return N
Always choose the minimum of the two percentages to be faster.
Or faster :
For each value of N starting à 1
i = floor(N*46.77/100)
For j = i or i+1
If round(j/N) = 46.77 and round((N-j)/N) = 53.23
return N
For the third example :
605/1138 = .5316344464
(1138-605)/1138 = .4683655536
but
606/1138 = .5325131810
(1138-606)/1138 = .4674868190
It can't be 1138...
But 62 is working :
33/62 = .5322580645
(62-33)/62 = .4677419355
Rounded it's giving you the good values.
(After some extensive edits:)
If you only have 2 voters, then you can only generate the following percentages for candidates A and B:
0+100, 100+0, or 50+50
If you have 3 voters, then you have
0+100, 100+0, 33.33+66.67, 66.67+33.33 [notice the rounding]
So this is a fun problem about fractions.
If you can make 25% then you have to have at least 4 people (so you can do 1/4, since 1/2 and 1/3 won't cut it). You can do it with more (i.e. 2/8 = 25%) but the problem asks for the least.
However, more interesting fractions require numbers greater than 1 in the numerator:
2/5 = 40%
Since you can't get that with anything but a 2 or more in the numerator (1/x will never cut it).
You can compare at each step and increase either the numerator or denominator, which is much more efficient than iterating over the whole sample space for j and then incrementing i;
i.e. if you have a percentage of 3%, checking solutions all the way up in the fashion of 96/99, 97/99, 98/99 before even getting to x/100 is a waste of time. Instead, you can increment the numerator or denominator based on how well your current guess is doing (greater than or less than) like so
int max = 5000; //we only need to go half-way at most.
public int minVoters (double onePercentage) {
double checkPercentage = onePercentage;
if (onePercentage > 50.0)
checkPercentage = 100-onePercentage; //get the smaller percentage value
double i=1;
double j=1; //arguments of Math.round must be double or float
double temp = 0;
while (j<max || i<max-1) { //we can go all the way to 4999/5000 for the lesser value
temp = (i/j)*100;
temp = Math.round(temp);
temp = temp/100;
if (temp == checkPercentage)
return j;
else if (temp > checkPercentage) //we passed up our value and need to increase the denominator
j++;
else if (temp < checkPercentage) //we are too low and increase the numerator
i++;
}
return 0; //no such solution
}
Step-wise example for finding the denominator that can yield 55%
55/100 = 11/20
100-55 = 45 = 9/20 (checkPercentage will be 45.0)
1/1 100.0%
1/2 50.00%
1/3 33.33%
2/3 66.67%
2/4 50.00%
2/5 40.00%
3/5 60.00%
3/6 50.00%
3/7 42.86% (too low, increase numerator)
4/7 57.14% (too high, increase denominator)
4/8 50.00%
4/9 44.44%
5/9 55.56%
5/10 50.00%
5/11 45.45%
6/11 54.54%
6/12 50.00%
6/13 46.15%
6/14 42.86%
7/14 50.00%
7/15 46.67%
7/16 43.75%
8/16 50.00%
8/17 47.06%
8/19 42.11%
9/19 47.37%
9/20 45.00% <-bingo
The nice thing about this method is that it will only take (i+j) steps where i is the numerator and j is the denominator.
I cannot see the relevance of this question to a position as junior developer.
Then answer that jumped into my head was more of a brute-force approach. There can be at most 5001 unique answers because there 5001 unique numbers between 00.00 and 50.00 . Consequently, why not create and save a look-up table. Obviously, there won't be 5001 unique answer because some answers will be repeated. The point is, there are only 5001 valid fractions because we are rounding to two digits.
int[] minPossible = new int[5001];
int numSolutionsFound = 0;
N = 2;
while(numSolutionsFound < 5001) {
for(int i = 0 ; i <= N/2 ; i++) {
//compute i/N
//see if the corresponding table entry is set
//if not write N there and increment numSolutionsFound
}
N++;
}
//Save answer here
Now the solution is merely a table look up.
FWIW I realize the euclidean solution is "correct". But I'd NEVER come up with that mid interview. However, I'd know something like that was possible -- but I won't be able to whip it out on the spot.
Edit: Since it appears nobody is reading the original question this links to, let me bring in a synopsis of it here.
The original problem, as asked by someone else, was that, given a large number of values, where the sum would exceed what a data type of Double would hold, how can one calculate the average of those values.
There was several answers that said to calculate in sets, like taking 50 and 50 numbers, and calculating the average inside those sets, and then finally take the average of all those sets and combine those to get the final average value.
My position was that unless you can guarantee that all those values can be split into a number of equally sized sets, you cannot use this approach. Someone dared me to ask the question here, in order to provide the answer, so here it is.
Basically, given an arbitrary number of values, where:
I know the number of values beforehand (but again, how would your answer change if you didn't?`)
I cannot gather up all the numbers, nor can I sum them (the sum will be too big for a normal data type in your programming language)
how can I calculate the average?
The rest of the question here outlines how, and the problems with, the approach to split into equally sized sets, but I'd really just like to know how you can do it.
Note that I know perfectly well enough math to know that in math theory terms, calculating the sum of A[1..N]/N will give me the average, let's assume that there are reasons that it isn't just as simple, and I need to split up the workload, and that the number of values isn't necessarily going to be divisable by 3, 7, 50, 1000 or whatever.
In other words, the solution I'm after will have to be general.
From this question:
What is a good solution for calculating an average where the sum of all values exceeds a double’s limits?
my position was that splitting the workload up into sets is no good, unless you can ensure that the size of those sets are equal.
Edit: The original question was about the upper limit that a particular data type could hold, and since he was summing up a lot of numbers (count that was given as example was 10^9), the data type could not hold the sum. Since this was a problem in the original solution, I'm assuming (and this is a prerequisite for my question, sorry for missing that) that the numbers are too big to give any meaningful answers.
So, dividing by the total number of values directly is out. The original reason for why a normal SUM/COUNT solution was out was that SUM would overflow, but let's assume, for this question that SET-SET/SET-SIZE will underflow, or whatever.
The important part is that I cannot simply sum, I cannot simply divide by the number of total values. If I cannot do that, will my approach work, or not, and what can I do to fix it?
Let me outline the problem.
Let's assume you're going to calculate the average of the numbers 1 through 6, but you cannot (for whatever reason) do so by summing the numbers, counting the numbers, and then dividing the sum by the count. In other words, you cannot simply do (1+2+3+4+5+6)/6.
In other words, SUM(1..6)/COUNT(1..6) is out. We're not considering NULL's (as in database NULL's) here.
Several of the answers to that question alluded to being able to split the numbers being averaged into sets, say 3 or 50 or 1000 numbers, then calculating some number for that, and then finally combining those values to get the final average.
My position is that this is not possible in the general case, since this will make some numbers, the ones appearing in the final set, more or less valuable than all the ones in the previous sets, unless you can split all the numbers into equally sized sets.
For instance, to calculate the average of 1-6, you can split it up into sets of 3 numbers like this:
/ 1 2 3 \ / 4 5 6 \
| - + - + - | + | - + - + - |
\ 3 3 3 / \ 3 3 3 / <-- 3 because 3 numbers in the set
---------- -----------
2 2 <-- 2 because 2 equally sized groups
Which gives you this:
2 5
- + - = 3.5
2 2
(note: (1+2+3+4+5+6)/6 = 3.5, so this is correct here)
However, my point is that once the number of values cannot be split into a number of equally sized sets, this method falls apart. For instance, what about the sequence 1-7, which contains a prime number of values.
Can a similar approach, that won't sum all the values, and count all the values, in one go, work?
So, is there such an approach? How do I calculate the average of an arbitrary number of values in which the following holds true:
I cannot do a normal sum/count approach, for whatever reason
I know the number of values beforehand (what if I don't, will that change the answer?)
Well, suppose you added three numbers and divided by three, and then added two numbers and divided by two. Can you get the average from these?
x = (a + b + c) / 3
y = (d + e) / 2
z = (f + g) / 2
And you want
r = (a + b + c + d + e + f + g) / 7
That is equal to
r = (3 * (a + b + c) / 3 + 2 * (d + e) / 2 + 2 * (f + g) / 2) / 7
r = (3 * x + 2 * y + 2 * z) / 7
Both lines above overflow, of course, but since division is distributive, we do
r = (3.0 / 7.0) * x + (2.0 / 7.0) * y + (2.0 / 7.0) * z
Which guarantees that you won't overflow, as I'm multiplying x, y and z by fractions less than one.
This is the fundamental point here. Neither I'm dividing all numbers beforehand by the total count, nor am I ever exceeding the overflow.
So... if you you keep adding to an accumulator, keep track of how many numbers you have added, and always test if the next number will cause an overflow, you can then get partial averages, and compute the final average.
And no, if you don't know the values beforehand, it doesn't change anything (provided that you can count them as you sum them).
Here is a Scala function that does it. It's not idiomatic Scala, so that it can be more easily understood:
def avg(input: List[Double]): Double = {
var partialAverages: List[(Double, Int)] = Nil
var inputLength = 0
var currentSum = 0.0
var currentCount = 0
var numbers = input
while (numbers.nonEmpty) {
val number = numbers.head
val rest = numbers.tail
if (number > 0 && currentSum > 0 && Double.MaxValue - currentSum < number) {
partialAverages = (currentSum / currentCount, currentCount) :: partialAverages
currentSum = 0
currentCount = 0
} else if (number < 0 && currentSum < 0 && Double.MinValue - currentSum > number) {
partialAverages = (currentSum / currentCount, currentCount) :: partialAverages
currentSum = 0
currentCount = 0
}
currentSum += number
currentCount += 1
inputLength += 1
numbers = rest
}
partialAverages = (currentSum / currentCount, currentCount) :: partialAverages
var result = 0.0
while (partialAverages.nonEmpty) {
val ((partialSum, partialCount) :: rest) = partialAverages
result += partialSum * (partialCount.toDouble / inputLength)
partialAverages = rest
}
result
}
EDIT:
Won't multiplying with 2, and 3, get me back into the range of "not supporter by the data type?"
No. If you were diving by 7 at the end, absolutely. But here you are dividing at each step of the sum. Even in your real case the weights (2/7 and 3/7) would be in the range of manageble numbers (e.g. 1/10 ~ 1/10000) which wouldn't make a big difference compared to your weight (i.e. 1).
PS: I wonder why I'm working on this answer instead of writing mine where I can earn my rep :-)
If you know the number of values beforehand (say it's N), you just add 1/N + 2/N + 3/N etc, supposing that you had values 1, 2, 3. You can split this into as many calculations as you like, and just add up your results. It may lead to a slight loss of precision, but this shouldn't be an issue unless you also need a super-accurate result.
If you don't know the number of items ahead of time, you might have to be more creative. But you can, again, do it progressively. Say the list is 1, 2, 3, 4. Start with mean = 1. Then mean = mean*(1/2) + 2*(1/2). Then mean = mean*(2/3) + 3*(1/3). Then mean = mean*(3/4) + 4*(1/4) etc. It's easy to generalize, and you just have to make sure the bracketed quantities are calculated in advance, to prevent overflow.
Of course, if you want extreme accuracy (say, more than 0.001% accuracy), you may need to be a bit more careful than this, but otherwise you should be fine.
Let X be your sample set. Partition it into two sets A and B in any way that you like. Define delta = m_B - m_A where m_S denotes the mean of a set S. Then
m_X = m_A + delta * |B| / |X|
where |S| denotes the cardinality of a set S. Now you can repeatedly apply this to partition and calculate the mean.
Why is this true? Let s = 1 / |A| and t = 1 / |B| and u = 1 / |X| (for convenience of notation) and let aSigma and bSigma denote the sum of the elements in A and B respectively so that:
m_A + delta * |B| / |X|
= s * aSigma + u * |B| * (t * bSigma - s * aSigma)
= s * aSigma + u * (bSigma - |B| * s * aSigma)
= s * aSigma + u * bSigma - u * |B| * s * aSigma
= s * aSigma * (1 - u * |B|) + u * bSigma
= s * aSigma * (u * |X| - u * |B|) + u * bSigma
= s * u * aSigma * (|X| - |B|) + u * bSigma
= s * u * aSigma * |A| + u * bSigma
= u * aSigma + u * bSigma
= u * (aSigma + bSigma)
= u * (xSigma)
= xSigma / |X|
= m_X
The proof is complete.
From here it is obvious how to use this to either recursively compute a mean (say by repeatedly splitting a set in half) or how to use this to parallelize the computation of the mean of a set.
The well-known on-line algorithm for calculating the mean is just a special case of this. This is the algorithm that if m is the mean of {x_1, x_2, ... , x_n} then the mean of {x_1, x_2, ..., x_n, x_(n+1)} is m + ((x_(n+1) - m)) / (n + 1). So with X = {x_1, x_2, ..., x_(n+1)}, A = {x_(n+1)}, and B = {x_1, x_2, ..., x_n} we recover the on-line algorithm.
Thinking outside the box: Use the median instead. It's much easier to calculate - there are tons of algorithms out there (e.g. using queues), you can often construct good arguments as to why it's more meaningful for data sets (less swayed by extreme values; etc) and you will have zero problems with numerical accuracy. It will be fast and efficient. Plus, for large data sets (which it sounds like you have), unless the distributions are truly weird, the values for the mean and median will be similar.
When you split the numbers into sets you're just dividing by the total number or am I missing something?
You have written it as
/ 1 2 3 \ / 4 5 6 \
| - + - + - | + | - + - + - |
\ 3 3 3 / \ 3 3 3 /
---------- -----------
2 2
but that's just
/ 1 2 3 \ / 4 5 6 \
| - + - + - | + | - + - + - |
\ 6 6 6 / \ 6 6 6 /
so for the numbers from 1 to 7 one possible grouping is just
/ 1 2 3 \ / 4 5 6 \ / 7 \
| - + - + - | + | - + - + - | + | - |
\ 7 7 7 / \ 7 7 7 / \ 7 /
Average of x_1 .. x_N
= (Sum(i=1,N,x_i)) / N
= (Sum(i=1,M,x_i) + Sum(i=M+1,N,x_i)) / N
= (Sum(i=1,M,x_i)) / N + (Sum(i=M+1,N,x_i)) / N
This can be repeatedly applied, and is true regardless of whether the summations are of equal size. So:
Keep adding terms until both:
adding another one will overflow (or otherwise lose precision)
dividing by N will not underflow
Divide the sum by N
Add the result to the average-so-far
There's one obvious awkward case, which is that there are some very small terms at the end of the sequence, such that you run out of values before you satisfy the condition "dividing by N will not underflow". In which case just discard those values - if their contribution to the average cannot be represented in your floating type, then it is in particular smaller than the precision of your average. So it doesn't make any difference to the result whether you include those terms or not.
There are also some less obvious awkward cases to do with loss of precision on individual summations. For example, what's the average of the values:
10^100, 1, -10^100
Mathematics says it's 1, but floating-point arithmetic says it depends what order you add up the terms, and in 4 of the 6 possibilities it's 0, because (10^100) + 1 = 10^100. But I think that the non-commutativity of floating-point arithmetic is a different and more general problem than this question. If sorting the input is out of the question, I think there are things you can do where you maintain lots of accumulators of different magnitudes, and add each new value to whichever one of them will give best precision. But I don't really know.
Here's another approach. You're 'receiving' numbers one-by-one from some source, but you can keep track of the mean at each step.
First, I will write out the formula for mean at step n+1:
mean[n+1] = mean[n] - (mean[n] - x[n+1]) / (n+1)
with the initial condition:
mean[0] = x[0]
(the index starts at zero).
The first equation can be simplified to:
mean[n+1] = n * mean[n] / (n+1) + x[n+1]/(n+1)
The idea is that you keep track of the mean, and when you 'receive' the next value in your sequence, you figure out its offset from the current mean, and divide it equally between the n+1 samples seen so far, and adjust your mean accordingly. If your numbers don't have a lot of variance, your running mean will need to be adjusted very slightly with the new numbers as n becomes large.
Obviously, this method works even if you don't know the total number of values when you start. It has an additional advantage that you know the value of the current mean at all times. One disadvantage that I can think of is the it probably gives more 'weight' to the numbers seen in the beginning (not in a strict mathematical sense, but because of floating point representations).
Finally, all such calculations are bound to run into floating-point 'errors' if one is not careful enough. See my answer to another question for some of the problems with floating point calculations and how to test for potential problems.
As a test, I generated N=100000 normally distributed random numbers with mean zero and variance 1. Then I calculated their mean by three methods.
sum(numbers) / N, call it m1,
my method above, call it m2,
sort the numbers, and then use my method above, call it m3.
Here's what I found: m1 − m2 ∼ −4.6×10−17, m1 − m3 ∼ −3×10−15, m2 − m3 ∼ −3×10−15. So, if your numbers are sorted, the error might not be small enough for you. (Note however that even the worst error is 10−15 parts in 1 for 100000 numbers, so it might be good enough anyway.)
Some of the mathematical solutions here are very good. Here's a simple technical solution.
Use a larger data type. This breaks down into two possibilities:
Use a high-precision floating point library. One who encounters a need to average a billion numbers probably has the resources to purchase, or the brain power to write, a 128-bit (or longer) floating point library.
I understand the drawbacks here. It would certainly be slower than using intrinsic types. You still might over/underflow if the number of values grows too high. Yada yada.
If your values are integers or can be easily scaled to integers, keep your sum in a list of integers. When you overflow, simply add another integer. This is essentially a simplified implementation of the first option. A simple (untested) example in C# follows
class BigMeanSet{
List<uint> list = new List<uint>();
public double GetAverage(IEnumerable<uint> values){
list.Clear();
list.Add(0);
uint count = 0;
foreach(uint value in values){
Add(0, value);
count++;
}
return DivideBy(count);
}
void Add(int listIndex, uint value){
if((list[listIndex] += value) < value){ // then overflow has ocurred
if(list.Count == listIndex + 1)
list.Add(0);
Add(listIndex + 1, 1);
}
}
double DivideBy(uint count){
const double shift = 4.0 * 1024 * 1024 * 1024;
double rtn = 0;
long remainder = 0;
for(int i = list.Count - 1; i >= 0; i--){
rtn *= shift;
remainder <<= 32;
rtn += Math.DivRem(remainder + list[i], count, out remainder);
}
rtn += remainder / (double)count;
return rtn;
}
}
Like I said, this is untested—I don't have a billion values I really want to average—so I've probably made a mistake or two, especially in the DivideBy function, but it should demonstrate the general idea.
This should provide as much accuracy as a double can represent and should work for any number of 32-bit elements, up to 232 - 1. If more elements are needed, then the count variable will need be expanded and the DivideBy function will increase in complexity, but I'll leave that as an exercise for the reader.
In terms of efficiency, it should be as fast or faster than any other technique here, as it only requires iterating through the list once, only performs one division operation (well, one set of them), and does most of its work with integers. I didn't optimize it, though, and I'm pretty certain it could be made slightly faster still if necessary. Ditching the recursive function call and list indexing would be a good start. Again, an exercise for the reader. The code is intended to be easy to understand.
If anybody more motivated than I am at the moment feels like verifying the correctness of the code, and fixing whatever problems there might be, please be my guest.
I've now tested this code, and made a couple of small corrections (a missing pair of parentheses in the List<uint> constructor call, and an incorrect divisor in the final division of the DivideBy function).
I tested it by first running it through 1000 sets of random length (ranging between 1 and 1000) filled with random integers (ranging between 0 and 232 - 1). These were sets for which I could easily and quickly verify accuracy by also running a canonical mean on them.
I then tested with 100* large series, with random length between 105 and 109. The lower and upper bounds of these series were also chosen at random, constrained so that the series would fit within the range of a 32-bit integer. For any series, the results are easily verifiable as (lowerbound + upperbound) / 2.
*Okay, that's a little white lie. I aborted the large-series test after about 20 or 30 successful runs. A series of length 109 takes just under a minute and a half to run on my machine, so half an hour or so of testing this routine was enough for my tastes.
For those interested, my test code is below:
static IEnumerable<uint> GetSeries(uint lowerbound, uint upperbound){
for(uint i = lowerbound; i <= upperbound; i++)
yield return i;
}
static void Test(){
Console.BufferHeight = 1200;
Random rnd = new Random();
for(int i = 0; i < 1000; i++){
uint[] numbers = new uint[rnd.Next(1, 1000)];
for(int j = 0; j < numbers.Length; j++)
numbers[j] = (uint)rnd.Next();
double sum = 0;
foreach(uint n in numbers)
sum += n;
double avg = sum / numbers.Length;
double ans = new BigMeanSet().GetAverage(numbers);
Console.WriteLine("{0}: {1} - {2} = {3}", numbers.Length, avg, ans, avg - ans);
if(avg != ans)
Debugger.Break();
}
for(int i = 0; i < 100; i++){
uint length = (uint)rnd.Next(100000, 1000000001);
uint lowerbound = (uint)rnd.Next(int.MaxValue - (int)length);
uint upperbound = lowerbound + length;
double avg = ((double)lowerbound + upperbound) / 2;
double ans = new BigMeanSet().GetAverage(GetSeries(lowerbound, upperbound));
Console.WriteLine("{0}: {1} - {2} = {3}", length, avg, ans, avg - ans);
if(avg != ans)
Debugger.Break();
}
}