These for-loops are among the first basic examples of formal correctness proofs of algorithms. They have different but equivalent termination conditions:
1 for ( int i = 0; i != N; ++i )
2 for ( int i = 0; i < N; ++i )
The difference becomes clear in the postconditions:
The first one gives the strong guarantee that i == N after the loop terminates.
The second one only gives the weak guarantee that i >= N after the loop terminates, but you will be tempted to assume that i == N.
If for any reason the increment ++i is ever changed to something like i += 2, or if i gets modified inside the loop, or if N is negative, the program can fail:
The first one may get stuck in an infinite loop. It fails early, in the loop that has the error. Debugging is easy.
The second loop will terminate, and at some later time the program may fail because of your incorrect assumption of i == N. It can fail far away from the loop that caused the bug, making it hard to trace back. Or it can silently continue doing something unexpected, which is even worse.
Which termination condition do you prefer, and why? Are there other considerations? Why do many programmers who know this, refuse to apply it?
I tend to use the second form, simply because then I can be more sure that the loop will terminate. I.e. it's harder to introduce a non-termination bug by altering i inside the loop.
Of course, it also has the slightly lazy advantage of being one less character to type ;)
I would also argue, that in a language with sensible scope rules, as i is declared inside the loop construct, it shouldn't be available outside the loop. This would mitigate any reliance on i being equal to N at the end of the loop...
We shouldn't look at the counter in isolation - if for any reason someone changed the way the counter is incremented they would change the termination conditions and the resulting logic if it's required for i==N.
I would prefer the the second condition since it's more standard and will not result in endless loop.
In C++, using the != test is preferred for generality. Iterators in C++ have various concepts, like input iterator, forward iterator, bidirectional iterator, random access iterator, each of which extends the previous one with new capabilities. For < to work, random access iterator is required, whereas != merely requires input iterator.
If you trust your code, you can do either.
If you want your code to be readable and easily understood (and thus more tolerant to change from someone who you've got to assume to be a klutz), I'd use something like;
for ( int i = 0 ; i >= 0 && i < N ; ++i)
I always use #2 as then you can be sure the loop will terminate... Relying on it being equal to N afterwards is relying on a side effect... Wouldn't you just be better using the variable N itself?
[edit] Sorry...I meant #2
I think most programmers use the 2nd one, because it helps figure out what goes on inside the loop. I can look at it, and "know" that i will start as 0, and will definitely be less than N.
The 1st variant doesn't have this quality. I can look at it, and all I know is that i will start as 0 and that it won't ever be equal to N. Not quite as helpful.
Irrespective of how you terminate the loop, it is always good to be very wary of using a loop control variable outside the loop. In your examples you (correctly) declare i inside the loop, so it is not in scope outside the loop and the question of its value is moot...
Of course, the 2nd variant also has the advantage that it's what all of the C references I have seen use :-)
In general I would prefer
for ( int i = 0; i < N; ++i )
The punishment for a buggy program in production, seems a lot less severe, you will not have a thread stuck forever in a for loop, a situation that can be very risky and very hard to diagnose.
Also, in general I like to avoid these kind of loops in favour of the more readable foreach style loops.
I prefer to use #2, only because I try not to extend the meaning of i outside of the for loop. If I were tracking a variable like that, I would create an additional test. Some may say this is redundant or inefficient, but it reminds the reader of my intent: At this point, i must equal N
#timyates - I agree one shouldn't rely on side-effects
I think you stated very well the difference between the two. I do have the following comments, though:
This is not "language-agnostic", I can see your examples are in C++ but there
are languages where you are not allowed to modify the loop variable inside the
loop and others that don't guarantee that the value of the index is usable after
the loop (and some do both).
You have declared the i
index within the for so I would not bet on the value of i after the loop.
The examples are a little bit misleading as they implictly assume that for is
a definite loop. In reality it is just a more convenient way of writing:
// version 1
{ int i = 0;
while (i != N) {
...
++i;
}
}
Note how i is undefined after the block.
If a programmer knew all of the above would not make general assumption of the value of i and would be wise enough to choose i<N as the ending conditions, to ensure that the the exit condition will be eventually met.
Using either of the above in c# would cause a compiler error if you used i outside the loop
I prefer this sometimes:
for (int i = 0; (i <= (n-1)); i++) { ... }
This version shows directly the range of values that i can have. My take on checking lower and upper bound of the range is that if you really need this, your code has too many side effects and needs to be rewritten.
The other version:
for (int i = 1; (i <= n); i++) { ... }
helps you determine how often the loop body is called. This also has valid use cases.
For general programming work I prefer
for ( int i = 0; i < N; ++i )
to
for ( int i = 0; i != N; ++i )
Because it is less error prone, especially when code gets refactored. I have seen this kind of code turned into an infinite loop by accident.
That argument made that "you will be tempted to assume that i == N", I don't believe is true. I have never made that assumption or seen another programmer make it.
From my standpoint of formal verification and automatic termination analysis, I strongly prefer #2 (<). It is quite easy to track that some variable is increased (before var = x, after var = x+n for some non-negative number n). However, it is not that easy to see that i==N eventually holds. For this, one needs to infer that i is increased by exactly 1 in each step, which (in more complicated examples) might be lost due to abstraction.
If you think about the loop which increments by two (i = i + 2), this general idea becomes more understandable. To guarantee termination one now needs to know that i%2 == N%2, whereas this is irrelevant when using < as the condition.
Related
I'm trying to write up some examples to explain when a while loop should be used, and when a for loop should be used.
When looking for 'interesting' cases to show young and novice programmers, I realized that the vast majority of textbook examples for while loops will look something like this:
i = 0
while i < 10:
do something
i = i + 1
'do something' might be printing the odd numbers, squaring i, etc... However all these are obviously easier written with a for loop!
I'm looking for more interesting examples. They would have to be:
Suitable for younger programmers (e.g. not too much math such as numerical root finding or the sequence in Collatz conjecture)
Easier (or more intuitive) to be solved with while loops rather than for.
Have some real use to it (e.g. I could do while random() < 0.95, but what's a real use for this?)
The only example I could come up with is when getting a list input from the user one-by-one (e.g. numbers to be summed), but the user will have to terminate it with a special input, and also this seems pointless as the user could just say in advance how many entries there will be in the sequence.
The fundamental difference between a FOR loop and a WHILE loop is that for a FOR loop, the number of iterations is bounded by a constant that is known before the loop starts, whereas for a WHILE loop, the number of iterations can be unbounded, unknown, or infinite.
As a result, a language offering only WHILE loops is Turing-complete, a language offering only FOR loops is not.
So, the first obvious thing that only a WHILE loop can do, is an infinite loop. Things that are easily modeled as infinite loops are, for example, a web server, a Netflix client, a game loop, a GUI event loop, or an operating system:
WHILE (r = nextHttpRequest):
handle(r)
END
WHILE (p = nextVideoStreamPacket):
frame = decode(p)
draw(frame)
END
WHILE (a = playerAction):
computeNextFrame(a)
END
WHILE (e = nextEvent):
handle(e)
END
WHILE (s = sysCall):
process(s)
END
A good example where the loop is not infinite, but the bound is not known in advance, is (as you already mentioned in your question) asking for user input. Something like this:
WHILE (askBoolean("Do you want to play again?")):
playGame()
END
Another good example is processing a C-like string, where the length of the string is unknown but finite. This is the same situation for a linked list, or for any data structure where there is a notion of "next", but not a notion of "size", instead there is some sentinel value that marks the end (e.g. NUL-terminated strings in C) or a way to check whether there is a next element (e.g. Iterator in Java):
WHILE ((element = getNext()) != END_MARKER):
process(element)
END
WHILE (hasNextElement):
process(getNext())
END
There are also situations that can be handled with a FOR loop, but a WHILE loop is more elegant. One situation I can think of, is that the bound for the number of iterations is known in advance, it is constant, but the known bound is ridiculously large, and the actual number of iterations required is significantly less than the bound.
Unfortunately, I cannot come up with a good real-life example of this, maybe someone else can. A FOR loop for this will then typically look like this, in order to skip the iterations from the actual number of iterations up to the upper bound:
FOR (i FROM 1 TO $SOME_LARGE_UPPER_BOUND):
IF (terminationConditionReached):
NOOP()
ELSE:
doSomethingInteresting()
END
END
Which would much better be expressed as
WHILE (NOT terminationConditionReached):
doSomethingInteresting()
END
Using the FOR loop could make sense in this situation, if the value of i is of interest:
FOR (i FROM 1 TO $SOME_LARGE_UPPER_BOUND):
IF (terminationConditionReached):
NOOP()
ELSE:
doSomethingInterestingWithI(i)
END
END
A last situation I can think of, where a WHILE loop is more appropriate than a FOR loop, even though the number of iterations is bounded by a known constant, is if that constant is not "semantically interesting" for the loop.
For example, a game loop for Tic-Tac-Toe only needs at most 9 moves, so it could be modeled as a FOR loop:
FOR (i FROM 1 TO 9):
IF (player1Won OR player2Won):
NOOP
ELSE:
makeMove()
END
END
But, the number "9" is not really interesting here. It's much more interesting whether one player has one or the board is full:
WHILE (NOT (player1Won OR player2Won OR boardFull)):
makeMove()
END
[Note: at least if playing against a child, this is also an example of the second-to-last situation, namely that the upper bound is known to be 9, but a lot of games will be shorter than 9 moves. However, I would still like to find an example for that, which is not also an example of a semantically un-interesting termination condition.]
So, we have two classes of situations here: one, where a FOR loop simply cannot be used (when the bound is unknown, non-existant, or infinite), and one, where a FOR loop can be used, but a WHILE loop is more intention-revealing.
I am trying to parallelize this for loop inside a function using OpenMP, but when I compile the code I still have an error =(
Error 1 error C3010: 'return' : jump out of OpenMP structured block not allowed.
I am using Visual studio 2010 C++ compiler. Can anyone help me? I appreciate any advice.
int match(char* pattern, int patternSize, char* string, int startFrom, unsigned int &comparisons) {
comparisons = 0;
#pragma omp for
for (int i = 0; i < patternSize; i++){
comparisons++;
if (pattern[i] != string[i + startFrom])
return 0;
}
return 1;
}
As #Hristo has already mentioned, you are not allowed to branch out of a parallel region in OpenMP. Among other reasons, this is not allowed because the compiler cannot know a priori how many iterations each thread should work on when it splits up a for loop like the one that you have written among the different threads.
Furthermore, even if you could branch out of your loop, you should be able to see that comparisons would be computed incorrectly. As is, you have an inherently serial algorithm that breaks at the first different character. How could you split up this work such that throwing more threads at this algorithm possibly makes it faster?
Finally, note that there is very little work being done in this loop anyway. You would be very unlikely to see any benefit from OpenMP even if you could rewrite this algorithm into a parallel algorithm. My suggestion: drop OpenMP from this loop and look to implement it somewhere else (either at a higher level - maybe you call this method on different strings? - or in a section of your code that does more work).
What are the pros and cons of using one or the other iteration functions ?
function (int n) {
for (int i = 1; i != n; ++i) { ... }
}
vs
function (int n) {
for (int i = 1; i < n; i++) { ... }
}
I think the main argument against the first version is that it is a much less common idiom.
Remembering that code is read more often than it is written, it does not make sense to use a less familiar form of for loop if there isn't a very clear advantage to doing so. All it achieves is distracting anyone working on the code in future.
So primarily for code maintenance reasons (by others as well as the original coder) I would favour the more common second format.
The version with < will work correctly if n is less than 1. The version with != will go into an infinite loop (well, probably not infinite, as integer variables wrap around in most languages).
Using < also generalizes better. E.g.
for (i = start; i < end; i += increment)
This will work even if end - start is not a multiple of increment.
The first one is quite dangerous and could cause an infinite loop.
If n is ever less than 1, the loop will never exit.
Also if something changes i inside the loop, so that it skips the value of n, then again the loop will never exit.
Edit: OK to be more precise when I say never exit, it will ultimately exit one way or another, but it won't be in the manner most sane developers expect. I can just imagine the look on the poor guy that debugs your code that calls the database 2 billion times.
I've recently noticed a coworker of mine doing
int len = foo.length();
for (int i = 0; i < len; ++i)
doStuff(foo[i]);
I'm aware that this was considered good practice in C, where strlen() ran in O(length_of_string). But I'd expect newer languages (say, Java or Python) to store the length of the String alongside the characters, thus allowing length() to run in O(1). I usually write:
for (int i = 0; i < foo.length(); ++i)
doStuff(foo[i]);
Saving a line of code. But my Co-Worker got me wondering.... is this really good practice, or is it unreasonable to expect the O(1) behaviour?
(As a related question: can't modern compilers extract the strlen() call from inside the for-header automatically these days?)
These statements are actually two different statements.
int len = foo.length(); // Will run once
for (int i = 0; i<len;++i)
Here i<len will be checked every loop, len is just a variable that can be read though.
for (int i = 0; i < foo.length(); ++i)
Here i < foo.length() contains a function call, and since the length of foo can change within the loop itself (You could e.g. strip characters off of foo instead of incrementing i) the function foo.length() will be called every iteration.
There are some languages in which foo might be a constant and foo.length() could be optimised out by the compiler, but it's better to be save than sorry.
Additionally some languages might allow something like this:
for (int i=0, len=foo.length();i<len;++i)
which still saves you the line.
First, call the function foo.length() each iteration of the loop in any case would require more resources than using a temporary variable to store result of the call foo.length().
Furthermore, the use of your code may cause errors when refactoring code. For example, this cycle will never end:
for (int i = 0; i < foo.length(); ++i)
{
doStuff(foo[i]);
// few line of code, written another man
doWork(foo); // Passing by reference
}
void doWork(Foo fooObj)
{
// some work
fooObj.Add(new SomeObject());
}
This isn't very language agnostic. It depends on how smart is the compiler. If you can explicitly state that the length is constant, the whole loop can be inlined, so no tests happen at all. When it comes to Java, I would bet compiler can get pretty smart, so you don't have to be explicit that much. You are right about java.lang.String precomputing its length. When it comes to complexity, it is practical to define what are the important operations you are counting. Strictly speaking on a Turing machine you have to be O(n) in order to find the end ("$") of the input.
As I am using for-loops on large multi-dim arrays, any saving on the for-loop mechanism itself is meaningful.
Accordingly, I am looking for any tips on how to reduce this overhead.
e.g. : counting down using uint instead of int and != 0 as stop instead of >0 allows the CPU to do less work (heard it once, not sure it is always true)
One important suggestion: move as much calculation to the outer loop as possible. Not all compilers can do that automatically. For eample, instead of:
for row = 0 to 999
for col = 0 to 999
cell[row*1000+col] = row * 7 + col
use:
for row = 0 to 999
x = row * 1000
y = row * 7
for col = 0 to 999
cell[x+col] = y + col
Try to make your loops contiguous in memory, this will optimize cache usage. That is, don't do this:
for (int i = 0; i < m; i++)
for (j = 0; j < n; j++)
s += arr[j][i];
If processing images, convert two loops to one loop on the pixels with a single index.
Don't make loops that will run zero times, as the pipeline is optimized to assume a loop will continue rather than end.
Have you measured the overhead? Do you know how much time is spent processing the for loops vs. how much time is spent executing your application code? What is your goal?
Loop-unrolling can be one way. That is:
for (i=0; i<N; i++) {
a[i]=...;
}
transforms into:
for (i=0; i<N; i+=4) {
a[i]=...;
a[i+1]=...;
a[i+2]=...;
a[i+3]=...;
}
You will need special handling when N is not a multiple of 4 in the example above.
First, don't sweat the small stuff. Details like counting up versus counting down are usually completely irrelevant in running time. Humans are notoriously bad at spotting areas in code that need to be sped up. Use a profiler. Pay little or no attention to any part of the loop that is not repeated, unless the profiler says otherwise. Remember that what is written in an inner loop is not necessarily executed in an inner loop, as modern compilers are pretty smart about avoiding unnecessary repetition.
That being said, be very wary of unrolling loops on modern CPUs. The tighter they are, the better they will fit into cache. In a high-performance application I worked on last year, I improved performance significantly by using loops instead of straight-line code, and tightening them up as much as I could. (Yes, I profiled; the function in question took up 80% of the run time. I also benchmarked times over typical input, so I knew the changes helped.)
Moreover, there's no harm in developing habits that favor efficient code. In C++, you should get in the habit of using pre-increment (++i) rather than post-increment (i++) to increment loop variables. It usually doesn't matter, but can make a significant difference, it doesn't make code less readable or writable, and won't hurt.
This isn't a language agnostic question, it depends highly on not only language, but also compiler. Most compilers I believe will compile these two equivalently:
for (int i = 0; i < 10; i++) { /* ... */ }
int i = 0;
while (i < 10) {
// ...
i++;
}
In most languages/compilers, the for loop is just syntactic sugar for the later while loop. Foreach is another question again, and is highly dependant on language/compiler as to how it's implemented, but it's generally less efficient that a normal for/while loop. How much more so is again, language and compiler dependant.
Your best bet would probably be to run some benchmarks with several different variations on a theme and see what comes out on top.
Edit: To that end, the suggestions here will probably save you more time rather than worrying about the loop itself.
BTW, unless you need post-increment, you should always use the pre-increment operator. It is only a minor difference, but it is more efficient.
Internally this is the difference:
Post Increment
i++;
is the same as:
int postincrement( int &i )
{
int itmp = i;
i = i + 1;
return itmp;
}
Pre Increment
++i;
is the same as:
int preincrement( int &i )
{
i = i + 1;
return i;
}
I agree with #Greg. First thing you need to do is put some benchmarking in place. There will be little point optimising anything until you prove where all your processing time is being spent. "Premature optimisation is the root of all evil"!
As your loops will have O(n^d) complexity (d=dimension), what really counts is what you put INTO the loop, not the loop itself. Optimizing a few cycles away in the loop framework from millions of cycles of an inefficient algorithm inside the loop is just snake oil.
By the way, is it good to use short instead of int in for-loop if Int16 capacity is guaranteed to be enough?
There is not enough information to answer your question accurately. What are you doing inside your loops? Does the calculation in one iteration depend on a value calculated in a previous iteration. If not, you can almost cut your time in half by simply using 2 threads, assuming you have at least a dual core processor.
Another thing to look at is how you are accessing your data, if you are doing large array processing, to make sure that you access the data sequentially as it is stored in memory, avoiding flushing your L1/L2 cache on every iteration (seen this before on smaller L1 caches, the difference can be dramatic).
Again, I would look at what is inside the loop first, where most of the gains (>99%) will be, rather than the outer loop plumbing.
But then again, if your loop code is I/O bound, then any time spent on optimization is wasted.
I think most compilers would probably do this anyway, stepping down to zero should be more efficient, as a check for zero is very fast for the processor. Again though, any compiler worth it's weight would do this with most loops anyway. You need to loo at what the compiler is doing.
There is some relevant information among the answers to another stackoverflow question, how cache memory works. I found the paper by Ulrich Drepper referred to in this answer especially useful.