Compute Visual Profiler doesn't Fill the .csv files - cuda

Im trying to benchmark my CUDA application with Compute Visual Profiler. However the program is unable to fill any data in the .csv files. All the paths to CUDA are set properly in the profiler application.
After few runs on the exe file it returns the error:
Error in Profiler data file
'C:/..../temp_compute_profiler_0_0.csv'
at line number 1. No column found.

There are many possible reasons... some of them to check for
the execution time out. make sure that the profiler is not set to time out too soon
the program doesn't finish executing (even if the kernel does). make sure there isn't a getchar at the end of your code
try adding an explicit call to cudaThreadExit at the end of your code, and check for errors.

One of the most common reason for that kind of error is that your program never manages to launch a CUDA kernel or that it failed during its execution.

Related

CUDA kernel launched from Nsight Compute gives inconsistent results

I have completed writing my CUDA kernel, and confirmed it runs as expected when I compile it using nvcc directly, by:
Validating with test data over 100 runs (just in case)
Using cuda-memcheck (memcheck, synccheck, racecheck, initcheck)
Yet, the results printed into the terminal while the application is getting profiled using Nsight Compute differs from run to run. I am curious if the difference is a cause for concern, or if this is the expected behavior.
Note: The application also gives correct & consistent results while getting profiled bu nvprof.
I followed up on the NVIDIA forums but will post here as well for tracking:
What inconsistencies are you seeing in the output? Nsight Compute runs a kernel multiple times to collect all of its information. So things like print statements in the kernel will show up multiple times. Could it be related to that or is it a value being calculated differently? One other issue is with Unified Memory (UVM) or zero copy memory Nsight Compute is not able to restore those values before each replay. Are you using that in your application? If so, the application replay mode could help. It may be worth trying to see if anything changes.
I was able to resolve the issue by addressing my shared memory initializations. Since Nsight Compute runs a kernel multiple times as #Jackson stated, the effects of uninitialized memory were amplified (I was performing atomicAdd into uninitialized memory).

Can not profile a cuda code with nvprof when using CUPTI functions inside

I'm doing a simple experiment. Everyone may know about callback_metric sample code of CUPTI (located in CUPTI folder: /usr/local/cuda/extras/CUPTI/sample/callback_metric). It contains only a simple code for reading a metric when running a vectorAdd kernel. Everything works when I compile and run the code.
But when I run this code under nvprof command (nvprof ./callback_metric), I get an error message as:
Error: incompatible CUDA driver version
both nvprof and other CUPTI-based codes work fine separately.
The profilers are not intended to be used in this way with applications that make use of CUPTI.
This is documented in the profiler documentation:
Here are a couple of reasons why Visual Profiler may fail to gather metric or event information.
More than one tool is trying to access the GPU. To fix this issue please make sure only one tool is using the GPU at any given point. Tools include the CUDA command line profiler, Parallel NSight Analysis Tools and Graphics Tools, and applications that use either CUPTI or PerfKit API (NVPM) to read event values.

Cuda iterative program stopping unusually. Runs only when PC is restarted Each time

I have a iterative cuda program which iterates new values as required.
It is a confidential code so I cant share, but I want to discuss the problem.
The iterative program runs properly on my PC when I work with less data.
I have proper allocation and deallocation codes.
No matter how many times I run the program it runs properly with less data.
But in case of huge data, It runs properly one time but not multiple times providing an error "****.exe has stopped working.....".
Same error persists until I restart the PC...each time.
It is not feasible to restart the PC each time for me to start the program. So What might be the reason behind it?
Most likely a memory error.
You should try running cuda-memcheck, this will make obvious any memory errors.
Other options include using error handling within your code, this would catch the problems as they arise.

Can GPU counters be read transparently to the application code

I am trying to profile the CUDA rodinia benchmarks executing on a GTX 650.
I am using the code /usr/local/cuda-5.0/extras/CUPTI/samples/event_sampling to read
the instructions executed counter. It seems strange that I do not see any change in the
values reported by the event_sampling whether I am executing the CUDA benchmarks or not.
The event_sampling code also has some calculations of its own for which it measures the instructions executed. Unlike CPU, do I need to make changes to the source code of the application to be able to read the GPU counters such as instruction_executed?
CUPTI will only give you counter updates for kernels in the same process. You can get some of these values, though not to the same level of precision, with the NVIDIA visual profiler or related environment variables without modifying the code however.

Runtime vs. Compile time

What is the difference between run-time and compile-time?
The difference between compile time and run time is an example of what pointy-headed theorists call the phase distinction. It is one of the hardest concepts to learn, especially for people without much background in programming languages. To approach this problem, I find it helpful to ask
What invariants does the program satisfy?
What can go wrong in this phase?
If the phase succeeds, what are the postconditions (what do we know)?
What are the inputs and outputs, if any?
Compile time
The program need not satisfy any invariants. In fact, it needn't be a well-formed program at all. You could feed this HTML to the compiler and watch it barf...
What can go wrong at compile time:
Syntax errors
Typechecking errors
(Rarely) compiler crashes
If the compiler succeeds, what do we know?
The program was well formed---a meaningful program in whatever language.
It's possible to start running the program. (The program might fail immediately, but at least we can try.)
What are the inputs and outputs?
Input was the program being compiled, plus any header files, interfaces, libraries, or other voodoo that it needed to import in order to get compiled.
Output is hopefully assembly code or relocatable object code or even an executable program. Or if something goes wrong, output is a bunch of error messages.
Run time
We know nothing about the program's invariants---they are whatever the programmer put in. Run-time invariants are rarely enforced by the compiler alone; it needs help from the programmer.
What can go wrong are run-time errors:
Division by zero
Dereferencing a null pointer
Running out of memory
Also there can be errors that are detected by the program itself:
Trying to open a file that isn't there
Trying find a web page and discovering that an alleged URL is not well formed
If run-time succeeds, the program finishes (or keeps going) without crashing.
Inputs and outputs are entirely up to the programmer. Files, windows on the screen, network packets, jobs sent to the printer, you name it. If the program launches missiles, that's an output, and it happens only at run time :-)
I think of it in terms of errors, and when they can be caught.
Compile time:
string my_value = Console.ReadLine();
int i = my_value;
A string value can't be assigned a variable of type int, so the compiler knows for sure at compile time that this code has a problem
Run time:
string my_value = Console.ReadLine();
int i = int.Parse(my_value);
Here the outcome depends on what string was returned by ReadLine(). Some values can be parsed to an int, others can't. This can only be determined at run time
Compile-time: the time period in which you, the developer, are compiling your code.
Run-time: the time period which a user is running your piece of software.
Do you need any clearer definition?
(edit: the following applies to C# and similar, strongly-typed programming languages. I'm not sure if this helps you).
For example, the following error will be detected by the compiler (at compile time) before you run a program and will result in a compilation error:
int i = "string"; --> error at compile-time
On the other hand, an error like the following can not be detected by the compiler. You will receive an error/exception at run-time (when the program is run).
Hashtable ht = new Hashtable();
ht.Add("key", "string");
// the compiler does not know what is stored in the hashtable
// under the key "key"
int i = (int)ht["key"]; // --> exception at run-time
Translation of source code into stuff-happening-on-the-[screen|disk|network] can occur in (roughly) two ways; call them compiling and interpreting.
In a compiled program (examples are c and fortran):
The source code is fed into another program (usually called a compiler--go figure), which produces an executable program (or an error).
The executable is run (by double clicking it, or typing it's name on the command line)
Things that happen in the first step are said to happen at "compile time", things that happen in the second step are said to happen at "run time".
In an interpreted program (example MicroSoft basic (on dos) and python (I think)):
The source code is fed into another program (usually called an interpreter) which "runs" it directly. Here the interpreter serves as an intermediate layer between your program and the operating system (or the hardware in really simple computers).
In this case the difference between compile time and run time is rather harder to pin down, and much less relevant to the programmer or user.
Java is a sort of hybrid, where the code is compiled to bytecode, which then runs on a virtual machine which is usually an interpreter for the bytecode.
There is also an intermediate case in which the program is compiled to bytecode and run immediately (as in awk or perl).
Basically if your compiler can work out what you mean or what a value is "at compile time" it can hardcode this into the runtime code. Obviously if your runtime code has to do a calculation every time it will run slower, so if you can determine something at compile time it is much better.
Eg.
Constant folding:
If I write:
int i = 2;
i += MY_CONSTANT;
The compiler can perform this calulation at compile time because it knows what 2 is, and what MY_CONSTANT is. As such it saves itself from performing a calculation every single execution.
Hmm, ok well, runtime is used to describe something that occurs when a program is running.
Compile time is used to describe something that occurs when a program is being built (usually, by a compiler).
Compile Time:
Things that are done at compile time incur (almost) no cost when the resulting program is run, but might incur a large cost when you build the program.
Run-Time:
More or less the exact opposite. Little cost when you build, more cost when the program is run.
From the other side; If something is done at compile time, it runs only on your machine and if something is run-time, it run on your users machine.
Relevance
An example of where this is important would be a unit carrying type. A compile time version (like Boost.Units or my version in D) ends up being just as fast as solving the problem with native floating point code while a run-time version ends up having to pack around information about the units that a value are in and perform checks in them along side every operation. On the other hand, the compile time versions requiter that the units of the values be known at compile time and can't deal with the case where they come from run-time input.
As an add-on to the other answers, here's how I'd explain it to a layman:
Your source code is like the blueprint of a ship. It defines how the ship should be made.
If you hand off your blueprint to the shipyard, and they find a defect while building the ship, they'll stop building and report it to you immediately, before the ship has ever left the drydock or touched water. This is a compile-time error. The ship was never even actually floating or using its engines. The error was found because it prevented the ship even being made.
When your code compiles, it's like the ship being completed. Built and ready to go. When you execute your code, that's like launching the ship on a voyage. The passengers are boarded, the engines are running and the hull is on the water, so this is runtime. If your ship has a fatal flaw that sinks it on its maiden voyage (or maybe some voyage after for extra headaches) then it suffered a runtime error.
Following from previous similar answer of question What is the difference between run-time error and compiler error?
Compilation/Compile time/Syntax/Semantic errors: Compilation or compile time errors are error occurred due to typing mistake, if we do not follow the proper syntax and semantics of any programming language then compile time errors are thrown by the compiler. They wont let your program to execute a single line until you remove all the syntax errors or until you debug the compile time errors.
Example: Missing a semicolon in C or mistyping int as Int.
Runtime errors: Runtime errors are the errors that are generated when the program is in running state. These types of errors will cause your program to behave unexpectedly or may even kill your program. They are often referred as Exceptions.
Example: Suppose you are reading a file that doesn't exist, will result in a runtime error.
Read more about all programming errors here
Here is a quote from Daniel Liang, author of 'Introduction to JAVA programming', on the subject of compilation:
"A program written in a high-level language is called a source program or source code. Because a computer cannot execute a source program, a source program must be translated into machine code for execution. The translation can be done using another programming tool called an interpreter or a compiler." (Daniel Liang, "Introduction to JAVA programming", p8).
...He Continues...
"A compiler translates the entire source code into a machine-code file, and the machine-code file is then executed"
When we punch in high-level/human-readable code this is, at first, useless! It must be translated into a sequence of 'electronic happenings' in your tiny little CPU! The first step towards this is compilation.
Simply put: a compile-time error happens during this phase, while a run-time error occurs later.
Remember: Just because a program is compiled without error does not mean it will run without error.
A Run-time error will occur in the ready, running or waiting part of a programs life-cycle while a compile-time error will occur prior to the 'New' stage of the life cycle.
Example of a Compile-time error:
A Syntax Error - how can your code be compiled into machine level instructions if they are ambiguous?? Your code needs to conform 100% to the syntactical rules of the language otherwise it cannot be compiled into working machine code.
Example of a run-time error:
Running out of memory - A call to a recursive function for example might lead to stack overflow given a variable of a particular degree! How can this be anticipated by the compiler!? it cannot.
And that is the difference between a compile-time error and a run-time error
For example: In a strongly typed language, a type could be checked at compile time or at runtime. At compile time it means, that the compiler complains if the types are not compatible. At runtime means, that you can compile your program just fine but at runtime, it throws an exception.
In simply word difference b/w Compile time & Run time.
compile time:Developer writes the program in .java format & converts in to the Bytecode which is a class file,during this compilation any error occurs can be defined as compile time error.
Run time:The generated .class file is use by the application for its additional functionality & the logic turns out be wrong and throws an error which is a run time error
Compile time:
Time taken to convert the source code into a machine code so that it becomes an executable is called compile time.
Run time:
When an application is running, it is called run time.
Compile time errors are those syntax errors, missing file reference errors.
Runtime errors happen after the source code has been compiled into an executable program and while the program is running. Examples are program crashes, unexpected program behavior or features don't work.
Run time means something happens when you run the program.
Compile time means something happens when you compile the program.
Imagine that you are a boss and you have an assistant and a maid, and you give them a list of tasks to do, the assistant (compile time) will grab this list and make a checkup to see if the tasks are understandable and that you didn't write in any awkward language or syntax, so he understands that you want to assign someone for a Job so he assign him for you and he understand that you want some coffee, so his role is over and the maid (run time)starts to run those tasks so she goes to make you some coffee but in sudden she doesn’t find any coffee to make so she stops making it or she acts differently and make you some tea (when the program acts differently because he found an error).
Compile Time:
Things that are done at compile time incur (almost) no cost when the resulting program is run, but might incur a large cost when you build the program.
Run-Time:
More or less the exact opposite. Little cost when you build, more cost when the program is run.
From the other side; If something is done at compile time, it runs only on your machine and if something is run-time, it run on your users machine.
I have always thought of it relative to program processing overhead and how it affects preformance as previously stated. A simple example would be, either defining the absolute memory required for my object in code or not.
A defined boolean takes x memory this is then in the compiled program and cannot be changed. When the program runs it knows exactly how much memory to allocate for x.
On the other hand if I just define a generic object type (i.e. kind of a undefined place holder or maybe a pointer to some giant blob) the actual memory required for my object is not known until the program is run and I assign something to it, thus it then must be evaluated and memory allocation, etc. will be then handled dynamically at run time (more run time overhead).
How it is dynamically handled would then depend on the language, the compiler, the OS, your code, etc.
On that note however it would really depends on the context in which you are using run time vs compile time.
Here is an extension to the Answer to the question "difference between run-time and compile-time?" -- Differences in overheads associated with run-time and compile-time?
The run-time performance of the product contributes to its quality by delivering results faster. The compile-time performance of the product contributes to its timeliness by shortening the edit-compile-debug cycle. However, both run-time performance and compile-time performance are secondary factors in achieving timely quality. Therefore, one should consider run-time and compile-time performance improvements only when justified by improvements in overall product quality and timeliness.
A great source for further reading here:
we can classify these under different two broad groups static binding and dynamic binding. It is based on when the binding is done with the corresponding values. If the references are resolved at compile time, then it is static binding and if the references are resolved at runtime then it is dynamic binding. Static binding and dynamic binding also called as early binding and late binding. Sometimes they are also referred as static polymorphism and dynamic polymorphism.
Joseph Kulandai‏.
The major difference between run-time and compile time is:
If there are any syntax errors and type checks in your code,then it throws compile time error, where-as run-time:it checks after executing the code.
For example:
int a = 1
int b = a/0;
here first line doesn't have a semi-colon at the end---> compile time error after executing the program while performing operation b, result is infinite---> run-time error.
Compile time doesn't look for output of functionality provided by your code, whereas run-time does.
here's a very simple answer:
Runtime and compile time are programming terms that refer to different stages of software program development.
In order to create a program, a developer first writes source code, which defines how the program will function. Small programs may only contain a few hundred lines of source code, while large programs may contain hundreds of thousands of lines of source code. The source code must be compiled into machine code in order to become and executable program. This compilation process is referred to as compile time.(think of a compiler as a translator)
A compiled program can be opened and run by a user. When an application is running, it is called runtime.
The terms "runtime" and "compile time" are often used by programmers to refer to different types of errors. A compile time error is a problem such as a syntax error or missing file reference that prevents the program from successfully compiling. The compiler produces compile time errors and usually indicates what line of the source code is causing the problem.
If a program's source code has already been compiled into an executable program, it may still have bugs that occur while the program is running. Examples include features that don't work, unexpected program behavior, or program crashes. These types of problems are called runtime errors since they occur at runtime.
The reference
Look into this example:
public class Test {
public static void main(String[] args) {
int[] x=new int[-5];//compile time no error
System.out.println(x.length);
}}
The above code is compiled successfully, there is no syntax error, it is perfectly valid.
But at the run time, it throws following error.
Exception in thread "main" java.lang.NegativeArraySizeException
at Test.main(Test.java:5)
Like when in compile time certain cases has been checked, after that run time certain cases has been checked once the program satisfies all the condition you will get an output.
Otherwise, you will get compile time or run time error.
You can understand the code compile structure from reading the actual code. Run-time structure are not clear unless you understand the pattern that was used.
public class RuntimeVsCompileTime {
public static void main(String[] args) {
//test(new D()); COMPILETIME ERROR
/**
* Compiler knows that B is not an instance of A
*/
test(new B());
}
/**
* compiler has no hint whether the actual type is A, B or C
* C c = (C)a; will be checked during runtime
* #param a
*/
public static void test(A a) {
C c = (C)a;//RUNTIME ERROR
}
}
class A{
}
class B extends A{
}
class C extends A{
}
class D{
}
It's not a good question for S.O. (it's not a specific programming question), but it's not a bad question in general.
If you think it's trivial: what about read-time vs compile-time, and when is this a useful distinction to make? What about languages where the compiler is available at runtime? Guy Steele (no dummy, he) wrote 7 pages in CLTL2 about EVAL-WHEN, which CL programmers can use to control this. 2 sentences are barely enough for a definition, which itself is far short of an explanation.
In general, it's a tough problem that language designers have seemed to try to avoid.
They often just say "here's a compiler, it does compile-time things; everything after that is run-time, have fun". C is designed to be simple to implement, not the most flexible environment for computation. When you don't have the compiler available at runtime, or the ability to easily control when an expression is evaluated, you tend to end up with hacks in the language to fake common uses of macros, or users come up with Design Patterns to simulate having more powerful constructs. A simple-to-implement language can definitely be a worthwhile goal, but that doesn't mean it's the end-all-be-all of programming language design. (I don't use EVAL-WHEN much, but I can't imagine life without it.)
And the problemspace around compile-time and run-time is huge and still largely unexplored. That's not to say S.O. is the right place to have the discussion, but I encourage people to explore this territory further, especially those who have no preconceived notions of what it should be. The question is neither simple nor silly, and we could at least point the inquisitor in the right direction.
Unfortunately, I don't know any good references on this. CLTL2 talks about it a bit, but it's not great for learning about it.