Running a simple application on nvidia Visual Profiler shows the error:
Encountered invalid option : --openacc-profiling
======== Use "nvprof --help" to get more information.
Any gpu applicatiion I try to profile gets the same error.
I tried to uncheck the option "Enable OpenACC profiling" and got the same error.
Versions:
nvprof --version
nvprof: NVIDIA (R) Cuda command line profiler
Copyright (c) 2013 - 2014 NVIDIA Corporation
Release version 6.5.14 (21)
And
NVIDIA Visual Profiler
Version: 6.5
It appears (based on comments above) that the issue here was a mixed configuration - a CUDA 8 version of nvvp (the visual profiler) calling a CUDA 6.5 version of nvprof.
The visual profiler performs some of its work by calling nvprof to do low-level profiling. As a result, it is passing command-line switches to nvprof, and so nvprof is expected to match, version-wise, the version of nvvp that is being used. If that is not the case, problems like this can occur.
The solution is to have a consistent install. It should be possible to have both CUDA 6.5 and CUDA 8 installed on the same machine, but it's necessary for the PATH and LD_LIBRARY_PATH variables to be set in such a way that the CUDA 8 version of nvvp will find/invoke the CUDA 8 version of nvprof, for example. Generally, the instructions contained in the linux install guide for setting of these variables should be sufficient, but care should be taken, for example, to be sure that there is not some previous version of nvprof that will be found due to the PATH setting when using CUDA 8. It's not possible cover all the possible ways in which this may happen, so some rudimentary linux administration skills will be necessary to ensure such a configuration is internally consistent.
Otherwise, if these skills don't exist, the linux install instructions may provide the best solution - remove all previous versions of CUDA when installing a new version. That is another possible approach which, if done correctly, should absolutely prevent a problem such as this from occurring.
Related
I tried to install cuda toolkit without display driver in CentOS 6. It gets installed properly. I was able to compile but it is compiling without performing any operation and I get garbage values in array addition. For cudaGetDeviceCount(&count) I am getting value as "o" which means I don't have any card on my machine.
You can install the CUDA toolkit without installing the driver.
You can then compile CUDA codes that use the runtime API.
You will not be able to run those codes unless you have a proper CUDA driver and GPU installed in the machine, however.
Codes that depend on the driver API will also not be compilable in this configuration, on older CUDA toolkits, without additional work. Newer CUDA toolkits provide stub libraries for driver libraries, which can be linked against.
This answer covers the method to install the CUDA toolkit without the driver.
If you want just run the codes and profiling the performance and other parameters, it would be helpful if you install GPGPU-sim simulator. It doesn't need any graphic card on your machine.
Since newer drivers ship with the CUDA runtime (I can choose 9.1 or 9.2 in the drivers download page) my question is: should my library (which uses a CUDA kernel internally) be shipped with -lcudart_static?
I had issues launching kernels compiled with 9.2 on systems which used 9.1 CUDA drivers. What's the most 'compatible' way of ensuring my library will run everywhere a recent CUDA driver is installed? (I'm already compiling for a virtual architecture)
Since newer drivers ship with the CUDA runtime (I can choose 9.1 or 9.2 in the drivers download page)
No, that's incorrect. That choice in the drivers download page is related to the fact that each CUDA version has a minimum required driver version associated with it. It does not mean that the driver ships with the CUDA runtime (stated another way, the driver does not install libcudart.so on linux and never has - with some careful experimentation on a clean install, you can prove this to yourself.)
Some additional comments:
-lcudart_static is actually the default for current/recent versions of nvcc. You can discover this by reading the nvcc manual. Therefore, by default, your executable, when compiled/built with nvcc should already be statically linked to the CUDA runtime library corresponding to the version of nvcc that you are using for compilation. The reason you might need to specify this or something like this is if you are building an application with e.g. the gnu toolchain (on linux) rather than nvcc.
The purpose of static linking to the CUDA runtime library is, as you surmise, so that an application can be built in such a way that it does not need an installation of the CUDA toolkit to run properly. It only needs a machine with a proper GPU driver install.
The most compatible way to ensure that an application will run on a range of machines with a range of GPU driver installs is to compile your application using the oldest CUDA toolkit required to meet the needs of the earliest GPU driver in the range you intend to cover. Again, you can refer to the table here.
I recently upgraded from msvc 2005 to 2015.
I have compiled my code with revision 4.2 of cuda toolkit for year. I'm now learning the hard way that there is no forward compatibility betweend visual and cuda, however Google shows that some trick exists to force the compilation (messing up with .props and .targets files).
From what I understand, cuda 4.2 is a no-go. nvcc seems to have an hardcoded check on the msvc revision.
My questions are:
is there a way to compile with cuda 5.x or 6.x?
worst case scenario is that I have to update to cuda 7.5, does it even work?
Thanks for your help.
Update: CUDA 8RC supports VS2015 Update 1 officially (not update 2).
For CUDA toolkits prior to CUDA 8RC, none officially list MSVC 2015 as a supported environment, including CUDA 7.5 (the most recent production toolkit, at the moment).
For recent CUDA toolkits, the official support matrix is given in the windows getting started guide or installation guide which you would have to review for each toolkit version, to find the support for that version.
Since support for a VS version means that the CUDA toolkit will make changes to the VS environment (e.g. installing CUDA build customization rules, what you refer to as "messing with .props and .targets") and also provide appropriate project definition files for each of the cuda sample projects, if you wanted to work around this, you would have to duplicate those functions yourself. There might be non-standard ways to do this, but you would be operating in unsupported territory.
CUDA 8 is the first version to support MSVC 2015, including the community edition (with the exception of cross compiling). At the time of writing this, CUDA 8 is available as a release candidate if you are signed up for the NVIDIA "Accelerated Computing Developer Program".
With a fresh CUDA 5.0 Linux install on CentOS 5.5, I am not able to gdb. So I am wondering if you still need a dedicated GPU for the Linux cuda-gdb? I tried it with the Vesa device driver for X11, but get the same result. Profiling works, running the app works, but trying to run cuda-gdb gives :
warning: no loadable sections found in added symbol-file system-supplied DSO at 0x2aaaaaaab000
Any suggestions?
cuda-gdb still needs a GPU that is not used by graphical environment (e.g. if you are running Gnome/KDE/etc. you need to have system with several GPUs - not necessary all of them must be NVIDIA GPUs)
This particular message is not about this problem - you can ignore it. cuda-gdb will tell if it fails because no GPU can be used for debugging.
I get the error -
cutilCheckMsg() CUTIL CUDA error : Kernel execution failed
: CUDA version is insufficient for CUDART version.
when I run the sample code. The code however builds successfully
Details of the environment Im running the program -
Windows XP with NO NVIDIA driver
Visual Studio 2008 Express Edition
Cuda toolkit, sdk 3.0
Emulation mode
Here is a similar question asked on SO before but in that case the person had NVIDIA card. I do not have NVIDIA card on my machine. - CUDA driver version is insufficient for CUDA runtime version
Please suggest a solution
Currently the only way to run CUDA code without a GPU is to use CUDA x86 from PGI. Emulation mode was dropped from CUDA several versions ago (current version is CUDA 4.2).