How to debug CUDA code on TCC enabled device on a remote server? - cuda

I would like to start a remote debugging session from my development pc on our soon-to-be production server. On the server I start NSight using the remote desktop, and then I try to Start CUDA Debugging on my local machine (given the server as target before). The result is, that the debug session disconnects saying "The remote system is logged in through remote desktop. WDDM adapters will not be debuggable".
Is this intended behaviour, a bug, wrong configuration? And if there is no solution involveing remote desktop, how could I start the NSight monitor, so that I can start a debugging session?
On the target server I have two Tesla K10 and a Quadro FX (for Display). All Tesla devices (which are actually four), show TCC turned on.
I am using NSight 3.0, CUDA 5.0 and Visual Studio 2008 (the latter two only on the development workstation).
One last thing: Copying the application to the server and executing through remote desktop works fine.

The message you refer to is a warning to let users know that Nsight has detected that you have a RDP session running on the target side and that it won't be able to do Nsight debugging on any WDDM cards in it, if there is one (or more) - which looks like you do, with that Quadro FX that you mentioned).
This message is a Warning, and it should continue to let you run or debug the application under Nsight and debug the application assuming GPU code runs on the TCC devices. Are you sure the application ran successfully on the target machine? Can you double check that you have set a GPU breakpoint and see if you hit those breakpoint(s)?
Are there any other messages shown? - you can check the VS Output window.
Which driver version are you using?
Can you try running (double clicking) the application on the server but from the directory that Nsight synced the application to? It should be under %appdata\NVIDIA Corporation\Nsight\Monitor\Mirror\<hostdev_machine_name>\<path_to_the_sync_app>
I know you mentioned you tried copying it, but I'd like to see if everything that Nsight sync is what's required by the application and that maybe there's not something missing that you need to specify to sync (more info on syncing is here: http://http.developer.nvidia.com/NsightVisualStudio/3.0/Documentation/UserGuide/HTML/Content/Synchronization.htm)
Thank you

Related

Nsight Debugging using single GPU

I have a single GPU in my Windows 7 system. Would like to debug my gpu code locally on this machine.
There is a confusion regarding this. Do I need to do headless debugging, (may be making my on-board display as display driver), as explained in Setup Local Headless GPU Debugging?
Or do I need not do any thing like that?
You cannot do local headless debugging with only a single GPU. Headless means there is no monitor or active display attached to the GPU that is running the code under debug. If you are debugging locally, you need this display to see the nsight GUI and your windows desktop.
Single GPU local (non-headless) debugging is covered in the nsight manual.
If you can enable another GPU (need not be an NVIDIA GPU), then you can use that GPU for your windows display, and do headless debugging on the NVIDIA GPU.

Developing using CUDA on several computers, when only one has a GPU installed

I am a Java developer. To speed some of our algorithms, we have decided to try CUDA.
But the Issue is, currently we have only one server with GPU installed and 3 developers have to work on it (by transferring the file each time over ssh and compiling and running it over there). This obviously is a tedious process.
What I would like to know is: On my machine which does not have GPU, can I using NSight work on CUDA by compiling and generating files locally. This can automatically be transferred to server to get the result.
If we can at least work on algorithm locally using NSight (or any other IDE) and not pure vim and then compile it to remove compile time errors, this would save quite some time.
On Linux you can do remote debugging using Nsight Eclipse Edition as documented here. This requires 5.5 or later. On Windows you need to start the Nsight monitor on the server and then just configure Nsight Visual Studio Edition to use the remote machine.

NVIDIA Nsight Debugging on GTX 480

I have one machine with GeForce GTX 480 but I can't debug or run analysis activity on it.
This error appears when I debug or run analysis activity:
The remote system is logged in through Remote Desktop. CUDA debugging
does not support Remote Desktop unless a TCC adapter is installed.
My Nsight version is 2.2. Whats wrong?
You would need a TCC capable card (Quadro or Tesla series) if you want to use the GPU from inside a RPD session.
When you do not have a TCC capable GPU, you can for example install VNC Server as a service and use that to access the machine.
Alternatively you can configure windows autologin to log you in automatically and have have the Nsight Monitor in the startup group (but to access the desktop remotely you would still need some kind of remote access software like VNC).
after some test and try ,I found that this problem is because of RDC and if you connect to the target machine from one computer with remote desktop connection even if you disconnect all remote desktop and try to debug with Nsight remotely or locally you get this error .
I were connected to the target machine with remote desktop and configure and run Nsight monitor and then close the RDC . and then I tried to debug the project remotely from host on the target and because of it I get this error
I found that one solution will be restart the target machine and configure the Monitor physically at front of the target machine and then DO NOT remote to the target from anywhere, now I can debug my project from host machine on the target machine .

CUDA Parallel NSight Debugging host and device simultaneously

Does anyone know if its possible to Debug CUDA using parallel NSight on a remote machine? I am able to step into CUDA code but not my host code. It says CUDA has the capability to generate host debug information so debugging remotely and locally should be possible.
My card is a 580 GTX.
//device code <-- able to debug device code
//host code <---- when device code returns, should be able to debug host code
Thanks!
Simultaneous GPU/CPU debugging from a single IDE instance is unfortunately not possible with the current releases of Nsight and Visual Studio.
As a workaround, you can start GPU debugging from one copy of Visual Studio, then open a second IDE instance and attach its CPU debugger. They won't have unified stepping, but you can at least set breakpoints independently.
It should now be possible to attach both the Visual Studio default debugger and NSight in the same VS instance. Then this should work.

Debugger in CUDA 5

Nvidia has released extended eclipse for CUDA 5. They have Nsight plugin for VS2010 also. In VS2010 we can stop program execution at breakpoint in kernel but how to achieve this functionality in eclipse on Linux? I don't see any nsight specific keys to stop execution. I tried changing perspective but it debugs as a normal C/C++ application. I'm using Tesla C2070, Intel Xeon 8 core machine with Linux.
I'm from Nsight Eclipse Edition team.
Our goal is specifically for the application to be debugged as a normal C/C++ application. This means that you can set breakpoints, use "run to line", etc. regardless of whether you debug host or device code.
Basically, the process is quite standard for Eclipse:
Create a project (you can also import existing executable)
Click debug button
Debugger will run and by default will break in the main function. Note that no device code posted on the device so you will only see the host thread.
Set a breakpoint in the device code and hit resume (note that Breakpoints view toolbar also allows you setting breakpoint on any CUDA kernel launch)
Debugger will break when device code reaches the breakpoint. You can inspect your application state using visual debugger UI.
Couple things, and not sure which solved the issue. Drivers updated to latest ones with RC5.0, but I chose to run VNC server instead of native X server. Then the CUDA card(s) are dedicated to my apps and debugging, and it works like a charm, and now accessible from everywhere.
Eugene,
I just installed Cuda 5, and I wasn't able to break in any kernel code. It was a clean install of centos 5.5, with a fresh download of cuda-5, and i am running on a asus g71x laptop which has a gtx260m installed.
I thought maybe you cant run display and dedbug on one device still, so i switched to non-nv x display, but still had same issue, cant stop in the kernel code.
Have you tried CUDA 5.0 RC1? It is available now. You can download and try it. And I have tried the Nsight in it, it works well for debugging.
Best regards!
The 304.43 NVIDIA Driver does not let users other than root debug their CUDA application.
That problem is not present in any past or future public releases. The CUDA documentation recommends using only drivers listed in the CUDA DevZone. The 304.43 driver is not one of them.
That may or may not be the issue you are hitting. But I thought it was worth mentioning.