CudaMemcpyToSymbol fails when called from constructor [duplicate] - cuda

I have a class that calls a kernel in its constructor, as follows:
"ScalarField.h"
#include <iostream>
void ERROR_CHECK(cudaError_t err,const char * msg) {
if(err!=cudaSuccess) {
std::cout << msg << " : " << cudaGetErrorString(err) << std::endl;
std::exit(-1);
}
}
class ScalarField {
public:
float* array;
int dimension;
ScalarField(int dim): dimension(dim) {
std::cout << "Scalar Field" << std::endl;
ERROR_CHECK(cudaMalloc(&array, dim*sizeof(float)),"cudaMalloc");
}
};
"classA.h"
#include "ScalarField.h"
static __global__ void KernelSetScalarField(ScalarField v) {
int index = threadIdx.x + blockIdx.x * blockDim.x;
if (index < v.dimension) v.array[index] = 0.0f;
}
class A {
public:
ScalarField v;
A(): v(ScalarField(3)) {
std::cout << "Class A" << std::endl;
KernelSetScalarField<<<1, 32>>>(v);
ERROR_CHECK(cudaGetLastError(),"Kernel");
}
};
"main.cu"
#include "classA.h"
A a_object;
int main() {
std::cout << "Main" << std::endl;
return 0;
}
If i instantiate this class on main (A a_object;) i get no errors. However, if I instantiate it outside main, just after defining it (class A {...} a_object;) I get an "invalid device function" error when the kernel launches. Why does that happen?
EDIT
Updated code to provide a more complete example.
EDIT 2
Following the advice in the comment by Raxvan, I wanted to say i have the dimensions variable used in ScalarField constructor also defined (in another class) outside main, but before everything else. Could that be the explanation? The debugger was showing the right value for dimensions though.

The short version:
The underlying reason for the problem when class A is instantiated outside of main is that a particular hook routine which is required to initialise the CUDA runtime library with your kernels is not being run before the constructor of class A is being called. This happens because there are no guarantees about the order in which static objects are instantiated and initialised in the C++ execution model. Your global scope class is being instantiated before the global scope objects which do the CUDA setup are initialised. Your kernel code is never being loaded into the context before it is call, and a runtime error results.
As best as I can tell, this is a genuine limitation of the CUDA runtime API and not something easily fixed in user code. In your trivial example, you could replace the kernel call with a call to cudaMemset or one of the non-symbol based runtime API memset functions and it will work. This problem is completely limited to user kernels or device symbols loaded at runtime via the runtime API. For this reason, an empty default constructor would also solve your problem. From a design point of view, I would be very dubious of any pattern which calls kernels in the constructor. Adding a specific method for class GPU setup/teardown which doesn't rely on the default constructor or destructor would be a much cleaner and less error prone design, IMHO.
In detail:
There is an internally generated routine (__cudaRegisterFatBinary) which must be run to load and register kernels, textures and statically defined device symbols contained in the fatbin payload of any runtime API program with the CUDA driver API before the kernel can be called without error. This is a part of the "lazy" context initialisation feature of the runtime API. You can confirm this for yourself as follows:
Here is a gdb trace of the revised example you posted. Note I insert a breakpoint into __cudaRegisterFatBinary, and that isn't reached before your static A constructor is called and the kernel launch fails:
talonmies#box:~$ gdb a.out
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.launchpad.net/gdb-linaro/>...
Reading symbols from /home/talonmies/a.out...done.
(gdb) break '__cudaRegisterFatBinary'
Breakpoint 1 at 0x403180
(gdb) run
Starting program: /home/talonmies/a.out
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Scalar Field
[New Thread 0x7ffff5a63700 (LWP 10774)]
Class A
Kernel : invalid device function
[Thread 0x7ffff5a63700 (LWP 10774) exited]
[Inferior 1 (process 10771) exited with code 0377]
Here is the same procedure, this time with A instantiation inside main (which is guaranteed to happen after the objects which perform lazy setup have been initialised):
talonmies#box:~$ cat main.cu
#include "classA.h"
int main() {
A a_object;
std::cout << "Main" << std::endl;
return 0;
}
talonmies#box:~$ nvcc --keep -arch=sm_30 -g main.cu
talonmies#box:~$ gdb a.out
GNU gdb (Ubuntu/Linaro 7.4-2012.04-0ubuntu2.1) 7.4-2012.04
Copyright (C) 2012 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
For bug reporting instructions, please see:
<http://bugs.launchpad.net/gdb-linaro/>...
Reading symbols from /home/talonmies/a.out...done.
(gdb) break '__cudaRegisterFatBinary'
Breakpoint 1 at 0x403180
(gdb) run
Starting program: /home/talonmies/a.out
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Breakpoint 1, 0x0000000000403180 in __cudaRegisterFatBinary ()
(gdb) cont
Continuing.
Scalar Field
[New Thread 0x7ffff5a63700 (LWP 11084)]
Class A
Main
[Thread 0x7ffff5a63700 (LWP 11084) exited]
[Inferior 1 (process 11081) exited normally]
If this is really a crippling problem for you, I would suggest contacting NVIDIA developer support and raising a bug report.

Related

Unable to create a thrust device vector

So I'm trying to start on GPU programming and using the Thrust library to simplify things.
I have created a test program to work with it and see how it works, however whenever I try to create a thrust::device_vector with non-zero size the program crashes with "Run-time Check Failure #3 - The variable 'result' is being used without being initialized.' (this comes from the allocator_traits.inl file) And... I have no idea how to fix this.
The following is all that is needed to cause this error.
#include <thrust/device_vector.h>
int main()
{
int N = 100;
thrust::device_vector<int> d_a(N);
return 0;
}
I suspect it may be a problem with how the environment is set up so the details on that are...
Created using visual studio 2019, in a CUDA 11.0 Runtime project (the example program given when opening this project works fine, however), Thrust version 1.9, and the given GPU is a GTX 970.
This issue only seems to manifest with the thrust version (1.9.x) associated with CUDA 11.0, and only in debug projects on windows/Visual Studio.
Some workarounds would be to switch to building a release project, or just click "Ignore" on the dialogs that appear at runtime. According to my testing this allows ordinary run or debug at that point.
I have not confirmed it, but I believe this issue is fixed in the latest thrust (1.10.x) just released (although not part of any formal CUDA release at this moment, I would expect it to be part of some future CUDA release).
Following the Answer of Robert Crovella, I fixed this issue by changing the corresponding lines of code in the thrust library with the code from GitHub. More precisely, in the file ...\CUDA\v11.1\include\thrust\detail\allocator\allocator_traits.inl I replaced the following function
template<typename Alloc>
__host__ __device__
typename disable_if<
has_member_system<Alloc>::value,
typename allocator_system<Alloc>::type
>::type
system(Alloc &)
{
// return a copy of a default-constructed system
typename allocator_system<Alloc>::type result;
return result;
}
by
template<typename Alloc>
__host__ __device__
typename disable_if<
has_member_system<Alloc>::value,
typename allocator_system<Alloc>::type
>::type
system(Alloc &)
{
// return a copy of a default-constructed system
return typename allocator_system<Alloc>::type();
}

Invalid device symbol error depending on nvcc flags

Toy program:
#include <iostream>
#include <vector>
// Matrix side size (they are square).
const int N = 3;
const int num_mats = 14;
// Rotation matrices.
__constant__ float rot_mats_device[num_mats*N*N];
int main() {
std::vector<float> rot_mats_host(num_mats*N*N);
for (int i = 0; i < rot_mats_host.size(); i++)
rot_mats_host[i] = i;
auto errMemcpyToSymbol = cudaMemcpyToSymbol(rot_mats_device,
rot_mats_host.data(),
sizeof(rot_mats_device));
if (errMemcpyToSymbol != cudaSuccess) {
std::cout << "MemcpyToSymbol error: " <<
cudaGetErrorString(errMemcpyToSymbol) << std::endl;
}
}
Compiled with
nvcc -arch=sm_52 -std=c++11 cuda_invalid_symbol_error.cu -o cuda_invalid_symbol_error
does not give any error during runtime. However, with
nvcc -gencode arch=compute_52,code=sm_52 -std=c++11 cuda_invalid_symbol_error.cu -o cuda_invalid_symbol_error
it will fail with the message MemcpyToSymbol error: invalid device symbol.
Why do the latter instructions for compilation give the runtime error?
Specs: cuda 8.0, Ubuntu 16.04, GeForce GTX 1060 (I know the cc of this card is 6.1).
Why do the latter instructions for compilation give the runtime error?
-arch=sm_xx is shorthand for:
-gencode arch=compute_xx,code=sm_xx -gencode arch=compute_xx,code=compute_xx
In your case, where xx is 52, this command embeds both cc 5.2 PTX code (the second gencode instance) and cc 5.2 SASS code (the first gencode instance). The SASS code for cc 5.2 will not run on your cc6.1 device, so the runtime JIT-compiles the cc 5.2 PTX code to create an object compatible with your cc 6.1 architecture. All is happy and everything works.
When you instead compile with:
nvcc -gencode arch=compute_52,code=sm_52 ...
you are omitting the PTX code from the compiled object. Only the cc 5.2 SASS code is present. This code will not run on your cc6.1 device, and the runtime has no other options, so a "hidden" error of NO_BINARY_FOR_GPU occurs when the runtime attempts to load the GPU image for your program. Since no image gets loaded, no device symbol is present/usable. Since it is not present/usable, you get the invalid device symbol error when you attempt to refer to it using the CUDA runtime API.
If you had performed another CUDA runtime API call prior to this which forced a sufficient or equivalent level of initialization of the CUDA runtime (and checked the returned error code), you would have received a NO_BINARY_FOR_GPU error or similar. Certainly, for example, if you had attempted to launch a GPU kernel, you would receive that error. There may be other CUDA runtime API calls that would force an equivalent or sufficient level of lazy initialization, but I don't have a list of those.

Online compilation of single CUDA function

I have a function in my program called float valueAt(float3 v). It's supposed to return the value of a function at the given point. The function is user-specified. I have an interpreter for this function at the moment, but others recommended I compile the function online so it's in machine code and is faster.
How do I do this? I believe I know how to load the function when I have PTX generated, but I have no idea how to generate the PTX.
CUDA provides no way of runtime compilation of non-PTX code.
What you want can be done, but not using the standard CUDA APIs. PyCUDA provides an elegant just-in-time compilation method for CUDA C code which includes behind the scenes forking of the toolchain to compile to device code and loading using the runtime API. The (possible) downside is that you need to use Python for the top level of your application, and if you are shipping code to third parties, you might need to ship a working Python distribution too.
The only other alternative I can think of is OpenCL, which does support runtime compilation (that is all it supported until recently). The C99 language base is a lot more restrictive than what CUDA offers, and I find the APIs to be very verbose, but the runtime compilation model works well.
I've thought about this problem for a while, and while I don't think this is a "great" solution, it does seem to work so I thought I would share it.
The basic idea is to use linux to spawn processes to compile and then run the compiled code. I think this is pretty much a no-brainer, but since I put together the pieces, I'll post instructions here in case it's useful for somebody else.
The problem statement in the question is to be able to take a file that contains a user-defined function, let's assume it is a function of a single variable f(x), i.e. y = f(x), and that x and y can be represented by float quantities.
The user would edit a file called fx.txt that contains the desired function. This file must conform to C syntax rules.
fx.txt:
y=1/x
This file then gets included in the __device__ function that will be holding it:
user_testfunc.cuh:
__device__ float fx(float x){
float y;
#include "fx.txt"
;
return y;
}
which gets included in the kernel that is called via a wrapper.
cudalib.cu:
#include <math.h>
#include "cudalib.h"
#include "user_testfunc.cuh"
__global__ void my_kernel(float x, float *y){
*y = fx(x);
}
float cudalib_compute_fx(float x){
float *d, *h_d;
h_d = (float *)malloc(sizeof(float));
cudaMalloc(&d, sizeof(float));
my_kernel<<<1,1>>>(x, d);
cudaMemcpy(h_d, d, sizeof(float), cudaMemcpyDeviceToHost);
return *h_d;
}
cudalib.h:
float cudalib_compute_fx(float x);
The above files get built into a shared library:
nvcc -arch=sm_20 -Xcompiler -fPIC -shared cudalib.cu -o libmycudalib.so
We need a main application to use this shared library.
t452.cu:
#include <stdio.h>
#include <stdlib.h>
#include "cudalib.h"
int main(int argc, char* argv[]){
if (argc == 1){
// recompile lib, and spawn new process
int retval = system("nvcc -arch=sm_20 -Xcompiler -fPIC -shared cudalib.cu -o libmycudalib.so");
char scmd[128];
sprintf(scmd, "%s skip", argv[0]);
retval = system(scmd);}
else { // compute f(x) at x = 2.0
printf("Result is: %f\n", cudalib_compute_fx(2.0));
}
return 0;
}
Which is compiled like this:
nvcc -arch=sm_20 -o t452 t452.cu -L. -lmycudalib
At this point, the main application (t452) can be executed and it will produce the result of f(2.0) which is 0.5 in this case:
$ LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH ./t452
Result is: 0.500000
The user can then modify the fx.txt file:
$ vi fx.txt
$ cat fx.txt
y = 5/x
And just re-run the app, and the new functional behavior is used:
$ LD_LIBRARY_PATH=.:$LD_LIBRARY_PATH ./t452
Result is: 2.500000
This method takes advantage of the fact that upon recompilation/replacement of a shared library, a new linux process will pick up the new shared library. Also note that I've omitted several kinds of error checking for clarity. At a minimum I would check CUDA errors, and I would also probably delete the shared object (.so) library before recompiling it, and then test for its existence after compilation, to do a basic test that the compilation proceeded successfully.
This method entirely uses the runtime API to achieve this goal, so as a result the user would have to have the CUDA toolkit installed on their machine and appropriately set up so that nvcc is available in the PATH. Using the driver API with PTX code would make this process much cleaner (and not require the toolkit on the user's machine), but AFAIK there is no way to generate PTX from CUDA C without using nvcc or a user-created toolchain built on the nvidia llvm compiler tools. In the future, there may be a more "integrated" approach available in the "standard" CUDA C toolchain, or perhaps even by the driver.
A similar approach can be arranged using separate compilation and linking of device code, such that the only source code that needs to be exposed to the user is in user_testfunc.cu (and fx.txt).
EDIT: There is now a CUDA runtime compilation facility, which should be used in place of the above.

thrust::device_vector error

I'm new to Thrust. I'm trying to copy from a thrust::host_vector to a thrust::device_vector, both of type Sequence which is a class I already implemented.
I do however get an error "Invalid Device Function".
I'm using CUDA 4.0 VS2010 on a GeForce GT 540.
thrust::host_vector <Sequence> Ind_Tabel_V;
void Ind_Table_Filling()
{
//some Code
Sequence s;
// some code
Ind_Tabel_V.push_back(s);
try
{
thrust::device_vector<Sequence> d_vec=Ind_Tabel_V;
}
catch (thrust::system_error &e)
{
std::cerr << "Error accessing vector element: " << e.what() << std::endl;
}
}
Can anyone help please?
That error message typically means the runtime cannot find a binary matching your GPU architecture, i.e. you have not included the correct GPU SM version in your compilation. Since you're using VS2010 the GPU architecture is usually set via the build customisation. In the project properties under CUDA C/C++, Device you should see the "Code Generation" option. I'm not sure what generation your GPU is but you could try "compute_20,sm_20;compute_20,sm_21" to build for both Fermi architectures.

CUDA External calls not supported

I am developing a CUDA 4.0 application running on a Fermi card. According to the specs, Fermi has Compute Capability 2.0 and therefore should support non-inlined function calls.
I compile every class I have with nvcc 4.0 in a distinct obj file. Then, I link them all with g++-4.4.
Consider the following code :
[File A.cuh]
#include <cuda_runtime.h>
struct A
{
__device__ __host__ void functionA();
};
[File B.cuh]
#include <cuda_runtime.h>
struct B
{
__device__ __host__ void functionB();
};
[File A.cu]
#include "A.cuh"
#include "B.cuh"
void A::functionA()
{
B b;
b.functionB();
}
Attempting to compile A.cu with nvcc -o A.o -c A.cu -arch=sm_20 outputs Error: External calls are not supported (found non-inlined call to _ZN1B9functionBEv).
I must be doing something wrong, but what ?
As explained on this thread on the NVidia forums, it appears that even though Fermi supports non-inlined functions, nvcc still needs to have all the functions available during compilation, i.e. in the same source file: there is no linker (yep, that's a pity...).
functionB is not declared and therefore considered external call. As the error said external calls are not supported. Implement functionB and it will work.
True, CUDA 5.0 does it. I can't get it to expose external device variables but device methods work just fine. Not by default.
The nvcc option is "-rdc=true". In Visual Studio and Nsight it is an option in the project properties under Configuration Properties -> CUDA C/C++ -> Common -> Generate Relocatable Device Code.