I'm trying to create a CUDA program (which I'm new at) that involves first grabbing information from a remote MySQL database. I'm using the Connector/C library from the MySQL website inside the program, before the CUDA calls.
I'm able to compile my program with MySQL when using gcc (without any CUDA code), but not with nvcc (the CUDA compiler). A peer who is familiar with CUDA mentioned to me that he had to compile some libjpg stuff he was doing with nvcc to avoid 'wrong architecture' and linking problems. He suggested that I compile the Connector/C library with nvcc. However, the Connector/C library uses CMake instead of a regular Makefile.
So, being new to CMake, I researched some stuff and found the toolchain file which sounded a lot like what I needed (found here). However, I am running into problems during the compile where all of the default includes and libraries used in Connector/C are not included. Specifically
-- Looking for include files HAVE_ALLOCA_H
-- Looking for include files HAVE_ALLOCA_H - not found.
and
-- Looking for strstr
-- Looking for strstr - not found
Those are just a couple examples, there are many more files that are not found.
Am I approaching this problem correctly? Is there a more obvious workaround that I am just not considering? If I am right in trying to compile MySQL Connector/C with CUDA, are there any suggestions for properly including the files and libraries required for Connector/C?
Thanks for your help.
If you can separate out the CUDA kernels from your mysql calls and place them in separate files, you can use your Makefile.
I keep all of the cuda kernels and such in .cu files and then I have a definition:
#
# CUDA Compilation Rules
#
define cuda-compile-rule
$1: $(call generated-source,$2) \
$(call source-dir-to-build-dir, $(subst .cu,.cubin, $2)) \
$(call source-dir-to-build-dir, $(subst .cu,.ptx, $2))
$(NVCC) $(CUBIN_ARCH_FLAG) $(NVCCFLAGS) $(INCFLAGS) $(DEFINES) -o $$# -c $$<
$(call source-dir-to-build-dir, $(subst .cu,.cubin, $2)): $(call generated-source,$2)
$(NVCC) -cubin -Xptxas -v $(CUBIN_ARCH_FLAG) $(NVCCFLAGS) $(INCFLAGS) $(DEFINES) $(SMVERSIONFLAGS) -o $$# $$<
$(call source-dir-to-build-dir, $(subst .cu,.ptx, $2)): $(call generated-source,$2)
$(NVCC) -ptx $(CUBIN_ARCH_FLAG) $(NVCCFLAGS) $(INCFLAGS) $(DEFINES) $(SMVERSIONFLAGS) -o $$# $$<
endef
I've also included three functions for ease of use:
generated-source = $(filter %.cpp, $1) $(filter %.c, $1) $(filter %.f, $1) $(filter %.F, $1) $(filter %.cu, $1)
source-dir-to-build-dir = $(addprefix $(BUILDDIR)/, $1)
source-to-object = $(call source-dir-to-build-dir, \
$(subst .f,.o,$(filter %.f,$1)) \
$(subst .F,.o,$(filter %.F,$1)) \
$(subst .c,.o,$(filter %.c,$1)) \
$(subst .cpp,.o,$(filter %.cpp,$1)) \
$(if $(filter 1,$(USE_CUDA)),$(subst .cu,.cu.o,$(filter %.cu,$1))))
Then all you need to do is build up a list of source files and call:
$(foreach f,$(filter %.cu, $listOfFiles),$(call cuda-compile-rule,$(call source-to-object,$f),$f))
Note that in the function source-to-object there is a variable which I use to conditionally disable CUDA compilation USE_CUDA.
Related
I've been trying to convert a cpp file to wasm without emscripten recently and I've been running with some errors, for example, when I run these commands:
clang++ --target=wasm32 -nostdlib -O3 -o public/main.o -c src/*.cpp
wasm-ld --no-entry --export-all --lto-O3 --allow-undefined --import-memory public/main.o -o public/main.wasm
it gives me this error:
wasm-ld: error: unknown file type public/main.o
Here are the versions of clang and lld that I currently have:
clang version 12.0.1
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /nix/store/jp4r5v8pla63qam5w34jvfyfmq8p74am-clang-12.0.1/bin
LLD 12.0.1
Also, I'm running the code on replit
Thank you
Ok, with this commands it works :D
clang++ --target=wasm32 -emit-llvm -c -S -o public/files/main.ll src/main.cpp
llc -march=wasm32 -filetype=obj -o public/files/main.o public/files/main.ll
wasm-ld --no-entry --export-all -o public/main.wasm public/files/main.o
The only thing is that you need to create some new files (main.ll and main.o) but it doesn't matter.
The place where I obtained the solution is here: https://surma.dev/things/c-to-webassembly/index.html
It was really useful
I have a project in C called triple where you can add, delete and match some Triplets. The idea now is to transform it to html using emcc and emmake.
I tried to compile it with:
emmake make
And then use:
emcc triple.o -s WASM=1 -o triple.html
But I get the error :
WARNING:root:triple.o is not valid LLVM bitcode
ERROR:root:no input files
note that input files without a known suffix are ignored, make sure
your input files end with one of: ('.c', '.C', '.i', '.cpp', '.cxx',
'.cc', '.c++', '.CPP', '.CXX', '.CC', '.C++', '.ii', '.m', '.mi',
'.mm', '.mii', '/dev/null', '.bc', '.o', '.obj', '.lo', '.dylib',
'.so', '.a', '.ll', '.h', '.hxx', '.hpp', '.hh', '.H', '.HXX', '.HPP',
'.HH')
What am I missing? Is there another way to use the make file with emcc instead with gcc?
Here is the make file I am using.
CC=emcc
triple : triple.o insert.o match.o Delete.o printList.o writeList.o
$(CC) -o triple triple.o insert.o match.o Delete.o printList.o
writeList.o
triple.o : triple.c
$(CC) -g -c triple.c
insert.o : insert.c
$(CC) -g -c insert.c
match.o : match.c
$(CC) -g -c match.c
Delete.o : Delete.c
$(CC) -g -c Delete.c
printList.o : printList.c
$(CC) -g -c printList.c
writeList.o : writeList.c
$(CC) -g -c writeList.c
The emcc program is a compiler front-end. This means it takes source code as input. You do not need to compile the code first with GCC. The emscripten website says it best: "use Emscripten Compiler Frontend (emcc) as a drop-in replacement for gcc in your existing project."
If you have the source there seems to be no good reason to compile to LLVM first.
What you need to do is simply replace any reference to gcc in your Makefile with emcc.
Even better - add a CC variable and use this. eg
CC=emcc
then replace all references to the compiler with $(CC). the $ bit is how to access a variable in a Makefile. Using a variable means you can easily change the compiler later.
You haven't specified an output target for the .o lines, so I think it will build to a.o.
Try modifying your makefile to say:
$(CC) -g -c triple.c -o triple.o
I recently tried to build my https://github.com/eyalroz/cuda-api-wrappers/ library's examples after switching to another Linux distribution on the same machine. Strangely enough, I encountered a linking issue. The command:
/usr/bin/c++ -Wall -std=c++11 -g CMakeFiles/device_management.dir/examples/by_runtime_api_module/device_management.cpp.o -o examples/bin/device_management -rdynamic lib/libcuda-api-wrappers.a -Wl,-Bstatic -lcudart_static -Wl,-Bdynamic -lpthread -ldl -lrt
fails to find the CUDA runtime library, and I get:
CMakeFiles/device_management.dir/examples/by_runtime_api_module/device_management.cpp.o: In function `cuda::device::peer_to_peer::get_attribute(cudaDeviceP2PAttr, int, int)':
/home/eyalroz/src/mine/cuda-api-wrappers/src/cuda/api/device.hpp:38: undefined reference to `cudaDeviceGetP2PAttribute'
collect2: error: ld returned 1 exit status
but if I add -L/usr/local/cuda/lib64 it builds fine. This didn't use to happen before; and it doesn't happen on another machine I've checked on, nor does it even happen to other targets using the CUDA runtime in the same CMakeLists.txt (like version_managament).
FindCUDA seems to be finding everything, as the value of ${CUDA_LIBRARIES} is /usr/local/cuda/lib64/libcudart_static.a;-lpthread;dl;/usr/lib/x86_64-linux-gnu/librt.so. And the target lines in CMakeLists.txt are:
add_executable(device_management EXCLUDE_FROM_ALL examples/by_runtime_api_module/device_management.cpp)
target_link_libraries(device_management cuda-api-wrappers ${CUDA_LIBRARIES})
as is suggested in answers to other related questions (e.g. here). Why is this happening? Should I "manually" add the -L switch?
Edit: Following #RobertCrovella's suggestion, here are the ld search paths:
$ gcc -print-search-dirs | sed '/^lib/b 1;d;:1;s,/[^/.][^/]*/\.\./,/,;t 1;s,:[^=]*=,:;,;s,;,; ,g' | tr \; \\012 | tr ':' "\n" | tail -n +3
/usr/local/cuda/lib64/x86_64-linux-gnu/5/
/usr/local/cuda/lib64/x86_64-linux-gnu/
/usr/local/cuda/lib/
/usr/lib/gcc/x86_64-linux-gnu/5/
/usr/x86_64-linux-gnu/lib/x86_64-linux-gnu/5/
/usr/x86_64-linux-gnu/lib/x86_64-linux-gnu/
/usr/x86_64-linux-gnu/lib/
/usr/lib/x86_64-linux-gnu/5/
/usr/lib/x86_64-linux-gnu/
/usr/lib/
/lib/x86_64-linux-gnu/5/
/lib/x86_64-linux-gnu/
/lib/
/usr/lib/x86_64-linux-gnu/5/
/usr/lib/x86_64-linux-gnu/
/usr/lib/
/usr/local/cuda/lib64/
/usr/x86_64-linux-gnu/lib/
/usr/lib/
/lib/
/usr/lib/
$ ld --verbose | grep SEARCH_DIR | tr -s ' ;' \\012
SEARCH_DIR("=/usr/local/lib/x86_64-linux-gnu")
SEARCH_DIR("=/lib/x86_64-linux-gnu")
SEARCH_DIR("=/usr/lib/x86_64-linux-gnu")
SEARCH_DIR("=/usr/local/lib64")
SEARCH_DIR("=/lib64")
SEARCH_DIR("=/usr/lib64")
SEARCH_DIR("=/usr/local/lib")
SEARCH_DIR("=/lib")
SEARCH_DIR("=/usr/lib")
SEARCH_DIR("=/usr/x86_64-linux-gnu/lib64")
SEARCH_DIR("=/usr/x86_64-linux-gnu/lib")
Notes:
Yes, I know the CMakeLists.txt there is ugly.
TL;DR:
After the FindCUDA invocation, add the lines:
get_filename_component(CUDA_LIBRARY_DIR ${CUDA_CUDART_LIBRARY} DIRECTORY)
set(CMAKE_EXE_LINKER_FLAGS ${CMAKE_EXE_LINKER_FLAGS} "-L${CUDA_LIBRARY_DIR}")
and building should succeed on both systems.
Discussion:
(Paraphrasing #RobertCrovella and myself in the comments:)
OP was expecting, that if the following hold:
FindCUDA succeeds
${CUDA_LIBRARIES} includes a valid full path to either the static or the dynamic CUDA runtime library
the library dependency is indicated using target_link_libraries(relevant_target ${CUDA_LIBRARIES})
... then the CMake-based build he was attempting should succeed on a variety of valid CUDA installations. That is (unfortunately) not the case, since while FindCUDA does locate the CUDA library path, it does not actually make your linker search that path. So a failure should actually be expected. The build had worked on OP's old system due to a "fluke", or rather, due to OP having added the CUDA library directory to the linker's search path, somehow, apriori.
The linking command must be issued with the -L/path/to/cuda/libraries switch, so that the linker knows where to looks for the (unspecified-path) libraries referred to be the CUDA-related -l switches (in OP's case, -lcudart_static).
This answer discusses how to do that in CMake for different kinds of targets. You might also want to have a look at man gcc (the GCC manual page, also available here) regarding the -l and -L options, if you are not familiar with them.
I have two versions of a function in an application, one implemented in CUDA and the other in standard C. They're in separate files, let's say cudafunc.h and func.h (the implementations are in cudafunc.cu and func.c). I'd like to offer two options when compiling the application. If the person has nvcc installed, it'll compile the cudafunc.h. Otherwise, it'll compile func.h.
Is there anyway to check if a machine has nvcc installed in the makefile and thus adjust the compiler accordingly?
Thanks a bunch,
You could try a conditional, like
ifeq (($shell which nvcc),) # No 'nvcc' found
func.o: func.c func.h
HEADERS += func.h
else
func.o: cudafunc.cu cudafunc.h
nvcc -o $# -c $< . . .
CFLAGS += -DUSE_CUDA_FUNC
HEADERS += cudafunc.h
endif
And then in the code that will call this function, it can test #if USE_CUDA_FUNC to decide which header to include and which interface to call.
This should work, included in your Makefile:
NVCC_RESULT := $(shell which nvcc 2> NULL)
NVCC_TEST := $(notdir $(NVCC_RESULT))
ifeq ($(NVCC_TEST),nvcc)
CC := nvcc
else
CC := g++
endif
test:
#echo $(CC)
For GNU make, the recipe line(s) (after test:) actually start with tab characters.
With the above preamble, you can use conditionals based on the CC variable to control the remainder of your Makefile.
You would change the test: target to whatever target you want to be built conditionally.
As a test, just run the above with make. You should get output of nvcc or g++ based on whatever was detected.
I'm just trying to build an old project of mine using cuda 5.0 preview.
I get an Error when linking, telling me that certain cuda functions can not be found. For example:
undefined reference to 'cudaMalloc'.
My linking command includes the following options for cuda :
-L/usr/local/cuda/lib64 -L/home/myhome/NVIDIA_CUDA_Samples/C/lib -L/home/myhome/NVIDIA_CUDA_Samples/C/common/lib/linux -lcudart
ls -lah /usr/local/cuda/lib64/ gives me 8 cuda libraries including libcudart.so.5.0.7 with symlinks using only the .so-file-ending.
ls /home/myhome/NVIDIA_CUDA_Samples/C/lib/ gives me an empty directory, which is kind of strange?
ls /home/myhome/NVIDIA_CUDA_Samples/C/common/lib/linux/ gives me two directories: i686 and x86_64 both containing only libGLEW.a
I have no idea which way to look for a solution. Any help is appreciated!
EDIT:
Here is my complete linking command (TARGET_APPLICATION is my binary and x86_64/Objectfiles.o stands for all (23) object files including the object file compiled with nvcc):
/home/myhome/nullmpi-0.7/bin/mpicxx -CC=g++ -I. -I/home/myhome/nullmpi-0.7/src -I/usr/lib/openmpi/include -L/usr/local/cuda/lib64 -L/home/myhome/NVIDIA_CUDA_Samples/C/lib -L/home/myhome/NVIDIA_CUDA_Samples/C/common/lib/linux -lcudart -o TARGET_APPLICATION x86_64/Objectfiles.o /usr/lib/liblapack.so /usr/lib/libblas.so /home/myhome/nullmpi-0.7/lib/libnullpmpi.a -lm
I use nullmpi for compilation and linking (project uses MPI and CUDA), which internally uses g++ as can be seen by -CC=g++, i wanted to keep this stuff out.
The compilation command for my cuda object file:
/usr/local/cuda/bin/nvcc -c -arch=sm_21 -L/home/myhome/NVIDIA_CUDA_Samples/C/lib -O3 kernelwrapper.cu -o x86_64/kernelwrapper.RELEASE.2.o
echo $LD_LIBRARY_PATH results in:
/usr/local/cuda/lib64:/usr/local/cuda/lib:
echo $PATH results in:
otherOptions:/usr/local/cuda/bin:/home/myhome/nullmpi-0.7/bin
I'm building 64-bit. For the sake of completeness I'm building on Ubuntu 12.04. (64bit).
Building the CUDA Samples works fine.
SOLUTION (thanks to talonmies for pointing me to it):
This is the correct linking command:
/home/myhome/nullmpi-0.7/bin/mpicxx -CC=g++ -I. -I/home/myhome/nullmpi-0.7/src -I/usr/lib/openmpi/include -L/usr/local/cuda/lib64 -L/home/myhome/NVIDIA_CUDA_Samples/C/lib -L/home/myhome/NVIDIA_CUDA_Samples/C/common/lib/linux -o TARGET_APPLICATION x86_64/Objectfiles.o /usr/lib/liblapack.so /usr/lib/libblas.so /home/myhome/nullmpi-0.7/lib/libnullpmpi.a -lcudart -lm
You have your linking statements in the incorrect order. It should be something more like this:
/home/myhome/nullmpi-0.7/bin/mpicxx -CC=g++ -I. -I/home/myhome/nullmpi-0.7/src \
-I/usr/lib/openmpi/include -L/usr/local/cuda/lib64 \
-L/home/myhome/NVIDIA_CUDA_Samples/C/lib \
-L/home/myhome/NVIDIA_CUDA_Samples/C/common/lib/linux \
-o TARGET_APPLICATION x86_64/Objectfiles.o \
/home/myhome/nullmpi-0.7/lib/libnullpmpi.a -llapack -lblas -lm -lcudart
The source of your problem is that you have specified the CUDA runtime library before the object file that contains a dependency to it. The linker simply discards libcudart.so from the linkage because there are no dependencies to it at the point when it is processed. Golden rule in POSIX style compilation statements: linkage statements are parsed left-to-right; so objects containing external dependencies first, libraries satisfying those dependencies afterwards.