This problem comes when, I try to import theano with gpu mode. While importing the theano, it tries to compile some code, make a shared library of it and tries to load it.
Here is the command to make the so file.
nvcc -shared -O3 -m64 -Xcompiler -DCUDA_NDARRAY_CUH=mc72d035fdf91890f3b36710688069b2e,\
-DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION,-fPIC,-fvisibility=hidden -Xlinker \
-rpath,/home/jay/.theano/compiledir_Linux-4.8--ARCH-x86_64-with-arch-Arch-Linux--3.6.0-64/cuda_ndarray \
-I/usr/lib/python3.6/site-packages/Theano-0.9.0b1-py3.6.egg/theano/sandbox/cuda \
-I/usr/lib/python3.6/site-packages/numpy-1.13.0.dev0+72839c4-py3.6-linux-x86_64.egg/numpy/core/include \
-I/usr/include/python3.6m -I/usr/lib/python3.6/site-packages/Theano-0.9.0b1-py3.6.egg/theano/gof \
-L/usr/lib -o /home/jay/.theano/compiledir_Linux-4.8--ARCH-x86_64-with-arch-Arch-Linux--3.6.0-64/cuda_ndarray/cuda_ndarray.so \
mod.cu -lcublas -lpython3.6m
It compiles the successfully but can't find the correct library for cudart. The ldconfig seems to know the location of this library, which is correct.
$ ldconfig -p | grep libcuda
libcudart.so.8.0 (libc6,x86-64) => /opt/cuda/lib64/libcudart.so.8.0
libcudart.so (libc6,x86-64) => /opt/cuda/lib64/libcudart.so
libcuda.so.1 (libc6,x86-64) => /usr/lib/libcuda.so.1
libcuda.so (libc6,x86-64) => /usr/lib/libcuda.so
But however when I inspect the library, it states the problem with libcudart.
$ ldd cuda_ndarray.so | grep cuda
libcublas.so.8.0 => /opt/cuda/lib64/libcublas.so.8.0 (0x00007f006dd1b000)
libcudart.so.7.5 => not found
reading elf header
$ readelf -a cuda_ndarray.so | grep NEEDED
0x0000000000000001 (NEEDED) Shared library: [libcublas.so.8.0]
0x0000000000000001 (NEEDED) Shared library: [libpython3.6m.so.1.0]
0x0000000000000001 (NEEDED) Shared library: [libcudart.so.7.5]
0x0000000000000001 (NEEDED) Shared library: [librt.so.1]
0x0000000000000001 (NEEDED) Shared library: [libpthread.so.0]
0x0000000000000001 (NEEDED) Shared library: [libdl.so.2]
0x0000000000000001 (NEEDED) Shared library: [libstdc++.so.6]
0x0000000000000001 (NEEDED) Shared library: [libm.so.6]
0x0000000000000001 (NEEDED) Shared library: [libgcc_s.so.1]
0x0000000000000001 (NEEDED) Shared library: [libc.so.6]
What went wrong so that it is picking the wrong library, cudart7.5 instead of cudart8.0 ?
here is a output of my nvcc -V
$ nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Sun_Sep__4_22:14:01_CDT_2016
Cuda compilation tools, release 8.0, V8.0.44
and my libcudart.so is pointing to correct library version
$ ls -la | grep libcudart
lrwxrwxrwx 1 root root 16 Jan 10 06:10 libcudart.so -> libcudart.so.8.0
lrwxrwxrwx 1 root root 19 Jan 10 06:10 libcudart.so.8.0 -> libcudart.so.8.0.44
-rwxr-xr-x 1 root root 415432 Jan 10 06:10 libcudart.so.8.0.44
-rw-r--r-- 1 root root 775162 Jan 10 06:10 libcudart_static.a
and one more general question how does the linker resolves the actual file location of input like -lm or -lcudart or any shorthand notation used while compiling ?
I tried two of the sample programs shipped with cuda, which includes the libcudart library
$ grep -rnw . -e 'lcudart'
./3_Imaging/cudaDecodeGL/Makefile:329:LIBRARIES += -lcudart -lnvcuvid
./0_Simple/simpleMPI/Makefile:284:LIBRARIES += -L$(CUDA_PATH)/lib$(LIBSIZE) -lcudart
out of these two simpleMPI ran without error.
$ ./simpleMPI
Running on 1 nodes
Average of square roots is: 0.667242
PASSED
The other one failed with earlier error
$ ./cudaDecodeGL
./cudaDecodeGL: error while loading shared libraries: libcudart.so.7.5: cannot open shared object file: No such file or directory
I installed the CUDA8, above the my old installation of CUDA7.5, so it moved old cuda libraries to cuda/lib64/stubs. After removing that directory everything worked as I expected.
Related
The site I administer has some CGI scripts that run scripts of the form:
#!/usr/bin/env bash
perl my-script.pl
my-script.pl uses DBD::mysql.
use DBD::mysql;
My scripts use many CPAN modules and I do not want to pollute the "system" Perl (5.16) installed by the Linux distro. Our security policy requires that httpd run as user "apache" and that apache not have a home directory on our server, so my solution has been to install Perl with perlbrew under a different home dir I have access to. Then the Apache config file for the virtual host sets some env vars to access it.
SetEnv PATH /export/home/user1/perl5/perlbrew/perls/perl-5.30.2/bin:${PATH}
SetEnv PERL5LIB /export/home/user1/perl5/perlbrew/perls/perl-5.30.2/lib # this may not be needed
This works well-enough for loading most modules. For example, apache can run:
perl -mDateTime -e 'print $DateTime::VERSION' # prints "1.52"
but if apache attempts:
perl -mDBD::mysql -e 'print $DBD::mysql::VERSION'
it barfs:
Can't locate loadable object for module DBD::mysql in #INC (#INC contains: /export/home/user1/perl5/perlbrew/perls/perl-5.30.2/lib/site_perl/5.30.2/x86_64-linux /export/home/user1/perl5/perlbrew/perls/perl-5.30.2/lib/site_perl/5.30.2 /export/home/user1/perl5/perlbrew/perls/perl-5.30.2/lib/5.30.2/x86_64-linux /export/home/user1/perl5/perlbrew/perls/perl-5.30.2/lib/5.30.2) at -e line 0.
Compilation failed in require.
BEGIN failed--compilation aborted.
The error message "Can't locate ..." is misleading. I confirmed that DBD::mysql is available from the 3rd path in #INC:
$ find ~user1/perl5/perlbrew/perls/perl-5.30.2/lib/site_perl/5.30.2 -name mysql -ls
16540213 4 drwxr-x--- 2 user1 user1 4096 Apr 21 12:51 /export/home/user1/perl5/perlbrew/perls/perl-5.30.2/lib/site_perl/5.30.2/x86_64-linux/auto/DBD/mysql
16540211 4 drwxr-xr-x 2 user1 user1 4096 Apr 21 11:26 /export/home/user1/perl5/perlbrew/perls/perl-5.30.2/lib/site_perl/5.30.2/x86_64-linux/DBD/mysql
Furthermore, user1 can load DBD::mysql with no problem:
perl -mDBD::mysql -e 'print $DBD::mysql::VERSION' # prints 4.050
Therefore, I suspect that the above error message should have read "Can't load libmysqlclient.so ..."
libmysqlclient.so is located in /usr/lib64/mysql/
ls -l /usr/lib64/mysql/
total 3076
lrwxrwxrwx 1 root root 17 Apr 16 11:59 libmysqlclient_r.so -> libmysqlclient.so
lrwxrwxrwx 1 root root 20 Apr 16 11:59 libmysqlclient.so -> libmysqlclient.so.18
lrwxrwxrwx 1 root root 24 Apr 16 11:57 libmysqlclient.so.18 -> libmysqlclient.so.18.0.0
-rwxr-xr-x 1 root root 3135664 Aug 18 2019 libmysqlclient.so.18.0.0
-rwxr-xr-x 1 root root 6758 Aug 18 2019 mysql_config
drwxr-xr-x. 2 root root 4096 Apr 16 11:57 plugin
If user1 runs perl -V, the Linker and Dynamic Linking sections show the following:
Linker and Libraries:
ld='cc'
ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib /lib/../lib64 /usr/lib/../lib64 /lib /lib64 /usr/lib64 /usr/local/lib64
libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.17.so
so=so
useshrplib=false
libperl=libperl.a
gnulibc_version='2.17'
Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E'
cccdlflags='-fPIC'
lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'
If I run this same perl as apache, it will produce the same result:
sudo -u apache bash
PATH=~user1/perl5/perlbrew/perls/perl-5.30.2/bin:/usr/local/bin:/usr/bin:/usr/X11R6/bin
perl -V
...
Linker and Libraries:
ld='cc'
ldflags =' -fstack-protector-strong -L/usr/local/lib'
libpth=/usr/local/lib /usr/lib /lib/../lib64 /usr/lib/../lib64 /lib /lib64 /usr/lib64 /usr/local/lib64
libs=-lpthread -lnsl -lgdbm -ldb -ldl -lm -lcrypt -lutil -lc -lgdbm_compat
perllibs=-lpthread -lnsl -ldl -lm -lcrypt -lutil -lc
libc=libc-2.17.so
so=so
useshrplib=false
libperl=libperl.a
gnulibc_version='2.17'
Dynamic Linking:
dlsrc=dl_dlopen.xs
dlext=so
d_dlsymun=undef
ccdlflags='-Wl,-E'
cccdlflags='-fPIC'
lddlflags='-shared -O2 -L/usr/local/lib -fstack-protector-strong'
How come user1 perl can load DBD::mysql but apache can't even though both are running the same Perl with the same #INC paths and their dynamic library loading paths look identical? Does anyone know what else can I do to get to the bottom of this?
For starters, you should never do
SetEnv PERL5LIB /export/home/user1/perl5/perlbrew/perls/perl-5.30.2/lib
If you use .../perl-5.30.2/bin/perl, it will know to look in .../perl-5.30.2/lib, and that's the only perl that should look in that directory.
Ideally, you wouldn't do the following either:
SetEnv PATH /export/home/user1/perl5/perlbrew/perls/perl-5.30.2/bin:${PATH}
The shebang of the script should point to the perl it's meant to use (the one with which it was tested and known to work).
In other words, use the following in the bash script:
./my-script.pl
And use the following shebang in my-script.pl:
#!/export/home/user1/perl5/perlbrew/perls/perl-5.30.2/bin/perl
What you are currently doing isn't terrible, but could bite you if you try to upgrade something.
Finally, perl can't find the module because of permission issues. Assuming the apache user isn't a member of the user1 group, you showed that apache user can't access lib/site_perl/5.30.2/x86_64-linux/auto/DBD/mysql (and it might not be able access other pertinent files either).
Fix:
chmod go+X \
/export \
/export/home \
/export/home/user1\
/export/home/user1/perl5 \
/export/home/user1/perl5/perlbrew \
/export/home/user1/perl5/perlbrew/perls
chmod -R go+rX /export/home/user1/perl5/perlbrew/perls/perl-5.30.2
System Configuration: Ubuntu 16.04, Nvidia GTX 1060
Cuda Tool Kit: 9.0
I installed Cuda 9.0 on my system and am able to output nvidia-smi
However, when I am trying to make darknet with GPU I am getting the following error:
nvcc -gencode arch=compute_30,code=sm_30 -gencode
arch=compute_35,code=sm_35 -gencode
arch=compute_50,code=[sm_50,compute_50] -gencode
arch=compute_52,code=[sm_52,compute_52] -gencode
arch=compute_61,code=[sm_61,compute_61] -DGPU
-I/usr/local/cuda/include/ --compiler-options "-Wall -Wfatal-errors -Wno-unused-result -Wno-unknown-pragmas -Ofast -DGPU" -c ./src/convolutional_kernels.cu -o obj/convolutional_kernels.o /bin/sh:
nvcc: command not found make: *** [obj/convolutional_kernels.o] Error
127
I had the same error and found out that the cuda path was not correctly added.
These are some post installation steps necessary after CUDA installation.
You might as well add them to your ~/.bashrc file.
$ export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}
$ export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
Post Installation Steps
its fixed for me thx.
export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
i am using cuda 10.1 version and this also working for that version.
It also worked for me thanks.
export PATH=/usr/local/cuda-11.5/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-11.5/lib${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
I am using cuda 11.5 version so I just changed it.
I have searched many places but ALL I get is HOW to install it, not how to verify that it is installed. I can verify my NVIDIA driver is installed, and that CUDA is installed, but I don't know how to verify CuDNN is installed. Help will be much appreciated, thanks!
PS.
This is for a caffe implementation. Currently everything is working without CuDNN enabled.
The installation of CuDNN is just copying some files. Hence to check if CuDNN is installed (and which version you have), you only need to check those files.
Install CuDNN
Step 1: Register an nvidia developer account and download cudnn here (about 80 MB). You might need nvcc --version to get your cuda version.
Step 2: Check where your cuda installation is. For most people, it will be /usr/local/cuda/. You can check it with which nvcc.
Step 3: Copy the files:
$ cd folder/extracted/contents
$ sudo cp include/cudnn.h /usr/local/cuda/include
$ sudo cp lib64/libcudnn* /usr/local/cuda/lib64
$ sudo chmod a+r /usr/local/cuda/lib64/libcudnn*
Check version
You might have to adjust the path. See step 2 of the installation.
$ cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2
edit: In later versions this might be the following (credits to Aris)
$ cat /usr/local/cuda/include/cudnn_version.h | grep CUDNN_MAJOR -A 2
Notes
When you get an error like
F tensorflow/stream_executor/cuda/cuda_dnn.cc:427] could not set cudnn filter descriptor: CUDNN_STATUS_BAD_PARAM
with TensorFlow, you might consider using CuDNN v4 instead of v5.
Ubuntu users who installed it via apt: https://askubuntu.com/a/767270/10425
My answer shows how to check the version of CuDNN installed, which is usually something that you also want to verify. You first need to find the installed cudnn file and then parse this file. To find the file, you can use:
whereis cudnn.h
CUDNN_H_PATH=$(whereis cudnn.h)
If that doesn't work, see "Redhat distributions" below.
Once you find this location you can then do the following (replacing ${CUDNN_H_PATH} with the path):
cat ${CUDNN_H_PATH} | grep CUDNN_MAJOR -A 2
The result should look something like this:
#define CUDNN_MAJOR 7
#define CUDNN_MINOR 5
#define CUDNN_PATCHLEVEL 0
--
#define CUDNN_VERSION (CUDNN_MAJOR * 1000 + CUDNN_MINOR * 100 + CUDNN_PATCHLEVEL)
Which means the version is 7.5.0.
Ubuntu 18.04 (via sudo apt install nvidia-cuda-toolkit)
This method of installation installs cuda in /usr/include and /usr/lib/cuda/lib64, hence the file you need to look at is in /usr/include/cudnn.h.
CUDNN_H_PATH=/usr/include/cudnn.h
cat ${CUDNN_H_PATH} | grep CUDNN_MAJOR -A 2
Debian and Ubuntu
From CuDNN v5 onwards (at least when you install via sudo dpkg -i <library_name>.deb packages), it looks like you might need to use the following:
cat /usr/include/x86_64-linux-gnu/cudnn_v*.h | grep CUDNN_MAJOR -A 2
For example:
$ cat /usr/include/x86_64-linux-gnu/cudnn_v*.h | grep CUDNN_MAJOR -A 2
#define CUDNN_MAJOR 6
#define CUDNN_MINOR 0
#define CUDNN_PATCHLEVEL 21
--
#define CUDNN_VERSION (CUDNN_MAJOR * 1000 + CUDNN_MINOR * 100 + CUDNN_PATCHLEVEL)
#include "driver_types.h"
indicates that CuDNN version 6.0.21 is installed.
Redhat distributions
On CentOS, I found the location of CUDA with:
$ whereis cuda
cuda: /usr/local/cuda
I then used the procedure about on the cudnn.h file that I found from this location:
$ cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2
To check installation of CUDA, run below command, if it’s installed properly then below command will not throw any error and will print correct version of library.
function lib_installed() { /sbin/ldconfig -N -v $(sed 's/:/ /' <<< $LD_LIBRARY_PATH) 2>/dev/null | grep $1; }
function check() { lib_installed $1 && echo "$1 is installed" || echo "ERROR: $1 is NOT installed"; }
check libcuda
check libcudart
To check installation of CuDNN, run below command, if CuDNN is installed properly then you will not get any error.
function lib_installed() { /sbin/ldconfig -N -v $(sed 's/:/ /' <<< $LD_LIBRARY_PATH) 2>/dev/null | grep $1; }
function check() { lib_installed $1 && echo "$1 is installed" || echo "ERROR: $1 is NOT installed"; }
check libcudnn
OR
you can run below command from any directory
nvcc -V
it should give output something like this
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2016 NVIDIA Corporation
Built on Tue_Jan_10_13:22:03_CST_2017
Cuda compilation tools, release 8.0, V8.0.61
Installing CuDNN just involves placing the files in the CUDA directory. If you have specified the routes and the CuDNN option correctly while installing caffe it will be compiled with CuDNN.
You can check that using cmake. Create a directory caffe/build and run cmake .. from there. If the configuration is correct you will see these lines:
-- Found cuDNN (include: /usr/local/cuda-7.0/include, library: /usr/local/cuda-7.0/lib64/libcudnn.so)
-- NVIDIA CUDA:
-- Target GPU(s) : Auto
-- GPU arch(s) : sm_30
-- cuDNN : Yes
If everything is correct just run the make orders to install caffe from there.
Getting cuDNN Version [Linux]
Use following to find path for cuDNN:
cat $(whereis cudnn.h) | grep CUDNN_MAJOR -A 2
If above doesn't work try this:
cat $(whereis cuda)/include/cudnn.h | grep CUDNN_MAJOR -A 2
Getting cuDNN Version [Windows]
Use following to find path for cuDNN:
C:\>where cudnn*
C:\Program Files\cuDNN6\cuda\bin\cudnn64_6.dll
Then use this to dump version from header file,
type "%PROGRAMFILES%\cuDNN6\cuda\include\cudnn.h" | findstr "CUDNN_MAJOR CUDNN_MINOR CUDNN_PATCHLEVEL"
Getting CUDA Version
This works on Linux as well as Windows:
nvcc --version
When installing on ubuntu via .deb you can use sudo apt search cudnn | grep installed
I have cuDNN 8.0 and none of the suggestions above worked for me. The desired information was in /usr/include/cudnn_version.h, so
cat /usr/include/cudnn_version.h | grep CUDNN_MAJOR -A 2
did the trick.
On Ubuntu 20.04LTS:
cat /usr/local/cuda/include/cudnn_version.h | grep CUDNN_MAJOR
returned the expected results
torch.backends.cudnn.version()
should do the trick
How about checking with python code:
from tensorflow.python.platform import build_info as tf_build_info
print(tf_build_info.cudnn_version_number)
# 7 in v1.10.0
Run ./mnistCUDNN in /usr/src/cudnn_samples_v7/mnistCUDNN
Here is an example:
cudnnGetVersion() : 7005 , CUDNN_VERSION from cudnn.h : 7005 (7.0.5)
Host compiler version : GCC 5.4.0
There are 1 CUDA capable devices on your machine :
device 0 : sms 30 Capabilities 6.1, SmClock 1645.0 Mhz, MemSize (Mb) 24446, MemClock 4513.0 Mhz, Ecc=0, boardGroupID=0
Using device 0
For CUDnn 8.1 and above use the following command:
cat /usr/local/cuda/include/cudnn_version.h | grep CUDNN_MAJOR -A 2
torch.backends.cudnn.m.is_available()
I'm building Octave from sources in order to include the ATLAS libraries. Did I get them included correctly? I don't know what to expect from the Octave configure script. I find "-llapack" suspiciously generic.
./configure --with-lapack=/usr/local/atlas
Source directory: .
Installation prefix: /usr/local
C compiler: gcc -Wall -W -Wshadow -Wformat -Wpointer-arith -Wmissing-prototypes -Wstrict-prototypes -Wwrite-strings -Wcast-align -Wcast-qual -g -O2 -pthread
C++ compiler: g++ -Wall -W -Wshadow -Wold-style-cast -Wformat -Wpointer-arith -Wwrite-strings -Wcast-align -Wcast-qual -g -O2 -pthread
Fortran compiler: gfortran -O
Fortran libraries: -L/usr/lib/gcc/x86_64-linux-gnu/4.8 -L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../x86_64-linux-gnu -L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../../../lib -L/lib/x86_64-linux-gnu -L/lib/../lib -L/usr/lib/x86_64-linux-gnu -L/usr/lib/../lib -L/usr/lib/gcc/x86_64-linux-gnu/4.8/../../.. -lgfortran -lm -lquadmath
Lex libraries:
LIBS: -lutil -lm
...
HDF5 libraries: -lhdf5
Java home: /usr/lib/jvm/java-7-openjdk-amd64
Java JVM path: /usr/lib/jvm/java-7-openjdk-amd64/jre/lib/amd64/server
Java CPPFLAGS: -I/usr/lib/jvm/java-7-openjdk-amd64/include -I/usr/lib/jvm/java-7-openjdk-amd64/include/linux
Java libraries:
LAPACK libraries: -llapack
LLVM CPPFLAGS:
LLVM LDFLAGS:
LLVM libraries:
Magick++ CPPFLAGS: -I/usr/include/GraphicsMagick
Magick++ LDFLAGS:
Magick++ libraries: -lGraphicsMagick++ -lGraphicsMagick
...
allusers#vbubuntu:~/Downloads/octave-3.8.1$ ll -R /usr/local/atlas/
/usr/local/atlas/:
total 16
drwxr-xr-x 4 root root 4096 May 25 23:01 ./
drwxr-xr-x 13 root root 4096 May 25 23:01 ../
drwxr-xr-x 3 root root 4096 May 25 23:01 include/
drwxr-xr-x 2 root root 4096 May 25 23:01 lib/
/usr/local/atlas/include:
total 60
drwxr-xr-x 3 root root 4096 May 25 23:01 ./
drwxr-xr-x 4 root root 4096 May 25 23:01 ../
drwxr-xr-x 2 root root 4096 May 25 23:01 atlas/
-rw-r--r-- 1 root root 33962 May 25 23:06 cblas.h
-rw-r--r-- 1 root root 9708 May 25 23:06 clapack.h
/usr/local/atlas/include/atlas:
total 604
drwxr-xr-x 2 root root 4096 May 25 23:01 ./
drwxr-xr-x 3 root root 4096 May 25 23:01 ../
-rw-r--r-- 1 root root 2089 May 25 23:06 atlas_buildinfo.h
-rw-r--r-- 1 root root 90 May 25 23:06 atlas_cacheedge.h
...
-rw-r--r-- 1 root root 2716 May 25 23:06 zmm.h
-rw-r--r-- 1 root root 552 May 25 23:06 zXover.h
/usr/local/atlas/lib:
total 26548
drwxr-xr-x 2 root root 4096 May 25 23:01 ./
drwxr-xr-x 4 root root 4096 May 25 23:01 ../
-rw-r--r-- 1 root root 14165306 May 25 23:06 libatlas.a
-rw-r--r-- 1 root root 455844 May 25 23:06 libcblas.a
-rw-r--r-- 1 root root 572392 May 25 23:06 libf77blas.a
-rw-r--r-- 1 root root 10942494 May 25 23:06 liblapack.a
-rw-r--r-- 1 root root 456426 May 25 23:06 libptcblas.a
-rw-r--r-- 1 root root 572788 May 25 23:06 libptf77blas.a
allusers#vbubuntu:~/Downloads/octave-3.8.1$
Additional info:
After spamming echo statements in the config file I've noticed the following:
This line:
$as_echo "$as_me:${as_lineno-$LINENO}: checking for $cheev in $LAPACK_LIBS" >&5
has the correct $LAPACK_LIBS variable in it (the one I passed in). It's this line that appears to be the first failure to find something in the lapack libraries I'm telling it about:
if ac_fn_c_try_link "$LINENO"; then :
Just before that line I see the config file define some c code that I believe it's running to identify whether whatever 'cheeve' is, is found in the libraries.
checking for cheev_ in /usr/local/atlas/lib/... no
checking for cheev_... no
checking for cheev_ in -llapack... yes
configuration script
cat confdefs.h - <<_ACEOF >conftest.$ac_ext
/* end confdefs.h. */
/* Override any GCC internal prototype to avoid an error.
Use char because int might match the return type of a GCC
builtin and then its argument prototype would still apply. */
#ifdef __cplusplus
extern "C"
#endif
char $cheev ();
#ifdef F77_DUMMY_MAIN
# ifdef __cplusplus
extern "C"
# endif
int F77_DUMMY_MAIN() { return 1; }
#endif
int
main ()
{
return $cheev ();
;
return 0;
}
_ACEOF
At this point the C code has gone beyond my comprehension level. It seems like it has something to do with whether the F77 compiler (compiler translator??) is being invoked or not.
Well, I think I worked this out after a marathon debugging session.
Octave doesn't appear to recognize the atlas libraries unless they're in shared format (.so files not the .a files that are generated by default).
When I build ATLAS with the --shared option added, and I reference the .so files generated by ATLAS, the Octave config script accepts them. Note: Make sure you use libtatlas.so, not libsatlas.so, assuming you want the multithreaded libraries.
Reference material:
ATLAS ./configure arguments:
../configure --shared -b 64 -D c -DPentiumCPS=3000 --with-netlib-lapack-tarfile=/home/allusers/Downloads/lapack-3.5.0.tgz
Octave ./configure arguments:
./configure --with-lapack=/usr/local/atlas/lib/libtatlas.so --with-blas=/usr/local/atlas/lib/libtatlas.so
Expected Octave ./configure output:
...
BLAS libraries: /usr/local/atlas/lib/libtatlas.so
...
LAPACK libraries: /usr/local/atlas/lib/libtatlas.so
...
Incorrect Octave ./configure output:
...
BLAS libraries: -lblas
...
LAPACK libraries: -llapack
...
My full build process for ATLAS and Octave:
ATLAS setup:
bunzip2 -c atlas3.10.x.tar.bz2 | tar xfm -
mv ATLAS atlas3.10.1
cd atlas3.10.1
mkdir build_vbubuntu
cd build_vbubuntu
sudo apt-get install gfortran f2c libcnf-dev # ???
../configure --shared -b 64 -D c -DPentiumCPS=3000 --with-netlib-lapack-tarfile=/home/allusers/Downloads/lapack-3.5.0.tgz
make build
make check # test serial routines
make ptcheck # check parallel routines
make time
sudo make install
Octave setup:
sudo apt-get build-dep octave
./configure --with-lapack=/usr/local/atlas/lib/libtatlas.so --with-blas=/usr/local/atlas/lib/libtatlas.so
sudo make install
Full disclosure: While I've written up this answer because I got octave to admit that the atlas libraries exist (and I don't want to forget to write it later), the end result is still not working, a large scale matrix multiplication doesn't use multiple cores. Hence, if the cause of that issue is related I may be back to edit this answer in the future.
My successful attempt to compile octave (3.8.2) on CENTOS including atlas:
(make sure to remove blas-devel and lapack-devel, just in case)
> yum install atlas-sse3.x86_64
> setenv LDFLAGS -L/usr/lib64/atlas-sse3
>./configure --with-lapack=-latlas --with-blas=-latlas --enable-jit
> make -j20
(as root)> make install
After configure you should see:
BLAS libraries: -lcblas -lf77blas -latlas
LAPACK libraries: -llapack
I have the cuda 4.0 toolkit installed in my Ubuntu 12.04 server. When I am trying to build a cuda code present in Rodinia benchmark suite. I was getting the following error. i.e, after running the make command
ncclab#slave13:~/Downloads/rodinia_2.4$ make
cd cuda/cfd; make; cp euler3d euler3d_double pre_euler3d pre_euler3d_double /home/ncclab/Downloads/rodinia_2.4/bin/linux/cuda
make[1]: Entering directory `/home/ncclab/Downloads/rodinia_2.4/cuda/cfd'
nvcc -O2 -Xptxas -v --gpu-architecture=compute_20 --gpu-code=compute_20 euler3d.cu -o euler3d -I/if10/kw5na/NVIDIA_GPU_Computing_SDK4/C/common/inc -L/if10/kw5na/NVIDIA_GPU_Computing_SDK4/C/lib
euler3d.cu:35: warning: #warning "the kernels may fail too launch on some systems if the block length is too large"
euler3d.cu:35: warning: #warning "the kernels may fail too launch on some systems if the block length is too large"
By here the euler3d was built
nvcc -Xptxas -v -O3 --gpu-architecture=compute_20 --gpu-code=compute_20 euler3d_double.cu -o euler3d_double -I/if10/kw5na/NVIDIA_GPU_Computing_SDK4/C/common/inc -L/if10/kw5na/NVIDIA_GPU_Computing_SDK4/C/lib
/tmp/tmpxft_0000532d_00000000-12_euler3d_double.o: In function `main':
tmpxft_0000532d_00000000-1_euler3d_double.cudafe1.cpp:(.text+0x22d9): undefined reference to `cutCreateTimer'
tmpxft_0000532d_00000000-1_euler3d_double.cudafe1.cpp:(.text+0x22f7): undefined reference to `cutStartTimer'
tmpxft_0000532d_00000000-1_euler3d_double.cudafe1.cpp:(.text+0x2457): undefined reference to `cutStopTimer'
tmpxft_0000532d_00000000-1_euler3d_double.cudafe1.cpp:(.text+0x246c): undefined reference to `cutGetAverageTimerValue'
collect2: ld returned 1 exit status
make[1]: *** [euler3d_double] Error 1
make[1]: Leaving directory `/home/ncclab/Downloads/rodinia_2.4/cuda/cfd'
cp: cannot stat `euler3d_double': No such file or directory
cp: cannot stat `pre_euler3d': No such file or directory
cp: cannot stat `pre_euler3d_double': No such file or directory
make: *** [CUDA] Error 1
By searching I came to know that there may be a problem with libcutil
But I had the libcutil_x86_64.a in ~/NVIDIA_GPU_Computing_SDK/C/lib
Edit: The contents of various folders are
ncclab#slave13:~/NVIDIA_GPU_Computing_SDK/C/lib$ ls
libcutil_x86_64.a libparamgl_x86_64.a librendercheckgl_x86_64.a
~/Downloads/rodinia_2.4/cuda/cfd$ ls
euler3d euler3d_double.cu Makefile~ pre_euler3d.cu README
euler3d.cu Makefile Makefile_nvidia pre_euler3d_double.cu run
ncclab#slave13:~/NVIDIA_GPU_Computing_SDK/C/common/inc$ ls
bank_checker.h cutil_inline.h GL nvVector.h stopwatch_base.h
cmd_arg_reader.h cutil_inline_runtime.h multithreading.h nvWidgets.h stopwatch_base.inl
cuda_drvapi_dynlink.c cutil_math.h nvGlutWidgets.h paramgl.h stopwatch.h
cutil_gl_error.h dynlink nvGLWidgets.h param.h stopwatch_linux.h
cutil_gl_inline.h dynlink_d3d10.h nvMath.h rendercheck_d3d10.h
cutil.h dynlink_d3d11.h nvMatrix.h rendercheck_d3d11.h
cutil_inline_bankchecker.h error_checker.h nvQuaternion.h rendercheck_d3d9.h
cutil_inline_drvapi.h exception.h nvShaderUtils.h rendercheck_gl.h
the partial contents of the makefile are
all: euler3d euler3d_double pre_euler3d pre_euler3d_double
euler3d: euler3d.cu
nvcc -O2 -Xptxas -v --gpu-architecture=compute_20 --gpu-code=compute_20 euler3d.cu -o euler3d -I$(CUDA_SDK_PATH)/common/inc -L$(CUDA_SDK_PATH)/lib $(CUTIL_LIB)
euler3d_double: euler3d_double.cu
nvcc -Xptxas -v -O3 --gpu-architecture=compute_20 --gpu-code=compute_20 euler3d_double.cu -o euler3d_double -I$(CUDA_SDK_PATH)/common/inc -L$(CUDA_SDK_PATH)/lib $(CUTIL_LIB)
pre_euler3d: pre_euler3d.cu
nvcc -Xptxas -v -O3 --gpu-architecture=compute_20 --gpu-code=compute_20 pre_euler3d.cu -o pre_euler3d -I$(CUDA_SDK_PATH)/common/inc -L$(CUDA_SDK_PATH)/lib $(CUTIL_LIB)
pre_euler3d_double: pre_euler3d_double.cu
nvcc -Xptxas -v -O3 --gpu-architecture=compute_20 --gpu-code=compute_20 pre_euler3d_double.cu -o pre_euler3d_double -I$(CUDA_SDK_PATH)/common/inc -L$(CUDA_SDK_PATH)/lib $(CUTIL_LIB)
The euler3d has been successfully built. The errors are occurring for all the remaining three files.So I don't think there is any problems in the directories they are referring to
I really don't understand why the undefined reference to `cutCreateTimer' and others are occurring. But searching in SE I came to know that when there is any problem with cutil.h or libcutil these problems exits. I was a newbie of linux environment. So please help me
Edit 2: In a include file named make.config the following lines were present
# CUDA SDK installation path
#SDK_DIR = $(HOME)/NVIDIA_GPU_Computing_SDK/C
SDK_DIR =/if10/kw5na/NVIDIA_GPU_Computing_SDK4/C
#SDK_DIR =/if10/kw5na/NVIDIA_CUDA_Computing_SDK4/C
Here by default the 2nd option was selected and the euler3d was built. So I thought there may be a problem because of this. But changing it does not change the problem I was facing
and about cutil library present in make file
CUDA_SDK_PATH := $(SDK_DIR)
# Determine the correct version of the cutil library
CUTIL_LIB = # -lcutil
ifeq ($(shell uname -m), x86_64)
ifeq ($(shell if test -e $(SDK_DIR)/lib/libcutil_x86_64.a; then echo T; else echo F; fi), T)
CUTIL_LIB = #-lcutil_x86_64
endif
endif
Which I think was correct
The problem is that your compile command is specifying the library path, but not the library itself that is providing the cut... functionality.
This is apparently the makefile line that is generating the nvcc compile command for euler3d_double:
euler3d_double: euler3d_double.cu
nvcc -Xptxas -v -O3 --gpu-architecture=compute_20 --gpu-code=compute_20 euler3d_double.cu -o euler3d_double -I$(CUDA_SDK_PATH)/common/inc -L$(CUDA_SDK_PATH)/lib $(CUTIL_LIB)
This is the generated compile command:
nvcc -Xptxas -v -O3 --gpu-architecture=compute_20 --gpu-code=compute_20 euler3d_double.cu -o euler3d_double -I/if10/kw5na/NVIDIA_GPU_Computing_SDK4/C/common/inc -L/if10/kw5na/NVIDIA_GPU_Computing_SDK4/C/lib
You'll note that the last two tokens in the makefile line are:
-L$(CUDA_SDK_PATH)/lib $(CUTIL_LIB)
The -L$... token is getting converted to the correct library path. But the $(CUTIL_LIB) token should be getting converted to something like -lcutil or perhaps -lcutil_x86_64, but that is not happening. As a result, the library that provides the cut...Timer... functions is missing, and so those appear to be undefined.
I would inspect the Makefile carefully to see if there are any end-of-line characters or other oddities that are different between the line that specifies how euler3d_double should be built and the previous lines that specify how euler3d should be built.
You should also be able to confirm this by looking at the nvcc compile command line generated for euler3d (which you don't show) and comparing it to the one for euler3d_double (which is failing).