Trying to get RmySQL to work but not understanding bash's export or filesystem conventions - mysql

I am trying to install RMySQL on my mac (mavericks) and it errors out when I try to build it from source, saying:
Configuration error: could not find the MySQL installation include
and/or library directories. Manually specify the location of the
MySQL libraries and the header files and re-run R CMD INSTALL.
INSTRUCTIONS:
Define and export the 2 shell variables PKG_CPPFLAGS and PKG_LIBS to include the directory for header files (*.h) and
libraries, for example (using Bourne shell syntax):
export PKG_CPPFLAGS="-I"
export PKG_LIBS="-L -lmysqlclient"
Re-run the R INSTALL command:
R CMD INSTALL RMySQL_.tar.gz
I tried to follow the instructions by entering:
export PKG_CPPFLAGS="-I/usr/local/mysql/include" export
PKG_LIBS="-L/usr/local/mysql/lib -lmysqlclient"
but when I re-run RMySQL it still doesn't work. Moreover, if I type
$PKG_LIBS
to see what that variable holds, I get
-bash: -L/usr/local/mysql/lib: No such file or directory'
I know that /usr/local/mysql/lib exists and it does contain a mySQL header. Am I misunderstanding the instructions?
I'm asking here only after a lot of effort to find solutions and/or work arounds. Sucks being a noob sometimes.

I am going to assume you're trying to get RmySQL to run on R 3.1.0 on Mavericks? Rather than worry about exporting variables etc, here is a simple clean solution for you that should avoid the headaches.
The RMySQL install link Pascal provided above really is your solution. You're probably just stumbling on syntax, or getting things to work from the terminal.
Even if you're a "noob", you should be able to get this working. I'll try to offer a "dummy's guide" walk through here, as I bet there are many others who have this problem too, even after trying to read the RMySQL installation readme.
I would bet with very high confidence the problem is just that you aren't specifying correctly the locations of the library and header folders for compiling. Read the errors carefully when you try to compile... the errors will probably tell you a file/header is missing, or some .so file (shared object) is missing.
One simple way compile RMySQL from source on R 3.1.0, mavericks is as follows (this does not require you to set any environmental variables, no editing of the Renviron file, etc):
Does MySQL work by itself? i.e. Can you open/run it no problems? If not, fix that first.
Find the precise location of your mysql installation. For me, on Mavericks, I see mysql installed at /usr/local/mysql-5.6.17-osx10.7-x86_64 (your version number may be different). There is also another folder /usr/local/mysql which is an alias to /usr/local/mysql-5.6.17-osx10.7-x86_64 (/usr/local/mysql finds the current version of mysql you are using, if multiple mysql file folders exist, I think). In this directory, I see two sub directories (among many) called "include" and "lib". Take a look; "include" will contain header files (include as in #include , etc, in simple C++ programs). The "lib" folder contains compiled source code of the mysql library.
An easy way to compile and install RMySQL which doesn't exactly follow the suggested way to do it in the installation guide is this. Note that this is doing the same thing as in the installation guide, just a little easier as it's one command line from the terminal, once you know where your mysql install folder is. Go to the terminal, and type the following exactly, with one space between each chunk (with your mysql folder name adjusted appropriately for the version number):
PKG_CPPFLAGS="-I/usr/local/mysql/include/" PKG_LIBS="-L/usr/local/mysql/lib/ -lmysqlclient" R CMD INSTALL RMySQL_0.9-3.tar.gz
OR (the same thing, just more typing)
PKG_CPPFLAGS="-I/usr/local/mysql-5.6.17-osx10.7-x86_64/include/" PKG_LIBS="-L/usr/local/mysql-5.6.17-osx10.7-x86_64/lib/ -lmysqlclient" R CMD INSTALL RMySQL_0.9-3.tar.gz
Note for dummies: Make sure when you run this command, that you are doing it from the terminal in the directory that contains the RMySQL_0.9-3.tar.gz file (or whatever the name of your folder is that contains the RMySQL source code)
and RMySQL compiles!
Don't be afraid about trying to compile source code -- it's not just for 'compiled language programmers' or 'computer science graduates'. Most of the time when compiling fails it's just because files are "missing" (there is no corruption on the source code) -- the user hasn't properly specified the locations of the header and libraries (shared objects). Now pull your big boy/girl panties up and just do it .... it's easy.
Notes for people clueless about compiling source code for packages in R:
a) pay special attention to the spacing in the above, otherwise it may not work. Do not have any spaces between the = and the variable/file names (e.g. don't try and have in the above PKG_CPPFLAGS ="-I/usr/local/mysql/include/" as it won't work)
b) When compiling, you want to specify the locations of the header files and the library files and this is what the "-I/ .... " and "-L/ ...." are doing. The -I directory specifies the location of the header files, and the -L the location of the library files. The library files also require the -l[name of library] extension (the -l is short for -lib in the library object names).
c) Note that in the directory /usr/local/mysql-5.6.17-osx10.7-x86_64/lib/ I do not see a file called "lmysqlclient", or even "libmysqlclient", but I do see files named (among others) "libmysqlclient.a" and "libmysqlclient.18.dylib". So don't worry about your MySQL installation not being correct if you don't see a file just called "libmysqlclient" in the lib folder.

Related

MySQL Connector/c (libmysqlclient) missing header file my_config.h file

I am trying to build a MySQL client only in C using Code Composer Studio to run on a micro without an RTOS. Its only function is to push a fixed format record to the server with no other features. I have downloaded the Connector/c source files and added all the "include" files to my build. I am getting 1 build error indicating that the #include "my_config.h" file can't be found. The reason is that it doesn't exist in the distribution from MySQL.com. The documentation seems to indicate that it is part of the server package but I downloaded that also and it is not included. The comments indicate this file defines the platform specific characteristics but gives no clue as to what they might be. Where can I get this file and what information should it contain??

Compiling a program on a server

I'm new to servers and programming in general, and I have a question regarding remote acces to a server, and how much I can actually do on it.
The thin is I have a working program on a linux server, which I acces with my windows machine using mobaxterm. I can acces the server, I see folders and a cmd line, where I can compile a makefile. Everything runs well, however when I run the makefile it just compiles, and doesn't do anything. No error messages, but also no opening of a program. I don't understand anything. Is it a delimitation of the servers structure, that it can only store files on it?
When you compile under linux using a make, it produces an executable but does not run it. Make builds executable objects, but it does not run them. You should include your makefile in the question (reduced to a minimum if it is large). Inside it, you will see that it generates a executable file with a specific name. To run it, you need to invoke this from the command line.
To find out what it is building, a quick way is to type "make clean" (press enter of course) to clean up any built objects. Then type the "ls" command to see what is in your directory.
Next, build the program with the "make" command, then type "ls" to see what has been added. Ignore any new files that end in .o or .a or .so and look for any new files. These are the files built by make and at least one of them is the program you built.
Assuming you found a new file called "myprogram". To run it, type:
./myprogram

Building MySQL from source changes Makefile due to cmake

I am working on MySQL optimization with another researcher and we are using git for version control. The problem is that each of us has to compile those sources on separate machines and running cmake . generates different versions of makefile on our machine. If we think about the following cases
1. A changes source
2. A runs cmake, builds the source, and test performance
3. B pulls the code change
4. B changes source, runs cmake and builds the source
After the step 4, B will have a different version of Makefile and files such as cmake_install.cmake that depend on users and user paths.
For example, some of the files have the following diffs.
# The program to use to edit the cache.
-CMAKE_EDIT_COMMAND = /usr/local/bin/ccmake
+CMAKE_EDIT_COMMAND = /usr/bin/ccmake
# The top-level source directory on which CMake was run.
-CMAKE_SOURCE_DIR = /home/dcslab/userA/mysql/mysql-5.6.21-original
+CMAKE_SOURCE_DIR = /home/dcslab/userB/mysql-5.6.21-original
# The top-level build directory on which CMake was run.
-CMAKE_BINARY_DIR = /home/dcslab/userA/mysql/mysql-5.6.21-original
+CMAKE_BINARY_DIR = /home/dcslab/userB/mysql-5.6.21-original
These are all user-dependent paths generated by cmake commands. The direct way to resolve this conflict is to untrack Makefiles or any file generated by cmake after initially committing them. I am wondering if there is any better and legit way of managing projects using cmake for more than one user. Thanks for your help in advance!
Files generated by CMake are machine-dependent, so they will not work on any machine except one where they has been generated. Because of that, they are useless on for others and there is no needs to track them in git. There are two ways for achive this:
Tune gitignore for ignore files, generated by CMake, on commit. Patterns for such files are relatively simple and can be found by googling. Disadvantage of this approach is that files, generated by project's CMake scripts (configure_file, add_custom_command) will not be automatically ignored and will require explicit notion in gitignore
Perform out-of-source builds, that is not run cmake from source directory. CMake generates additional files only in build directory, correct project's CMake scripts also should follow this rule. So git repo will be clean without any gitignore patterns.
It is common practice to perform out-of-source build in ./build subdirectory of source directory. In this case you can add /build/** to gitignore, and everything will work.
An important part of good engineering -- and especially in research -- is reproducability. It is unfortunate that the code you are working on can be influenced by the environment in which it is built (you may want to look at the Bazel for future projects to reduce external dependencies). Given that this code already has this external dependency problem, you can at least counter the effects by using a consistent environment via virtualization. In particular, you may want to take a look at Docker, which would allow you and your collaborators to build/run code using a common system image, thereby ensuring that all builds and executions are derived from a predictable, consistent environment.

caffe: libglog.so.0 missing (error while loading shared libraries)

I've installed caffe on a server a while ago, and back then it worked properly.
Now I'm following the LeNet MNIST tutorial again (http://caffe.berkeleyvision.org/gathered/examples/mnist.html), and running
./examples/mnist/create_mnist.sh
returns
build/examples/mnist/convert_mnist_data.bin: error while loading shared libraries: libglog.so.0: cannot open shared object file: No such file or directory
I've noticed that liibglog.so.0 is not in /lib which might be the reason for it, but I'm not allowed to copy that file into /lib directory, since I'm not a root user.
Is there workaround for this?
The easiest way to work around the lack of shared libraries in system directories is to use LD_LIBRARY_PATH with the directory where the shared library lives.
Before running the the command that requires a library, run the following in the following the same shell.
export LD_LIBRARY_PATH=~/local/lib
You can also stick this in your .bashrc for convenience.
An alternate solution is to use the following command line flag while compiling, but that requires mucking with other people's build scripts.
-Wl,-rpath,$(DEFAULT_LIB_INSTALL_PATH)

Mercurial: Commit files with more than 255 characters path length (Windows)?

I try to commit a folder hierarchy into out Mercurial repository, which contains files, whiches absolute path length exceeds 255 characters (Windows max. path length).
For these files I receive an error message saying
The system cannot find the path specified
We use TortoiseHG and an Eclipse plugin for Mercurial, both don't work.
Has anybody found a solution for this?
(I do not want to change the repository's location on my HD)
There exists an extension which is aimed at solving this exact problem. It is: https://www.mercurial-scm.org/wiki/Win32LongFileNamesExtension
It uses \\?\ style names to transparently handle long files.
I'm the author, let me know if it works for you.
I've just installed Aaron Cohen' extension, as he suggested.
And it perfectly works with my TortoiseHG 2.6.1 ! Thanks, Aaron!
Though, I'd like to add a detailed guide here, because I cannot find one...
(At least here's what I did on my Win7 x64 - I'm not sure this is the shortest way possible)
1. Download Mercurial-py
Note the Python version required
I've downloaded "Mercurial-2.4.2 (64-bit py2.7)"
2. Download Python
Make sure you're downloading compatible version.
I used "Windows X86-64 MSI Installer (2.7.3)" link
3. Install Python
I've installed it to "D:\Python27"
4. Download pywin32
It's required by the Win32LongFileNamesExtension.
Note the Python version number in pywin32's filename.
I used "pywin32-218.win32-py2.7.exe"
5. Install pywin32
Make sure installer detected correct Python installation
In my case, it's installed in "d:\Python27\Lib\site-packages\pywin32_system32\"
6. Install Mercurial
Make sure installer detected correct Python installation
In my case, it's installed in "d:\Python27\Lib\site-packages\mercurial\"
7. Set PYTHONPATH enviroment variable
setx PYTHONPATH d:\Python27\Lib\site-packages\win32lfn\src;d:\Python27\Lib\site-packages\mercurial\
Use this cli command, or do the same using some other method
Of course, you should adapt paths to your needs
Restart your cli after this, to make sure env. variable is now properly set
8. Download win32lfn
Check for availible downloads on the project's repository page
If there's still nothing, just clone a repository from https://bitbucket.org/remleduff/win32lfn to "d:\Python27\Lib\site-packages\win32lfn\"
Now "win32lfn.pyc" should be in "d:\Python27\Lib\site-packages\win32lfn\src\"
9. Do interanal win32lfn tests
cd /D D:\Python27
python d:\Python27\Lib\site-packages\win32lfn\tests\testwin32lfn.py
10. Create a backup of your repository.
For me, everything went just fune, but you never know....
11. Add win32lfn to hgrc
[extensions]
win32lfn = d:\Python27\Lib\site-packages\win32lfn\src\win32lfn.py
you can find "hgrc" in your ".hg" folder, inside your repository
12. Test it!
The quick and dirty solution is to map a network drive.
For the path c:\some long path\project folder
Map \\localhost\c$\some long path\ to drive Z:\
cd z:\project folder
hg push
We are using this successfully as an interim solution, before migrating to shorter paths.
The mercurial plugins above look good but unfortunately there are numerous non-mercurial bugs related to path greater than 255 characters. For example the VS2010 failure at exactly 259 characters is a real corker!
http://support.microsoft.com/kb/2516078
See https://www.mercurial-scm.org/wiki/Win32LongFileNamesExtension (Aaron pointed to it via the mercurial-devel mailing list).
Another workaround without changing the path to the repo could be to create a second path to it by means of directory junction points. It may work because the reparsing is done at a very low level by the file system driver (or rather some installed filter), so the full (Unicode) path is known by that time and the expansion to beyond 260 characters should work fine. Give it a try. You can use the tool mklink on Windows Vista or 7 and junction.exe from Sysinternals on Windows 2000 or later. For mklink make sure to create a junction point. I'm not sure the reparsing mechanism works the same for directory symlinks (although I faintly remember that it should).
If you don't have a Unicode version of the program available, the limit is 260 characters (including drive letter part). There is nothing to get around it.
However, all ANSI functions are implemented by means of their Unicode counterpart and therefore you may get lucky by providing the full path prepended with \\?\. This may work, but likely won't because the program itself didn't consider anything beyond MAX_PATH (= 260). Ask the author to compile a Unicode version and use the prefix I mentioned. This will fix the issue.
This is a limit of the Win32 subsystem. The absolute path length limit is approximately 32,767 characters. Approximately because the object manager of Windows may expand it (symlinks in the object namespace and the likes).
Windows 10 system running the mercurial 4.4.1 client
Aaron Cohen extension will work I did need make a small one small tweak
based on a comment from mhaecki on this thread: https://bitbucket.org/remleduff/win32lfn/issues/13/not-compatible-with-version-431
in the win32lfn.py file I changed:
from mercurial import util, osutil,cmdutil
from mercurial.i18n import _
to:
from mercurial import util, cmdutil
from mercurial.cext import osutil
from mercurial.i18n import _