I have the following problem.
Background:
I want to connect to a MySQL database and download tables into R.
The database (the MySQL prompt) can only be accessed via SSH tunnel to another server.
I am using Mac OSX El Capitan on a MacBook Pro (2015).
Problem:
From the 'terminal', I can easily SSH to the server, and thereon enter the MySQL prompt and run the queries I need.
I cannot, however, perform the latter tasks from within RStudio.
The code I tried is:
> system('ssh -f <server_user>#<server_ip> -N sleep 20')
> db <- dbConnect(MySQL(), host="hostname", user="username", pwd="password", dbname="databasename", port=3306)
> sql1 <- paste("SELECT * FROM databasename.tablename", sep="")
> results <- dbGetQuery(con, sql1)
> dbDisconnect(con)
While I can see from ps -A | grep ssh that the ssh process is running (for the 20 seconds I want it to), and in R the command executed normally (not waiting for more input), I cannot connect to the relevant database. The error message was something to effect of "could not connect", and RStudio hangs for at least a minute before the error appears.
The questions:
How does one
(1) Use SSH in conjunction with
(2) MySQL on the remote server to
(3) load tables directly into an R data frame?
Thanks.
You should use something which is called port forwarding. Some details are here (https://help.ubuntu.com/community/SSH/OpenSSH/PortForwarding)
For example, say you wanted to connect from your laptop to http://www.ubuntuforums.org using an SSH tunnel. You would use source port number 8080 (the alternate http port), destination port 80 (the http port), and destination server www.ubuntuforums.org. :
ssh -L 8080:www.ubuntuforums.org:80 <host>
Where <host> should be replaced by the name of your laptop.
This is done for whole computer so you dont need to do this from r studio.
Offcourse you need to forward your port to 3036. But you need special privilige on the server. Because on most hosting you can only connect from localhost (for example from php)
Related
How to connect R to mysql? Do we need to install mysql on local host separately? When I try to connect it says
"Error in .local(drv, ...) : Failed to connect to database: Error:
Can't connect to MySQL server on 'localhost' (0)".
What does it mean? I am using R on windows by the way.
You don't show your R code in the post so it is hard to "fix" what might be wrong. Instead I'll just show you some code that works.
To access a mysql database you can use the RMySQL package. For this to work your mysql server must accept remote requests and you must specify the
address (for example the IP address where the server is running)
the username
the password
the database name
Your mysql server should be set up to accept non-local requests unless it's already running locally on the machine where you run R.
library(RMySQL)
mydb = dbConnect(MySQL(), user="username", password="PASSword",
dbname="dbname", host="192.168.13.15")
then you can submit queries in standard SQL language.
indata <- dbGetQuery(mydb, "select * from students")
You do not need a sql client for the R code to work, but if you cannot get it to run within R then you should try to see if you can log onto the server using a manual client. If that is not possible your server is not setup to allow the logins.
I'm a little green at this, and I hope the issue I'm having is a simple one...edit: new information at bottom
I need to make a connection to a remote mysql (Amazon RDS) database.
After following a few tutorials, I have unixodbc and libmyodbc installed and configured on the client, but when I try to connect via isql, I get the error
[08S01][unixODBC][MySQL][ODBC 5.1 Driver]Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)
[ISQL]ERROR: Could not SQLConnect
The most confusing part about this error is that I'm not trying to connect to a local database, but rather to a remote one. I do not have a mysql.sock file on the client...this isn't the issue though is it?
I'm sensing a configuration error but I'm just not sure what it could be.
If I run odbcinst -j then the output is:
DRIVERS............: /etc/unixODBC/odbcinst.ini
SYSTEM DATA SOURCES: /etc/unixODBC/odbc.ini
USER DATA SOURCES..: /root/.odbc.ini
The content of /etc/unixODBC/odbcinst.ini is:
[MySQL]
Description = ODBC for MySQL
Driver = /usr/lib64/libmyodbc5.so
Setup = /usr/lib64/unixODBC/libodbcmyS.so
UsageCount = 5
[MySQL ODBC 515 Driver]
Description = ODBC 5.515 for MySQL
DRIVER = /usr/lib64/libmyodbc5-5.1.5.so
SETUP = /usr/lib64/unixODBC/libodbcmyS.so
UsageCount = 3
Please note that I had to make up this configuration myself, I did a find for libmyodbc* and found these two .so files, thus set up a driver for each of them. A search for libodbcmyS* yields:
/usr/lib64/unixODBC/libodbcmyS.so.1
/usr/lib64/unixODBC/libodbcmyS.so
/usr/lib64/unixODBC/libodbcmyS.so.1.0.0
So, I don't know what else that configuration could be.
The content of /etc/unixODBC/odbc.ini is:
[target_db]
Driver = MySQL
Server = [servername.com]
Port = 3306
Database = [databasename]
Option = 2
User = [username]
Password = [password]
I've tried different options in "Driver", changing it from MySQL, to MySQL ODBC 515 Driver, to the path to the .so file (eg: /usr/lib64/libmyodbc5.so) and all yield the same result.
I'm running:
odbcinst -i -d -f /etc/unixODBC/odbcinst.ini
Followed by:
odbcinst -i -s -l -f /etc/unixODBC/odbc.ini
Followed by:
odbcinst -s -q
Which prints out the name of my connection, ie [target_db]
Then, I try the connect:
isql -v target_db user password
or just
isql -v target_db
and get the error shown above.
Anyone happen to know what I'm doing wrong here? Thanks a bunch-
EDIT:
Wanted to mention that I'm able to connect to the database from this server using the mysql command line tools.
I installed a local mysql database, and I'm able to connect to this using isql. It seems to be ignoring my odbc.ini file entirely, i have to enter a name with the command, ie isql -v test-database, but it still tries to connect to localhost despite my settings.
I feel as though I've tried everything but will keep at it and will post if i find a solution.
You could try to connect using the ip of your server instead of the dns entry on the "Server" line of odbc.ini.. Have you verified the driver is installed with phpinfo()?
Try to set the environment variable ODBCINI with the path of your odbc.ini file.
Keep in mind that the odbc.ini file you point to must be "write-accessible" by the user that is running the program (i.e. the user must have permissions to write in this file).
chmod g+w .odbc.ini did it for me since we run the DB with ORACLE-Start and the crs-User seems to be in charge
//update: when feeding mysql passwords, -ppassword works. -p password does -not- work. Problem solved.//
We have hundreds of databases on our work servers, and I'd like to write a ruby script that automates the process of creating duplicate versions of them on my local machine for development purposes.
I've been trying to use net-ssh to create the tunnel, but (host, user, pass, etc. are censored for obvious reasons):
require 'net/ssh'
HOST = 'xxx'
USER = 'yyy'
PASS = 'ppp'
Net::SSH.start( HOST, USER, :password => PASS ) do|ssh|
puts "inside ssh tunnel"
puts ssh.exec!('ruby -v')
puts ssh.exec!('mysql -u zzz -p pswrd -h c3 will_il_raw -e "select * from deeds limit 1"')
end
results in the output:
inside ssh tunnel
ruby 1.8.7 (2010-08-16 patchlevel 302) [x86_64-linux]
followed by an indefinite hang. Interestingly, if I ssh into the same computer via the console, 'ruby -v' returns 1.9.3. If I enter that mysql query there, it successfully returns one line of the named table.
I assume the solution has something to do with port forwarding, but here my limited knowledge begins to fail me utterly.
The machine running the mysql server is not the same machine as I am accessing it from, which is not, in turn, the machine I am actually sitting at. I need to connect the dots and apparently have no idea how to go about this properly.
Any protips would be much appreciated.
In MySQL syntax, when feeding a password using -p[password], there is no space between -p and the password.
Because the script's version of the syntax had a space, the result of the query was a prompt requesting the password, which caused the hang on the far end of the SSH tunnel.
I have access to a MySQL database through ssh,
Could someone direct me to a MySQL-python code that will let me do this?
I need to save my query results on my local WINDOWS computer,
Thanks,
You can use SSH port forwarding to do this.. in fact first google hit looks to walk you through this exact thing:
http://www.howtogeek.com/howto/ubuntu/access-your-mysql-server-remotely-over-ssh/
And since you're on windows, translate that to using PuTTY:
https://intranet.cs.hku.hk/csintranet/contents/technical/howto/putty-portforward.jsp
You'll then connect to localhost:3306 with your python script, SSH will forward that over to the other machine and you'll end up connecting to the remote mysql instance.
You need to open up an SSH Tunnel to your sql server and then you can run paramiko to connect locally to the port you are using locally. This is done quite easily in *nix systems and I am sure you can download ssh command line too for windows. Try putty or plink, see here. What I do is I run a shell script like so, then I execute my paramiko python script, then I kill the
ssh -N remote_server#54.221.226.240 -i ~/.ssh/my_ssh_key.pem -L 5433:localhost:5432
python paramiko_connect.py
kill pkill -f my_ssh_key.pem # kill using the pattern,
#see ''ps aux | grep my_ssh_key.pem'' to see what it will kill
-N means don't execute any commands, -L is the local port to tunnel from, followed by the remotes server port, assuming you are connected to that server already.
Works like a charm for me for my postgres server & I did try it on mysql too.
I have query regarding connecting mysql to comand prompt.
I did:
open cmd prompt
telnet localhost com 3306
I RECEIVED REPLY as---
some instructions mentioning
telnet [-a][-e escape char][-f log file][-1 user][-t term][host [port]]
-a attempt automatic logon. same as -1 option except uses the currently logged on user's name.
-e Escape char to enter telnet client prompt.
and some more...
but is it right?? or i am lost???
kindly help.
do not telnet to your mysql database.
Instead use the mysql command, a UI like db visualizer (they have a pay version and a free version), or the free ui that comes with maria db (a drop in replacement for mysql).
more more info on the mysql command, try running mysql --help or find it in the mysql reference manual
Edit: more info added here.
Telnet is not a "command prompt", it is a communication protocol (check out telnet protocol on wikipedia) and a program (that uses the communication protocol to communicate). You can not connect to mySql with telnet because mySql does not use the telnet protocol for communication.
I have only accessed mySql for jdbc, so I'm not sure how to solve your problem. I know there is a c api interface for mySql as well. Sections "20 Connectors and APIs," and "15 Replication" in the mysql reference might be helpful.