How to sync ShinyApps with MySQL From Localhost to Public - mysql

I am currently developing an app using R Shiny and finishing my ShinyApps, now I am trying to deploy the apps to Shinyapps.io so multiple user can reach and use it, but I have an issue for the deployment.
my Apps is about a Pharmacy management, it controls a CRUD operation, so clearly it is binded with db Connection using these options configuration (running this in locally)
options(mysql = list(
"host" = "127.0.0.1",
"port" = 3306,
"user" = "root",
"password" = ""
))
one more thing, to connect to database, I usually started my XAMPP apps and switched on mysql admin so my apps can connect the database locally. it worked flawlessly and clear before deploy
but it crashed instantly when I try to run it in the shiny.io after delpoy (what I mean is disconnected automatically). so I did try to change the host ip to publically like this , (I am trying to get ip address on user local machine)
configA <- system("ipconfig", intern=TRUE)
configB <- configA[grep("IPv4", configA)]
configC <- gsub(".*? ([[:digit:]])", "\1", configB)
options(mysql = list(
"host" = configC,
"port" = 3306,
"user" = "root",
"password" = ""
))
the ConfigC variable stores IPv4 address to get the public IP on local machine, but still these doesn't work, I attached a log in below link
how can I connect and sync my apps with MySQL in Shinyapps.io ? I use DBI and RMySQL package..do I need to host MySQL first so i can sync my apps? can anyone brief me with step by step explanation how to? thankyou in advance
here is my error log from shinyapps.io
http://textuploader.com/dulzh

For people who have same problem & didn't know how, i'll share what have work for me:
1) I Recommend Host your MySQL Database to AWS (Amazon Web Service), it is free and great performance to sync any Web Service update online especially shinyapps.io, with creating an Account first
2) Validate your AWS Account with full information including credit card, so you can access its Services
3) Click Service > Database > RDS
4) Then you will be redirected to AWS RDS Dasboard, here you can manage Instances of your MySQL Database, to create one new Instance, click Launch DB Instance
5) Here is my Instance settings:
Engine Options: MySQL
Use Case: Production- MySQL
DB Instance Class: db.r4.large 2vCPU, 15.25 GiB RAM (i believe this setting is subjectively based on our CPU performance)
Multi AZ Deployment: No
Storage Type: Provisioned IOPS
Allocated Storage: 100 GiB
Provisioned IOPS: Depends on your Allocated Storage (I use 4000)
6) Then in Settings tab, fill your db instance identifier, master username & password, after that when you click Next, there is advanced configuration, fill again db name and then you will want to check all Log Reports in hope an easier maintenance later, after finished > Launch DB Instance
7) Wait until your instances status become Available (keep refreshing to know)
8) After the Instance become Available, check the Instances and scroll down until you found Connect section, remember and save the Endpoint, Security Group Rules, master username & password Instances from Detail section
9) In your server.R, edit your MySQL connection options, from localhost to AWS RDS..
options(mysql = list(
"host" = "your Endpoint",
"port" = 3306,
"user" = "your master username of db instance",
"password" = "your master password of db instance"
))
10) Before deploying your MySQL database from localhost to AWS RDS, firstly Go to your AWS > Services > VPC > Security Group > (Click one of Group Name that is actively used by your Instances)> Inbound Rules
11) In Inbound Rules you must whitelist all External IP that you or other PC access to your shinyapps http://whatsmyip.org, and whitelist all shinyapps IP address based on this http://docs.rstudio.com/shinyapps.io/applications.html#accessing-databases-with-odbc in section 3.8.4
12) And now lastly, to deploy your MySQL Database from localhost to AWS RDS cannot be done directly, I Recommend installing MySQL Workbench to do it, after done installing, launch MySQL WorkBench
13) Create new MySQL Connection adn then fill the connection form:
Connection Name: (anything you like)
Connection method: TCP/IP
Hostname : (paste your Endpoint)
Port 3306
Username: (your master username of db instance)
Password: (your master password of db instance)
14) After a successful Connection to AWS RDS, open your connection, and then MySQL WorkBench UI will open, Import your .bak files (MySQL database) from Navigator > Management > Data Import > Select Import from Self Contained Files > browse your file> and then start Import
15) You have successfully deployed your database to AWS RDS! you can use query in WorkBench to see all your table/database information
16) RunApp your ShinyApps and test it, and DONE!
(if you EVER found message can't connect to your AWS RDS host, probably that your External IP is changed to new one, and to solve it you need to whitelist again your IP to AWS VPC in step 10)
I hope these are helpful for you!

Related

Why can't I connect to AWS RDS?

I'm trying to connect to my new AWS RDS I just made.
I followed the "Setting up for RDS" (https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_SettingUp.html), then the "Tutorial: Create an Amazon VPC for Use with a DB Instance" (https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_Tutorials.WebServerDB.CreateVPC.html), then the "Creating a MySQL DB Instance and Connecting to a Database on a MySQL DB Instance" (https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/CHAP_GettingStarted.CreatingConnecting.MySQL.html) but I'm not able to connect to my DB from my computer or my dedicated server on the web.
Following the previous docs, I have this config :
My DB instance
The VPC
The subnetworks
Example of subnetwork's details :
The first security group :
The second security group, calling the first one :
For the first security group, I put both my private IP and the IP of my dedicated server, and their ports.
I even tried to put 0.0.0.0/0 for SSH and TCP, it didn't work either.
For the DB instance, I tried to add the two security group instead of only the db-securitygroup, it didn't work.
I tried to use a different Port for the DB instance, it didn't work.
With MySQL Workbench or with PDO on my dedicated server, I'm unable to connect to the DB : "SQLSTATE[HY000] [2003] Can't connect to MySQL server on [...]"
I think your security groups are incorrect. If the RDS instance is the only thing you currently have running in the VPC, then you should only have one security group, which is assigned to the RDS server, and that security group should have a rule for port 3306 that allows ingress from your personal IP address, and your dedicated server's IP address.
Take a look to this instruction, pay attention to step 3, 4 and 5. It is for ElasticSearch but I think in your case steps are similar

Why don't I have access to the database from aws lambda but have from a local computer with the same login data?

I created a new Mysql database in Amazon RDS, and on my local computer I get access through the console
mysql -u username -p -h test.c2nfdg67dbdpb.us-east-1.rds.amazonaws.com
Now I try to connect via Aws Lamda using python and pymysql module
rds_host = "test.c2nfdg67dbdpb.us-east-1.rds.amazonaws.com"
name = 'username'
password = 'pass'
db_name = 'dbtest'
conn = pymysql.connect(rds_host, user=name, passwd=password, db=db_name, connect_timeout=5)
and got an error ERROR] 2019-11-20T14:30:05.261Z 7b1ed2e0-03ba-4c62-9525-22d84582f (1049, "Unknown database 'dbtest'")
It seems strange that I don't use the database name at all through the local console and I get access.
p.s.I run show databases; on local console and cant see dbtest in databaselist. I was confused because when I created the new database, I entered a name, but only default databases are listed. What am I connected to then?
p.s.s
hm why DB name is undefined ?
Your code explicitly tries to connect a specific database named dbtest. It will either connect to this database or fail, contrary to your mysql client which can connect without a database selection. Clearly, the database is missing, since you verified that show databases results does not include dbtest.
Then only possible explanation is that the database was never created.
The RDS console provides the ability to create a database with default settings. That's the way I used it. But no physical database is created, only database instances.
I can't found a way to add a new database to the created instance. Now, after a few hours of troubleshooting, I recreated the new database without using the default settings and there was a point - database name entry. I added this name and everything worked out.
But here's what I don't understand is why AWS added a quick creation way, which in the end doesn't actually create a database that you can join.
If you have created the database in Amazon RDS and are able to connect to it from your local computer, it may be a connection/permission issue with your lambda. Your Lambda function may be inside a VPC subnet with no internet access. Either pull it out of the VPC or grant internet access to the Lambda function like this:
https://aws.amazon.com/premiumsupport/knowledge-center/internet-access-lambda-function/
However, if your Lambda function will be communicating with RDS instances, you should consider doing this via private channels for security purposes.
Update
A third possibility is that the database in question doesn't exist even though the asker implies that there was a successful connection to the database from his/her local computer.

failure to connect to Google SQL First gen (and Second gen?)

I am receiving an error when trying to load up my webpage
Failed to connect to MySQL: (2005) Unknown MySQL server host ':/cloudsql/testsite:europe-west1:testdatabase' (2)Error:
I have a Google Compute Engine VM set up with a LAMP stack (Apache/2.4.10 (Debian)/ Database client version: libmysql - 5.5.55 / PHP extension: mysqli)
I also have set up an instance on Google SQL with user credentials for aforementioned VM (i have set up both First Gen and Second Gen)
I can access both a local MySQL database on the VM as well as the Google SQL databases via phpAdmin installed locally
HOWEVER i appear to have an issue with the DB_HOST credentials in my config.php file when i run the script
path = /var/www/html/includes/config.php
I get
usually for local MYSQL databases i use
// The MySQL credentials
$CONF['host'] = 'localhost';
$CONF['user'] = 'YOURDBUSER';
$CONF['pass'] = 'YOURDBPASS';
$CONF['name'] = 'YOURDBNAME';
Documentation (and github links) recommend path
:/cloudsql/project-id:region:sql-db-instance-name
which is what i have done (see above) - but i keep getting the error message.
Am i typing the host description incorrectly? Or have i missed a configuration step?
Thanks in advance
It seems as if i have erred and that the credentials format i stated earlier are for Google App Engine
If you are on Google Compute Engine, you have two options:
Connect to the public IP address of your Cloud SQL instance. This requires you whitelist your GCE instance on the ACL for the Cloud SQL instance.
Use the Cloud SQL proxy. This is a extra daemon you run on your GCE instance that allows you to connect via TCP on localhost or a socket.

Unable to connect to Mysql from Django app - AWS, EC2, RDS

I am having trouble connecting to my RDS MySQL Database.
My current Set up:
Django 1.9+
AWS - Launched an EC2 Instance (Amazon Linux). Launched an RDS instance of MySQL (5.6.7). I created a Virtual PC (VPC) for the EC2 instance and RDS instance, and in that group created several security rules so the EC2 instance could connect to the MySQL RDS. Below are the security rules I use and associate with the VPC.
Inbound Rules: (Type, Protocol, Port Range, Source)
1. Http , TCP, 80 , 0.0.0.0/0
2. SSH , TCP, 22 , <homePC address>/32
3. HTTPS, TCP, 443, 0.0.0.0/0
4. MYSQL/Aurora, TCP, 3306, source (my created security group)
Nginx - Serving my content redirecting from port 80 to port 443. I created SSL certificates in order to make my site secure. I correctly served my Django application, and the static files. Nginx serves everything great!
Previously I was able to connect to my RDS instance. I did not set up an elastic IP at that point in time, as it took some time to transfer my domain over to Route53 (amazons DNS). I successfully was able to transfer it, and associate it with my EC2 instance and even start my application
The problems -
First Problem I cannot connect to the RDS from my EC2 instance. I have checked my django settings file and the secret file (the one that feeds my settings file with sensitive information which I keep out of version control). Here are examples of my settings for the mysql connection - I changed them obviously
"DATABASE_NAME":"myDatabase",
"DATABASE_USERNAME": "myUsername",
"DATABASE_PASSWORD": "myPassword123",
I double checked the username and password in the RDS console, and even reset the password.
After I went ahead and activated my virtual environment and attempted to connect.
python manage.py dbshell
My Error was
ERROR 1045 (28000): Access denied for user 'myUsername'#'<private IP address from Elastic IP>' (using password: YES)
Notive the user name is not what I utilized in the settings. It changed. I feel this is the root cause of the problem, but am unsure why Django would change my username in such a manner.
Second Problem - I also try to connect to the database directly using mysql. I double check my endpoint, username and password are correct, but also run into trouble connecting.
Note - As I said, I was able to connect in both manners listed above before acquiring my IP address. Thus my troubles are configuring security rules to allow a permanent IP address, and being unable to use TCP connect to use the Mysql client to log in.
Thanks in advance.

Connect to MySQL on AWS from local machine

I am trying to set up a dev environment on my local machine that accesses a MySQL DB on AWS, but I keep getting a "Can't connect" message.
mysql_connect('xxx.xxx.xxx.xxx:3306', 'USERNAME', 'PASSWORD');
I also commented out the bind-address in the my.cnf file, and granted permissions to the IP address that is connecting.
Anyone ever successfully get this working?
My experience in Aug-2013 was as follows for an RDS instance created through Elastic Beanstalk.
0) Assuming the RDS instance has already been created
1) Log in to the management console: https://console.aws.amazon.com/console/home
2) Select Services->VPC
3) Select Security Groups (on the left hand side)
4) Select the group whose description says "Security Group for RDS DB..."
5) In the Security Group Selected panel at the bottom of the page, choose "Inbound"
6) Select MySQL as the rule.
7) Type the ip address of my local machine e.g. 145.23.32.15/32
8) Click Add Rule and Apply Rule Changes
After doing this I could connect to the database using mysql from my local machine.
a) From management console select Services->RDS
b) Click on DB Instances (I have only one) and select "Go to Details Page" for the required instance
c) Obtain Host and Port from the endpoint
d) From a terminal session do soemthing like: mysql --host blah.blah.blah.us-west-2.rds.amazonaws.com --port 3306 -u my-user-name -p
If you are using MySql on AWS via an RDS instance you must add the IP address you want to connect from to the "DB Security Groups". To do this go to your AWS Managment Console and select RDS.
1. Select "DB Security Groups" on the left panel
2. Select "default"
3. Select "CIDR/IP" from the select box and enter your workstations public IP address. Example:
23.234.192.123/32 (dont forget the /32 for a single ip)
4. Click "Add"
5. Wait a few minutes for it to go into effect and then connect your MySql client.
This only applies for RDS instances, if you are using MySql installed on an EC2 instance then the instructions are the same as accessing MySql from any remote machine.
I suppose this is firewalled by Amazon, try using a SSH tunnel:
http://blogs.oracle.com/divyen/entry/connecting_mysql_server_on_amazon
Note: Do not open MySQL to the public internet, not even when using IP filtering. SSH tunnels are way more secure. Best part of it: The tunnel could be accessible with localhost:3306 on your machine, no need to change the config : )
I am on a Windows 7 machine, and had to make the following 3 changes to be able to connect to AWS RDB.
VPC Security Group update in AWS Console (similar to what mikemay has above)
From https://console.aws.amazon.com, click on Services (top left) and choose VPC.
Next select Security Groups
Click on the Security Group which has the description "Security Group for RDS DB..."
On the "Inbound" tab, choose "MYSQL" in the Create a New Rule dropdown.
Add your IP address in CIDR format and click on Add Rule.
Click on Apply Rule Changes.
my.cnf update in local MySQL configuration
Change "bind-address = 127.0.0.1" to "bind-address = 0.0.0.0"
Comment out "skip-networking"
Turn OFF Windows Firewall
Go to Control Panel/System and Security/Windows Firewall and turn OFF Windows Firewall.
After these changes, I am able to connect through both
MySQL WorkBench using Database->Connect to Database
Command Prompt with
mysql.exe -h <AWS DB Endpoint> -U <UserName> -P <Port Number, likely 3306> -p
I have been using MySQL Workbench http://www.mysql.com/products/workbench/ with RDS and it works great. Very easy to create and save a new database service instance. Click "New Server Instance" under "Server Administration" and follow the prompts. You will need to enter the information provided in the AWS RDS webpage for that instance (for example, it's endpoint).
NOTE: In order for you to actually connect, you MUST add your IP address in the "DB Security Groups." The link is in the left-hand column, which is titled "Navigation." I use the "CIDR/IP" option (the other is EC2 Security Group). Make sure to include a "/##" after the IP, such as the "/32" they use in the example (you will see it on the page). In a few seconds, the IP address should be authorized.
After that, go back to MySQL Workbench and complete the New Server Instance creation process.
To use the connection, your code might look something like this (that excerpts of my Java code):
String url = "jdbc:mysql://yourdatabasename.foo.us-east-1.rds.amazonaws.com:3306/";
String userName = "your_user_name";
String password = "your_password";
String dbName = "your_db_name";
String driver = "com.mysql.jdbc.Driver";
Connection connection = DriverManager.getConnection(url + dbName, userName, password);
When we create RDS need to configured the firewall to accept mySQL connections from local or other instance, as such the packet is being dropped at the firewall level, to resolve this you need to:
Login into your AWS console Go to RDS
Note down the security group of your mySQL server (in my case awseb-e)
click security groups
click your group in the center menu awseb
click inbound tab
Select mySQL from the list, add the details of your client server and save the rule
NOTE : once you choose my IP ,AWS select your IP , if you need other ip use
https://www.whatismyip.com/my-ip-information/
and add your IP4 IP