I'm trying to use an AWS data pipeline to pull data from an externally hosted MySQL datasource into an RDS MySQL instance.
Is this even possible? How can it be configured? I can't find anything about this in the documentation.
If it's not possible, is there any better option, or am I better to set up a manual push from the external server to the RDS instance?
Have you looked at SqlDataNode?
http://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-sqldatanode.html
This lets you provide a jdbc connection string to any MySql database.
Related
I am very new to DB.
Is it possible to transfer data from LDAP server to mysql?
Also is it possible to transfer data from LDAP server to elasticsearch?
I am using Grafana and trying to integrate it with LDAP data (if possible directly)
Since Grafana can connect with MySQL and Elasticsearch, I want to transfer data from LDAP server (ldif , csv, or any) to MySQL or Elasticsearch.
If anyone had done this before, could you give me some tips on how?
I think you can do that with slapcat: https://serverfault.com/questions/435261/ldap-export-and-import
To import into mysql or similar, you'll have to understand the dump format and put it into mysql.
How is it so difficult to find out how to do this? All I need is to run some simple SQL to restore our existing database on a new RDS database.
Tried connecting with SQL Server Management Studio, it won't connect.
Tried using the Database Migration Tool, it doesn't make any sense.
All the documentation for this seems to assume I'm trying to move some monumental database with zero downtime. Literally all I need to do is run an SQL file on a RDS database. Does anybody on earth understand how this works??
You can use Native backup and Restore. Amazon RDS supports native backup and restore for Microsoft SQL Server databases using full backup files.
Though it wouldn't work on db.t1.micro DB instance class.
You can add Native Backup and restore option by doing following according to the AWS docs.
Create a new option group, or copy or modify an existing option group.
Add the option to the option group.
Associate the option group with the DB instance.
After you add the Native Backup and Restore option, you don't need to restart your DB instance. As soon as the option group is active, you can begin backing up and restoring immediately.
If you cannot connect to the you need to check your RDS security group. Make sure port 3389 is open to your IP,
There is good documentation for connecting to RDS from the Management Studio here:
http://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER_ConnectToMicrosoftSQLServerInstance.html
It states the following:
After Amazon RDS provisions your DB instance, you can use any standard
SQL client application to connect to the DB instance. In this topic
you connect to your DB instance by using either Microsoft SQL Server
Management Studio (SSMS) or SQL Workbench/J.
Also double check that the RDS instance is on a public subnet and that you have allowed public access to it. See those settings in Configure Advanced Settings:
This may sound stupid and it is. But i'm wondering if it is possible to create a GCE instance that its sole purpose is to copy data from another GCE's MySQL database and copies all data to a Google Cloud SQL instance every few minutes and essentially updates the GCloud SQL.
Essentially i'm trying to get around how GAE can't connect to a GCE MySQL database but you can connect to a Google cloud SQL database.
I have tried "FEDERATED Tables" however Google Cloud SQL doesn't support that. So this is my last resort.
Thanks
Why do you need the GCE database at all? That is, why can't you just use a Cloud SQL database for all of your database needs?
You could try manually replaying the binary log to your Cloud SQL instance, ie:
Enable binary logging on your GCE MySQL instance.
Use mysqlbinlog to periodically extract the latest contents of the log as SQL statements. Use the positioning functions to make sure each run starts where the last finished.
Send the SQL outputted by mysqlbinlog to your Cloud SQL instance.
I am working on a symfony website and I want to connect it to a remote RDS on amazon on which i will be running some Select queries to fetch some data every time page refreshes.
Is there an easy way to do so? I was reading something about doctrine but I didn't get it how to get it to use a remote database. Is it possible to do without using doctrine?
Amazon RDS is a meta-layer to manage the database server itself. I assume you want to create a database, e.g. in MySQL, on Amazon RDS and use this database with Symfony and Doctrine.
If this is true, it is very simple: In your app/config/parameters.yml file, you have various database_* parameters. Just set the database_host to the IP address or domain of your DB server as provided by Amazon, set the correct user/password/port, and you're ready to go.
i have 3 servers for three applications on aws ec2 using MySql database,
now each of the application is having amember that is client subscription app,
it connects with sql databse that is created in each instance
so in this way every amamber app is having diffrent database in each server,
now we are working with a device ROKU we need to pass the XML attributes from amember to it
to varify the user so he can watch online streaming tv.
the objective
now i need to make one database that will be connected with each server using amember
so each server access one database .
Options
my options are aws RDS ,dynamoDb
Now can anyone put me in the right direction, for that.
in simple Words
need to connect my multiple apps (same app) with one database
HELLLLP
If you need to connect to a mysql database, DynamoDB is not the answer. It isn't a mysql database.
RDS is a mysql database. It connects like any other mysql database. You haven't mentioned what language[s] you are using, however. Googling "connect to mysql with [language]" should help.
I think it would be best to stick with relational databases such as MySQL.
Amazon RDS is a managed MySQL solution, but you don't have to use it for your needs.
You can use one of your EC2 instances or a new EC2 instance as the central DB and connect all the other servers to it for quires. There are pros and cons for choosing RDS over your own SQL server. If you have any questions there, feel free to edit your question and add them.
EDIT according to comment
In order to connect your application with the local MySQL. Your are probably using a connection string that points to either "localhost" or "127.0.0.1"... That is the IP of your local machine. You will have to change it to the remote IP of the machine where the DB is stored remotely.