We are planning to use Amazon Redshift and planning to use Federated Queries to connect and insert data from our Aurora RDS (MySQL, Provisioned). I followed this documentation to setup the secrets manager store, IAM role for redshift, security group etc.:
https://docs.aws.amazon.com/redshift/latest/dg/getting-started-federated-mysql.html
So now when I try to run the query CREATE EXTERNAL SCHEMA .... FROM MYSQL, I get this error message:
ERROR: CREATE EXTERNAL SCHEMA .. FROM MYSQL is not enabled.
I am running this examples on Redshift Query Editor.
Example query from documentation:
CREATE EXTERNAL SCHEMA <schema_name>
FROM MYSQL
DATABASE '<database_name>'
URI '<endpoint_of_rds>'
IAM_ROLE '<iam_role_attached_to_redshift>'
SECRET_ARN '<secret_arn_contains_rds_credentials>';
I tried to go around Redshift console to check for any setting to enable it but could not find any. Anyone familiar with the setup?
Only question asked so far is this but it is using redshift spectrum: What are the steps to use Redshift Spectrum.?
Please advise. Thanks!
Federated queries are in preview at the momement:
The following is prerelease documentation for the federated query to MySQL feature for Amazon Redshift, which is in preview release.
This means that you have to explicit set your cluster to enable preview features.
You do this by setting maintenance track of your cluster to sql_preview.
It's worth noting that this feature is coming out of preview very soon/already. According to the AWS redshift forums:
https://forums.aws.amazon.com/ann.jspa?annID=8900
Although I should add that I get the same 'not enabled' error and my cluster version is now 1.0.31186, so I'm not sure what's going on with that.
Also this should be a a reply to Marcin's answer but I lack the reputation to use such features.
Related
I want to establish 2 way sync between MS SQL Database with WordPress Mysql Database. I even tried to use Workbench or Microsoft SQL Server Migration Assistant for MySQL. But nothing worked. Every time i get some "Connection to MySQL Failed" Error.
I have even tried all option i have seen online but noting worked.
You have to tell us more about what you are trying to do.
As far as I can tell yo have 2 options:
Use PolyBase: Which is very impractical, close to retirement, lacking in documentation, and personally I wasn't able to install it not even once.
Use Linked Servers: As I posted in the reply you can follow the step by step guide and create a linked server to MySQL. From there you can query the linked server and eventually create a stored procedure that import/export data but forget about an on-time sync
Azure Data Factory: If you are on Azure SQL Database you can setup Azure Data Factory and create a pipeline for sync data. Or eventually use SSIS if you are on prem.
sorry if title is not so clear, probably I am not finding what I need due I do not know how to search
I have few MySQL servers is separated online servers (from different wordpress) and I want to load some of the data on those databases/tablets into a SQL database located on Azure.
inside Azure portal itself I do not see where to establish external connections, neither at server level, neither at database level
I download and install Microsoft SQL Server Management Studio, connect to the server, I can see my databse and the master one, Security with logins, and Integration Service Catalog, nothing else.
I was looking for something like:
https://www.jetbrains.com/help/go/db-tutorial-connecting-to-ms-sql-server.html#step-3-connect-to-microsoft-sql-server-with-datagrip
but nowhere ...
maybe something like this:
https://www.devart.com/odbc/mysql/docs/microsoft_sql_server_manager_s.htm
but no Servers objects option available on my SSMS
Can be this done?
Note: Azure database is a basic wfor now, if that is a limitation
Some choices.
In your SQL Server Management Studio create a linked server pointing to each MySQL instance. You found the instructions for that. https://www.devart.com/odbc/mysql/docs/microsoft_sql_server_manager_s.htm But it probably will not work in Azure SQL Server; you don't have access to the underlying Windows OS to install stuff like MySQL ODBC drivers, which you need. (You could ask Azure techsupport if they can help.)
In each MySQL instance, try creating a federated table connection to appropriate table in SQL Server. That cross-vendor federation stuff only works in MariaDB, however; MySQL's federation only goes MySQL <--> MySQL.
Write yourself a purpose-built extract / transform / load (ETL) program, and arrange to run it every so often. Program it to connect to all the servers involved, retrieve the data needing to be transferred from your MySQL servers, and update / insert that data on the SQL server.
(edit) You may be able to use command-line SQL client programs. mysqldump, with its --compatible option, may generate usable INSERT statements in a file. You then may be able to use sqlcmd to run those INSERTs on your Azure server. It's going to take some hacking, and may take using sed(1) or awk(1) to make the MySQL output compatible with SQL Server.
I believe the third option is the most robust one for production use.
Is it possible to create a MySQL Schema from a running AWS-based RDS database? I am a newbie and not sure whether there is a tool like what is available in MySQL workbench type dashboard to create schema at ease?
I am also new to this but I recently created a MYSQL database on AWS RDS.
(I am new to MySQL and AWS).
The answer is yes, we can create schemas. However, as much as I understand, there is no concept of schema in MySQL. There is one database where you create the different database objects and then we just query those using one user or the other.
Edit:: Database and Schema are same thing as per the comments. So yes, we can create schemas and then users to access them.
Anyways, you can use the MySQL workbench to connect your AWS RDS database.
Or you can do the same using any coding language. For both the above options, make sure your database is created with 'Publicly accessible' option enabled.
Refer to this - https://aws.amazon.com/getting-started/hands-on/create-mysql-db/
If you follow all steps, your database on AWS should work like one installed locally on your machine.
I had created a MySQL DB instance on RDS, following are the details:
Instance class - db.t2.micro
Engine version - 8.0.17
It is present in us-east-1 region.
Now I have to move my MySQL DB to Amazon Aurora, following were the documentation guides which i followed:
1- https://docs.amazonaws.cn/en_us/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Migrating.RDSMySQL.Replica.html
On following the above guide i am only able to create a read replica. The option for creating a aurora replica is not visible. Also, when i create read replica it is created for mysql db and not gives me option to create for any other engine type.
2- https://docs.amazonaws.cn/en_us/AmazonRDS/latest/AuroraUserGuide/AuroraMySQL.Migrating.RDSMySQL.Import.html
On following the above guide, i am able to create snapshot for my mysql db instance. But, the option to 'Migrate Snapshot' is disabled. Could anyone please help me with this.
Could anyone also suggest me any other way in which i can migrate my MySQL db on RDS to Amazon Aurora?
Aurora MySQL is only supported for MySQL 5.6 and 5.7 unfortunately so the read replica solution would not work.
However, Amazon have you covered with the Database Migration Service.
AWS published a blog post on this, so take a read.
We're running Linux VM's with MySQL on Azure and want to start using Azure SQL, but need to get the data from one into the other, initially.
Is there a way to dump a mysql database and then import that into an Azure sql database?
I'm on a Mac (or can be on Linux), so the .net tools won't help.
I've tried having Azure use the mysql dump. Reads it, but nope.
I've tried selecting the mysql tables from an open connection and drop them on the Azure db, also in an open connection, via Navicat. Nope.
I also tried looking for something in SQLPro for MSSQL. Nope.
Also, I'm willing to edit the mysql dump if there are minor global things to do so that Azure sql will read it.
You can:
1. Install mysql instance on windows based server.
2. Dump all your databases into there using mysql dump.
3. Use all the spectrum of microsoft tools for your goal.