how to install & configure mysql in hadoop cluster (os centos) to access the created database in one node and accesing in other node.? can anyone post the step by step installation & confiuration procedure?
If you're looking for a distributed SQL type solution, i would suggest you look at Cloudera Impala, Apache Spark SQL, or the Amazon solution Redshift.
MySQL is not designed to be distributed horizontally but rather vertically.
Related
This is something i need to figure out, my company runs a number of prod RDS on AWS. Some of the mysql RDS run with 5.7 , i need to downgrade the mysql to 5.6 or 5.5 . Is this functionality provided by AWS.
Scenario: A mysql server already up and running with mysql version 5.7, Downgrade this to 5.6
-> If this is possible then what are the possible ways ?
-> How to do this ?
This is not something that AWS provides out of the box, however it can be solved with below 2 approaches depending on your database size and downtime that you can accept.
It might worth considering fixing application compatibility instead of downgrading DB which is more risky operation.
1. Dump, restore and switch method
Dump your currently running database with mysqldump utility. Start a new RDS instance with downgraded engine, load your dumped data into it. Switch your application to use RDS instance with downgraded engine.
2. Dump, restore, replicate & switch method
Dump your currently running database with mysqldump utility. Start a new RDS instance with downgraded MySQL engine, load your dumped data into it.
Set the new, downgraded DB instance as read replica of your old DB instance using mysql.rds_set_external_master and then start replication using mysql.rds_start_replication. Stop writes to your original DB, once the read replica catches up (you must monitor replication lag), run mysql.rds_reset_external_master which will promote your downgraded instance and turn off replication. Point your application to the downgraded RDS DB instance.
Method 2 will shorten your downtime to minimum, but is a bit more complex to execute. Here is a command reference to get familiar with to help you succeed: MySQL on Amazon RDS SQL Reference
You will find a great amount of examples in RDS documentation also - Importing and Exporting Data From a MySQL DB Instance:
I have a task to Migrate MySQL DB to Oracle (its my requirement) i tried to Migrate using SQL developer as defined in below link.
https://www.packtpub.com/books/content/migrating-mysql-table-using-oracle-sql-developer-15
As the DB is huge, the constraints are not copied properly from MySQL to Oracle, i need to define/alter/add constraints explicitly, which is time consuming (SQL developer migrates data 300rec/min from mysql to Oracle) & the entire procedure, views, functions is need to re write.
How can i ensure that data has migrated properly or not.?
Is this is a right approach to migrate?
Should i move to any tool which helps to Migrate? If yes please suggest the tool..!!
Or it is the right thing to Move from MySQL to Oracle.
Thanks in Advance.
No specific answer, but some genaral thoughts based on my experiences with migration.
I've found that there normally isn't one tool that does the whole migration job well, and by whole job I mean:
Fast
Handles all data types, scenarios
And that is from Oracle to Oracle!!
Last project we tried Oracle Golden Gate, and found there were issues with that.
We always end up with a hybrid approach, somethings like:
Extract all DDL manually and pre-create objects - there are weaknesses in the stagndard tools that confound them when extracting DDL, e.g. we found 10g expdp did not handle some quirky PLSQL well, so we resorted to extracting this ourselves.
Some tables work well with SQL Loader, others with GG, others (rare) with a custom extract and load process. We had over 3,500 tables and identified about 100 that worked better done as SQLLoader rather than GG. When I say better I mean with data handling and speed of migration. We created different groups of processing each group having a different method.
Once we have an overall hybrid scheme that works, we tune, mainly by splitting that task into parallel processes, both the export and import side.
All my migrations have been big projects where we have shifted from one Oracle system/server to another, always with the target being a newer version of OS and Oracle.
So, I would imagine that migration between non-Oracle and Oracle will through up even more challenges, and probably not as trivial as imply clicking a few buttons in SQL*Developer.
You may find the expected content from the SQL developer documentation at the Oracle website.
There are migration information available for all Microsoft Access users, MySQL users, Microsoft SQL Server and Sybase Adaptive Server users.
You can also download the tutorial in forms of PDF (best for offline viewing and printing), ePub (best for most mobile devices) and Mobi (best for Amazon Kindle devices).
Recently, I have successfully migrated the MySQL database to Oracle database. Below are detail steps:
Operating System: Desktop Ubuntu local and Desktop Ubuntu on amazon aws
Please Note: Here I am using aws desktop ubuntu server because my mysql
database was pretty big. In my case there were 800 tables, 200 views,
procedures, triggers, and functions. The total size of the database was almost
20GB. In case of small database I would recommend to use local ubuntu server.
Tools Used: SQL Developer, VNCServer, Remote Desktop Client, JAVA 8, Third Party MySql JDBC Driver
1. Setup ec2 ubuntu desktop server : https://www.youtube.com/watch?v=ljvgwmJCUjw
2. Install SQL Developer on #1
Download the SQL Developer package from this link :
http://www.oracle.com/technetwork/developer-tools/sql-developer/downloads/index.html
Accept the license agreement and download "Other Platforms" for ubuntu.
Install the SQL developer package as the following.
sudo apt-get install sqldeveloper-package debhelper openjdk-7-jdk
openjdk-7-jre icedtea-7-plugin
Now all that you need to do is to run the command (you might have a
different version)
make-sqldeveloper-package sqldeveloper-4.1.3.20.78-no-jre.zip
This will generate a debian package that you can use to install SQL developer.
Now install the resulting .deb package using the command (Your deb
might have a different version too)
sudo dpkg -i sqldeveloper_4.1.3.20.78+0.2.4-1_all.deb
In my case, I have used java 8.
3. Once you have done with your SQL developer installation on your newly created ec2 instance with VNCServer then all you need to do is to connect to that ec2 instance with the Remote Desktop Client by default available in your ubuntu local machine.
Use IP:1 with user/pass setup for VNCServer in #1
You can see the remote ec2 ubuntu desktop server. You have to grab the keyboard inputs from the Remote Desktop tool if you want to tab inside the remote server.
Once you get connected with the remote client, open SQL Developer from the terminal or from the explorer.
sqldeveloper
Follow the migration steps provided by Oracle corporation:
http://www.oracle.com/technetwork/database/migration/mysql-093223.html
Please Note: While following the migration steps provided by the
oracle they will ask for the destination database connection i.e. oracle
database connection. This is not the database where your MySQL
database will be migrated. Instead, this database connection will
be used for the migration process. Your database connection user
must have user and database create privileges. Once your connection
have user create privilege, then migration process automatically create
the corresponding database user in Oracle database[if you have mysql_test_db in MySQL
database, same mysql_test_db will be created in Oracle db too].
I recently used sqline's tool http://www.sqlines.com/cmd to convert a dump from mysql in the form of an .sql script to an (almost) Oracle-compatible sql script.
sqlines31113\sqlines.exe -s=mysql -t=oracle "-in=$infile"
I just had to (semi-manually) fix some things in the output and then I could run it on my oracle database.
We're running Linux VM's with MySQL on Azure and want to start using Azure SQL, but need to get the data from one into the other, initially.
Is there a way to dump a mysql database and then import that into an Azure sql database?
I'm on a Mac (or can be on Linux), so the .net tools won't help.
I've tried having Azure use the mysql dump. Reads it, but nope.
I've tried selecting the mysql tables from an open connection and drop them on the Azure db, also in an open connection, via Navicat. Nope.
I also tried looking for something in SQLPro for MSSQL. Nope.
Also, I'm willing to edit the mysql dump if there are minor global things to do so that Azure sql will read it.
You can:
1. Install mysql instance on windows based server.
2. Dump all your databases into there using mysql dump.
3. Use all the spectrum of microsoft tools for your goal.
I followed 1 guide and was able to setup, hadoop and hive on my virtual machine ubuntu. Now, i want to get hiveserver2 to start in ubuntu. I am not able to find any guide which will tell me how to get started with hiveserver2.
my plan is to get hiveserver2 then beeline and mysql, later connect to mysql in HDFS to tomcat and use eclipse to develop some DB software.
I am not expecting big answers from anyone, just some references to study from.
Thank You
You could try the documentation for hive provided by Cloudera.
Cloudera Installation Guide
I have a mySQL database on my Windows PC. I'm pretty sure I've found the relevant files, namely the following:
formula.frm
formula.ibd
db.opt
What is the natural way to inspect, edit, and generally play with the contents of these files?
You do not view the binary database files directly. MySQL is a service that you connect to with a client and then perform SQL commands. You will need a client (such as MySQL Workbench) to work with the server.
MySQL Workbench is the GUI tool that allows you to connect to a MySQL database and perform actions on it including querying and creating/modifying the various parts of the database.
MySQL Workbench intro: http://dev.mysql.com/doc/workbench/en/wb-intro.html
Getting started with MySQL: http://dev.mysql.com/doc/refman/5.6/en/tutorial.html
There is also the command-line utility that is included when you install the server. It will be in the BIN folder of the MySQL install directory.
Command-line client info: http://dev.mysql.com/doc/refman/5.1/en/mysql.html
Use a tool like Mysql Workbench to connect to the DB. You do nothing directly to the files. You connect to the service and use the DB.
William, it sounds like your question is "how do I take mysql binary files and turn them into something usable on my machine?". If that's the case, you'll want to first install MySQL on your machine if you haven't already. Then you might have a look here for how to recreate a database from a .ibd file.