I followed 1 guide and was able to setup, hadoop and hive on my virtual machine ubuntu. Now, i want to get hiveserver2 to start in ubuntu. I am not able to find any guide which will tell me how to get started with hiveserver2.
my plan is to get hiveserver2 then beeline and mysql, later connect to mysql in HDFS to tomcat and use eclipse to develop some DB software.
I am not expecting big answers from anyone, just some references to study from.
Thank You
You could try the documentation for hive provided by Cloudera.
Cloudera Installation Guide
Related
I'm very new to MySQL.
My question is about installation of MySQL on my Mac (Mojave OS) to learn and practice it to later use it with node.js.
First I got it with Homebrew, but then I wasn't able to use it with the Workbench - I set up the password for root user but I didn't find any commands list to work with it since it refused to do any operations with it in workbench and I didn't find any command.
Then I downloaded Community Server - I have a server through system prefs, but I don't have a command line interface - which I need for writing queries.
After that I downloaded MySQL Shell. But as far as I researched I found that is it not standard CLI (found it here https://stackoverflow.com/questions/58367813/how-to-uninstall-mysql-shell-on-macos).
My question is next.
As an example, when I started learning MongoDB I had MongoDB Compass and MongoDB Shell - one helped me to work with commands, the other one helped to see results visually.
MySQL has a language which I want to learn and practice with (SELECT, USE etc.). How can I do the same what I did with MongoDB but in the case of MySQL? What should I install, and why Homebrew MySQL version is different than any other MySQL version (even commands - such as mysql.server start) and there is no documentation for it in MySQL docs?
Unfortunately, there's no such nice installer for macOS as there is for Windows. Instead you have to install the different parts individually.
For the type of work you plan to do I recommend to install at least 2 pieces:
The MySQL server, which holds your data. On macOS this comes with a preferences pane, which you can use do some basic management of the server (start/stop, directory settings).
MySQL Workbench, a visual design, query and management tool. That will give you GUI support for your DB work.
The server also comes with the classic shell, which you can launch by running mysql in a terminal. That's sometimes quicker for simple work, compared to launching a full GUI.
A better command line tool is the MySQL Shell, a new and highly improved variant of the classic shell, which will eventually replace the old classic shell entirely in the future. So it's certainly no mistake to install that too.
I hope that clarifies which pieces are required to start working with MySQL.
Cloudera claim to have a Quick Start approach. That is not working for me I note.
When I invoke spark-shell I get:
... WARN metastore.ObjectStore: Version information not found in metastore. hive.metastore.schema.verification is not enabled so recording the schema version
I find it confusing, this is after all a Quick Start and this looks odd.
So:
I see that there is mysql running with metastore db. I can access this fine.
Do I need to start hive metastore if using mysql as hive metastore? I think so, but ...
Do I need hive server 2 now to run locally? Or can I run without?
The Cloudera Manager on the Hive Tab tells me I am using mysql and I see an auto generated hive-site.xml.
In short I am not sure how nto proceed to fix this. One of the logs is talking about failure to create derby e.g. ...
Caused by: java.sql.SQLException: Failed to create database 'metastore_db', see the next exception for details.
In short I am seeking guidance on how to fix this.
Before one of the numerous crashes I have had, I had an sbt assembly of SPARK / SCALA working fine accessing a remote MYSQL db, so I am wondering if that is the way to go and that the spark-shell and the local Cloudera VM are all to unstable.
Seeking guidance amidst frustration. Data Bricks works like a dream.
Thanks in advance.
Install 5.13, other problems but these ones disappear. Noted however what the cause is.
When a clean install is done and
sudo jps
executed, then all Hadoop services are fine and working. Checked this.
What is then noted is that the Cloudera Manager Console (CMS) never shows. Advice on Internet is to execute the command to invoke CM Express.
Once you do that, then the CMS shows, but many Hadoop Services need to be (re-)started. Point then is that spark-shell goes haywire and the metastore no longer accessible. All in all a sorry mess for which the solution is not so obvious.
Manual install of Hadoop may well be the best option, but a definitive integrated spec is needed. Then also have issues with Spark 2.x not being supported and KUDU not there, parcel vs. packages.
I am wondering if there is anyone try to use mysql workbench to connect to openedx fullstack version. I have a devstack version which is running on my local machine and doesn't have any issues. Also, I am able to connect to mysql database by using SSH on mysql workbench(devstack).
Please see the official document here: https://github.com/edx/edx-platform/wiki/Developing-on-the-edX-Developer-Stack#the-edx-platform-database-that-devstack-uses
But I am not able to find any document regrading the process of fullstack mysql connection configuration.
Any ideas or suggestions? Appreciate your help!
how to install & configure mysql in hadoop cluster (os centos) to access the created database in one node and accesing in other node.? can anyone post the step by step installation & confiuration procedure?
If you're looking for a distributed SQL type solution, i would suggest you look at Cloudera Impala, Apache Spark SQL, or the Amazon solution Redshift.
MySQL is not designed to be distributed horizontally but rather vertically.
I know the question I am about to ask is a very stupid once but I am new to MYSQL so please bear with me. Want I want to do is create a database on my local machine and also access/manipulate it from their. I thought MYSQL WB was the all in one tool to do this i.e it will also allow me to create a database and also make a MYSQL server from which to access it. But I came to know the Workbench is just a front end and you need to have a some kind of a seperate MYSQL server module for it to access.
The problem is I can't seem to find any link for a simple a server module on the MYSQL download site.
http://dev.mysql.com/downloads/ .
It has one Cluster Server but I don't think this is what I am looking for. If anyone could provide a link to the MYSQL server module using which I could connect MYSQL Workbench on my local host, and the steps to connect them that would be great.
The one your looking for is named MySQL Community Edition. The current version is 5.6.14. Just select your platform (mac, linux or windows) and follow the installation guide.
If you wish to have php/mysql then you can download xampp. It includes all the necessary tools. Or if you wish to install it separately then follow this for Windows,
http://dev.mysql.com/downloads/installer/