I have a AWS EMR and I want to get access to table in meta store. I found the hive-site.xml with
javax.jdo.option.ConnectionURL
jdbc:mysql://ip-XX-XX-XX-XX.eu-central-1.compute.internal:3306/hive?createDatabaseIfNotExist=true
javax.jdo.option.ConnectionDriverName
org.mariadb.jdbc.Driver
javax.jdo.option.ConnectionUserName
hive
...
hive.metastore.uris
thrift://ip-172-31-40-111.eu-central-1.compute.internal:9083
I tried to connect directly from masternode but there was no access.
Could you please help to connect from master node?
Related
I currently deployed my laravel app to the AWS elastic beanstalk. I created RDS mysql database. I imported my sql dump from my local project and deployed it to the AWS EC2 server. But I'm getting this error.
SQLSTATE[HY000] [2002] No such file or directory (SQL: select * from settings where code = company-name limit 1).
Is there any way how to check if there are data in my DB ? Just to note I'm still learning moving around AWS. Thanks.
EDIT :
I'm also getting correct Environment & details like RDS_PORT RDS_HOST... , on error page where I can see the error displaying
You can follow this tutorial to connect to a MYSQL RDS Database on AWS.
So you can check if all your tables have been created successfully.
AWS Connect to Database Tutorial
Make sure that you open the port of the RDS database in the security group. Otherwise your local machine can't connect to the Database.
I just installed MySQL and phpMyAdmin on my AWS EC2 instance running Linux AMI. I would like to find out how to change the default URL to access phpMyAdmin, which is currently /phpMyAdmin. I found an article on how to do this:
https://www.tecmint.com/change-secure-phpmyadmin-login-url-page/
but there is no such file: /etc/httpd/conf.d/phpMyAdmin.conf. The only config file I can find is the sample config file: /var/www/html/phpMyAdmin/config.sample.inc.php. But this sample config file doesn't seem to have anything about url alias.
Would appreciate some help on this, thanks!
I have confluent platform on my local machine i am just trying to read the data from aws ec2 instance i have credentials like hostname, DB name, pwd etc. I am using JDBC source connector. connector config is
name=test
connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
connection.url=jdbc:mysql://ab.bca.bdc.aaa:abcd/DB?user=abc&password=bca
table.whitelist=ppp
mode=incrementing
after running connect standalone got error like
Invalid value java.sql.SQLNonTransientConnectionException: Cannot load connection class because of underlying exception: com.mysql.cj.exceptions.WrongArgumentException: Malformed database URL, failed to parse the connection string near
According to the JDBC MySQL syntax, user and password go before the database address as colon separated, not after
user:password#host_or_host_sublist
Not sure if using RDS or your own EC2 database would change the syntax of that
If you want to capture all database events, though, Debezium (your old question) would be what you want. Using the JDBC connector won't capture deletes (or events created and deleted between polls) and puts unnecessary strain on your database
It is a MySQL database server. When I login in AWS management console, it looks normal. But cannot connect to the server from any applications (PUTTY, MySQL clients, etc). Other than create a new instance, any ideas?
You cannot create new instance on aws. You can install phpmyadmin https://www.phpmyadmin.net/downloads/ and just upload it using sftp and extract then access on the browser http://ec2/phpmyadmin_directory_name.
If you are trying to access ssh then you have to create pem key link : https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-key-pairs.html
then add the key to your putty. Thats it.
I am creating a EMR cluster with default hive meta store , after which i am overriding the hive-site.xml with some property which are pointing the aws rds instance as hive metastore , everything is fine , but after restarting the hive server , i am not able to use RDS as hive metastore. It is still usin the default hive metastore created by EMR.
You can override the default configurations for applications by supplying a configuration object for applications when you create a cluster. The configuration object is referenced as a JSON file. Configuration objects consist of a classification, properties, and optional nested configurations. Properties are the settings you want to change in that file. You can specify multiple classifications for multiple applications in a single JSON object.
For overriding hive-site.xml with your external mysql metastore information, create a configuration file called hiveConfiguration.json containing edits to hive-site.xml:
[
{
"Classification": "hive-site",
"Properties": {
"javax.jdo.option.ConnectionURL": "jdbc:mysql:\/\/hostname:3306\/hive?createDatabaseIfNotExist=true",
"javax.jdo.option.ConnectionDriverName": "org.mariadb.jdbc.Driver",
"javax.jdo.option.ConnectionUserName": "username",
"javax.jdo.option.ConnectionPassword": "password"
}
}
]
Use hiveConfiguration.json with the following AWS CLI command to create the cluster:
aws emr create-cluster --release-label emr-5.11.0 --instance-type m3.xlarge --instance-count 2 \
--applications Name=Hive --configurations ./hiveConfiguration.json --use-default-roles
Reference :
https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hive-metastore-external.html
Why do you restart Hive-server2 ? If you are changing properties like hive.metastore.* , then you will need to restart Hive Metastore daemon instead, which on EMR is part of Hcatalog. Actually, you might not need to even restart anything for just updating the metastore DB. You can just run offline schematool -initSchema to point to the new DB.
see https://cwiki.apache.org/confluence/display/Hive/Hive+Schema+Tool