Unable to start Hive metastore related to java exception - mysql

I installed Apache hive 3, Apache Hadoop 3, Trino and Mysql (mysql Ver 14.14 Distrib 5.7.20, for Linux (x86_64) using EditLine wrapper).
When I start Hive metastore I had an exception, it blocked the start of the metastore:
Exception:
Exception in thread "main" java.lang.NoSuchMethodError: com/google/common/base/Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V (loaded from file:/root/trino_poc/trino-server-351/etc/catalog/apache-hive-3.1.2-bin/lib/guava-19.0.jar by jdk.internal.loader.ClassLoaders$AppClassLoader#8753f89b) called from class org.apache.hadoop.conf.Configuration (loaded from file:/root/trino_poc/hadoop-3.2.2/share/hadoop/common/hadoop-common-3.2.2.jar by jdk.internal.loader.ClassLoaders$AppClassLoader#8753f89b).
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
My solution was (I found it on github) to replace the guava-19.0.jar by guava-22.0.jar
So the metastore started well on localhost:9083
But the port is not in listen: netstat -na | grep :9083 ==> nothing
in hive log I have this error:
Unable to open a test connection to the given database. JDBC url = jdbc:mysql://localhost:3306/metastore_dbs?useSSL=false;createDatabaseIfNotExist=true, username = root. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------^M
java.sql.SQLException: The connection property 'useSSL' only accepts values of the form: 'true', 'false', 'yes' or 'no'. The value 'false;createDatabaseIfNotExist=true' is not in this set
So the issue is here: 'false;createDatabaseIfNotExist=true' the ';' is not supported in xml file because this code is exist in hive-site.xml and I need to properly escape the characters.
I used '?' instead, Hivemetastore started but port 9083 is not listen, and I got almost the same exception'false?createDatabaseIfNotExist=true'
I used '&' ==> Hive metastore is not started (failed) and I got this exception:
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.common.base.internal.Finalizer (file:/root/trino_poc/trino-server-351/etc/catalog/apache-hive-3.1.2-bin/lib/guava-22.0.jar) to field java.lang.Thread.inheritableThreadLocals
WARNING: Please consider reporting this to the maintainers of com.google.common.base.internal.Finalizer
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
MetaException(message:org.jets3t.service.security.ProviderCredentials)
If I resume:
To resolve exception to initialize Database and start hive metastore I replace the guava-19.0.jar by guava-22.0.jar but port 9083 is not listen.
Then to resolve the issue of port listening:
I Used '?' instead of ';' ==> hive metastore still started well
but port 9083 in not listen.
I tried using '&' instead of '?' ==> Hive metastore not
started (failed) and I got the exception above related to
guava-22.0.jar
I hope that I was clear. Really I need to your help, because I spent a lot of hours trying to resolve the issue.
Thanks in advance

Related

Migrate mysql to postgres using pgloader

I'm trying to migrate mysql to postgres using pgloader, since I'm using window so, I installed pgloader in
windows linux subsystem, and I have used following command to migrate the DB
pgloader mysql://root#localhost/f1db pgsql://postgres#localhost:5433/f1db
KABOOM!
FATAL error: Failed to connect to pgsql at "localhost" (port 5433) as user "postgres": Server requested md5-password authentication, but no password was given.
An unhandled error condition has been signalled:
Failed to connect to pgsql at "localhost" (port 5433) as user "postgres": Server requested md5-password authentication, but no password was given.
What I am doing here?
Failed to connect to pgsql at "localhost" (port 5433) as user "postgres": Server requested md5-password authentication, but no password was given.
Waiting for the monitor thread to complete.
Please tell that how to pass password for both MYsql and Postgres in the command line and if I have # in my password how to use it in the command because we already have user#localhost in the command.
From pgloader Reference Manual (https://pgloader.readthedocs.io/en/latest/pgloader.html):
command line:
pgloader SOURCE TARGET
Source Connection String
db://user:pass#host:port/dbname
Where db might be of sqlite, mysql or mssql.
Connection String
The parameter is expected to be given as a Connection URI as documented in the PostgreSQL documentation at http://www.postgresql.org/docs/9.3/static/libpq-connect.html#LIBPQ-CONNSTRING.
postgresql://[user[:password]#][netloc][:port][/dbname][?option=value&...]
user
Can contain any character, including colon (:) which must then be doubled (::) and at-sign (#) which must then be doubled (##).
When omitted, the user name defaults to the value of the PGUSER environment variable, and if it is unset, the value of the USER environment variable.
password
Can contain any character, including the at sign (#) which must then be doubled (##). To leave the password empty, when the user name ends with at at sign, you then have to use the syntax user:#.
When omitted, the password defaults to the value of the PGPASSWORD environment variable if it is set, otherwise the password is left unset.
When no password is found either in the connection URI nor in the environment, then pgloader looks for a .pgpass file as documented at https://www.postgresql.org/docs/current/static/libpq-pgpass.html

Knowage installation, database error, mysql

I am trying to install Knowage to my Centos VM in command line and get the following error:
WARNING: please provide a database user that can create schemas. The
following schemas will be created (or overwritten):
knowage_demo
foodmart_demo
JDBC connection failed Database Management System Configuration Use an
already installed DBMS [1, Enter]
Select DBMS for metadata:
MariaDB [1, Enter]
MySQL [2]
2
[jdbc:mysql://localhost:3306]
Username: [root]
Password:
WARNING: please provide a database user that can create schemas.
Any ideas on how to make it working? It keeps getting back to this step.

How to create a new MySQL server instance in workbench?

I'm using:
Mac OS X
MySQL Ver 14.14 Distrib 5.6.27
And latest MySQL workbench, which, for some reason, still telling me, that it uses MySQl version 5.6.24.
So, I could create new connections, but when I try to configure local management, in stage of test settings I always get:
Error: error calling Python module function
WbAdmin.testInstanceSettingByName
Check if /etc/my.cnf can be accessed File was found at the expected
location Check if section for instance mysqld exists in /etc/my.cnf
Traceback (most recent call last): File
"/Applications/MySQLWorkbench.app/Contents/Resources/plugins/wb_admin_grt.py",
line 777, in testInstanceSettingByName
if check_if_config_file_has_section(open(config_file, "r"), section): IOError: [Errno 13] Permission denied: '/etc/my.cnf'
Operation failed: error calling Python module function
WbAdmin.testInstanceSettingByName
I suppose, that it is caused by absence of any server instances. When I try to Rescan for Local MySQL Servers, I get message:
No servers were found.
So, what's wrong? How to solve this problem, how to create a server instance?

Unable to start mysql in redmine

Hi I'm unable to start my mysql and subversion using root as user.When im trying to start it by ./ctlscript.sh start mysql it is showing that mysql could not be started.The same is happening in the case of subversion also.In logs its showing that
error: Can't connect to local MySQL server through socket '/data/redmine/mysql/tmp/mysql.sock' (2) (Mysql2::Error)
error: [ pid=6349 thr=139977144923904 file=ext/common/agents/HelperAgent/RequestHandler.h:1731 time=2014-09-29 12:54:58.7821 ]: [Client 23] Cannot checkout session. An error occured while starting up the preloader.
Can any one help?
Please check whether your root user has the permission to start the mysql and other services.
If not please give the permissions using CHMOD

Hive metastore configuration

I configured hive with MYSQL as repository. When I start the hive server using my standard user (infa_hadoop) it is giving me an error "cant connect to metastore using the URI provided".
But if I login as root and start the hive server it starts well.
command used:
hive --service hiveserver
But when I tried to execute the ETL job (informatica) it is giving me the Access control exception!
Error :
Function [INFASQLExecute] failed in adapter [/u01/app/informatica/plugins/dynamic/hiveruntime/libhive.so] with error code [-1].
FnName: INFASQLExecute -- execute(). SQLException: Query returned non-zero code: 12, cause: FAILED: Hive Internal Error: java.lang.RuntimeException(org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="scratchdir":Infa_Linux:supergroup:rwxr-xr-x)
Function [INFASQLGetDiagRecW] failed in adapter [/u01/app/informatica/plugins/dynamic/hiveruntime/libhive.so] with error code [100].
FnName: INFASQLExecute -- execute(). SQLException: Query returned non-zero code: 12, cause: FAILED: Hive Internal Error: java.lang.RuntimeException(org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="scratchdir":Infa_Linux:supergroup:rwxr-xr-x)
Function [INFASQLGetDiagRecW] failed in adapter [/u01/app/informatica/plugins/dynamic/hiveruntime/libhive.so] with error code [100].].
But hive is working fine in command promt ? Any suggestions..
First, please check if your hive thrift server is up and running .
It is advised to use the following command to start the hive thrift server
hive --service hiveserver -p 10001
Telnet and check if the server is running in port 10001 , If yes I suppose your issue would be resolved