I configured hive with MYSQL as repository. When I start the hive server using my standard user (infa_hadoop) it is giving me an error "cant connect to metastore using the URI provided".
But if I login as root and start the hive server it starts well.
command used:
hive --service hiveserver
But when I tried to execute the ETL job (informatica) it is giving me the Access control exception!
Error :
Function [INFASQLExecute] failed in adapter [/u01/app/informatica/plugins/dynamic/hiveruntime/libhive.so] with error code [-1].
FnName: INFASQLExecute -- execute(). SQLException: Query returned non-zero code: 12, cause: FAILED: Hive Internal Error: java.lang.RuntimeException(org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="scratchdir":Infa_Linux:supergroup:rwxr-xr-x)
Function [INFASQLGetDiagRecW] failed in adapter [/u01/app/informatica/plugins/dynamic/hiveruntime/libhive.so] with error code [100].
FnName: INFASQLExecute -- execute(). SQLException: Query returned non-zero code: 12, cause: FAILED: Hive Internal Error: java.lang.RuntimeException(org.apache.hadoop.security.AccessControlException: org.apache.hadoop.security.AccessControlException: Permission denied: user=root, access=WRITE, inode="scratchdir":Infa_Linux:supergroup:rwxr-xr-x)
Function [INFASQLGetDiagRecW] failed in adapter [/u01/app/informatica/plugins/dynamic/hiveruntime/libhive.so] with error code [100].].
But hive is working fine in command promt ? Any suggestions..
First, please check if your hive thrift server is up and running .
It is advised to use the following command to start the hive thrift server
hive --service hiveserver -p 10001
Telnet and check if the server is running in port 10001 , If yes I suppose your issue would be resolved
Related
I have a remote MySQL server which I can connect to via my command line.
But when I try to run npx prisma db pull
it gives me this error:
Error: Error in connector: Error querying the database: Error querying the database: Error querying the database: Server error: ERROR 08S01 (1043): Bad handshake'
What did I do
My prisma database url is: mysql://sven:<URL-ENCODEDPASSWORD>#<IP-ADDRESS>3306/portfolio?sslidentity=client-identity.p12&sslpassword=<SSL-PASSORD>&sslcert=ca.pem
I copied the ssl certificate and keys like this, I made sure I copied it right.
:
sudo cat /var/lib/mysql/client-key.pem
sudo cat /var/lib/mysql/client-cert.pem
sudo cat /var/lib/mysql/ca.pem
following the documention I made the one sslindentiy of it.
When connecting from my command line I do this with the following command: mysql -usven -h<ip-address> -p
Does anyone have an idea how to fix this issue?
I installed Apache hive 3, Apache Hadoop 3, Trino and Mysql (mysql Ver 14.14 Distrib 5.7.20, for Linux (x86_64) using EditLine wrapper).
When I start Hive metastore I had an exception, it blocked the start of the metastore:
Exception:
Exception in thread "main" java.lang.NoSuchMethodError: com/google/common/base/Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V (loaded from file:/root/trino_poc/trino-server-351/etc/catalog/apache-hive-3.1.2-bin/lib/guava-19.0.jar by jdk.internal.loader.ClassLoaders$AppClassLoader#8753f89b) called from class org.apache.hadoop.conf.Configuration (loaded from file:/root/trino_poc/hadoop-3.2.2/share/hadoop/common/hadoop-common-3.2.2.jar by jdk.internal.loader.ClassLoaders$AppClassLoader#8753f89b).
at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
My solution was (I found it on github) to replace the guava-19.0.jar by guava-22.0.jar
So the metastore started well on localhost:9083
But the port is not in listen: netstat -na | grep :9083 ==> nothing
in hive log I have this error:
Unable to open a test connection to the given database. JDBC url = jdbc:mysql://localhost:3306/metastore_dbs?useSSL=false;createDatabaseIfNotExist=true, username = root. Terminating connection pool (set lazyInit to true if you expect to start your database after your app). Original Exception: ------^M
java.sql.SQLException: The connection property 'useSSL' only accepts values of the form: 'true', 'false', 'yes' or 'no'. The value 'false;createDatabaseIfNotExist=true' is not in this set
So the issue is here: 'false;createDatabaseIfNotExist=true' the ';' is not supported in xml file because this code is exist in hive-site.xml and I need to properly escape the characters.
I used '?' instead, Hivemetastore started but port 9083 is not listen, and I got almost the same exception'false?createDatabaseIfNotExist=true'
I used '&' ==> Hive metastore is not started (failed) and I got this exception:
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by com.google.common.base.internal.Finalizer (file:/root/trino_poc/trino-server-351/etc/catalog/apache-hive-3.1.2-bin/lib/guava-22.0.jar) to field java.lang.Thread.inheritableThreadLocals
WARNING: Please consider reporting this to the maintainers of com.google.common.base.internal.Finalizer
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
MetaException(message:org.jets3t.service.security.ProviderCredentials)
If I resume:
To resolve exception to initialize Database and start hive metastore I replace the guava-19.0.jar by guava-22.0.jar but port 9083 is not listen.
Then to resolve the issue of port listening:
I Used '?' instead of ';' ==> hive metastore still started well
but port 9083 in not listen.
I tried using '&' instead of '?' ==> Hive metastore not
started (failed) and I got the exception above related to
guava-22.0.jar
I hope that I was clear. Really I need to your help, because I spent a lot of hours trying to resolve the issue.
Thanks in advance
I'm trying to get a file from Mysql webpage from command line Ubuntu:
wget "http://dev.mysql.com/get/mysql-apt-config_0.8.3-1_all.deb"
But I'm getting error:
--2017-10-09 15:22:23-- http://dev.mysql.com/get/mysql-apt-config_0.8.3-1_all.deb
Resolving dev.mysql.com (dev.mysql.com)... 137.254.60.11
Connecting to dev.mysql.com (dev.mysql.com)|137.254.60.11|:80... failed: Connection refused.
What is the reason for this connection refuse error ?
Does MySql workbench support the DB2 to MySql migration ?
Steps I followed :
Installed DB2 ODBC driver and and defined the conection parameters for my DB2 database in 'Source Selection'.
I am using Database System as "Generic RDBMS" and Connection Method as "ODBC Data Source" and DSN with DB2 ODBC driver.
Defined the MySql database details in 'Target Selection'
When I do next and it goes to 'Fetch Schema List', I get error as below. Although my source and target database connection successful.
Error logs from workbench as follows :
Starting...
Connect to source DBMS...
Connecting to source...
Connection to apparently lost, reconnecting...
Connecting to ...
Opening ODBC connection to DSN=EESBPTS...
Connected
Connect to source DBMS done
Check target DBMS connection...
Connecting to target...
Check target DBMS connection done
Retrieve schema list from source....
Checking connection...
Connection to apparently lost, reconnecting...
Connecting to ...
Opening ODBC connection to DSN=EESBPTS...
Connected
Fetching catalog names...
Fetching schema names...
Fetching schema names from None...
Traceback (most recent call last):
File "C:\Program Files\MySQL\MySQL Workbench 6.2 CE\workbench\wizard_progress_page_widget.py", line 192, in thread_work
self.func()
File "C:\Program Files\MySQL\MySQL Workbench 6.2 CE\modules\migration_source_selection.py", line 456, in task_fetch_schemata
self.main.plan.migrationSource.doFetchSchemaNames(only_these_catalogs)
File "C:\Program Files\MySQL\MySQL Workbench 6.2 CE\modules\migration.py", line 255, in doFetchSchemaNames
self.state.sourceSchemataNames.append("%s.%s" % (self._db_module.quoteIdentifier(catalog), self._db_module.quoteIdentifier(schema)))
TypeError: DbGenericRE.quoteIdentifier(): argument 1 must be a string but is None
ERROR: Retrieve schema list from source: DbGenericRE.quoteIdentifier(): argument 1 must be a string but is None
Failed
Please help me to proceed further.
I have replaced sqoop metatsore with a MySQL db.
When I try to run a saved sqoop job from the command line, it "asks" me for the password and run the job when I supply it.
However, I need to run this job through oozie.
--password is not recognized as a valid argument here in command line while executing sqoop saved job and anyway, it keeps asking for password through the prompt.
Now, when I try to run it through oozie,
I get
Caused by: java.sql.SQLException: The database is already in use by
another process: org.hsqldb.persist.NIOLockFile#950abfc6[file
=/home/yarn/.sqoop/metastore.db.lck, exists=false, locked=false,
valid=false, fl =null]: java.io.FileNotFoundException:
/home/yarn/.sqoop/metastore.db.lck (No such file or directory)
What is this and why is this? I am directly connecting to my MySQL metatsore and have configured sqoop-site.xml accordingly.
Why is this trying to to connect to hsqldb?
Why is it locked?
How do I fix this?
Also, how do I supply password for execution of sqoop job in oozie?