I am trying to perform an sqoop export on HDP sandbox 2.1 via Oozie. When I run the Oozie job I get the following java runtime exception.
'>>> Invoking Sqoop command line now >>>
7598 [main] WARN org.apache.sqoop.tool.SqoopTool - $SQOOP_CONF_DIR
has not been set in the environment. Cannot check for additional
configuration.
7714 [main] INFO org.apache.sqoop.Sqoop - Running Sqoop version:
1.4.4.2.1.1.0-385
7760 [main] WARN org.apache.sqoop.SqoopOptions - Character argument
'\t' has multiple characters; only the first will be used.
7791 [main] WARN org.apache.sqoop.ConnFactory - $SQOOP_CONF_DIR has
not been set in the environment. Cannot check for additional
configuration.
7904 [main] INFO org.apache.sqoop.manager.MySQLManager - Preparing
to use a MySQL streaming resultset.
7905 [main] INFO org.apache.sqoop.tool.CodeGenTool - Beginning code
generation
7946 [main] ERROR org.apache.sqoop.Sqoop - Got exception running
Sqoop: java.lang.RuntimeException: Could not load db driver class:
com.mysql.jdbc.Driver Intercepting System.exit(1)
I have copied jdbc driver file "mysql-connector-java.jar" to Oozie's shared library folder which I believe is "/usr/lib/oozie/share/lib/sqoop/". I have restarted my sandbox and tried to perform the export with Oozie again and I still get the same error.
The export works perfectly fine when I try performing it only via sqoop, so I presume Oozie needs its own set of drivers.
My question is, which Oozie directory am I suppose to copy my jdbc drivers to?
If you guys think I'm doing something wrong or you need further information, please let me know.
Thank you for your time.
Normally for Oozie the sharelib directory is /user/oozie/share/lib/ on HDFS where "oozie" would be the name of the user which is used to start the Oozie Server. I don't know what that is in case of HDP sandbox 2.1 , but you can use ps command to figure that out.
And for jars needed for sqoop action, I think you should copy the jar to /user/oozie/share/lib/sqoop/ folder.
Related
<>#BWEclipseAppNode> 12:12:32.545 INFO [main] com.tibco.thor.frwk.Deployer - Started by BusinessStudio.
12:12:34.970 INFO [main] com.tibco.bw.frwk.engine.BWEngine - TIBCO-BW-FRWK-300002: BW Engine [Main] started successfully.
12:12:35.355 INFO [Framework Event Dispatcher: Equinox Container: 80edabb3-8e47-001c-18c5-90bdbc610de0] com.tibco.thor.frwk.Deployer - TIBCO-THOR-FRWK-300001: Started OSGi Framework of AppNode [BWEclipseAppNode] in AppSpace [BWEclipseAppSpace] of Domain [BWEclipseDomain]
12:12:35.483 INFO [Framework Event Dispatcher: Equinox Container: 80edabb3-8e47-001c-18c5-90bdbc610de0] com.tibco.thor.frwk.Application - TIBCO-THOR-FRWK-300018: Deploying BW Application [RESTservice2.application:1.0].
12:12:40.067 INFO [Framework Event Dispatcher: Equinox Container: 80edabb3-8e47-001c-18c5-90bdbc610de0] com.tibco.thor.frwk.Application - TIBCO-THOR-FRWK-300021: All Application dependencies are resolved for Application [RESTservice2.application:1.0]
12:12:41.431 INFO [Thread-28] com.tibco.thor.frwk.Application - TIBCO-THOR-FRWK-300019: BW Application [RESTservice2.application:1.0] is impaired.
12:12:41.435 INFO [Framework Event Dispatcher: Equinox Container: 80edabb3-8e47-001c-18c5-90bdbc610de0] com.tibco.thor.frwk.Application - Started by BusinessStudio, ignoring .enabled settings.
12:12:41.438 ERROR [CM Configuration Updater (Update: pid=bw.resource.jdbc.916120c9-fbcb-45de-a13c-b22e3edf76ec)] com.tibco.bw.sharedresource.runtime.dependency.ReferenceDependency - TIBCO-BW-SR-FRWK-503000: Unable to start SharedResource [restservice2.JDBCConnectionResource] from Module [RESTservice2:1.0.0.qualifier], DeploymentUnit [RESTservice2.application:1.0]. <Reason>: TIBCO-BW-SR-JDBC-500003: The database driver [com.mysql.jdbc.Driver] is not found. Ensure that DataSourceFactory bundle providing this driver is available in the environment.
so this is the error i am getting as a result i cant run OSGi commands to request endpoint urls and for my rest service even though my code has no errors before debug ,i am getting this during debug ,please help.
In order to use the MySql JDBC driver you need to install it in the BusinessWorks environment of your machine.
This is done by running the command 'bwinstall mysql-driver' from the <TIBCO_HOME>bw/6.X/bin folder.
As Emmanuel suggested, according to the error, you probably missed the required JDBC driver installation.
Once you execute the command, it will start successfully.
I am trying to connect an application with VisualVM, but VisualVM unable to connect with application: Below is the env:
JBoss EAP 7.3
Java 11
OpenShift
I have tried to configure it in different ways, but all failed:
Config try 1:
Use few env variables in script file, so that it could execute first (file contents are mentioned below):
echo *** Adding system property for VisulVM ***
batch
/system-property=jboss.modules.system.pkgs:add(value="org.jboss.byteman,com.manageengine,org.jboss.logmanager")
/system-property=java.util.logging.manager:add(value="org.jboss.logmanager.LogManager")
run-batch
I can see that above commands executed successfully and above properties are available in JBoss config (I verified using Jboss cli command).
JAVA_TOOLS_OPTIONS: -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=n -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=3000 -Dcom.sun.management.jmxremote.rmi.port=3001 -Djava.rmi.server.hostname=127.0.0.1 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Xbootclasspath/a:/opt/eap/modules/system/layers/base/org/jboss/log4j/logmanager/main/log4j-jboss-logmanager-1.2.0.Final-redhat-00001.jar -Xbootclasspath/a:/opt/eap/modules/system/layers/base/org/jboss/logmanager/main/jboss-logmanager-2.1.14.Final-redhat-00001.jar
Result:
- java.lang.RuntimeException: WFLYCTL0079: Failed initializing module org.jboss.as.logging
- Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: WFLYLOG0078: The logging subsystem requires the log manager to be org.jboss.logmanager.LogManager. The subsystem has not be initialized and cannot be used. To use JBoss Log Manager you must add the system property "java.util.logging.manager" and set it to "org.jboss.logmanager.LogManager"
Config 2:
JAVA_TOOL_OPTIONS= -agentlib:jdwp=transport=dt_socket,server=y,address=8000,suspend=n -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=3000 -Dcom.sun.management.jmxremote.rmi.port=3001 -Djava.rmi.server.hostname=127.0.0.1 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Djava.util.logging.manager=org.jboss.logmanager.LogManager -Djboss.modules.system.pkgs=org.jboss.byteman,org.jboss.logmanager -Xbootclasspath/a:/opt/eap/modules/system/layers/base/org/jboss/log4j/logmanager/main/log4j-jboss-logmanager-1.2.0.Final-redhat-00001.jar -Xbootclasspath/a:/opt/eap/modules/system/layers/base/org/jboss/logmanager/main/jboss-logmanager-2.1.14.Final-redhat-00001.jar
Result:
WARNING: Failed to instantiate LoggerFinder provider; Using default.
java.lang.IllegalStateException: The LogManager was not properly installed (you must set the "java.util.logging.manager" system property to "org.jboss.logmanager.LogManager")
Config 3:
• Modify in standalone.conf, where I put all required configuration in this file.
Result:
WARNING: Failed to instantiate LoggerFinder provider; Using default.
java.lang.IllegalStateException: The LogManager was not properly installed (you must set the "java.util.logging.manager" system property to "org.jboss.logmanager.LogManager")
Kindly suggest that what is the correct configurations?
I am using ofbiz in my organization. I want to migrate ofbiz from derby to MySQL.
I refer the steps from
(https://cwiki.apache.org/confluence/display/OFBIZ/How+to+migrate+OfBiz+from+Derby+to+MySQL+database) here, but at the i got stuck at the end.
At the end when I type (java -jar ofbiz.jar -install) this command I am getting an exception,
C:\Users\sagar_vinod_khanke\Sagar\Apache OFBiz\Ofbiz\13.07>java -jar ofbiz.jar -
install
Exception in thread "main" org.ofbiz.base.start.StartupException: Couldn't not f
etch config instance
at org.ofbiz.base.start.Start.init(Start.java:202)
at org.ofbiz.base.start.Start.main(Start.java:127)
Caused by: java.io.IOException: Cannot load configuration properties : org/ofbiz
/base/start/-install.properties
at org.ofbiz.base.start.Config.getPropertiesFile(Config.java:229)
at org.ofbiz.base.start.Config.readConfig(Config.java:297)
at org.ofbiz.base.start.Config.getInstance(Config.java:58)
at org.ofbiz.base.start.Start.init(Start.java:200)
... 1 more
Can you please help me?
don't use - with install.
See revised Step-V
Step-V
1. Run the following command from command prompt:
java -jar ofbiz.jar install
2. Start OfBiz
3. Use webtools to import all data from XML:
a. Navigate to http://localhost:8080/catalog/
b. Go to Applications>WebTools
c. Go to section 'Entity XML Tools' and click the link 'XML Data Import Dir' -> In the 'Absolute directory path:' enter the full path of the directory where you exported the data in Step - II
I have installed Hive and mysql successfully, i did the configuration for Hive as suggested in link. But i see an error as below:
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
..
..
Caused by: org.datanucleus.exceptions.NucleusException: Attempt to invoke the "BONECP" plugin to create a ConnectionPool gave an error : The specified datastore driver ("com.mysql.jdbc.Driver") was not found in the CLASSPATH. Please check your CLASSPATH specification, and the name of the driver.
so i added the mysql-connector-java.jar in lib of Hive. Now hive just hangs, i dont get the shell at all.
Kindly suggest how i can resolve it
You need to add mysql connector to the classpath in the hive. It is looking for that connector in your classpath and its not able to find it. Download mysql connector and put it to the following path
/usr/lib/hive/apache-hive-0.13.0-bin/lib
I am using shared node cluster
Hadoop 2.5.0-cdh5.3.2
Please share the names all compatible version of MySql jar files to be loaded and all the path folders for any successful import and export between HDFS and MySQL.
I am currently getting below error message
ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
so far, I have loaded MySql connector and hadoop jar files in /usr/lib/sqoop/lib
mysql-connector-java-5.0.8-bin
hadoop-core-1.0.3
Please let me know if I need to add more files and specify the path
Is t
If it is a version incompatibility issue, then you can give a try to mysql-connector-java-5.1.31.jar as I am using mysql-connector-java-5.1.31.jar with sqoop version 1.4.5. For me, it works for both data import and export use cases.