For the data import handler I have moved he mysql and postgresql jar files to the solr lib directory (/opt/solr/lib).
My issue is in the data-config.xml I have put two datasources, however, I am stuck on what to put for the driver values and the urls.
<dataSource name="mysql" driver="<driver>" url="<url>" user="<user>" password="<pass>" />
<dataSource name="postgresql" driver="<driver>" url="<url>" user="<user>" password="pass>"/>
Is anyone able to tell me hat I should be putting for these values please?
MySQL
Driver - com.mysql.jdbc.Driver
URL - jdbc:mysql://localhost/dbname
Postgresql
Driver - org.postgresql.Driver
URL - jdbc:postgresql://localhost:5432/xxxxx
Related
I am new in Moqui and I start learning it so in first step I need configure MySQL database so I read the document in Moqui website and write below code in dev config but the error happen
Error initializing webapp context: java.lang.ClassNotFoundException: Class [com.mysql.jdbc.jdbc2.optional.MysqlXADataSource] not found
Do you know what should I do to solve this problem?
<datasource group-name="transactional" database-conf-name="mysql" schema-name="">
<inline-jdbc><xa-properties user="root" password="" pinGlobalTxToPhysicalConnection="true" serverName="127.0.0.1" port="3306" databaseName="MM" autoReconnectForPools="true" useUnicode="true" encoding="UTF-8"/></inline-jdbc>
</datasource>
Hi mahdi you can use this config
<datasource group-name="transactional" database-conf-name="mysql" schema-name="db_name" runtime-add-missing="false" startup-add-missing="true">
<inline-jdbc jdbc-uri="jdbc:mysql://127.0.0.1:3306/db_name?autoReconnect=true&useUnicode=true&characterEncoding=UTF-8" jdbc-username="root" jdbc-password="MySqlServerPassword" pool-minsize="2" pool-maxsize="50"/>
</datasource>
u can try this solution.It will work for you.
<!-- add datasource elements here to configure databases -->
<datasource group-name="transactional" database-conf-name="mysql" schema-name="">
<inline-jdbc jdbc-uri="jdbc:mysql://127.0.0.1:3306/database-name?useSSL=false&autoReconnect=true&useUnicode=true&characterEncoding=UTF-8"
jdbc-username="root" jdbc-password="password" pool-minsize="20" pool-maxsize="50"/>
I am trying to import a simple mySQL table into Solr and keep failing.
Having ploughed through:
https://gist.github.com/maxivak/3e3ee1fca32f3949f052
https://wiki.apache.org/solr/DataImportHandler
How to import data from mysql to solr
and half a dozen other posts that are related to Solr4, I am in desperate need of some community help.
I need specific instructions for Solr 6, it's installed on RedHat to connect to a mySQL DB. Thanks a lot.
Please find the steps below..
Install Java Runtime Environment (JRE) version 1.8 for Solr 6.1
Set Java home "export JAVA_HOME=/usr/lib/jvm/jdk1.8.0_101/"
Download solr 6.1 "solr-6.1.0.tgz" from http://www-eu.apache.org/dist/lucene/solr/6.1.0/
Extract tgz file using "tar zxf solr-6.1.0.tgz".
Open the port 8983 as to communicate with solr.
Go to the path ../solr-6.1.0/ and start the server with "bin/solr start"
Create a config folder named myConfig under configsets directory , move the conf folder into it. Add the data-config.xml into "/myConfig/conf".
Modified the schema.xml under the folder "/myConfig/conf" and add the data-config.xml.
add teh entry in solconfig.xml
data-config.xml
Fire the url to create the core.
http://localhost:8983/solr/admin/cores?action=CREATE&name=mycore&instanceDir=my_instance&configSet=myConfig
Add the jars "solr-dataimporthandler-6.1.0.jar", "solr-dataimporthandler-extras-6.1.0.jar" to path "/home/abhijit/Downloads/solr-6.1.0/server/lib".
Add the "solr-core-6.1.0.jar" jar to the path "/home/abhijit/Downloads/solr-6.1.0/server/lib".
Add the "solr-solrj-6.1.0.jar" jar to the path "/home/abhijit/Downloads/solr-6.1.0/server/lib".
If you do the above steps you will ge the error like :
"Error Instantiating requestHandler, org.apache.solr.handler.dataimport.DataImportHandler failed to instantiate org.apache.solr.request.SolrRequestHandler"
This problem occurred because dataimporthandler and requesthandler are loaded by two different class loaders. To solve it, ensure that solr loads its jars only from the same class loader.
Option to steps 5,6,7 create a folder lib inside the solr folder at path "/home/abhijit/Downloads/solr-6.1.0/server/solr"
and add the entry of the same to solrConfig.xml as "".
Comment the other lib entries from solrConfig.xml
the jars "solr-dataimporthandler-6.1.0.jar", "solr-dataimporthandler-extras-6.1.0.jar" to path "/home/abhijit/Downloads/solr-6.1.0/server/solr/lib".
Commented the updateRequestProcessorChain in solrConfig.xml.
Command for full import
http://localhost:8983/solr/mycore/dataimport?command=full-import&clean=true&commit=true
if you have any params to pass on then do something like this
http://localhost:8983/solr/mycore/dataimport?command=full-import&clean=true&commit=true&cabinetId=5654174
I'm fighting with this problem for several days and I has not been able to solve it. I've got a server with an instance of SQL Server 2008 R2 and a JBoss 7.1 installed (I'm using standalone configuration). I was trying to configure a datasource in the application server to connect to database, using JTDS drivers. The application server starts correctly but when I tried to test the datasource through the admin console it spools out the following error:
17:49:42,117 WARN [org.jboss.jca.core.connectionmanager.pool.strategy.OnePool] (HttpManagementService-threads - 1) IJ000604: Throwable while attempting to get a new connection: null: javax.resource.ResourceException: Co
uld not create connection
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:277) [ironjacamar-jdbc-1.0.9.Final.jar:1.0.9.Final]
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.createManagedConnection(LocalManagedConnectionFactory.java:235) [ironjacamar-jdbc-1.0.9.Final.jar:1.0.9.Final]
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.createConnectionEventListener(SemaphoreArrayListManagedConnectionPool.java:761) [ironjacamar-core-impl-1.0.9.Final.jar:1.0.
9.Final]
at org.jboss.jca.core.connectionmanager.pool.mcp.SemaphoreArrayListManagedConnectionPool.getConnection(SemaphoreArrayListManagedConnectionPool.java:343) [ironjacamar-core-impl-1.0.9.Final.jar:1.0.9.Final]
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getSimpleConnection(AbstractPool.java:397) [ironjacamar-core-impl-1.0.9.Final.jar:1.0.9.Final]
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.getConnection(AbstractPool.java:365) [ironjacamar-core-impl-1.0.9.Final.jar:1.0.9.Final]
at org.jboss.jca.core.connectionmanager.pool.AbstractPool.internalTestConnection(AbstractPool.java:627) [ironjacamar-core-impl-1.0.9.Final.jar:1.0.9.Final]
at org.jboss.jca.core.connectionmanager.pool.strategy.OnePool.testConnection(OnePool.java:88) [ironjacamar-core-impl-1.0.9.Final.jar:1.0.9.Final]
at org.jboss.as.connector.pool.PoolOperations$TestConnectionInPool.invokeCommandOn(PoolOperations.java:121) [jboss-as-connector-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.connector.pool.PoolOperations$1.execute(PoolOperations.java:60) [jboss-as-connector-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:385) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:272) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:200) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.connector.pool.PoolOperations.execute(PoolOperations.java:74) [jboss-as-connector-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:385) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:272) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:200) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.controller.ModelControllerImpl$DefaultPrepareStepHandler.execute(ModelControllerImpl.java:466) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.controller.AbstractOperationContext.executeStep(AbstractOperationContext.java:385) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.controller.AbstractOperationContext.doCompleteStep(AbstractOperationContext.java:272) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.controller.AbstractOperationContext.completeStep(AbstractOperationContext.java:200) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.controller.ModelControllerImpl.execute(ModelControllerImpl.java:121) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:309) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.controller.ModelControllerImpl$1.execute(ModelControllerImpl.java:299) [jboss-as-controller-7.1.1.Final.jar:7.1.1.Final]
at org.jboss.as.domain.http.server.DomainApiHandler.processRequest(DomainApiHandler.java:294)
at org.jboss.as.domain.http.server.DomainApiHandler.doHandle(DomainApiHandler.java:201)
at org.jboss.as.domain.http.server.DomainApiHandler.handle(DomainApiHandler.java:208)
at org.jboss.as.domain.http.server.security.SubjectAssociationHandler.handle(SubjectAssociationHandler.java:51)
at org.jboss.com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:78)
at org.jboss.sun.net.httpserver.AuthFilter.doFilter(AuthFilter.java:69)
at org.jboss.com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:81)
at org.jboss.sun.net.httpserver.ServerImpl$Exchange$LinkHandler.handle(ServerImpl.java:710)
at org.jboss.com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:78)
at org.jboss.as.domain.http.server.RealmReadinessFilter.doFilter(RealmReadinessFilter.java:54)
at org.jboss.com.sun.net.httpserver.Filter$Chain.doFilter(Filter.java:81)
at org.jboss.sun.net.httpserver.ServerImpl$Exchange.run(ServerImpl.java:682)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) [rt.jar:1.7.0_11]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) [rt.jar:1.7.0_11]
at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_11]
at org.jboss.threads.JBossThread.run(JBossThread.java:122) [jboss-threads-2.0.0.GA.jar:2.0.0.GA]
Caused by: java.sql.SQLException: Cannot open database "SQLEXPRESS" requested by the login. The login failed.
at net.sourceforge.jtds.jdbc.SQLDiagnostic.addDiagnostic(SQLDiagnostic.java:368)
at net.sourceforge.jtds.jdbc.TdsCore.tdsErrorToken(TdsCore.java:2820)
at net.sourceforge.jtds.jdbc.TdsCore.nextToken(TdsCore.java:2258)
at net.sourceforge.jtds.jdbc.TdsCore.login(TdsCore.java:603)
at net.sourceforge.jtds.jdbc.ConnectionJDBC2.<init>(ConnectionJDBC2.java:345)
at net.sourceforge.jtds.jdbc.ConnectionJDBC3.<init>(ConnectionJDBC3.java:50)
at net.sourceforge.jtds.jdbc.Driver.connect(Driver.java:184)
at org.jboss.jca.adapters.jdbc.local.LocalManagedConnectionFactory.getLocalManagedConnection(LocalManagedConnectionFactory.java:249) [ironjacamar-jdbc-1.0.9.Final.jar:1.0.9.Final]
... 39 more
It seems a simple authentication problem, but the user/pwd is absolutely correct! Through SQL Server Management Studio I'm able to correctly connect to the db using the following:
Server name: DAN-Aladino-vs.usersad.everis.int\SQLEXPRESS
Authentication: SQL Server Authentication
Login: aladinoDs
Password: aladinoDs
To configure the datasource I've made the following steps:
1) In JBoss I've created the directory "modules\net\sourceforge\jtds\main".
Inside it I've put the jtds-1.2.5.jar and a new module.xml with the following content:
<?xml version="1.0" encoding="UTF-8"?>
<module xmlns="urn:jboss:module:1.0" name="net.sourceforge.jtds">
<resources>
<resource-root path="jtds-1.2.5.jar"/>
<!-- Insert resources here -->
</resources>
<dependencies>
<module name="javax.api"/>
<module name="javax.transaction.api"/>
</dependencies>
</module>
2) I've modified the standalone.xml configuration file adding the followings:
<datasource jndi-name="java:jboss/datasources/AladinoDS" pool-name="AladinoDS" enabled="true" use-java-context="true">
<connection-url>jdbc:jtds:sqlserver://DAN-Aladino-vs.usersad.everis.int:1433/SQLEXPRESS</connection-url>
<driver>JTDS</driver>
<new-connection-sql>select 1</new-connection-sql>
<transaction-isolation>TRANSACTION_READ_COMMITTED</transaction-isolation>
<pool>
<min-pool-size>5</min-pool-size>
<max-pool-size>50</max-pool-size>
</pool>
<security>
<user-name>aladinoDs</user-name>
<password>aladinoDs</password>
</security>
<validation>
<check-valid-connection-sql>select 1</check-valid-connection-sql>
</validation>
<timeout>
<set-tx-query-timeout>true</set-tx-query-timeout>
<blocking-timeout-millis>5000</blocking-timeout-millis>
<idle-timeout-minutes>15</idle-timeout-minutes>
</timeout>
<statement>
<track-statements>false</track-statements>
</statement>
</datasource>
and, in the <drivers> section:
<driver name="JTDS" module="net.sourceforge.jtds">
<driver-class>net.sourceforge.jtds.jdbc.Driver</driver-class>
</driver>
According to what I've found on the web, it should be correct, but it still not work. By the way, I don't want to use Windows Authentication for the datasource but I tried also that way, unsuccessfully.
I hope someone can find something wrong in my caonfiguration. If it's correct, could be a problem of the database server/instance configuration? I'm puzzled... through SQL Server Mgmt Studio all seems to work.
Thank you all,
Luca
(Answered in comments and edits. See Question with no answers, but issue solved in the comments (or extended in chat) )
The OP wrote:
#Jon Skeet: The SQL Server error log shows the following:
01/25/2013 09:47:02,Logon,Unknown,Login failed for user 'aladinoDs'. Reason: Failed to open the explicitly specified database.
So, the problem should be that I'm not using the proper database name.
#CoolBeans: I'm not used to SQL Server (usually I work with Oracle DB), and I'm not familiar with its distinction between databases/instances/logins, but if I'm using the credentials above to successfully logon into db through SQL Server Mgmt Studio, why am I not able to use them for datasource? When I connect to db server, I can see in SQL Server Mgmt Studio, into "databases" folder, two objects: System Databases and AladinoSFA2. Should I use the latest one as server name in the connection string? This will be my next try. I'll post an update.
SOLVED:
I corrected the connection string: <connection-url>jdbc:jtds:sqlserver://DAN-Aladino-vs.usersad.everis.int:1433/AladinoSFA2</connection-url>
It seems that i would have to use the single database name and not the database server name into the connection string. I don't really understand why using SQL Server Mgmt Studio I do not need to specify db instance name, but only db server name, and in the datasource, on the contrary, the database instance name is the only one needed.
However now it all works.
Hi I have copied my Solr config from a working windows server to a new one, and it can't seem to run an import.
They're both using win server 2008 and SQL 2008R2. This is the data import config:
<dataConfig>
<dataSource type="JdbcDataSource" name="ds1"
driver="com.microsoft.sqlserver.jdbc.SQLServerDriver"
url="jdbc:sqlserver://localhost;databaseName=DB"
user="Solr"
password="pwd"/>
<document name="datas">
<entity name="data" dataSource="ds1" pk="key"
query="EXEC SOLR_COMPANY_SEARCH_DATA"
deltaImportQuery="SELECT * FROM Company_Search_Data WHERE [key]='${dataimporter.delta.key}'"
deltaQuery="SELECT [key] FROM Company_Search_Data WHERE modify_dt > '${dataimporter.last_index_time}'">
<field column="WorkDesc_Comments" name="WorkDesc_Comments_Split" />
<field column="WorkDesc_Comments" name="WorkDesc_Comments_Edge" />
</entity>
</document>
</dataConfig>
I can use MS SQL Profiler to watch the Solr user log in successfully, but then nothing. It doesn't seem to even try and execute the stored procedure. Any ideas why this would be working one server and not on another?
FTR the only thing in the tomcat catalina log is:
org.apache.solr.handler.dataimport.JdbcDataSource$1 call
INFO: Creating a connection for entity data with URL: jdbc:sqlserver://localhost;databaseName=CATLive
UPDATE:
Me and Yavar Husain from the Solr Mailing list both came up with the solution of replacing the MS JDBC Driver with an open source one - this seems to work, and means this must be a compatibility problem between the latest versions of Java, the DIH and the MS JDBC driver.
UPDATE 2:
Issues have been reported with Java 1.6.0_29 - which I am running!
http://blogs.msdn.com/b/jdbcteam/archive/2011/11/07/supported-java-versions-november-2011.aspx
This looks to have been caused by a release of Java 1.6.0_29 - which I was running:
http://blogs.msdn.com/b/jdbcteam/archive/2011/11/07/supported-java-versions-november-2011.aspx
Seeing as the problem could only be with the JDBC drivers or their config, I have worked around this issue by replacing them with the jTDS Open Source JDBC driver. My config ended up like this when I got it working with that:
<dataConfig>
<dataSource type="JdbcDataSource"
driver="net.sourceforge.jtds.jdbc.Driver"
url="jdbc:jtds:sqlserver://localhost;databaseName=DBName"
user="Solr"
password="password" name="ds1"/>
<document>
<entity dataSource="ds1" name="sometext"
query="SELECT * FROM mytable">
</entity>
</document>
</dataConfig>
HTH someone - but I'm not marking this as the answer because I can't see why I can't use the MS drivers...
I guess you are missing this
url="jdbc:mysql://localhost/dbname"
dbname -> \SQL2008
I presume when you copied your Solr config to the new server you've taken the index file too. I suspect the problem is due to the fact that Solr has already indexed this data. Your query states that you only want to index files that have changes since the last index.
Does this make sense?
The issue appears to occur only on very specific releases of the java runtime, and affects applications beyond just solr. This leads me to believe that the root cause is an incompatibility or bug between the Java runtime and Microsoft SQL Server JDBC drivers rather than being a Solr issue.
In my particular case we had two groups consisting of 3 servers each. One group showed no symptoms of this problem, and the other group all exhibited issues. All servers were running Solr 3.4.0, and the same release of SQL Server JDBC drivers 3.0. All servers were also running JRE 1.6.0_20, but upon closer inspection it was identified that the problem servers were using a slightly newer package for 1.6.0_20 (a different RPM distributed via yum for CentOS).
By downgrading the problem servers to the exact version of java on the known-good servers the problem immediately went away.
Specifically,
working:
http://pkgs.org/centos-5-rhel-5/centos-rhel-x86_64/java-1.6.0-openjdk-1.6.0.0-1.22.1.9.8.el5_6.x86_64.rpm.html
non-working:
http://pkgs.org/centos-5-rhel-5/centos-rhel-updates-x86_64/java-1.6.0-openjdk-src-1.6.0.0-1.23.1.9.10.el5_7.x86_64.rpm.html
Can we configure database connection from JBOSS? If it is possible, than is there any configuration file in JBOSS to configure database connection from JBOSS?
You need two things:
Make the JDBC driver available to your Applications Server
Write a Data Source configuration
For #1, you can download the JAR containing JDBC driver and put it in the following directory:
$JBOSS_HOME/server/default/lib
Assuming that $JBOSS_HOME points to your JBoss installation, and you are using default installation.
For #2, you will find a lot of examples here:
$JBOSS_HOME/docs/examples/jca
There are examples for most of database products around. Here's PostgreSQL's:
<datasources>
<local-tx-datasource>
<jndi-name>PostgresDS</jndi-name>
<connection-url>jdbc:postgresql://[servername]:[port]/[database name]</connection-url>
<driver-class>org.postgresql.Driver</driver-class>
<user-name>x</user-name>
<password>y</password>
<!-- sql to call when connection is created. Can be anything, select 1 is valid for PostgreSQL
<new-connection-sql>select 1</new-connection-sql>
-->
<!-- sql to call on an existing pooled connection when it is obtained from pool. Can be anything, select 1 is valid for PostgreSQL
<check-valid-connection-sql>select 1</check-valid-connection-sql>
-->
<!-- corresponding type-mapping in the standardjbosscmp-jdbc.xml (optional) -->
<metadata>
<type-mapping>PostgreSQL 7.2</type-mapping>
</metadata>
</local-tx-datasource>
</datasources>
Well, this looks like a good guide for configuring a datasource for JBoss.
A JDBC driver is needed for your database (a jar file, refer to your db's documentation) and afterwards, configuration. You should be able to configure it using JBoss admin console.
After your datasource is configured, you can use JNDI or some other mechanism for obtaining it in your application. Then you use JDBC for actually interacting with your database. Another popular alternative is using JPA since JBoss already has Hibernate built in.
Follow the following steps:
Add my sql connector jar in main folder (eg. \modules\system\layers\base\com\mysql\main).
Add a module.xml file to this folder consisting of the following config:
<?xml version="1.0" encoding="UTF-8"?>
<module xmlns="urn:jboss:module:1.0" name="com.mysql">
<resources>
<resource-root path="mysql-connector-java-5.1.17-bin.jar"/>
</resources>
<dependencies>
<module name="javax.api"/>
<module name="javax.transaction.api"/>
<module name="javax.servlet.api" optional="true"/>
</dependencies>
</module>
Now in stanalone.xml file add mysql driver under drivers tag as following:
<driver name="mysql" module="com.mysql">
<driver-class>com.mysql.jdbc.Driver</driver-class>
</driver>
start the jboss server from command prompt by going to location of the jboss/bin folder and executing standalone.bat
Now in a browser, open localhost:8080, click on administration console and under create a datasource click datasource. Then click on add.
Add the following details:
Name: MysqlDS5
JNDI Name : java:/mysql
click next and under "detected driver" select mysql.
click next
Connection url : jdbc:mysql://localhost:3306/sampledb
UserName:****
Password:****
click done, select the MysqlDS and click on enable.
click on test connection and it will successfully connect.