Couchbase server timeout expired after upgrade to 3.1.0 - couchbase

We recently upgraded our cluster from Couchbase 2.5.1 to 3.1.0. But when trying to update documents or read from an index, we're getting 'timeout expired' errors. That happens from .NET client and curl/postman. We use .NET client 2.1.2 if that helps.
When I looked into the logs, I found errors for document updates in couchdb.log looking like this: couch_log:error:44]dcp client (bucket_name, mapreduce_view: bucket_name _design/main (prod/main)): Obtaining mutation from server timed out after 60.0 seconds [RequestId 318, PartId 825, StartSeq 0, EndSeq 34450]. Waiting...
Something is wrong with compacting/updating indexes: Set view bucket_name, main (prod) group _design/main, compactor process <0.27790.45> died with unexpected reason: {updater_died, {updater_error, {view_group_index_updater_exit, 96, <<>>}}}
What does it mean?

We ended up dropping views data and recreating views from scratch

Related

karaf + pax-jdbc the connection pool had reached the limit

I have a problem with the pool connections of pax-jdbc in karaf, I'm trying to inject a Mysql DataSource (DS) through
blueprint.xml into my project, for test it, I have built a karaf command where injects the DS into karaf command class
and execute a query with that connection. That it's OK, but the problem is when I execute the command a lot times, for
each execution a new instance of the DS is created and the pool connection cannot open new connections to MySQL, because
the pool had reached the limit.
I have uploaded my code to github in this link: https://github.com/christmo/karaf-pax-jdbc , you can give a pull request
if you find an error in this project.
For test this project you can do:
1. Download karaf 4.0.4 or apache-karaf-4.1.0-SNAPSHOT
2. Copy the file karaf-pax-jdbc/etc/org.ops4j.datasource-my-ds.cfg to ${karaf}/etc, this file have the mysql
configuration change with your mysql configuration data.
4. Start mysql database engine
3. Start karaf -> cd ${karaf}/bin/; ./karaf
4. Add the repo of this project with this karaf command: feature:repo-add mvn:pax/features/1.0-SNAPSHOT/xml/features
5. Install the feature created for this project: feature:install mysql-test
6. Execute the command for test this problem: mysql-connection, this command only execute "Select 1" in mysql
If you execute 9 times this command "mysql-connection", it will freeze the prompt of karaf and if you interrupt the
execution you can get this exception:
java.sql.SQLException: Cannot get a connection, general error at
org.apache.commons.dbcp2.PoolingDataSource.getConnection(PoolingDataSource.java:146)
at com.twim.OrmCommand.execute(OrmCommand.java:53) at
org.apache.karaf.shell.impl.action.command.ActionCommand.execute(ActionCommand.java:83)
at
org.apache.karaf.shell.impl.console.osgi.secured.SecuredCommand.execute(SecuredCommand.java:67)
at
org.apache.karaf.shell.impl.console.osgi.secured.SecuredCommand.execute(SecuredCommand.java:87)
at org.apache.felix.gogo.runtime.Closure.executeCmd(Closure.java:480)
at
org.apache.felix.gogo.runtime.Closure.executeStatement(Closure.java:406)
at org.apache.felix.gogo.runtime.Pipe.run(Pipe.java:108) at
org.apache.felix.gogo.runtime.Closure.execute(Closure.java:182) at
org.apache.felix.gogo.runtime.Closure.execute(Closure.java:119) at
org.apache.felix.gogo.runtime.CommandSessionImpl.execute(CommandSessionImpl.java:94)
at
org.apache.karaf.shell.impl.console.ConsoleSessionImpl.run(ConsoleSessionImpl.java:270)
at java.lang.Thread.run(Thread.java:745) Caused by:
java.lang.InterruptedException at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.java:2014)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2048)
at
org.apache.commons.pool2.impl.LinkedBlockingDeque.takeFirst(LinkedBlockingDeque.java:583)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:442)
at
org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at
org.apache.commons.dbcp2.PoolingDataSource.getConnection(PoolingDataSource.java:134)
... 12 more
The problem in your code is in the line System.out.println("--DS--: " + ds.getConnection());.
There you create a connection but never close it. So with every call you drain the pool.

OpenMrs sync module didn't start

I am a beginner in openmrs developping. I work in a project in with we have download the openmrs core master in github and deploy it in a server an in a pc. We want the server and the pc to synchronize their data; the server will be the father and the pc the child.
The problem is when I install the sync module, I have the following error:
Error while trying to start module
Error while running sql: INSERT INTO scheduler_task_config (name, description, schedulable_class, start_time, start_time_pattern, repeat_interval, start_on_startup, started, created_by, date_created, changed_by, date_changed, uuid) VALUES ('Cleanup Old Sync Records', 'This task deletes old rows in the sync_record and sync_server_record tables. The default settings work on all servers, but for advanced users, you can change the properties to suit your needs.', 'org.openmrs.module.sync.scheduler.CleanupSyncTablesTask', '2009-12-18 17:26:31', 'MM/dd/yyyy HH:mm:ss', '604800', '1', '0', '1', '2009-12-18 17:28:39', null, null, 'd3122955-00d7-454c-b17f-e3f87206c74b') . Message: Duplicate entry 'd3122955-00d7-454c-b17f-e3f87206c74b' for key 'scheduler_task_config_uuid_index'
So I have look for a solution and one of them said that it is a problem of version. The version of my openmrs is 1.12.0 and in the downloading sync module page, it is specify that the version required is 1.9.0. I have supposed that it is the smallest version, otherwise openmrs 1.12 could not be synchronised. One other solution told me to check the version in global_property table and I do it and the problem stay.
Please, anyone of you, can he know how could I start sync module in openmrs 1.12.0? Thanks in advance.
An openmrs developper said to me that to his knowledge, he did not yet see an openmrs project with use sync module that has a version greater than 1.9. Base on that, and on the fact that in the mostly modules we talk of versions 1.7, 1.8, 1.9, I decided to come back to an openmrs core version 1.9.x and at the first try it works. The sync module started perfectly.

Play TypeSafe Activator fails to start - IllegalArgumentException "Failed to download new template catalog properties"

Moving from play 2.2.x to latest activator last night. Downloaded minimal 1.2.10, extracted it in program file (x86)\typesafe... and put the directory into the system path variable. cloned my repository, and when i executed activator run it downloaded the required modules and my app is up and running. All great so far. run works!
Then I tried to create a new app, and activator fails, with the following trace:
Checking for a newer version of Activator (current version 1.2.10)...
... our current version 1.2.10 looks like the latest.
Found previous process id: 9632
FOUND REPO = activator-local # file:////C:/Program%20Files%20(x86)/Typesafe/activator-1.2.10-minimal/repository
Play server process ID is 9760
[info] play - Application started (Prod)
[info] play - Listening for HTTP on /127.0.0.1:8888
[info] a.e.s.Slf4jLogger - Slf4jLogger started
[WARN] [10/30/2014 10:47:13.972] [default-akka.actor.default-dispatcher-2] [ActorSystem(default)] Failed to download new template ca
talog properties: java.lang.IllegalArgumentException: requirement failed: Source file 'C:\Users\admin\.activator\1.2.10\templates\in
dex.db_6e0565f0c8826b17.tmp' is a directory.
[ERROR] [10/30/2014 10:47:13.972] [default-akka.actor.default-dispatcher-2] [akka://default/user/template-cache] Could not find a te
mplate catalog. (activator.templates.repository.RepositoryException: We don't have C:\Users\admin\.activator\1.2.10\templates\cache.
properties with an index hash in it, even though we should have downloaded one
activator.templates.repository.RepositoryException: We don't have C:\Users\admin\.activator\1.2.10\templates\cache.properties with a
n index hash in it, even though we should have downloaded one
at activator.cache.TemplateCacheActor.preStart(TemplateCacheActor.scala:184)
at akka.actor.Actor$class.aroundPreStart(Actor.scala:470)
at activator.cache.TemplateCacheActor.aroundPreStart(TemplateCacheActor.scala:25)
at akka.actor.ActorCell.create(ActorCell.scala:580)
at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456)
at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263)
at akka.dispatch.Mailbox.run(Mailbox.scala:219)
at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
I've taken a look at several similar issues on SO and elsewhere. I've deleted .activator directory and retried, I've tried this process from behind a proxy and not, as well as offline (surely offline should work!), but it consistently gives the above error. activator ui gives the same error. I'm stuck and any suggestions would be appreciated. (Edit. tried with full activator download, rather than minimal, and I get the same error.)
Look for reasons it might be impossible to create or access 'C:\Users\admin.activator\1.2.10\templates\in
dex.db_6e0565f0c8826b17.tmp' ... maybe a permissions issue?
The failed check is for "is a directory" but that also fails if it just doesn't exist or can't be accessed.

Getting 'java.sql.SQLException: com.mysql.jdbc.Driver' with grails run-app (when BuildConfig.groovy doesn't need to be recompiled)

I've upgraded my grails application from 1.3.9 to 2.2.3 and then to 2.3.3. I read the release and upgrade notes for 1.3.9->2.2.3 and then from 2.2.3->2.3.3
I am using OpenJDK 6, Jetty 6 and the plugin jetty 1.1, MySQL 5.5 and I have the connector library under lib
Now my issue is if I run grails clean and then grails run-app the application runs without any problems but if I stop it and run grails run-app again I get a gigantic error (see here: http://pastebin.com/36MpXhir)
I also found that changing something like adding a space somewhere in BuildConfig.groovy (anything that makes it be recompiled) makes the application run normally.
Looking at the stacktrace the first thing that puzzles me is
[02.12.13 16:13:59.919] [main] pool.ConnectionPool Unable to create initial connections of pool.
java.sql.SQLException: com.mysql.jdbc.Driver
at org.apache.tomcat.jdbc.pool.PooledConnection.connectUsingDriver(PooledConnection.java:254)
at org.apache.tomcat.jdbc.pool.PooledConnection.connect(PooledConnection.java:182)
at org.apache.tomcat.jdbc.pool.ConnectionPool.createConnection(ConnectionPool.java:701)
at org.apache.tomcat.jdbc.pool.ConnectionPool.borrowConnection(ConnectionPool.java:635)
at org.apache.tomcat.jdbc.pool.ConnectionPool.init(ConnectionPool.java:486)
at org.apache.tomcat.jdbc.pool.ConnectionPool.<init>(ConnectionPool.java:144)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.pCreatePool(DataSourceProxy.java:116)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.createPool(DataSourceProxy.java:103)
at org.apache.tomcat.jdbc.pool.DataSourceProxy.getConnection(DataSourceProxy.java:127)
at org.springframework.jdbc.datasource.LazyConnectionDataSourceProxy.afterPropertiesSet(LazyConnectionDataSourceProxy.java:162)
There are references to org.apache.tomcat even though I'm using jetty (and removed tomcat from BuildConfig.groovy).
Did anyone else encounter such a problem?
Don't put jar files in the lib directory if they're available in a public Maven repo. It's far better to download jars once and keep them in a local cache, and reuse them as needed.
The MySQL driver is used as the commented-out example in the generated BuildConfig.groovy - just un-comment it :) You might want to bump up the version to the latest, e.g.
dependencies {
runtime 'mysql:mysql-connector-java:5.1.27'
}
This is a good site for finding Maven artifacts: http://mvnrepository.com/artifact/mysql/mysql-connector-java
If you do have a jar that's not in a Maven repo (e.g. one with shared code at your company) then you can put it in the lib directory, but it's not auto-discovered. Run grails compile --refresh-dependencies to get it added to the classpath.
For me same error has occurred while running the Grails Application.Then I debug and view the code history of my code which was committed recently.
From that I found the issue that was:
Inside the controller file I send the instance with-out properly
Eg:
**def list=[personInstance.]---> error occurred.**
**render list as JSON**
Then I correct my mistake-->clean the app --> run the app
Now its working fine.

"File not found Exception" at runtime in Glassfish ESB 2.1

Good wishes of the day..!
In production we have Glassfish 2.1 server hosting ESB Applications in two instances each under two Linux Boxes. Functionality of the ESB app to takes client request and transform to destination, again receive the response and sent back to the client.
From past few days we are seeing "File not found Exception" in the logs throwing by WsdlQueryHelper of HTTP BC. We analyzed the logs and came to know that it is happening for only one instance (Instance 2 of Server 1), that to for few requests in that instance. We checked the service of that instance from SOAP tool and it is giving appropriate response, Understood that WsdlQueryHelper failed to process few requests at runtime. Below exception details for the same in logs,
*[#|2012-12-13T18:29:24.526+1100|FINE|sun-appserver2.1|com.sun.jbi.httpsoapbc.WsdlQueryHelper|_ThreadID=319;_ThreadName=httpWorkerThread-7092-0;ClassName=com.sun.jbi.httpsoapbc.WsdlQueryHelper;MethodName=;_RequestID=6fdd0535-24d4-4878-8c98-b48e2dea39eb;|init
query helper failed. javax.wsdl.WSDLException: WSDLException (at
/definitions/types/xsd:schema): faultCode=OTHER_ERROR: An error
occurred trying to resolve schema referenced at 'RouterSchema_v4.xsd',
relative to ''.: java.io.FileNotFoundException: This file was not
found:
file:/home/glassfish/GlassFishESBv21/glassfish/nodeagents/GLASSFISH-001-NA/GLASSFISH-001-instB/RouterSchema_v4.xsd
at com.ibm.wsdl.xml.WSDLReaderImpl.parseSchema(WSDLReaderImpl.java:918)
at com.ibm.wsdl.xml.WSDLReaderImpl.parseSchema(WSDLReaderImpl.java:678)
at com.ibm.wsdl.xml.WSDLReaderImpl.parseTypes(WSDLReaderImpl.java:639)
at com.ibm.wsdl.xml.WSDLReaderImpl.parseDefinitions(WSDLReaderImpl.java:339)
at com.ibm.wsdl.xml.WSDLReaderImpl.readWSDL(WSDLReaderImpl.java:2324)
at com.ibm.wsdl.xml.WSDLReaderImpl.readWSDL(WSDLReaderImpl.java:2288)
at com.ibm.wsdl.xml.WSDLReaderImpl.readWSDL(WSDLReaderImpl.java:2341)
at com.ibm.wsdl.xml.WSDLReaderImpl.readWSDL(WSDLReaderImpl.java:2362)
at com.sun.jbi.httpsoapbc.WsdlQueryHelper.(WsdlQueryHelper.java:105)
at com.sun.jbi.httpsoapbc.embedded.JAXWSGrizzlyRequestProcessor.processSynchronousQueryResource(JAXWSGrizzlyRequestProcessor.java:293)
at com.sun.jbi.httpsoapbc.embedded.JAXWSGrizzlyRequestProcessor.service(JAXWSGrizzlyRequestProcessor.java:217)
at com.sun.enterprise.web.connector.grizzly.DefaultProcessorTask.invokeAdapter(DefaultProcessorTask.java:647)
at com.sun.jbi.httpsoapbc.embedded.JBIGrizzlyAsyncFilter.doFilter(JBIGrizzlyAsyncFilter.java:95)
at com.sun.enterprise.web.connector.grizzly.async.DefaultAsyncExecutor.invokeFilters(DefaultAsyncExecutor.java:175)
at com.sun.enterprise.web.connector.grizzly.async.DefaultAsyncExecutor.interrupt(DefaultAsyncExecutor.java:153)
at com.sun.enterprise.web.connector.grizzly.async.AsyncProcessorTask.doTask(AsyncProcessorTask.java:92)
at com.sun.enterprise.web.connector.grizzly.TaskBase.run(TaskBase.java:265)
at com.sun.enterprise.web.connector.grizzly.WorkerThreadImpl.run(WorkerThreadImpl.java:116)
Caused by: java.io.FileNotFoundException: This file was not found:
file:/home/glassfish/GlassFishESBv21/glassfish/nodeagents/GFESB_ASPAC_001-NA/GFESB_ASPAC_001-instB/RouterSchema_v4.xsd
at com.ibm.wsdl.util.StringUtils.getContentAsInputStream(StringUtils.java:199)
at com.ibm.wsdl.xml.WSDLReaderImpl.parseSchema(WSDLReaderImpl.java:840)
... 17 more |#]*
Could you please help us to resolve the issue.
Regards,
Ram