Redis exception with transactions with Jedis client - exception

In order to avoid duplicates in my redis channel I'm checking if the message is already there by keeping an index in Redis set. Following is my implementation. However, it is giving an exception.
redis.clients.jedis.exceptions.JedisDataException: Please close pipeline or multi block before calling this method.
at redis.clients.jedis.Response.get(Response.java:23)
Here is the implementation.
Jedis jedis = pool.getResource();
String id = message.getId();
Transaction transaction = jedis.multi();
redis.clients.jedis.Response<java.lang.Boolean> response = transaction.sismember(ID_SET_REDIS_KEY, id);
if (response != null && !response.get().booleanValue()) {
//add it to the
transaction.sadd(ID_SET_REDIS_KEY, id);
transaction.publish(redisChannelName, message);
}
transaction.exec();
pool.returnResource(jedis);
I need to do the get inside the transaction because there are multiple publishers who may publish the exact same message.

You can't have the result of your get before you end the transaction.
If you are using Redis > 2.6.X, what you can do is use a Lua Script to create a function with you logic. See Redis Lua
This is exactly what I did to guarantee concurrency in my project.
Edit: Including a more complete example
You should create something like a PUBLISHNX script (not tested):
local shouldPublish = redis.call('SISMEMBER', KEYS[1], ARGV[1])
if shouldPublish == 0
redis.call('SADD', KEYS[1], ARGV[1])
redis.call('PUBLISH', ARGV[2], ARGV[3])
end
And you pass all the arguments necessary, channel, messageId, message, controlKey.
PS. Wei Li is right, you could accomplish the same result using WATCH and a loop for retrying in case of concurrency, but I still prefer using a Lua script.

Based on #Axexandre's comment above I used the following piece of code to do perform the operation.
import redis.clients.jedis.Jedis;
public class RedisLuaDemo {
public static void main(String args[])
{
Jedis jedis = new Jedis("localhost");
jedis.sadd("a", "b");
int numberOfKeys = 1 //we are using only one Redis set 'setvar'
jedis.eval("if redis.call('sismember', KEYS[1], ARGV[1]) == 1 then return ARGV[2] else redis.call('sadd', KEYS[1], ARGV[1]); redis.call('publish', 'channel.mychannel', ARGV[2]) end", numberOfKeys, "setvar", "joe", "message from joe!");
}
}
Here is some more information about the script. It took some time to understand the syntax.
if redis.call('sismember', KEYS[1], ARGV[1]) == 1 eqavalent to SISMEMBER setvar joe
redis.call('sadd', KEYS[1], ARGV[1]);
For some reason if I don't have this jedis.sadd("a", "b"); line I get an exception (see below).
Exception in thread "main" java.lang.NullPointerException
at redis.clients.jedis.Connection.setTimeoutInfinite(Connection.java:41)
at redis.clients.jedis.Jedis.eval(Jedis.java:2763)
at redis.RedisLuaDemo.main(RedisLuaDemo.java:13)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)

Related

How to resolve NullPointerException when updating database

I'm a jooq neophyte. I'm trying to update a column called account status in my users table from UNLOCKED to LOCKED. I'm using the update query using Jooq's dslContext.
I'm calling this updateAccountStatus method in my Controller. But this method is giving me a NullPointerException.
public boolean updateAccountStatus(String userhandle,UsersAccStatus usersAccStatus) {
try {
dslContext
.update(USERS)
.set(USERS.ACC_STATUS,UsersAccStatus.LOCKED)
.where(USERS.USER_HANDLE.equal(userHandle))
.execute();
}
catch(Exception e) {
// code
}
}
I expect the output of this method to update my account_status enum in my database.
Error message :-
java.lang.NullPointerException: null
at com.core.admin.controller.AdminAccountStatus.execute(AdminAccountStatus.java:84) ~[classes/:na]
Any help would be appreciated.
1) In following line, you are directly using the class UsersAccStatus, if you have declared LOCKED as static then it is fine else you have to use the object of class (UsersAccStatus) which is usersAccStatus (in small letters)
2) Modify the following code
FROM:
.set(USERS.ACC_STATUS,UsersAccStatus.LOCKED)
TO:
.set(USERS.ACC_STATUS,usersAccStatus!=null ? null : usersAccStatus.LOCKED)
Let me know if this worked for you.
Regards,
Anurag

How to handle Modbus exception 0x5

I'm writing a Modbus client program using Qt5 and the QModbusTcpClient class. Here the code I'm using for open a connection and read something:
QModbusClient *_modbus;
bool ModbusMaster::open(QString host, int port)
{
// Disconnect and delete any existing instance
if (_modbus)
{
_modbus->disconnectDevice();
delete _modbus;
}
// Create and open the new connection
_modbus = new QModbusTcpClient(this);
_modbus->setConnectionParameter(QModbusDevice::NetworkPortParameter, port);
_modbus->setConnectionParameter(QModbusDevice::NetworkAddressParameter, host);
_modbus->setTimeout(250);
_modbus->setNumberOfRetries(1);
return _modbus->connectDevice();
}
bool ModbusMaster::read(QModbusDataUnit::RegisterType type, int startAddress, quint16 count)
{
if (!_modbus) return false;
if (_modbus->state() != QModbusDevice::ConnectedState) return false;
QModbusDataUnit req(type, startAddress, count);
if (auto *reply = _modbus->sendReadRequest(req, _id))
{
if (!reply->isFinished()) connect(reply, &QModbusReply::finished, this, &ModbusMaster::readReady);
else delete reply;
return true;
}
return false;
}
void ModbusMaster::readReady()
{
auto reply = qobject_cast<QModbusReply *>(sender());
if (!reply) return;
reply->deleteLater();
if (reply->error() == QModbusDevice::NoError)
{
// do something
}
else if (reply->error() == QModbusDevice::ProtocolError)
{
qDebug() << QString("Read response error: %1 (Mobus exception: 0x%2)").
arg(reply->errorString()).
arg(reply->rawResult().exceptionCode(), -1, 16);
} else {
qDebug() << QString("Read response error: %1 (code: 0x%2)").
arg(reply->errorString()).
arg(reply->error(), -1, 16);
}
}
Sometimes when I read something from the remote device it happens the device returns the exception 0x5. Reading the official Modbus documentation, at page 48 I read:
Specialized use in conjunction with programming
commands.
The server has accepted the request and is
processing it, but a long duration of time will be
required to do so. This response is returned to
prevent a timeout error from occurring in the
client. The client can next issue a Poll Program
Complete message to determine if processing is
completed.
[bold is mine]
I cannot find a description of this "Poll Program Complete message" that seems I must use to handle the exception 0x5.
Did I search wrong? Is there another way to handle this exception?
It depends on type of an equipment, you are working with. You just have to follow the logic, described in equipment mans for this particular exception.
In general there is no special 'Program Complete' event. That means, as it is written for 0x5 - "Specialized use in conjunction with programming commands.". So you just have to poll (read) some flag from your device meaning the internal process in device, which caused this exception, is complete.
Just as an example, I've met with such exception in relay protection device, which issued it when it has been in a process of writing a disturbance record. I had just to check for that record readiness in some time.

How to recover from database errors in Grails within a transaction

In short, what I am trying solve is how to recover from certain database errors in a Grails application using Hibernate and continue on with the transaction skipping over the failed row updates that are part of a batch of changes.
The application uses Grails 2.3.11 but I have also tried with version 1.3.8 with similar failed results.
Basically there is a Grails service class that iterates over a list of imported records and attempts to update associated master records appropriately. In certain situations exceptions might occur during the domain.save(flush:true) call e.g. org.hibernate.exception.DataException thrown due to issues like (Data truncation: Data too long for column ...).
At this point I have tried:
Disabling transactions
Using domainObj.withTransaction() for each individual record
Trying various #Transactional annotations
Calling domain.clearErrors() and domain.discard() after catching the exception
Tried using a nested service with Transactional annotation with noRollbackFor as shown below
A number of other approaches but nothing I've tried has worked
Example code:
#Transactional
class UpdateService {
public updateBatch(Integer batchId) {
...
list.each { record ->
record.value = 123
try {
nestedService.saveDomain()
} catch (e) {
record.clearErrors()
record.discard()
}
}
batch.status = "POSTED"
batch.save()
}
}
#Transactional
class NestedService {
#Transactional(propagation = Propagation.REQUIRED, noRollbackFor = RuntimeException.class)
public void saveDomain(domainObj) throws RuntimeException {
if (domainObj.validate() && domainObj.save(flush:true) {
log.info "domain $domain was saved"
}
}
}
Once an error occurs I can't seem to clear out the hibernate session. On each subsequent record being updated I receive the error:
org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction
where it indicates the original failed domain id.
Revision:
Vahid, Thanks for the suggestions. I have tried that. I realized one issue is that I am passing objects across transactional boundaries. So I experimented with the NestedService class do something along the lines of:
#Transactional(propagation = Propagation.REQUIRE_NEW)
public void saveDomain(domainObj) {
def newObj = new Domain.get(domainObj.id)
newObj.properties = domainObj.properties
if (newObj.validate() && newObj.save(force:true) ) { ... }
I expected that to work but the original domainObj still fails even though I'm not calling the save on it. Very strange...
A simple approach would be to loop and then use validate(). If it does fail, then just store the id of the failed entity and proceed.
if(!domainObject.validate()){
// store Id for trying it again later ?
}else{
// Save
}

Send a specific response, or at least a specific HTTP status code, using an exception

In Django, there are a couple of exceptions that are designed to be intercepted by the framework and turned into specific HTTP response codes, such as 404 Not Found and 403 Forbidden.
This is especially useful for request validation, because it allows you to factor out common validation logic into utility functions and cleanup your controller actions.
Whenever the utility functions decide that the current request must be aborted with a specific HTTP error code, they can do so by throwing the relevant exception, without any support code in the controller action, in the form of a return statement or a try/catch.
For example, given a tree of nested REST resources:
static mappings = {
"/authors" (resources: "author") {
"/sagas" (resources: "saga") {
"/books" (resources: "book") {
}
}
}
Then the URL pattern for the Book resource is /authors/$authorId/sagas/$sagaId/books/$id, which means that any of the show(), delete(), or update() actions in BookController have this signature and must include some boilerplate validation logic:
def actionName(int authorId, int sagaId, Book book) {
// -- common validation logic ----------
// fetch parent objects
def author = Author.get(authorId)
def saga = Saga.get(sagaId)
// check that they exists
if (author == null || saga == null || book == null) {
return render(status: NOT_FOUND)
}
// check consistency
if (book.author != author || book.saga != saga || saga.author != author) {
return render(status: BAD_REQUEST)
}
// -- end of commond code --------------
...
}
What is the Grails way of factoring this out into a common method, while still allowing it to terminate request processing whenever an exceptional condition occurs?
I would think the best way is a NotFoundException, ForbiddenException, BadRequestException, and so on, or maybe a generic exception that accepts a HTTP status code. Is there anything like it in Grails? If not, where is the best place to add it? A filter?
Edit: I see now that the standard method is to add an error controller with a matching URL pattern, such as:
"500" (controller: "error")
The problem with this is that Grails will still log full stacktraces for all exceptions, including those that are not programming errors. This spams log files with all sorts of useless tracebacks.
Is there a solution?
You catch the exception in the beforeInterceptor closure of your controller. I resolved this same problem by examining the exception thrown and then acting accordingly. For example:
class BaseController {
/**
* Define DRA exception handlers. This prevents the default Grails
* behavior of returning an HTTP 500 error for every exception.
*
* Instead the exceptions are intercepted and modified according to
* the exception that was thrown. These exceptions are not logged
* whereas application exceptions are.
*/
def beforeInterceptor = {
request.exceptionHandler = { exception ->
def cause = exception.cause
def exceptionBody = [:]
if(cause.class == BadRequestException) {
response.setStatus(HttpStatus.BAD_REQUEST.value()) // HTTP 400 BAD REQUEST
exceptionBody.httpStatus = HttpStatus.BAD_REQUEST.value()
exceptionBody.error = cause.message
}
// render the exception body, the status code is set above.
render exceptionBody as JSON
return true
}
}
}
In order to get this to work you will have to create an ErrorController or something where all server errors are processed and rendered. For example:
class ErrorController {
def serverError() {
def handler = request.exceptionHandler
if(handler) {
request.exceptionHandler = null
if(handler.call(request.exception)) {
return
}
}
}
I have tested this an it does work. I copied the code from a running project that I have been working on. You can build out the if statement in the beforeInterceptor to catch any type of Exception you wish.

How to know exception occurred within grails transaction?

I have a service method which does some operation inside a transaction.
public User method1() {
// some code...
Vehicle.withTransaction { status ->
// some collection loop
// some other delete
vehicle.delete(failOnError:true)
}
if (checkSomething outside transaction) {
return throw some user defined exception
}
return user
}
If there is a runtime exception we dont have to catch that exception and the transaction will be rolled back automatically. But how to determine that transaction rolled back due to some exception and I want to throw some user friendly error message. delete() call also wont return anything.
If I add try/catch block inside the transaction by catching the Exception (super class) it is not getting into that exception block. But i was expecting it to go into that block and throw user friendly exception.
EDIT 1: Is it a good idea to add try/catch arround withTransaction
Any idea how to solver this?? Thanks in advance.
If I understand you question correctly, you want to know how to catch an exception, determine what the exception is, and return a message to the user. There are a few ways to do this. I will show you how I do it.
Before I get to the code there are a few things I might suggest. First, you don't need to explicitly declare the transaction in a service (I'm using v2.2.5). Services are transactional by default (not a big deal).
Second, the transaction will automatically roll back if any exception occurs while executing the service method.
Third, I would recommend removing failOnError:true from save() (I don't think it works on delete()... I may be wrong?). I find it is easier to run validate() or save() in the service then return the model instance to the controller where the objects errors can be used in a flash message.
The following is a sample of how I like to handle exceptions and saves using a service method and try/catch in the controller:
class FooService {
def saveFoo(Foo fooInstance) {
return fooInstance.save()
}
def anotherSaveFoo(Foo fooInstance) {
if(fooInstance.validate()){
fooInstance.save()
}else{
do something else or
throw new CustomException()
}
return fooInstance
}
}
class FooController {
def save = {
def newFoo = new Foo(params)
try{
returnedFoo = fooService.saveFoo(newFoo)
}catch(CustomException | Exception e){
flash.warning = [message(code: 'foo.validation.error.message',
args: [org.apache.commons.lang.exception.ExceptionUtils.getRootCauseMessage(e)],
default: "The foo changes did not pass validation.<br/>{0}")]
redirect('to where ever you need to go')
return
}
if(returnedFoo.hasErrors()){
def fooErrors = returnedFoo.errors.getAllErrors()
flash.warning = [message(code: 'foo.validation.error.message',
args: [fooErrors],
default: "The foo changes did not pass validation.<br/>${fooErrors}")]
redirect('to where ever you need to go')
return
}else {
flash.success = [message(code: 'foo.saved.successfully.message',
default: "The foo was saved successfully")]
redirect('to where ever you need to go')
}
}
}
Hope this helps, or gets some other input from more experienced Grails developers.
Here are a few other ways I've found to get exception info to pass along to your user:
request.exception.cause
request.exception.cause.message
response.status
A few links to other relevant questions that may help:
Exception handling in Grails controllers
Exception handling in Grails controllers with ExceptionMapper in Grails 2.2.4 best practice
https://commons.apache.org/proper/commons-lang/javadocs/api-2.6/org/apache/commons/lang/exception/ExceptionUtils.html