How to deal with errors and exceptions during dependency injection - exception

As far as I have understood, dependency injection separates the application wiring logic from the business logic. Additionally, I try to adhere to the law of Demeter by only injecting direct collaborators.
If I understand this article correctly, proper dependency injection means that collaborators should be fully initialized when they are injected, unless lazy instantiation is required. This would mean (and is actually mentioned in the article) that objects like database connections and file streams should be up and ready at injection time.
However, opening files and connections could result in an exception, which should be handled at some point. What is the best way to go about this?
I could handle the exception at 'wire time', like in the following snippet:
class Injector:
def inject_MainHelper(self, args):
return MainHelper(self.inject_Original(args))
def inject_Original(self, args):
return open(args[1], 'rb')
class MainHelper:
def __init__(self, original):
self.original = original
def run(self):
# Do stuff with the stream
if __name__ == '__main__':
injector = Injector()
try:
helper = injector.inject_MainHelper(sys.argv)
except Exception:
print "FAILED!"
else:
helper.run()
This solution, however, starts to mix business logic with wiring logic.
Another solution is using a provider:
class FileProvider:
def __init__(self, filename, load_func, mode):
self._load = load_func
self._filename = filename
self._mode = mode
def get(self):
return self._load(self._filename, self._mode)
class Injector:
def inject_MainHelper(self, args):
return MainHelper(self.inject_Original(args))
def inject_Original(self, args):
return FileProvider(args[1], open, 'rb')
class MainHelper:
def __init__(self, provider):
self._provider = provider
def run(self):
try:
original = self._provider.get()
except Exception:
print "FAILED!"
finally:
# Do stuff with the stream
if __name__ == '__main__':
injector = Injector()
helper = injector.inject_MainHelper(sys.argv)
helper.run()
The drawback here is the added complexity of a provider and a violation of the law of Demeter.
What is the best way to deal with exceptions like this when using a dependency-injection framework as discussed in the article?
SOLUTION, based on the discussion with djna
First, as djna correctly points out, there is no actual mixing of business and wiring logic in my first solution. The wiring is happening in its own, separate class, isolated from other logic.
Secondly, there is the case of scopes. Instead of one, there are two smaller scopes:
The scope where the file is not verified yet. Here, the injection engine cannot assume anything about the file's state yet and cannot build objects that depend on it.
The scope where the file is successfully opened and verified. Here, the injection engine can create objects based on the extracted contents of the file, without the worry of blowing up on file errors.
After entering the first scope and obtaining enough information on opening and validating a file, the business logic tries to actually validate and open the file (harvesting the fruit, as djna puts it). Here, exceptions can be handled accordingly. When it is certain the file is loaded and parsed correctly, the application can enter the second scope.
Thirdly, not really related to the core problem, but still an issue: the first solution embeds business logic in the main loop, instead of the MainHelper. This makes testing harder.
class FileProvider:
def __init__(self, filename, load_func):
self._load = load_func
self._filename = filename
def load(self, mode):
return self._load(self._filename, mode)
class Injector:
def inject_MainHelper(self, args):
return MainHelper(self.inject_Original(args))
def inject_Original(self, args):
return FileProvider(args[1], open)
def inject_StreamEditor(self, stream):
return StreamEditor(stream)
class MainHelper:
def __init__(self, provider):
self._provider = provider
def run(self):
# In first scope
try:
original = self._provider.load('rb')
except Exception:
print "FAILED!"
return
# Entering second scope
editor = Injector().inject_StreamEditor(original)
editor.do_work()
if __name__ == '__main__':
injector = Injector()
helper = injector.inject_MainHelper(sys.argv)
helper.run()
Note that I have cut some corners in the last snippet. Refer to the mentioned article for more information on entering scopes.

I've had discussion about this in the contect of Java EE, EJB 3 and resources.
My understanding is that we need to distinguish between injection of the Reference to a resource and the actual use of a resource.
Take the example of a database connection we have some pseudo-code
InjectedConnectionPool icp;
public void doWork(Stuff someData) throws Exception{
Connection c = icp.getConnection().
c.writeToDb(someData);
c.close(); // return to pool
}
As I understand it:
1). That the injected resource can't be the connection itself, rather it must be a connection pool. We grab connections for a short duration and return them.
2). That any Db connection may be invalidated at any time by a failure in the DB or network. So the connection pooling resource must be able to deal with throwing away bad connections and getting new ones.
3). A failure of injection means that the component will not be started. This could happen if, for example, the injection is actually a JNDI lookup. If there's no JNDI entry we can't find the connection pool definition, can't create the pool and so can't start the component. This is not the same as actually opening a connection to the DB ...
4). ... at the time of initialisation we don't actually need to open any connections, a failure to do so just gives us an empty pool - ie. exactly the same state as if we had been running for a while and the DB went away, the pool would/could/should throw away the stale connections.
This model seems to nicely define a set of responsibilities that Demeter might accept. Injection has respobilitiy to prepare the ground, make sure that when the code needs to do something it can. The code has the responsibility to harvest the fruit, try to use the prepared material and cope with actual resource failures and opposed to failures to find out about resources.

Related

override SQlAlchemy context manager for all tests

I use FastApi with SqlAlchemy as context manager
#contextmanager
def get_session(): # need to patch all over tests files
session_ = SessionLocal()
try:
yield session_
except Exception as e:
session_.rollback()
router.py
#router.get('/get_users'):
def get_users(): # No dependencies
with get_session() as session:
users = session.query(Users).all()
return users
I need to override get_session during all my tests (I use pytest)
I could do it with #patch and patch all test. But it's not most effective way because i need to use decorator to patch in each test file, correct specify full path.
I wonder is there quick way to do it in one place or maybe use fixture?
You could try the approach in this answer to a similar question: define a pytest fixture with arguments scope="session", autouse=True, patching the context manager. If you need, you can also provide a new callable:
from contextlib import contextmanager
from unittest.mock import patch
import pytest
#pytest.fixture(scope="session", autouse=True, new_callable=fake_session)
def default_session_fixture():
with patch("your_filename.get_session"):
yield
#contextmanager
def fake_session():
yield ... # place your replacement session here
As a side note, I would highly recommend using FastAPI's dependency injection for handling your SQLAlchemy session, especially since it has built in support for exactly this kind of patching.

Accessing regmap RegFields

I am trying to find a clean way to access the regmap that is used with *RegisterNode for creating documentation and testing files. The TLRegisterNode has methods for generating the json through some Annotations. These are done in the regmap method by adding them to the ElaborationArtefacts object. Other protocols don't seem to have these annotations.
Is there anyway to iterate over the "regmap" Register Fields post elaboration or during?
I cannot just access the regmap as it's not really a val/var since it's a method. I can't quite figure out where this information is being stored. I don't really believe it's actually "storing" any information as much as it is simply creating the hardware to attach the specified logic to the RegisterNode based logic.
The JSON output is actually fine for me as I could just write a post processing script to convert JSON to my required formats, but I'm wondering if I can access this information OR if I could add a custom function call at the end. I cannot extend the case class *RegisterNode, but I'm not sure if it's possible to add custom functions to run at the end of the regmap method.
Here is something I threw together quickly:
//in *RegisterRouter.scala
def customregmap(customFunc: (RegField.Map*) => Unit, mapping: RegField.Map*) = {
regmap(mapping:_*)
customFunc(mapping:_*)
}
def regmap(mapping: RegField.Map*) = {
//normal stuff
}
A user could then create a custom function to run and pass it to the regmap or to the RegisterRouter
def myFunc(mapping: RegField.Map*): Unit = {
println("I'm doing my custom function for regmap!")
}
// ...
node.customregmap(myFunc,
0x0 -> coreControlRegFields,
0x4 -> fdControlRegFields,
0x8 -> fdControl2RegFields,
)
This is just a quick example I have. I believe what would be better, if something like this was possible, would be to have a Seq of functions that could be added to the RegisterNode that are ran at the end of the regmap method, similar to how TLRegisterNode currently works. So a user could add an arbitrary number and you still use the regmap call.
Background (not directly part of question):
I have a unified register script that I have built over the years in which I describe the registers for a particular IP. It works very similar to the RegField/node.regmap, except it obviously doesn't know about diplomacy and the like. It will generate the Verilog, but also a variety of files for DV (basic `defines for simple verilog simulations and more complex uvm_reg_block defines also with the ability to describe multiple of the IPs for a subsystem all the way up to an SoC level). It will also print out C Header files for SW and Sphinx reStructuredText for documentation.
Diplomacy actually solves one of the main issues I've been dealing with so I'm obviously trying to push most of my newer designs to Chisel/Diplo.
I ended up solving this by creating my own RegisterNode which is the same as the rocketchip RegisterNodes except that I use a different Elaboration Artifact to grab the info and store it for later.

SQLAlchemy session within Huey periodic task

I'm integrating Huey with a simple pyramid app. I'm not using a global SQLAlchemy session in the app (I'm making use of the latest alchemy scaffold). However, there seems to be no other straightforward way to provide a session to periodic tasks.
from huey import RedisHuey
huey = RedisHuey(password=os.environ.get('REDIS_PASSWORD', ''))
DBSession = scoped_session(sessionmaker())
#huey.periodic_task(crontab(minute='*/1'))
def notify_not_confirmed_assignments():
# TODO: Use a non-global DB session
assignments = DBSession.query(Assignment).filter_by(date=next_date).all()
Does Huey offer hooks to close the DB connection on task completion? What is the best way to provide a thread-safe connection to these tasks?
Thanks in advance!
You can build session object with a factory in the tasks:
factory = sessionmaker()
factory.configure(bind=engine)
session = factory()
No need to use scoped session, just initialize the engine and pass it to factory.
scoped_session provides you with a contextual/thread-local session (i.e. it corresponds to an individual DB connection in each thread, and it's also possible to configure a custom scope when you need a separate session per something which is not a thread.
So, basically, all you need to do is to have a properly-configured pseudo-global variable (similar to what you have now) and make sure you call DBSession.begin() at the start of the task and DBSession.commit() at the end - doing that manually is probably a chore but it can easily be abstracted into a context manager
def my_task():
with magically_start_session() as session:
session.query(...)
or into a decorator:
#huey.periodic_task(crontab(minute='*/1'))
#start_session
def notify_not_confirmed_assignments(session):
# TODO: Use a non-global DB session
assignments = session.query(...)

Grails 2.4.4: How to reliably rollback in a complex service method

Consider the following service (transactional by default). A player must always have one account. A player without at least one corresponding account is an error state.
class playerService {
def createPlayer() {
Player p new Player(name: "Stephen King")
if (!p.save()) {
return [code: -1, errors:p.errors]
}
Account a = new Account(type: "cash")
if (!a.save()) {
// rollback p !
return [code: -2, errors:a.errors]
}
// commit only now!
return [code: 0, player:p]
}
}
I have seen this pattern by experienced grails developers, and when I tell them that if creation of the account of the player fails for any reason, it wont rollback the player, and will leave the DB in an invalid state, they look at me like I am mad because grails handles rolling back the player because services are transactional right?
So then, being a SQL guy, I look for a way to call rollback in grails. There isn't one. According to various posts, there are only 2 ways to force grails to rollback in a service:
throw an unchecked exception. You know what this is right?
don't use service methods or transactional annotations, use this construct:
.
DomainObject.withTransaction {status ->
//stuff
if (someError) {
status.setRollbackOnly()
}
}
1. throw an unchecked exception
1.1 So we must throw runtime exceptions to rollback. This is ok for me (I like exceptions), but this wont gel with the grails developers we have who view exceptions as a throwback to Java and is uncool. It also means we have to change the whole way the app currently uses its service layer.
1.2 If an exception is thrown, you lose the p.errors - you lose the validation detail.
1.3 Our new grails devs don't know the difference between an unchecked and an checked exception, and don't know how to tell the difference. This is really dangerous.
1.4. use .save(failOnError: true)
I am a big fan of using this, but its not appropriate everywhere. Sometimes you need to check the reason before going further, not throw an exception. Are the exceptions it can generate always checked, always unchecked, or either? I.e. will failOnError AWLAYS rollback, no matter what the cause? No one I have asked knows the answer to this, which is disturbing, they are using blind faith to avoid corrupted/inconsistent DBs.
1.5 What happens if a controller calls service A, which calls Service B, then service C. Service A must catch any exception and return a nicely formatted return value to the controller. If Service C throws an exception, which is caught by Service A, will service Bs transactions be rolled back? This is critical to know to be able to construct a working application.
UPDATE 1:
Having done some tests, it appears that any runtime exception, even if thrown and caught in some unrelated child calls, will cause everything in the parent to rollback. However, it is not easy to know in the parent session that this rollback has happened - you need to make sure that if you catch any exception, you either rethrow, or pass some notice back to the caller to show that it has failed in such a way that everything else will be rolled back.
2. withTransaction
2.1 This seems a bazaar construct. How do I call this, and what do I pass in for the "status" parameter? What is "setRollbackOnly" exactly. Why is it not just called "rollback". What is the "Only" part? It is tied to a domain object, when your method may want to do update several different domain objects.
2.2 Where are you supposed to put this code? In with the DomainObject class? In the source folder (i.e. not in a service or controller?)? Directly in the controller? (we don't want to duplicate business logic in the controllers)
3. Ideal situation.
3.1 The general case is we want every thing we do in a service method to roll back if anything in that service method cant be saved for any reason, or throws any exception for any reason (checked or unchecked).
3.2 Ideally I would like service methods to "always rollback, unless I explicitly call commit", which is the safest strategy , but this is not possible I believe.
The question is how do I achieve the ideal situation?
Will calling save(failOnError:true) ALWAYS rollback everything, no matter what the reason for failing? This is not perfect, as it is not easy for the caller to know which domain object save caused the issue.
Or do people define lots of exception classes which subclass runtimeException, then explicit catch each of them in the controller to create the appropriate response? This is the old Java way, and our groovy devs pooh pooh this approach due to the amount of boiler plate code we will have to write.
What methods do people use to achieve this?
I wouldn't call myself an expert, and this question is over a year old, but I can answer some of these questions, if only for future searchers. I'm just now refactoring some controllers to use services in order to take advantage of transactions.
I have seen this pattern by experienced grails developers, and when I tell them that if creation of the account of the player fails for any reason, it wont rollback the player, and will leave the DB in an invalid state, they look at me like I am mad because grails handles rolling back the player because services are transactional right?
I'm not seeing in the documentation where it explicitly states that returning from a service method does not rollback the transaction, but I can't imagine that this would be a very sane behavior. Still, testing is an easy way to prove yourself.
1.2 If an exception is thrown, you lose the p.errors - you lose the validation detail.
Since you're the one throwing the exception, you can throw the errors along with it. For instance:
// in service
if (!email.save()) {
throw new ValidationException("Couldn't save email ${params.id}", email.errors)
}
When you catch the exception, you reload the instance (because throwing an exception clears the session), put the errors back into the instance, and then pass that to the view as usual:
// in controller
} catch (ValidationException e) {
def email = Email.read(id)
email.errors = e.errors
render view: "edit", model: [emailInstance: email]
}
This is discussed under the heading "Validation Errors and Rollback", down the page from http://grails.github.io/grails-doc/2.4.4/guide/single.html#transactionsRollbackAndTheSession.
1.4. use .save(failOnError: true) I am a big fan of using this, but its not appropriate everywhere. Sometimes you need to check the reason before going further, not throw an exception. Are the exceptions it can generate always checked, always unchecked, or either? I.e. will failOnError AWLAYS rollback, no matter what the cause? No one I have asked knows the answer to this, which is disturbing, they are using blind faith to avoid corrupted/inconsistent DBs.
failOnError will cause save() to throw a ValidationException, so yes, if you're in a transaction and aren't checking that exception, the transaction will be rolled back.
Generally speaking, it seems to be un-"Grailsy" to use failOnError a lot, probably for the reasons you listed (e.g., lack of control). Instead, you check whether save() failed (if (!save()) ...), and take action based on that.
withTransaction
I'm not sure the point of this, because SpringSource really encourages the use of services for everything. I personally don't like it, either.
If you want to make a particular service non-transactional, and then make one method of it transactional, you can just annotate that one method with #Transactional (unless your developers also dislike annotations because they're too "Java" ;) ).
Note! As soon as you mark a single method with #Transactional, the overall service will become non-transactional.
3.1 The general case is we want every thing we do in a service method to roll back if anything in that service method cant be saved for any reason, or throws any exception for any reason (checked or unchecked).
I feel like checked exceptions are generally considered not "Groovy" (which also makes them not Grails-y). Not sure about the reason for that.
However, it looks like you can tell your service to rollback on your checked exceptions, by listing them in the rollbackFor option to #Transactional.
Or do people define lots of exception classes which subclass runtimeException, then explicit catch each of them in the controller to create the appropriate response? This is the old Java way, and our groovy devs pooh pooh this approach due to the amount of boiler plate code we will have to write.
The nice thing about Groovy is that you can write your boiler plate once and then call it repeatedly. A pattern I've seen a lot, and am currently using, is something like this:
private void validate(Long id, Closure closure) {
try {
closure()
} catch (ValidationException e) {
def email = Email.read(id)
email.errors = e.errors
render view: "edit", model: [emailInstance: email]
} catch (OtherException e) {
def email = Email.read(id)
flash.error = "${e.message}: ${e.reasons}"
render view: "show", model: [emailInstance: email]
} catch (Throwable t) {
flash.error = "Unexpected error $t: ${t.message}"
redirect action: "list"
}
}
And then call it in each controller action like so:
def update(Long id, Long version) {
withInstance(id, version) { Email emailInstance ->
validate(emailInstance.id) {
emailService.update(emailInstance, params)
flash.message = "Email $id updated at ${new Date()}."
redirect action: "show", id: emailInstance.id
}
}
}
(withInstance is another similar method that DRYs up the check for existence and optimistic locking.)
This approach has downsides. You get the same set of redirects in every action; you probably want to write one set of methods for each controller; and it seems kind of silly to pass a closure into a method and expect the method to know what exceptions the closure will throw. But hey, programming's all about tradeoffs, right?
Anyway, hope that is at least interesting.
If you have a service such as:
In a Grails 2 app, the recommended way would be to use transactionStatus.setRollbackOnly().
import grails.transaction.Transactional
Class RoleService {
#Transactional
Role save(String authority) {
Role roleInstance = new Role(authority: authority)
if ( !roleInstance.save() ) {
// log errors here
transactionStatus.setRollbackOnly()
}
roleInstance
}
}
See: https://github.com/grails/grails-core/issues/9212

swing uncaughtexceptionhandler

I am trying to build a general exception handler for a swing application as described here: http://www.javaspecialists.eu/archive/Issue081.html
I work in jython (python syntax getting compiled to java and executed). My code looks roughly like this (updated):
def launcher(func):
class launcherThread(Runnable):
def __init__(self):
super(launcherThread, self).__init__()
def run(self):
func()
#trying to get the name which can be used to instantiate this in java
cls = ExceptionGroup().getClass()
fullName = cls.__module__ + '.' + cls.__name__
System.setProperty("sun.awt.exception.handler", fullName)
Thread(ExceptionGroup(), launcherThread(), 'Cross ExceptionHandlerThread').start()
class ExceptionGroup(ThreadGroup):
def __init__(self):
super(ExceptionGroup, self).__init__("HardenedGroup")
def uncaughtException(self, thread, exception):
#make a fancy dialog displaying str(exception)
If I test it it works fine however in the production enviornment it failes.
For testing I launch my program in Eclipse (PyDev), the production enviornment is a third party application written in Java, that has a Jython console build in. The application supports adding of custom menu entries, and putting jython scripts on these.
The main difference I see between testing and production enviornment is that in the production enviornment the swing threads are allready started (the third party application utilitizes swing). Does this cause my ThreadGroup setting to fail, or is there another reason why this is not working?
How can I get the Involved threads (exceptions ar thrown as a result of buttonActions) to check their defaultException handlers? If (as I am afraid) it should turn out that the third party installed its own handler (all exceptions are written to a log file) how can I make a new swing worker thread? (I don't want to catch the exceptions created by the host application after all)
Question recap:
1. How can I check which threads are started for the function func passed into the launcher function and see thier uncaught exception handler?
2. Can I enforce a seperate swing dispatcher for my gui part and the main applications gui part? (If I exitOnClos on a frame of my add in, the third party application closes)?
Update:
Considering the anwser from lbalazscs I am trying to use the sun.awt.exception.handler property, but it has no effect, the exceptions still end up in the log file (applications dfeault behaviour). Am I using it right? (p.s.: I am on Java 1.6)
If you have Java 5 or higher, you can also use Thread.setDefaultUncaughtExceptionHandler(), which is also described in a newer "Java Specialists' Newsletter":
http://www.javaspecialists.eu/archive/Issue089.html
And here is the newest Java 7 version:
http://www.javaspecialists.eu/archive/Issue196.html
Also see this:
Why bother with setting the "sun.awt.exception.handler" property?
EDIT: This is how I use Thread.setDefaultUncaughtExceptionHandler (in Java...):
public static void setupGlobalExceptionHandling() {
Thread.setDefaultUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() {
#Override
public void uncaughtException(Thread t, Throwable e) {
handleException(e);
}
});
}