I use FastApi with SqlAlchemy as context manager
#contextmanager
def get_session(): # need to patch all over tests files
session_ = SessionLocal()
try:
yield session_
except Exception as e:
session_.rollback()
router.py
#router.get('/get_users'):
def get_users(): # No dependencies
with get_session() as session:
users = session.query(Users).all()
return users
I need to override get_session during all my tests (I use pytest)
I could do it with #patch and patch all test. But it's not most effective way because i need to use decorator to patch in each test file, correct specify full path.
I wonder is there quick way to do it in one place or maybe use fixture?
You could try the approach in this answer to a similar question: define a pytest fixture with arguments scope="session", autouse=True, patching the context manager. If you need, you can also provide a new callable:
from contextlib import contextmanager
from unittest.mock import patch
import pytest
#pytest.fixture(scope="session", autouse=True, new_callable=fake_session)
def default_session_fixture():
with patch("your_filename.get_session"):
yield
#contextmanager
def fake_session():
yield ... # place your replacement session here
As a side note, I would highly recommend using FastAPI's dependency injection for handling your SQLAlchemy session, especially since it has built in support for exactly this kind of patching.
I am trying to find a clean way to access the regmap that is used with *RegisterNode for creating documentation and testing files. The TLRegisterNode has methods for generating the json through some Annotations. These are done in the regmap method by adding them to the ElaborationArtefacts object. Other protocols don't seem to have these annotations.
Is there anyway to iterate over the "regmap" Register Fields post elaboration or during?
I cannot just access the regmap as it's not really a val/var since it's a method. I can't quite figure out where this information is being stored. I don't really believe it's actually "storing" any information as much as it is simply creating the hardware to attach the specified logic to the RegisterNode based logic.
The JSON output is actually fine for me as I could just write a post processing script to convert JSON to my required formats, but I'm wondering if I can access this information OR if I could add a custom function call at the end. I cannot extend the case class *RegisterNode, but I'm not sure if it's possible to add custom functions to run at the end of the regmap method.
Here is something I threw together quickly:
//in *RegisterRouter.scala
def customregmap(customFunc: (RegField.Map*) => Unit, mapping: RegField.Map*) = {
regmap(mapping:_*)
customFunc(mapping:_*)
}
def regmap(mapping: RegField.Map*) = {
//normal stuff
}
A user could then create a custom function to run and pass it to the regmap or to the RegisterRouter
def myFunc(mapping: RegField.Map*): Unit = {
println("I'm doing my custom function for regmap!")
}
// ...
node.customregmap(myFunc,
0x0 -> coreControlRegFields,
0x4 -> fdControlRegFields,
0x8 -> fdControl2RegFields,
)
This is just a quick example I have. I believe what would be better, if something like this was possible, would be to have a Seq of functions that could be added to the RegisterNode that are ran at the end of the regmap method, similar to how TLRegisterNode currently works. So a user could add an arbitrary number and you still use the regmap call.
Background (not directly part of question):
I have a unified register script that I have built over the years in which I describe the registers for a particular IP. It works very similar to the RegField/node.regmap, except it obviously doesn't know about diplomacy and the like. It will generate the Verilog, but also a variety of files for DV (basic `defines for simple verilog simulations and more complex uvm_reg_block defines also with the ability to describe multiple of the IPs for a subsystem all the way up to an SoC level). It will also print out C Header files for SW and Sphinx reStructuredText for documentation.
Diplomacy actually solves one of the main issues I've been dealing with so I'm obviously trying to push most of my newer designs to Chisel/Diplo.
I ended up solving this by creating my own RegisterNode which is the same as the rocketchip RegisterNodes except that I use a different Elaboration Artifact to grab the info and store it for later.
I'm integrating Huey with a simple pyramid app. I'm not using a global SQLAlchemy session in the app (I'm making use of the latest alchemy scaffold). However, there seems to be no other straightforward way to provide a session to periodic tasks.
from huey import RedisHuey
huey = RedisHuey(password=os.environ.get('REDIS_PASSWORD', ''))
DBSession = scoped_session(sessionmaker())
#huey.periodic_task(crontab(minute='*/1'))
def notify_not_confirmed_assignments():
# TODO: Use a non-global DB session
assignments = DBSession.query(Assignment).filter_by(date=next_date).all()
Does Huey offer hooks to close the DB connection on task completion? What is the best way to provide a thread-safe connection to these tasks?
Thanks in advance!
You can build session object with a factory in the tasks:
factory = sessionmaker()
factory.configure(bind=engine)
session = factory()
No need to use scoped session, just initialize the engine and pass it to factory.
scoped_session provides you with a contextual/thread-local session (i.e. it corresponds to an individual DB connection in each thread, and it's also possible to configure a custom scope when you need a separate session per something which is not a thread.
So, basically, all you need to do is to have a properly-configured pseudo-global variable (similar to what you have now) and make sure you call DBSession.begin() at the start of the task and DBSession.commit() at the end - doing that manually is probably a chore but it can easily be abstracted into a context manager
def my_task():
with magically_start_session() as session:
session.query(...)
or into a decorator:
#huey.periodic_task(crontab(minute='*/1'))
#start_session
def notify_not_confirmed_assignments(session):
# TODO: Use a non-global DB session
assignments = session.query(...)
Consider the following service (transactional by default). A player must always have one account. A player without at least one corresponding account is an error state.
class playerService {
def createPlayer() {
Player p new Player(name: "Stephen King")
if (!p.save()) {
return [code: -1, errors:p.errors]
}
Account a = new Account(type: "cash")
if (!a.save()) {
// rollback p !
return [code: -2, errors:a.errors]
}
// commit only now!
return [code: 0, player:p]
}
}
I have seen this pattern by experienced grails developers, and when I tell them that if creation of the account of the player fails for any reason, it wont rollback the player, and will leave the DB in an invalid state, they look at me like I am mad because grails handles rolling back the player because services are transactional right?
So then, being a SQL guy, I look for a way to call rollback in grails. There isn't one. According to various posts, there are only 2 ways to force grails to rollback in a service:
throw an unchecked exception. You know what this is right?
don't use service methods or transactional annotations, use this construct:
.
DomainObject.withTransaction {status ->
//stuff
if (someError) {
status.setRollbackOnly()
}
}
1. throw an unchecked exception
1.1 So we must throw runtime exceptions to rollback. This is ok for me (I like exceptions), but this wont gel with the grails developers we have who view exceptions as a throwback to Java and is uncool. It also means we have to change the whole way the app currently uses its service layer.
1.2 If an exception is thrown, you lose the p.errors - you lose the validation detail.
1.3 Our new grails devs don't know the difference between an unchecked and an checked exception, and don't know how to tell the difference. This is really dangerous.
1.4. use .save(failOnError: true)
I am a big fan of using this, but its not appropriate everywhere. Sometimes you need to check the reason before going further, not throw an exception. Are the exceptions it can generate always checked, always unchecked, or either? I.e. will failOnError AWLAYS rollback, no matter what the cause? No one I have asked knows the answer to this, which is disturbing, they are using blind faith to avoid corrupted/inconsistent DBs.
1.5 What happens if a controller calls service A, which calls Service B, then service C. Service A must catch any exception and return a nicely formatted return value to the controller. If Service C throws an exception, which is caught by Service A, will service Bs transactions be rolled back? This is critical to know to be able to construct a working application.
UPDATE 1:
Having done some tests, it appears that any runtime exception, even if thrown and caught in some unrelated child calls, will cause everything in the parent to rollback. However, it is not easy to know in the parent session that this rollback has happened - you need to make sure that if you catch any exception, you either rethrow, or pass some notice back to the caller to show that it has failed in such a way that everything else will be rolled back.
2. withTransaction
2.1 This seems a bazaar construct. How do I call this, and what do I pass in for the "status" parameter? What is "setRollbackOnly" exactly. Why is it not just called "rollback". What is the "Only" part? It is tied to a domain object, when your method may want to do update several different domain objects.
2.2 Where are you supposed to put this code? In with the DomainObject class? In the source folder (i.e. not in a service or controller?)? Directly in the controller? (we don't want to duplicate business logic in the controllers)
3. Ideal situation.
3.1 The general case is we want every thing we do in a service method to roll back if anything in that service method cant be saved for any reason, or throws any exception for any reason (checked or unchecked).
3.2 Ideally I would like service methods to "always rollback, unless I explicitly call commit", which is the safest strategy , but this is not possible I believe.
The question is how do I achieve the ideal situation?
Will calling save(failOnError:true) ALWAYS rollback everything, no matter what the reason for failing? This is not perfect, as it is not easy for the caller to know which domain object save caused the issue.
Or do people define lots of exception classes which subclass runtimeException, then explicit catch each of them in the controller to create the appropriate response? This is the old Java way, and our groovy devs pooh pooh this approach due to the amount of boiler plate code we will have to write.
What methods do people use to achieve this?
I wouldn't call myself an expert, and this question is over a year old, but I can answer some of these questions, if only for future searchers. I'm just now refactoring some controllers to use services in order to take advantage of transactions.
I have seen this pattern by experienced grails developers, and when I tell them that if creation of the account of the player fails for any reason, it wont rollback the player, and will leave the DB in an invalid state, they look at me like I am mad because grails handles rolling back the player because services are transactional right?
I'm not seeing in the documentation where it explicitly states that returning from a service method does not rollback the transaction, but I can't imagine that this would be a very sane behavior. Still, testing is an easy way to prove yourself.
1.2 If an exception is thrown, you lose the p.errors - you lose the validation detail.
Since you're the one throwing the exception, you can throw the errors along with it. For instance:
// in service
if (!email.save()) {
throw new ValidationException("Couldn't save email ${params.id}", email.errors)
}
When you catch the exception, you reload the instance (because throwing an exception clears the session), put the errors back into the instance, and then pass that to the view as usual:
// in controller
} catch (ValidationException e) {
def email = Email.read(id)
email.errors = e.errors
render view: "edit", model: [emailInstance: email]
}
This is discussed under the heading "Validation Errors and Rollback", down the page from http://grails.github.io/grails-doc/2.4.4/guide/single.html#transactionsRollbackAndTheSession.
1.4. use .save(failOnError: true) I am a big fan of using this, but its not appropriate everywhere. Sometimes you need to check the reason before going further, not throw an exception. Are the exceptions it can generate always checked, always unchecked, or either? I.e. will failOnError AWLAYS rollback, no matter what the cause? No one I have asked knows the answer to this, which is disturbing, they are using blind faith to avoid corrupted/inconsistent DBs.
failOnError will cause save() to throw a ValidationException, so yes, if you're in a transaction and aren't checking that exception, the transaction will be rolled back.
Generally speaking, it seems to be un-"Grailsy" to use failOnError a lot, probably for the reasons you listed (e.g., lack of control). Instead, you check whether save() failed (if (!save()) ...), and take action based on that.
withTransaction
I'm not sure the point of this, because SpringSource really encourages the use of services for everything. I personally don't like it, either.
If you want to make a particular service non-transactional, and then make one method of it transactional, you can just annotate that one method with #Transactional (unless your developers also dislike annotations because they're too "Java" ;) ).
Note! As soon as you mark a single method with #Transactional, the overall service will become non-transactional.
3.1 The general case is we want every thing we do in a service method to roll back if anything in that service method cant be saved for any reason, or throws any exception for any reason (checked or unchecked).
I feel like checked exceptions are generally considered not "Groovy" (which also makes them not Grails-y). Not sure about the reason for that.
However, it looks like you can tell your service to rollback on your checked exceptions, by listing them in the rollbackFor option to #Transactional.
Or do people define lots of exception classes which subclass runtimeException, then explicit catch each of them in the controller to create the appropriate response? This is the old Java way, and our groovy devs pooh pooh this approach due to the amount of boiler plate code we will have to write.
The nice thing about Groovy is that you can write your boiler plate once and then call it repeatedly. A pattern I've seen a lot, and am currently using, is something like this:
private void validate(Long id, Closure closure) {
try {
closure()
} catch (ValidationException e) {
def email = Email.read(id)
email.errors = e.errors
render view: "edit", model: [emailInstance: email]
} catch (OtherException e) {
def email = Email.read(id)
flash.error = "${e.message}: ${e.reasons}"
render view: "show", model: [emailInstance: email]
} catch (Throwable t) {
flash.error = "Unexpected error $t: ${t.message}"
redirect action: "list"
}
}
And then call it in each controller action like so:
def update(Long id, Long version) {
withInstance(id, version) { Email emailInstance ->
validate(emailInstance.id) {
emailService.update(emailInstance, params)
flash.message = "Email $id updated at ${new Date()}."
redirect action: "show", id: emailInstance.id
}
}
}
(withInstance is another similar method that DRYs up the check for existence and optimistic locking.)
This approach has downsides. You get the same set of redirects in every action; you probably want to write one set of methods for each controller; and it seems kind of silly to pass a closure into a method and expect the method to know what exceptions the closure will throw. But hey, programming's all about tradeoffs, right?
Anyway, hope that is at least interesting.
If you have a service such as:
In a Grails 2 app, the recommended way would be to use transactionStatus.setRollbackOnly().
import grails.transaction.Transactional
Class RoleService {
#Transactional
Role save(String authority) {
Role roleInstance = new Role(authority: authority)
if ( !roleInstance.save() ) {
// log errors here
transactionStatus.setRollbackOnly()
}
roleInstance
}
}
See: https://github.com/grails/grails-core/issues/9212
I am trying to build a general exception handler for a swing application as described here: http://www.javaspecialists.eu/archive/Issue081.html
I work in jython (python syntax getting compiled to java and executed). My code looks roughly like this (updated):
def launcher(func):
class launcherThread(Runnable):
def __init__(self):
super(launcherThread, self).__init__()
def run(self):
func()
#trying to get the name which can be used to instantiate this in java
cls = ExceptionGroup().getClass()
fullName = cls.__module__ + '.' + cls.__name__
System.setProperty("sun.awt.exception.handler", fullName)
Thread(ExceptionGroup(), launcherThread(), 'Cross ExceptionHandlerThread').start()
class ExceptionGroup(ThreadGroup):
def __init__(self):
super(ExceptionGroup, self).__init__("HardenedGroup")
def uncaughtException(self, thread, exception):
#make a fancy dialog displaying str(exception)
If I test it it works fine however in the production enviornment it failes.
For testing I launch my program in Eclipse (PyDev), the production enviornment is a third party application written in Java, that has a Jython console build in. The application supports adding of custom menu entries, and putting jython scripts on these.
The main difference I see between testing and production enviornment is that in the production enviornment the swing threads are allready started (the third party application utilitizes swing). Does this cause my ThreadGroup setting to fail, or is there another reason why this is not working?
How can I get the Involved threads (exceptions ar thrown as a result of buttonActions) to check their defaultException handlers? If (as I am afraid) it should turn out that the third party installed its own handler (all exceptions are written to a log file) how can I make a new swing worker thread? (I don't want to catch the exceptions created by the host application after all)
Question recap:
1. How can I check which threads are started for the function func passed into the launcher function and see thier uncaught exception handler?
2. Can I enforce a seperate swing dispatcher for my gui part and the main applications gui part? (If I exitOnClos on a frame of my add in, the third party application closes)?
Update:
Considering the anwser from lbalazscs I am trying to use the sun.awt.exception.handler property, but it has no effect, the exceptions still end up in the log file (applications dfeault behaviour). Am I using it right? (p.s.: I am on Java 1.6)
If you have Java 5 or higher, you can also use Thread.setDefaultUncaughtExceptionHandler(), which is also described in a newer "Java Specialists' Newsletter":
http://www.javaspecialists.eu/archive/Issue089.html
And here is the newest Java 7 version:
http://www.javaspecialists.eu/archive/Issue196.html
Also see this:
Why bother with setting the "sun.awt.exception.handler" property?
EDIT: This is how I use Thread.setDefaultUncaughtExceptionHandler (in Java...):
public static void setupGlobalExceptionHandling() {
Thread.setDefaultUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() {
#Override
public void uncaughtException(Thread t, Throwable e) {
handleException(e);
}
});
}