I am trying to insert data into a database using the following code:
print("New Account: {}".format(new_acc))
session.add(new_acc)
if pers_inf is not None:
session.add(pers_inf)
print(session.dirty)
session.commit()
print("Number of Accounts after adding: {}".format(session.query(dbquery).filter_by(**param).count()))
session.close()
new_acc is not empty and session.add() does not throw any exception but session.dirty returns an empty IdentitySet and nothing is inserted into the database. Can anyone give me a hint on how to continue debugging here?
Related
Using Postgres, SQLAlchemy 1.4 ORM, Python 3.7, Pytest.
I have a script in myproject/src/db.py and the tests for it are located in myproject/tests/.
In db.py I have a function to drop any given table, it works as expected:
async def delete_table(self, table_name):
table = self.meta.tables[table_name]
async with self.engine.begin() as conn:
await conn.run_sync(Base.metadata.drop_all(sync_engine, [table], checkfirst=True))
It gets called like:
asyncio.run(db.delete_table('user'))
In conftest.py I have a fixture like this:
#pytest.fixture(scope='function')
def test_drop_table():
def get_delete_tables(table_name):
return asyncio.run(DB.delete_table(table_name))
return get_delete_tables
In test.py I run the test like this:
#pytest.mark.parametrize('function_input, expected',
[('user', 'user'),
pytest.param('intentional failure', 'intentional failure',
marks=pytest.mark.xfail(reason='testing incorrect table name fails'))])
def test_drop_table(test_drop_table, function_input, expected):
# Drop the table
test_drop_table(function_input)
# Check that it no longer exists
with pytest.raises(KeyError) as error_info:
test_table_commits(function_input, expected)
raise KeyError
assert error_info.type is KeyError
When I run this test I get this error:
self = <postgresdb.PostgresDb object at 0x7f4bbd87cc18>, table_name = 'user'
async def delete_table(self, table_name):
> table = self.meta.tables[table_name]
E KeyError: 'user'
I verified that this table can be dropped in the main script. I then recommit the table, verify it is present, and then try to drop it with the test but will continually receive a KeyError that the table is not present even though when checking the database that table is actually present.
I'm not sure what to test or adjust in the code to get Pytest working with this function. I appreciate any help!
I think for the first time it deletes the table named user, but the second input in pytest.mark.parametrize is also the name user, so it may be throwing error. If you need to test 2 different scenarios, it's better to have 2 different test functions. By doing this, you can have all your code under with pytest.raises(KeyError) as error_info in the 2nd test function.
I'm trying to call a stored procedure on a multi-db Django installation, but am not having any luck getting results. The stored procedure (which is on the secondary database) always returns an empty array in Django, but the expected result does appear when executed in a mysql client.
My view.py file
from SomeDBModel import models
from django.db import connection
def index(request, someid):
#Some related django-style query that works here
loc = getLocationPath(someid, 1)
print(loc)
def getLocationPath(id, someval):
cursor = connection.cursor()
cursor.callproc("SomeDB.spGetLocationPath", [id, someval])
results = cursor.fetchall()
cursor.close()
return results
I have also tried:
from SomeDBModel import models
from django.db import connections
def index(request, someid):
#Some related Django-style query that works here
loc = getLocationPath(someid, 1)
print(loc)
def getLocationPath(id, someval):
cursor = connections["SomeDB"].cursor()
cursor.callproc("spGetLocationPath", [id, someval])
results = cursor.fetchall()
cursor.close()
return results
Each time I print out the results, I get:
[]
Example of data that should be retrieved:
{
Path: '/some/path/',
LocalPath: 'S:\Some\local\Path',
Folder: 'SomeFolderName',
Code: 'SomeCode'
}
One thing I also tried was to print the result of cursor.callproc. I get:
(id, someval)
Also, printing the result of cursor._executed gives:
b'SELECT #_SomeDB.spGetLocationPath_arg1, #_SomeDB.spGetLocationPath_arg2'
Which seems to not have any reference to the stored procedure I want to run at all. I have even tried this as a last resort:
cursor.execute("CALL spGetLocationPath("+str(id)+","+str(someval)+")")
but I get an error about needing multi=True, but putting it in the execute() function doesn't seem to work like some sites have suggested, and I don't know where else to put it in Django.
So...any ideas what I missed? How can I get stored procedures to work?
These are the following steps that I took:
Made my stored procedure dump results into a temporary table so as to flatten the result set to a single result set. This got rid of the need for multi=True
In addition, I made sure the user at my IP address had access to call stored procedures in the database itself.
Finally, I continued to research the callproc function. Eventually someone on another site suggested the following code, which worked:
cur = connections["SomeDB"].cursor()
cur.callproc("spGetLocationPath", [id, someval])
res = next(cur.stored_results()).fetchall()
cur.close()
I have a problem to catch util.JDBCExceptionReporter during save().
Code:
membership.withTransaction { status ->
try{
def membership = membership.findByUserId(userId)
if(!membership){
membership = new membership(userId: userId, email: userEmail, baseLevel: membership.findByName(membershipName.MEMBER)).save(flush: true)
}
}catch(org.springframework.dao.DataIntegrityViolationException e){
status.setRollbackOnly()
return false
}catch(all){
println "Insert Membership Exception.\n User Id: $userId\n " + e
}
When I create two thread to run this code, it throw a error:
2014-05-06 12:53:07,034 [Actor Thread 5] ERROR util.JDBCExceptionReporter - Duplicate entry 'test#gmail.com' for key 'email'
I don't want to show this error every time when their has two threads doing the same insert, because the first thread will go through and insert successfully, so I don't really care about second one.
My question is how to catch util.JDBCExceptionReporter?
Thanks.
Just guessing:
By default Grails doesn't throw exceptions when saving. To throw integrity exceptions you have to use save(failOnError: true).
So in this case, it's just an internal trace (util.JDBCExceptionReporter is not an exception).
In your case, instead of capturing exceptions I'd use validate before saving so you can get the integrity errors before trying to save.
As lsidroGH said, util.JDBCExceptionReporter is not an exception, it's a log message. It logs both SQLExceptions and SQLWarnings. There is no problem with your code, as one thread will have a save() call that returns true and the other thread's save() will get false.
If you don't want this message to show up in your logs, you will need to increase your log level for org.hibernate.util.JDBCExceptionReporter from ERROR to FATAL but this will potentially exclude valid exceptions you would want logged. Your best bet is to ignore it, as your code works.
I'm trying some db schema changes to my db, using the sqlalchemy table.create and sqlalchemy-migrate table.rename methods, plus some insert into select statments. I want to wrap all of this in a transaction. I can't figure out how to do this. This is what I tried:
engine = sqlalchemy.engine_from_config(conf.local_conf, 'sqlalchemy.')
trans = engine.connect().begin()
try:
old_metatadata.tables['address'].rename('address_migrate_tmp', connection=trans)
new_metatadata.tables['address'].create(connection=trans)
except:
trans.rollback()
raise
else:
trans.commit()
But it errors with:
AttributeError: 'RootTransaction' object has no attribute '_run_visitor'
(I tried using sqlalchemy-migrate column.alter(name='newname') but that errors, and does not work in a transaction, and so leaves my db in a broken state. I also need to rename multiple columns, and so I decide to roll my own code.)
Ah - I need to simply use the connection that the transaction was created on.
engine = sqlalchemy.engine_from_config(conf.local_conf, 'sqlalchemy.')
conn = engine.connect()
trans = conn.begin()
try:
old_metatadata.tables['address'].rename('address_migrate_tmp', connection=conn)
new_metatadata.tables['address'].create(bind=conn)
except:
trans.rollback()
raise
else:
trans.commit()
I am experimenting with ErlyDB in a non-erlyweb environment and I am not having much luck.
I have a 'Thing' table for testing, and a corresponding Thing module:
-module(thing).
-export([table/0, fields/0]).
table() ->
thing.
fields() ->
[name, value].
The module itself works - I can query the database fine using ([Thing] = thing:find({name, '=', "test"})).
When I try and save a new record, however things aren't so good.
I consistently see the following error:
mysql_conn:426: fetch <<"BEGIN">> (id <0.97.0>)
mysql_conn:426: fetch <<"INSERT INTO thing(value,name) VALUES ('vtha','blah')">> (id <0.97.0>)
mysql_conn:426: fetch <<"ROLLBACK">> (id <0.97.0>)
** exception exit: {{'EXIT',{badarg,[{erlang,hd,[[]]},
{erlydb_base,'-do_save/1-fun-0-',4},
{mysql_conn,do_transaction,2},
{mysql_conn,loop,1}]}},
{rollback_result,{updated,{mysql_result,[],[],0,[]}}}}
in function erlydb_base:do_save/1
in call from erlydb_base:hook/4
in call from test_thing:test/0
called as test_thing:test()
The table exists, the credentials work, and the SQL itself is fine, as I can execute the command directly on the database.
The code I am using to save is:
erlydb:start(mysql, Database),
Thing = thing:new(<<"hello">>, <<"world">>),
thing:save(Thing),
Is there something I am missing?
Is there some way of viewing some more helpful error messages from the database?
Looking at the source of erlydb_base, the exception happens when erlydb calls your thing module's db_pk_fields() function. That function should return a list, but apparently it doesn't.
I can confirm that altering the code in erlydb.erl fixes this problem (from this reference on the mailing list).
Change Line 561 of erlydb.erl from
lists:map(fun({_Name, _Atts} = F) -> F;
(Name) -> {Name, []}
end, lists:usort(DefinedFields)),
To:
lists:map(fun({_Name, _Atts} = F) -> F;
(Name) -> {Name, []}
end, DefinedFields),