If there's a table with a column that I want to get the number of occurrences of the columns 'id' in another tables column?
So if there was a table 'player' of every player, and a table 'goals' that listed every goal scored, is there an easy way to autoupdate the player column every time a goal they score is added to the goal table?
another example would be a 'team' and 'players' table, where the table updates the team.number_of_players every time a player is added with player.team_name == team.name or something like that.
Would using JSON as a way of holding {'username': True} or something like that for each user be worthwhile?
You have several ways to implement you idea:
Easiest way: you can update your columns with update query, something like this:
try:
player = Player(name='New_player_name', team_id=3)
Session.add(player)
Session.flush()
Session.query(Team).filter(Team.id == Player.team_id).update({Team.players_number: Team.players_number + 1})
Session.commit()
except SQLAlchemyError:
Session.rollback()
# error processing
You can implement sql-trigger. But an implementation is different for different DBMS. So, you can read about it in the documentation of your DBMS.
You can implement SQLAlchemy trigger, like this:
from sqlalchemy import event
class Team(Base):
...
class Player(Base):
...
#staticmethod
def increment_players_number(mapper, connection, player):
try:
Session.query(Team).filter(Team.id == player.team_id)\
.update({Team.players_number: Team.players_number + 1})
except SQLAlchemyError:
Session.rollback()
# error processing
event.listen(Player, 'after_insert', Player.increment_players_number)
As you see, there are always two queries, because you should perform two procedures: insert and update. I think (but I'm not sure) that some DBMS can process queries like this:
UPDATE table1 SET column = column + 1 WHERE id = SOMEID AND (INSERT INTO table2 values (VALUES))
Related
I want to specify the return values for a specific update in sqlalchemy.
The documentation of the underlying update statement (sqlalchemy.sql.expression.update) says it accepts a "returning" argument and the docs for the query object state that query.update() accepts a dictionary "update_args" which will be passed as the arguments to the query statement.
Therefore my code looks like this:
session.query(
ItemClass
).update(
{ItemClass.value: value_a},
synchronize_session='fetch',
update_args={
'returning': (ItemClass.id,)
}
)
However, this does not seem to work. It just returns the regular integer.
My question is now: Am I doing something wrong or is this simply not possible with a query object and I need to manually construct statements or write raw sql?
The full solution that worked for me was to use the SQLAlchemy table object directly.
You can get that table object and the columns from your model easily by doing
table = Model.__table__
columns = table.columns
Then with this table object, I can replicate what you did in the question:
from your_settings import db
update_statement = table.update().returning(table.id)\
.where(columns.column_name=value_one)\
.values(column_name='New column name')
result = db.session.execute(update_statement)
tuple_of_results = result.fetchall()
db.session.commit()
The tuple_of_results variable would contain a tuple of the results.
Note that you would have to run db.session.commit() in order to persist the changes to the database as you it is currently running within a transaction.
You could perform an update based on the current value of a column by doing something like:
update_statement = table.update().returning(table.id)\
.where(columns.column_name=value_one)\
.values(like_count=table_columns.like_count+1)
This would increment our numeric like_count column by one.
Hope this was helpful.
Here's a snippet from the SQLAlchemy documentation:
# UPDATE..RETURNING
result = table.update().returning(table.c.col1, table.c.col2).\
where(table.c.name=='foo').values(name='bar')
print result.fetchall()
I have a bunch of MySQL tables I work with where the ultimate data source from a very slow SQL server administered by someone else. My predecessors' solution to dealing with this is to do queries more-or-less like:
results = python_wrapper('SELECT primary_key, col2, col3 FROM foreign_table;')
other_python_wrapper('DELETE FROM local_table;')
other_python_wrapper('INSERT INTO local_table VALUES() %s;' % results)
The problem is this means you can never use values in local_table as foreign key constraints for other tables because they are constantly being deleted and added back into the table whenever you update it from the foreign source. However, if a record really does dis sapper in the results to the query on the foreign server, than that usually means you would want to trigger a cascade effect to drop records in other local tables that you've linked with a foreign key constraint to data propagated from the foreign table.
The only semi-reasonable solution I've come up with is to do something like:
results = python_wrapper('SELECT primary_key, col2, col3 FROM foreign_table;')
other_python_wrapper('DELETE FROM local_table_temp;')
other_python_wrapper('INSERT INTO local_table_temp VALUES() %s;' % results)
other_python_wrapper('DELETE FROM local_table WHERE primary_key NOT IN local_table_temp;')
other_python_wrapper('INSERT INTO local_table SELECT * FROM local_table_temp ON DUPLICATE KEY UPDATE local_table.col2 = local_table_temp.col2, local_table.col3 = local_table_temp.col3
The problem is there's a fair number of these tables and many of the tables have a large number of columns that need to be updated so it's tedious to write the same boiler-plate over & over. And if the table schema changes, there's more than one place you need to update the listing of all columns.
Is there any more concise way to do this with the SQL code?
Thanks!
I have a somewhat un-satisfactory answer to my own question. Since I'm using python to query the foreign Oracle database and put that into SQL, and I trust the format of the table and column names to be pretty well behaved, I can just wrap the whole procedure in python code and have python generate the update SQL update queries based off inspecting the tables.
For a number of reasons, I'd still like to see a better way to do this, but it works for me because:
I'm using an external scripting language that can inspect the database schema anyway.
I trust the database, column, and table names I'm working with to be well-behaved because these are all things I have direct control over.
My solution depends on the local SQL table structure; specifically which keys are primary keys. The code won't work without properly structured tables. But that's OK, because I can restructure the MySQL tables to make my python code work.
While I do hope someone else can think up a more-elegant and/or general-purpose solution, I will offer up my own python code to anyone who is working on a similar problem who can safely make the same assumptions I did above.
Below is a python wrapper I use to do simple SQL queries in python:
import config, MySQLdb
class SimpleSQLConn(SimpleConn):
'''simplified wrapper around a MySQLdb.connection'''
def __init__(self, **kwargs):
self._connection = MySQLdb.connect(host=config.mysql_host,
user=config.mysql_user,
passwd=config.mysql_pass,
**kwargs)
self._cursor = self._connection.cursor()
def query(self, query_str):
self._cursor.execute(query_str)
self._connection.commit()
return self._cursor.fetchall()
def columns(self, database, table):
return [x[0] for x in self.query('DESCRIBE `%s`.`%s`' % (database, table))g]
def primary_keys(self, database, table):
return [x[0] for x in self.query('DESCRIBE `%s`.`%s`' % (database, table)) if 'PRI' in x]
And here is the actual update function, using the SQL wrapper class above:
def update_table(database,
table,
mysql_insert_with_dbtable_placeholder):
'''update a mysql table without first deleting all the old records
mysql_insert_with_dbtable_placeholder should be set to a string with
placeholders for database and table, something like:
mysql_insert_with_dbtable_placeholder = "
INSERT INTO `%(database)s`.`%(table)s` VALUES (a, b, c);
note: code as is will update all the non-primary keys, structure
your tables accordingly
'''
sql = SimpleSQLConn()
query ='DROP TABLE IF EXISTS `%(database)s`.`%(table)s_temp_for_update`' %\
{'database': database, 'table': table}
sql.query(query)
query ='CREATE TABLE `%(database)s`.`%(table)s_temp_for_update` LIKE `%(database)s`.`%(table)s`'%\
{'database': database, 'table': table}
sql.query(query)
query = mysql_insert_with_dbtable_placeholder %\
{'database': database, 'table': '%s_temp_for_update' % table}
sql.query(query)
query = '''DELETE FROM `%(database)s`.`%(table)s` WHERE
(%(primary_keys)s) NOT IN
(SELECT %(primary_keys)s FROM `%(database)s`.`%(table)s_temp_for_update`);
''' % {'database': database,
'table': table,
'primary_keys': ', '.join(['`%s`' % key for key in sql.primary_keys(database, table)])}
sql.query(query)
update_columns = [col for col in sql.columns(database, table)
if col not in sql.primary_keys(database, table)]
query = '''INSERT into `%(database)s`.`%(table)s`
SELECT * FROM `%(database)s`.`%(table)s_temp_for_update`
ON DUPLICATE KEY UPDATE
%(update_cols)s
''' % {'database': database,
'table': table,
'update_cols' : ',\n'.join(['`%(table)s`.`%(col)s` = `%(table)s_temp_for_update`.`%(col)s`' \
% {'table': table, 'col': col} for col in update_columns])}
sql.query(query)
I am using the code below to extract table names on a database at a GET call in a Flask app.:
session = db.session()
qry = session.query(models.BaseTableModel)
results = session.execute(qry)
table_names = []
for row in results:
for column, value in row.items():
#this seems like a bit of a hack
if column == "tables_table_name":
table_names.append(value)
print('{0}: '.format(table_names))
Given that tables in the database may added/deleted regularly, is the code above an efficient and reliable way to get the names of tables in a database?
One obvious optimization is to use row["tables_table_name"] instead of second loop.
Assuming that BaseTableModel is a table, which contains names of all other tables, than you're using the fastest approach to get this data.
I am able to do mySQL data insert using following,
from twisted.enterprise.adbapi import ConnectionPool
.
.
self.factory.pool.runOperation ('insert into table ....')
But, somehow unable to figure out how to do a simple select from an adbapi call to mySQL like following,
self.factory.pool.runOperation('SELECT id FROM table WHERE name = (%s)',customer)
How do I retrieve the id value from this partilcar call? I was working OK with plain python but somehow really fuzzed up with the twisted framework.
Thanks.
runOperation isn't for SELECT statements. It is for statements that do not produce rows, eg INSERT and DELETE.
Statements that produce rows are supported by runQuery. For example:
pool = ...
d = pool.runQuery("SELECT id FROM table WHERE name = (%s)", (customer,))
def gotRows(rows):
print 'The user id is', rows
def queryError(reason):
print 'Problem with the query:', reason
d.addCallbacks(gotRows, queryError)
In this example, d is an instance of Deferred. If you haven't encountered Deferreds before, you definitely want to read up about them: http://twistedmatrix.com/documents/current/core/howto/defer.html
I need to get last record from db. I'm using sqlalchemy.
At the moment, I'm doing like that:
obj = ObjectRes.query.all()
return str(obj[-1].id)
But it's too heavy query. How can I get last record better?
Take a look at Query.first(). If you specify a sort on the right column, the first will be your last. An example could look like this:
obj = session.query(ObjectRes).order_by(ObjectRes.id.desc()).first()
Sometimes it is difficult to reformulate simple things:
SELECT * FROM ObjectRes WHERE id IN (SELECT MAX(id) FROM ObjectRes)
but this worked for me:
session.query(ObjectRes).filter(ObjectRes.id == session.query(func.max(ObjectRes.id)))
Don't forget to disable existing ordering if needed
In my case I have dynamic ordered relationships:
class Match:
...
records = relationship("Record", backref="match", lazy="dynamic", order_by="Record.id")
And when I tried accepted answer I got first record, not the last, cause ORDER BY were applied twice and spoiled the results.
According to documentation:
All existing ORDER BY settings can be suppressed by passing None
So the solution will be:
match = db_session.query(Match).first()
last_record = match.records.order_by(None).order_by(Record.id.desc()).first()
This answer modifies the others to allow for cases where you don't know what the primary key is called.
from sqlalchemy.inspection import inspect
# ...
def last_row(Table: type, *, session): # -> Table
primary_key = inspect(Table).primary_key[0].name # must be an arithmetic type
primary_key_row = getattr(Table, primary_key)
# get first, sorted by negative ID (primary key)
return session.query(Table).order_by(-primary_key_row).first()