I have in MySQL database table containing "tasks". Each task have flag (if it's taken or not).
And now for example 3 threads do:
query_base = session.query(PredykcjaRow).filter(
PredykcjaRow.predyktor == predictor,
PredykcjaRow.czy_wziete == False
)
query_disprot = query_base.join(NieustrRow, NieustrRow.fastaId == PredykcjaRow.fastaId)
query_pdb = query_base.join(RawBialkoRow, RawBialkoRow.fasta_id == PredykcjaRow.fastaId)
response = query_pdb.union(query_disprot)
response = response.with_for_update()
response = response.first()
if response is None:
return None
response.czy_wziete = True
try:
session.commit()
return response
except:
return None
each thread have own session (ScopedSession) but all 3 threads get the same object.
In configuration
tx_isolation..... REPEATABLE-READ
Assuming the scoped session is created like this:
Session = scoped_session(sessionmaker(bind=engine))
Make sure you aren't doing this
session = Session()
give_to_thread1(session)
give_to_thread2(session)
With a scoped session, you can use it directly, e.g.
Session.query(...)
So your threads should do this:
def runs_in_thread():
Session.add(...)
# or
session = Session()
session.add(...)
The problem is union statement. MySQL does not provide accumulative SELECTS with FOR UPDATE - it execute without warning, but row is not locked.
I found this information in official documentation but now I can't. If anyone can, please post comment.
Related
I have a SQLAlchemy query that I build, such as :
query_one = User.query.filter(User.id == 1) # Note that I don't call .first() or .all() as I want the "select" instance.
I want to store this Select query in such a way that I can retrieve it by having the same query :
stored_queries = {}
stored_queries[hash(query_one)] = query_one
# ... later on:
query_two = User.query.filter(User.id == 1)
if hash(query_two) in stored_queries:
# Execute custom code because it's the same query
Of course, hash in that case does not work, but is there a SQLAlchemy method that works in the same way?
I thought of str(query_one), but that query only consider the request, without the value. I need both.
Thank you in advance.
You can compile the query to get access to the parameters, and use those as part of your key:
def query_key(query):
statement = query_one.statement.compile()
return str(statement), str(statement.params)
query_key(query_one)
('SELECT user.id, ... FROM user WHERE user.id = :id_1', "{'id_1': 1}")
See https://docs.sqlalchemy.org/en/14/core/selectable.html#sqlalchemy.sql.expression.TableClause.compile
public void updateUserState(User user) {
Session sess=getSession();
sess.setFlushMode(FlushMode.MANUAL);
String queryStr = "update User usr set usr.logCount = :logCount , usr.isLocked = :isLocked , usr.lastLogin = :lastLogin where usr.userId=:userId";
Query query=null;
query = sess.createNativeQuery(queryStr);
query.setParameter("logCount", user.getLogCount());
query.setParameter("isLocked", user.getIsLocked());
query.setParameter("lastLogin", user.getLastLogin());
query.setParameter("userId", user.getUserId());
query.executeUpdate();
}
This is my code. This does not update mu user table in database , neither does this throw any error. It reflects the correct value till set parameter but after executeUpdate, I cannot see any update in my table. It would be really nice if anyone of you can tell me, where am I going wrong. Thanks in advance!
According to the hibernate documentation flush type MANUAL assume:
The Session flushing is delegated to the application, which must call Session.flush() explicitly in order to apply the persistence context changes.
So, you should explicitly call Session.flush() in the end of your method.
Also your updateUserState method should be ran inside a transaction:
Session sess = getSession();
sess.setFlushMode(FlushMode.MANUAL);
Transaction txn = sess.beginTransaction();
// ...
query.executeUpdate();
sess.flush();
txn.commit();
session.close();
I stack on this error for a long time, I try to find some records by passing the where condition with variable. for some reason the query code that I wrote in python those not get the variable and return this error :
self._connection.handle_unread_result().
raise errors.InternalError("Unread result found").
mysql.connector.errors.InternalError: Unread result found.
and here is the code I am using to function to execute!
def unfollow_user(username, update):
if update == True:
get_list_of_not_following_back()
sql.cur.execute("SELECT user_id FROM not_following_back WHERE username = (%s) AND of_user =
(%s)",(username, current_user[0]))
unfollow_user_id = sql.cur.fetchone()
def unfollow_number_of_follower():
sql.cur.execute("SELECT username,user_id FROM not_following_back WHERE of_user =(%s)",
(current_user[0],))
list_of_user_to_unfollow = sql.cur.fetchmany(number_of_user_to_unfollow)
for each_user_to_unfollow in list_of_user_to_unfollow:
unfollow_user(each_user_to_unfollow[0], False)
I figured out the result to this problem all you need to do is defined your cur as
cur = conn.cursor(buffered=True)
I am not sure why but it work
I am using celery to archive the async job in python, my code flow is as following:
celery task get some data from remote api
celery beat get the celery task result from celery backend which is redis and then insert the result into redis
but in step 2, before I insert result data into mysql, I check if the data is existed.although I do the check, the duplicate data still be inserted.
my code is as following:
def get_task_result(logger=None):
db = MySQLdb.connect(host=MYSQL_HOST, port=MYSQL_PORT, user=MYSQL_USER, passwd=MYSQL_PASSWD, db=MYSQL_DB, cursorclass=MySQLdb.cursors.DictCursor, use_unicode=True, charset='utf8')
cursor = db.cursor()
....
....
store_subdomain_result(db, cursor, asset_id, celery_task_result)
....
....
cursor.close()
db.close()
def store_subdomain_result(db, cursor, top_domain_id, celery_task_result, logger=None):
subdomain_list = celery_task_result.get('result').get('subdomain_list')
source = celery_task_result.get('result').get('source')
for domain in subdomain_list:
query_subdomain_sql = f'SELECT * FROM nw_asset WHERE domain="{domain}"'
cursor.execute(query_subdomain_sql)
sub_domain_result = cursor.fetchone()
if sub_domain_result:
asset_id = sub_domain_result.get('id')
existed_source = sub_domain_result.get('source')
if source not in existed_source:
new_source = f'{existed_source},{source}'
update_domain_sql = f'UPDATE nw_asset SET source="{new_source}" WHERE id={asset_id}'
cursor.execute(update_domain_sql)
db.commit()
else:
insert_subdomain_sql = f'INSERT INTO nw_asset(domain) values("{domain}")'
cursor.execute(insert_subdomain_sql)
db.commit()
I first select if the data is existed, if the data not existed, I will do the insert, the code is as following:
query_subdomain_sql = f'SELECT * FROM nw_asset WHERE domain="{domain}"'
cursor.execute(query_subdomain_sql)
sub_domain_result = cursor.fetchone()
I do this, but it still insert duplicate data, I can't understand this.
I google this question and some one says use insert ignore or relace into or unique index, but I want to know why the code not work as expectedly?
also, In my opinion, I think if there is some cache in mysql, when I do the select, the data not really into mysql it just in the flush, so the select will return none?
The older MTurk API (and boto2) had an UpdateQualificationScore method that would allow users to update the score of a specific worker, but this seems to have disappeared in the latest version(s) based on boto3.
The latest MTurk API has a GetQualificationScore method (which actually returns a full worker qualification record, not just the score), but no corresponding UpdateQualificationScore method. What is the mechanism to update a score for an existing worker?
As best as I can tell, the proper way to do this with the boto3 is to use the AssociateQualificationWithWorker endpoint:
session = boto3.Session(profile_name='mturk')
client = session.client('mturk')
response = client.associate_qualification_with_worker(
QualificationTypeId=qualification_type_id,
WorkerId=worker_id,
IntegerValue=score,
SendNotification=False,
)
This seems to work, especially when taken alongside GetQualificationScore returning the "full" qualification record instead of just the score.
ex-nerd's answer is correct. Building off the Python sample available at http://requester.mturk.com/developer, the following works to assign a QualificationType then change the score for that Worker:
import boto3
region_name = 'us-east-1'
aws_access_key_id = 'YOUR_ACCESS_ID'
aws_secret_access_key = 'YOUR_SECRET_KEY'
endpoint_url = 'https://mturk-requester-sandbox.us-east-1.amazonaws.com'
# Uncomment this line to use in production
# endpoint_url = 'https://mturk-requester.us-east-1.amazonaws.com'
client = boto3.client(
'mturk',
endpoint_url=endpoint_url,
region_name=region_name,
aws_access_key_id=aws_access_key_id,
aws_secret_access_key=aws_secret_access_key,
)
# This will assign the QualificationType
client.associate_qualification_with_worker(QualificationTypeId = '3KIOU9ULHKIIS5OPUVORW7OE1070V0', WorkerId = 'A39ECJ12CY7TE9', IntegerValue = 100)
# This will set the QualificationScore from 100 to 90
client.associate_qualification_with_worker(QualificationTypeId = '3KIOU9ULHKIIS5OPUVORW7OE1070V0', WorkerId = 'A39ECJ12CY7TE9', IntegerValue = 90)