I am making a login and registration project. The login code looks similar to this where the query is obviously not an insert statement. This is the registration code below to register a user. The front end sends their json to me as a back end and I parse it. I remember this was working yesterday. Now today, I've noticed it's not inserting data into the database, yet I am getting the true response that it inserted. I check the table and nothing is getting inserted.
def auth(n):
cnx = mysql.connector.connect(user='dbuser', password='dbpass', host='localhost', port='3306', database='dbname')
cursor = cnx.cursor(buffered = True)
value_list = list()
for value in n.values():
value_list.append(value)
value_string = str(value_list)
a = value_string.strip("[")
b = a.strip("]")
c = b.replace("'", "")
d = c.split(', ')
authquery=("INSERT INTO members (id, firstname, lastname, email, password, history) VALUES (id, %s, %s, %s, %s);")
cursor.execute(authquery, d)
if cursor.rowcount:
return "true"
else:
return "false"
cursor.close()
cnx.commit()
cnx.close()
Anyone know why it is doing this? I even tried creating a new table and even a new database.
Related
connection = request.form.get('connection_type_id')
tariff = request.form.get('tariff_id')
house = request.form.get('house_type_id')
status = 1
mySql_insert_query = "INSERT INTO tbl_consumer(consumer_no, connection_type_id, tariff_id, house_type_id, status) VALUES (%s,%s,%s,%s,%s)"
cursor = connection.cursor()
result = cursor.execute(mySql_insert_query,(consumer, connection, tariff, house, status))
connection.commit()
print("Record inserted successfully into Laptop table")
cursor.close()
return redirect(url_for('admin_add_consumer'))
connection = request.form.get('connection_type_id') was this supposed to be connection_type_id = ....? request.form is by default ImmutableMultiDict with form data, your connection variable is set to form element with name connection_type_id, that's why you are getting error. I assume connection is mysql connection object from global space.
Please change to connection_type_id = request.form.get('connection_type_id') and it's references as well.
I am using celery to archive the async job in python, my code flow is as following:
celery task get some data from remote api
celery beat get the celery task result from celery backend which is redis and then insert the result into redis
but in step 2, before I insert result data into mysql, I check if the data is existed.although I do the check, the duplicate data still be inserted.
my code is as following:
def get_task_result(logger=None):
db = MySQLdb.connect(host=MYSQL_HOST, port=MYSQL_PORT, user=MYSQL_USER, passwd=MYSQL_PASSWD, db=MYSQL_DB, cursorclass=MySQLdb.cursors.DictCursor, use_unicode=True, charset='utf8')
cursor = db.cursor()
....
....
store_subdomain_result(db, cursor, asset_id, celery_task_result)
....
....
cursor.close()
db.close()
def store_subdomain_result(db, cursor, top_domain_id, celery_task_result, logger=None):
subdomain_list = celery_task_result.get('result').get('subdomain_list')
source = celery_task_result.get('result').get('source')
for domain in subdomain_list:
query_subdomain_sql = f'SELECT * FROM nw_asset WHERE domain="{domain}"'
cursor.execute(query_subdomain_sql)
sub_domain_result = cursor.fetchone()
if sub_domain_result:
asset_id = sub_domain_result.get('id')
existed_source = sub_domain_result.get('source')
if source not in existed_source:
new_source = f'{existed_source},{source}'
update_domain_sql = f'UPDATE nw_asset SET source="{new_source}" WHERE id={asset_id}'
cursor.execute(update_domain_sql)
db.commit()
else:
insert_subdomain_sql = f'INSERT INTO nw_asset(domain) values("{domain}")'
cursor.execute(insert_subdomain_sql)
db.commit()
I first select if the data is existed, if the data not existed, I will do the insert, the code is as following:
query_subdomain_sql = f'SELECT * FROM nw_asset WHERE domain="{domain}"'
cursor.execute(query_subdomain_sql)
sub_domain_result = cursor.fetchone()
I do this, but it still insert duplicate data, I can't understand this.
I google this question and some one says use insert ignore or relace into or unique index, but I want to know why the code not work as expectedly?
also, In my opinion, I think if there is some cache in mysql, when I do the select, the data not really into mysql it just in the flush, so the select will return none?
First of all, sorry for my lack of knowledge regarding databases, this is my first time working with them.
I am having some issues trying to get the data from an excel file and putting it into a data base.
Using answers from the site, I managed to kind of connect to the database by doing this.
import pandas as pd
import pyodbc
server = 'XXXXX'
db = 'XXXXXdb'
# create Connection and Cursor objects
conn = pyodbc.connect('DRIVER={SQL Server};SERVER=' + server + ';DATABASE=' + db + ';Trusted_Connection=yes')
cursor = conn.cursor()
# read data from excel
data = pd.read_excel('data.csv')
But I dont really know what to do now.
I have 3 tables, which are connected by a 'productID', my excel file mimics the data base, meaning that all the columns in the excel file have a place to go in the DB.
My plan was to read the excel file and make lists with each column, then insert into the DB each column value but I have no idea how to create a query that can do this.
Once I get the query I think the data insertion can be done like this:
query = "xxxxxxxxxxxxxx"
for row in data:
#The following is not the real code
productID = productID
name = name
url = url
values = (productID, name, url)
cursor.execute(query,values)
conn.commit()
conn.close
Database looks like this.
https://prnt.sc/n2d2fm
http://prntscr.com/n2d3sh
http://prntscr.com/n2d3yj
EDIT:
Tried doing something like this, but i'm getting 'not all arguments converted during string formatting' Type error.
import pymysql
import pandas as pd
connStr = pymysql.connect(host = 'xx.xxx.xx.xx', port = xxxx, user = 'xxxx', password = 'xxxxxxxxxxx')
df = pd.read_csv('GenericProducts.csv')
cursor = connStr.cursor()
query = "INSERT INTO [Productos]([ItemID],[Nombre])) values (?,?)"
for index,row in df.iterrows():
#cursor.execute("INSERT INTO dbo.Productos([ItemID],[Nombre])) values (?,?,?)", row['codigoEspecificoProducto'], row['nombreProducto'])
codigoEspecificoProducto = row['codigoEspecificoProducto']
nombreProducto = row['nombreProducto']
values = (codigoEspecificoProducto,nombreProducto)
cursor.execute(query,values)
connStr.commit()
cursor.close()
connStr.close()
I think my problem is in how I'm defining the query, surely thats not the right way
Try this, you seem to have changed the library from pyodbc to mysql, it seems to expect %s instead of ?
import pymysql
import pandas as pd
connStr = pymysql.connect(host = 'xx.xxx.xx.xx', port = xxxx, user = 'xxxx', password = 'xxxxxxxxxxx')
df = pd.read_csv('GenericProducts.csv')
cursor = connStr.cursor()
query = "INSERT INTO [Productos]([ItemID],[Nombre]) values (%s,%s)"
for index,row in df.iterrows():
#cursor.execute("INSERT INTO dbo.Productos([ItemID],[Nombre]) values (%s,%s)", row['codigoEspecificoProducto'], row['nombreProducto'])
codigoEspecificoProducto = row['codigoEspecificoProducto']
nombreProducto = row['nombreProducto']
values = (codigoEspecificoProducto,nombreProducto)
cursor.execute(query,values)
connStr.commit()
cursor.close()
connStr.close()
This question already has an answer here:
Python sqlite3 parameterized drop table
(1 answer)
Closed 5 years ago.
I am having a problem with WTForms in Flask, I want to create a add_menu function which adds menu to the database. User can choose from SelectField "Appetizer", "Main Dish", or "Drinks" accordingly. So whenever user chooses the value from SelectField it adds to the corresponding table in a database. (I use MySQL). For some reason when I use menu_type = form.menu_type.data it gives me the following error
mysql_exceptions.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''main_dishes'(name,ingredients,price) VALUES('Salmon', 'duude,frv', '35')' at line 1")
It takes the right value, but I have this awkward '' signs infront of main_dishes string
My code looks as follows:
class MenuForm(Form):
menu_type = SelectField('Menu Type', [validators.DataRequired()], choices=[('appetizers','Appetizer'),('main_dishes','Main Dish'),('desserts','Dessert'),('drinks','Drinks')], coerce=str)
name = StringField('Name', [validators.Length(min=1, max=2000)])
ingredients = TextAreaField('Ingredients', [validators.Length(min=10)])
price = DecimalField('Price (Manat)', [validators.DataRequired()])
#app.route('/add_menu', methods=['GET','POST'])
#is_logged_in
def add_menu():
form = MenuForm(request.form)
if request.method == 'POST' and form.validate():
menu_type = form.menu_type.data # <---Here is the problem
name = form.name.data
ingredients = form.ingredients.data
price = form.price.data
#Create cursor
cur = mysql.connection.cursor()
#execute
cur.execute("INSERT INTO %s(name,ingredients,price) VALUES(%s, %s, %s)", (menu_type,name,ingredients,price))
#Commit to DB
mysql.connection.commit()
#CLose connection
cur.close()
flash('Menu is Added', 'success')
return redirect(url_for('dashboard'))
return render_template('add_menu.html', form=form)
The table name is substituted as a quoted string and the query executed as such.
You may want to build your query with the table name before binding parameterized values.
query = "INSERT INTO {}(name,ingredients,price) VALUES(%s, %s, %s)".format(menu_type)
cur.execute(query, (name,ingredients,price))
In one of the projects I am working on, I am using the transactions explicitly as follows:
from django.db import transaction
#transaction.commit_on_succcess
def some_view(request):
""" Renders some view
"""
I am using Django 1.5.5 and in the docs it says:
The recommended way to handle transactions in Web requests is to tie them to the request and response phases via Django’s TransactionMiddleware.
It works like this: When a request starts, Django starts a transaction. If the response is produced without problems, Django commits any pending transactions. If the view function produces an exception, Django rolls back any pending transactions.
To activate this feature, just add the TransactionMiddleware middleware to your MIDDLEWARE_CLASSES setting:
I want to use the transactions on requests instead of tying them to a particular view that requires it but I am a little confused about how this'd work. Say I have a view as follows:
def some_view(request):
""" Creates a user object.
"""
context = {}
first_name = request.POST['first_name']
last_name = request.POST['last_name']
email = request.POST['email']
try:
create_user(first_name, last_name, email)
context['success'] = 'User %s was added to the database.' % (first_name)
except IntegrityError, err:
context['failure'] = 'An error occurred while adding user to the database: %s' % (str(err))
except Exception, err:
context['failure'] = '%s' % (str(err))
return json_response(context)
In the above view we are handling the exceptions and returning a response and in the docs it states:
If the response is produced without problems, Django commits any pending transactions.
Q: Will the transactions be committed in the above mentioned view even if it raises an exception ?
What if we want to create multiple objects in a single request and only want to rollback a single entry that raises an exception but commit all other ? So for example, we read a data from the file and for each row we want to create a user object, we want all the users to be inserted into the database except for the ones that raise an error:
def some_view(request):
""" Creates a user object.
"""
context = {}
data = # Read from file
for row in data:
first_name, last_name, email = row.split(",")
try:
create_user(first_name, last_name, email)
context['success'] = 'User %s was added to the database.' % (first_name)
except IntegrityError, err:
context['failure'] = 'An error occurred while adding user to the database: %s' % (str(err))
except Exception, err:
context['failure'] = '%s' % (str(err))
return json_response(context)
Q: How would the transactions work in this case ? Is it better to explicitly use transactions here ?
Update:
In my case I am using an Inherited model class. Example:
class BaseUser(models.Model)
first_name = models.CharField(max_length=100)
last_name = models.CharField(max_length=100)
email = models.EmailField(max_length=100, unique=True)
class UserA(BaseUser):
phone = models.BigIntegerField()
type = models.CharField(max_length=32)
So in case I am trying to create a UserA type object using the above view and it raises an exception, The BaseUSer object is created with the given data but UserA type object is not. So, what I am trying to do is to either create the TypeA object or do not commit any changes. Currently I am using transactions manually (as follows) and it seem to work fine, It's just that I want to switch to using transactions on HTTP requests instead.
from django.db import transaction
transaction.commit_on_success()
def some_view(request):
""" Creates a user object.
"""
context = {}
data = # Read from file
for row in data:
first_name, last_name, email = row.split(",")
try:
sid = transaction.savepoint()
create_user(first_name, last_name, email)
context['success'] = 'User %s was added to the database.' % (first_name)
transaction.savepoint_commit()
except IntegrityError, err:
context['failure'] = 'An error occurred while adding user to the database: %s' % (str(err))
transaction.savepoint_rollback()
except Exception, err:
transaction.savepoint_rollback()
context['failure'] = '%s' % (str(err))
return json_response(context)
You don't need transactions at all here. If an IntegrityError is raised, that means the database update couldn't even be done, so there is nothing to roll back.
Transactions are useful if you want to roll back all the updates if a single one fails.