I'm creating some tests in my django web app with selenium.
Django create a test database with all tables.
In my case however I have a model Users.
I want that django create also users table when I run this command in terminal
python3.9 manage.py test -v3
This command goes in error:
Running post-migrate handlers for application contenttypes
Adding content type 'contenttypes | contenttype'
Adding permission 'Permission object (None)'
Adding permission 'Permission object (None)'
Adding permission 'Permission object (None)'
Adding permission 'Permission object (None)'
Running post-migrate handlers for application sessions
Adding content type 'sessions | session'
Adding permission 'Permission object (None)'
Adding permission 'Permission object (None)'
Adding permission 'Permission object (None)'
Adding permission 'Permission object (None)'
Running post-migrate handlers for application pygiustizia
Adding content type 'pygiustizia | member'
Adding permission 'Permission object (None)'
Adding permission 'Permission object (None)'
Adding permission 'Permission object (None)'
Adding permission 'Permission object (None)'
System check identified no issues (0 silenced).
test_login (pygiustizia.tests.test_views_topics.ViewsTopicsTestCase) ... nel costruttore model Users
ERROR
======================================================================
ERROR: test_login (pygiustizia.tests.test_views_topics.ViewsTopicsTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/home/nicola/.local/lib/python3.9/site-packages/django/db/backends/utils.py", line 89, in _execute
return self.cursor.execute(sql, params)
File "/home/nicola/.local/lib/python3.9/site-packages/django/db/backends/mysql/base.py", line 75, in execute
return self.cursor.execute(query, args)
File "/home/nicola/.local/lib/python3.9/site-packages/pymysql/cursors.py", line 148, in execute
result = self._query(query)
File "/home/nicola/.local/lib/python3.9/site-packages/pymysql/cursors.py", line 310, in _query
conn.query(q)
File "/home/nicola/.local/lib/python3.9/site-packages/pymysql/connections.py", line 548, in query
self._affected_rows = self._read_query_result(unbuffered=unbuffered)
File "/home/nicola/.local/lib/python3.9/site-packages/pymysql/connections.py", line 775, in _read_query_result
result.read()
File "/home/nicola/.local/lib/python3.9/site-packages/pymysql/connections.py", line 1156, in read
first_packet = self.connection._read_packet()
File "/home/nicola/.local/lib/python3.9/site-packages/pymysql/connections.py", line 725, in _read_packet
packet.raise_for_error()
File "/home/nicola/.local/lib/python3.9/site-packages/pymysql/protocol.py", line 221, in raise_for_error
err.raise_mysql_exception(self._data)
File "/home/nicola/.local/lib/python3.9/site-packages/pymysql/err.py", line 143, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.ProgrammingError: (1146, "Table 'test_civiledb.users' doesn't exist")
EDIT
I access to the table users that I want create manually for test by this model.
from django.db import models
from django.db import connection
import pymysql
class Users(models.Model):
firstname=models.CharField(max_length=100)
lastname=models.CharField(max_length=100)
username=models.CharField(max_length=50)
password=models.CharField(max_length=255)
admin=models.IntegerField()
createdAt=models.DateTimeField(db_column="created_at")
def __init__(self,*args, **kwargs):
super().__init__(*args, **kwargs)
print("nel costruttore model Users")
def getAdmin(self):
return self.admin
class Meta:
db_table = 'users'
def getUser(self,username):
cursor = connection.cursor()
cursor.execute('''SELECT id,
username,
password,
admin,
firstname,
lastname,
created_at
FROM users
WHERE username = %s''',[username])
result = cursor.fetchone()
return result
def setPassword(self,userId,password):
cursor = connection.cursor()
try:
cursor.execute('''UPDATE users
SET password = %s
WHERE id = %s''',[password,userId])
connection.commit()
except Exception as e:
print(e)
connection.rollback()
return False
finally:
connection.close()
return True
A solution is to create "sql create" schema run it in testcase in setUp and drop it on tearDown.
I think the concept you are looking for here are unmanaged models. By setting the appropriate Meta option you can tell Django that it shouldn't handle the table lifecycle for you.
To answer your question, using SchemaEditor on test setup and teardown seems like a particularly clean way to achieve what you are looking for.
Related
I have a Flask website that uses the peewee ORM. The connection is managed by FlaskDB.
When I only use 1 gunicorn worker, it works well. But as soon as I use 2 or more gunicorn workers, I start getting this error:
Traceback (most recent call last):
File "/home/user/.local/lib/python3.10/site-packages/flask/app.py", line 2073, in wsgi_app
response = self.full_dispatch_request()
File "/home/user/.local/lib/python3.10/site-packages/flask/app.py", line 1519, in full_dispatch_request
return self.finalize_request(rv)
File "/home/user/.local/lib/python3.10/site-packages/flask/app.py", line 1540, in finalize_request
response = self.process_response(response)
File "/home/user/.local/lib/python3.10/site-packages/flask/app.py", line 1888, in process_response
self.session_interface.save_session(self, ctx.session, response)
File "/home/user/project/session.py", line 113, in save_session
saved_session.save()
File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 6497, in save
rows = self.update(**field_dict).where(self._pk_expr()).execute()
File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 1886, in inner
return method(self, database, *args, **kwargs)
File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 1957, in execute
return self._execute(database)
File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 2442, in _execute
cursor = database.execute(self)
File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 3112, in execute
return self.execute_sql(sql, params, commit=commit)
File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 3096, in execute_sql
with __exception_wrapper__:
File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 2873, in __exit__
reraise(new_type, new_type(exc_value, *exc_args), traceback)
File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 183, in reraise
raise value.with_traceback(tb)
File "/home/user/.local/lib/python3.10/site-packages/peewee.py", line 3099, in execute_sql
cursor.execute(sql, params or ())
peewee.DatabaseError: error with status PGRES_TUPLES_OK and no message from the libpq
The peewee documentation states that the connection should be thread-safe, but it seems there are thread-safety issues here.
I use playhouse.pool.PooledPostgresqlDatabase if that matters.
What is the solution to this problem?
I believe this is possibly due to the connection being opened before the workers are forked. I'm not too familiar w/gunicorns worker model, but googling your error reveals similar problems w/multiprocessing. Specifically,
When a program uses multiprocessing or fork(), and an Engine object is copied to the child process, Engine.dispose() should be called so that the engine creates brand new database connections local to that fork. Database connections generally do not travel across process boundaries.
That is for SQLAlchemy, but the same should apply to Peewee. Just ensure that all your connections are closed (pool.close_all()) when your workers first start running. Or similarly, if you're opening any database connections at module scope, ensure you call close_all() after using them. This way when your workers start up they will each have an empty pool of connections.
db = FlaskDB(app)
# ...
db.database.close_all()
I'm trying to connect to a database and create a table based on the user's input, but I'm getting this error: TypeError: __init__() takes 1 positional argument but 6 were given
Here is my code:
import mysql.connector
def add_tb():
host='localhost'
port='3306'
user='root'
password=''
database =input('type the database name you want to add the table inside of it: ')
mydb = mysql.connector.connect(host, port, user, password, database)
mycursor = mydb.cursor()
tablename=input('what the name of table u want to create ?:')
mycursor.execute(f'create table {tablename} (name varchar (200), kelas int (4))')
add_tb()
But I'm gettting this error:
Traceback (most recent call last):
File "c:\Users\NBUSER\Documents\ARTOFWAR\PITON\02-piton\ALDOBOT\buattabel.py", line 18, in <module>
add_tb()
File "c:\Users\NBUSER\Documents\ARTOFWAR\PITON\02-piton\ALDOBOT\buattabel.py", line 13, in add_tb
mydb = mysql.connector.connect(host, port, user, password, database)
File "C:\Users\NBUSER\AppData\Local\Programs\Python\Python39\lib\site-packages\mysql\connector\__init__.py", line 273, in connect
return MySQLConnection(*args, **kwargs)
PS C:\Users\NBUSER\Documents\ARTOFWAR\PITON\02-piton> & C:/Users/NBUSER/AppData/Local/Programs/Python/Python39/python.exe c:/Users/NBUSER/Documents/ARTOFWAR/PITON/02-piton/ALDOBOT/buattabel.py
type database you wanttest123
Traceback (most recent call last):
File "c:\Users\NBUSER\Documents\ARTOFWAR\PITON\02-piton\ALDOBOT\buattabel.py", line 18, in <module>
add_tb()
File "c:\Users\NBUSER\Documents\ARTOFWAR\PITON\02-piton\ALDOBOT\buattabel.py", line 13, in add_tb
mydb = mysql.connector.connect(host, port, user, password, database)
File "C:\Users\NBUSER\AppData\Local\Programs\Python\Python39\lib\site-packages\mysql\connector\__init__.py", line 273, in connect
return MySQLConnection(*args, **kwargs)
File "C:\Users\NBUSER\AppData\Local\Programs\Python\Python39\lib\site-packages\mysql\connector\connection.py", line 72, in __init__
TypeError: __init__() takes 1 positional argument but 6 were given
PS C:\Users\NBUSER\Documents\ARTOFWAR\PITON\02-piton> & C:/Users/NBUSER/AppData/Local/Programs/Python/Python39/python.exe c:/Users/NBUSER/Documents/ARTOFWAR/PITON/02-piton/ALDOBOT/buattabel.py
type the database name you want to add the table inside of it: test123
Traceback (most recent call last):
File "c:\Users\NBUSER\Documents\ARTOFWAR\PITON\02-piton\ALDOBOT\buattabel.py", line 16, in <module>
add_tb()
File "c:\Users\NBUSER\Documents\ARTOFWAR\PITON\02-piton\ALDOBOT\buattabel.py", line 10, in add_tb
mydb = mysql.connector.connect(host, port, user, password, database)
File "C:\Users\NBUSER\AppData\Local\Programs\Python\Python39\lib\site-packages\mysql\connector\__init__.py", line 273, in connect
return MySQLConnection(*args, **kwargs)
File "C:\Users\NBUSER\AppData\Local\Programs\Python\Python39\lib\site-packages\mysql\connector\connection.py", line 72, in __init__
super(MySQLConnection, self).__init__(*args, **kwargs)
TypeError: __init__() takes 1 positional argument but 6 were given
the error you get is because mysql.connector.connect() uses **kwargs in order to be able to take an arbitrary number of keyword arguments ("kwargs" means "keyword arguments"), that's why it does not take arguments in a specific order.
It needs to be provided with key=value pair format.
This is how you need to use it:
mydb = mysql.connector.connect(host=host, port=port, user=user, password=password, database=database)
see more argument names here
I want to use the users from a legacy database but keep everything else on the 'default' database.
The main issue I have atm is that I can't get the database router to properly forward queries to the appropriate database. Namely when I run the migrations 2 times, the second time I get an error
~> DJANGO_SETTINGS_MODULE=settings python manage.py makemigrations
Migrations for 'myapp':
myapp/migrations/0001_initial.py
- Create model CustomUser
~> DJANGO_SETTINGS_MODULE=settings python manage.py makemigrations
Traceback (most recent call last):
File "manage.py", line 21, in <module>
main()
File "manage.py", line 17, in main
execute_from_command_line(sys.argv)
File "/home/manos/workspace/umed/common/oauth2_test/.env/lib/python3.7/site-packages/django/core/management/__init__.py", line 401, in execute_from_command_line
utility.execute()
File "/home/manos/workspace/umed/common/oauth2_test/.env/lib/python3.7/site-packages/django/core/management/__init__.py", line 395, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/home/manos/workspace/umed/common/oauth2_test/.env/lib/python3.7/site-packages/django/core/management/base.py", line 328, in run_from_argv
self.execute(*args, **cmd_options)
File "/home/manos/workspace/umed/common/oauth2_test/.env/lib/python3.7/site-packages/django/core/management/base.py", line 369, in execute
output = self.handle(*args, **options)
File "/home/manos/workspace/umed/common/oauth2_test/.env/lib/python3.7/site-packages/django/core/management/base.py", line 83, in wrapped
res = handle_func(*args, **kwargs)
File "/home/manos/workspace/umed/common/oauth2_test/.env/lib/python3.7/site-packages/django/core/management/commands/makemigrations.py", line 101, in handle
loader.check_consistent_history(connection)
File "/home/manos/workspace/umed/common/oauth2_test/.env/lib/python3.7/site-packages/django/db/migrations/loader.py", line 299, in check_consistent_history
connection.alias,
django.db.migrations.exceptions.InconsistentMigrationHistory: Migration admin.0001_initial is applied before its dependency myapp.0001_initial on database 'auth_db'.
So I assume it tries to run the admin stuff inside the auth_db instead of default database. But I can't really figure out what I'm doing wrong since the routing seems correct.
Simplified code below
models.py
class CustomUser(AbstractUser):
password = models.CharField(max_length=128)
username = models.CharField(unique=True, max_length=32)
class Meta:
managed = False
db_table = 'myauthdb_user'
objects = UserManager()
db.py
class AuthRouter:
""" Forwards queries for users models to the auth database. """
def db_for_read(self, model, **hints):
if model.__name__ == 'CustomUser':
return 'auth_db'
return None
settings.py
DATABASE_ROUTERS = ['db.AuthRouter']
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.sqlite3',
'NAME': join(BASE_DIR, 'db.app'),
},
'auth_db': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': 'postgres',
'USER': 'postgres',
'PASSWORD': 'postgres',
'HOST': 'localhost',
},
}
I can use the ORM just fine e.g. CustomUser.objects.all(). But the migration issue is a huge problem. A naive solution is to disable the admin but I feel the underlying problem is still there.
Is it possible to use one database for user authentifications in django and another database for working with django-forms? I work with non-relational data structures and feel comfortable with mongodb forms, but I unable to authorize my users, using mongodb. All posts describing mongodb-authentification with django referer to deprecated modules of mongoengine, which is no longer support django. So I decide to use mysql for user authentification and mongodb as main database, but cannot find information about defining special database for user authorization.
UPD:
I add mysql to setings:
DATABASES = {
'default': {
'ENGINE' : 'django.db.backends.dummy',
'NAME' : 'my_database'
},
'users': {
'NAME': 'my_users',
'ENGINE': 'django.db.backends.mysql',
'USER': 'django'
}
}
Migrate:
python3 manage.py migrate --database=users
It return:
Operations to perform:
Apply all migrations: admin, auth, contenttypes, sessions
Running migrations:
Applying contenttypes.0001_initial... OK
Applying auth.0001_initial... OK
Applying admin.0001_initial... OK
Applying admin.0002_logentry_remove_auto_add... OK
Applying contenttypes.0002_remove_content_type_name... OK
Applying auth.0002_alter_permission_name_max_length... OK
Applying auth.0003_alter_user_email_max_length... OK
Applying auth.0004_alter_user_username_opts... OK
Applying auth.0005_alter_user_last_login_null... OK
Applying auth.0006_require_contenttypes_0002... OK
Applying auth.0007_alter_validators_add_error_messages... OK
Applying auth.0008_alter_user_username_max_length... OK
Applying sessions.0001_initial... OK
Then I tried to create a user:
set DJANGO_SETTINGS_MODULE=my_site.settings
,
>>> import django
>>> django.setup()
>>> from django.contrib.auth.models import User
>>> user = User.objects.create_user('john', 'lennon#thebeatles.com', 'johnpassword')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/dist-packages/django/contrib/auth/models.py", line 159, in create_user
return self._create_user(username, email, password, **extra_fields)
File "/usr/local/lib/python3.5/dist-packages/django/contrib/auth/models.py", line 153, in _create_user
user.save(using=self._db)
File "/usr/local/lib/python3.5/dist-packages/django/contrib/auth/base_user.py", line 80, in save
super(AbstractBaseUser, self).save(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/django/db/models/base.py", line 807, in save
force_update=force_update, update_fields=update_fields)
File "/usr/local/lib/python3.5/dist-packages/django/db/models/base.py", line 834, in save_base
with transaction.atomic(using=using, savepoint=False):
File "/usr/local/lib/python3.5/dist-packages/django/db/transaction.py", line 158, in __enter__
if not connection.get_autocommit():
File "/usr/local/lib/python3.5/dist-packages/django/db/backends/base/base.py", line 385, in get_autocommit
self.ensure_connection()
File "/usr/local/lib/python3.5/dist-packages/django/db/backends/dummy/base.py", line 20, in complain
raise ImproperlyConfigured("settings.DATABASES is improperly configured. "
django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.
>>>
Guys, please, advise me, what was the wrong?
UPD2:
Add a router to settings.py:
class AuthRouter(object):
"""
A router to control all database operations on models in the
auth application.
"""
def db_for_read(self, model, **hints):
"""
Attempts to read auth models go to auth_db.
"""
if model._meta.app_label == 'auth':
return 'users'
return 'default'
def db_for_write(self, model, **hints):
"""
Attempts to write auth models go to auth_db.
"""
if model._meta.app_label == 'auth':
return 'users'
return 'default'
def allow_relation(self, obj1, obj2, **hints):
"""
Allow relations if a model in the auth app is involved.
"""
if obj1._meta.app_label == 'auth' or \
obj2._meta.app_label == 'auth':
return True
return False
def allow_migrate(self, db, app_label, model_name=None, **hints):
"""
Make sure the auth app only appears in the 'auth_db'
database.
"""
return db == 'auth_db'
DATABASE_ROUTERS = [AuthRouter]
Now i have enother error, when create user:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/dist-packages/django/contrib/auth/models.py", line 159, in create_user
return self._create_user(username, email, password, **extra_fields)
File "/usr/local/lib/python3.5/dist-packages/django/contrib/auth/models.py", line 153, in _create_user
user.save(using=self._db)
File "/usr/local/lib/python3.5/dist-packages/django/contrib/auth/base_user.py", line 80, in save
super(AbstractBaseUser, self).save(*args, **kwargs)
File "/usr/local/lib/python3.5/dist-packages/django/db/models/base.py", line 766, in save
using = using or router.db_for_write(self.__class__, instance=self)
File "/usr/local/lib/python3.5/dist-packages/django/db/utils.py", line 267, in _route_db
chosen_db = method(model, **hints)
TypeError: db_for_write() missing 1 required positional argument: 'model'
What's wrong in my router? I copied it from oficial docs.
I successfully uploaded my webapp onto webfaction, but I noticed, when using peewee to connect to a MySQL database on my webfaction account, I got this error:
ProgrammingError: (1146, "Table 'TABLEGOESHERE' doesn't exist")
Exact error is in the error log file below
Some background information:
I created a MySQL database on webfaction
I did not create any tables within the control panel provided by the service. It's completely empty.
I can run my flask app successfully through the terminal, but I am about to make it live on a web server, and so I am very new to this process.
I assumed that when you're using peewee you can create tables from within your program like so:
models.py
# -- Peewe Modules
from peewee import *
DATABASE = MySQLDatabase("DBNAMEGOESHERE", host="HOSTGOESHERE", port=PORTGOESHERE, user="USERGOESHERE", passwd="PASSGOESHERE")
# -- DATABASE OBJECTS GO HERE:
#-- INIT
def initialize():
DATABASE.connect()
DATABASE.create_tables([Post, etc...],safe=True)
DATABASE.close()
The initialize function is called in the __init__.py file at the bottom of the file like so:
if __name__ == "__main__":
models.initialize()
try:
models.User.create_user(
username = 'user',
email = 'email',
password = 'pass',
is_admin = True,
confirmed = True,
confirmed_on = datetime.datetime.now(),
)
except ValueError:
pass
app.run()
My index view, which routes to ('/'), in my __init__.py file makes a call to the count method like so:
count = models.Post.select().count()
And I believe this line caused my site to display a 500 internal server error which resulted in this error log (Timestamps have been removed for simplicity):
return self.wsgi_app(environ, start_response)
File "/home/username/webapps/myapp/myapp/__init__.py", line 49, in __call__
return self.app(environ, start_response)
File "/home/username/lib/python2.7/Flask-0.10.1-py2.7.egg/flask/app.py", line 1820, in wsgi_app
response = self.make_response(self.handle_exception(e))
File "/home/username/lib/python2.7/Flask-0.10.1-py2.7.egg/flask/app.py", line 1403, in handle_exception
reraise(exc_type, exc_value, tb)
File "/home/username/lib/python2.7/Flask-0.10.1-py2.7.egg/flask/app.py", line 1817, in wsgi_app
response = self.full_dispatch_request()
File "/home/username/lib/python2.7/Flask-0.10.1-py2.7.egg/flask/app.py", line 1477, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/username/lib/python2.7/Flask-0.10.1-py2.7.egg/flask/app.py", line 1381, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "/home/username/lib/python2.7/Flask-0.10.1-py2.7.egg/flask/app.py", line 1475, in full_dispatch_request
rv = self.dispatch_request()
File "/home/username/lib/python2.7/Flask-0.10.1-py2.7.egg/flask/app.py", line 1461, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/username/webapps/myapp/myapp/__init__.py", line 587, in index
count = models.Post.select().count()
File "/home/username/lib/python2.7/peewee-2.7.3-py2.7.egg/peewee.py", line 2792, in count
return self.aggregate(convert=False) or 0
File "/home/username/lib/python2.7/peewee-2.7.3-py2.7.egg/peewee.py", line 2785, in aggregate
return self._aggregate(aggregation).scalar(convert=convert)
File "/home/username/lib/python2.7/peewee-2.7.3-py2.7.egg/peewee.py", line 2564, in scalar
row = self._execute().fetchone()
File "/home/username/lib/python2.7/peewee-2.7.3-py2.7.egg/peewee.py", line 2555, in _execute
return self.database.execute_sql(sql, params, self.require_commit)
File "/home/username/lib/python2.7/peewee-2.7.3-py2.7.egg/peewee.py", line 3366, in execute_sql
self.commit()
File "/home/username/lib/python2.7/peewee-2.7.3-py2.7.egg/peewee.py", line 3212, in __exit__
reraise(new_type, new_type(*exc_args), traceback)
File "/home/username/lib/python2.7/peewee-2.7.3-py2.7.egg/peewee.py", line 3359, in execute_sql
cursor.execute(sql, params or ())
File "/usr/lib64/python2.7/site-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib64/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
ProgrammingError: (1146, "Table 'DATABASENAMEHERE.post' doesn't exist")
Can anyone help me identify and fix this issue? I have no idea how to get my flask app to cooperate with my MySQL database on webfaction.
Are you sure your application is being run by executing it directly from the command-line? i.e. the __name__ == '__main__' block is actually running? Is it possible you're using a dedicated WSGI server instead?
Moved my server initialization code into a file that runs the Flask app on Webfaction. It is not recommended to place additional code under the conditional as its meant for the command line.