Django 1.6 w/ gunicorn - OperationalError: (2006, 'MySQL server has gone away') - mysql

In the process of upgrading to Django 1.6 I've started to get a frequent OperationalError: (2006, 'MySQL server has gone away') message on requests to the gunicorn server I use to run the django app. These errors occur instantly from the moment the server is started, on requests that should only take a second which makes me doubt that it is a timeout issue. This error isn't present on the old 1.4 branch of the project and the 1.6 branch doesn't behave this way if it's served simply through django-admin.py runserver.
I generally run gunicorn through an sv process (though it errors if I run it manually too) with the command django-admin.py run_gunicorn --workers=4 -b localhost:8000 which results in many requests, even ones for static media, returning with:
[ERROR] 2015-07-29 14:30:27,931 - gunicorn.error:260 - Error handling request
Traceback (most recent call last):
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 125, in handle_request
respiter = self.wsgi(environ, resp.start_response)
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 187, in __call__
self.load_middleware()
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/core/handlers/base.py", line 47, in load_middleware
mw_instance = mw_class()
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/middleware/locale.py", line 24, in __init__
for url_pattern in get_resolver(None).url_patterns:
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 365, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/core/urlresolvers.py", line 360, in urlconf_module
self._urlconf_module = import_module(self.urlconf_name)
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/utils/importlib.py", line 40, in import_module
__import__(name)
File "/opt/apps/maplecroft/versions/current/websites/maplecroft/urls.py", line 2, in <module>
from maplecroft.views import RiskAtlasesLandingView
File "/opt/apps/maplecroft/versions/current/maplecroft/views.py", line 40, in <module>
import maplecroft.search as _search
File "/opt/apps/maplecroft/versions/current/maplecroft/search.py", line 71, in <module>
class MaplecroftSearchForm(SearchForm):
File "/opt/apps/maplecroft/versions/current/maplecroft/search.py", line 111, in MaplecroftSearchForm
choices=model_choices(),
File "/opt/apps/maplecroft/versions/current/maplecroft/search.py", line 57, in model_choices
for category in reversed(Category.objects.filter(parent=None)):
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/db/models/query.py", line 77, in __len__
self._fetch_all()
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/db/models/query.py", line 857, in _fetch_all
self._result_cache = list(self.iterator())
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
for row in compiler.results_iter():
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 713, in results_iter
for rows in self.execute_sql(MULTI):
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 786, in execute_sql
cursor.execute(sql, params)
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/db/backends/util.py", line 69, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
return self.cursor.execute(query, args)
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/opt/envs/maplecroft/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
OperationalError: (2006, 'MySQL server has gone away')
However, if I drop to --workers=1 everything seems to run smoothly so my current thoughts are that it is an issue with the threaded workers feature of gunicorn?
Edit: I just tried upgrading gunicorn to the latest version (was on 0.17.2) but that doesn't seem to have made a difference.
I am wondering if this question: https://serverfault.com/questions/407612/error-2006-mysql-server-has-gone-away is relevant, but struggling to overlay it with my current issues

It appears that it was the gunicorn version after all- I tried upgrading gunicorn but was still using the django-admin.py run_gunicorn method of starting the server. Upgrading gunicorn and switching to the non-deprecated gunicorn wsgi:application method solved it.

Related

Airflow upon starting - MySQL server has gone away

We added a second airflow database to support our staging airflow instance, and now both staging and production seem to have intermittent connection issues. We recently upgraded to 2.2.4, but looking through past RDS logs, I see aborted connections (Got an error reading communication packets) prior to the upgrade as well (previous version 1.10.11). Both instances have a webserver UI that pull in past DAG runs fine, and the scheduler is running DAGs appropriately and storing dag_run data. When the service is started however, we receive an error "MySQL server has gone away" (see below for stack trace). airflow db check and airflow db shell both return successful connections. Some relevant airflow config values are:
sql_alchemy_pool_enabled = True
sql_alchemy_pool_size = 5
sql_alchemy_max_overflow = 10
sql_alchemy_pool_recycle = 1800
sql_alchemy_pool_pre_ping = True
sql_alchemy_schema =
sql_alchemy_reconnect_timeout = 300
I've also tried upping the pool size to 20, and also confirmed that our db size should be able to handle at least 90 concurrent connections.
[2022-04-28 16:32:03,212] {manager.py:512} WARNING - Refused to delete permission view, assoc with role exists DAG Runs.can_create Admin
Process ForkProcess-19:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 206, in execute
res = self._query(query)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 319, in _query
db.query(q)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 254, in query
_mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (2006, 'MySQL server has gone away')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/local/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.6/site-packages/airflow/dag_processing/manager.py", line 287, in _run_processor_manager
processor_manager.start()
File "/usr/local/lib/python3.6/site-packages/airflow/dag_processing/manager.py", line 520, in start
return self._run_parsing_loop()
File "/usr/local/lib/python3.6/site-packages/airflow/dag_processing/manager.py", line 585, in _run_parsing_loop
self._find_zombies()
File "/usr/local/lib/python3.6/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
File "/usr/local/lib/python3.6/site-packages/airflow/dag_processing/manager.py", line 1079, in _find_zombies
LJ.latest_heartbeat < limit_dttm,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3373, in all
return list(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3535, in __iter__
return self._execute_and_instances(context)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3560, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1130, in _execute_clauseelement
distilled_params,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 206, in execute
res = self._query(query)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 319, in _query
db.query(q)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 254, in query
_mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (2006, 'MySQL server has gone away')
[SQL: SELECT task_instance.try_number AS task_instance_try_number, task_instance.task_id AS task_instance_task_id, task_instance.dag_id AS task_instance_dag_id, task_instance.run_id AS task_instance_run_id, task_instance.start_date AS task_instance_start_date, task_instance.end_date AS task_instance_end_date, task_instance.duration AS task_instance_duration, task_instance.state AS task_instance_state, task_instance.max_tries AS task_instance_max_tries, task_instance.hostname AS task_instance_hostname, task_instance.unixname AS task_instance_unixname, task_instance.job_id AS task_instance_job_id, task_instance.pool AS task_instance_pool, task_instance.pool_slots AS task_instance_pool_slots, task_instance.queue AS task_instance_queue, task_instance.priority_weight AS task_instance_priority_weight, task_instance.operator AS task_instance_operator, task_instance.queued_dttm AS task_instance_queued_dttm, task_instance.queued_by_job_id AS task_instance_queued_by_job_id, task_instance.pid AS task_instance_pid, task_instance.executor_config AS task_instance_executor_config, task_instance.external_executor_id AS task_instance_external_executor_id, task_instance.trigger_id AS task_instance_trigger_id, task_instance.trigger_timeout AS task_instance_trigger_timeout, task_instance.next_method AS task_instance_next_method, task_instance.next_kwargs AS task_instance_next_kwargs, dag.fileloc AS dag_fileloc, dag_run_1.state AS dag_run_1_state, dag_run_1.id AS dag_run_1_id, dag_run_1.dag_id AS dag_run_1_dag_id, dag_run_1.queued_at AS dag_run_1_queued_at, dag_run_1.execution_date AS dag_run_1_execution_date, dag_run_1.start_date AS dag_run_1_start_date, dag_run_1.end_date AS dag_run_1_end_date, dag_run_1.run_id AS dag_run_1_run_id, dag_run_1.creating_job_id AS dag_run_1_creating_job_id, dag_run_1.external_trigger AS dag_run_1_external_trigger, dag_run_1.run_type AS dag_run_1_run_type, dag_run_1.conf AS dag_run_1_conf, dag_run_1.data_interval_start AS dag_run_1_data_interval_start, dag_run_1.data_interval_end AS dag_run_1_data_interval_end, dag_run_1.last_scheduling_decision AS dag_run_1_last_scheduling_decision, dag_run_1.dag_hash AS dag_run_1_dag_hash
FROM task_instance INNER JOIN job ON task_instance.job_id = job.id AND job.job_type IN (%s) INNER JOIN dag ON task_instance.dag_id = dag.dag_id INNER JOIN dag_run AS dag_run_1 ON dag_run_1.dag_id = task_instance.dag_id AND dag_run_1.run_id = task_instance.run_id
WHERE task_instance.state = %s AND (job.state != %s OR job.latest_heartbeat < %s)]
[parameters: ('LocalTaskJob', <TaskInstanceState.RUNNING: 'running'>, <TaskInstanceState.RUNNING: 'running'>, datetime.datetime(2022, 4, 28, 16, 27, 2, 870459))]
(Background on this error at: http://sqlalche.me/e/13/e3q8)```

How to fix AttributeError: 'bytes' object has no attribute 'encode' if using mysql?

Django development server was running fine using mysql up to yesterday. But today I get the error AttributeError: 'bytes' object has no attribute 'encode'.
I created a new blank django instance by running django-admin startproject newProject. It runs with 'ENGINE': 'django.db.backends.sqlite3'. But produces same error when I change it to 'ENGINE': 'django.db.backends.mysql'
System specification:
os: manjaro linux
virtual environment python version: 3.6
Django version: 2.2
MariaDB version: 10.3.15-1
Have not updated any related package in last 3 days.
The error:
Watching for file changes with StatReloader
Performing system checks...
System check identified no issues (0 silenced).
Exception in thread django-main-thread:
Traceback (most recent call last):
File "/usr/lib64/python3.6/threading.py", line 916, in _bootstrap_inner
self.run()
File "/usr/lib64/python3.6/threading.py", line 864, in run
self._target(*self._args, **self._kwargs)
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/utils/autoreload.py", line 54, in wrapper
fn(*args, **kwargs)
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/core/management/commands/runserver.py", line 120, in inner_run
self.check_migrations()
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/core/management/base.py", line 453, in check_migrations
executor = MigrationExecutor(connections[DEFAULT_DB_ALIAS])
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/db/migrations/executor.py", line 18, in __init__
self.loader = MigrationLoader(self.connection)
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/db/migrations/loader.py", line 49, in __init__
self.build_graph()
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/db/migrations/loader.py", line 212, in build_graph
self.applied_migrations = recorder.applied_migrations()
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 73, in applied_migrations
if self.has_table():
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/db/migrations/recorder.py", line 56, in has_table
return self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor())
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/db/backends/base/base.py", line 256, in cursor
return self._cursor()
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/db/backends/base/base.py", line 233, in _cursor
self.ensure_connection()
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/db/backends/base/base.py", line 217, in ensure_connection
self.connect()
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/db/backends/base/base.py", line 197, in connect
self.init_connection_state()
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 232, in init_connection_state
if self.features.is_sql_auto_is_null_enabled:
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/utils/functional.py", line 80, in __get__
res = instance.__dict__[self.name] = self.func(instance)
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/db/backends/mysql/features.py", line 82, in is_sql_auto_is_null_enabled
cursor.execute('SELECT ##SQL_AUTO_IS_NULL')
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/db/backends/utils.py", line 103, in execute
sql = self.db.ops.last_executed_query(self.cursor, sql, params)
File "/srv/http/python/env/env36/lib/python3.6/site-packages/django/db/backends/mysql/operations.py", line 147, in last_executed_query
query = query.encode(errors='replace')
AttributeError: 'bytes' object has no attribute 'encode'
How do I fix this ?
Thank to Martijn, I have fixed this.
In the source, stable 2.2 branch line 146 of django/db/backends/mysql/operations.py is
query = query.decode(errors='replace')
In my system path/to/virtualenv/django/db/backends/mysql/operations.py was like
# query = query.decode(errors='replace')
query = query.encode(errors='replace')
I changed it to
query = query.decode(errors='replace')
# query = query.encode(errors='replace')
It is working correctly now.

1146 "app_name.django_site missing"

I am using Django 1.11 for making an app 'cnfs', and I am using MYSQL database with it. I am constantly facing this issue where I am getting an error like this when I type the following code:
$python manage.py migrate
System check identified some issues: WARNINGS: ?: (mysql.W002) MySQL Strict Mode is not set for database connection 'default' HINT: MySQL's Strict Mode fixes many data integrity problems in MySQL, such as data truncation upon insertion, by escalating warnings into errors. It is strongly recommended you activate it. See: https://docs.djangoproject.com/en/1.11/ref/databases/#mysql-sql-mode Operations to perform: Apply all migrations: admin, auth, cnfs, contenttypes, sites Running migrations: No migrations to apply. Traceback (most recent call last): File "manage.py", line 22, in execute_from_command_line(sys.argv) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/core/management/init.py", line 363, in execute_from_command_line utility.execute() File "/home/ubuntu/.local/lib/python2.7/site-packages/django/core/management/init.py", line 355, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv self.execute(*args, **cmd_options) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute output = self.handle(*args, **options) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 227, in handle self.verbosity, self.interactive, connection.alias, apps=post_migrate_apps, plan=plan, File "/home/ubuntu/.local/lib/python2.7/site-packages/django/core/management/sql.py", line 53, in emit_post_migrate_signal **kwargs File "/home/ubuntu/.local/lib/python2.7/site-packages/django/dispatch/dispatcher.py", line 193, in send for receiver in self._live_receivers(sender) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/contrib/sites/management.py", line 20, in create_default_site if not Site.objects.using(using).exists(): File "/home/ubuntu/.local/lib/python2.7/site-packages/django/db/models/query.py", line 670, in exists return self.query.has_results(using=self.db) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 517, in has_results return compiler.has_results() File "/home/ubuntu/.local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 845, in has_results return bool(self.execute_sql(SINGLE)) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 886, in execute_sql raise original_exception django.db.utils.ProgrammingError: (1146, "Table 'cnfs.django_site' doesn't exist")
I have literally spent days on this, I did not face this issue when i was using the default django database engine, sqlite3.
I have tried doing this:
$python manage.py migrate sites
My SITE_ID setting is set to 1.
I tried removing the 'django.contrib.sites' thing from the INSTALLED_APPS setting, but that throws this error:
Internal Server Error: / Traceback (most recent call last): File "/home/ubuntu/.local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner response = get_response(request) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/utils/deprecation.py", line 138, in call response = self.process_request(request) File "/home/ubuntu/.local/lib/python2.7/site-packages/subdomains/middleware.py", line 62, in process_request super(SubdomainURLRoutingMiddleware, self).process_request(request) File "/home/ubuntu/.local/lib/python2.7/site-packages/subdomains/middleware.py", line 38, in process_request (self.get_domain_for_request(request), request.get_host())) File "/home/ubuntu/.local/lib/python2.7/site-packages/subdomains/middleware.py", line 31, in get_domain_for_request return get_domain() File "/home/ubuntu/.local/lib/python2.7/site-packages/subdomains/utils.py", line 12, in current_site_domain from django.contrib.sites.models import Site File "/home/ubuntu/.local/lib/python2.7/site-packages/django/contrib/sites/models.py", line 84, in class Site(models.Model): File "/home/ubuntu/.local/lib/python2.7/site-packages/django/db/models/base.py", line 118, in new "INSTALLED_APPS." % (module, name) RuntimeError: Model class django.contrib.sites.models.Site doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.
Any suggestions would be much appreciated.
Thanks.
uncomment the django.contrib.sites from installed apps
then
python manage.py migrate sites
python manage.py migrate
if you have few table in the database, drop the tables and then run this

Celery Django Error in timer: OperationalError(2006, 'MySQL server has gone away')

I am using Django and celery to run some tasks. i have 6 workers with concurrency 1, and different tasks are sent to different workers. one of witch raises a Error in timer: OperationalError(2006, 'MySQL server has gone away').
system runs normally until something happens and worker stops running tasks until manually restarted.
I have:
Django==1.6
celery==3.1.5
django-celery==3.1.1
MySQL-python==1.2.4
kombu==3.0.6
Traceback:
[2015-12-10 09:50:43,300: ERROR/MainProcess] Error in timer: OperationalError(2006, 'MySQL server has gone away')
Traceback (most recent call last):
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/kombu/async/timer.py", line 171, in apply_entry
entry()
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/kombu/async/timer.py", line 64, in __call__
return self.fun(*self.args, **self.kwargs)
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/kombu/async/timer.py", line 132, in _reschedules
return fun(*args, **kwargs)
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/celery/events/snapshot.py", line 73, in capture
self.state.freeze_while(self.shutter, clear_after=self.clear_after)
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/celery/events/state.py", line 428, in freeze_while
return fun(*args, **kwargs)
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/celery/events/snapshot.py", line 70, in shutter
self.on_shutter(self.state)
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/djcelery/snapshot.py", line 139, in on_shutter
_handle_tasks()
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/djcelery/snapshot.py", line 135, in _handle_tasks
self.handle_task(task)
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/djcelery/snapshot.py", line 82, in handle_task
(task.worker.hostname, task.worker),
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/djcelery/snapshot.py", line 72, in handle_worker
defaults={'last_heartbeat': self.get_heartbeat(worker)},
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/djcelery/managers.py", line 87, in update_or_create
return get_queryset(self).update_or_create(**kwargs)
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/djcelery/managers.py", line 70, in update_or_create
obj, created = self.get_or_create(**kwargs)
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/django/db/models/query.py", line 373, in get_or_create
return self.get(**lookup), False
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/django/db/models/query.py", line 301, in get
num = len(clone)
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/django/db/models/query.py", line 77, in __len__
self._fetch_all()
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/django/db/models/query.py", line 854, in _fetch_all
self._result_cache = list(self.iterator())
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/django/db/models/query.py", line 220, in iterator
for row in compiler.results_iter():
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 710, in results_iter
for rows in self.execute_sql(MULTI):
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 781, in execute_sql
cursor.execute(sql, params)
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/django/db/utils.py", line 99, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/django/db/backends/util.py", line 53, in execute
return self.cursor.execute(sql, params)
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
return self.cursor.execute(query, args)
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File "/opt/virtualenvs/simocEnv/local/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
OperationalError: (2006, 'MySQL server has gone away')
if you use flask, you can use db.session.close() before some CRUD operator.
Reference:
OperationalError: (2006, 'MySQL server has gone away after upgrading from celery 4.1.0 to 4.2.0')
OperationalError: (2006, 'MySQL server has gone away')

Django: "Unknown Column" when run makemigrations on an app after adding an Integer Field

First of all, I have recently upgraded from Django 1.6 to 1.8, and never used South, so I'm new to migrations.
I just ran:
> python manage.py makemigrations myapp
> python manage.py migrate --fake initial
to create the initial migrations for my model with existing MySQL tables. All seemed good so far.
I then added an IntegerField to my model:
new_integer_field = models.IntegerField(default = 50) #can include blank=True, null=True; seems to make no difference
Now when I run:
>python manage.py makemigrations myapp
I get
django.db.utils.OperationalError: (1054, "Unknown column 'myapp_mymodel.new_integer_field' in 'field list'")
The traceback (starting where the problem occured) is:
Traceback (most recent call last):
File "./manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File ".../django/core/management/__init__.py", line 338, in execute_from_command_line
utility.execute()
File ".../django/core/management/__init__.py", line 312, in execute
django.setup()
File ".../django/__init__.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File ".../django/apps/registry.py", line 115, in populate
app_config.ready()
File ".../autocomplete_light/apps.py", line 9, in ready
autodiscover()
File ".../autocomplete_light/registry.py", line 298, in autodiscover
autodiscover_modules('autocomplete_light_registry')
File ".../django/utils/module_loading.py", line 74, in autodiscover_modules
import_module('%s.%s' % (app_config.name, module_to_search))
File ".../python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File ".../mymapp/autocomplete_light_registry.py", line 42, in <module>
x = other_model.other_model_manager.manager_function(),
File ".../myapp/models.py", line 759, in get_a_queryset
stus = a_queryset
File ".../myapp/models.py", line 92, in get_another_queryset
if obj.model_function(prog):
File ".../myapp/models.py", line 402, in model_function
z = object.MyManagerObjects.manager_function(self)
File ".../myapp/models.py", line 573, in get_type
curstart = mymodel.MyManagerObjects.get().old_field
File ".../python2.7/site-packages/django/db/models/manager.py", line 127, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File ".../python2.7/site-packages/django/db/models/query.py", line 328, in get
num = len(clone)
File ".../python2.7/site-packages/django/db/models/query.py", line 144, in __len__
self._fetch_all()
File ".../python2.7/site-packages/django/db/models/query.py", line 965, in _fetch_all
self._result_cache = list(self.iterator())
File ".../python2.7/site-packages/django/db/models/query.py", line 238, in iterator
results = compiler.execute_sql()
File ".../python2.7/site-packages/django/db/models/sql/compiler.py", line 840, in execute_sql
cursor.execute(sql, params)
File ".../python2.7/site-packages/django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File ".../python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File ".../python2.7/site-packages/django/db/utils.py", line 97, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File ".../python2.7/site-packages/django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File ".../python2.7/site-packages/django/db/backends/mysql/base.py", line 124, in execute
return self.cursor.execute(query, args)
File ".../python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File ".../python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
django.db.utils.OperationalError: (1054, "Unknown column 'myapp_mymodel.new_integer_field' in 'field list'")
MyManager is simple, as follows:
class MyManager(models.Manager):
def get_queryset(self):
return super(MyManager, self).get_queryset().filter(...some filters...) #this is guaranteed to return a queryset with one object in it.
And inside myapp/models.py:
MyManagerObjects = MyManager()
So, my question is, why can't I add a field to mymodel and run makemigrations? What is breaking in the manager? I'm starting the stacktrace there because I've tried a variety of different commenting out strategies that seem to point to this manager regardless of what the prior calls are.
I'm sure hoping someone can tell me it's something quick and simple that I'm just not seeing, but I'm not sure how that will be the case! Anyone else had this trouble or have any suggestions?
----****----
Update 1: the error doesn't just occur with makemigrations - since some commentors asked what happens when I run python manage.py migrate I thought I'd try it for things and giggles, even though there is nothing to migrate. I was expecting to be told there was nothing to migrate, but I got the same error. So it's not makemigrations code itself; it's something not able to function because it can't find the new field.
Somehow it is looking at the model and finding a field that it doesn't recognize. But why?!
----****----
Update 2: I added the start of the traceback... I'm loathe to post any of the actual paths/model names, hence the edits. Hope that's ok. A note - it is not, as far as I can tell, related to autocomplete_light (unless it is!) because when I comment out the line x = other_model.other_model_manager.manager_function() (line 42 in autocomplete_light_registry.py) the error occurs without reference to autocomplete_light. I've added a second traceback here in case that helps too!
Traceback (most recent call last):
File "./manage.py", line 10, in <module>
execute_from_command_line(sys.argv)
File ".../django/core/management/__init__.py", line 338, in execute_from_command_line
utility.execute()
File ".../django/core/management/__init__.py", line 312, in execute
django.setup()
File ".../django/__init__.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File ".../django/apps/registry.py", line 115, in populate
app_config.ready()
File ".../django/contrib/admin/apps.py", line 22, in ready
self.module.autodiscover()
File ".../django/contrib/admin/__init__.py", line 24, in autodiscover
autodiscover_modules('admin', register_to=site)
File ".../django/utils/module_loading.py", line 74, in autodiscover_modules
import_module('%s.%s' % (app_config.name, module_to_search))
File ".../python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File ".../myapp/admin.py", line 151, in <module>
class MyListFilter(SimpleListFilterWithDefault):
File ".../myapp/admin.py", line 154, in MyListFilter
x = mymodel.MyManagerObjects.get()
File ".../django/db/models/manager.py", line 127, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File ".../django/db/models/query.py", line 328, in get
num = len(clone)
File ".../django/db/models/query.py", line 144, in __len__
self._fetch_all()
File ".../django/db/models/query.py", line 965, in _fetch_all
self._result_cache = list(self.iterator())
File ".../django/db/models/query.py", line 238, in iterator
results = compiler.execute_sql()
File ".../django/db/models/sql/compiler.py", line 840, in execute_sql
cursor.execute(sql, params)
File ".../django/db/backends/utils.py", line 79, in execute
return super(CursorDebugWrapper, self).execute(sql, params)
File ".../django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File ".../django/db/utils.py", line 97, in __exit__
six.reraise(dj_exc_type, dj_exc_value, traceback)
File ".../django/db/backends/utils.py", line 64, in execute
return self.cursor.execute(sql, params)
File ".../django/db/backends/mysql/base.py", line 124, in execute
return self.cursor.execute(query, args)
File ".../MySQLdb/cursors.py", line 205, in execute
self.errorhandler(self, exc, value)
File ".../MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
django.db.utils.OperationalError: (1054, "Unknown column 'myapp_mymodel.new_integer_field' in 'field list'")
It looks like you are making a db query either when your module is imported or when the app is being registered:
curstart = mymodel.MyManagerObjects.get().old_field
This means when you run any management command like runserver, makemigrations or migrate, if you've changed your models then they will be out of sync with your database, and making the query will throw an exception before the command can do what it's supposed to (like make your migrations).
If possible, modify your code so MyManagerObjects.get() isn't called at load time, eg by using a lambda function or wrapping it in a method that you can call at run time.
If that's not possible, you'll need to comment out that section of the code every time you make or run migrations, but that's really messy.
Update:
In the full traceback there's the line:
File ".../myapp/admin.py", line 154, in MyListFilter
x = mymodel.MyManagerObjects.get()
which means the get is being run when the admin file is imported. I think you may need to put the call to mymodel.MyManagerObjects.get() in MyListFilter.queryset() instead of in the class declaration.
https://docs.djangoproject.com/en/1.8/ref/contrib/admin/#django.contrib.admin.ModelAdmin.list_filter
It shouldn't be:
python manage.py makemigrations mymodel
it should be:
python manage.py makemigrations myapp
So comment out new_integer_field and do it again I think.
You also need to run migrate to add the new_integer_fieldcolumn in the database table.
python manage.py makemigrations myapp
./manage.py migrate --fake
# add new_integer_field
./manage.py makemigrations myapp
./manage.py migrate