SQLAlchemy and SQLite errors appearing on PythonAnywhere but it was fine on localhost.
So I just uploaded my Flask site to pythonanywhere, and when I tried to access it it can't access my database, but is getting a bunch of errors instead.
2021-01-30 10:42:05,740: Exception on /m/550 [GET]
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
self.dialect.do_execute(
File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 580, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: near "(": syntax error
**NO MATCH**
I was not getting these exceptions on my local computer. A little googling and it seems like this is because PA uses an older version of SQLite? Which seems a bit stupid to begin with but it also seems like I can't update it?
So I'm wondering if there is some smart solution to this, or if it would perhaps be easier to switch to MySQL? Would there be other problems in doing that instead? Or do I just need to switch out the SQLALCHEMY_DATABASE_URI to fit MySQL instead? My Current one looks like this:
SQLALCHEMY_DATABASE_URI = 'sqlite:////home/movieelo/cinelow/my_server/database/database.db'
SQLALCHEMY_TRACK_MODIFICATIONS = False
All help and guidence will be greatly appreciated!
UPDATE:
I managed to switch to MySQL but am still getting SQL syntax related errors, such as:
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in
your SQL syntax; check the manual that corresponds to your MySQL server
version for the right syntax to use near '(PARTITION BY
movie_category_scores.category_id ORDER BY movie_category_scores.s' at line 2
**NO MATCH**
UPDATE 2
I think that I have now identiefied the single function that is causing the errors, and it looks like this:
def get_top_movies_by_category(category_id):
query = db.session.query(
MovieCategoryScores,
func.rank()\
.over(
order_by=MovieCategoryScores.score.desc(),
partition_by=MovieCategoryScores.category_id,
)\
.label('rank')
).filter(MovieCategoryScores.votes >= 10)
# now filter
query = query.filter(MovieCategoryScores.category_id == category_id)
query = query.order_by(MovieCategoryScores.category_id, 'rank')
movies = query.all()
return movies
Which generates this error:
Traceback (most recent call last):
File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
self.dialect.do_execute(
File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 580, in do_execute
cursor.execute(statement, parameters)
sqlite3.OperationalError: near "(": syntax error
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/movieelo/cinelow/my_server/database/pers_movie_dbf.py", line 137, in get_top_movies_by_category
movies = query.all()
File "/usr/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3186, in all
return list(self)
File "/usr/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3342, in __iter__
return self._execute_and_instances(context)
File "/usr/lib/python3.8/site-packages/sqlalchemy/orm/query.py", line 3367, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 988, in execute
return meth(self, multiparams, params)
File "/usr/lib/python3.8/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1101, in _execute_clauseelement
ret = self._execute_context(
File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1252, in _execute_context
self._handle_dbapi_exception(
File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1473, in _handle_dbapi_exception
util.raise_from_cause(sqlalchemy_exception, exc_info)
File "/usr/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 398, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/lib/python3.8/site-packages/sqlalchemy/util/compat.py", line 152, in reraise
raise value.with_traceback(tb)
File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context
self.dialect.do_execute(
File "/usr/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 580, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) near "(": syntax error
[SQL: SELECT movie_category_scores.votes >= ? AS anon_1, movie_category_scores.movie_id AS movie_category_scores_movie_id, movie_category_scores.category_id AS movie_category_scores_category_id, movie_category_scores.score AS movie_category_scores_score, movie_category_scores.votes AS movie_category_scores_votes, rank() OVER (PARTITION BY movie_category_scores.category_id ORDER BY movie_category_scores.score DESC) AS rank
FROM movie_category_scores
WHERE movie_category_scores.votes >= ? AND movie_category_scores.category_id = ? ORDER BY movie_category_scores.category_id, rank]
[parameters: (10, 10, 11)]
(Background on this error at: http://sqlalche.me/e/e3q8)
Giles Thomas (of PythonAnywhere) confirmed that the SQLite version available on PythonAnywhere doesn't support window functions.
Window functions were added to SQLite version 3.25.0. In the future, you can check whether the SQLite version included with your Python installation supports windows functions by running:
import sqlite3
if sqlite3.sqlite_version_info > (3, 25, 0):
print("Window functions are supported!")
else:
print("Window functions are not supported :(")
Related
We added a second airflow database to support our staging airflow instance, and now both staging and production seem to have intermittent connection issues. We recently upgraded to 2.2.4, but looking through past RDS logs, I see aborted connections (Got an error reading communication packets) prior to the upgrade as well (previous version 1.10.11). Both instances have a webserver UI that pull in past DAG runs fine, and the scheduler is running DAGs appropriately and storing dag_run data. When the service is started however, we receive an error "MySQL server has gone away" (see below for stack trace). airflow db check and airflow db shell both return successful connections. Some relevant airflow config values are:
sql_alchemy_pool_enabled = True
sql_alchemy_pool_size = 5
sql_alchemy_max_overflow = 10
sql_alchemy_pool_recycle = 1800
sql_alchemy_pool_pre_ping = True
sql_alchemy_schema =
sql_alchemy_reconnect_timeout = 300
I've also tried upping the pool size to 20, and also confirmed that our db size should be able to handle at least 90 concurrent connections.
[2022-04-28 16:32:03,212] {manager.py:512} WARNING - Refused to delete permission view, assoc with role exists DAG Runs.can_create Admin
Process ForkProcess-19:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 206, in execute
res = self._query(query)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 319, in _query
db.query(q)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 254, in query
_mysql.connection.query(self, query)
MySQLdb._exceptions.OperationalError: (2006, 'MySQL server has gone away')
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/multiprocessing/process.py", line 258, in _bootstrap
self.run()
File "/usr/local/lib/python3.6/multiprocessing/process.py", line 93, in run
self._target(*self._args, **self._kwargs)
File "/usr/local/lib/python3.6/site-packages/airflow/dag_processing/manager.py", line 287, in _run_processor_manager
processor_manager.start()
File "/usr/local/lib/python3.6/site-packages/airflow/dag_processing/manager.py", line 520, in start
return self._run_parsing_loop()
File "/usr/local/lib/python3.6/site-packages/airflow/dag_processing/manager.py", line 585, in _run_parsing_loop
self._find_zombies()
File "/usr/local/lib/python3.6/site-packages/airflow/utils/session.py", line 70, in wrapper
return func(*args, session=session, **kwargs)
File "/usr/local/lib/python3.6/site-packages/airflow/dag_processing/manager.py", line 1079, in _find_zombies
LJ.latest_heartbeat < limit_dttm,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3373, in all
return list(self)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3535, in __iter__
return self._execute_and_instances(context)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/orm/query.py", line 3560, in _execute_and_instances
result = conn.execute(querycontext.statement, self._params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1130, in _execute_clauseelement
distilled_params,
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1317, in _execute_context
e, statement, parameters, cursor, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1511, in _handle_dbapi_exception
sqlalchemy_exception, with_traceback=exc_info[2], from_=e
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
raise exception
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/base.py", line 1277, in _execute_context
cursor, statement, parameters, context
File "/usr/local/lib/python3.6/site-packages/sqlalchemy/engine/default.py", line 608, in do_execute
cursor.execute(statement, parameters)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 206, in execute
res = self._query(query)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/cursors.py", line 319, in _query
db.query(q)
File "/usr/local/lib/python3.6/site-packages/MySQLdb/connections.py", line 254, in query
_mysql.connection.query(self, query)
sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (2006, 'MySQL server has gone away')
[SQL: SELECT task_instance.try_number AS task_instance_try_number, task_instance.task_id AS task_instance_task_id, task_instance.dag_id AS task_instance_dag_id, task_instance.run_id AS task_instance_run_id, task_instance.start_date AS task_instance_start_date, task_instance.end_date AS task_instance_end_date, task_instance.duration AS task_instance_duration, task_instance.state AS task_instance_state, task_instance.max_tries AS task_instance_max_tries, task_instance.hostname AS task_instance_hostname, task_instance.unixname AS task_instance_unixname, task_instance.job_id AS task_instance_job_id, task_instance.pool AS task_instance_pool, task_instance.pool_slots AS task_instance_pool_slots, task_instance.queue AS task_instance_queue, task_instance.priority_weight AS task_instance_priority_weight, task_instance.operator AS task_instance_operator, task_instance.queued_dttm AS task_instance_queued_dttm, task_instance.queued_by_job_id AS task_instance_queued_by_job_id, task_instance.pid AS task_instance_pid, task_instance.executor_config AS task_instance_executor_config, task_instance.external_executor_id AS task_instance_external_executor_id, task_instance.trigger_id AS task_instance_trigger_id, task_instance.trigger_timeout AS task_instance_trigger_timeout, task_instance.next_method AS task_instance_next_method, task_instance.next_kwargs AS task_instance_next_kwargs, dag.fileloc AS dag_fileloc, dag_run_1.state AS dag_run_1_state, dag_run_1.id AS dag_run_1_id, dag_run_1.dag_id AS dag_run_1_dag_id, dag_run_1.queued_at AS dag_run_1_queued_at, dag_run_1.execution_date AS dag_run_1_execution_date, dag_run_1.start_date AS dag_run_1_start_date, dag_run_1.end_date AS dag_run_1_end_date, dag_run_1.run_id AS dag_run_1_run_id, dag_run_1.creating_job_id AS dag_run_1_creating_job_id, dag_run_1.external_trigger AS dag_run_1_external_trigger, dag_run_1.run_type AS dag_run_1_run_type, dag_run_1.conf AS dag_run_1_conf, dag_run_1.data_interval_start AS dag_run_1_data_interval_start, dag_run_1.data_interval_end AS dag_run_1_data_interval_end, dag_run_1.last_scheduling_decision AS dag_run_1_last_scheduling_decision, dag_run_1.dag_hash AS dag_run_1_dag_hash
FROM task_instance INNER JOIN job ON task_instance.job_id = job.id AND job.job_type IN (%s) INNER JOIN dag ON task_instance.dag_id = dag.dag_id INNER JOIN dag_run AS dag_run_1 ON dag_run_1.dag_id = task_instance.dag_id AND dag_run_1.run_id = task_instance.run_id
WHERE task_instance.state = %s AND (job.state != %s OR job.latest_heartbeat < %s)]
[parameters: ('LocalTaskJob', <TaskInstanceState.RUNNING: 'running'>, <TaskInstanceState.RUNNING: 'running'>, datetime.datetime(2022, 4, 28, 16, 27, 2, 870459))]
(Background on this error at: http://sqlalche.me/e/13/e3q8)```
Hi I'm new to the PYTHON and MYSQL. I'm trying to update the data base using the python where I'm taking user input and trying to update into the database.
Can any buddy help me in this error.
Traceback (most recent call last):
File "C:\Users\YGIN\Anaconda3\envs\MyTest\lib\site-packages\mysql\connector\network.py", line 148, in send_plain
self.sock.sendall(packet)
File "C:\Users\YGIN\Anaconda3\envs\MyTest\lib\ssl.py", line 1037, in sendall
return super().sendall(data, flags)
OSError: [WinError 10038] An operation was attempted on something that is not a socket
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/YGIN/PycharmProjects/MyTest/BankApplication.py", line 181, in <module>
c1.customer_closing_account()
File "C:/Users/YGIN/PycharmProjects/MyTest/BankApplication.py", line 146, in customer_closing_account
dbq.delete_cust_account(self.account_no)
File "C:\Users\YGIN\PycharmProjects\MyTest\DB_queryies.py", line 71, in delete_cust_account
my_cursor.execute(mysqlinsertquery, recordstodel)
File "C:\Users\YGIN\Anaconda3\envs\MyTest\lib\site-packages\mysql\connector\cursor.py", line 569, in execute
self._handle_result(self._connection.cmd_query(stmt))
File "C:\Users\YGIN\Anaconda3\envs\MyTest\lib\site-packages\mysql\connector\connection.py", line 590, in cmd_query
result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
File "C:\Users\YGIN\Anaconda3\envs\MyTest\lib\site-packages\mysql\connector\connection.py", line 344, in _send_cmd
packet_number, compressed_packet_number)
File "C:\Users\YGIN\Anaconda3\envs\MyTest\lib\site-packages\mysql\connector\network.py", line 151, in send_plain
errno=2055, values=(self.get_address(), _strioerror(err)))
mysql.connector.errors.OperationalError: 2055: Lost connection to MySQL server at 'localhost:3306', system error: 10038 An operation was attempted on something that is not a socket
I am using Django 1.11 for making an app 'cnfs', and I am using MYSQL database with it. I am constantly facing this issue where I am getting an error like this when I type the following code:
$python manage.py migrate
System check identified some issues: WARNINGS: ?: (mysql.W002) MySQL Strict Mode is not set for database connection 'default' HINT: MySQL's Strict Mode fixes many data integrity problems in MySQL, such as data truncation upon insertion, by escalating warnings into errors. It is strongly recommended you activate it. See: https://docs.djangoproject.com/en/1.11/ref/databases/#mysql-sql-mode Operations to perform: Apply all migrations: admin, auth, cnfs, contenttypes, sites Running migrations: No migrations to apply. Traceback (most recent call last): File "manage.py", line 22, in execute_from_command_line(sys.argv) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/core/management/init.py", line 363, in execute_from_command_line utility.execute() File "/home/ubuntu/.local/lib/python2.7/site-packages/django/core/management/init.py", line 355, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv self.execute(*args, **cmd_options) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute output = self.handle(*args, **options) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 227, in handle self.verbosity, self.interactive, connection.alias, apps=post_migrate_apps, plan=plan, File "/home/ubuntu/.local/lib/python2.7/site-packages/django/core/management/sql.py", line 53, in emit_post_migrate_signal **kwargs File "/home/ubuntu/.local/lib/python2.7/site-packages/django/dispatch/dispatcher.py", line 193, in send for receiver in self._live_receivers(sender) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/contrib/sites/management.py", line 20, in create_default_site if not Site.objects.using(using).exists(): File "/home/ubuntu/.local/lib/python2.7/site-packages/django/db/models/query.py", line 670, in exists return self.query.has_results(using=self.db) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/db/models/sql/query.py", line 517, in has_results return compiler.has_results() File "/home/ubuntu/.local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 845, in has_results return bool(self.execute_sql(SINGLE)) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/db/models/sql/compiler.py", line 886, in execute_sql raise original_exception django.db.utils.ProgrammingError: (1146, "Table 'cnfs.django_site' doesn't exist")
I have literally spent days on this, I did not face this issue when i was using the default django database engine, sqlite3.
I have tried doing this:
$python manage.py migrate sites
My SITE_ID setting is set to 1.
I tried removing the 'django.contrib.sites' thing from the INSTALLED_APPS setting, but that throws this error:
Internal Server Error: / Traceback (most recent call last): File "/home/ubuntu/.local/lib/python2.7/site-packages/django/core/handlers/exception.py", line 41, in inner response = get_response(request) File "/home/ubuntu/.local/lib/python2.7/site-packages/django/utils/deprecation.py", line 138, in call response = self.process_request(request) File "/home/ubuntu/.local/lib/python2.7/site-packages/subdomains/middleware.py", line 62, in process_request super(SubdomainURLRoutingMiddleware, self).process_request(request) File "/home/ubuntu/.local/lib/python2.7/site-packages/subdomains/middleware.py", line 38, in process_request (self.get_domain_for_request(request), request.get_host())) File "/home/ubuntu/.local/lib/python2.7/site-packages/subdomains/middleware.py", line 31, in get_domain_for_request return get_domain() File "/home/ubuntu/.local/lib/python2.7/site-packages/subdomains/utils.py", line 12, in current_site_domain from django.contrib.sites.models import Site File "/home/ubuntu/.local/lib/python2.7/site-packages/django/contrib/sites/models.py", line 84, in class Site(models.Model): File "/home/ubuntu/.local/lib/python2.7/site-packages/django/db/models/base.py", line 118, in new "INSTALLED_APPS." % (module, name) RuntimeError: Model class django.contrib.sites.models.Site doesn't declare an explicit app_label and isn't in an application in INSTALLED_APPS.
Any suggestions would be much appreciated.
Thanks.
uncomment the django.contrib.sites from installed apps
then
python manage.py migrate sites
python manage.py migrate
if you have few table in the database, drop the tables and then run this
I run into such problems when I running my Django project(a week ago, the project work properly, today find this problem):
My Django version is 1.10.2 with python version 3.5.2, MySQL version is 5.5 on ubuntu 14.0.
/Users/deja/Virtualenv/python3.5/bin/python3.5 /Applications/PyCharm.app/Contents/helpers/pydev/pydevd.py --multiproc --qt-support --client 127.0.0.1 --port 60850 --file /Users/mozat/project/crawler_management_system/manage.py runserver 0.0.0.0:6380
pydev debugger: process 81527 is connecting
Connected to pydev debugger (build 145.1504)
pydev debugger: process 81528 is connecting
Performing system checks...
System check identified no issues (0 silenced).
You have 13 unapplied migration(s). Your project may not work properly until you apply the migrations for app(s): admin, auth, contenttypes, sessions.
Run 'python manage.py migrate' to apply them.
January 03, 2017 - 07:55:47
Django version 1.10.2, using settings 'crawler_management_system.settings'
Starting development server at http://0.0.0.0:6380/
Quit the server with CONTROL-C.
Internal Server Error: /
Traceback (most recent call last):
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/network.py", line 226, in recv_plain
chunk = self.sock.recv(4 - packet_len)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/mozat/project/crawler_management_system/crawler_management_system/mysql_utility.py", line 32, in open_db
raise err
File "/Users/mozat/project/crawler_management_system/crawler_management_system/mysql_utility.py", line 28, in open_db
yield cursor
File "/Users/mozat/project/crawler_management_system/crawler_management_system/views.py", line 40, in select_batch_records
cursor.execute(sql.format(table=self.table,times=times))
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/cursor.py", line 515, in execute
self._handle_result(self._connection.cmd_query(stmt))
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/connection.py", line 488, in cmd_query
result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/connection.py", line 267, in _send_cmd
return self._socket.recv()
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/network.py", line 255, in recv_plain
errno=2055, values=(self.get_address(), _strioerror(err)))
mysql.connector.errors.OperationalError: 2055: Lost connection to MySQL server at '127.0.0.1:10189', system error: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/connection.py", line 710, in reset_session
self.cmd_reset_connection()
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/connection.py", line 1046, in cmd_reset_connection
raise errors.NotSupportedError("MySQL version 5.7.2 and "
mysql.connector.errors.NotSupportedError: MySQL version 5.7.2 and earlier does not support COM_RESET_CONNECTION.
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/protocol.py", line 267, in parse_eof
unpacked = struct_unpack('<xxxBBHH', packet)
struct.error: unpack requires a bytes object of length 9
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/django/core/handlers/exception.py", line 39, in inner
response = get_response(request)
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/django/core/handlers/base.py", line 249, in _legacy_get_response
response = self._get_response(request)
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/django/core/handlers/base.py", line 187, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/django/core/handlers/base.py", line 185, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/Users/mozat/project/crawler_management_system/crawler_management_system/views.py", line 99, in mainpage
result = batch_record_repo.select_batch_records(times)
File "/Users/mozat/project/crawler_management_system/crawler_management_system/views.py", line 41, in select_batch_records
return cursor.fetchall()
File "/Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/contextlib.py", line 77, in __exit__
self.gen.throw(type, value, traceback)
File "/Users/mozat/project/crawler_management_system/crawler_management_system/mysql_utility.py", line 40, in open_db
connection.close()
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/pooling.py", line 117, in close
cnx.reset_session()
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/connection.py", line 713, in reset_session
self._database, self._charset_id)
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/connection.py", line 661, in cmd_change_user
self._post_connection()
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/abstracts.py", line 695, in _post_connection
self.set_charset_collation(self._charset_id)
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/abstracts.py", line 654, in set_charset_collation
charset_name, collation_name))
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/connection.py", line 869, in _execute_query
self.cmd_query(query)
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/connection.py", line 488, in cmd_query
result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/connection.py", line 393, in _handle_result
return self._handle_eof(packet)
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/connection.py", line 344, in _handle_eof
eof = self._protocol.parse_eof(packet)
File "/Users/deja/Virtualenv/python3.5/lib/python3.5/site-packages/mysql/connector/protocol.py", line 269, in parse_eof
raise errors.InterfaceError(err_msg)
mysql.connector.errors.InterfaceError: Failed parsing EOF packet.
[03/Jan/2017 07:56:40] "GET / HTTP/1.1" 500 172448
anyone happen to see same problem?Could you please give me some suggestion.
all ,nothing wrong with mysql, and Django. It's for I use a complex SQL to access mysql:
SELECT *
FROM
(SELECT *,
#batch_rank := IF(#current_batch = spider , #batch_rank + 1, 1) AS batch_rank,
#current_batch := spider
FROM deja_crawler.crawl_batch_record
where spider in (select distinct spider from deja_crawler.crawl_batch_record where spider like '%_new' or spider like '%_update')
ORDER BY spider, create_time DESC)
ranked
WHERE batch_rank <= 3
the tables used to have not too much data, so it works, later, the table became huge, and the access time cost is almost 30s, but the connection time out is set as 20s, so it will raise time out error.
My legacy mysql 4.0.20 server is on a windows machine.
I'm developing a new system (python based) on a linux that needs to connect to the legacy server and make queries etc.
I have successfully connected using both plain MySQLdb and django. I'm having trouble connecting using sqlalchemy. Here is the code:
conn_str = "mysql://user:pass#192.168.171.233/dbd"
engine = create_engine(conn_str, echo=True)
metadata = MetaData(engine)
connection = engine.connect()
and the error stack I'm getting:
2011-03-01 08:35:04,613 INFO sqlalchemy.engine.base.Engine.0x...b42c SELECT DATABASE()
2011-03-01 08:35:04,613 INFO sqlalchemy.engine.base.Engine.0x...b42c ()
Traceback (most recent call last):
connection = engine.connect()
File "/usr/local/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1811, in connect
return self.Connection(self, **kwargs)
File "/usr/local/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 832, in init
self.connection = connection or engine.raw_connection()
File "/usr/local/lib/python2.6/dist-packages/sqlalchemy/engine/base.py", line 1874, in raw_connection
return self.pool.unique_connection()
File "/usr/local/lib/python2.6/dist-packages/sqlalchemy/pool.py", line 142, in unique_connection
return _ConnectionFairy(self).checkout()
File "/usr/local/lib/python2.6/dist-packages/sqlalchemy/pool.py", line 369, in __init
rec = self._connection_record = pool.get()
File "/usr/local/lib/python2.6/dist-packages/sqlalchemy/pool.py", line 213, in get
return self.do_get()
File "/usr/local/lib/python2.6/dist-packages/sqlalchemy/pool.py", line 732, in do_get
con = self.create_connection()
File "/usr/local/lib/python2.6/dist-packages/sqlalchemy/pool.py", line 147, in create_connection
return _ConnectionRecord(self)
File "/usr/local/lib/python2.6/dist-packages/sqlalchemy/pool.py", line 258, in init
l.first_connect(self.connection, self)
File "/usr/local/lib/python2.6/dist-packages/sqlalchemy/engine/strategies.py", line 151, in first_connect
dialect.initialize(c)
File "/usr/local/lib/python2.6/dist-packages/sqlalchemy/dialects/mysql/base.py", line 1753, in initialize
default.DefaultDialect.initialize(self, connection)
File "/usr/local/lib/python2.6/dist-packages/sqlalchemy/engine/default.py", line 159, in initialize
self.returns_unicode_strings = self._check_unicode_returns(connection)
File "/usr/local/lib/python2.6/dist-packages/sqlalchemy/engine/default.py", line 205, in _check_unicode_returns
unicode_for_varchar = check_unicode(sqltypes.VARCHAR(60))
File "/usr/local/lib/python2.6/dist-packages/sqlalchemy/engine/default.py", line 195, in check_unicode
]).compile(dialect=self)
File "/usr/local/lib/python2.6/dist-packages/MySQL_python-1.2.3c1-py2.6-linux-i686.egg/MySQLdb/cursors.py", line 173, in execute
self.errorhandler(self, exc, value)
File "/usr/local/lib/python2.6/dist-packages/MySQL_python-1.2.3c1-py2.6-linux-i686.egg/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
ProgrammingError: (1064, "You have an error in your SQL syntax. Check the manual that corresponds to your MySQL server version for the right syntax to use near '(60)) AS anon_1' at line 1")
Looks like MySQL is reporting a syntax error in one of your SQL statements:
<snip>
1064, "You have an error in your SQL syntax.
Check the manual that corresponds to your MySQL server
version for the right syntax to use near
'(60)) AS anon_1' at line 1
</snip>
It looks like MySQL is objecting to a statement that includes the string:
(60)) AS anon_1
NOTE: I am not familiar with Sqlalchemy, but it looks to me like the exception (the SQL error from the statement) is getting propagated up to the handler specified in the connection.
FOLLOWUP: It's likely that a query that Sqlalchemy is executing is not compatible with the older version of MySQL. Looks like it's in the "_check_unicode_returns" function.
Have you tried disabling unicode support?
conn_str = "mysql://user:pass#192.168.171.233/dbd?use_unicode=0"