I first started using MySQL in one of my apps and I am now thinking of moving from MySQL to PostgreSQL.
I have South installed for migrations.
When I set up a new DB in postgres I successfully synced my apps and got to a complete halt in one of my last migrations.
> project:0056_auto__chg_field_project_project_length
Traceback (most recent call last):
File "./manage.py", line 11, in <module>
execute_manager(settings)
File "/Users/ApPeL/.virtualenvs/fundedbyme.com/lib/python2.7/site-packages/django/core/management/__init__.py", line 438, in execute_manager
utility.execute()
File "/Users/ApPeL/.virtualenvs/fundedbyme.com/lib/python2.7/site-packages/django/core/management/__init__.py", line 379, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/ApPeL/.virtualenvs/fundedbyme.com/lib/python2.7/site-packages/django/core/management/base.py", line 191, in run_from_argv
self.execute(*args, **options.__dict__)
File "/Users/ApPeL/.virtualenvs/fundedbyme.com/lib/python2.7/site-packages/django/core/management/base.py", line 220, in execute
output = self.handle(*args, **options)
File "/Library/Python/2.7/site-packages/south/management/commands/migrate.py", line 105, in handle
ignore_ghosts = ignore_ghosts,
File "/Library/Python/2.7/site-packages/south/migration/__init__.py", line 191, in migrate_app
success = migrator.migrate_many(target, workplan, database)
File "/Library/Python/2.7/site-packages/south/migration/migrators.py", line 221, in migrate_many
result = migrator.__class__.migrate_many(migrator, target, migrations, database)
File "/Library/Python/2.7/site-packages/south/migration/migrators.py", line 292, in migrate_many
result = self.migrate(migration, database)
File "/Library/Python/2.7/site-packages/south/migration/migrators.py", line 125, in migrate
result = self.run(migration)
File "/Library/Python/2.7/site-packages/south/migration/migrators.py", line 99, in run
return self.run_migration(migration)
File "/Library/Python/2.7/site-packages/south/migration/migrators.py", line 81, in run_migration
migration_function()
File "/Library/Python/2.7/site-packages/south/migration/migrators.py", line 57, in <lambda>
return (lambda: direction(orm))
File "/Users/ApPeL/Sites/Django/fundedbyme/project/migrations/0056_auto__chg_field_project_project_length.py", line 12, in forwards
db.alter_column('project_project', 'project_length', self.gf('django.db.models.fields.IntegerField')())
File "/Library/Python/2.7/site-packages/south/db/generic.py", line 382, in alter_column
flatten(values),
File "/Library/Python/2.7/site-packages/south/db/generic.py", line 150, in execute
cursor.execute(sql, params)
File "/Users/ApPeL/.virtualenvs/fundedbyme.com/lib/python2.7/site-packages/django/db/backends/postgresql_psycopg2/base.py", line 44, in execute
return self.cursor.execute(query, args)
django.db.utils.DatabaseError: column "project_length" cannot be cast to type integer
I am wondering if there is some workaround for this?
You current migration works that way:
Alter column "project_length" to another type.
It is broken because you are making alter that is not supported by PostgreSQL.
You must fix your migration. You can change it to following migration (it will work, but probably can be done easier):
Create another column project_length_tmp with type you want to project_length have and some default value.
Make data migration from column project_length to project_lenght_tmp (see data migrations in south docs).
Remove column project_length.
Rename column project_length_tmp to project_length.
Kind complicated migration but it have two major strengths:
1. It will work on all databases.
2. It is compatible with your old migration, so you can just override old migration (change the file) and it will be fine.
Approach 2
Another approach to your problem would be just to remove all your migrations and start from scratch. If you have only single deployment of your project it will work fine for you.
You don't provide any details of the SQL being executed, but it seems unlikely that it's an ALTER TYPE failing - assuming the SQL is correct.
=> CREATE TABLE t (c_text text, c_date date, c_datearray date[]);
CREATE TABLE
=> INSERT INTO t VALUES ('abc','2011-01-02',ARRAY['2011-01-02'::date,'2011-02-03'::date]);
INSERT 0 1
=> ALTER TABLE t ALTER COLUMN c_text TYPE integer USING (length(c_text));
ALTER TABLE
=> ALTER TABLE t ALTER COLUMN c_date TYPE integer USING (c_date - '2001-01-01');
ALTER TABLE
=> ALTER TABLE t ALTER COLUMN c_datearray TYPE integer USING (array_upper(c_datearray, 1));
ALTER TABLE
=> SELECT * FROM t;
c_text | c_date | c_datearray
--------+--------+-------------
3 | 3653 | 2
(1 row)
There's not much you can't do. I'm guessing it's incorrect SQL being generated from this Django module you are using.
Related
A process I have that's been running for a couple of years suddenly stopped working.
I avoided updating much in the way of python and packages to avoid that..
I've now updated ib_insync to the latest version, and no improvement. debugging a little gives me this:
the code
import ib_insync as ibis
ib = ibis.IB()
contract = ibis.Contract()
contract.secType = 'STK'
contract.currency = 'USD'
contract.exchange = 'SMART'
contract.localSymbol = 'AAPL'
ib.qualifyContracts(contract)
Result:
File "/Users/macuser/anaconda3/lib/python3.6/site-packages/ib_insync/client.py", line 244, in send
if field in empty:
File "/Users/macuser/anaconda3/lib/python3.6/site-packages/ib_insync/contract.py", line 153, in hash
raise ValueError(f'Contract {self} can't be hashed')
ValueError: Contract Contract(secType='STK', exchange='SMART', currency='USD', localSymbol='AAPL') can't be hashed
Exception ignored in: <bound method IB.del of <IB connected to 127.0.0.1:7497 clientId=6541>>
Traceback (most recent call last):
File "/Users/macuser/anaconda3/lib/python3.6/site-packages/ib_insync/ib.py", line 233, in del
File "/Users/macuser/anaconda3/lib/python3.6/site-packages/ib_insync/ib.py", line 281, in disconnect
File "/Users/macuser/anaconda3/lib/python3.6/logging/init.py", line 1306, in info
File "/Users/macuser/anaconda3/lib/python3.6/logging/init.py", line 1442, in _log
File "/Users/macuser/anaconda3/lib/python3.6/logging/init.py", line 1452, in handle
File "/Users/macuser/anaconda3/lib/python3.6/logging/init.py", line 1514, in callHandlers
File "/Users/macuser/anaconda3/lib/python3.6/logging/init.py", line 863, in handle
File "/Users/macuser/anaconda3/lib/python3.6/logging/init.py", line 1069, in emit
File "/Users/macuser/anaconda3/lib/python3.6/logging/init.py", line 1059, in _open
NameError: name 'open' is not defined
| => python --version
Python 3.6.4 :: Anaconda, Inc.
I am not OP, but have had a similar problem. I am attempting to send an order to IB using ib_insync.
contract = Stock('DKS','SMART','USD')
order = LimitOrder('SELL', 1, 1)
try:
ib.qualifyContracts(contract)
trade = ib.placeOrder(contract, order)
except Exception as e:
print(e)
Here is the exception that is returned:
Contract Stock(symbol='DKS', exchange='SMART', currency='USD') can't be hashed
I understand that lists and other mutable can't be hashed, but I don't understand why this wouldn't work. It pretty clearly follows the examples in the ib_insync docs.
FYI - I was able to resolve this issue by updating to the latest ib_insync version. Perhaps this will help you as well.
I'm getting a ValueError: Lock objects should only be shared between processes through inheritance when writing a xarray.DataArray to_netcdf().
Everything works until writing to disk. But I found a workaround which is to use dask.config.set(scheduler='single-threaded').
Is everyone supposed to use dask.config.set(scheduler='single-threaded') before writing to disk?
Am I missing something?
I tested two schedulers:
1) from dask.distributed import Client; client = Client()
2) import dask.multiprocessing; dask.config.set(scheduler=dask.multiprocessing.get)
python=2.7, xarray=0.10.9, traceback:
File "/home/py_user/miniconda2/envs/v0/lib/python2.7/site-packages/xarray/core/dataarray.py", line 1746, in to_netcdf
return dataset.to_netcdf(*args, **kwargs)
File "/home/py_user/miniconda2/envs/v0/lib/python2.7/site-packages/xarray/core/dataset.py", line 1254, in to_netcdf
compute=compute)
File "/home/py_user/miniconda2/envs/v0/lib/python2.7/site-packages/xarray/backends/api.py", line 724, in to_netcdf
unlimited_dims=unlimited_dims, compute=compute)
File "/home/py_user/miniconda2/envs/v0/lib/python2.7/site-packages/xarray/core/dataset.py", line 1181, in dump_to_store
store.sync(compute=compute)
...
File "/home/py_user/miniconda2/envs/v0/lib/python2.7/multiprocessing/synchronize.py", line 95, in __getstate__
assert_spawning(self)
File "/home/py_user/miniconda2/envs/v0/lib/python2.7/multiprocessing/forking.py", line 52, in assert_spawning
' through inheritance' % type(self).__name__
As #jhamman mentioned in the comments. This may have been fixed in a newer version of Xarray.
my knowledge about database driven app and database ORM is wee, I have written this model in peewee https://codereview.stackexchange.com/q/210293/22943
and I want to be able to update the 3 tables Patient, Relative and PatientAttendOnVisit at the same time with Relative and PatientAttendOnVisit foreign key to Patient ID in Patient table.
I tried straight forward:
def add_patient_visit(data=None):
"""
Adds new visit to clinic of patient
for new or follow up patient.
"""
if not data:
raise ValueError("Please pass the user info.")
try:
patient = _clinic.Patient.get(name=data["name"])
if patient:
print "Patient exists with same name."
response_object = {
"status": "fail",
"message": "Patient already in record."
}
return response_object, 400
except peewee.DoesNotExist as er:
patient = _clinic.Patient.create(
name=data["name"],
townCity=data["townCity"],
contactnumber=data["contactnumber"],
age=data["age"],
gender=data["gender"],
email=data["email"],
postalAddress=data["postalAddress"])
relative = _clinic.Relative.create(relation=data["relation"],
relativeName=data["relativeName"])
attendence = _clinic.PatientAttendOnVisit.create(
dateTimeofvisit=data["dateTimeofvisit"],
attendtype=data["attendtype"],
department=data["department"]
)
but attempting to do so give me below error:
return controller.add_patient_visit(data=data) File
"/Users/ciasto/Development/python/clinic-backend/app/api/clinic/controller.py",
line 35, in add_patient_visit
relativeName=data["relativeName"]) File "/Users/ciasto/Development/python/clinic-backend/clinic_venv/lib/python2.7/site-packages/peewee.py",
line 5580, in create
inst.save(force_insert=True) File "/Users/ciasto/Development/python/clinic-backend/clinic_venv/lib/python2.7/site-packages/peewee.py",
line 5727, in save
pk_from_cursor = self.insert(**field_dict).execute() File "/Users/ciasto/Development/python/clinic-backend/clinic_venv/lib/python2.7/site-packages/peewee.py",
line 1622, in inner
return method(self, database, *args, **kwargs) File "/Users/ciasto/Development/python/clinic-backend/clinic_venv/lib/python2.7/site-packages/peewee.py",
line 1693, in execute
return self._execute(database) File "/Users/ciasto/Development/python/clinic-backend/clinic_venv/lib/python2.7/site-packages/peewee.py",
line 2355, in _execute
return super(Insert, self)._execute(database) File "/Users/ciasto/Development/python/clinic-backend/clinic_venv/lib/python2.7/site-packages/peewee.py",
line 2118, in _execute
cursor = database.execute(self) File "/Users/ciasto/Development/python/clinic-backend/clinic_venv/lib/python2.7/site-packages/peewee.py",
line 2724, in execute
return self.execute_sql(sql, params, commit=commit) File "/Users/ciasto/Development/python/clinic-backend/clinic_venv/lib/python2.7/site-packages/peewee.py",
line 2718, in execute_sql
self.commit() File "/Users/ciasto/Development/python/clinic-backend/clinic_venv/lib/python2.7/site-packages/peewee.py",
line 2509, in exit
reraise(new_type, new_type(*exc_args), traceback) File "/Users/ciasto/Development/python/clinic-backend/clinic_venv/lib/python2.7/site-packages/peewee.py",
line 2711, in execute_sql
cursor.execute(sql, params or ()) File "/Users/ciasto/Development/python/clinic-backend/clinic_venv/lib/python2.7/site-packages/MySQLdb/cursors.py",
line 205, in execute
self.errorhandler(self, exc, value) File "/Users/ciasto/Development/python/clinic-backend/clinic_venv/lib/python2.7/site-packages/MySQLdb/connections.py",
line 36, in defaulterrorhandler
raise errorclass, errorvalue IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails
(clinic_backend.relative, CONSTRAINT relative_ibfk_1 FOREIGN KEY
(patient_id) REFERENCES patient (id))')
Nothing much complicated, I figured out,
def add_patient_visit(data=None):
"""
Adds new visit to clinic of patient
for new or follow up patient.
"""
if not data:
raise ValueError("Please pass the user info.")
patient = _clinic.Patient.create(
name=data["name"],
townCity=data["townCity"],
contactnumber=data["contactnumber"],
age=data["age"],
gender=data["gender"],
email=data["email"],
postalAddress=data["postalAddress"])
relative = _clinic.Relative.create(
patient=patient,
relation=data["relation"],
relativeName=data["relativeName"])
attendence = _clinic.PatientAttendOnVisit.create(
patient=patient,
dateTimeofvisit=data["dateTimeofvisit"],
attendtype=data["attendtype"],
department=data["department"]
)
I am attempting to transfer a database from sqlite to mysql.
I've googled this error and found Stack Overflow matches, but havent seen how to debug/identify the offending "0 value AutoField" fields. I've tried skirting the issue by dumping/loading different tables, but all seem to generate the same error.
I've attempted appending -e contenttypes, --natural-foreign, and --natural-primary to my datadump command, e.g.,
python manage.py dumpdata -e contenttypes --natural-foreign --natural-primary --indent=4 > datadump_3-7-18.json
After running python manage.py loaddata --traceback datadump_3-7-18.json
It produces the traceback error:
(venv) ➜ bikerental git:(additional-features-march) ✗ python manage.py loaddata --traceback datadump_3-7-18.json
Traceback (most recent call last):
File "manage.py", line 15, in <module>
execute_from_command_line(sys.argv)
File "/rentals/venv/lib/python3.6/site-packages/django/core/management/__init__.py", line 371, in execute_from_command_line
utility.execute()
File "/rentals/venv/lib/python3.6/site-packages/django/core/management/__init__.py", line 365, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/rentals/venv/lib/python3.6/site-packages/django/core/management/base.py", line 288, in run_from_argv
self.execute(*args, **cmd_options)
File "/rentals/venv/lib/python3.6/site-packages/django/core/management/base.py", line 335, in execute
output = self.handle(*args, **options)
File "/rentals/venv/lib/python3.6/site-packages/django/core/management/commands/loaddata.py", line 72, in handle
self.loaddata(fixture_labels)
File "/rentals/venv/lib/python3.6/site-packages/django/core/management/commands/loaddata.py", line 113, in loaddata
self.load_label(fixture_label)
File "/rentals/venv/lib/python3.6/site-packages/django/core/management/commands/loaddata.py", line 177, in load_label
obj.save(using=self.using)
File "/rentals/venv/lib/python3.6/site-packages/django/core/serializers/base.py", line 205, in save
models.Model.save_base(self.object, using=using, raw=True, **kwargs)
File "/rentals/venv/lib/python3.6/site-packages/django/db/models/base.py", line 759, in save_base
updated = self._save_table(raw, cls, force_insert, force_update, using, update_fields)
File "/rentals/venv/lib/python3.6/site-packages/django/db/models/base.py", line 842, in _save_table
result = self._do_insert(cls._base_manager, using, fields, update_pk, raw)
File "/rentals/venv/lib/python3.6/site-packages/django/db/models/base.py", line 880, in _do_insert
using=using, raw=raw)
File "/rentals/venv/lib/python3.6/site-packages/django/db/models/manager.py", line 82, in manager_method
return getattr(self.get_queryset(), name)(*args, **kwargs)
File "/rentals/venv/lib/python3.6/site-packages/django/db/models/query.py", line 1125, in _insert
return query.get_compiler(using=using).execute_sql(return_id)
File "/rentals/venv/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1280, in execute_sql
for sql, params in self.as_sql():
File "/rentals/venv/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1233, in as_sql
for obj in self.query.objs
File "/rentals/venv/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1233, in <listcomp>
for obj in self.query.objs
File "/rentals/venv/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1232, in <listcomp>
[self.prepare_value(field, self.pre_save_val(field, obj)) for field in fields]
File "/rentals/venv/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1172, in prepare_value
value = field.get_db_prep_save(value, connection=self.connection)
File "/rentals/venv/lib/python3.6/site-packages/django/db/models/fields/__init__.py", line 767, in get_db_prep_save
return self.get_db_prep_value(value, connection=connection, prepared=False)
File "/rentals/venv/lib/python3.6/site-packages/django/db/models/fields/__init__.py", line 940, in get_db_prep_value
value = connection.ops.validate_autopk_value(value)
File "/rentals/venv/lib/python3.6/site-packages/django/db/backends/mysql/operations.py", line 163, in validate_autopk_value
raise ValueError('The database backend does not accept 0 as a '
ValueError: Problem installing fixture '/rentals/bikerental/datadump_3-7-18.json': The database backend does not accept 0 as a value for AutoField.
I've noticed this seems to have something to do with Foreignkey values, so I'll post the one I have:
bike = models.ManyToManyField(Bike, blank=True)
Any way to more easily identify where in the database I assume this is coming from?
I've solved the problem by manually editing the data dump. In the table to which the ManyToManyField refers, there was a record whose ID was 0. This was due to my initial manually entered set of records, where I began the increment at 0. Removing this record, and removing references to it in the ManyToManyField allowed the loaddata command to work without error. For the record, I would just like to state that error handling could/should be improved here by being more explicit, for it had me scratching my head for nearly an entire work day.
For illustration in datadump_3-7-18.json:
{ <----- I GOT DELETED
"model": "inventory.bike", <----- I GOT DELETED
"pk": 0, <----- I GOT DELETED
"fields": { <----- I GOT DELETED
... <----- I GOT DELETED
} <----- I GOT DELETED
}, <----- I GOT DELETED
{
"model": "inventory.bike",
"pk": 1,
"fields": {
...
}
},
{
"model": "inventory.bike",
"pk": 2,
"fields": {
...
}
},
And later on in datadump_3-7-18.json, the records containing the 0 ManyToManyField foreignkey:
{
"model": "reservations.reservation",
"pk": 55,
"fields": {
...
"bike": [
0, <----- I GOT DELETED
1,
2
]
}
},
{
"model": "reservations.reservation",
"pk": 28,
"fields": {
...
"bike": [
0, <----- I GOT DELETED
1,
2,
3
]
}
},
I want to update a table with the following query. I am getting multiple errors on the following. What is the correct syntax for writing the query below
cursor.execute("""UPDATE `%s` SET `content`=%s WHERE link=%s""", (feed,cnews,news_url))
The error I get when running the above is
Traceback (most recent call last):
File "digger_1.py", line 34, in <module>
cursor.execute("""UPDATE `%s` SET `content`=%s WHERE link=%s""", (feed,cnews,news_url))
File "/usr/lib/python2.7/dist-packages/MySQLdb/cursors.py", line 174, in execute
self.errorhandler(self, exc, value)
File "/usr/lib/python2.7/dist-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
raise errorclass, errorvalue
_mysql_exceptions.ProgrammingError: (1146, "Table 'newstracker.'NDTV'' doesn't exist")
The error I picked was the table newstracker.NDTV doesn't exist, which is not right as it does exist, and I believe the error is something else which is wrong with the syntax.
You cannot specify metadata such as database, table, or field names using a parametrized query. You must substitute them using normal string formatting and then use the result string as your parametrized query.
...("""UPDATE `%s` SET `content`=%%s WHERE link=%%s""" % (feed,), ...)