I recently tried to change a column in my database with a south migration for my django project from a CharField to a IntegerField
I did a python manage.py schemamigration appname --auto
i then applied the migration and it just hung. I stupidly ran this right on my production server before trying out locally. Once I tried locally I got this error in my traceback
File "/Library/Python/2.7/site-packages/MySQL_python-1.2.3-py2.7-macosx-10.7-intel.egg/MySQLdb/cursors.py", line 92, in _warning_check
_mysql_exceptions.Warning: Data truncated for column 'pickup_id' at row 1
When I go into mysql workbench and just try to revert the changes back to a CharField it just hangs in the "preparing" state and eventually times out. My site just hangs wherever this table is used and I cant preform any queries on this particular table.
I'm open to any suggestion. Thank you
What ended up working was applying the --fake parameters to my migrate call
./manage migrate appname --fake
Related
I can't seem to get Flask to migrate my models. I'm following along with the Flask Mega-Tutorial series. At the database setup part of the tutorial, I'm just trying to substitute MySQL for SQLite that is used in the tutorial.
I followed SQLAlchemy's instructions for connecting to a MySQL database, and I've included mysqlclient in my pipfile.
But when I run:
flask db init
flask db migrate
I get the following:
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
ERROR [root] Error: Can't locate revision identified by '7bb962b87f19'
I've tried deleting the migrations folder, and I've deleted the environment and recreated to see if that would fix anything. But it keeps saying it can't locate revision. I thought maybe it did something with the database, I've read other solutions saying you have to flush the database. But, there aren't any tables created or any schemas in the database.
Other info:
DATABASE_URL="mysql://root:mypassword#localhost/flask_tutorial"
SQLite database runs fine and makes the migrations. So it must be something I've done with the mysql setup.
Any suggestions or ideas that may lead me in the right direction would be greatly appreciated.
The session was storing incorrect environmental variables exported. Had to completely restart my machine for the settings to reset. I even tried unset for the variables, which didn't remove the variables. Once Flask was able to pick up the correct variable settings for my configuration everything worked as planned.
Please make sure you have used this "SQLALCHEMY_TRACK_MODIFICATIONS= True"
Did you install MySqldb lib. to connect with your mysql db ?
I had the same issue. I had deleted the migrations folder but hadn't done the same for the app.db file. So, every time I tried to create any db migrations, it failed because it was using the old (and still existing) app.db file.
This is how my database was configured:
class Config(object):
# Database configuration
SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URL') or \
'sqlite:///' + os.path.join(basedir, 'app.db')
SQLALCHEMY_TRACK_MODIFICATIONS = False
Deleting the app.db file solved the error. Subsequent db migrations and upgrades worked well.
Background info:
I am quite beginner with django, better equipped with python and quite beginner in mysql (and thus mariaDB) as well.
I work in Windows 7 environment with the following versions:
django 1.9.1
mariaDB 10.0.20
Python 3.4.4
mysql-connector\Python 2.1.3 for Python 3.4
Goal:
I am trying to build mariaDB database and web based user interface for it with django. I have built the database beforehand in mysql and am now trying to create the django project using this database as legacy database.
Problem:
I have retrieved the models to my django project as follows:
python manage.py inspectdb > models.py
and it worked fine. Checked the models.py, as well, and they seemed as they should.
Now, if I try to migrate to create the tables based on django to my legacy db as follows:
python manage.py migrate
I keep getting plenty of nested errors last one being:
django.db.migrations.exceptions.MigrationsSchemaMissing: Unable to create the django_migrations table (Table 'mydb'.'django_migrations' already exists. Please DISCARD the tablespace before IMPORT.)
I figured the table was left in my database folder from previous attempt, but removing it does not help. Even if I check from the database command prompt that the only tables left are native to my database, this same error message pops up still. At some point I wondered if there is some issue with the connector I am using as it is not the recommended one (mysqlclient).
Somewhere, I found that the migration would not even be required. This would obviously save me, but I have no idea, if that is a solution I can go for. Ideally the database would be managed later online and users with admin accounts could add entries there. Otherwise, however the database should be just returning the data.
EDIT: Fixed the error message the python manage.py inspectdb > models.py line
EDIT 29.1.
Running the migration as:
python manage.py migrate --fake
results in the same error.
The issue persists even after removing the django project and starting it fresh.
EDIT v2 29.1.
There is only one of the required table files for the django_migrations table within the database folder. There should be django_migrations.ibd AND django_migrations.frm. However, after running the migration, there is only one file: django_migrations.ibd.
There was something similar going on here, with different file
mysqldump problems with restore error: 'Please DISCARD the tablespace before IMPORT'. It is well explained in the chosen answer.
I think this is related to my issue, but cannot figure out what could help and what to do with this.
EDIT v3 29.1.
And finally I noticed the file giving the django error. It is django's migration recorder at C:\Python34\lib\site-packages\django\db\migrations\recorder.py on line 59.
def ensure_schema(self):
"""
Ensures the table exists and has the correct schema.
"""
# If the table's there, that's fine - we've never changed its schema
# in the codebase.
if self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()):
return
# Make the table
try:
with self.connection.schema_editor() as editor:
editor.create_model(self.Migration)
except DatabaseError as exc:
raise MigrationSchemaMissing("Unable to create the django_migrations table (%s)" % exc)
Where the last line raising the error is the line 59. Funnily, the function should work, if the table exists. Only thing I can come up with (as in my previous edit), is that indeed for some reason the frm-file is not created during the migration process.
Why that happens is way beyond me.
I've been working on making an auto-deployment system to make updating my django site easier, and it mostly works, but right now my database changes are screwy. I have the auto deployment run "makemigrations" "migrate" then "syncdb"
The trouble I have is when I run migrate, it tells me that my table already exists and can't be created. Previous answers have said to run "migrate --fake", but if I do this, it appears to think everything is up to date. The issue then is that I run "makemigrations" or "migrate" and it says no changes detected (even though there is a missing column from my database). I tried to run "sqlall" to figure out what it thinks the database should be, and it tells me that I have pending migrations. So I tried running "migrate" and it said no migrations exist.
How do I manage this? And in the future, what should I do for database migrations to troubleshoot/fix these problems?
Thanks
I fixed the problem. The issue stemmed from a misunderstanding of how Django migrations worked. Also, as per the comments, syncdb is no longer a command in manage.py.
The original issue was that I had been using a different database for deployment and then I switched and my migrations were out of sync from my development database, so I had to clear out both databases and migrations and start over with my data. Then I run makemigrations on my local machine and migrate there (for my dev DB). Then I upload to the server and have the fab script run migrate on the server production DB without calling makemigrations. This seems to have fixed the issue.
I have a django application and until now I used sqLite as database backend. Now when it's quite close to production I thought of moving it all to mySQL which will be used on the box.
I reconfigured my settings to a mysql database and run
manage.py syncdb --migrate
it started creating tables and all but failed in the first (out of 40) migration with an old error (can't insert blob without key length and all that).
I first thought of manually fixing the migration files but quickly realized there is too much manual work.
So I thought ok I'll run manage.py migrate core 0040 (the last migration and that will do the trick) but it still attempts to run the initial one as well:
File "D:\~Sasha\Portman\core\migrations\0001_initial.py", line 23, in forwards
('name', self.gf('django.db.models.fields.TextField')(unique=True)),
... error message
Is there a way to migrate my models somehow without manually fixing the initial migration file and all the other magic?
First, you can't pick and choose migrations to run. migrate core 0040 means run all migrations up to 0040. In other words, it wouldn't run 0041, but it would run 0001-0040.
Now, it's side-stepping your question a bit, but if you have not moved this project to production yet, you don't actually need all those migrations. Assuming they're all schemamigrations you can rollback to zero with:
python manage.py migrate core zero
Then, delete them all (including 0001_initial.py) and simply run again:
python manage.py schemamigration --initial core
To regenerate the initial migration. It will base it off the current state of your models, negating the need for 40 migrations.
It's always a good idea to compress your migrations like this before you move new code to production. Since this is the first launch, you can remove them all and start from scratch, but in future iterations, if you generate 5 migrations during the course of development, before you commit, rollback to before the first of those, then delete those 5 and then generate a new schemamigration. The result is just one migration with all the changes of those 5. Then, you can commit that and migrate in production.
It may not solve your problem completely here, but it will definitely make things simpler to debug.
Very occasionally, I've seen problems with migrations when deploying a project on a mysql backend.
Since you are deploying a fresh copy, you do have a couple options for sidestepping the need to run all of your mightations:
First, if you want to maintain your migration history as-is, you can force syncdb to create all of your tables regardless of migrations, and then run fake migrations to get your new database up to date. That would look like:
python manage.py syncdb --all
python manage.py migrate --fake
Alternatively, if you don't care about your historical migrations, you can clear the old migrations out (just delete them) and then create a new initial migration, like:
python manage.py schemamigration --initial core
Just be sure that you also clear out the south_migrationhistory table in your dev database so that everything stays in sync between dev and prod
I have created an app using a badly named database, all alterations to important data in the database have been done in fixtures so that I can just drop the database, recreate it and then rake migrate the database tables and fill them with the initial data from the fixtures.
I would like to change the name of my database now, so I updated the database.yml file to reflect another database name. I created the database in mysql and then tried to run the migration and fixtures.
Running the migration with trace shows that it is running commands to create tables etc. However once I am finished I get errors in my application saying that the tables don't exist in the new database.
I go into mysql and check the database and it is completely empty. I have tried wiping everything and running the migrations a few times but nothing changes. Is there something I am missing?
I don't know what commands you used to do the migration, but to migrate a production database (which I infer from the tag, "production-environment"), you have to do:
RAILS_ENV=production rake db:migrate
If that's not the answer, then please provide the command you used to perform the migration, which database you expected to be affected, and the relevant bits of database.yml.