I've been working on making an auto-deployment system to make updating my django site easier, and it mostly works, but right now my database changes are screwy. I have the auto deployment run "makemigrations" "migrate" then "syncdb"
The trouble I have is when I run migrate, it tells me that my table already exists and can't be created. Previous answers have said to run "migrate --fake", but if I do this, it appears to think everything is up to date. The issue then is that I run "makemigrations" or "migrate" and it says no changes detected (even though there is a missing column from my database). I tried to run "sqlall" to figure out what it thinks the database should be, and it tells me that I have pending migrations. So I tried running "migrate" and it said no migrations exist.
How do I manage this? And in the future, what should I do for database migrations to troubleshoot/fix these problems?
Thanks
I fixed the problem. The issue stemmed from a misunderstanding of how Django migrations worked. Also, as per the comments, syncdb is no longer a command in manage.py.
The original issue was that I had been using a different database for deployment and then I switched and my migrations were out of sync from my development database, so I had to clear out both databases and migrations and start over with my data. Then I run makemigrations on my local machine and migrate there (for my dev DB). Then I upload to the server and have the fab script run migrate on the server production DB without calling makemigrations. This seems to have fixed the issue.
Related
I can't seem to get Flask to migrate my models. I'm following along with the Flask Mega-Tutorial series. At the database setup part of the tutorial, I'm just trying to substitute MySQL for SQLite that is used in the tutorial.
I followed SQLAlchemy's instructions for connecting to a MySQL database, and I've included mysqlclient in my pipfile.
But when I run:
flask db init
flask db migrate
I get the following:
INFO [alembic.runtime.migration] Context impl SQLiteImpl.
INFO [alembic.runtime.migration] Will assume non-transactional DDL.
ERROR [root] Error: Can't locate revision identified by '7bb962b87f19'
I've tried deleting the migrations folder, and I've deleted the environment and recreated to see if that would fix anything. But it keeps saying it can't locate revision. I thought maybe it did something with the database, I've read other solutions saying you have to flush the database. But, there aren't any tables created or any schemas in the database.
Other info:
DATABASE_URL="mysql://root:mypassword#localhost/flask_tutorial"
SQLite database runs fine and makes the migrations. So it must be something I've done with the mysql setup.
Any suggestions or ideas that may lead me in the right direction would be greatly appreciated.
The session was storing incorrect environmental variables exported. Had to completely restart my machine for the settings to reset. I even tried unset for the variables, which didn't remove the variables. Once Flask was able to pick up the correct variable settings for my configuration everything worked as planned.
Please make sure you have used this "SQLALCHEMY_TRACK_MODIFICATIONS= True"
Did you install MySqldb lib. to connect with your mysql db ?
I had the same issue. I had deleted the migrations folder but hadn't done the same for the app.db file. So, every time I tried to create any db migrations, it failed because it was using the old (and still existing) app.db file.
This is how my database was configured:
class Config(object):
# Database configuration
SQLALCHEMY_DATABASE_URI = os.environ.get('DATABASE_URL') or \
'sqlite:///' + os.path.join(basedir, 'app.db')
SQLALCHEMY_TRACK_MODIFICATIONS = False
Deleting the app.db file solved the error. Subsequent db migrations and upgrades worked well.
First of all, I am not sure how I came to this state. I use mySQL5.7 and I recently upgraded few dependencies for my gradle project. After the upgrade it didn't like previous comments in my database migration files. By the way I didn't change database version. Some of the comments were made without a space like bellow and it was fine before. If I add spaces after '--' it works but it's an issue with other environments to which
migration is already applied since migration files checksum is changed. Could anyone suggest me a solution? I initialize my local environment frequently and I need to make this work for all environments.
--My comments start here
Your best option is to fix your scripts, so that they contain valid MySQL syntax and then run Flyway's repair command to realign the checksums.
Been able to make migrations (add columns etc) on my local machine with mysql.
But when trying to push these migrations to Heroku, they continue to fail.
The first table in my migration file gets tagged with:
Mysql2::Error: Table 'xxxxx' already exists
locally all my migrations showing as:
up 20171127214206 Add tags to business
but when running heroku run rake db:migrate:status
down 20171127214206 Add tags to business
i don't mind losing all the data at this point as i'm working in a development version on Heroku and will merge later with production
been working on this issue for over a day, so any and all advice appreciated.
according to this and many other sources, it appears you're better to use postgresql adapter with heroku. If you want to do that, you need to edit your database.yml similar to this
I know this one sounds kind of backwards! I've been working on a cloud-based case-management application, and was working on a support-ticketing feature. We have our development database (MySQL) that has all the same data as our production database (It's a very large app). Development is basically a "sandbox" environment, hence why the development db has all the same as the production db. This morning, I ran into a problem in my local development server for:
Migrations are pending; run 'bin/rake db:migrate RAILS_ENV=development' to resolve this issue.
Alright, did that, even though it didn't make any sense. It err'd out, because it was trying to create tables that already existed (I had already ran my migrations, the features were completed a day ago! And everything worked great).
The only thing that I did the moment before I got this message in my development server was uncomment an entirely-commented-out file to try and fix problems with TinyMCE (/config/tinymce.yml, text editor), followed by restarting my rails dev server. All this file was, was a bunch of defaults and plugins. When I saw the problem, I first commented them all back out. Still the same error. Proceeded with rake db: tasks.
Started looking for answers.. kept getting errors. Made a noob mistake, and ultimately did a
rake db:migrate:reset
which I knew would rebuild the schema at the cost of hosing the db. This failed.. But we do have the db in more than one place (since it's the same as production.) Also, just to try and bring all the tables back:
rake db:schema:load
....failed on one of the tables anyway with
ActiveRecord::StatementInvalid: Mysql2::Error: Specified key was too long; max key length is 767 bytes: CREATE UNIQUE INDEX `login` USING btree ON `users` (`login`, `intranet_id`)
..and all the tables are empty (as expected) that were successfully created. I'm by no means a database pro (I haven't dev'd very long). Am wondering, is there a way to copy the entire production database into the devel database? Tables, data, and all. Or any other suggestions? Really lost on this one.
You can use mysql_dump.
If you connect to the database everything is there - you just have to do a dump of the full database and then do the dump again into the production tables.
As you clearly see you can output from the database with ">" and you can import with the "<".
I have a django application and until now I used sqLite as database backend. Now when it's quite close to production I thought of moving it all to mySQL which will be used on the box.
I reconfigured my settings to a mysql database and run
manage.py syncdb --migrate
it started creating tables and all but failed in the first (out of 40) migration with an old error (can't insert blob without key length and all that).
I first thought of manually fixing the migration files but quickly realized there is too much manual work.
So I thought ok I'll run manage.py migrate core 0040 (the last migration and that will do the trick) but it still attempts to run the initial one as well:
File "D:\~Sasha\Portman\core\migrations\0001_initial.py", line 23, in forwards
('name', self.gf('django.db.models.fields.TextField')(unique=True)),
... error message
Is there a way to migrate my models somehow without manually fixing the initial migration file and all the other magic?
First, you can't pick and choose migrations to run. migrate core 0040 means run all migrations up to 0040. In other words, it wouldn't run 0041, but it would run 0001-0040.
Now, it's side-stepping your question a bit, but if you have not moved this project to production yet, you don't actually need all those migrations. Assuming they're all schemamigrations you can rollback to zero with:
python manage.py migrate core zero
Then, delete them all (including 0001_initial.py) and simply run again:
python manage.py schemamigration --initial core
To regenerate the initial migration. It will base it off the current state of your models, negating the need for 40 migrations.
It's always a good idea to compress your migrations like this before you move new code to production. Since this is the first launch, you can remove them all and start from scratch, but in future iterations, if you generate 5 migrations during the course of development, before you commit, rollback to before the first of those, then delete those 5 and then generate a new schemamigration. The result is just one migration with all the changes of those 5. Then, you can commit that and migrate in production.
It may not solve your problem completely here, but it will definitely make things simpler to debug.
Very occasionally, I've seen problems with migrations when deploying a project on a mysql backend.
Since you are deploying a fresh copy, you do have a couple options for sidestepping the need to run all of your mightations:
First, if you want to maintain your migration history as-is, you can force syncdb to create all of your tables regardless of migrations, and then run fake migrations to get your new database up to date. That would look like:
python manage.py syncdb --all
python manage.py migrate --fake
Alternatively, if you don't care about your historical migrations, you can clear the old migrations out (just delete them) and then create a new initial migration, like:
python manage.py schemamigration --initial core
Just be sure that you also clear out the south_migrationhistory table in your dev database so that everything stays in sync between dev and prod