I wanted to create a auto-generated script of database with sql extension using git. I know it is stupid question, but how comfortable would it be, when I know starter developers are also working at the same database and I can give them freedom to use live database as per their own without loosing my important data and chance to revert at live.
The main concern is it must be using GIT. My one command and creation of script become ready.
Any suggestion will be helpful. Thanks in advance.
Here are some suggestions that can help with database management in a multi-developer, version-controlled environment.
Use a migrations library. There might be one for your stack (e.g. South if you're working with Django 1.6 or earlier, Doctrine Migrations for Doctrine, Active Record Migrations for Rails) or you could use a general purpose tool like Liquibase.
In general, when working with database migrations libraries you end up with a number of versioned files that modify your database in some way. For example you might have a migrations directory containing
0001_create_initial_schema.sql
0002_add_user_model.sql
0003_replace_plaintext_passwords_with_hashed_passwords.sql
Depending on your library, these scripts may logically be written at the database level or at the ORM level. In general, each script is reversible.
This lets developers easily upgrade their database schemas as development continues.
Set each developer up with their own copy of the database. You could ask them to install MySQL locally and have initialization instructions along the lines of "create database, create user, add credentials to config file, run migrations".
Alternatively you could use something like Vagrant to easily create self-contained virtual machines for developers to use. Vagrant can easily install MySQL and create a database.
Finally, you could create databases for your developers on a centralized server, e.g. app1_john, app1_cindy, app1_joel.
Using these guidelines, developers have the flexibility to modify their own databases as needed. They can share these modifications cleanly by adding migrations to the code repository. Any developer can reset their database at any time, either by running all the migrations in reverse or by simply dropping the database.
Using this model there should be no need for most developers to have administrative access to the production database. Only allow trusted users to modify the production database, and always upgrade your database schema using the well-tested migrations scripts that you have used everywhere else.
And, of course, keep regular backups.
Related
How to upgrade a production DB schema from a dev DB schema automatically, via command line? The dev version has changes to the schema that need to be made in the production schema, but I cannot lose the data in production.
Schema Migrations
Most modern projects use a tool to track each individual change to the database, and associate some version number with the change. The database must also have some table to store its current version. That way the tool can query the current version and figure out which (if any) changes to apply.
There are several free tools to do this, like:
Liquibase
Flyway
Rails Migrations
Doctrine Migrations
SQLAlchemy Migrate
All of these require that you write meticulous code files for each change as you develop. It would be hard to reverse-engineer a project if you haven't been following the process of creating schema change code all along.
There are tools like mysqldbcompare that can help you generate the minimal ALTER TABLE statements to upgrade your production database.
There is also a newer free tool called Shift (I work with the engineer who created it), which helps to automate the process of upgrading your database. It even provides a nice web interface for entering your schema changes, running them as online changes, and monitoring their progress. But it requires quite a lot of experience to use this tool, I wouldn't recommend it for a beginner.
You can also use MySQL WorkBench: https://dev.mysql.com/downloads/workbench/
if you want an easy GUI, and cross OS compatible ;)
I started learning Ruby and Rails over the past week or so. I have completely destroyed my app and had to redo everything a couple of times. haha
It is a giant pain rebuilding everything using only a backup folder of some copied files. I'm sure I'll break some more stuff in the future.
Are there any good gems or tools that will help me backup both the App and the MySQL database?
I have used this option for MySQL but it doesn't seem to work well when working with Rails:
mysqldump database_name > database_name.sql
First of all: you don't need to do backup of your application. You should use version control system (git, for example) and automated deployment (capistrano, chef or whatever). Using this two principles you can always redeploy your application on the broken VPS or transfer it to another server.
Then, about DB backup - just do it using your DB abilities (mysqldump in your case) :) You can set up, for example, CRON task, which will backup your database once a day. But don't forget that it will be much better if you will store your dump on the other server - in case of your server being completely broken.
I am new to Database programming and there is some general questions that I would like to ask. I created the schema in my localhost using mySQL and linked to eclipse. There are some problems that I do not know how to approach.
One of my friends would like to help to develop at his personal machine, but he could not link to my database server. So one way is to copy the schema to his mySQL and change the connection string, are there any better ways?
If I would to release the project and run it on different machines, will it affect the databases operation since the schema resides in my local server.
Is there ways to just like attach the database inside the project since it is a local database and I am not accessing it from any other programs.
Sorry if my questions sound very stupid. I am really new.
Your approach of running local test MySQL instance for every developer sounds fine.
However, if your application never needs any data shared (essentially database is always local as you stated), then you should ask yourself if MySQL really an appropriate solution for your application. I think SQLite is a better choice in that situation - it is likely to work faster than MySQL for local access, and it has essentially zero setup - no database daemon to run or anything like that which greatly simplifies application install.
Granted, SQLite has its limits, but I often use SQLite for my projects, and only if they grow large, I might migrate them to MySQL or PostgreSQL if task requires it.
Typical signs that SQLite may not cut it:
Many clients (10+) need to access database for writing
Total database size is very large - more than 10GB total
I'm working with MySQL databases.
To simplify the problem, let's say I have two environments : the local one (development) and the remote one (production mode).
In the database, I have some tables that contain configuration data.
How can I automate cleanly the delivery from the development mode to the production mode when I modify the database schema and the configuration tables content ?
For instance, I dot it manually by doing a diff between the local and remote databases. But, I find that method not so clean and I believe there is good practice allowing that.
This might be helpful in cases where you have multiple environments and multiple developers making schema changes very often and using php.. https://github.com/davejkiger/mysql-php-migrations
Introduce parameter "version" for your database. This version should be written somewhere in your code and somewhere in your database. Your code will work with database only if they have equal versions
Create a wrapper around your MySQL connection. This wrapper should check versions and if versions are not compatible, it should start upgrade.
"Upgrade" is a process of sequential applying the list of *.sql files with SQL commands, which will move your database from one state to another. It can be schema changes or data manipulation commands.
When you do something with database, do it only through adding new *.sql file and incrementing version.
As a result, when you deploy your database from development enviroment to production, your database will be upgraded automatically in the same way as it was upgraded during development.
I've seen LiquiBase http://www.liquibase.org/ a lot in Java environments.
In most of my projects I use sqlalchemy(a Python tool to manage db plus an ORM). If you have some experience(little more than beginner) with Python I highly recommend using it.
You can check this tool with a little help of that. This is also very useful for migrating your db to other rdbms(for example mysql to postgres or oracle).
My father has a small business and asked me to make him a quick program to help automate several of his daily tasks. Due to the nature of these tasks, it absolutely requires a local database (not distributed with connections from multiple clients, just 1 local client and 1 local MySql datastore).
I'm pretty accustomed to MySql programming but have never had to install it and don't really know the details about how it works under the hood.
What I'd like to have is a final deliverable where I send him a folder that has an executable JAR (I've chosen Java since its a good fit here) as well as a preconfigured MySql database on it. I'll run all the scripts, initialize all the tables and populate them, etc. before deploying the the distributable (folder).
This way, no config is necessary for him. My dad gets the folder, plunks it down wherever he wants it on his file system, and double-clicks the JAR to launch the program every time. I will preconfigure the JDBC code to look and connect to the local Mysql database also living under the same root folder ("MyDadsApp/").
Is this even possible to do? If not, why? If so, how?
Here's what I'm thinking as a tentative deployed folder structure:
MyDadsApp/
MyDadsApp.jar
res/
user-manual/
mysql/
MyDadsAppDatabase.myd (???) - the actual database
MySql.exe (???) - MySql Server
Any input/recommendations will be met with open arms!
Thanks
This is actually a perfect scenario for SQLite. It's got a lot less features than MySQL, but it's whole shtick is that it's server-less and self-contained. Sounds perfect for what you're doing.