I am developing a web application in Django. My application is already up, and some users are are using it (say about 5-10). The database is SQLite. Should I move to MySQL now?
Or, wait till the user base increases? I don't have any user registration feature, yet. The basic usage of app is - problems are served n users solve them.
Move now. It'll be a pain to move later. At least right now if you take your website offline for a few hours it won't be noticeable. Later, that will be a problem. (Not to mention, you'll probably have to write a script to move data from your SQLite database to MySQL, which is a pain in the ass in and of itself.)
I don't get why using SQLite for development and then deploying it with MySQL.
Why don't develop and deploy the same RDMS?
Definitely move to MySQL now - on both development and production (and staging?). The earlier you do it, the less users you disrupt and the smaller and simpler the migration will be.
Do it on development first so you see what problems you're going to run into, and resolve them before migrating to production. If you were to keep using SQLite for development, and MySQL for production - you would run into problems with the differences eventually.
Related
I hvae django app that needs to be extremely fast, and it works good for now.
So my question is, is it better to put django app on one server and mysql on another server, or on one server both?
I ask because of communication between then.
I use digitalocean, and both are on one server.
It depends how well the application is written.
Poorly written django will generate a lot of queries so maybe it's beneficial to have it on the same server. Well written Django should leverage the database to do the heavy lifting, in which case its better to have it on a separate server, so the server can be tuned for a database. (In general having a separate database server is the way to go).
The best thing to do would be to add Django debug toolbar to your application and see if it is generating a lot of queries or not, and tune the application from there.
You have couple of options but let's stick to these two.
One server for everything
Good for setting up an application quickly, as it is the simplest setup possible, but it offers little in the way of scalability and component isolation.
There are a lot of pros, it's fast, simple to work with. It does not meet latency problems. From cons: you cannot horizontally scale.
Server for web application and server for database.
First of all, I would recommend to use Postgres, since the latest version (9.6) can now work on multiple cores, which makes it way faster than mysql.
It is good for setting up an application quickly, but keeps application and database from fighting over the same system resources.
From pros it does not fight over resources (RAM / CPU / I/O).
It may also increase security by removing database from DMZ.
From cons, it is harder to setup and when high-latency is going on, the queries might take longer to execute.
To sum up. I would use first option for small and medium applications which does not require a lot of requests.
I would consider moving DB to another server/servers, whenever the application hosts thousands of users per day.
I want to upgrade my appserv mysql instalation from 5.0.x to 5.x.
I have some tables and views relationed with various web proyects and VB.net aplications in that.
Any body can help me to do that without data loss?
(Putting this in an answer as it's too long for a comment)
NB - I've not used AppServ so this answer is generic
The versions of software within AppServ appear to be old. Very old. MySQL 5.0.51b, PHP 5.2.6, Apache 5.2.8 are way behind with regards to security and features. The best thing you can do is to replace the whole stack with a newer one
If you do a quick Google search for WAMP installer, a plethora of available stacks are listed. The first one in the list uses MySQL 5.6.17, PHP 5.5.12, Apache 2.4.9. Again, not the newest, but much more recent and feature rich. It's also available in 32 and 64 bit versions
The first thing to do is to download a virtual machine system. (VirtualBox is a pretty simple one to get to grips with and runs on a variety of platforms). This is so that you can practise.
Spool up an instance of Windows (which is as close as possible to your live setup) and install your current version of AppServ and your applications which use it, take a snapshot (so you can roll back) and then work out slowly how to update to a new stack. Take lots of snapshots as you go.
You need to make note of your MySQL data directories and back up your Apache, MySQL and PHP configurations
It will take time to iron out the bugs and problems you find along the way. Do not be downhearted.
Once you have worked out how to update your stack without data loss, try your applications on the virtual machine. There is no point in upgrading your stack if your software is going to bomb out the second it start to run.
Once you're satisfied what all the steps you need are, roll back to the snap shot you took at the start and go through all the steps again. Then again. And keep on restoring/upgrading it until you are confident that you can do the update with the minimum of fuss and panic on the live system
I would recommend doing your update over two sessions. For both sessions, choose a quiet time to do it. Essentially, out of office hours is the best, early morning (after a good sleep) is even better.
During the first session (SESSION-1) the server offline, backup everything, then return the server to live. And when I say "backup everything", I mean EVERYTHING! Take this backup and restore it to a virtual machine. Go through your steps that you worked out before on this restored version to make sure everything is going to work. Make a note of anything that is different to the steps you worked out earlier.
When you've done your testing, you can do session two (SESSION-2). Again, take the server offline, run a differential backup on the system and a full backup of the MySQL databases. Update your WAMP stack (using the steps you worked out in SESSION-1) and bring it back online. Check that all your URLs and code still works.
After you've completed your checks, send triumphant emails to whoever needs to know, put a smug smile on your face for a job well done, pour yourself a large glass of whiskey (other drinks are available) and relax - you've earned it
Sorry that I can't give you definitive steps but I use Linux for all my PHP stacks so these steps are what I would do if I was upgrading them. I spent 3 months practising upgrading my servers then did all of them in a single night (I have separate MySQL servers so it was only the Apache/PHP side I was updating - much easier and quicker)
Hopefully some of this helps. Good luck
I work with a small web team that is currently in the process of getting GIT integrated into our development process. We develop locally, have a central bare repository and then pull changes down to separate test and production servers. This is working great for our files but we are hitting roadblocks when it comes to syncing MySQL databases.
We have a lot of sites built with Wordpress and the issues are more prominent here:
Wordpress inserts the domain name into the DB. Right now, we get around this by doing a find and replace whenever we move the sites from local, to testing and then to production. It would be nice if we didn't have to do this, though.
The production server site DBs are constantly changing (comments, etc.) and the testing server and our local servers are not in sync. This makes it difficult to send changes (after adding a plugin, page, etc.) to the production DB from the test server.
It would be great if we could find something that could integrate with GIT (maybe through githooks) that would allow us to sync the databases across different development and production servers. Moreover, it would be a bonus if there were a way to track changes within the database itself -- allowing us to merge changes (development edits and production changes) when pushing to production.
And finally, it would be even better if this could all work across multiple domains (local, testing and production); in other words, it would have to find and replace the URLs in the sql on each push/pull.
Thanks a bunch for any insight.
You might want to check out http://www.liquibase.org/. It's a database refactoring tool made for creating and modifying database schema, creating rollbacks and code for SQL generation. I was introduced to it a long while back and can't remember it that well, but it seems like it's made for what you need and from what I remember it kicks ass.
I was originally planning on using a local machine on our network as the development server.
Then I had the idea of using a subdomain.
So if the site was at www.example.com then the development could be done at dev.example.com.
If I did this, I would know that the entire software stack was configured exactly the same for development and production. Also development could use the same database as production removing the hassle of syncing the data. I could even use the same media (images, videos, etc.)
I have never heard of anyone else doing this, and with all these pros I am wondering why not?
What are the cons to this approach?
Update
OK, so its seems the major no no of this approach is using the same DB for dev and production. If you take that out of the equation, is it still a terrible idea?
The obvious pro is what you mentioned: no need to duplicate files, databases, or even software stacks. The obvious con is slightly bigger: you're using the exact same files, databases, or even software stacks. Needless to say: if your development isn't working correctly (infinite loops, and whatnot), production will be pulled down right alongside with it. Obviously, there are possibilities to jail both environments within the OS, but in that case you're back to square one.
My suggestion: use a dedicated development machine, not the production server, for development. You want to split it for stability.
PS: Obviously, if the development environment missed a "WHERE id = ?", all information in the production database is removed. That sounds like a huge problem, doesn't it? :)
People do do this.
However, it is a bad idea to run development against a production database.
What happens if your dev code accidentally overwrites a field?
We use subdomains of the production domain for development as you suggest, but the thought of the dev code touching the prod database is a bit hair-raising.
In my experience, using the same database for production and development is nonsence. How would you change your data model without changing your code?
And also 2 more things:
Its wise to prepare all changes in SQL script, that is run after testing from different environment not your console. Some accidental updates to live system made me headake for weeks.
Once happend to me, that restored backup didn't reproduced live system problem, because of unordered query result. This strange baviour of backup later helped us find the real problem simplier, than retrying on live system.
Using the production machine for development takes away your capacity to experiment. Trying out new modules/configurations can be very risky in a live environment. If I mess up our dev machine with an error in the apache conf, I will just slightly inconvenience my fellow devs. You will be shutting down the live server while people are trying to give you their money.
Not only that but you will be sharing resources with the live enviroment. You can forget about stress testing when the dev server also has to deal with actual customers. Any mistakes that can cause problems on the development server (infinite loop taking up the entire CPU, running out of HDD space, etc) suddenly become a real issue.
I have several databases hosted on a shared server, and a local testing server which I use for development.
I would like to keep both set of databases somewhat synchronized (more or less daily).
So far, my ideas to solve the problem seem very clumsy. Anyway, for reference, here is what I have considered so far:
Make a database dump from online databases, trash local databases, and recreate the databases from the dump. It's a lot of work and requires a lot of download time (which guarantees I won't do it as much as I would like it to be done)
Write a small web service to access the new data, and write a small application locally to communicate with said web service, download the newest data, and update the local databases.
Both solutions sound like a lot of work for a problem that is probably already solved a zillion times over. Or maybe it's even an existing feature which I completely overlooked.
Is there an easy way to keep databases more or less in synch? Ideally something that I can set up once, schedule and forget about.
I am using MySQL 5 (MyISAM) databases on both servers.
=============
Edit: I had a look at replication, but it seems that I can't go that route because the shared hosting does not give me enough control on the server itself (I got most permissions on my databases, but not on the MySQL server itself)
I only need to keep the data synchronized, nothing else. Is there any other solution that doesn't require full control on the server?
Edit 2:
Sorry, I forgot to mention I am running on a LAMP stack on the shared server, so Windows-only solutions won't work.
I am surprised to see that there is no obvious off-the-shelves solution for this problem.
Have you considered replication? It's not to be trifled with but may be what you want. See here for more details... http://dev.mysql.com/doc/refman/5.0/en/replication-configuration.html
Take a look at Microsoft Sync Framework - you will need to code in .net, but it can resolve your issues.
http://msdn.microsoft.com/en-in/sync/default(en-us).aspx
Here is a sample for SQL server, but it can be adapted to mysql as well using ado.net provider for Mysql.
http://code.msdn.microsoft.com/sync/Release/ProjectReleases.aspx?ReleaseId=4835
You will need the additional tables for change tracking and anchors (keeping track of last synchronization) for this to work, in your mysql database, but you wont need full control as long as you can access the db.
Replication would have simpler :), but this might just work in your case.