I need to add a column to my current table.
This table is used a lot during the day and night. i found out i need to alter using the alter command found here
http://dev.mysql.com/doc/refman/5.1/en/alter-table.html
I tested it on a development server.
i took about 2 hours to complete. Now I want to execute this on the production server?
will this stop my website?
Why not display a message on the site saying you will perform maintenance from midnight UTC time January 7 2012.
This way, you won't break any data, you will not get any mysql errors. you execute your ALTER and you start the site again once its completed (don't forget your code to make sure you have the right field etc..). Easy solution.
Stackoverflow does it, why not yours?
Yes, during an ALTER TABLE all reads and writes are blocked. If your website needs to use that table, requests will hang.
Try pt-online-schema-change. It allows reads and writes to continue, while it captures changes to be replayed against the altered table once the restructure is done.
Test carefully on your development server so you know how it works and what to expect.
It won't stop your website, but it will likely make it throw errors.
Of course there is no way to answer this without looking at all the code of your application.
The bottom line is, when in doubt schedule a maintenance window.
Make the production server to point to dev db (or) mirror of prodcution db for some time.
Alter the table in production
Deploy the code which talks to production db (with the new attributes)
P.S: I feel this is safer and a fool proof way (based on my experience).
Related
I'm new to MYSQL and there is something really weird happened and I can't figure out why.
Recently, the INSERT query to some of the table become extremely slow. Weirdly enough, the query time all around 60 secs.
The tables are all with the only 10k to 35k entries, so I think they are not that big.(But indeed they are the biggest one in the database, though.)
And the slowness is only with INSERT query, DELETE, UPDATE, SELECT are all executed with 0.000x sec.
Can some help me find out why is this happening?
UPDATE: So I turned on the general log and noticed all my INSERT queries are followed with 'DO sleep(60)'. It seems my server got hacked?
Where can I find this malicious script inject the sleep() command after INSERT?
If you use code to build the queries, copy the code base off the server to your machine (ideally in a VM, just in case) and search for the changes within the code. Alternatively, you could restore the code base from source control (you use source control, right?!).
If it's store procedures you use, you'll need to change them back to a working version without the sleep. Check previous backups to try and find out when this happened, which might help a wider investigation as to how they got in and did what they did.
You'll also need to think about the wider implications of this. Do you store user data? If so, then you'll need to inform them that you've had your database compromised and therefore they should assume their accounts are and change their passwords.
Finally, wipe the server. A hacked server is no longer in your control (or that's how you should look at it). Wipe it, reinstall everything, and put in changes to help prevent the same hack happening again.
I've been doing a lot of research, reading on replication, etc but just not sure as to what mysql solution would work.
This is what I'm looking at:
when my mysql fails for some reason or there are certain queries that are taking really long to execute and locking some tables, I want the other insert/update/select queries to still function at normal speed without having to wait for locks to be released or for the main database to be back up. I'm thinking there should be a second mysql server for this to happen, but is what I mentioned possible even if there is and would it involve a lot of change in my existing programming logic?
when my database is being backed up, I would still like my site to function normally, all inserts/selects/updates should function as normal.
when I need to alter a large table, I wouldn't like it to affect my application, there should be a backup server to work from.
So what do I need to do to get all this done and also would it require changing plenty of existing coding to suit the new set up? [My site has a lot of reads and writes]
There's no easy way. You're asking for a highly-available MySQL-based setup, and that requires a lot of work at the server and client ends.
Some issues, for example:
when I need to alter a large table, I wouldn't like it to affect my application, there should be a backup server to work from.
If you're altering the table, you can't trivially create a copy to work from during the update. What about the changes that are made to your copy while the first update is taking place?
Have a search for "High Availability MySQL". It's mostly a solved problem, but the solution depends heavily on your exact requirements. You cannot just ask for "I want my SQL server to run at full speed always forever no matter what I throw at it".
Not a MySQL specific answer, but a general one. Have a read only copy of your DB for site to render, which will be synced to the master DB regularly. This way, you can keep your site working even if the master DB is under load/lock due to insert/delete. For efficiency, keep this copy as denormalized as you can.
I have a site set up using CakePHP and MySQL and I want to work on a test database without disrupting my live site in case something goes wrong.
I have another busy site, but my test site runs off the live database which can be occasionally nerve wracking.
What do I do if I change a table name in the test db and I want it changed in the live database? Or if I remove a record from the test database. Is there a way to diff the changes? How do I even merge those changes?
How does this interfere with live user edits and things of that nature?
Hopefully some of you working devs can share some insight!
As I said in the comment, there are too many questions at once here IMO.
However, as for this question:
What do I do if I change a table name in the test db and I want it changed in the live database
this is comparably easy to do manually: Any mySQL client will show you the exact SQL query that was made to change a table or a record. You would keep track of every change, and build "changesets" from those queries, i.e. just series of queries that you then run on your live database, for example after putting the site into maintenance mode for a moment.
This is enough in many, many small to mid-size use cases.
To get answers on more sophisticated topics like database replication, clustering and such, I think you will need to refine your answer.
How can I convert a database from MySQL to MS SQL Server 2005?
You can use SSIS to copy over the table data to the new structure, but that is the easy part. Next you need to check all your sql code to make sure it will still work. This link can help see the differences between how each of the databases implements SQL
http://troels.arvin.dk/db/rdbms/
While you are converting, you might consider if now might not be the best time to do some refactoring as well.
The key piece of doing a conversion though is to make sure that everything is automated and reproducable. You are going to want to do this several times in dev before moving to prod data. And when you go to prod, you will need to take the database down for maintanance or you will end up having data added to the old database after you have moved the data from that table to the new one. You might even want to build the process to copy over the bulk of the rcords before the maintenance window and then during the maintenace window only move the new records or records which hae changed since the main move. This will depend on on how big your database is and how long you will have to be down to move the records. If it can be done in one step without being down for longer than your system can tolerate, it is better to do that. Another choice for a large database might be a client by client data movement, so that instead of being down for everyone for a full day, you are only down for a couple of hours per client. Again this depends on your database design and how possible that might be to set up and do.
Whatever you do, make sure the users are fully aware of what you are doing and when in advance so they can plan. Also avoid times of the month for the change that would coincide with a need for the database to be up and running - I'm thinking in terms of don't close the payroll database the day that payroll runs or the finanical database when end of the fiscal year tasks need to be done or monthly reports run, etc. I don't know if you have any of those issues, but is is good to consider if you do and work around those periods. If the users say, "No we can't do that on Friday", then find out why - they may have a really good reason why the day you choose to implement is bad for their own work schedules.
Here is a application that will do the conversion for you:
http://www.spectralcore.com/fullconvert/tutorials/convert-mysql-to-mssql-sql-server.php
This white paper by Microsoft may also help too:
http://technet.microsoft.com/en-us/library/cc966396.aspx
I wish to migrate the database of a legacy web app from SQL Server to MySQL. What are the limitations of MySQL that I must look out for ? And what all items would be part of a comprehensive checklist before jumping into actually modifying the code ?
First thing I would check is the data types - the exact definition of datatypes varies from database to database. I would create a mapping list that tellme what to map each of the datatypes to. That will help in building the new tables. I would also check for data tables or columns that are not being used now. No point in migrating them. Do the same with functions, job, sps, etc. Now is the time to clean out the junk.
How are you accessing the data through sps or dynamic queries from the database? Check each query by running it aganst a new dev database and make sure they still work. Again there are differences between how the two flavors of SQl work. I've not used my sql so I'm not sure what some of the common failure points are. While you are at it you might want to time new queries and see if they can be optimized. Optimization also varies from database to database and while you are at it, there are probably some poorly performing queries right now that you can fix as part of the migration.
User defined functions will need to be looked at as well. Don't forget these if you are doing this.
Don't forget scheduled jobs, these will need to be checkd and recreated in myslq as well.
Are you importing any data ona regular schedule? All imports will have to be rewritten.
Key to everything is to use a test database and test, test, test. Test everything especially quarterly or annual reports or jobs that you might forget.
Another thing you want to do is do everything through scripts that are version controlled. Do not move to production until you can run all the scripts in order on dev with no failures.
One thing I forgot, make sure the dev database you are running the migration from (the sql server database) is updated from production immediately before each test run. Hate to have something fail on prod because you were testing against outdated records.
Your client code is almost certain to be the most complex part to modify. Unless your application has a very high quality test suite, you will end up having to do a lot of testing. You can't rely on anything working the same, even things which you might expect to.
Yes, things in the database itself will need to change, but the client code is where the main action is, it will need heaps of work and rigorous testing.
Forget migrating the data, that is the last thing which should be on your mind; the database schema can probably be converted without too much difficulty; other database objects (SPs, views etc) could cause issues, but the client code is where the focus of the problems will be.
Almost every routine which executes a database query will need to be changed, but absolutely all of them will need to be tested. This will be nontrivial.
I am currently looking at migrating our application's main database from MySQL 4.1 to 5, that is much less of a difference, but it will still be a very, very large task.