The main bottle neck of a web server locates usually in the database,in my case,MYSQL.
More specifically,fulltext search and master-slave replication.
And sphinx is a probable solution for fulltext-search,so master-slave replication is the
final pain in ass.
Is it possible to boost the performance significantly with the technology of Cloud Coumputing,
for instance,by services offered by Amazon?
Just a wild guess!
EDIT:what about MySQL and Google App Engine?
Of course. MySQL Enterprise for Amazon EC2 is one MySQL package for Amazon EC2. See also Setting Up MySQL on an EC2 AMI and this tutorial/blog post.
EDIT: App Engine is higher-level than EC2 and is really designed for BigTable/GQL only. However, look at approcket, which allows replicating between AppEngine and MySQL.
You may want to be careful with just switching your web app to use an external data base (ie amazon, et.al.), you want to understand where exactly is your bottleneck or you may end up introducing more performance problems... Remember that by going to an external DB, you're introducing more latency into each query compared to a local (box or net) query.
If your problem is performance, try to find out exactly where the problem lies first, and then you may want to explore other options like query optimization, caching, etc.
Possible - for sure. See for example, xeround, rightscale, Amazon and phpfog. There are probably at least a few more with more to come. They come in varying degrees of "freeness" (How's that for a made up word?) too.
The question, it seems to me, will be performance and reliability.
Who knows, localhost may become a thing of the past for development.
Related
This is going to be a generic question.
We are a young startup faced with the inevitable problem of scaling and during our research, Apache Mesos seemed like a good fit for our architecture, which is –
Core Scala based microservices, each responsible for dealing with a
part of our database, which is mainly MySQL
Middleware microservices,
to deal with some other persistent data-storage systems like MongoDB,
Elasticsearch etc.
Which basically means that we can containerise all of our services and ship them to a single datacenter which can then deploy these containers in a topographically agnostic way.
What we are currently stumped by is –
Mesos doesn't seem to have any native support for MySQL
Container based persistence seems awfully tricky and hard to manage/maintain.
We'd like to continue using MySQL/MongoDB/ElasticSearch because migrating to Cassandra etc. at this stage (we are a small team) is too much of an overhead and hence not an option.
What are the best strategies for this problem?
Mesos provide Persistent resources support for storage-like services.
If you want to use MySQL on mesos, please consider try https://github.com/apache/incubator-cotton
After some research we decided not to try Cotton but we're still sticking to deploying our services across a Mesos cluster.
Instead of hosting our own MySQL database, we decided to outsource it to Amazon RDS. But we're now faced with problems like doing something for our other databases like MongoDB.
What's the advantage of using Google's Cloud SQL offering versus installing/maintaining instances yourself?
It seems to me that most of the features they list can be implemented fairly trivially by experienced mysql admins. Am I wrong?
CLARIFICATION: I am not asking which product is better. I am trying to understand what features Cloud SQL adds on top of self-hosted MySQL installations.
You can look at these types of questions from several angles: money, maintenance effort and quality of service.
Money-wise there is probably not that much of a difference, although it would not hurt to make the calculations for your usage scenario just to be sure.
In terms of maintenance-effort you are probably better of with using the Cloud SQL since most mundane tasks (such as creating back-ups and upgrading versions) are already handled for you.
Depending on your context, the quality of service of a 'hand-maintained' instance might be good enough, but if you need to support multiple sites then it quickly becomes a nuisance to create proper replications across the world (which from the looks of it is a standard feature of Cloud SQL).
Overall I would say it is greatly a matter of convenience.
My MYSQL servers are not configured properly with failover. I am thinking of using Redhat cluster or heartbeat. Also, I need to achieve all tasks w/ one floating IP since application does not know about multiple databases. Can someone suggest what route would be the very for best?
If you decide on using regular replication I would recommend you using this:
MHA tool for MySQL replication high availability
I didn't have a good experience with the ways you want to use, they are not as good as a cluster and not so much better than normal replication with the tool I posted. I do believe you should take a look at this tool and maybe stick with regular replication or jump to serious HA with MySQL cluster. Configuring the cluster might be hard but using Several Nines Solutions things might be a bit easier.
We are currently evaluating failover support in different databases.
We were earlier using HSQLDB but it seems that it does not have clustering/replication support.
Our requirement is simply to have two database servers, one being only for synchronous backup but if the primary server is down, then the secondary should automatically start acting as the primary server.
Has anyone evaluated MySQL, PostgreSQL or any other DB server for such a use case?
Edit: We had thought of using MySQL cluster but it now seems that it is under GPL license which we won't be able to work with. Could anyone please suggest a synchronous replication/clustering solution which can be used? We are currently using HSQL, so a solution with HSQL used in clustered mode will be ideal for us but we are open for change.
Stackoverflow resources
MySQL supports replication out of the box: see this question for MySQL: Scaling solutions for MySQL (Replication, Clustering)
PostgreSQL also support replication, see this question for that: PostgreSQL replication strategies
If your requirements are simple MySQL will work
I've used MySQL is a simple master-master failover scenario using the setup I read in High Performance MySQL. I highly recommend the book if you're keen on using MySQL.
It has worked well for me, because I just wanted a simple fail-over.
If your use case is just as simple. It will work well.
Just for completeness, the H2 database has some clustering support, but compared to the MySQL and PostgreSQL features it is very limited, it's really only failover. I would first look at HA-JDBC.
for a simple failover where servers are on the same location. you can use DRBD and Heartbeat.
In a nutshell: DRBD stores the data on 2 servers on the same time. fully transparent to the system. with heartbeat the standby checks against the main server, if its not reachable, it takes over the resource, mounts it and starts the database daemon. (works with mysql, postgres and most probably with most other daemons out there)
There is a third-party product that works with HSQLDB:
http://ha-jdbc.sourceforge.net/
Not sure this is within the desired price range of most FOSS-type people :-) but we use DB2 9.7 for exactly this purpose (actually, we mostly use DB2/z on the mainframe for it, but some customers like the DB2/LUW (Linux/UNIX/Windows) option for smaller systems).
DB2 comes with high availability (HA) features built in and you can use db2haicu, the DB2 High Availability Instance Configuration Utility (gotta love those acronym generators employed by Big Blue) to configure things relatively painlessly.
It's active/passive as you desired, although DB2 is certainly capable of active/active setups for load balancing.
The particular setups we're most familiar with at the low end (everything other than a mainframe) are actually shared disk ones, with the HA applying to only DBMS resources and not data, but you can separate the data with DB2 replication features as well.
We've had one client (at least) using Q replication, which is a very low latency replication method, close to synchronous but not quite. DB2 does actually provide real synchronous replication as well.
DeveloperWorks has an interesting article on how this all hangs together, along with the various options.
The only good reference that I can find on the internet is this whitepaper, which explains what database tiering is, but not how it works:
The concept behind database tiering is
the seamless co-existence of multiple
(legacy and new) database technologies
to best solve a business problem.
But, how does it implemented? How does it work?
Any links regarding this would also be helpful. Thanks.
I think the idea of that document is you to put "cheap" databases in front of the "expensive" databases to reduce costs.
For example. Let's assume you have an "expensive" db...something like Oracle, or DB2 or even MSSQL (more realistically it's probably more of an issue with a legacy DB system that is not supported much or you need specialized resources to maintain). A database engine that costs a lot to purchase and maintain (arguably these are not expensive when you take all factors into consideration. But let's use them for the example).
Now if you suddenly get famous and your server starts to get overloaded what do you do? Do you buy a bigger server and migrate all your data to that new server? That could be incredibly expensive.
With the tiering solution you put several "cheap" databases in front of you "expensive" database to take the brunt of the work. So your web servers (or app servers) talk to a bunch of MySQL servers, for example, instead of directly to the your expensive server. Then these MySQL servers handle the majority of the calls. For example, they could handle all read-only calls completely on their own and only need to pass write-calls back to the main database server. These MySQL servers are then kept in sync via standard replication practices.
Using methods like this you could in theory scale out your expensive server to dozens, if not hundreds, of "cheap" database servers and handle a much higher load.
Database tiering is just a specific style of tiering. There are also application tiering and service tiering. It's a form of scalability.
What exactly are you asking? This question is rather vague.
This is a PDF from a course at Ohio State. What it discusses is a bit over my head, but hopefully you might understand it better.