My company has a requirement that we want to build a private DB cloud service for internal use.
Requirements are:
User can easily request a new mysql instance, and terminate it.
Each mysql instances are isolated with each other.
One of the solution we have is just using to create different user and schema for each user. Something similar to what the cPanel is doing.
But I wonder is there better option available?
Honestly, I am don't think putting everybody on the single big MySQL instance is a good idea.
First, we can't do much about resources management. And I am afraid having a problem in the database (can't boot it up for example) is going to kill everybody.
To minimize the risk of single point of failure, I am looking for something like the Amazon RDS and Azure MySQL. What we want is very similar to that.
Does anybody know how are they do that? Is there is open source or commerical version we can buy?
Thanks you.
Without knowing the why it is hard to give you a good answer to your question. If Amazon RDS or Azure MySQL look like a good solution to you, I would suggest using those. Building such a service yourself and making sure it will scale well will probably cost a lot more money.
I mean, sure you could set up Kubernetes or Hashicorp Nomad and deploy containers there but you would need to figure out how these tools work, how to let MySQL run in a scaleable fashion, and build some kind of UI to easily launch and stop MySQL instances.
Related
I am making a Javafx program and need to use a small mySQL database. Currently I am hosting one on my computer but I can't access it on other computers on other networks. I need the mySQL server to be accessible from anywhere. How do I host one that does that? Thanks in advance, all help is welcome.
Well you have a few options depending on how important this MySQL database is to you, how you intend to connect to it from outside, and what you want to do with it.
The naive implementation would involve opening your firewall and directing all incoming traffic using whatever port you have configured MySQL for to point to the ip address of your server. If you do this you absolutely must secure your database with a password!!! You'll also need to keep the server's public ip address handy so you know how to find it when you go out.
Use Amazon AWS, Google Compute, Google App Engine, or some other cloud platform to host a MySQL instance. All the big players also tend to host pretty awesome RDBMS solutions. The advantage here is that you're not exposing your home computer to malice and you are connecting into an ecosystem that will answer a lot of other questions for you as they come up along the way (IE - how do you ensure redundancy? Backups? Scale your network for traffic?). There's a ton of other advantages too. It's the cloud... dude...
Use a SaaS DB service such as Firebase (Note: We are leaving MySQL and SQL database territory with Firebase)
If you plan to let other parties access your MySQL instance to make use of your data, you might also want to consider implementing a REST API (or SOAP API if you hate the future) which acts as an abstraction layer to interact with and provide the data from your database in a consistent and reliable format.
Best answer I can give with the details afforded - look around though the options in this arena are near limitless depending on how and what you're trying to do.
You should be able to access your machine from your LAN pretty easily unless there is some firewall rules preventing opening connection to your machine. Another way is there are many cloud shosting providers has free tier you can signup to bring up a test instance of mysql. Example: Open Shift.
I have several databases hosted on a shared server, and a local testing server which I use for development.
I would like to keep both set of databases somewhat synchronized (more or less daily).
So far, my ideas to solve the problem seem very clumsy. Anyway, for reference, here is what I have considered so far:
Make a database dump from online databases, trash local databases, and recreate the databases from the dump. It's a lot of work and requires a lot of download time (which guarantees I won't do it as much as I would like it to be done)
Write a small web service to access the new data, and write a small application locally to communicate with said web service, download the newest data, and update the local databases.
Both solutions sound like a lot of work for a problem that is probably already solved a zillion times over. Or maybe it's even an existing feature which I completely overlooked.
Is there an easy way to keep databases more or less in synch? Ideally something that I can set up once, schedule and forget about.
I am using MySQL 5 (MyISAM) databases on both servers.
=============
Edit: I had a look at replication, but it seems that I can't go that route because the shared hosting does not give me enough control on the server itself (I got most permissions on my databases, but not on the MySQL server itself)
I only need to keep the data synchronized, nothing else. Is there any other solution that doesn't require full control on the server?
Edit 2:
Sorry, I forgot to mention I am running on a LAMP stack on the shared server, so Windows-only solutions won't work.
I am surprised to see that there is no obvious off-the-shelves solution for this problem.
Have you considered replication? It's not to be trifled with but may be what you want. See here for more details... http://dev.mysql.com/doc/refman/5.0/en/replication-configuration.html
Take a look at Microsoft Sync Framework - you will need to code in .net, but it can resolve your issues.
http://msdn.microsoft.com/en-in/sync/default(en-us).aspx
Here is a sample for SQL server, but it can be adapted to mysql as well using ado.net provider for Mysql.
http://code.msdn.microsoft.com/sync/Release/ProjectReleases.aspx?ReleaseId=4835
You will need the additional tables for change tracking and anchors (keeping track of last synchronization) for this to work, in your mysql database, but you wont need full control as long as you can access the db.
Replication would have simpler :), but this might just work in your case.
I have been having the hardest time for the last couple of months to really grasp mysql. Everytime no matter what the framework, programming language, computers (pc's), I have had no luck.
My question is this, is there some functionality in mysql preventing the database crud operations from being run on a different domain name? Could it be my firewall settings? I even have a hard time connecting to local databases. Look at my past questions to see what I mean. Is it at all easier to use mysql on a mac / linux?
I'll take the question and break it into parts.
1.) is there some functionality in mysql preventing the database crud operations from being run on a different domain name?
1a.) Don't know what you really mean. Are you trying to ask if it's possible to yank out the actual "data" in the database and store it on one server, and have another server perform crud on it?
2.)Could it be my firewall settings?
2a.) Could always be firewall settings. if you're having trouble connecting, double check them, but often times it is simply the case that you haven't enabled a user to connect remotely. See this guide for step-by-step: http://help.hardhathosting.com/question.php/87
3.)Is it at all easier to use mysql on a mac / linux?
3a.) If you're used to a linux environment, i'd say yes. If you're a pure windows guy, then it'll probably be easier in the short-to-mid term to stay on windows. But Linux is fun once you know it :-)
The main bottle neck of a web server locates usually in the database,in my case,MYSQL.
More specifically,fulltext search and master-slave replication.
And sphinx is a probable solution for fulltext-search,so master-slave replication is the
final pain in ass.
Is it possible to boost the performance significantly with the technology of Cloud Coumputing,
for instance,by services offered by Amazon?
Just a wild guess!
EDIT:what about MySQL and Google App Engine?
Of course. MySQL Enterprise for Amazon EC2 is one MySQL package for Amazon EC2. See also Setting Up MySQL on an EC2 AMI and this tutorial/blog post.
EDIT: App Engine is higher-level than EC2 and is really designed for BigTable/GQL only. However, look at approcket, which allows replicating between AppEngine and MySQL.
You may want to be careful with just switching your web app to use an external data base (ie amazon, et.al.), you want to understand where exactly is your bottleneck or you may end up introducing more performance problems... Remember that by going to an external DB, you're introducing more latency into each query compared to a local (box or net) query.
If your problem is performance, try to find out exactly where the problem lies first, and then you may want to explore other options like query optimization, caching, etc.
Possible - for sure. See for example, xeround, rightscale, Amazon and phpfog. There are probably at least a few more with more to come. They come in varying degrees of "freeness" (How's that for a made up word?) too.
The question, it seems to me, will be performance and reliability.
Who knows, localhost may become a thing of the past for development.
I am just beginning to do research into the feasibility of using Amazon's SimpleDB service as the datastore for RoR application I am planning to build. We will be using EC2 for the web server, and had planned to also use EC2 for the MySQL servers. But now the question is, why not use SimpleDB?
The application will (if successful) need to be very scalable in terms of # of users supported, will need to maintain a simple and efficient code base, and will need to be reliable.
I'm curious as to what the SO communities thoughts are on this.
The Ruby SimpleDB library is not as complete as ActiveRecord (the default Rails DB adapter), so many of the features you're used to will not be there.
On the plus side it's schemaless, scalable and works well with ec2.
If you're going to do things like full text search in your app then SimpleDB might not be the best choice, stick with AR + sphinx.
Well, considering simple DB doesn't use SQL, or even have tables, means that it's a completely different beast than MySQL and other SQL-based things (http://aws.amazon.com/simpledb/). There are no constraints, triggers, or joins. Good luck.
Here's one way of getting it up and running:
http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1242
(via http://rubyforge.org/projects/aws-sdb/ )
I suppose if you're never going to need to query the data outside of rails, then SimpleDB may prove to be OK. But as it's not a first-class supported DB, you're likely to run into bugs that are difficult to fix. I wouldn't want to run a production rails app in a semi-beta backend.
To me this just feels like, "Hey there are these neat tools out there, I should go build a project using them," rather than actually needing to use these specific tools. Maybe I'm just being crabby but it feels like a classic case of premature optimization. You're trying to use an external service that costs money for an app that isn't even written yet and you don't say you've got a guaranteed audience or one that will necessarily scale to a level that warrants that.
"The application will (if successful) need to be very scalable in terms of # of users supported", seriously, that describes half the Internet. It's the "if successful" part that's really the question. Just concentrate on building the application quickly and easily. The easiest way to do that is just use ROR as it is out-of-the-box so to speak. Pair it with a database, use ActiveRecord and get something built and attracting users.
In fact, I'll go further and say that EC2 is rather expensive for always on servers. Deploy it over on Slicehost or another hosted solution and then move it to EC2 if you need to in order to support demand.
I myself am very interested in this topic. Right now I'm on a cloud computing high so I'd say go with SimpleDB since it'll probably scale better in the sense that you'll have high availability, but that's just my thoughts as of the moment. Not from experience yet.
Edit: It's true that SimpleDB has no normal features a "normal" database, but it should do the trick if you only need a simple CRUD layer to work against, which is my case
There's a library called SimpleRecord that is a drop in replacement for ActiveRecord, but uses SimpleDB as its backend data store.