I'm a totally newbie with cloud computing, but I want to try it.
I want to relocate a public app developed by myself and currently served by a traditional hosting.
The requirements are just the basics.
- Windows Server 2008 R2 or later
minimal ram
.net 2.0 and .net 4.0 support
IIS
SQL Server 2008
Some 20gb, for the app, db, and files
Some 10gb more (DB backup)
With win2 + sql server I can't use the free testing. I know that and I'm ready to pay the platform needed. In the current hosting I'm paying for that too.
I just want see if someone can validate my configuration and say if I'm forgetting something:
1 instance EC2 Windows and Std. SQL Server m4.large
1 Amazon EBS Volumes 100gb, 3IOPS, 30% snapshots
1 Elastic IP for the unique instance
5GB Transfer in, out, interdata
http://calculator.s3.amazonaws.com/index.html#r=IAD&s=EC2&key=calc-D29B73A3-A7C5-4CEE-9C4F-3ED5A74D2420
Basically, the server will run the web app who has connection with the database installed in the same machine (maybe in the future with more visits the db will be moved to another vm). The web app will be public access.
Someone see something missing?
Should I buy amazon s3 storage for the backups?
Where will be saved the ebs volume backups? or with my configuration I can't have backups?
At the moment I'll keep using my own dns server and I hope be capable to configure for redirect to aws without need of amazon route 53.
at are you seeing, I need some orientation, because I'm a little lost in this new world. What services I need? what about the backups?
I'm not thinking at this moment in optimization (cache, load balance, ...). If the things go in the right way, in the future. Right now I just want the most simply installation. My problem, windows and sql, those can't be used in free version and I can't change the app.
I feel shame for be so annoying, but I still have those doubts.
There are a lot of pieces to this. I will try and break some of it down.
Should I buy amazon s3 storage for the backups? Where will be saved
the ebs volume backups? or with my configuration I can't have backups?
You can script a backup of the drive to S3, or back up the volume entirely as a 'volume'. Totally up to you. Backing up contents of the drive to S3 with a proper script (like this) would be a great idea.
At the moment I'll keep using my own dns server and I hope be capable to configure for redirect to aws without need of amazon route 53.
Route 53 is amazing and an extremely small charge for the flexibility it gives. I would recommend using it because it ties in neatly to other AWS services you might need (like load balancers) in the future.
I just want see if someone can validate my configuration and say if I'm forgetting something:
1 instance EC2 Windows and Std. SQL Server m4.large
1 Amazon EBS Volumes 100gb, 3IOPS, 30% snapshots
1 Elastic IP for the unique instance
5GB Transfer in, out, interdata
With an m4.large, you get 2 VCPUs and 8gb of memory, and 450mbps throughput. Do you need all that horsepower? You can try a t2.medium if that sounds like a lot for some burstable cpu performance. It all depends on what utilization you're at now, and if you're CPU/memory/bandwidth bound.
Feel free to ask more questions.
For backups of EBS volumes to S3, the best way to backup would be us EBS snapshots.
Related
I'm wondering can we setup SQL Server inside Amazon Lightsail? So I don't need to purchase Amazon RDS Plan. What's the difference if I prefer Amazon RDS as my database instance?
I want to build simple Point of Sales web with node.js and there are only about 10 users which use it.
You can actually have your database hosted a couple of different ways with Lightsail.
Lightsail has a managed database option that is built on top of RDS, so if you don't want to have to install and manage MySQL you can just use the managed offering. It will likely be more expensive than self-hosting, but comes with the benefit of not having to manage the underlying system, optional one-click high availability mode, and automatic backups. Today the managed service only supports MySQL - but we've announced that we're adding Postgres shortly.
If you'd rather pay a little less, you can (as stated above) install just about anything you want into a Lightsail instance. So you could provision a Lightsail instance and install MySQL (or Postgres or Mariadb or whatever).
Now if you want to run MSSQL, there isn't a managed offering, but there is a blue print for SQL express on Windows server available as well.
You can install whatever you want on Amazon Lightsail, as long as it is supported by the operating system. Lightsail is just an environment to launch virtual machines.
Amazon RDS is a managed database service, which means AWS takes care of many things you need to do manually otherwise. With RDS you get managed backups, easy on-demand snapshots, automatic patching, the possibility to scale up and down in a simple way, and advanced monitoring, performance metrics, and alerts. You also get a nice API to control your SQL Server/MySQL instance.
I see a Lightsail managed DB as an inferior option to RDS. Their logging system starts from the top so it is close to useless, so you can only take advantage of backups and snapshots. Also, the free tier of Lightsail only lasts for 3 months, and then you pay 15 USD monthly for 1GB in RAM and 2 vCPU. The free tier of Amazon RDS also gives you 1GB in RAM and 2 vCPU for 1 year, but with much better features and more future-proof.
To connect RDS with Lightsail you can follow these instructions from the AWS blog. The important bit is modifying the inbound configuration of the VPC security to allow Lightsail IPs.
So, based on this, this is how we use databases with lightsail:
We install Postgres or MySQL directly on our dev or testing Lightsail instances, those don't need backups and can be recreated every build, so no need for an extra service or cost. You can SSH into your instance for debugging and monitoring.
We use RDS for production and maybe staging, since this will provides us with backups, good monitoring, etc.
We avoid Lightsail DBs because their logging and monitoring is close to useless and costs the same as RDS.
From what I gather, the only way to use a MySQL database with Azure websites is to use Cleardb but can I install MySQL on VMs provided in Azure Cloud Services. And if so how?
This question might get closed and moved to ServerFault (where it really belongs). That said: ClearDB provides MySQL-as-a-Service in Azure. It has nothing to do with what you can install in your own Virtual Machines. You can absolutely do a VM-based MySQL install (or any other database engine that you can install on Linux or Windows). In fact, the Azure portal even has a tutorial for a MySQL installation on OpenSUSE.
If you're referring to installing in web/worker roles: This simply isn't a good fit for database engines, due to:
the need to completely script/automate the install with zero interaction (which might take a long time). This includes all necessary software being downloaded/installed to the vm images every time a new instance is spun up.
the likely inability for a database cluster to cope with arbitrary scale-out (the typical use case for web/worker roles). Database clusters may or may not work well when a scale-out occurs (adding an additional vm). Same thing when scaling in (removing a vm).
less-optimal attached-storage configuration
inability to use Linux VMs
So, assuming you're still ok with Virtual Machines (vs stateless Cloud Service vm's): You'll need to carefully plan your deployment, with decisions such as:
Distro (Ubuntu, CentOS, etc). Azure-supported Linux distro list here
Selecting proper VM size (the DS series provide SSD attached disk support; the G series scale to 448GB RAM)
Azure Storage attached disks being non-Premium or Premium (premium disks are SSD-backed, durable disks scaling to 1TB/5000 IOPS per disk, up to 32 disks per VM depending on VM size)
Virtual network configuration (for multi-node cluster)
Accessibility of database cluster (whether your app is in the vnet or accesses it through a public endpoint; and if the latter, setting up ACL's)
Backup / HA / DR planning
Someone else mentioned using a pre-built VM image from VM Depot. Just realize that, if you go that route, you're relying on someone else to configure the database engine install for you. This may or may not be optimal for what you're trying to achieve. And the images may or may not be up-to-date with the latest versions, patches, etc.
Of course, what I wrote applies to any database engine you install in your own virtual machines, where a service provider (such as ClearDB) tends to take care of most of these things for you.
If you are talking about standard VMs then you can use a pre-built images on VMDepot for that.
If you are talking about web or worker roles (PaaS) I wouldn't recommend it, but if you really want to you could. You would need to fully script the install of the solution on the host. The only downside (and it's a big one) you would have would be the that the host will be moved to a new host at some point which would mean your MySQL data files would be lost - if you backed up frequently and were happy to lose some data then this option may work for you.
I think, that the main question is "what You want to achieve?". As I see, You want to use PaaS solution with Web Apps or Cloud Service and You need a MySQL database. If Yes, You have two options (both technically as David Makogon said). First one is to deploy Your own (one) server with MySQL and connect to it from the outside (internet side). Second solution is to create one MySQL server or cluster and connect Your application internally in Azure virtual network. WIth Cloud Service it is simple but with Web App it is not. You must create VPN gateway in Azure VM and connect Your Web App to this gateway. In this way You will have internal connection wfrom Your application to Your own MySQL cluster.
Ok, I have my server built on ec2. My stack is Nginx as a load balancer, supervisord for managing processes for node.js i.e. one process for each cpu, and redis, master and slave on separate boxes. I have stress tested by testing failover and taking services offline. Using apache AB, on the server I can get up to 6500 QPS.
Now, I need to load test remotely. What are the best open source tools to accomplish this or even the most cost effective SaaS method to do this. I do expect 6500 QPS per server in production and need to extend the isolation of apache AB to remote testing. E.g. I will have servers in singapore and I need to test 6500 QPS from Japan and the effect of latency. I am aware of apache Jmeter but looking for a best practice solution.
Thanks
I have successfully used jMeter for load testing at significant scale.
If a single load generation client cannot output enough load, you can configure jMeter with multiple load generation clients, with the load coordinated by a master instance.
Using "open source tools" implies that you have the ability to spin up servers in the zones you're interested in (e.g. Japan). If you locate a cloud provider in that region, you can spin up as many load generation instances as needed. You may, however, need quite a few instances depending on the network connectivity offered to individual instances. The nice thing about jMeter is that it can coordinate many load generation instances.
You can use blazemeter as SaaS solution. It's 100% Jmeter compatible. There is Japan(Tokyo) load origin location which you need.
I have developed several websites on my local host for school. I can program database driven aspects of a webisite such as a blog on my own no problem. At school I used a MySQL DB which was hosted on a dedicated server, so when the server went down I could not have access to my data. I also have a mysql DB on my localhost but...my question is this: If i use the MySQL DB on my localhost, won't I only have access to the data when my computer is running so that my site can make server requests?
How can I set up any kind of DB for my site that is reliable and doesn't depend on my localhost to act as a DB Server?
I am now developing in .NET
Thanks. PS I don't have access to schools' server any more.
PPS I still don't have a site set up through a hosting provider...probalby going to go with godaddy
You'll need a server for that. A server will cost you on a monthly basis, I haven't heard of any free mysql servers, someone else may be able to help you with that.
Regular hosting providers supply PHP and MySQL server. I run my sites on Dreamhost (www.dreamhost.com)
It might also be worthwhile to look into Amazon EC2 : http://aws.amazon.com/ec2/
Also Amazon RDS for pure database only usage : http://aws.amazon.com/rds/
They provide a pay as you use service and the tier 1 service is free for the first year as I understand
There's no magic, you have to host your DB on a server that will run 24/7. Could be on the internet, it depends on the firewalls surrounding you.
The website will always depend on some sort of database server, why so worried that your database will be unreachable? Your webserver can go down also and will have the same consequences as the db.
Why don't you get some paid hosting? It is fairly cheap these days, for around 30dollar/year you will have some reliable hosting. When the DB will break down, it is not your responsibility.
If you don't want te be dependent of MySQL, you will need to use plain-text database that will be maintained by PHP or something.
You might consider running the MYSQL in i.e. a cloud provider, like Amazon EC2. It will certainly cost some money.
We have a Java EE application (EAR file deployed on JBoss, MySQL, MongoDB) which we would like to deploy on an Amazon EC2 instance. I have several questions regarding deployment best practices.
What is the most commonly used Linux AMI which we can rely on for a robust deployment (There are so many Linux variants, and I am not sure which AMI is commonly used, is it Fedora, CentOS, Red Hat, SUSE ...)
How do we handle production upgrades (EAR file modifications or schema upgrades). Are there any tools which are available to handle this installation or rollback of these changes.
What kind of data backup capability is available for the database?
Should I rely on Amazon RDS for MySQL support?
How should I handle support for MongoDB?
This is the first time, I am hosting an web-app and would appreciate some inputs on how to manage the production instance.
I agree with Mark Robinson's answer: Use whichever Unix variant you're most comfortable with. It may pay to pick one with decent cloud support. For my site I use Ubuntu.
I have a common image which is the base of every version deploy I do. I have www.mysite.com pointing to an Elastic IP so I can decide which instance it goes to. The common image has all the software I need installed (Postgres/Postgis/Tomcat/etc) but the database and web server data folders and symlinked to Elastic Block Store (EBS) instances.
When it comes time to do a deploy I start a new instance up, freeze and snapshot the EBS volumes on production and make new volumes. I point my new instance at the new volumes and then install whatever I need to onto that. Once I've smoke tested everything successfully I can switch the Elastic IP to point to the new instance and everything keeps on going.
I'll note that I currently have the advantage where only I can modify the database; no users can. This will become a problem shortly.
If you use the XFS filesystem on top of the EBS volume then you can tell XFS to freeze the file system (so no updates happen) then call the EC2 api to snapshot the volume then unfreeze the file system. The result is that the snapshot is taken quickly and sent to S3. I have a nightly script which does this.
If RDS looks like it will suit your needs then use it. Amazon is building lots of solid tools quickly and this will ease your scalability issues if you have any.
I'm sorry, I have no idea.
Good question!
1) I would recommend going with whatever Linux variant you are most comfortable with. If you have someone who is really keen on CentOS, go with that. Once you have selected your AMI, take it and customize it by configuring how you want it. Then save that AMI as you base-layout. It will make rolling out new machines much easier and save your bacon if EC2 goes down.
2) Upgrades with EC2 can be tres cool. Instead of upgrading a live system, take your pre-configured AMI, update that and save that AMI as myAMI-1.1 (or whatever). That way, you can flip over to the new system almost instantly AND roll back to a previous version in case something breaks. You can also back-up DB instances to S3. It's cheap at about $0.10/GB/Month.
3) It depends where you are storing your DB. If you are storing it on your EC2 instance you are in trouble. The EC2 instances have no persistence storage. So if your machine crashes, you lose everything. I'm not familiar with Amazon DB system but you should also look into Elastic Block Store. It's basically an actual hard-drive you can write to. When you want to upgrade your schema, do a full DB dump to S3 and then do an upgrade of your actual schema. If something goes wrong, you can pull the previous version out of S3.
4) & 5) I have never used those so I can't help you.
What is the most commonly used Linux AMI which we can rely on for a robust deployment (There are so many Linux variants, and I am not sure which AMI is commonly used, is it Fedora, CentOS, Red Hat, SUSE ...)
How do we handle production upgrades (EAR file modifications or schema upgrades). Are there any tools which are available to handle this installation or rollback of these changes.
What kind of data backup capability is available for the database?
Should I rely on Amazon RDS for MySQL support?
How should I handle support for MongoDB?
Any Linux AMI will do the job, what you need is a JRE only. (assuming development work not required). If you need to monitor the JVM behavior then get JConsole installed.
Easiest and painless way is to SSH into the local home directory, transfer the updated class file/EAR file (depends the number of changes applied) and copy and replace into the Tomcat deployment directory, restart apache. (make sure you tested locally before upload to production).
Depends on which database you are using, if you are using MySQL then just do scheduled backup that writes to your home directory so that from time to time you could SSH in and download a copy for backup purpose.
I would not consider reply on Amazon RDS for MySQL support due to 2 reasons: MySQL is small enough and manageable, and also I would want to have total complete control of the database and why pay for more when you can do it yourself FOC?
The usage of MongoDB should be align with the purpose of your application and benefits you gain from that. I would recommend you use MongoDB for static data retrieval like state, country, area etc... where MySQL to be use for transaction data only.
If you can live with deploying your Java EE application on TomEE instead of JBoss, Boxfuse does what you want.
For you Java EE application you literally only have to execute (TomEE uses war files instead of ear files):
boxfuse run my-tomee-app-1.0.war -env=prod
This will
Create AMI containing TomEE and your application ready to boot
Create an Elastic IP or ELB
Create a security group with the correct ports defined
Create an auto-scaling group
Launch your instance(s)
Any subsequent update will be done as a zero downtime blue/green deployment.
More info: https://boxfuse.com/blog/javaee-aws