here is the issue:
I used the CloudSQL price calculator to estimate the price of running a website, my website has 1000-2000 URLs and each URL will use the DB in some way, I don't have more than 1GB of data, and I mostly deal with reads, for a small 50k record table, nothing super-complicated, I don't currently have very complex queries either, and I write into the db only once a week maybe a couple records here and there, I've even considered SQLITE tbh.
I don't currently have a lot of traffic, maybe people come visit once a day, however, GoogleBot will continuously try to index the website via the sitemap, which causes some times lots of requests on the server.
Currently, I have a normal php+mysql website which does the job on a DigitalOcean instance, which doesn't take a lot of resources, however, I want to move to Cloud Run in order to try the Cloud Run technology, but running MySQL directly on the VM is discouraged (as per this question Should I run mysql on google cloud run? (or any database))
So I'm kind of afraid of using CloudSQL and then having GoogleBot destroying my credit card by doing lots of concurrent requests into the CloudSQL Database during daily indexation.
Traffic doesn't scare me (I don't have any), but crawlers do.
Should I use CloudSQL for this usecase?
Will my credit card be destroyed?
Are these valid concerns?
Any opinion from experienced CloudSQL Users would be greatly appreciated.
If you consider fully managed database instance Google Cloud is definitely good choice for you.
If you want to optimize GoogleBot crawling, you can do it from here
However, if you experience high server load from specific sites/services you may consider blocking them or using Google Cloud CDN caching
Please read this article will explain how to deal with heavy bot load on the website
Your concerns do not sound valid to me, since you can limit GoogleBot crawling rate.
Since Cloud Run is compute platform STATELESS container service, it is not suited to install MySQL. If you are searching to install your own MySQL server and manage it, you can do it on Cloud Compute Engine using one click solution from Marketplace
Related
I've been researching the new options to run Docker containers in Google Cloud Run, however, there seems to be no advice on whether or not one should run MySQL on Cloud run, apparently, I know it isn't a web service, and I understand in the Official Google Documentation for GCP, Google would probably just tell people to kindly use Cloud SQL (their SQL Offering), I haven't found any advice online about "running mysql on cloud run", so I thought I'd ask here.
Will startup times from cold starts decrease performance of the solution? (assuming one uses a Bucket for storing the stuff)
Running a SQL database is not a good fit for Cloud Run.
First of all, the contract between the deployed container and Cloud Run is that the container needs to run an HTTP server on port 8080. That's not really the way MySQL works.
Second of all, the container is going to be limited to the filesystem that was included in the container image. This same image is going to be instantiated many times over as the service handles load. There will be no way to persist the data written to MySQL. You could have read-only data stored in that image that only changes when a new image is published, but that's not really what you would expect to use a relational database for.
Cloud Run is really good at operating HTTP/web services in a serverless and scalable way. These web services typically make use of other APIs and service deployed to Google Cloud, or third party services. It's not really meant to offer persistent, scalable, ACID-compliant database services - this is a whole different sort of problem space.
I have looked everywhere...
whats the difference between a hosted database and a cloud database they seem like the same things?
Thanks
Both "hosted database" and "cloud database" mean that the database lives on the servers of some external provider/hoster.
The hoster might even be the same in both cases.
The main difference is that the "cloud" plans are usually meant to scale more (at a higher monthly fee), so you'd use them when you expect your site to get huge soon and need to quickly adjust server capacity when needed.
On the other hand, "hosted" plans are not that expensive, but have more limitations (server speed, database size...) and are more suited for "small" websites.
Of course this isn't by any means an "official" description of the two terms, but that's the impression that I get every time I see "cloud" or "hosted" webspaces/databases/services/whatever.
It depends on the context in which they're being used, but, yes, they usually mean the same thing. When I see the term cloud database being used they are usually referencing some cloud platform like Amazon EC2 or Microsoft Azure instead of GoDaddy or HostGator or something. Plus, cloud is the new buzz word - I'm sure it sells better. Lol.
As Christian Specht said, the cloud servers really scale more. So why you need more scaling? and why there are many featured options in cloud database service selection?
Things are not like before. Before smartphones and earlier pc operating systems, users gets information from the server only when they log on the specific web page using their credentials. But now apps like facebook shows notifications, provide ads etc and collect/push other data in parallel while we are looking at something else irrelevant.
Hosted database are reliable to access the database when users log onto the web page. But in case of the lastest smart phone applications, it needs to access the database everytime starting from its birth (installation on the device). So for each installation, the minimum workload over the server is expected to raise up.
So more scalability is required here. More simultaneous connections, Input/Output operation requests are expected daily. So with the dedicated servers with the core purpose, and with the configurable package selection based on your expectation of user count and bandwidth usage, Cloud Service is not yet another marketing term, but is a helpful service.
My partner and I are trying to start a website hosted in cloud. It has pretty heavy ajax traffic and the backend handles money transactions so we need ACID in some of the DB tables.
Currently everything is running off a single server. Some of the AJAX traffic are cached in text files.
Question:
What's the best way to scale the database server? I thought about moving mysql to separate instances and do master-master duplication. However this seems tough and I heard I might lose ACID properties even with InnoDB? Is Amazon RDS a good solution?
The web server is relatively stateless except for some custom log files and the ajax cache files. What's a good way to scale to multiple web servers? I guess the custom log files can be moved to a reliable shared file system or DB but not sure what to do about the AJAX cache file coherency across multiple servers. (I dont care about losing /var/log/* if web server dies)
For performance it might be cheaper to go with larger instance with more cores and memory but eventually I would need redundancy so wondering what's the best way to do this cheaply.
thanks
take a look at this post. there is plenty of presentations on the net discussing scalability. few things i suggest to keep in mind:
plan early for the data sharding [even if you are not going to do it immediately]
try using mechanisms like memcached to limit number of queries sent to the database
prepare to serve static content from other domain, in the longer run - from ngin-x-alike server and later CDN
redundancy - depends on your needs. is 'read-only' mode acceptable for your site? if so - go with mysql replication + rsync of static files and in case of failover have your site work in that mode till you recover the master node. if you need high availability - then take a look either at drbd replication [at least for mysql] or setup with automated promotion of slave server to become master node.
you might find following interesting:
http://yoshinorimatsunobu.blogspot.com/2011/08/mysql-mha-support-for-multi-master.html
http://mysqlperformanceblog.com
http://highscalability.com
http://google.com - search for scalability, lamp, failover... there are tones of case studies and horror stories from the trench lines :-]
Another option is using a scaleable platform such as Amazon Web Services. You can start out with a micro instance and configure load balancing to fire up more instances as needed.
Once you determine average resource requirements you can then resize your image to larger or smaller depending on your needs.
http://aws.amazon.com
http://tuts.pinehead.tv/2011/06/26/creating-an-amazon-ec2-instance-with-linux-lamp-stack/
http://tuts.pinehead.tv/2011/09/11/how-to-use-amazon-rds-relation-database-service-to-host-mysql/
Amazon allows you to either load balance or change instance size based off demand.
I have developed a social networking site using the elgg framework and I am hosting it on amazon cloud (Amazon EC2, the free tier micro instance service) and thus develop a benchmark for it.I am creating around 200 columns describing each user (most of them dummy) and after that I should create around a million users with each users profile updated with some data.This is done to reflect the image of big data.When hosted on cloud we should measure the cloud's performance based on a query and an update action for all users. The problem is how to create so many users? Which tool would be optimal to choose? Done this, I should also consider storage on a file system(HDFS) and do the same with some modifications (The output should be a row and the input should be an unstructured data).
For elgg framework we are using mysql as backend. I have no idea how to start with it. Any suggestions would be really helpful.
Thank you.
I had to perform a similar task recently and came up with this script.. maybe it can help you
I don't think I fully understand Amazon Web Services yet, which is why I'm asking this question. I want to know if AWS would be a nice host for a CakePHP application that of course runs off PHP, and MySQL?
Would I have to change or add anything to my code if used a service like EC2? I also noticed that Amazon has it's own database system, does that mean no mysql.
Also as a side question, what do you guys think of the cost AWS?
Sorry if my question is a little scattered, or somewhat "noobish," but that is why I ask it.
Thanks in advance!
It seems this question has been answered 2 weeks ago. Yes, my current company hosted their CakePHP app (1.2.1, production & development) on AWS Asia Pacific (singapore) and EU West (Ireland). Both production and development servers are m1.xlarge (standard 4-core, 15GB, Fedora 8 64-bit). We tried LAMP starter before, but it's too small and cannot handle web traffic demand.
Basically you don't have change your code, unless you have reached the milestone having enough traffic that forces you to switch to CloudFront (CDN), RDS, and Load Balancer features (that's currently I am doing now.) But you have been warned to plan it as early as possible.
For the cost, the bill we received for June is around USD$550 ~ 600 (roughly m1.xlarge server running 24x7), and for July is USD$1100 ~ 1200 (roughly two m1.xlarge server running 24x7.) This would give you the idea how much it cost.
unless you need a dedicated instance just for your cakephp application
you may be better off using a host that provides a full lamp hosting stack on a shared server.
with aws you would need to create an instance, configure that instance, eg install mysql if that is what you want, etc.
then you pay-per-hour while it is running, so for a full year it would be like 24*365*12c = $1050 approx. (you will need to check the exact rates for what you want)
AWS is great for automatically balancing varying loads across your server. Imagine you're running a Christmas special -- you don't need 50x servers all year (waste of money). AWS allows you to scale (automatically) from your normal 1x instance to 50x.
Based on your question, I'm going to guess you're not an expert in infrastructure. Amazon does offer a data store, which could replace MySQL. But if you're not aware of the tradeoffs, I'd advise against going this route until you become more familiar with AWS.
You should sign up for an account and play with it. You only pay per compute-hour, so you can setup and run a server for under a dollar a day. Check it out!
Price wise, I think they're a little more expensive than some of the other cloud computing options. The most frequent names I hear in that space are AWS, linode (more VPS than true "cloud"), slicehost, and (more frequently) Media Temple.
AWS is like a Virtual Private Server (VPS) with additional API. If you haven't used a VPS before, it would be good to explore into that before you use AWS.
If you do want to try AWS, it's available at an affordable per hour rate. You can try for a couple of hours to see how it's like too.
Have fun!