I've been using autoscaling based on cpu usage. We used to set it up using a single zone, but to ensure instance availability we are now creating it with multi-zone enabled.
Now it seems to create much more instances than required according to cpu usage. I believe it has to do with the fact that instances are created among different zones and the total usage calculation somehow is not taking that into consideration.
From the documentation, the regional autoscaler will need at least 3 instances that will be located in 3 different zones, even though your utilisation is lower and it could be served from an instance in a single zone. This is to provide resiliency, because a region is less likely to go down than a single zone.
Related
I set up Google Cloud MySQL, I store there just one user (email, password, address) and I'm querying it quite often due to testing purposes of my website. I set up minimal zone availability, the lowest SSD storage, memory 3.75GB, 1vCPUs, automatic backups disabled but running that database from the last 6 days costing me £15... How can I decrease the costs of having MySQL database in the cloud? I'm pretty sure paying that amount is way too much. Where is my mistake?
I suggest using the Google Pricing Calculator to check the different configurations and pricing you could have for a MySQL database in Cloud SQL.
Choosing Instance type
As you've said in your question, you're currently using the lowest standard instance, which is based on CPU and memory pricing.
As you're currently using your database for testing purposes, I could suggest to configure your database with the lowest Shared-Core Machine Type which is db-f1-micro, as shown here. But note that
The db-f1-micro and db-g1-small machine types are not included in the Cloud SQL SLA. These machine types are designed to provide low-cost test and development instances only. Do not use them for production instances.
Choosing Storage type
As you have selected the lowest allowed disk space, you could lower cost changing the storage type to HDD instead of a SSD if you haven't done so, as stated in the documentation:
Choosing SSD, the default value, provides your instance with SSD storage. SSDs provide lower latency and higher data throughput. If you do not need high-performance access to your data, for example for long-term storage or rarely accessed data, you can reduce your costs by choosing HDD.
Note that Storage type could only be selected when you're creating the instance and could not be changed later, as stated in the message when creating your instance.
Choice is permanent. Storage type affects performance.
Stop instance when is not in use
Finally, you could lower costs by stopping the database instance when it is not in use as pointed in the documentation.
Stopping an instance suspends instance charges. The instance data is unaffected, and charges for storage and IP addresses continue to apply.
Using Google Pricing Calculator
The following information is presented as a calculation exercise based in the Google Pricing Calculator
The estimated fees provided by Google Cloud Pricing Calculator are for discussion purposes only and are not binding on either you or Google. Your actual fees may be higher or lower than the estimate. A more detailed and specific list of fees will be provided at time of sign up
Following the suggestions above, you could get a monthly estimate of 6.41 GBP. Based on a 24 hour per 7 days running instance.
And using a SSD, it increases to 7.01 GBP. As said before, the only way to change the storage type would be to create a new instance and load your data.
And this could lower to 2.04 GBP if you only run it for 8 hours 5 days a week running on HDD.
I'm starting a project where a CloudSQL instance would be a great fit however I've noticed they are twice the price for the same specification VM on GCP.
I've been told by several devops guys I work with that they are billed by usage only. Which would be perfect for me. However on their pricing page it states "Instance pricing for MySQL is charged for every second that the instance is running".
https://cloud.google.com/sql/pricing#2nd-gen-pricing
I also see several people around the web saying they are usage only.
Cloud SQL or VM Instance to host MySQL Database
Am I interpreting Googles pricing pages incorrectly?
Am I going to be billed for the instance being on or for its usage?
Billed by usage
All depend what you mean by USAGE. When you run a Cloud SQL instance, it's like a server (compute engine). Until you stop it, you will pay for it. It's not a pay-per-request pricing, as you can have with BigQuery.
With Cloud SQL, you will also pay the storage that you use. And the storage can grow automatically according with the usage. Be careful the storage can't be reduce!! even if you delete data in database!
Price is twice a similar Compute engine
True! A compute engine standard1-n1 is about $20 per month and a same config on Cloud SQL is about $45.
BUT, what about the price of the management of your own SQL instance?
You have to update/patch the OS
You have to update/patch the DB engine (MySQL or Postgres)
You have to manage the security/network access
You have to perform snapshots, ensure that the restoration works
You have to ensure the High Availability (people on call in case of server issue)
You have to tune the Database parameters
You have to watch to your storage and to increase it in case of needs
You have to set up manually your replicas
Is it worth twice the price? For me, yes. All depends of your skills and your opinion.
There are a lot of hidden configuration options that when modified can quickly halve your costs per option.
Practically speaking, GCP's SQL product only works by running 24/7, there is no time-based 'by usage' option, short of you manually stopping and restarting the compute engine.
There are a lot of tricks you can follow to lower costs, you can read many of them here: https://medium.com/#the-bumbling-developer/can-you-use-google-cloud-platform-gcp-cheaply-and-safely-86284e04b332
The pricing page only gives this information for the 1st gen. Does anybody know the concurrent connection limits for the 2nd gen?
Second Generation instances are configured to allow up to 4000 connections though it does not mean that you can safely run your workload at 4000 connections for a given instance size. Different workloads will have different demands so you still need to monitor/benchmark your application to choose the appropriate instance size.
e.g. You might be able to make 4000 concurrent connections to a n1-standard-1 instance but it's unlikely to perform well for many workloads
Today I have notice that my Amazon RDS instance memory monitor threshold shows me with red line. Here I have attached screen shot for the same.
So, My question is what is that Memory threshold, and why it is crossing limit? Anything wrong with my instance? What is the solution to decrease/control this hike?
The Red line you see is a threshold set by AWS if the RDS is causing that threshold many times then there might be a performance issue that you need to take a look.
MySQL try to use all available memory as needed. However, the limits are defined by RDS' server parameters which you can modify and you may not need to scale up your server.
RDS instances are created with default values for those parameters (the most relevant of them being innodb_buffer_pool_size) to optimize memory usage. In order to see which server variables are applied to your instance, connect to it and execute the "show global variables" command.
It is normal for that number to go up and down as matter of course.
If you are seeing performance issues and you have no more freeable memory, then you should be looking at causes or upgrading to a larger instance.
Those values may not be right for all workloads, but you can adjust them as needed using parameter groups. This document explains how you can use parameter groups:
http://docs.amazonwebservices.com/AmazonRDS/latest/UserGuide/USER_WorkingWithParamGroups.html
I have been trying to determine what instances I should choose, for Compute engine and Cloud SQL, for when we lunch our product.
Initially I'm working on handling max 500 users per day, with peak traffic likely to occur during the evenings. The users are expected to stay on the site with constant interactions for a lengthily period of time (10min+).
So far my guess's lead me to the following:
Compute engine:
n1-standard-2 ->
2 virtuals cpu's, 3.75GB memory
Cloud SQL:
D2 ->
1GB ram, max 250 concurrent users
Am I in the right ball park, or can I use smaller/larger instances?
I'd say to use appropriate performance testing tools to simulate the traffic that will be hitting your server and estimate the amount of the resources you will require to handle the requests.
For Compute Engine VM instance, you can go with a lighter machine type and take advantage of the GCE Autoscaler to automatically add more resources to your front-end when the traffic goes high.
I recommend watching this video.