I created an Amazon Aurora instance few weeks ago , it was doing great at the beginning and memory current value was 13,000 MB , Everyday current value of memory is reduced by a certain amount and now it is only 781 MB.
I don't know if it is a cache problem or I have something wrong with my configuration any ideas ?
I found that MySQL tries to cache data and use memory as long as it has free space , so If you want to change this, you could use different configuration other than Amazon RDS configuration.
By default : innodb_buffer_pool_size is {DBInstanceClassMemory*3/4}
I changed it to innodb_buffer_pool_size = {DBInstanceClassMemory*1/2}
and that solved my problem.
Related
I am running Aurora MySql Engine version 5.6.10a in the production environment. Automated DB snapshot size on 9th May was 120 GB and this snapshot size increased by 27 GB to 147 GB. I have checked that DB size did not increase even by 1 GB. I looked on the internet for the reason why this happened but got nothing.
Graph for snapshot size for the last two weeks as:
It's pretty consistent till 9th May and after 10th May. Does anyone have insight into this issue?
The rate of increase in DB size:
VolumeBytesUsed Graph:
Your help will be much appreciated!!
Thanks,
Mayurkumar
There is bug in aurora storage provisioning , we have also faced the same issue for one of our client.
In our case the storage grew from 17gb to 22000 gb in 15 days where db size was just 16gb. We were charged a lot for that period , you can connect to aws support for the resolution of this problem.
How were you checking you snapshot size? One possibility is that may have configured daily backups or so, which might be the reason why your total backup size is growing. Would love to help out more if you can provide more details on the number of snapshots, how you were checking the size of the cluster and the snapshot etc.
I have a database running on Amazons RDS platform and it does not seem to be using the full amount of memory available.
The instance type is db.m4.xlarge, this should give me 16 GiB of memory but when i look at the monitoring page it shows I am reaching the threshold with a current value of 2460 MB.
When I look at the db-parameter-group it shows that the innodb_buffer_pool_size should be the 3/4 of the DBinstanceClassMemory however when i check the actual value set (by logging into the db and running show global variables) is it set as 12465471488 (i assume this is bytes?)
Does anyone know why this is and what options I should set to make the RDS instance take full advantage of the memory that is available?
The number shown in the console is free memory -- not used memory. It's arguably counter-intuitive, but that's what's being shown, here. Note that the small bar graph adjacent to the number is mostly full, not mostly empty.
in ejabberd.yml we have following line :
##
## Number of connections to open to the database for each virtual host
##
## odbc_pool_size: 10
we are running mysql enabled ejabberd server. MySql server connection limit is 300.
After doing research online (on very limited documentation available) , it seems like increase odbc_pool_size from default 10 mainly affects (decreases) the connecting time of client to server. we have an average of ~1500 users online at one given time instance.
My question : what exact purpose does odbc_pool_size variable serve. How will increasing the pool size affect server connect time / latency ?
UPDATE
Ejabberd Server stats :
8 gb RAM
Dual core
~2000 users (peak hours)
average cpu utilaztion 13.5%
MySql Server stats:
max supported simultaneous connection: 300
write IOPS (QPS) 23.1/sec
read IOPS 1/sec
Memory usage : 2.5/15gb
According to you what will be a good odbc_pool_size for above configuration? (I was thinking of something around 50?)
Like any pool, its size decide of the number of request that can be processed in parallel. If your pool size in 10, only 10 requests can be process in parallel, the other are queued. It means if you have 100 users that tried to connect at the same time, the last one to be process will have to wait for 10 batches of queries to have been processed, thus increasing the latency.
Increasing the pool size can help with latency, up to a point where database cannot cope with more parallelism and global performance will decrease. Good value depends on your database sizing, your use case and your overall architecture.
You need to perform benchmarks and experiment to adapt the sizing to your own case as it really depend on actual traffic patterns.
we have a 4 node cluster, with 24 GB RAM, out of which 18GB has been given to couchbase with zero replicaion.
We have a approx 10M of records in this cluster with ~2.5M/hour and expire old items.
My RAM Usage which is ~72GB is getting full every ~12 days, and i need to restart the cluster to fix this. After restart again the RAM usage is back to ~20GB.
Can someone please help to understand the reason for it.
FYI : Auto Compaction is set to 40% fragment level and Meta Data Purge Interval is set to 1 Day, -- which we reduced to do 2 hours. But it didn't help.
Under scenarios with very high memory allocation churn Couchbase can experience memory fragmentation, which would cause the effects you are describing. This was addressed in the 4.x release by switching to jemalloc on non-Windows OSes and using tcmalloc with agressive decommit on Windows. I would suggest you download the RC version of Couchbase 4 (http://www.couchbase.com/nosql-databases/downloads#Couchbase_Server) and give it a try to see if that fixes the issue.
When I launch an Amazon MySQL database instance in RDS, I choose the amount of Allocated Storage for it.
When I create a snapshot (either manually or with the automatic backup), it says under "Storage" the same size as the size allocated for the instance, even though my database did not reach that size.
Since the pricing (or the free tier) in Amazon is dependent on the amount of storage used, I would like to know the real storage size I'm using, rather than the size allocated by the original database.
From looking at the Account Activity, and from knowing how mysqldump works, I would guess the snapshot does not really include the empty space allocated.
I was interested in the answer to this question and a google search brought me here. I was surprised to see that although there is an accepted, upvoted answer, it does not actually answer the question that was asked.
The question asked is:
How can I tell the raw size of a MySQL DB snapshots in Amazon RDS?
However, the accepted answer is actually the answer to this question:
Am I charged for the allocated size of the source database when I take an RDS snapshot from it.
As to the the original question, AFAICT, there is no API or console function to determine the storage used by an RDS snapshot. The DBSnapshot resource has allocated_storage (ruby, java), but this returns the storage maximum size requested when the database was created. This mirrors the AWS RDS console:
One might have thought this would be broken out on the AWS bill, but it provides very little details. For RDS:
The S3 part of the bill is even less helpful:
Conclusion, there is no way to tell the raw size of a MySQL DB snapshot in Amazon RDS.
RDS is being stored through EBS according to FAQ:
Amazon RDS uses EBS volumes for database and log storage.
EBS doesn't store empty blocks, according to its pricing page:
Because data is compressed before being saved to Amazon S3, and Amazon EBS does not save empty blocks, it is likely that the snapshot size will be considerably less than your volume size.
And takes space only for changed blocks after initial snapshot was made, according to details page:
If you have a device with 100 GB of data but only 5 GB has changed after your last snapshot, a subsequent snapshot consumes only 5 additional GB and you are billed only for the additional 5 GB of snapshot storage, even though both the earlier and later snapshots appear complete.
RDS backups are block level full virtual machine snapshots; no mysqldump at all is involved. Given this fact, each of your snapshots will use exactly the same ammount of storage as your production instance at the moment the backup took place.