I have some queries regarding mysql memcached such as :
1 . What is the main difference between mysql cache and memcached ?mainly I want to ask what caching techniques are adopted in both of these?
2 .What is the benefit of using memcached over mysql cache and What addiional functionality does memcached provide over mysql cache??
3 .Can I use memcached in my existing java application without making any modifications to my existing JDBC code and if not then what necessary modifications do I need to make in my JDBC code.?? please explain with an example that how can i do this .
4. Is it good to use mysql memcached for frequently updating tables?
5 . I want to use memcached in my java application but I am unable to find relevant examples so please provide me some examples showing that how can I use it ??
I have searched for the answers on web but was unable to find the relevant examples. So please provide the answers with a detailed explanation and some examples.
Thanks in anticipation.
Depending on engine used, MySQL will typically outperform memcached. Reasoning and extended explanation found at the link
Probably none, if anything you can very well end up wasting RAM for no significant benefit. There is no reason why connecting to memcached bears less strain than connecting to MySQL - both should be equally fast/slow, and many languages and frameworks implement connection pooling so that concurrent requests are served over established MySQL connections, therefore the performance loss due to connecting to MySQL is circumvented.
N/A
How would you use memcached for frequently updated tables? The data must be written to the disk, how will memcached help there? MySQL's cache is faster than memcached so I don't see how memcached helps you at all in this scenario.
N/A
I didn't refer to points 3 and 5 because there are googlable examples all over the net.
Long story short, you don't need to use memcached with MySQL if you use MySQL properly (proper dedicated server, with adequate InnoDB / TokuDB configuration, using prepared statements to alleviate SQL lexing on frequent queries).
Related
Our application was using MySql version 4.0.24 for a long time. We are trying to migrate it to version 5.6.27.
But, on testing the performance on 5.6.27, even the simple selects and updates are 30-40% slower when we are doing load testing. The CPU and IO speeds are much better than the older server. The storage engine of the tables is MyIsam in both versions. There's only one connection to the database. We tried the following options:
Changing storage engine to InnoDb - this reduce the performance drastically (70% slower)
Changing the innodb log size and buffer size - didn't help much
Increasing key buffer size with MyIsam storage engine for tables. - It made no difference
We tried modifying other parameters like query cache, tmp_table_size, heap_table_size. But, none of them made any difference.
Can you please let me know if there's any other option that we can try?
Here's a copy of my.cnf:
lower-case-table-names=1
myisam-recover=FORCE
key_buffer_size=2000M
Some things you can look at are whether the two servers have the same amount of RAM or not as it may be that the old server has more RAM and so can cache more things in memory.
You can also look at how are you connecting to the MySQL server - is it over a network? Is that network speed / quality different? Is one server accessed locally and the other over a network.
You tried tuning some good parameters, but in case there are ones you're missing, you can run mysql -e 'show variables' on both servers and then use a tool like Winmerge to compare the values of the two and see what might be different that you might not have thought of.
After trying multiple options, here are the configurations that worked for us. There may be other solutions as well but this worked for us.
Option 1:
Turned off the performance schema
Added a couple of jdbc connection parameters: useConfigs=maxPerformance&maintainTimeStats=false
Option 2:
Migrate to MariaDB. From a performance perspective, this worked out really well. It was giving a 5% better performance compared to mysql for our system. But, we couldn't pursue this option due to non-technical reasons.
Thank you for your inputs.
I have a website with 100k+ daily visitors. We use MySQL 5.1.
Serving all these visitors and logging their data specially during rush hours put a lot of load on our server.
I just upgraded our server to EC2/c3.8xlarge with vCPU=32, ECU= 108 and memory = 60G
Any suggestion about how I should setup my mysql configuration to optimize our usage?
I know this is a very broad question and depend on the nature of the load, the answer might be different, but I appreciate any suggestion.
You might want to consider updating to MySQL 5.5 (or higher) since they included the performance_schema since that version. That'll help you monitor events alot better and determine were you have to improve on your configuration regarding performance.
Other than that i guess you check some memory settings in your my.cnf. I'm not really read into it but i think some variables you can check regarding performance are:
key_buffer_size,
table_cache,
sort_buffer_size,
read_buffer_size,
Upgrading your server is always good but i think you have to tune MySQL towards that aswell
I'm having trouble getting a clear understanding of what MySQL 5.6 is introducing w/r/t memcache.
As I understand it, memcache by itself is essentially a huge, shared, memory-resident hash table that is managed by a server, memcached. In particular, it knows nothing about a persistent data store, and offers no services in that regard. It simply knows about keys and values (like a Perl hash).
What I think mySQL 5.6 introduces is a NoSQL API, whereby mySQL clients can request data from the mySQL server by key, rather than by a SELECT statement. (And similarly, they can perform updates with key=value pairs). MySQL uses memcached to cache these in memory as a performance boost, but also takes care of things like writing updates back to the database before they age out of the cache, etc.
In other words, the use of memcached is an implementation detail of the mySQL 5.6 NoSQL feature, and is not something the application programmer needs to be aware of.
I'd welcome any corrections or amplification to my understanding.
Thanks,
Chap
I think it's quite simple (from the official documentation):
I disagree with your last sentence, the application programmer has to be really aware of the memcache plugin because having it onboard of the MySQL server means that he can decide (maybe he will be forced to) access data through a memcached language interface or via the SQL interface
To better understand the impact of this plugin onto an app design you should know that there are 3 configuration tables used by MySQL for a proper memcached management; understanding how the "cache_policies" works will shade some light to some of your doubts:
Table cache_policies specifies whether to use InnoDB as the data store of memcached (innodb_only), or to use the traditional memcached engine as the backstore (cache-only), or both (caching). In the last case, if memcached cannot find a key in memory, it searches for the value in an InnoDB table.
here is the link: innodb-memcached-internals
This quote above means that, depending on what you decided for a specific key-value, you will have different application scenarios :
innodb_only -> means that you can query the data via a sql interface or via a memcached interface, here is a link to some memcached language interface examples memcached-interfaces
cache-only -> means that you should query the data via the memchached interface only
caching -> means that you can use both the interfaces (note that the storage mechanism slightly changes)
Of course this latter configuration decision is strictly related to your specific needs
I don't really have a complete answer for you I'm afraid, as I too am struggling to find the detail I require before toying around with it.
That said however there is one important point which I have managed to uncover that you seem to have missed, namely that by accessing the InnoDB storage engine via the new plugin you are actually completely bypassing SQL and avoiding all the overhead that comes with it.
This of course makes it essentially a key/value store more akin to most NoSQL databases complete with all the drawbacks associated with them. i.e. no joins etc...
However on the flip side for many applications these days, this is exactly what we want. There has been only a handful of real world performance mentions that I have come across but all seem to point to this implementation significantly outperforming MongoDB and other similar NoSQL solutions (how much truth is in it I do not know) with even one (relatively in depth) comparison claiming as high as 700k qps on a commodity server (compared with around 100k on a well tuned MySQL setup), which is incredible if true.
Resource here:
http://yoshinorimatsunobu.blogspot.co.uk/search/label/handlersocket
Anyway, sorry I can't be any more help but its food for thought at least!
We are currently evaluating failover support in different databases.
We were earlier using HSQLDB but it seems that it does not have clustering/replication support.
Our requirement is simply to have two database servers, one being only for synchronous backup but if the primary server is down, then the secondary should automatically start acting as the primary server.
Has anyone evaluated MySQL, PostgreSQL or any other DB server for such a use case?
Edit: We had thought of using MySQL cluster but it now seems that it is under GPL license which we won't be able to work with. Could anyone please suggest a synchronous replication/clustering solution which can be used? We are currently using HSQL, so a solution with HSQL used in clustered mode will be ideal for us but we are open for change.
Stackoverflow resources
MySQL supports replication out of the box: see this question for MySQL: Scaling solutions for MySQL (Replication, Clustering)
PostgreSQL also support replication, see this question for that: PostgreSQL replication strategies
If your requirements are simple MySQL will work
I've used MySQL is a simple master-master failover scenario using the setup I read in High Performance MySQL. I highly recommend the book if you're keen on using MySQL.
It has worked well for me, because I just wanted a simple fail-over.
If your use case is just as simple. It will work well.
Just for completeness, the H2 database has some clustering support, but compared to the MySQL and PostgreSQL features it is very limited, it's really only failover. I would first look at HA-JDBC.
for a simple failover where servers are on the same location. you can use DRBD and Heartbeat.
In a nutshell: DRBD stores the data on 2 servers on the same time. fully transparent to the system. with heartbeat the standby checks against the main server, if its not reachable, it takes over the resource, mounts it and starts the database daemon. (works with mysql, postgres and most probably with most other daemons out there)
There is a third-party product that works with HSQLDB:
http://ha-jdbc.sourceforge.net/
Not sure this is within the desired price range of most FOSS-type people :-) but we use DB2 9.7 for exactly this purpose (actually, we mostly use DB2/z on the mainframe for it, but some customers like the DB2/LUW (Linux/UNIX/Windows) option for smaller systems).
DB2 comes with high availability (HA) features built in and you can use db2haicu, the DB2 High Availability Instance Configuration Utility (gotta love those acronym generators employed by Big Blue) to configure things relatively painlessly.
It's active/passive as you desired, although DB2 is certainly capable of active/active setups for load balancing.
The particular setups we're most familiar with at the low end (everything other than a mainframe) are actually shared disk ones, with the HA applying to only DBMS resources and not data, but you can separate the data with DB2 replication features as well.
We've had one client (at least) using Q replication, which is a very low latency replication method, close to synchronous but not quite. DB2 does actually provide real synchronous replication as well.
DeveloperWorks has an interesting article on how this all hangs together, along with the various options.
Currently have approximately 2000 simultaneouse connections. We average approximately 425 reads and writes per second. We have a read to write ration of 3:1. All of our tables are myisam. Can we expect better or worse performance when we go from mysql 4.1.22 to 5.0?
There's no way for anyone here to tell you without the schema, queries and test data.
Why not setup a dev environment on 5.0 and testing it out?
The main concern should be that the 5.0 Information Schemas, are a HUGE vulnerability and can be used to very easily gain access to the SQL server from remote locations simply by printing off the schema using injection will let an unwanted viewer, view all of the tables and capitalize off the knowledge to get passwords using the same schema for its columns.
The MySQL source tree includes a set of benchmark tests written as Perl scripts. See The MySQL Benchmark Suite for some information. You can download the source distribution for MySQL 5.0.91 at the archives.
Source distribution of MySQL 4.1 doesn't seem to be easily available anymore. You might have to check it old sources from LaunchPad unless you can find a copy of an old source distribution elsewhere on the internet.
However, the comparison that these benchmarks show is only of general interest. It may be irrelevant to how your application performs. For instance, your usage of the database may not take advantage of some performance improvements in MySQL 5.0, but it may run into some regressions in MySQL 5.0 that were necessary.
The only way to get an answer that is relevant to your application is to try the new software with a test instance of your application, using a sample of data that is a realistic model of the type and volume of data your application typically deals with. As #BenS says, no one on a site like StackOverflow can give an answer specific to your application.
You say in a comment that you're very concerned about performance, but if you don't have an instance of your application and database that you can run tests on, you aren't doing the work necessary to satisfy this concern.
I would strongly suggest moving straight to 5.1.45 with Innodb Support. Percona provides an excellent version with XtraDB that provides a number of performance related improvements. Moving off of your MyISAM tables and onto Innodb will provide a huge performance increase in almost all cases. If you are going to burn the QA/Testing time to move, do a full move now, not a half-way step.