The Impact of a Larger MemcachePosted: July 24, 2012
A larger memcache can sometimes enable better performance and scalability. But not always. Some of our customers have benefited from a larger cache, while others haven’t. In this post we’ll explain how to determine if you need a larger cache and how a larger cache improves performance. This post assumes you already have a basic knowledge of memcache — refer to our Memcache user guide if you don’t.
Do I need a larger cache?
Before considering increasing your memcache, you should understand how much memcache you’re using. Memcache provides a stats function (in Rails it’s Rails.cache.stats) that returns the number of bytes and the number of objects in your cache.
You shouldn’t consider increasing your cache size if you aren’t already using most of your cache. You’ll usually never see exactly 100% usage, though.  So consider increasing your cache size if your usage is consistently above 90%. Increasing your cache size with less usage won’t change performance at all.
What does a larger memcache do for me?
A larger cache will not affect latency for individual get and set operations. Every operation in memcache is always completed in constant time.
However, a larger cache enables you to keep more keys in memory, which will increase your overall cache hit rate. More database queries and rendered HTML will fit in your cache, which means that more page loads will be faster, too. An increased cache will improve your app’s average page load time. Consider the following diagram:
Memcache has a least recently used eviction algorithm, which keeps popular keys in the cache in favor of less popular keys. Your average page load time will increase as you increase memcache because more and more of your long tail will be cached.
 Memcache cannot guarantee that your cache will be 100% used for two main reasons. First. the sum of each value size, in bytes, may not equal the memcache limit. And second, your cache is spread across multiple cache servers. A cheap, less accurate, algorithm must be used to calculate cache limits in order to guarantee low latencies.