> For the complete documentation index, see [llms.txt](https://blog.sunilgudivada.dev/notebook/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://blog.sunilgudivada.dev/notebook/paper/research-papers/scaling-memcache-at-facebook.md).

# Scaling Memcache at Facebook

* **Enhancements to Memcached**:

  * **Lease Mechanism**: To mitigate the "thundering herd" problem, a lease mechanism is used. When a cache miss occurs, a lease token is assigned to one server, preventing multiple servers from simultaneously querying the backend database.
  * **Adaptive Slab Allocator**: This helps in better memory management by reducing fragmentation and optimizing memory allocation for different sizes of data​.

* **Handling Consistency and Stale Data**:

  * **McSqueal and Mcrouter**: These tools are used to ensure data consistency across different regions by broadcasting invalidations whenever the data is updated in the database​.
  * **Replication and Invalidation**: SQL statements modifying data include the required cache keys, ensuring that updates are propagated to all relevant caches​.

* **Scaling Strategies**:

  * **Regional Clusters**: Splitting servers into regional clusters reduces latency and improves reliability by keeping data closer to users and mitigating the effects of large-scale outages​.
  * **Different Pools for Different Data Types**: Assigning different pools to different types of data based on access patterns and memory requirements prevents negative interactions and optimizes performance​.

* **Load Balancing and Failure Handling**:

  * **Gutter Servers**: These are used as fallback when primary memcached servers fail, preventing cascading failures that can overload the system​​.
  * **Distributed Read Requests**: Read requests are distributed across multiple replicas to balance the load and improve performance​.

* **Performance Optimization**:
  * **Eviction Policies**: Utilizing Least Recently Used (LRU) eviction policies within each slab class ensures efficient memory use by discarding the least used items first​.
  * **High Availability**: The design incorporates mechanisms to handle server failures gracefully, ensuring high availability and reliability of the caching system​​.

Ref: <https://www.usenix.org/conference/nsdi13/technical-sessions/presentation/nishtala>


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://blog.sunilgudivada.dev/notebook/paper/research-papers/scaling-memcache-at-facebook.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.