Cassandra ships with three distinct caches, and the one most operators reach for first — row cache — is the one that’s off by default for a reason. Turning it on for the wrong workload doesn’t just fail to help; it adds CPU and memory overhead while quietly making writes slower, because every write to a cached partition evicts the entire cached row.

Situation

A read-heavy Cassandra table is serving elevated read latency despite adequate CPU, memory, and disk headroom. nodetool tablestats shows a low SSTable count per read, so it isn’t the read-amplification problem that a bad compaction strategy would cause. The obvious instinct is to enable row caching — after all, an in-memory cache should make reads faster. The team enables row cache on the hot table and read latency gets worse under write load, not better.

The Problem

Cassandra’s three caches solve different problems, and Cassandra’s own row cache implementation has a specific, non-obvious cost: it caches the entire deserialized row, not just a lookup index. Any write to that partition invalidates the whole cached row, forcing a full re-read from disk on the next access. For a table that’s read-heavy and write-heavy on the same rows — which describes most production Cassandra tables — row cache thrashes: it gets populated, immediately invalidated by the next write, and repopulated, burning CPU and heap on cache churn that never pays off in hit rate.

The core question isn’t “should I cache reads” — Cassandra already does that at two other layers by default. The question is which of the three caches actually matches this workload’s access pattern.

The Three-Cache Architecture

Cassandra’s read path touches up to three independent caches before it ever reads compressed data off disk, each with a different job and a different default.

flowchart TD
    Read[Read Request] --> KeyCache{Key Cache Hit?}
    KeyCache -->|Yes — skip partition index scan| ChunkCache{Chunk Cache Hit?}
    KeyCache -->|No| PartitionIndex[Scan Partition Index on Disk]
    PartitionIndex --> ChunkCache
    ChunkCache -->|Yes — decompressed chunk in memory| Serve[Serve Row]
    ChunkCache -->|No| DiskRead[Read and Decompress SSTable Chunk]
    DiskRead --> Serve
    RowCache{Row Cache Enabled and Hit?} -.->|Bypasses everything above| Serve

1. Key Cache — On by Default, Rarely Worth Touching

The key cache stores the on-disk location of a partition key within its SSTable, so a read can skip the partition index scan entirely. It defaults to auto — min(5% of heap, 100MB) — and is on-heap. This is close to free: it’s small, it has no write-invalidation problem (a key’s disk location doesn’t change on every write to that partition), and disabling it almost never helps. Leave it on; there’s rarely a reason to tune it beyond confirming it isn’t disabled.

2. Chunk Cache — The Lever DataStax Actually Recommends

The chunk cache (file_cache_size_in_mb in cassandra.yaml) caches decompressed SSTable data chunks in off-heap memory, defaulting to the smaller of 512MB or 1/4 of heap. Because it caches at the compression-chunk level rather than the whole-row level, a write to one row in a chunk doesn’t invalidate the other rows sharing that chunk the way row cache invalidation does. This is the cache DataStax’s own documentation points operators toward when row cache looks tempting: it gives most of the read-latency benefit without the write-invalidation cliff.

3. Row Cache — Off by Default, Narrow Use Case

Row cache stores the fully deserialized row, off-heap. It is disabled by default (row_cache_size_in_mb: 0) because the win only shows up for a specific shape of workload: partitions that are read far more often than they’re written, and ideally static or slowly-changing. A reference/lookup table that’s read thousands of times per write is a reasonable candidate. A table receiving continuous writes to the same partitions it’s also serving hot reads from — the shape most operational tables actually have — is exactly the case where row cache makes things worse.

In Practice

Check current cache effectiveness before changing anything:

# Per-table cache hit rates
nodetool info | grep -A 4 "Key Cache"
nodetool tablestats <keyspace>.<table> | grep -i cache

If chunk cache hit rate is low on a read-latency-sensitive table, the fix is usually more chunk cache headroom, not row cache:

# cassandra.yaml — raise chunk cache ceiling on a node with heap to spare
file_cache_size_in_mb: 1024

To actually validate whether a specific table is a legitimate row-cache candidate, check its write-to-read ratio and partition volatility before enabling it — not after:

-- Enable row cache only after confirming low write volume to this table
ALTER TABLE reference.currency_codes
WITH caching = {'keys': 'ALL', 'rows_per_partition': 'ALL'};

rows_per_partition can also be capped (e.g., '100') instead of 'ALL' to bound memory use per partition when only the most recent rows in a wide partition are actually hot.

Where It Breaks

Design ChoiceTradeoffMitigation
Row cache on a write-heavy tableEvery write invalidates the cached row, forcing a full re-read on next access — net negative on both CPU and latency.Reserve row cache for genuinely read-dominant, low-write reference tables. Check write volume before enabling.
Oversized chunk cache on a memory-constrained nodeChunk cache is off-heap; raising it without headroom pressures the OS page cache and can trigger swapping.Size chunk cache against actual free system RAM, not just “more is better.”
Disabling key cache to “save memory”Every read now pays a full partition-index scan even when the win from disabling was under 100MB of heap.Leave key cache on auto unless heap pressure is severe and every other lever has already been tried.

What to Do Next

  • Problem: Enabling row cache on the wrong workload adds overhead without improving read latency, because writes to hot partitions invalidate the cache faster than it can pay off.
  • Solution: Default to key cache (already on) and chunk cache (tune the ceiling) for general read-latency work; reserve row cache for a narrow set of read-dominant, low-write tables.
  • Proof: Cassandra’s own defaults reflect this — row cache ships disabled, chunk cache ships on, and DataStax’s documentation explicitly recommends chunk cache tuning over row cache for most workloads.
  • Action: Before enabling row cache on any table, run nodetool tablestats to check its write volume. If it’s taking meaningful write traffic, tune chunk cache instead.