Bigtable and DynamoDB both remove fleet management, but they do not remove data modeling: one makes ordered key ranges primary, while the other makes partition-key-local access primary.

Situation

Both services can support large operational datasets and automatic distribution, so architecture reviews often reduce the choice to cloud preference or an estimated request price. That misses the durable constraint: a Bigtable row key defines ordering and range locality; a DynamoDB primary key defines item placement and the boundary of efficient queries.

The comparison must therefore use a concrete envelope. Assume an event service with 2 KB writes, a 10:1 write-to-read ratio, burst traffic three times the sustained rate, 90-day retention, and two required reads: fetch one entity’s recent events and retrieve one event by external identifier. These are assumptions for reasoning, not vendor benchmarks.

The Problem

The entity-history query fits both products when the entity identifier distributes traffic. The external-identifier lookup creates a second access path. In Bigtable it requires another row representation or a continuous materialized view where its constraints fit. In DynamoDB it commonly requires a global secondary index. Either design adds physical writes, storage, lag semantics, and recovery work.

The decision is not “which database scales further?” It is: which key layout makes the dominant query local, and can the secondary path remain correct during skew, throttling, and recovery?

Ordered Rows Versus Partition-Key Queries

flowchart TD
    A["Application write"] --> B{"Primary access model"}
    B -->|Ordered history| C["Bigtable row key — entity and reversed time"]
    B -->|Item collection| D["DynamoDB key — entity and event time"]
    C --> E["Primary row mutation"]
    D --> F["Base table item"]
    E --> G["Derived lookup representation"]
    F --> H["Global secondary index update"]
    G --> I["Validate lag, restore, and hotspot behavior"]
    H --> I

Bigtable: Row Order Is a Performance Contract

Bigtable stores rows in lexicographic row-key order. Efficient operations use an exact row key, a prefix, or a bounded range. A key such as tenant#device#reverse_timestamp can co-locate recent history for a device. Starting with a timestamp is unsafe because sequential inserts can concentrate on one key range; Google explicitly recommends a high-cardinality prefix.

The transaction boundary is one row. Check-and-mutate and read-modify-write operations can protect an invariant contained in that row, but the design should not imply atomicity across multiple rows. Keep related atomic data together without producing oversized rows; Google recommends keeping rows below 100 MB and enforces a 256 MB row limit.

Bigtable separates compute from its distributed storage layer, and tablets split and rebalance as load and data change. This does not make key choice irrelevant. Key Visualizer and the hottest-node metrics are required proof that a prefix is distributing traffic. Autoscaling responds to CPU and storage targets, but it cannot repair a schema that directs most requests to one range.

DynamoDB: Capacity Is Consumed by Items and Indexes

DynamoDB supports a simple partition key or a composite partition-and-sort key. Queries within one partition key can use sort-key conditions, making an entity-plus-time item collection a natural history representation.

AWS documents per-physical-partition design limits of 1,000 write units and 3,000 read units per second. A write unit covers an item up to 1 KB, so the assumed 2 KB event consumes two write units before indexes or replication. Adaptive capacity can shift capacity toward unevenly active keys, but a single hot key still has a finite boundary.

A global secondary index has its own partition and sort key, projection, storage, and capacity behavior. AWS documents that insufficient provisioned GSI write capacity can throttle base-table writes. GSI reads are eventually consistent. This means the external-identifier path needs an explicit stale-read and duplicate-write policy rather than an assumption that it behaves like an immediately consistent second primary key.

Correctness and Geography

Bigtable replication creates independent cluster copies. With multi-cluster routing, replication is eventual; single-cluster routing can provide read-your-writes within that cluster but changes failover behavior. App profiles also affect whether single-row transactions are available. The topology must state which profile each workload uses.

DynamoDB strongly consistent reads are available on tables and local secondary indexes in a single Region, but not on GSIs. Global tables replicate across Regions. Multi-Region eventual consistency and multi-Region strong consistency are different modes with different topology and feature constraints; MRSC is not a checkbox that can be assumed for every existing table.

For either product, define:

  • Whether stale reads are allowed after a regional write.
  • How conflicts or duplicate retries are resolved.
  • Whether failover is automatic or an application/runbook action.
  • How derived access paths are rebuilt to the same recovery point as base data.

In Practice

The documented operational pattern is to validate distribution rather than infer it from aggregate traffic. Google’s schema workflow says to test a proposed Bigtable key and inspect Key Visualizer and monitoring for hotspots. AWS exposes throttling reason codes that distinguish table and index capacity from key-range limits.

For the assumed event service, a defensible evaluation would replay a measured key-frequency distribution—not uniform synthetic IDs—and then inject the largest observed tenant burst. Bigtable should be evaluated using hottest-node CPU, latency, and Key Visualizer. DynamoDB should be evaluated using consumed capacity, throttled requests, and throttling reasons for both the base table and GSI. The result is a workload-specific capacity envelope, not a universal throughput claim.

Choose Bigtable when ordered prefix and range reads dominate, Google Cloud is the intended failure domain, and the application can own explicit alternate representations. Choose DynamoDB when AWS-native key-value access, conditional item operations, and declared secondary indexes are the better fit. Neither conclusion follows from total request rate alone.

Verification Plan

  1. Replay P50, P95, and maximum observed item sizes.
  2. Preserve production tenant and key skew in the generator.
  3. Test sustained load, burst load, and one deliberately hot key separately.
  4. Measure the base table and every derived index independently.
  5. Exercise regional routing or failover and record observed staleness.
  6. Restore into an isolated environment and prove both access paths return the same logical dataset.
  7. Price the tested topology, including replicas, indexes, backups, network transfer, and headroom.

Where It Breaks

DesignFailure modeRequired control
Bigtable timestamp-first keySequential traffic heats one rangeHigh-cardinality prefix plus Key Visualizer validation
Bigtable arbitrary predicateFilter becomes a broad scanNew query-oriented row representation or reject the workload
DynamoDB low-cardinality partition keyOne key reaches a partition boundaryRedesign key or introduce write sharding with read fan-out
DynamoDB underprovisioned GSIIndex backpressure throttles base writesCapacity alarms, on-demand evaluation, and sparse projections
Either multi-region designApplication assumes immediate global visibilityExplicit consistency contract and failover test
Either secondary pathRestore produces mismatched base and derived dataVersioned rebuild and reconciliation procedure

What to Do Next

  • Problem: Aggregate throughput hides key skew and derived-write cost.
  • Solution: Model exact access paths, item sizes, consistency requirements, and regional topology.
  • Proof: Replay production-shaped traffic and validate the hottest range or partition under failure.
  • Action: Approve a product only with a reviewed key schema, capacity model, restore procedure, and revalidation trigger.

Sources