Adding S3 Tables Support to LocalStack

I hit a wall. We had a Terraform module landing that provisions AWS S3 Tables — the new Iceberg-backed table storage service — but there was no way to test it locally. S3 Tables didn’t exist in any tier of LocalStack: not Community, not Base, not Ultimate. Which meant every iteration of our Terraform required a round-trip to AWS.

So I forked LocalStack and added it.

The problem

S3 Tables is new enough that there’s no Botocore model for it. That means no auto-generated API types, no existing provider skeleton to copy from. I needed to implement the control plane from scratch using AWS docs and the Terraform provider source code as references.

The data plane was the harder question. S3 Tables is really an Iceberg catalog backed by S3 — if I wanted DuckDB to actually query tables created through the LocalStack provider (not just fake the API responses), I needed a real Iceberg catalog running locally.

The spike that decided things

Before writing any code, I ran a spike to evaluate options:

• Polaris (Snowflake’s Iceberg catalog): its credential vending scheme breaks when you point it at LocalStack’s S3 endpoint. Dead end.
• Pure in-process mock: would mean reimplementing the Iceberg REST catalog. Weeks of work for dubious fidelity.
• Nessie (Project Nessie, an open-source Iceberg REST catalog): worked end-to-end with DuckDB 1.4.4+. One Docker container, straightforward REST API, compatible with LocalStack’s S3 for actual data file storage.

Nessie won. The architecture: LocalStack handles the AWS control plane (create/get/list/delete for table buckets, namespaces, and tables), Nessie handles the Iceberg catalog, and LocalStack’s S3 stores the actual data files. Same code paths work locally and in production.

What I built

Eight commits, about 2,800 lines:

1. Design doc and implementation plan
2. Data models (table bucket, namespace, table metadata)
3. Hand-written API types (no Botocore model to generate from)
4. NessieManager — singleton Docker container lifecycle
5. S3TablesProvider — 12 CRUD handlers
6. Service registration in LocalStack’s plugin system
7. Integration tests
8. Terraform compatibility fixes from end-to-end testing

The NessieManager was the trickiest piece. It spins up a Nessie container on first use (lazy init), detects whether LocalStack itself is running in Docker (sibling container networking) or on the host, configures the warehouse location to point at LocalStack’s S3, and handles reconnection when containers get reused. On state restore, it replays namespace and table creation calls to Nessie so everything survives a LocalStack restart.

The provider implements the full CRUD surface: CreateTableBucket, GetTableBucket, ListTableBuckets, DeleteTableBucket, and the same for namespaces and tables. Plus UpdateTableMetadataLocation and a handful of other ops the Terraform provider needs to not error out.

The Terraform it enables

resource "aws_s3tables_table_bucket" "main" {
  name = "session-exchange"
}

resource "aws_s3tables_namespace" "ns" {
  table_bucket_arn = aws_s3tables_table_bucket.main.arn
  namespace        = ["session_exchange"]
}

resource "aws_s3tables_table" "sessions" {
  table_bucket_arn = aws_s3tables_table_bucket.main.arn
  namespace        = aws_s3tables_namespace.ns.namespace
  name             = "sessions"
  format           = "ICEBERG"
}

This terraform apply now works against LocalStack. And DuckDB can attach to the resulting Iceberg catalog and actually read/write data through it.

Then robotocore showed up

A few weeks after I built this, robotocore launched — a purpose-built local AWS emulator that includes S3 Tables support out of the box. It’s a cleaner solution to the same problem, built from the ground up rather than grafted onto LocalStack’s architecture.

That’s fine. The point was never to maintain a LocalStack fork forever. The point was to unblock a real project — and to demonstrate that when a tool doesn’t exist yet, you can build it. The spike, design doc, implementation, and testing took a couple of days with Claude Code handling the implementation while I focused on architecture decisions and the Nessie integration strategy.

What I’d do differently

The hand-written API types were the most tedious part. Since there’s no Botocore model, Claude Code had to piece together request/response shapes from the AWS docs and the Terraform provider source. It got there, but it took more back-and-forth than anything else in the project. If I did it again, I’d have Claude Code generate the types programmatically from the Terraform provider’s schema rather than transcribing them — it’s the most complete machine-readable description of the S3 Tables API that exists.

The dual-write architecture (LocalStack for control plane metadata, Nessie for catalog metadata) adds complexity. State replay on restart works but feels fragile. A single-source-of-truth approach would be cleaner, though harder to fit into LocalStack’s existing patterns.

---

Built with Claude Code. 8 commits, ~2,800 lines.