Skip to main content

Service shape - a monolith that can split

The core API is one deployable service made of many modules. Modules only talk through a bridge. When a module needs to become its own service, you swap the bridge's transport, not the code. Read this before you add a module or call one module from another.

How it works​

A modular monolith is one app, deployed as one unit, but split inside into modules with hard walls between them. You get the simple deploys of a monolith and the clean edges of microservices.

The bridge​

NeedCallExample
Ask one other module and wait for an answerbridge.request(action, payload) handled by bridge.handle(action, fn)Product module asks user for a display name
Tell anyone who cares that something happenedbridge.publish(event, payload) received by bridge.subscribe(event, fn)user publishes "user erased", every module deletes its rows

Later: split one module out​

Inside each module​

This is hexagonal architecture: the core logic depends only on interfaces (ports). Database and AWS code (adapters) plug in from outside. Tests swap in fakes.

  • Commands (writes) use the write connection. Queries (reads) use the read connection. Never both in one repository. This split is called CQRS.
  • Each module owns its own Postgres schema. It reads user_id but never writes another module's tables.
  • No foreign-key constraints in the database. They break when you split a module out, and they cannot cross tiers anyway.

How to build it​

When to split a module into its own service​

Split only when you can name the reason:

ReasonExample from our system
Different blast radiusStaff console traffic should not be able to slow the mobile API
Different scaling shapeOne workload needs many small tasks, another needs few large ones
Different network identityStaff traffic must arrive through a private network path only
Different owner or release paceA separate team ships on its own schedule

Our first split was not a code split. We ran the same image twice: a public service for apps and an admin service for staff, behind an internal load balancer reached through a zero-trust access proxy with company sign-in. Same schema, same database. It bought isolation and a private network path without touching the code.

One database per tier, not per module​

Do not give every module its own database cluster. That multiplies cost and makes every cross-module read a network call. Give each module its own schema inside the right tier's cluster. If a module is split out later, its schema moves with it.

Migrations run on boot​

An advisory lock makes tasks that start at the same time wait in line, so two tasks never run the same migration at once.

Rules​

Rule

Modules never import each other. Every cross-module call goes through the bridge. One direct import makes the later split a rewrite instead of a transport swap.

Rule

Publish events only after the transaction commits. If you publish inside it and it rolls back, other modules act on something that never happened.

Rule

Events carry everything a subscriber needs. A subscriber never calls the publisher back for more. Subscribers handle duplicates themselves.

Rule

Every schema change is a migration in the repo. Never rename a table or add a column by hand in any environment. A hand change means a fresh database can no longer be built.

Rule

Migrations must be additive. A rollback restores old code, not the old schema. Old code must still work on the new schema.

Mistakes we made​

  • A duplicate GraphQL type name crash-looped the API for an hour. A new module declared a type with the same name as one in another module. The server merges every module's schema into one at start-up, so it refused to boot. Type checks, lint and over 4,000 unit tests were all green, because the schema test only built one module's schema on its own. New tasks restarted every few minutes until the deploy's 60-minute health timeout rolled back. Instead: add one test that builds the full merged schema from every module. Set a short deploy health timeout in dev. Watch the ECS task sets: a new task set that never becomes steady is a crash loop.
  • A fresh content database could not be built. Two tables had been renamed and some columns added by hand in every environment. Later migrations assumed the new names. Existing databases worked. A new developer, or a disaster recovery from migrations, could not reproduce the schema. Instead: CI builds both databases from empty on every change.
  • New migrations were skipped silently. The migration runner only applied files listed in a journal. Newer files ran only in a fallback path that ran on failure. On success they were skipped, and a release shipped without a table it needed. Every staff sign-in failed. Instead: the runner always applies every new file, records each one, and fails the boot loudly on a real error.
  • Two services from one image drifted apart. The admin service had its own task definition file. Settings added to the public service's file were missing from the admin one, so two staff features silently did nothing in staging and prod. The task definition people naturally checked showed the setting present. Instead: generate both task definitions from one source, or diff them in CI. When you verify a setting, check the service that actually serves that screen.
  • We wanted to deploy the admin service alone. That was the wrong fix. Both services share the databases and both run migrations on boot. Deploying one alone would migrate the schema under the other's older code. Instead: one pipeline deploys both from the same image at the same time.
  • Rollback did not restore the schema. A scheduled job's task definition was re-registered on every build, and it ran migrations when it started. So new migrations landed even when the deploy itself rolled back. Instead: keep migrations additive, and know which tasks run them.
  • A placeholder image restart-looped and cost money. New services start on a placeholder image that has no health endpoint. The load balancer never marks it healthy, so ECS replaces it forever. Instead: set desired count to 0 until the first real image is pushed.

Checklist​

  • My module does not import another module.
  • I publish events after commit, with a full payload.
  • My subscriber is safe to run twice.
  • My tables live in my module's own schema, in the right tier.
  • My migration is idempotent and additive.
  • A test builds the full merged GraphQL schema.