Skip to main content

The whole picture

One product, many countries, one AWS design. Read this page first. Every other page in this module zooms into one box of the diagram below.

How it works​

  1. The app first asks the discovery service "which region do I talk to?". It gets back one API address.
  2. Each region runs its own copy of the core API, its own regional database for personal data, and its own cache.
  3. Personal data never leaves its home region. Shared app content lives in a global database that is copied to every region as read-only.
  4. The identity registry maps a hashed email or phone to the user's home region. That is all it stores.
  5. Every web product sends users to one account app to sign in. One user id works on every app, and one core API serves them all.
What was proven

In the system these lessons come from, one region was live. The second region was designed, its network space was reserved, and the gaps were written down, but it was not built. Each page says which parts were proven and which are the plan.

Words you will see​

TermWhat it means
RegionOne AWS location, for example ap-south-1 (Mumbai) or eu-west-1 (Ireland). Data stored there stays there unless you copy it.
AZ (Availability Zone)One data centre group inside a region. We run everything across two or more.
ALB (Application Load Balancer)Takes HTTPS traffic and sends it to the right service based on the hostname.
ECS FargateRuns our Docker containers without us managing servers.
Aurora PostgresAWS's managed PostgreSQL database.
DynamoDBAWS's key-value database. Fast single-key lookups.
KMSAWS's key service. Keys never leave it.
AppConfigAWS service that stores a config file and lets apps poll it.
CloudFrontAWS's CDN. Serves static files from locations near the user.
Data residencyThe rule that a person's data is stored in a specific country or region.

How to build it​

Read the pages in this order:

  1. Data residency - decide where each table lives.
  2. Region discovery - how a client finds its region.
  3. One identity, many apps - one sign-in for every product.
  4. AWS accounts and environments - dev, staging, prod, DNS.
  5. Network and edge - VPC, load balancers, WAF, CDN.
  6. Service shape - one API made of modules that can split later.
  7. Scaling to a new region - the checklist.
  8. Mistakes - everything that cost us time, in one place.

Rules​

Rule

Choose the data tier for a table before you write the schema. Moving a table from global to regional later means copying live data between countries.

Rule

Clients never hardcode a regional API address. They always ask discovery. A hardcoded fallback signs a foreign user up in the wrong country, and that cannot be undone easily.

Mistakes we made​

See Mistakes for the full list. The three biggest:

  • The "global" database was not content-only. It also held group chat and staff accounts. See Data residency.
  • One-region testing hid multi-region bugs. Country-to-region mapping was missing, and nobody could see it with one region. See Region discovery.
  • Terraform config fell behind the live infrastructure. A plain apply would have broken production email sign-in. See AWS accounts and environments.

Checklist​

  • I can draw this diagram from memory.
  • I know which tier my new data belongs to.
  • I know my client must call discovery first.
Last updated on