AWS accounts and environments

Each environment gets its own AWS account. DNS is split across accounts by delegation. Terraform keeps state in each account. Read this before you create an account, a DNS zone, or run terraform apply.

How it works

AWS Organizations groups many AWS accounts under one management account. That account pays the bills for all of them and runs central sign-in.

Why separate accounts

Reason	What it buys
Blast radius	A wrong command in dev cannot touch prod resources. They are in a different account.
Permissions	You can give a junior full access to dev and read-only access to prod.
Cost	Each account's bill is its own line. You see what dev costs without tagging tricks.
Quotas	AWS limits (like how many Fargate CPUs you can run) are per account and per region, so dev load tests cannot starve prod.

What each environment is for

	dev	staging	prod
Purpose	Daily builds, fast and cheap	Catch problems before prod	Real users
Compute	Fargate Spot (cheap, can be stopped by AWS)	On-demand	On-demand, autoscaling
Database	Aurora Serverless, short backups	Prod-like	Writer and reader, 14-day backups, deletion protection
NAT gateways	One, shared	One	One per AZ
VPC interface endpoints	Off, to save money	On, same as prod	On
Attestation	Bypassed	Should be on	On

A separate review stack can live in the prod account: its own API and its own empty database, used only by app-store reviewers. Personal data is never copied into it.

DNS: one domain, delegated across accounts

• Route 53 is AWS's DNS service. A hosted zone holds the records for one domain or subdomain.
• Delegation means the parent zone has an NS record that says "ask these other nameservers about this subdomain". The child zone can live in any account.
• Terraform writes records into another account's zone through a second provider alias that uses that account's credentials.
• Use one ACM certificate per zone. ACM checks you own a name by looking for a DNS record. With per-zone certs, Terraform adds that record and renewals are automatic forever. A wildcard on a zone you do not control means manual work on every renewal.

How to build it

Terraform state, one bucket per account

• Each environment folder (envs/dev, envs/staging, envs/prod) has its own state in its own account. State is Terraform's record of what it created.
• Turn on bucket versioning, encryption and block public access.
• Use S3 native locking (use_lockfile = true), so two people cannot apply at once.
• Never commit .tfstate files. If state breaks, restore an older version of the object.

New zone in a new account: two stages

ACM waits until DNS can prove you own the name. That cannot happen until the parent zone delegates to the new zone. One big apply would hang.

1. Apply only the new zones (-target the zone resources).
2. Add the NS records in the parent zone or at the registrar.
3. Check delegation with dig NS sub.example.com against a public resolver.
4. Apply the rest.

Central sign-in with IAM Identity Center

IAM Identity Center lets people sign in once with the company login and pick an account and role. They get credentials that expire in hours, not access keys that live forever.

Roll it out in this order, so nobody is locked out:

Leave service users (used by automation) alone during this. Move them separately.

Every account, from day one

• CloudTrail (a log of every AWS API call) turned on.
• GuardDuty (AWS threat detection) turned on, high-severity findings sent to one team channel.
• MFA on every human user that still exists.

Rules

Rule

One environment per AWS account. Never put dev and prod in the same account. Separate accounts are the only boundary that a wrong command cannot cross.

Rule

Staging copies prod's shape, including the network path. If prod uses VPC endpoints, staging uses them too. Otherwise an endpoint bug shows up first in prod.

Rule

Read the whole plan before every apply. If the plan changes anything you did not touch, stop. It is reverting someone's live fix.

Rule

Change prod through Terraform or the deploy pipeline, never by hand in the console. The next pipeline run or apply silently undoes a manual change.

Mistakes we made

• A plain terraform apply would have broken production sign-in. People fixed things directly in AWS and never updated the Terraform code. The live setup was right and the code was stale. One full plan showed eight changes. Five were reverts nobody asked for. One removed a cross-account trust that production used to send email sign-in codes. Another dropped allowed web origins and would have broken file uploads in the dev admin console. Instead: the dev environment became -target only. Then we copied the live values back into code, one area at a time, until a plan showed only intended changes.
• A whole environment's network was built by hand. Staging's VPC and its endpoints were made outside Terraform, so Terraform could only read them, not change them. Any change there had to be done with the CLI. Instead: build every environment from the same modules from day one.
• Importing a role policy deleted permissions. We imported a live IAM policy into a Terraform module. The live policy had an extra permission the module did not have, so the module's version won and the extra permission vanished. Builds failed minutes after a clean-looking apply. Instead: diff the live policy against the module before importing, and add any missing permission in the same apply.
• One hand-made secret failed the whole apply. Terraform tried to create a config parameter that already existed (someone had created it by hand). The error stopped every other change in the same run. Instead: for anything the plan says "will be created", check whether it already exists, and import it.
• Importing a task definition deregistered the running one. The Terraform task definition is a placeholder. The real revisions come from the deploy pipeline. Importing the live one made Terraform want to replace it. Instead: never import task definitions. Also pin the revision number when you update a service by hand, or you may get the placeholder.
• A DNS migration nearly lost five records. We rebuilt a zone by guessing names and testing them with dig. It matched 22 out of 22. The registrar's zone export then showed five more live records, including share links, device enrolment and email signing keys. Instead: always start from the full zone export. DNS cannot be listed from the outside.
• The apex domain could not point at a load balancer. The registrar's DNS had no ALIAS record type, and its "forwarding" redirected only the bare domain, returning 404 for every path. Instead: move the apex zone to Route 53. The registration can stay at the registrar. Only the nameservers change.
• A dig +short check passed while checking nothing. For a delegated child, the parent's server returns the NS records in a section that +short does not print. Empty matched empty. Instead: compare full dig output, or query a public resolver.
• Admin users had no MFA, and there was no CloudTrail. The first full security scan of prod found admin IAM users without MFA, no CloudTrail trail, no GuardDuty, and tokens printed in application logs. Instead: turn these on when the account is created, and move people to Identity Center.
• A new account's Fargate limit was too low. The new prod account allowed 6 Fargate vCPUs. Full autoscaling needed about 9.5, and blue/green deploys briefly double the service. Instead: request quota increases per account and per region before launch.

Checklist

• My environment has its own account.
• I know which account holds each DNS zone I need.
• I read the full plan, and every change in it is mine.
• I did not change anything in the console without putting it in Terraform.
• I sign in through Identity Center, not with a long-lived access key.
• CloudTrail and GuardDuty are on in the account I am using.