Multipart upload

Use this for big files and for video from phones. The file goes up as numbered parts. Each part has its own presigned URL. If one part fails, only that part is sent again. S3 joins the parts when you call complete.

How it works

If anything fails, the client calls abort. The server calls AbortMultipartUpload. A lifecycle rule is the safety net for clients that just disappear.

The S3 limits

Checked against the AWS docs in October 2026.

Item	Limit
Part size	5 MiB to 5 GiB
Last part	Can be smaller than 5 MiB
Parts per upload	10,000 at most, numbered 1 to 10,000
Parts returned by one ListParts call	1,000. Page through the rest.
Biggest object	About 50 TB
When AWS suggests multipart	From about 100 MB

Two results of these rules:

• A file under 5 MiB is a single part. That is allowed (the last part has no minimum).
• For a huge file, 5 MiB parts run out at 10,000 parts (about 48.8 GiB). Pick partSize = max(5 MiB, ceil(fileSize / 10000)).

How to build it

Plan the parts on the server

The server decides the part size. The client never picks its own.

const MIN_PART = 5 * 1024 * 1024;
const MAX_PARTS = 10_000;

export function planParts(fileSize: number, partSize: number) {
  if (!Number.isInteger(fileSize) || fileSize < 1) return null;
  if (!Number.isInteger(partSize) || partSize < MIN_PART) return null;
  const partCount = Math.ceil(fileSize / partSize);
  if (partCount > MAX_PARTS) return null;
  const parts = Array.from({ length: partCount }, (_, i) => {
    const n = i + 1;
    const contentLength = n < partCount ? partSize : fileSize - partSize * (partCount - 1);
    return { partNumber: n, contentLength };
  });
  return { partSize, partCount, parts };
}

Start the upload and sign each part

import {
  AbortMultipartUploadCommand,
  CompleteMultipartUploadCommand,
  CreateMultipartUploadCommand,
  ListPartsCommand,
  S3Client,
  UploadPartCommand,
} from "@aws-sdk/client-s3";
import { getSignedUrl } from "@aws-sdk/s3-request-presigner";

// Without WHEN_REQUIRED the SDK signs a CRC32 of an EMPTY body into every
// part URL, and S3 rejects every real chunk.
const s3 = new S3Client({ requestChecksumCalculation: "WHEN_REQUIRED" });
const BUCKET = "example-uploads";

export async function initiate(key: string, contentType: string) {
  const res = await s3.send(
    new CreateMultipartUploadCommand({ Bucket: BUCKET, Key: key, ContentType: contentType }),
  );
  if (!res.UploadId) throw new Error("no_upload_id");
  return res.UploadId;
}

export async function signParts(
  key: string,
  uploadId: string,
  parts: { partNumber: number; contentLength: number }[],
) {
  return Promise.all(
    parts.map(async (p) => ({
      partNumber: p.partNumber,
      contentLength: p.contentLength,
      // Content-Length is SIGNED, so S3 refuses a body of any other size.
      url: await getSignedUrl(
        s3,
        new UploadPartCommand({
          Bucket: BUCKET,
          Key: key,
          UploadId: uploadId,
          PartNumber: p.partNumber,
          ContentLength: p.contentLength,
        }),
        { expiresIn: 900, signableHeaders: new Set(["content-length"]) },
      ),
    })),
  );
}

A POST policy cannot be used for parts. Signing the exact Content-Length of each part is the multipart way to cap the size.

Client: send parts in parallel, retry each one

const PARALLEL = 3;
const RETRIES = 3;

async function sendAll(file: File, session: UploadSession) {
  const queue = session.parts.map((p) => p.partNumber);

  const sendPart = async (n: number) => {
    let part = session.parts[n - 1];
    const start = (n - 1) * session.partSize;
    // No type on the slice, so the PUT sends no Content-Type header.
    const chunk = file.slice(start, start + part.contentLength);
    for (let attempt = 1; ; attempt++) {
      try {
        const res = await fetch(part.url, { method: "PUT", body: chunk });
        if (!res.ok) throw new Error(`part_${n}_${res.status}`);
        return;
      } catch (err) {
        if (attempt >= RETRIES) throw err;
        await new Promise((r) => setTimeout(r, 500 * 2 ** attempt));
        [part] = await api.signParts([n]); // a fresh URL in case the old one expired
      }
    }
  };

  try {
    await Promise.all(
      Array.from({ length: Math.min(PARALLEL, queue.length) }, async () => {
        for (let n = queue.shift(); n !== undefined; n = queue.shift()) await sendPart(n);
      }),
    );
    return await api.complete(session.id);
  } catch (err) {
    void api.abort(session.id).catch(() => undefined);
    throw err;
  }
}

Three in flight is a good default for phones. More parallel parts do not help on a weak network. They just compete for the same bandwidth.

Server: complete from ListParts, not from the client

export async function complete(key: string, uploadId: string, plan: { partCount: number; fileSize: number }) {
  const parts: { PartNumber: number; ETag: string; Size: number }[] = [];
  let marker: string | undefined;
  for (;;) {
    const res = await s3.send(
      new ListPartsCommand({ Bucket: BUCKET, Key: key, UploadId: uploadId, PartNumberMarker: marker }),
    );
    for (const p of res.Parts ?? []) {
      if (p.PartNumber && p.ETag) parts.push({ PartNumber: p.PartNumber, ETag: p.ETag, Size: p.Size ?? 0 });
    }
    if (!res.IsTruncated || !res.NextPartNumberMarker) break;
    marker = res.NextPartNumberMarker;
  }

  const total = parts.reduce((sum, p) => sum + p.Size, 0);
  if (parts.length !== plan.partCount || total !== plan.fileSize) throw new Error("upload_incomplete");

  parts.sort((a, b) => a.PartNumber - b.PartNumber);
  await s3.send(
    new CompleteMultipartUploadCommand({
      Bucket: BUCKET,
      Key: key,
      UploadId: uploadId,
      MultipartUpload: { Parts: parts.map(({ PartNumber, ETag }) => ({ PartNumber, ETag })) },
    }),
  );
}

export async function abort(key: string, uploadId: string) {
  try {
    await s3.send(new AbortMultipartUploadCommand({ Bucket: BUCKET, Key: key, UploadId: uploadId }));
  } catch {
    // Already completed or aborted. The lifecycle rule is the backstop.
  }
}

ETags and CORS

CompleteMultipartUpload needs the ETag of every part. There are two ways to get them.

Way	How	Needs
Client reads them	Read the ETag response header of each part PUT and send the list to complete	Bucket CORS must have ExposeHeaders: ["ETag"]. Without it the browser hides the header and the value is empty.
Server reads them	Server calls ListParts at complete	Nothing on the client. The server also gets real sizes to check.

We use the second way. The client does not depend on CORS for ETags, and the server checks the real part count and total size. Keep ETag in ExposeHeaders anyway. Other upload code may read it.

resource "aws_s3_bucket_cors_configuration" "uploads" {
  bucket = aws_s3_bucket.uploads.id

  cors_rule {
    allowed_methods = ["PUT", "POST"]
    allowed_origins = ["https://app.example.com"]
    allowed_headers = ["*"]
    expose_headers  = ["ETag"]
    max_age_seconds = 3600
  }
}

Resume

A phone app can be killed mid-upload. To resume:

1. Keep key, uploadId and partSize on the database row. Never only on the device.
2. When the client comes back, it asks the API for progress. The API calls ListParts and returns the part numbers already stored.
3. The client signs and sends only the missing parts, using the same partSize. A different part size would cut the file in different places.
4. Then complete as usual.

A part URL can expire while the client waits. Signing a new URL for the same part number is safe. The new PUT replaces any earlier copy of that part.

Cleanup: the lifecycle rule

Parts of an upload that was never completed or aborted are billable storage that ListObjects does not show. There is no object yet, so an expiry rule cannot reach them. Only AbortIncompleteMultipartUpload does.

resource "aws_s3_bucket_lifecycle_configuration" "uploads" {
  bucket = aws_s3_bucket.uploads.id

  rule {
    id     = "abort-incomplete-multipart"
    status = "Enabled"
    filter {}

    abort_incomplete_multipart_upload {
      days_after_initiation = 1
    }
  }
}

Also run a small job that marks rows stuck in initiated as aborted and calls AbortMultipartUpload for them. Then the database and the bucket agree.

tip

Put the bucket's lifecycle rules in one lifecycle configuration. S3 keeps only one per bucket, so two Terraform resources for the same bucket overwrite each other.

Rules

Rule

The server plans the parts and signs each part's Content-Length. The client never chooses a part size. The server must know exactly how many bytes to expect.

Rule

Complete from ListParts, not from a list the client sends. Check that the part count and total size match the plan.

Rule

Every bucket that takes multipart uploads has an AbortIncompleteMultipartUpload lifecycle rule. Without it, abandoned parts cost money forever and nobody can see them.

Rule

Build the S3 client with requestChecksumCalculation: "WHEN_REQUIRED". Do not "simplify" that line away in a refactor.

Mistakes we made

• S3 rejected every chunk. The SDK default added a CRC32 checksum of an empty body to each part URL. The fix was one line on the S3 client. A comment next to it now says why it must stay.
• Lifecycle rules lived only in the live bucket. Someone added them by hand. The Terraform config did not have them, so the next full apply would have deleted them. Put every bucket rule in code.

Checklist

• Part size comes from server config, at least 5 MiB.
• Part count is at most 10,000.
• Each part URL signs content-length.
• The S3 client uses requestChecksumCalculation: "WHEN_REQUIRED".
• Complete reads ListParts (with paging) and checks count and total size.
• Any failure calls abort.
• A lifecycle rule aborts incomplete uploads after a short time.
• A job reaps rows stuck in initiated.
• Bucket CORS allows PUT from the web origins, with ETag exposed.
• Resume uses the stored uploadId and the same part size.