Building a WORM-Compliant Communications Archive for Under $100

The problem nobody budgets for

Sooner or later, every business that sends automated email or SMS hits one of two walls. Either a regulator asks to see a tamper-proof record of everything you sent, or a recipient disputes a message and you have no proof it existed. Both scenarios end the same way: you need an immutable communications archive.

If you search for solutions, you will find names like Global Relay, Smarsh, and Proofpoint. They are serious products. They are also priced for enterprises — typically $10,000 to $50,000 per year, with annual contracts, onboarding fees, and per-seat licensing. For a startup, a solo developer, or a small platform team, that pricing is a non-starter.

We needed WORM compliance for a production platform handling email campaigns, transactional sends, and SMS delivery across multiple projects. We built the archive ourselves, ran it in production for months, then extracted and packaged it. The result: 90 tests, zero dependencies on proprietary platforms, and a one-time price of $99.

What WORM actually means

WORM stands for Write Once Read Many. In a WORM archive, once a record is written, it cannot be modified, overwritten, or deleted — not by an admin, not by root, not by anyone — until a defined retention period expires. This is the standard that financial regulators (SEC Rule 17a-4, FINRA), healthcare compliance (HIPAA), and data retention laws reference when they say "tamper-proof records."

AWS provides this through S3 Object Lock in COMPLIANCE mode. Unlike Governance mode (which IAM admins can override), COMPLIANCE mode is truly immutable. Once you set a retention period on an object, even the root AWS account cannot delete it before that date. The S3 bucket itself cannot be deleted until every object's retention has expired. This is the real thing.

Fail-closed architecture: spool first, send second

The most critical design decision in a communications archive is not where you store the data. It is when you store the data relative to when you send the message. If you archive after sending, there is a window where a crash, timeout, or bug means the message was sent but never archived. That is a compliance gap.

The WORM Archive uses a fail-closed architecture for email: the message is written to a durable local spool with fsync before it is dispatched via SES. If the spool write fails, the send is aborted. The message is never sent without being archived first.

The send flow

// 1. Build the MIME document
// 2. Write to local spool (fsync) ← FAIL-CLOSED: if this fails, send is aborted
// 3. Dispatch via SES
// 4. S3 PUT with Object Lock COMPLIANCE (fire-and-forget)
// 5. If S3 fails, spool retains — cron drains later

import { sendEmail } from 'worm-archive';

const { id, s3ArchiveKey } = await sendEmail({
  project: 'myapp',
  from:    'noreply@example.com',
  to:      'customer@example.com',
  subject: 'Invoice #1234',
  html:    '<h1>Your invoice</h1><p>Amount: $99</p>',
  kind:    'transactional',
});

console.log(id);            // SES MessageId
console.log(s3ArchiveKey);  // email/myapp/outbound/2026/05/20/2026-05-20T14:30:00Z_a1b2c3.eml

SMS uses a fail-open strategy instead. If archiving a text message fails, the SMS is still sent. The reasoning: a missed SMS (verification code, appointment reminder) has immediate user impact, while the archive failure is recoverable from the Twilio delivery record. The local spool retains a copy for the external uploader cron to drain when S3 is available again.

The spool write: fsync is not optional

The spool is a JSONL index file plus raw blob files on disk. Every write is fsync-ed — the blob is fsynced first, then the index entry. This ordering means a crash between the two writes leaves a recoverable state: an orphan blob with no index entry (ignored by the uploader) rather than an index entry pointing to a missing blob.

Spool write internals

// Write blob file first, then spool index entry.
// Both are fsynced for crash safety.
const blobFd = fs.openSync(blobPath, 'w');
try {
  fs.writeSync(blobFd, buf);
  fs.fsyncSync(blobFd);        // durable on disk before proceeding
} finally {
  fs.closeSync(blobFd);
}

const spoolFd = fs.openSync(spoolFile, 'a');
try {
  fs.writeSync(spoolFd, JSON.stringify(entry) + '\n');
  fs.fsyncSync(spoolFd);        // index entry durable after blob
} finally {
  fs.closeSync(spoolFd);
}

S3 key schema and the .meta.json sidecar

Every archived message gets a deterministic, sortable S3 key that encodes channel, project, direction, and timestamp:

// S3 key schema:
// <channel>/<project>/<direction>/<yyyy>/<mm>/<dd>/<iso8601>_<hash12>[_<sid>].<ext>

email/myapp/outbound/2026/05/20/2026-05-20T14:30:00Z_a1b2c3d4e5f6.eml
email/myapp/outbound/2026/05/20/2026-05-20T14:30:00Z_a1b2c3d4e5f6.meta.json
sms/myapp/outbound/2026/05/20/2026-05-20T14:31:05Z_f6e5d4c3b2a1_SMabc123.json

Alongside every .eml or .json message blob, a .meta.json sidecar is uploaded containing the project name, capture timestamp, archive format version, and any metadata the caller provided (campaign tag, template ID, approver). This makes the archive queryable with standard S3 tools — you can use S3 Select, Athena, or a simple aws s3 ls prefix scan to find messages by date, project, or direction without any database.

.meta.json sidecar example

{
  "project": "myapp",
  "capturedAt": "2026-05-20T14:30:00Z",
  "archiveVersion": 1,
  "campaignTag": "onboarding-may",
  "templateId": "welcome-v3",
  "approver": "human"
}

Reputation pause: protecting your send reputation

The archive module includes a reputation pause feature. If SES bounce rates or complaint rates cross configurable thresholds, the module automatically pauses outbound email sends and writes the pause state to a durable file. Sends resume only when the pause is explicitly cleared. This prevents a bad campaign from burning your entire domain reputation while you are asleep.

How this compares to enterprise alternatives

The trade-off is real: enterprise platforms offer managed dashboards, eDiscovery integration, and pre-built compliance certifications. If you need those features and have the budget, use them. But if what you need is a tamper-proof, auditable, legally defensible record of every message your platform sends and receives — running in your own AWS account with full source code and 90 tests — you do not need to spend $10K/year for it.

What you get in the package

• archive.js — Core WORM capture primitives: spool write with fsync, S3 PUT with Object Lock COMPLIANCE, key schema builder, credential pre-checks
• email.js — SES send with archive-before-dispatch, MIME rendering, reputation pause, configuration set management
• sms.js — Twilio send with archive capture (fail-open), inbound SMS capture from webhooks
• 90 tests covering spool writes, S3 PUT logic, key format validation, fail-closed behaviour, reputation pause, and SMS wiring
• TypeScript type definitions included
• Node.js 22+, zero proprietary dependencies — just the AWS SDK and optionally Twilio