Autotests for agents

Verify hosted MailAgent on prod without human OTP checks and without DATABASE_URL. Contract tests inject mail via POST /v1/inboxes/:id/simulate.

Prod gate

npm ci

MAILAGENT_API_URL=https://api.webmailagent.com \
MAILAGENT_API_KEY=ma_… \
  npm run test:prod:gate   # CI after deploy

  npm run test:prod        # full — before merge / tag v*

CI runs test:prod:gate (smoke:agent → smoke:qa). Full suite adds all contract-qa* + Playwright.

Environment

VariableRequiredNotes
MAILAGENT_API_KEYyesTeam or legacy API key
MAILAGENT_API_URLnoDefault https://api.webmailagent.com
SMOKE_EXPECT_ATTACHMENTSCI onlySet to 1 in GitHub Actions

Test layers

CommandScope
npm run test:prod:gateCI smoke gate
npm run test:prodFull prod gate
npm run smoke:agentMCP, OAuth, DCR, Streamable HTTP
npm run smoke:qaInbox lifecycle on prod
npm run test:contract:all12 API contract scripts
npm run test:contract:qa:agentGET /v1/agent, /v1/me, /mcp/auth
npm run checkTypeScript (no prod key)

Agent workflow after a code change

  1. GET /v1/agent — tools and auth discovery
  2. npm run check if you changed src/
  3. Run one targeted npm run test:contract:qa:* for your area
  4. npm run test:prod before merge
  5. On failure: npm run doctor:qa or mailagent_diagnose_inbox

CI

GitHub Actions secret MAILAGENT_API_KEY is required. Without it, deploy and PR checks fail on purpose. Operator setup: docs/OPERATOR.md.

Add a new contract test

  1. Copy scripts/contract-qa.mjs
  2. Use scripts/lib/contract-api.mjs (contractSimulate)
  3. Register in scripts/test-contract-all.mjs + package.json
  4. Update docs/AUTOTESTS.md

More