​

Email Agent Packaging

​

Overview

​

Goals

• A host web app can run npm install @amd-gaia/agent-email and, with one build-time fetch step, get a working local email agent — no Python install, no C++ toolchain, no compile step.
• The npm package stays lightweight (JS/TS client + fetch CLI only); the large binaries live in R2 object storage, the single source of truth.
• One REST contract backs both the Python and C++ implementations; the host integrates once and is implementation-agnostic.
• The platform binary is present on disk at build/sign time (fetched before the host signs its app), so it’s covered by the host’s code signature and runs fully offline at runtime.
• Each platform fetches only its own binary, integrity-verified against a pinned SHA-256 — fail loudly on mismatch.
• Binaries are signed/notarized so a host app that bundles them stays signable.

​

Non-goals (this milestone)

• Re-enabling the paused PyPI publish path (tracked separately under #1179).
• An in-process Node native addon (N-API). Captured as a future fast-path in Deferred, not built here.
• Third-party host glue. We ship the agent + client + contract; an external host owns its own integration. Exception: the GAIA Agent UI is our reference consumer and its migration to the sidecar is in scope as the validation phase (Phase 6).
• New triage capabilities. This is a packaging and distribution milestone, not a feature milestone.

​

Current State

​

Target Architecture

​

Sidecar model

┌─────────────────────────┐        loopback REST          ┌──────────────────────────┐
│  Host web app (Node)     │  ───────────────────────────▶ │  Email agent sidecar      │
│                          │   POST /v1/email/triage        │  (frozen Python OR C++)   │
│  @amd-gaia/agent-email   │   GET  /health  /version       │  REST server + connectors │
│  (JS/TS client)          │ ◀───────────────────────────  │                           │
│   • spawns binary        │                                └──────────────────────────┘
│   • health/version check │
│   • typed REST calls     │
└─────────────────────────┘

​

Distribution — thin npm package + build-time R2 fetch

@amd-gaia/agent-email             ← npm: JS/TS client + `fetch` CLI + SHA-256 manifest. NO binaries.
        │  build-time: npx @amd-gaia/agent-email fetch
        ▼
R2 via Agent Hub Worker           ← canonical, immutable, versioned (existing POST /publish + bucket)
  agents/email/<version>/email-win32-x64.exe
  agents/email/<version>/email-darwin-arm64
  agents/email/<version>/email-darwin-x64
  agents/email/<version>/email-linux-x64
        │  verified against pinned SHA-256, written to the host's resources dir
        ▼
host app bundle (signed)          ← binary on disk at sign time → covered by the host signature

// @amd-gaia/agent-email/package.json (the only published binary-bearing artifact is the manifest)
{
  "name": "@amd-gaia/agent-email",
  "version": "0.1.0",
  "bin": { "agent-email": "./cli.js" },     // exposes `npx @amd-gaia/agent-email fetch`
  "files": ["dist/", "binaries.lock.json"]  // SHA-256 + R2 keys per platform, no binaries
}

// resolves the *fetched* binary (written by the build-time fetch step), not an npm sub-package
function binaryPath(resourcesDir: string): string {
  const ext = process.platform === "win32" ? ".exe" : "";
  return path.join(resourcesDir, `email-${process.platform}-${process.arch}${ext}`);
}

• Binaries in npm via optionalDependencies (esbuild/swc pattern) — considered, and viable; it’s the most reproducible (lockfile-pinned) and lowest-friction for the consumer (npm install just works). Rejected as the default here because frozen-Python binaries are large (~40–100 MB each), and we want one canonical binary store (R2) shared across the npm path, the gaia agent install Hub path, and the GitHub manual share — rather than duplicating tens of MB into the npm registry per release.
• postinstall download — breaks under npm install --ignore-scripts and behind proxies; rejected. The fetch is an explicit build step, not an install hook.
• Runtime / first-init download — rejected (signing + offline reasons above).

​

Lifecycle handshake

1. Resolve the platform binary (binaryPath() above) — the binary fetched at build time from R2 and SHA-verified.
2. Spawn it on an allocated loopback port (agent --port <p> --host 127.0.0.1).
3. Health — poll GET /health until ready or timeout (fail loudly on timeout — no silent “assume ready”).
4. Version-check — GET /version returns { apiVersion, agentVersion }; the client refuses a mismatched apiVersion with an actionable error rather than making calls against an incompatible binary.
5. Ready — typed REST calls; clean shutdown on host exit.

​

One contract, two implementations

​

Workstreams

​

Phase 1 — REST contract & client SDK

• Author the OpenAPI spec for the email agent REST surface (triage, connectors, health, version). Commit it to the repo as the source of truth.
• Add GET /version ({ apiVersion, agentVersion }) and confirm GET /health semantics on the existing Python server.
• Build @amd-gaia/agent-email (TypeScript): typed REST client + lifecycle helpers (spawn, health-poll, version-check, shutdown). Full type defs, no runtime binary yet.
• Unit + contract tests: client validated against the OpenAPI spec; lifecycle helpers tested against a stub server.

​

Phase 2 — Frozen Python binary

• Freeze gaia-agent-email (PyInstaller or Nuitka) into a single self-contained executable that boots the REST server.
• Per-platform builds: win32-x64, darwin-arm64, darwin-x64, linux-x64.
• Smoke test each frozen binary in a clean environment with no Python: spawn → health → version → triage.
• Compute and record each binary’s SHA-256 (feeds the Phase 3 pinned manifest); measure per-platform size for R2 storage/transfer.

​

Phase 3 — Fully-automated R2 distribution + thin npm package

• Fully-automated, tag-triggered publish workflow (no workflow_dispatch gate): on a namespaced tag, the workflow builds every platform binary → computes SHA-256 → pushes each to R2 via the existing Worker POST /publish (Bearer PUBLISH_TOKENS from a GH Actions secret) → writes the pinned manifest → publishes the npm package. Re-running a published version is a no-op (the Worker’s native version-immutability), not an overwrite.
• Reuse the existing pipeline rather than forking: extend build_agents.yml’s release-bundle leg (it already produces per-platform binaries + checksums) to call POST /publish instead of stopping at GitHub-artifact upload.
• Confirm/extend POST /publish to accept multiple per-platform binaries under one <id>/<version> (today it stores gaia-agent.yaml + an artifact; the four OS/arch binaries must coexist in one version without tripping per-version immutability).
• Build the thin @amd-gaia/agent-email npm package: JS/TS client + fetch CLI + a binaries.lock.json manifest pinning each platform’s R2 key and SHA-256.
• fetch CLI: resolve platform → download from R2 → verify SHA-256 (fail loudly on mismatch) → write to the host-specified resources dir → chmod +x on POSIX.
• Document the host integration: run npx @amd-gaia/agent-email fetch as a build step before signing, and (for Electron-style hosts) keep the fetched binary outside the asar (extraResources) so it can be spawned.
• Atomic release: R2 upload → manifest write → npm publish on one tag; manifest SHAs must match the uploaded R2 objects, and the npm version must reference an R2 version that exists, or the whole release fails.

​

Phase 4 — Code signing & notarization

• macOS: sign + notarize the darwin binaries (unsigned/un-notarized binaries break the host app’s notarization).
• Windows: Authenticode-sign the win32 binary (unsigned trips SmartScreen on the host’s signed app).
• Open decision: who holds the signing cert — we sign and publish signed binaries, or we ship reproducible builds the partner signs under their cert. Resolve before Phase 3 publish.

​

Phase 5 — C++ implementation & binary

• Stand up the C++ email agent server under hub/agents/cpp/email/, implementing the Phase 1 OpenAPI contract (same routes, same /version apiVersion).
• Build static / dependency-bundled binaries for the four platforms via the existing C++ build infrastructure (build_cpp.yml, cpp/packaging/).
• Distribute via the same R2 + thin-package mechanism — upload to the same R2 key layout, pin SHA-256 in the manifest; the host install + fetch path is unchanged, only the binary’s provenance differs.
• Conformance test: the C++ binary passes the same contract test suite as the Python binary (identical request/response behavior).

​

Phase 6 — Agent UI as first consumer (validation)

• Add @amd-gaia/agent-email as a dependency of the Electron app; run the build-time fetch in the app’s build; spawn the sidecar via the client’s lifecycle handshake (spawn → health → version → ready).
• Re-point the UI’s email features (ChatView, EmailPreScanCard, EmailConnectCta, useConnectorsSSE triage calls) at the sidecar instead of the backend’s mounted /v1/email router.
• Connectors via shared OS keyring (no token handoff): the backend keeps owning the OAuth connect flow and keyring writes (connect CTAs unchanged); the sidecar reads the same OS keyring through its bundled gaia.connectors. This relies on the connectors module’s already-documented “two processes share the keyring” behavior — note the process-local token cache + concurrent-refresh caveat, but no new auth protocol.
• Remove the built-in path: stop pulling the gaia-agent-email wheel into the UI backend’s dependency/auto-mount chain, so the sidecar is the only email surface — proving the package fully replaces the in-process agent.
• Keep the Python backend for everything non-email; only email migrates to the sidecar in this phase.

​

CI / Release Pipeline

• One tag-triggered workflow does the whole release: build matrix → chmod +x + sign + SHA-256 → push each binary to R2 via the Worker POST /publish → write the pinned manifest → npm publish. The current manual build_agent_package.yml (workflow_dispatch → GitHub release) is demoted to a fallback mirror, not the release path.
• Reuse the existing legs: extend build_agents.yml (C++ binaries + checksums, already tag-aware) and the build-agent-wheel action to push to R2 via POST /publish rather than stopping at GitHub-artifact upload — the workflow comment “R2 … handled by other issues” is exactly this work.
• Idempotent by construction: the Worker rejects republishing an existing <id>@<version> (native version-immutability), so a re-run or an unchanged-version tag is a safe no-op — no custom overwrite logic.
• Atomic release: R2 upload → manifest write → npm publish on a single tag. Manifest SHA-256s must match the uploaded R2 objects, and the npm version must reference an R2 version that exists, or the whole release fails — no client-without-binary or stale-SHA states.
• Auth: the publish step authenticates to the Worker with a Bearer PUBLISH_TOKENS secret stored in GitHub Actions (scoped to the email author); never embedded in the package.
• Canonical store = R2, mirror = GitHub: R2 (workers/agent-hub/) is the programmatic source of truth; the agent-pkg-email-* GitHub release stays as the manual, no-login mirror.
• Namespacing: keep release tags namespaced (agent-pkg-email-*) so they never fire the paused PyPI workflows (publish.yml / publish_agents.yml).
• Distribution registries: the existing @amd-gaia npm scope (already hosts @amd-gaia/agent-ui). The thin model needs one public package, @amd-gaia/agent-email — no per-platform sub-packages.
• npm auth = trusted publishing (OIDC), no stored token. The package is configured as an npm trusted publisher; the publish job requests id-token: write and runs npm publish --access public with provenance — no NPM_TOKEN secret to store or rotate. This mirrors the repo’s PyPI trusted-publishing pattern (publish_agents.yml). The publish must run from the exact workflow file registered as the trusted publisher on npm.

​

Versioning & Contract Governance

​

apiVersion bump policy

• Package version tracks the agent build; the npm package, the R2 binary version (agents/email/<version>/…), and the pinned binaries.lock.json manifest always publish in lockstep — the manifest references the matching R2 version, never a floating one.
• The OpenAPI spec (openapi.email.json) is the single source of truth — client, Python server, and C++ server are all validated against it in CI. A change to the contract that isn’t reflected in the committed artifact fails CI (test_rest_contract.py::test_committed_openapi_artifact_is_up_to_date), and a running server that doesn’t conform to the spec fails the conformance suite (tests/test_email_openapi_conformance.py).

​

Testing Strategy

​

Risks

• Runtime download mistaken for build-time fetch — if a host wires the fetch at runtime/first-init instead of at build time, the binary lands outside its code signature (Gatekeeper/SmartScreen blocks) and breaks offline first-run. Mitigation: make fetch an explicit, documented build step; the integration guide leads with “fetch before sign”; the runtime-fetch flag is opt-in and documented as server/dev-only.
• R2 availability / integrity at build time — a build can’t fetch if R2 is down or an object is corrupt. Mitigation: immutable versioned keys, SHA-256 pinned in the package (fail loudly on mismatch), and the GitHub release as a manual fallback mirror.
• Manifest ↔ R2 drift — a published SHA that doesn’t match the uploaded object bricks every consumer. Mitigation: the atomic release gates publish on manifest SHAs matching R2 objects.
• Signing ownership unresolved — blocks the first signed release. Mitigation: resolve the cert-ownership decision (us vs. partner) before Phase 4.
• Contract drift between Python and C++ — two implementations of one spec can diverge. Mitigation: shared OpenAPI source of truth + a single conformance suite both must pass in CI.
• Electron asar packaging — binaries inside an asar can’t be spawned. Mitigation: document the asarUnpack requirement prominently in the integration guide.
• Executable-bit loss — a binary published without the exec bit fails to spawn on POSIX. Mitigation: CI chmod +x + a post-pack assertion.

​

Deferred

• In-process Node native addon (N-API) for the C++ build — lowest latency, in-memory data sharing, but couples to the host’s exact Node/Electron ABI and makes a crash fatal to the host. Revisit as a v2 fast-path once the sidecar path is proven.
• PyPI publish — remains paused under #1179; the npm path does not depend on it.
• Additional agents — the npm mechanism is designed to generalize (swap the agent id), but onboarding other agents is out of scope for this milestone.

​

Exit Criteria (Milestone)

1. npm install @amd-gaia/agent-email + npx @amd-gaia/agent-email fetch on each of the four target platforms yields a working agent with no Python/C++ prerequisites, pulling only that platform’s binary from R2 and SHA-256-verifying it.
2. The JS/TS client runs the full lifecycle handshake and a triage round-trip against the frozen Python binary.
3. Binaries are signed/notarized and verified inside a test host app that fetched at build time; the agent then runs offline.
4. The C++ binary passes the same contract suite as the Python binary and is a drop-in behind the same client.
5. The thin npm package installs under --ignore-scripts; fetch fails loudly on a SHA-256 mismatch or when offline.
6. The OpenAPI contract is published and enforced in CI across client + both servers.
7. Deployment is fully automated: pushing a version tag publishes binaries to R2 and the npm package end-to-end with no manual step, and re-running the release is a safe no-op.
8. Dogfooded: the GAIA Agent UI runs email triage through the @amd-gaia/agent-email sidecar with the gaia-agent-email Python wheel removed from its backend, picking up connector tokens from the shared OS keyring.