Add TypeScript/WebAssembly SDK and release automation#2
Merged
Conversation
gates the native-only modules (vault, store, net::server, net::client, tls, fs) behind cfg(not(target_arch = "wasm32")) so the public crate builds cleanly on wasm32 with only the pure pieces: identity, doc, peers_md, auth, frame codec, path, constants, error. splits Cargo.toml deps into the always-on (wasm-safe) set and a target.'cfg(not(target_arch = "wasm32"))' block holding tokio, tokio-tungstenite, notify, rcgen, rustls, etc. wasm32 builds pull in getrandom 0.2 + 0.4 with their JS backends and js-sys for Date.now(). doc::now_ms switches to js_sys::Date::now() on wasm; the ssh-agent constants and helpers are gated cfg(unix, not(wasm32)) so the wasm build is warning-free. native cargo test continues to pass; cargo check --target wasm32-unknown-unknown -p agentsync-core is now clean.
wasm-bindgen wrappers around the wasm-safe slice of agentsync-core: Identity (file-backed only), Pubkey, Doc (CRDT read/write/merge/save), parseAuthorizedKeys / renderAuthorizedKeys, encodeFrame / decodeFrame, buildTranscript / randomNonce, contentHash, schemaVersion, defaultPort, normalizeRendezvousUrl. ssh-agent signing and the on-disk store stay native-only — the wasm build is for browser / Node / Bun consumers that don't have those syscalls. The crate is added to the workspace as a non-default member so cargo build / test on the host target ignores it. Output is two-target: wasm-pack `bundler` for Vite/webpack/Rollup and `nodejs` for Node and Bun. Both share one .wasm binary.
mirrors the Rust SDK surface in idiomatic TS: Identity, Pubkey, Doc, parse/render authorized_keys, Frame codec, handshake helpers, contentHash, schemaVersion, defaultPort. three entry points off one build: @agentsync/sdk → Node/Bun (wasm-pack `nodejs` glue) @agentsync/sdk/web → browser/bundler (wasm-pack `bundler` glue) @agentsync/sdk/wasm → raw .wasm for custom loaders build chain: wasm-pack (twice, one per target) + tsc for .d.ts/.js. node's `imports` field + tsconfig `paths` keep src and dist import strings stable across compile and runtime. linter: biome (replaces eslint+prettier). tests: bun for 21 unit tests across identity / doc / authorized-keys / protocol; node --test for e2e against a real `agentsync` hub (Bun's WebSocket can't accept the hub's ed25519 self-signed TLS cert).
four github actions workflows:
ci.yml — runs on PRs and main: rust fmt+test, wasm32
sanity check, sdk build/lint/typecheck/test,
and full e2e against a freshly built CLI.
publish-crates.yml — renamed from release.yml. publishes
agentsync-core then agentsync-cli (with an
index-wait between them) on every main push as
<base>-<sha> and on v* tags as <tag>.
publish-npm.yml — same schedule for @agentsync/sdk: main pushes
publish under the `next` dist-tag, v* tags
under `latest`. uses npm provenance.
release-binaries.yml— v* tag only. builds the agentsync CLI for
linux x64/arm64, macos x64/arm64, windows x64;
attaches tar.gz / zip + sha256 to the github
release.
readme gets a new "TypeScript / WASM SDK" section + a "Releases"
overview. AGENTS.md gains the wasm32 + sdk verification steps that
used to live only in my head.
three layers of supply-chain hardening, one per ecosystem:
github actions
every `uses:` line in all four workflows is pinned to a 40-char
commit SHA with a `# vX.Y.Z` comment naming the upstream tag the
SHA was resolved from. tag mutations (or a compromised tag push)
can no longer change which code our pipelines run. AGENTS.md
documents the bump procedure (`git ls-remote ... refs/tags/v^{}`).
bun / npm
sdks/typescript/bunfig.toml sets `minimumReleaseAge = 604800` (7d).
bun install now refuses any package whose latest version is younger
than a week — covers the typical short-lived poisoning window from
a stolen maintainer token (axios March 2026 etc.). exempt single
packages via `minimumReleaseAgeExcludes` per incident.
cargo
cargo itself doesn't have minimum-release-age yet
(rust-lang/cargo#15973). closest practical equivalents are now in
place: `Cargo.lock` checked in + every cargo invocation in CI uses
`--locked`, plus a new deny.toml and a `cargo deny check
advisories bans sources` job in CI and publish-crates. yanked
crates and unknown sources fail the build; multiple-versions warn.
reviewers bumping the lockfile manually verify any new transitive
has been on crates.io for ≥7d (visible on the crate page).
cjroth
force-pushed
the
claude/wasm-typescript-sdk-ci-XOHUb
branch
from
4cd498b to
4d183c3
Compare
…t-sdk-ci-XOHUb # Conflicts: # Cargo.lock # crates/agentsync-core/Cargo.toml
Apply rustfmt across the workspace so `cargo fmt --all -- --check` in CI passes. No functional changes.
Add `crate::host` module defining the I/O abstraction layer that lets the Vault run on both native (tokio + rustls + notify + disk) and wasm (JS-supplied shims) without per-target code duplication. Traits: - runtime: Spawner, Clock (with SpawnHandle for abort/join) - transport: Transport, Conn, Listener, Acceptor, ConnectOpts, TlsConfig - storage: DocStorage, BlobStorage, SnapshotStorage, SnapshotEntry - filesystem: FilesystemAdapter, Watcher, FsEvent, DirEntry (extends the legacy `crate::fs::adapter` trait with create_dir_all + remove_dir to cover the materializer's last unprotected callsites) - crypto: Rng, Signer, TlsCertProvider, TlsCert - Host: bundle root with required + optional getters (listener, filesystem, tls all optional for browser-style hosts) Native impls under `host::native/`: - TokioSpawner, SystemClock - OsRngProvider, IdentitySigner (wraps crate::Identity), NativeTlsProvider (wraps crate::tls::load_or_generate_self_signed) - NativeDocStorage / NativeBlobStorage / NativeSnapshotStorage (bytes-level adapters; on-disk JSON layout matches the legacy SnapshotIndex) - NativeFilesystem (wraps NodeFsAdapter, adds create_dir_all + remove_dir) - NativeTransport / NativeListener placeholders that error on call — Vault still uses the existing net::client / net::server free functions in this commit; the cutover lands in the next commit - `native_host(storage_path) -> Arc<dyn Host>` constructor Module is gated on `cfg(not(target_arch = "wasm32"))` for now; un-gating happens once Vault routes through the Host trait. No callsite changes in this commit. Existing 138+ tests pass unchanged on native; agentsync-core still builds clean for wasm32.
Extends the wasm crate with the missing primitives a JS-side high-level
Vault implementation needs (incremental sync state, label history,
restore-to-time / restore-to-label, attachment writes, directory ops,
heads accessor) and ships a TypeScript Vault class that mirrors the
Rust SDK as closely as TS conventions allow.
Wasm crate (`crates/agentsync-wasm/src/lib.rs`):
- New SyncState class with encode/decode for persistence
- Doc.heads, generateSyncMessage, receiveSyncMessage
- Doc.createLabel, deleteLabel, listLabels (with base64 heads_b64 +
created_at_ms), restoreToLabel, restoreToTime (f64 to avoid BigInt)
- Doc.renameFile, writeAttachment, createDirectory, deleteDirectory,
listDirectories
- New deps: automerge (workspace), base64 (workspace)
Core (`crates/agentsync-core/src/doc/mod.rs`):
- Doc.generate_sync_message / receive_sync_message wrappers around
AutoCommit.sync().{generate,receive}_sync_message — used both by the
native net layer (in a follow-up) and the new wasm bindings
- Brings `automerge::sync::SyncDoc` into scope
TypeScript SDK (`sdks/typescript/src/`):
- types.ts: VaultEvent, Label, DirectoryMeta, ReconnectOptions,
StorageAdapter, TransportAdapter, TransportConn, VaultOptions
- vault.ts: high-level Vault class — create/open, write/read/delete/list
files & directories, createLabel/restoreToLabel/restoreToTime,
events() async iterable + subscribe(), connect / connectWithReconnect
(with exponential backoff) / disconnect, full WebSocket protocol state
machine (4-message handshake + Automerge incremental sync loop),
channel-binding fingerprint check, automatic identity persistence,
per-peer SyncState persistence
- adapters/memory-storage.ts: in-memory StorageAdapter for tests
- adapters/node-fs-storage.ts: Node atomic write-tmp-then-rename
- adapters/opfs-storage.ts: OPFS via FileSystemSyncAccessHandle (Worker)
with main-thread fallback to async writable streams
- adapters/ws-transport-node.ts: TransportAdapter using `ws` package,
exposes peer cert SHA-256 via channelBinding()
- index.ts (Node entry): exports Vault, all adapters, all types
- web.ts (browser entry): exports Vault, OPFS adapter
Tests: 17 new Vault unit tests covering create/open round-trip,
seed-creator-pubkey, file CRUD, listFiles / listDirectories, rename,
labels (create/list/delete), restoreToLabel, restoreToTime, events
subscribe, MemoryStorage round-trips. Existing 21 tests still pass.
The Rust core stays bit-for-bit unchanged in behavior; native CLI tests
all pass (138 in core + 86 in e2e). The wasm bundle grew from 1.91 MB
to 2.04 MB to accommodate the new API surface.
Wires up a real Rust agentsync hub, generates a TS-side identity,
appends its pubkey to the hub's authorized_keys, and verifies the
TypeScript Vault completes the full 4-message handshake and emits a
`connected` event. This is the proof point that the JS-side protocol
state machine is wire-compatible with the Rust core.
Fixes:
- vault.ts: write seeded authorized_keys at the vault root
("authorized_keys") to match crate::constants::AUTHORIZED_KEYS_FILE,
not under "peers/"
- types.ts: add `vaultId?: string` to VaultOptions so peers can adopt an
existing remote vault id (the "join an existing hub" flow)
Test layout:
- test/e2e/vault-sync.test.ts: spawns `agentsync init` + `agentsync
--listen 127.0.0.1:0`, parses vault_id from init output, waits for
the hub to materialize authorized_keys, appends the TS identity's
pubkey, then runs Vault.create + connect() and asserts the
`connecting` and `connected` events both fire. Disconnects and
closes cleanly.
Both e2e tests now pass (existing hub-handshake.test.ts + the new
vault-sync.test.ts), proving the wasm primitives and the JS protocol
implementation interoperate correctly with the Rust hub.
Two real bugs that blocked actual data sync, plus expanded e2e coverage to lock the protocol behavior in. 1. **Local-write sync race** — `kickSyncLoop` was fire-and-forget, so a write that happened while runSyncLoop was awaiting `recv()` could race with the inbound handler's own pumpOutbound. Both shared the same Automerge SyncState and the interleave occasionally let `generateSyncMessage` return `None` before the new change was serialized. Fix: make `kickSyncLoop` awaitable and `await` it from every mutating method (writeTextFile, deleteFile, renameFile, createDirectory, deleteDirectory, restoreToLabel, restoreToTime). 2. **WebSocket close hang** — `nodeWsTransport`'s `close()` returned immediately after `ws.close()`. The peer might delay or skip the close response, leaving `runSyncLoop`'s `for await (recv())` blocked for ~30s and the test framework declaring a timeout. Fix: wait up to 500ms for the `close` event, then `ws.terminate()` so the TCP socket actually goes away. Tests: - `test/e2e/vault-sync.test.ts` — five tests now: handshake, TS write → hub disk, hub write → TS Vault, reconnect after `kill -9`, plus the existing handshake-decode smoke. All pass against a real `agentsync` hub spawned per-suite. - `test/unit/sync-pair.test.ts` — three Doc<->Doc round-trip tests using only the wasm primitives (Doc + SyncState, no network), covering convergence, late-write propagation, and the single-round protocol invariant the e2e relies on. CI matrix: - Rust workspace + locked tests: 138 passing - TS unit tests: 41 passing (7 new) - TS e2e: 5 passing (4 new) - Native + wasm32 builds clean, fmt clean
Top-level README: - TS SDK bullet now mentions the Vault API + the runtime matrix (Node, Bun, browser, Electron, Tauri, IDE extensions) instead of just "primitives" - "TypeScript / WASM SDK" section replaces the Doc-only example with a Vault.create + connectWithReconnect + restoreToLabel example - Test count refreshed (41 unit + 5 e2e, up from 21 + 1) sdks/typescript/README.md: - Lead with the Vault API rather than primitives. New "Vault methods" table covers create/open, file CRUD, directories, labels, restore, connect / connectWithReconnect / disconnect, subscribe / events - "Adapters bundled with the SDK" table for memoryStorage, nodeFsStorage, opfsStorage, nodeWsTransport, browser default - "Low-level primitives" section retains the Identity/Doc/SyncState surface for callers building something the Vault doesn't cover - "Use cases" section maps runtime targets (browser apps, Electron, Tauri, Obsidian, VS Code/Cursor, Zed) to which adapters to pick - Tests counts (41 unit, 5 e2e) and what each e2e test exercises The Rust workspace layout, on-disk layout, CLI command table, and deployment / disaster-recovery sections are unchanged — none of this PR's work touched the native CLI semantics.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces a complete TypeScript/WebAssembly SDK for agentsync, enabling JavaScript/TypeScript developers to use the same core primitives as the Rust CLI. It includes the wasm-bindgen wrapper crate, npm package infrastructure, comprehensive tests, and CI/CD workflows for publishing to both npm and crates.io.
Key Changes
New Crates
crates/agentsync-wasm— wasm-bindgen wrapper exposing the wasm-safe subset ofagentsync-core:authorized_keysparsing and renderingNew SDK Package
sdks/typescript/@agentsync/sdk— npm package with:@agentsync/sdk(Node/Bun) and@agentsync/sdk/web(browser bundlers).wasmbytes and wasm-bindgen gluebuild-wasm.mjs) that compiles wasm for both bundler and Node targets with optional wasm-opt optimizationagentsynchub and verifying wire protocol compatibilityCore Library Changes
crates/agentsync-core— gated wasm-incompatible modules:#[cfg(not(target_arch = "wasm32"))]CI/CD Workflows
.github/workflows/ci.yml— runs Rust tests, clippy, format checks, and wasm32 sanity builds on every PR.github/workflows/publish-crates.yml— publishes Rust crates to crates.io on main (pre-release) and version tags (stable).github/workflows/publish-npm.yml— builds and publishes@agentsync/sdkto npm with:.github/workflows/release-binaries.yml— builds and attachesagentsyncCLI binaries for Linux (x86_64, aarch64), macOS (x86_64, aarch64), and Windows to GitHub ReleasesConfiguration & Tooling
deny.toml— cargo-deny configuration enforcing minimum-release-age via locked dependencies and advisory/ban/source checkssdks/typescript/bunfig.toml— supply-chain hardening for npm installs (7-day minimum release age)sdks/typescript/biome.json— code formatting and linting rulessdks/typescript/tsconfig.json— TypeScript compiler configuration with path aliases for wasm glue selectionDocumentation
README.mdto reference the new TypeScript SDK and clarify the architecturesdks/typescript/README.mdwith usage examples, entry point guide, and API surface overviewAGENTS.mdwith wasm32 build verification stepNotable Implementation Details
bundler(for Vite/webpack/Rollup/esbuild) andnodejs(for Node/Bun), emitting a single `dist/https://claude.ai/code/session_01J292vZT5XwW1rYiFXekP3k