perf(build): replace specs glob with fs reads#2959
perf(build): replace specs glob with fs reads#2959atharvadeosthale wants to merge 4 commits intomainfrom
Conversation
The 11 import.meta.glob calls in specs.ts expanded into 58k lazy import statements, producing 49,123 server chunks (500 MB) and pushing peak build RSS to 7 GB. The .md and .json files were being treated as JS modules with full transform/sourcemap/chunk overhead, when they are just static text. Resolve @appwrite.io/specs at runtime via createRequire and load examples + OpenAPI specs with fs.readFile. Move the package from devDependencies to dependencies so it survives bun install --production and ships to the final image. Build wall: 192s -> 113s. Peak RSS: 7.0 GB -> 5.6 GB. Server chunks: 49,123 -> 3,194. Final image net ~125 MB smaller (build/server drops 421 MB, node_modules adds 296 MB). Drops vite-plugin-dynamic-import which is no longer needed.
Appwrite WebsiteProject ID: Website (appwrite/website)Project ID: Tip Trigger functions via HTTP, SDKs, events, webhooks, or scheduled cron jobs |
Greptile Summary
Confidence Score: 4/5Safe to merge after fixing the cache key; all other changes are well-structured and correct. One P1 defect: the apiCache key uses the full platform string rather than the derived mode, causing the same large JSON to be parsed and stored once per platform variant instead of once per mode. The rest of the PR — glob removal, fs reads, path validation, parallel example loading, and build-time copy — is correct and a clear improvement. src/routes/docs/references/[version]/[platform]/[service]/specs.ts — specifically the apiCache key on line 414. Important Files Changed
Reviews (4): Last reviewed commit: "move specs out of node_modules so doesnt..." | Re-trigger Greptile |
…RL params - Cache parsed OpenAPI documents in a module-scope Map keyed by version|platform. Restores the implicit caching the old import.meta.glob path got from Node's ESM module cache, avoids re-parsing multi-MB JSON on every request. - Replace the serial loadExample await loop with Promise.all over the prepared method list. Cuts I/O latency on services with many endpoints. - Validate version and platform against allowlists from references.ts before any path.join. URL segments shouldn't reach a filesystem read unguarded; a lone .. segment could escape examples/ otherwise.
getApi only interpolates a derived literal mode (server/client/console)
into the JSON path, never platform itself, so the platform allowlist
there was rejecting valid internal callers (model-markdown.ts and
models/[model]/+page.server.ts hardcode 'console-web' which isn't in
the public Platform enum).
Move the guard to getService where the raw platform value is actually
interpolated into examples/${version}/${platform}/... — that's the
real filesystem boundary. URL paths reaching getService still go
through the [platform] route segment whose values come from the
Platform enum, so the check stays correct there.
Fixes 69 failing redirect tests for /docs/references/cloud/models/*.
| const cacheKey = `${version}|${platform}`; | ||
| const cached = apiCache.get(cacheKey); | ||
| if (cached) { | ||
| return cached; | ||
| } | ||
|
|
||
| const isClient = platform.startsWith('client-'); | ||
| const mode = platform.startsWith('server-') ? 'server' : isClient ? 'client' : 'console'; | ||
| const filename = `open-api3-${version}-${mode}.json`; |
There was a problem hiding this comment.
The
apiCache key includes the full platform string (e.g. server-nodejs, server-python), but the JSON file actually read is determined by mode — one of three fixed literals (server, client, console). Every server-side platform for the same version maps to the same open-api3-${version}-server.json file, so each distinct platform string creates a separate cache entry that parses and stores an identical multi-MB document. On a live SSR server where different users request server-nodejs, server-python, server-php, etc. concurrently, the same JSON is parsed and allocated once per platform variant — partially undoing the memory benefit of the cache. The key should be ${version}|${mode} so all server platforms share one entry.
| const cacheKey = `${version}|${platform}`; | |
| const cached = apiCache.get(cacheKey); | |
| if (cached) { | |
| return cached; | |
| } | |
| const isClient = platform.startsWith('client-'); | |
| const mode = platform.startsWith('server-') ? 'server' : isClient ? 'client' : 'console'; | |
| const filename = `open-api3-${version}-${mode}.json`; | |
| const isClient = platform.startsWith('client-'); | |
| const mode = platform.startsWith('server-') ? 'server' : isClient ? 'client' : 'console'; | |
| const cacheKey = `${version}|${mode}`; | |
| const cached = apiCache.get(cacheKey); | |
| if (cached) { | |
| return cached; | |
| } | |
| const filename = `open-api3-${version}-${mode}.json`; |


Summary
The 11
import.meta.globcalls inspecs.ts(one per Appwrite version) expanded into ~58k lazyimport()statements, producing 49,123 server chunks (500 MB) and pushing peak build RSS to 7 GB. The matched.mdand.jsonfiles are static text — there's no reason to route them through Vite's module graph.This PR resolves
@appwrite.io/specsat runtime viacreateRequireand reads examples + OpenAPI specs withfs.readFile. The package moves fromdevDependenciestodependenciesso it survivesbun install --productionand ships to the final image.vite-plugin-dynamic-importis removed (no longer needed).Build impact
build/server/sizeTest plan
/docs/references/cloud/server-nodejs/databasesrenders code examples (Node.js)/docs/references/1.5.x/server-python/storagerenders (Python, older version)/docs/references/1.7.x/client-android-kotlin/accountrenders (exercises the Android special path inloadExample)