-
Notifications
You must be signed in to change notification settings - Fork 0
feat: add staging environment with prod sync and migration CI #30
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
61577b2
feat: prod/staging/local env setup with prod-to-target sync script
claude f162cd2
feat(sync): sync auth.users with email rewriting
claude 56216bd
feat(ci): auto-push migrations to staging on develop, prod on main
claude e0d8ead
chore(ci): drop develop branch — main-only flow
claude aa706c7
feat(ci): auto-migrate staging on PRs that touch migrations
claude 7655e83
feat(ci): comment on PR when staging migration fails
claude a0b26ce
fix(ci): read PROJECT_REF from vars, not secrets
claude 9588d37
fix: address PR #30 review feedback
claude 7b0751c
fix(supabase): throw on missing env vars instead of silently hitting …
claude 3b2a50b
fix(sync): collapse auth.users dump command to a single line
claude 7733c4a
fix(sync): use bash expansion for csv path instead of psql :var
claude 797dec5
fix(sync): use session_replication_role=replica instead of --disable-…
claude 4642811
fix(sync): anonymize profiles.email to avoid collision on new sign-in
claude 7f6ce79
feat(auth): commit magic-link email template
claude ee50fb2
docs(auth): track enable_confirmations and otp_length in config.toml
claude 41511ff
docs: rewrite ENVIRONMENTS.md as a concise setup checklist
claude 5312f18
fix(ci): wire VITE_SUPABASE_* into unit-tests workflow
claude a5cbb26
fix(ci): upsert single PR comment for migration status, delete on suc…
claude d08abaf
fix(ci): post success comment instead of deleting on green migrate
claude d2cf6b7
chore: pin node version via .nvmrc + engines
claude e310329
fix: address PR review — drop --mode local and clean env file confusion
claude b5f5a83
fix: address PR review — drop dev:prod, drop email from auth dump
claude File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| # Personal env file for local development. Loaded by Vite for every mode, | ||
| # but mode-specific files (.env.staging, .env.staging.local, etc.) override it. | ||
| # | ||
| # Copy this file to `.env.local` and adjust as needed. The defaults below | ||
| # match the Supabase CLI's local stack (`supabase start`). | ||
| VITE_SUPABASE_URL=http://127.0.0.1:54321 | ||
| VITE_SUPABASE_PUBLISHABLE_KEY=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJzdXBhYmFzZS1kZW1vIiwicm9sZSI6ImFub24iLCJleHAiOjE5ODM4MTI5OTZ9.CRXP1A7WOeoJeXxjNni43kdQwgnWNReilDMblYTn_I0 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| # Staging Supabase project (used by `pnpm run dev:staging` and `pnpm run build:staging`) | ||
| # Copy this file to `.env.staging.local` and fill in the values. | ||
| VITE_SUPABASE_URL=https://your-staging-project.supabase.co | ||
| VITE_SUPABASE_PUBLISHABLE_KEY=your-staging-anon-key |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,81 @@ | ||
| name: DB Migrate | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] | ||
| paths: | ||
| - "supabase/migrations/**" | ||
| pull_request: | ||
| branches: [main] | ||
| paths: | ||
| - "supabase/migrations/**" | ||
| - ".github/workflows/db-migrate.yml" | ||
| workflow_dispatch: | ||
| inputs: | ||
| target: | ||
| description: "Which environment to migrate" | ||
| required: true | ||
| type: choice | ||
| options: [staging, prod] | ||
|
|
||
| permissions: | ||
| contents: read | ||
| pull-requests: write | ||
|
|
||
| jobs: | ||
| migrate: | ||
| name: Push migrations | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 10 | ||
| environment: ${{ ((github.event_name == 'workflow_dispatch' && github.event.inputs.target == 'prod') || github.event_name == 'push') && 'production' || 'staging' }} | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - uses: supabase/setup-cli@v1 | ||
| with: | ||
| version: 2.58.5 | ||
|
|
||
| - name: Resolve target | ||
| id: target | ||
| run: | | ||
| case "${{ github.event_name }}" in | ||
| workflow_dispatch) TARGET="${{ github.event.inputs.target }}" ;; | ||
| pull_request) TARGET="staging" ;; | ||
| push) TARGET="prod" ;; | ||
| *) echo "Unknown event"; exit 1 ;; | ||
| esac | ||
| echo "target=$TARGET" >> "$GITHUB_OUTPUT" | ||
| echo "Migrating: $TARGET" | ||
|
|
||
| - name: Push migrations | ||
| env: | ||
| SUPABASE_ACCESS_TOKEN: ${{ secrets.SUPABASE_ACCESS_TOKEN }} | ||
| PROJECT_REF: ${{ steps.target.outputs.target == 'prod' && vars.PROD_PROJECT_REF || vars.STAGING_PROJECT_REF }} | ||
| DB_PASSWORD: ${{ steps.target.outputs.target == 'prod' && secrets.PROD_DB_PASSWORD || secrets.STAGING_DB_PASSWORD }} | ||
| run: | | ||
| supabase link --project-ref "$PROJECT_REF" --password "$DB_PASSWORD" | ||
| supabase db push --password "$DB_PASSWORD" | ||
|
|
||
| - name: Update PR comment with migration status | ||
| if: always() && github.event_name == 'pull_request' | ||
| env: | ||
| GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} | ||
| OUTCOME: ${{ job.status }} | ||
| PR: ${{ github.event.pull_request.number }} | ||
| REPO: ${{ github.repository }} | ||
| RUN_URL: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }} | ||
| TARGET: ${{ steps.target.outputs.target || 'staging' }} | ||
| MARKER: "<!-- db-migrate-status -->" | ||
| run: | | ||
| if [[ "$OUTCOME" == "success" ]]; then | ||
| BODY="$MARKER"$'\n'"✅ **DB Migrate succeeded** for \`$TARGET\` — [workflow run]($RUN_URL)." | ||
| else | ||
| BODY="$MARKER"$'\n'"❌ **DB Migrate failed** for \`$TARGET\` — [workflow run]($RUN_URL)." | ||
| fi | ||
| EXISTING=$(gh api "repos/$REPO/issues/$PR/comments" --jq ".[] | select(.body | contains(\"$MARKER\")) | .id" | head -n1) | ||
| if [[ -n "$EXISTING" ]]; then | ||
| gh api -X PATCH "repos/$REPO/issues/comments/$EXISTING" -f body="$BODY" | ||
| else | ||
| gh pr comment "$PR" --body "$BODY" | ||
| fi | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| 22 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,92 @@ | ||
| # Environments | ||
|
|
||
| | Env | Project | Used by | | ||
| | --- | --- | --- | | ||
| | **local** | Supabase CLI (`supabase start`) | `pnpm run dev` (default), e2e tests | | ||
| | **staging** | a second Supabase project | `pnpm run dev:staging`, Vercel preview deploys | | ||
| | **prod** | `qssmazlqrmxiudxckxvi` | Vercel production only | | ||
|
|
||
| The frontend reads `VITE_SUPABASE_URL` / `VITE_SUPABASE_PUBLISHABLE_KEY` from a Vite env file picked by `--mode`. Vite load order (later overrides earlier): | ||
|
|
||
| ``` | ||
| .env -> .env.local -> .env.[mode] -> .env.[mode].local | ||
| ``` | ||
|
|
||
| `*.local` files are gitignored; the `*.example` files are templates. | ||
|
|
||
| ## Setting up a new Supabase project (e.g. staging) | ||
|
|
||
| 1. **Create the project** in the Supabase dashboard. | ||
| 2. **Configure auth** (Authentication → Sign In / Providers → Email): | ||
| - "Confirm email" → **off** | ||
| - "Email OTP Length" → **6** | ||
| - Site URL and additional redirect URLs → match prod | ||
| 3. **Paste the magic-link email template** (Authentication → Email Templates → Magic Link) from `supabase/templates/magic_link.html`. Re-paste on every change. | ||
| 4. **Apply migrations**: GitHub → Actions → **DB Migrate** → Run workflow → target = `staging`. | ||
| 5. **Save the database password** (Project Settings → Database). You'll need it for GitHub secrets and `scripts/.env.sync`. | ||
|
|
||
| ## GitHub Actions config | ||
|
|
||
| Settings → Secrets and variables → Actions. | ||
|
|
||
| **Variables:** | ||
|
|
||
| | Name | Value | | ||
| | --- | --- | | ||
| | `PROD_PROJECT_REF` | `qssmazlqrmxiudxckxvi` | | ||
| | `STAGING_PROJECT_REF` | the staging project's ref | | ||
|
|
||
| **Secrets:** | ||
|
|
||
| | Name | Source | | ||
| | --- | --- | | ||
| | `SUPABASE_ACCESS_TOKEN` | https://supabase.com/dashboard/account/tokens | | ||
| | `PROD_DB_PASSWORD` | Project Settings → Database | | ||
| | `STAGING_DB_PASSWORD` | Same, on the staging project | | ||
|
|
||
| Add a **required-reviewer** rule to the `production` GitHub environment (Settings → Environments → production) so prod migrations pause for approval. | ||
|
|
||
| ## Vercel config | ||
|
|
||
| Project Settings → Environment Variables. For each Supabase var, add it twice: | ||
|
|
||
| | Variable | Production scope (main) | Preview scope (everything else) | | ||
| | --- | --- | --- | | ||
| | `VITE_SUPABASE_URL` | prod URL | staging URL | | ||
| | `VITE_SUPABASE_PUBLISHABLE_KEY` | prod anon key | staging anon key | | ||
| | `VITE_PUBLIC_POSTHOG_KEY` | PostHog key | same | | ||
| | `VITE_PUBLIC_POSTHOG_HOST` | PostHog host | same | | ||
|
|
||
| ## Local prerequisites | ||
|
|
||
| - **Supabase CLI** + **Docker** (for `supabase start`) | ||
| - **Postgres client tools** for the sync script: `brew install libpq` on macOS (and add `/opt/homebrew/opt/libpq/bin` to your PATH), `apt-get install postgresql-client` on Debian. | ||
| - Copy env templates: | ||
| ```bash | ||
| cp .env.local.example .env.local # local supabase | ||
| cp .env.staging.example .env.staging.local # staging | ||
| cp scripts/.env.sync.example scripts/.env.sync # prod + staging direct DB connection strings (for sync script) | ||
| ``` | ||
|
chiptus marked this conversation as resolved.
|
||
|
|
||
| ## Day-to-day commands | ||
|
|
||
| ```bash | ||
| pnpm run dev # local supabase (requires `supabase start`) | ||
| pnpm run dev:staging # staging | ||
| pnpm run db:sync:staging # overwrite staging public schema with prod data, anonymized | ||
| pnpm run db:sync:local # same, into local supabase | ||
|
chiptus marked this conversation as resolved.
|
||
| ``` | ||
|
|
||
| The sync script syncs `auth.users` (with anonymized emails, no passwords) and `public.*` (PII scrubbed via `scripts/anonymize.sql`). Existing target users on `auth.users` are preserved via `ON CONFLICT DO NOTHING`. Skip auth sync with `SYNC_AUTH=0`. | ||
|
|
||
| If you add a public-schema column that holds free-form user input or PII, update `scripts/anonymize.sql`. | ||
|
|
||
| ## Migration flow | ||
|
|
||
| - PR → `main`: Vercel preview points at staging. If the PR touches `supabase/migrations/**`, **DB Migrate** auto-pushes to staging. `supabase db push` is idempotent so re-runs on each commit are safe. | ||
| - Merge to `main`: **DB Migrate** auto-pushes to prod (gated by the `production` environment's reviewer rule, if configured). | ||
| - Manual: Actions → **DB Migrate** → Run workflow → pick target. | ||
|
|
||
| ## Auth email template | ||
|
|
||
| `supabase/templates/magic_link.html` is the source of truth. `supabase/config.toml` wires it into local supabase automatically. For staging and prod, paste it into the dashboard manually after each change (Authentication → Email Templates → Magic Link). Automating this via the Supabase Management API is a future improvement. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| # Copy this file to scripts/.env.sync and fill in values. | ||
| # scripts/.env.sync is gitignored — never commit real connection strings. | ||
| # | ||
| # Find these in Supabase Dashboard -> Project Settings -> Database | ||
| # -> Connection string -> URI (use the "Direct connection" string, not the pooler). | ||
|
|
||
| PROD_DB_URL="postgresql://postgres:PASSWORD@db.qssmazlqrmxiudxckxvi.supabase.co:5432/postgres" | ||
| STAGING_DB_URL="postgresql://postgres:PASSWORD@db.YOUR-STAGING-REF.supabase.co:5432/postgres" | ||
|
|
||
| # Optional — defaults to the Supabase CLI's local DB. | ||
| # LOCAL_DB_URL="postgresql://postgres:postgres@127.0.0.1:54322/postgres" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,30 @@ | ||
| -- Scrubs PII from a freshly-restored prod dump (public schema only). | ||
| -- Run after restoring data into staging or local. Idempotent. | ||
| -- | ||
| -- auth.users IS synced separately by sync-from-prod.sh, with emails rewritten | ||
| -- to user-<short-id>@example.test at load time and no password set. Existing | ||
| -- target accounts are preserved via ON CONFLICT (id) DO NOTHING. Skip the auth | ||
| -- sync entirely with SYNC_AUTH=0. | ||
| -- | ||
| -- Any new public-schema column that holds free-text user input should be | ||
| -- added below. | ||
|
|
||
| BEGIN; | ||
|
|
||
| -- profiles.username may contain a real handle, and profiles.email is UNIQUE | ||
| -- and would collide if a real user signs in to staging (the new auth.users | ||
| -- row created by Supabase would conflict with the synced profile's email). | ||
| -- Both replaced with synthetic values matching the auth.users anonymization. | ||
| UPDATE public.profiles | ||
| SET username = 'user_' || substring(id::text, 1, 8), | ||
| email = 'user-' || substring(id::text, 1, 8) || '@example.test'; | ||
|
|
||
| -- artist_notes.note_content is free-form user text. Wipe it. | ||
| UPDATE public.artist_notes | ||
| SET note_content = '[redacted]'; | ||
|
|
||
| -- group_invites.invite_token is a live secret — rotate so old links are dead. | ||
| UPDATE public.group_invites | ||
| SET invite_token = encode(gen_random_bytes(16), 'hex'); | ||
|
|
||
| COMMIT; |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.