test(flatkv): add EVM migration docker integration coverage by blindchaser · Pull Request #3400 · sei-protocol/sei-chain

blindchaser · 2026-05-06T20:42:49Z

Describe your changes and provide context

Testing performed to validate your change

Note

Medium Risk
Introduces a new FlatKV import CLI and modifies FlatKV import lifecycle/abort semantics; mistakes could lead to partial imports being committed or brittle migration behavior, though changes are mostly gated to an offline tool and tests.

Overview
Adds an end-to-end workflow to migrate the evm module’s SC data from memiavl into FlatKV and validate EVM queries across the migration boundary.

This introduces a new seidb import-flatkv-from-memiavl command (evm-only, optional --force, height selection) plus a flatkv.ImportTranslator that converts exported memiavl pairs into FlatKV’s physical key/value format while buffering/merging account fragments and dropping deletes.

FlatKV’s KVImporter is extended with Err() and an Abort() path, and Close() is made idempotent and skips finalize/snapshot when any pipeline error is recorded; new unit tests cover fail-fast, backpressure, multi-flush, and abort semantics. Docker integration tests add scripts/YAML to generate an EVM fixture, run historical EVM RPC checks pre/post import, perform a cluster-wide import+restart, and smoke-verify FlatKV contains migrated EVM storage.

^{Reviewed by Cursor Bugbot for commit 268cd4b. Bugbot is set up for automated code reviews on this repo. Configure here.}

github-actions · 2026-05-06T20:43:33Z

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

Build	Format	Lint	Breaking	Updated (UTC)
`✅ passed`	`✅ passed`	`✅ passed`	`✅ passed`	May 8, 2026, 9:08 PM

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a9518ab773

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

codecov · 2026-05-06T20:47:54Z

Codecov Report

❌ Patch coverage is 57.48299% with 125 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.17%. Comparing base (0543e0e) to head (268cd4b).

Files with missing lines	Patch %	Lines
...cmd/seidb/operations/import_flatkv_from_memiavl.go	42.37%	84 Missing and 18 partials ⚠️
sei-db/state_db/sc/flatkv/import_translator.go	77.14%	8 Missing and 8 partials ⚠️
sei-db/state_db/sc/flatkv/importer.go	86.95%	4 Missing and 2 partials ⚠️
sei-db/tools/cmd/seidb/main.go	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #3400      +/-   ##
==========================================
- Coverage   59.24%   59.17%   -0.08%     
==========================================
  Files        2110     2110              
  Lines      174149   173922     -227     
==========================================
- Hits       103175   102917     -258     
- Misses      62041    62097      +56     
+ Partials     8933     8908      -25

Flag	Coverage Δ
sei-chain-pr	`56.06% <57.48%> (?)`
sei-db	`70.41% <ø> (ø)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines	Coverage Δ
sei-db/tools/cmd/seidb/main.go	`0.00% <0.00%> (ø)`
sei-db/state_db/sc/flatkv/importer.go	`91.35% <86.95%> (+3.52%)`	⬆️
sei-db/state_db/sc/flatkv/import_translator.go	`77.14% <77.14%> (ø)`
...cmd/seidb/operations/import_flatkv_from_memiavl.go	`42.37% <42.37%> (ø)`

... and 31 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Add docker-based integration tests covering the FlatKV import path from a memiavl snapshot, plus the supporting `seidb import-flatkv-from-memiavl` tool. The CI job deploys an EVM fixture, exports state from a node, runs the importer against a separate cluster, and verifies the rebuilt FlatKV store matches the original. Implementation pieces: - new translator (`flatkv/import_translator.go`) that adapts memiavl snapshot nodes to FlatKV pairs - new `seidb import-flatkv-from-memiavl` operation (+ tests) wiring the translator into the existing importer - `Importer.Err()` method on the interface so callers can probe the pipeline without forcing Close (composite + memiavl + flatkv all implement it) - `KVImporter.Close()` is now idempotent (`finishOnce`) so the tool can invoke it on both the success and error paths without panicking on a double-close of the ingest channel - docker scripts + yaml runner config + CI matrix entries for the new EVM Module (FlatKV Import) integration test

The existing tests only ever drive a few key/value pairs through the importer, leaving the concurrency pipeline (setErr / closeOnce / done / finishOnce / Err) and the >importBatchSize flush branch entirely untested. Add direct unit tests so a future regression in batching, fail-fast error propagation, or Close idempotency surfaces in CI instead of in production. flatkv/importer_test.go (new): - CloseIdempotent_HappyPath / _AfterError: Close called multiple times must not panic on a double-close of ingestCh; the cached finishErr must round-trip on subsequent calls. - ErrLifecycle: Err() reports nil pre-error, surfaces the first pipeline error before Close is invoked, and stays stable afterwards. - SetErrAtomicCAS: only the first setErr wins (firstErr CAS) and closeOnce(done) prevents a double-close panic when workers race. - AddNodeAfterDoneDoesNotBlock: once done is closed, AddNode must exit via <-imp.done instead of stalling on a saturated ingestCh. - LargeImportTriggersMultipleFlushes: 3 * importBatchSize + 100 storage pairs hit dbWorker.flush() repeatedly mid-stream and the importStats pair count round-trips. operations/import_flatkv_from_memiavl_test.go: - Add HandlesLargeDataset (~70K source pairs across 10K addresses with mixed nonce / codeHash / code / storage entries) so the end-to-end memiavl→FlatKV path exercises the importBatchSize=2048 flush boundary and the translator's cross-batch account merge buffer without requiring a docker cluster. nonzeroByte() guards against flatkv's all-zero codeHash / storage tombstone semantics that would otherwise silently drop test data. Co-authored-by: Cursor <cursoragent@cursor.com>

Add a whitebox test that gates every dbWorker flush via a new flushHookForTest package var, sends >ingestChanSize+workerChanSize+ importBatchSize pairs at a single worker, and asserts: 1. Producer is observably blocked while flushes are gated (proves AddNode is hitting the ingestCh-full backpressure arm). 2. After gate release, producer drains, Close succeeds, and every pair is persisted across multiple flushes. Previously the only coverage of true backpressure was incidental (TestImportMemiavlModulesToFlatKVHandlesLargeDataset) — a regression that broke AddNode's <-imp.done arm or the dispatcher's worker.ch select would have silently passed as long as data still landed correctly. Production cost: a single nil load per flush. Co-authored-by: Cursor <cursoragent@cursor.com>

Before, the seidb import-flatkv-from-memiavl tool's deferred KVImporter.Close ran on every return path, including the failure paths inside the import loop (ctx cancellation, exporter error, translator error, etc.). KVImporter.Close only skips FinalizeImport + WriteSnapshot when its own pipeline error is set; CLI-side errors never propagate to imp.firstErr, so Close happily committed whatever pairs had been buffered and bumped FlatKV.Version to the target height — leaving an internally-self-consistent but partial copy of the source state on disk while the command reported failure. Operationally that meant a transient failure (ctx cancel, brief exporter hiccup) silently corrupted the migration: a retry without --force would refuse, and a retry with --force would mask the issue without surfacing the prior partial commit. Fix: introduce KVImporter.Abort(reason error). Abort records the reason via setErr so Close observes a non-nil pipeline error and skips finalize / snapshot, leaving the store at its pre-import committed version. The CLI now uses a named return + defer that routes any non-nil err through Abort instead of Close, with the explicit success-path Close unchanged. Tests: * TestKVImporter_AbortSkipsFinalize: store version unchanged after Abort despite buffered pairs. * TestKVImporter_AbortNilReasonStillAborts: Abort(nil) substitutes a generic reason rather than silently no-op'ing into a finalize. * TestKVImporter_AbortAfterCloseIsNoop: a successful Close cannot be retroactively rewound by a later Abort (finishOnce contract). A CLI-level end-to-end test of the failure path was prototyped using a mid-flight ctx cancel but tripped a pre-existing race in flatkv LoadVersion's pebble-recovery / dbcache pool interaction; that's tracked separately. The failure-path wiring is mechanical and covered by the unit-level Abort tests above. Co-authored-by: Cursor <cursoragent@cursor.com>

The FlatKV EVM import test failed intermittently with 'Error: error sending request for url (http://localhost:8545/)' on the post-import flatkv_evm_test.yaml run. Root cause: import_flatkv_evm_cluster.sh's only readiness gate after restarting seid was wait_for_height, which polls SyncInfo via tendermint RPC. Tendermint typically advances a height or two before the in-process EVM HTTP server finishes binding 8545, so the downstream test (which docker-execs 'cast' against localhost:8545) could race the restart and see connection refused. Add wait_for_evm_rpc that probes each node's EVM HTTP endpoint with eth_blockNumber and gates the script's exit on an actual EVM RPC response. Falls back to dump_node_log on timeout so a real EVM startup failure (vs. a slow bind) surfaces quickly. Co-authored-by: Cursor <cursoragent@cursor.com>

The seidb import-flatkv-from-memiavl CLI is the only caller of Err(), and it always operates on a *flatkv.KVImporter. Surface that via a type assertion in importerErr instead of widening types.Importer to require Err() across every implementation. This rolls back: * Err() on types.Importer interface * Err() stub on memiavl.MultiTreeImporter (always returned nil) * Err() join on composite.SnapshotImporter (state-sync never polls it) * The TestCompositeImporterErrJoinsChildren test and the now-unused err field on trackingImporter Net: -27 lines of speculative interface surface that no production caller (state-sync, rootmulti.Restore) actually uses, with zero behavior change for the CLI — *flatkv.KVImporter still exposes Err() and the import loop's fail-fast poll still surfaces worker errors between exporter reads. Co-authored-by: Cursor <cursoragent@cursor.com>

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 268cd4b. Configure here.}

cursor · 2026-05-08T21:11:29Z

+# Go lives at /usr/local/go/bin/go in the container (see docker/localnode/Dockerfile)
+# but is not on the default PATH for non-interactive shells, so call it absolutely.
+GO_BIN=${GO_BIN:-/usr/local/go/bin/go}
+docker exec -e GOPROXY="${GOPROXY:-https://proxy.golang.org,direct}" sei-node-0 bash -c "cd /sei-protocol/sei-chain && $GO_BIN build -o build/seidb ./sei-db/tools/cmd/seidb"


seidb binary built only on node-0 but executed on all nodes

High Severity

The seidb tool is compiled via docker exec only on sei-node-0, but the import loop at lines 94–96 runs build/seidb import-flatkv-from-memiavl on every node (sei-node-0 through sei-node-3). If the containers don't share the /sei-protocol/sei-chain/build/ directory (i.e., each has its own filesystem layer), the binary won't exist on nodes 1–3 and the import will fail with a "command not found" or "no such file" error.

Additional Locations (1)

integration_test/contracts/import_flatkv_evm_cluster.sh#L93-L96

^{Reviewed by Cursor Bugbot for commit 268cd4b. Configure here.}

blindchaser force-pushed the yiren/flatkv-docker-testings branch from a6335b1 to a9518ab Compare May 6, 2026 20:43

blindchaser added the non-app-hash-breaking label May 6, 2026

blindchaser force-pushed the yiren/flatkv-docker-testings branch from a9518ab to 1acc8aa Compare May 6, 2026 20:43

chatgpt-codex-connector Bot reviewed May 6, 2026

View reviewed changes

Comment thread sei-db/tools/cmd/seidb/operations/import_flatkv_from_memiavl.go Outdated

blindchaser force-pushed the yiren/flatkv-docker-testings branch 4 times, most recently from 2060562 to 04eaf2b Compare May 7, 2026 21:42

blindchaser force-pushed the yiren/flatkv-docker-testings branch from 04eaf2b to e664f79 Compare May 8, 2026 14:58

blindchaser and others added 6 commits May 8, 2026 13:15

Merge branch 'main' into yiren/flatkv-docker-testings

e0b0388

cursor Bot reviewed May 8, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test(flatkv): add EVM migration docker integration coverage#3400

test(flatkv): add EVM migration docker integration coverage#3400
blindchaser wants to merge 7 commits intomainfrom
yiren/flatkv-docker-testings

blindchaser commented May 6, 2026 •

edited by cursor Bot

Loading

Uh oh!

github-actions Bot commented May 6, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

codecov Bot commented May 6, 2026 •

edited

Loading

Uh oh!

cursor Bot left a comment

Uh oh!

cursor Bot May 8, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

blindchaser commented May 6, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Describe your changes and provide context

Testing performed to validate your change

Uh oh!

github-actions Bot commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

codecov Bot commented May 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot May 8, 2026

Choose a reason for hiding this comment

seidb binary built only on node-0 but executed on all nodes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

blindchaser commented May 6, 2026 •

edited by cursor Bot

Loading

github-actions Bot commented May 6, 2026 •

edited

Loading

codecov Bot commented May 6, 2026 •

edited

Loading