Skip to content

Feat btc canister#1295

Open
bysomeone wants to merge 92 commits into33cn:masterfrom
bysomeone:feat-btc-canister
Open

Feat btc canister#1295
bysomeone wants to merge 92 commits into33cn:masterfrom
bysomeone:feat-btc-canister

Conversation

@bysomeone
Copy link
Copy Markdown
Collaborator

@bysomeone bysomeone commented May 6, 2026

Summary

This PR introduces RGBX and Lightclient modules for BTC cross-chain workflows, fixes key EVM runtime/query issues, and updates CI/build scripts for better portability and reliability.

Why

Main Changes

  • Added new dApps:
    • plugin/dapp/rgbx/*
    • plugin/dapp/lightclient/*
  • EVM fixes:
    • gas multiple config handling
    • log address behavior + related tests
  • Build/CI updates:
    • build/autotest/run.sh portability improvements
    • workflow/config updates for RGBX/Lightclient paths
  • Dependency updates:
    • go.mod / go.sum (chain33 and related modules)

Compatibility / Risk

  • Includes config/fork related changes; deployment environments should verify effective fork heights and module enablement.

Notes

  • This PR is large and may be easier to review commit-by-commit by area:
    1. RGBX
    2. Lightclient
    3. EVM fixes
    4. CI/build/refactor

- format asset symbol to upper case
-  handle change amount when transfer asset in utxo mode
* support setting rgbx asset precision
* use string type for utxo hash value
- Removed the `isCrossChain` flag from the `TransferAsset` message and related functions.
- Updated transfer logic to handle cross-chain assets based on symbol prefixes instead.
- Adjusted related tests and error handling to reflect the new asset management approach.
- Enhanced precision handling for asset amounts during transfers.
…g in withdraw processing

- Changed the command flag for BTC header height from `-h` to `-t` for better clarity.
- Modified the `processWithdrawRequest` function to include error handling for panic recovery and improved UTXO management.
- Enhanced logging and error handling in various withdrawal-related functions to improve traceability and debugging.
- Introduced new fork configurations for light client and RGBX in `chain33.para.toml`.
- Enhanced the `Makefile` to support conditional builds based on the operating system.
- Updated `docker-compose` scripts to include volume mounts for persistent data storage.
- Improved error handling and added new utility functions in the `docker-compose-pre.sh` script.
- Adjusted `docker-compose.yml` to ensure proper service dependencies and configurations for the RGBX dApp.
- Added `endHeight` parameter to `ReqListPendingTx` for improved transaction querying.
- Introduced `requiredConfs` in the `rgbx` struct to manage block confirmations dynamically.
- Adjusted logging and error handling in transaction processing for better traceability.
@bysomeone bysomeone changed the title Feat btc canister WIP Feat btc canister May 6, 2026
@bysomeone bysomeone force-pushed the feat-btc-canister branch from 063d78d to 7d9330e Compare May 7, 2026 06:17
bysomeone and others added 3 commits May 7, 2026 15:35
- Remove replace directive pointing to bysomeone/chain33 fork
- Update to github.com/33cn/chain33@master

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
@bysomeone bysomeone force-pushed the feat-btc-canister branch from 7c9f5fe to e59d6d2 Compare May 8, 2026 03:43
bysomeone and others added 3 commits May 8, 2026 12:53
para1 neutrino node reads btcd RPC cert at startup, but had no dependency
on btcd being ready, causing exit 2 panic when cert file didn't exist yet.
Also skip rgbx/rollup in Jenkins CI since they are covered by GitHub Actions.

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
@bysomeone bysomeone changed the title WIP Feat btc canister Feat btc canister May 8, 2026
Copy link
Copy Markdown
Collaborator

@vipwzw vipwzw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

整体规模较大(91 commits / 122 文件 / +17402/-442),新增 RGBX 与 Lightclient 两个 dapp 用于 BTC 跨链 + TSS 出金,同时修复 EVM 两个历史问题,调整 CI/构建脚本。CI 全绿、mergeable: MERGEABLE,但代码本身还有几处比较硬的逻辑/健壮性问题,建议合并前修掉。

1. 总体印象

  • 模块边界清楚:lightclient 负责 BTC header 入链与 SPV 校验、Neutrino + TSS 客户端;rgbx 负责资产 mint/transfer/deposit/withdraw 与 confirm 的链上协议。
  • 测试覆盖明显补强(executor_test.gochecktx_test.goexec_transfer_confirm_test.govalidate_proof_test.golog_address_integration_test.go 等)。
  • EVM 修复点都对(log address + gasmultiple),但 quickEstimateGasValue 有一个不易察觉的语义回退(见下方 §2.1)。
  • btcec v1→v2 升级遍及 dpos/ticket/dposvote,是必要适配且看起来正确。
  • 最关键的问题集中在 neutrino 客户端的并发与 nil 取址,以及 rgbx 跨链 withdraw 的"超时分支无回滚"逻辑。

2. 重要问题(建议合并前处理)

2.1 EVM quickEstimateGasValue 丢掉了"30% 下限"语义

plugin/dapp/evm/executor/query.go

func quickEstimateGasValue(usedGas uint64, gasmultiple interface{}) uint64 {
    multiple := 1.3 // quick gas 默认增加30%
    switch v := gasmultiple.(type) {
    case float64:
        multiple = v
    case int64:
        multiple = float64(v)
    }
    return uint64(float64(usedGas) * multiple)
}

旧实现在 multiple < 1.3 时强制抬到 1.3("默认 +30% 下限");新实现把这条规则去掉了,运维只要把 gasmultiple 设到 1.0/0.5 就会让快估返回值低于实际所需 gas,链上交易会因 OOG 失败。注释还在写"默认增加 30%",与行为不一致。

建议:保留下限或显式去掉注释,明确 <1.0 / <1.3 的处理策略(同时给 int64=0 时一个合理回退)。另外测试中并未覆盖 multiple < 1 的回归。

2.2 processWithdrawRequestreq.stickyUTXO == nil 分支里解引用 nil

plugin/dapp/lightclient/rpc/lightclient/neutrino/bitcoin.go (L211–228)

stickyUTXO := lockedUTXOs[len(lockedUTXOs)-1]
expectedHash := n.getExpectedWithdrawHash(stickyUTXO.OutPoint.String())
...
// 提现交易构建后,则和最后一个utxo强绑定,后续不能更改
if req.stickyUTXO == nil {

    stickyUTXO := lockedUTXOs[len(lockedUTXOs)-1]
    if err = n.setWithdrawStickyUTXO(req.chain33WithDrawHash, stickyUTXO); err != nil {
        log.Error("processWithdrawRequest setWithdrawStickyUTXO", "txHash", txHash, "stickyUTXO", req.stickyUTXO.OutPoint.String(), "err", err)
        return err
    }
    req.stickyUTXO = stickyUTXO
}

进入分支的前提就是 req.stickyUTXO == nil,但日志里又用 req.stickyUTXO.OutPoint.String()——setWithdrawStickyUTXO 失败时必然 panic。虽然外层 defer recover() 兜住了,但相当于把"DB 失败"路径变成了 panic→错误日志变形→UTXO 释放,真正的错误信息被覆盖。

另外这里还存在变量遮蔽:外层 stickyUTXO (L211) 与内层重新声明的 stickyUTXO (L222) 同名,进一步增加阅读成本。

建议:

last := lockedUTXOs[len(lockedUTXOs)-1]
if req.stickyUTXO == nil {
    if err = n.setWithdrawStickyUTXO(req.chain33WithDrawHash, last); err != nil {
        log.Error("...", "stickyUTXO", last.OutPoint.String(), "err", err)
        return err
    }
    req.stickyUTXO = last
}

并对 lockedUTXOs 长度先做防御。

2.3 initCommitKey 用写锁包住一个无限循环 goroutine(潜在饥饿/死锁)

plugin/dapp/lightclient/rpc/lightclient/neutrino/utils.go

func (n *neutrinoClient) initCommitKey() {
    if n.commitKey != nil {
        return
    }
    n.lock.Lock()
    go func() {
        defer n.lock.Unlock()
        ticker := time.NewTicker(time.Second * 3)
        ...
        for {
            select {
            case <-n.ctx.Done():
                return
            case <-ticker.C:
                ...
            }
        }
    }()
}
  • 在父 goroutine Lock(),由子 goroutine 异步 Unlock(),整段 polling 期间持有写锁;
  • 同一把锁还服务于 getCommitKey()(RLock)、getBestBlock / setBestBlock(Lock/RLock),导致:
    • Start() 中后续启动的 handleBlockSyncsetBestBlock → 阻塞;
    • tss.init()getCommitKey() → 阻塞;
    • 直到钱包解锁/导出私钥成功才放开。
  • 还有一个数据竞争:if n.commitKey != nil 在加锁前裸读(虽然这里是初始化路径,问题较小)。

建议改为:用 chan struct{}/sync.Once + atomic.Pointer[crypto.PrivKey] 信号通知就绪,或者只用一把独立的 commitKeyMu,避免与 bestBlock 共用锁。

2.4 client.go: subMsg 永远输出 "receive invalid msg" 错误日志

plugin/dapp/lightclient/rpc/lightclient/neutrino/client.go (L164–170)

data, ok := msg.Data.(*types.TopicData)
if msg.Ty == types.EventReceiveSubData && ok && data.Topic == tssSignNotifyTopic {
    n.tss.subChan <- data
}
log.Error("SubMsg receive invalid msg", "ty", msg.Ty, "ok", ok)

log.Error 漏写在 else 分支里,每次正常订阅推送都会打一条 Error 日志,会污染监控告警。建议加 else { log.Error(...) }(或者降级为 Debug)。

2.5 confirmWithdrawSettlementTimeout 路径不退还锁定资产

plugin/dapp/rgbx/executor/exec.go

func (r *rgbx) Exec_Confirm(confirm *rtypes.ConfirmTx, tx *types.Transaction, index int) (*types.Receipt, error) {
    ...
    if confirm.GetTimeout() {
        return &types.Receipt{Ty: types.ExecOk}, nil
    }
    if confirm.ActionType == rtypes.TyWithDrawAsset {
        return r.confirmWithdrawSettlement(confirm, txHash, confirmHash)
    }
    ...
}

Withdraw 流程 Exec_Withdraw 已经把 tx.From() 的 XBTC 转入 crossChainLockAddress

lockReceipt, err := accDB.Transfer(tx.From(), lockAddr, withdraw.GetAmount())

如果对应的 Confirm 携带 Timeout=true(提现到 BTC 链失败),Exec_Confirm 直接返回 ExecOk、什么也不做——锁定地址里的资产永远卡死,用户无法找回。看起来缺少超时退还路径:

if confirm.GetTimeout() && confirm.ActionType == rtypes.TyWithDrawAsset {
    // refund: transfer from lockAddr back to original from
}

考虑到实际链上目前 timeout 路径主要由 RGBX UTXO 模式触发,但 checkConfirm 里允许 withdraw + timeout 同时通过校验,建议明确策略:要么在 checkTx 中拒绝 withdraw + timeout,要么补上退款逻辑。

2.6 多处未做切片下标边界校验

下面这些路径都接受外部数据,缺少长度判断:

  • plugin/dapp/lightclient/rpc/lightclient/neutrino/btcwallet.go L608–613:tx.TxIn[0].Witness 未检查 len(tx.TxIn) > 0
  • plugin/dapp/rgbx/executor/validate_proof.go hasDepositCommitmenttx.TxIn[0].PreviousOutPoint 同上
  • plugin/dapp/lightclient/rpc/lightclient/neutrino/rgbx.go createConfirmPayloadout.PkScript[0] 未检查 len > 0
  • parseOpReturnpkScript[2:] 由调用方保护(len > 2),但 out.PkScript 长度为 0 的输出(Value=0, PkScript=nil)能反序列化通过

建议统一加上 if len(...) == 0 { continue } / len(tx.TxIn) > 0 防御,避免一个畸形 BTC 交易把 chain33 共识节点送进 panic。

2.7 validate_proof.go 同包重复导入

"github.com/33cn/plugin/plugin/dapp/lightclient/lighttypes"
ltypes "github.com/33cn/plugin/plugin/dapp/lightclient/lighttypes"

同一个包既以默认名 lighttypes 又以别名 ltypes 引入。Go 允许,但 goimports/lint 会报,并让阅读者困惑——保留任意一个即可。


3. 一般性建议

  • Exec_ConfirmconfirmHash 来源于 confirm.GetTxHash(),但日志里两个名字混用(confirmTxconfirmHashconfirmTxHash),建议统一以减少调试成本。
  • pullPendingTxtx 直接发到 withdrawReqChan(容量 256),队列满会阻塞整个轮询;考虑 select { default: } 或按高度/索引分页消费。
  • r.client.cfg.MaxUtxoRescanTime *= int64(time.Hour / time.Second) 是"小时→秒",但配置类型只是 int64,注释和单位说明应该写到结构体注释里,避免运维误传秒数。
  • rgbx.go types 中 tlog 声明但未使用,TyUnknowAction 拼写错误(缺一个 n,对外可能成为公开符号),这两个虽然小但都属于一旦发布就难收回的 API 类问题。
  • Jenkinsfilerm -rf cross2eth mix rgbx rollup|| true:bash 解析没问题,但建议补回 || 前的空格以保持可读。
  • Makefile 跨平台 build_ciCGO_ENABLED=0 适合在 Linux 容器跑,但要注意 chain33 主程序是否依赖 CGO;这块需要在 mac→docker 流程做一次完整冒烟。
  • analyzeTransactionlen(witness) == 2 && witness[1] == tssPubKey 判断 TSS 输入,仅支持 P2WPKH;如果将来切到 Taproot/嵌套 SegWit,识别逻辑要换。建议留一个 // only P2WPKH 注释或抽出一个识别函数。
  • subMsg 还有一个潜在死锁:当 n.tss.subChan 满时(容量 100),subMsg 会阻塞在 n.tss.subChan <- data,导致 qclient 消费不动,进而 chain33 内部消息总线背压。建议非阻塞投递+丢弃或更大缓冲。

4. 测试 / CI

  • 新加的单元测试覆盖了 BTC header 校验、mapBtcHeaderVerifyErrcreateConfirmPayloadparseTxFromNotify、EVM log address/gasmultiple 等关键路径,质量不错。
  • 推荐补两类:
    1. quickEstimateGasValuegasmultiple < 1.0 时的回归测试(验证 §2.1 决定的语义)。
    2. processWithdrawRequestsetWithdrawStickyUTXO 失败的路径,避免 §2.2 的 nil 解引用回归。
  • ci_rgbx 工作流跑 make docker-compose dapp=rgbx + docker-compose-down,并通过 Dockerfile.btcd 起 regtest 集群,配合 cmd/ci/scripts/*.sh 是端到端覆盖;macOS 下 awk-based build/autotest/run.sh 改写也是合理的可移植性修复。

5. 结论

需要先修:

  • §2.1 quickEstimateGasValue 行为回退(语义/测试)
  • §2.2 processWithdrawRequest nil 解引用
  • §2.3 initCommitKey 锁与持锁 goroutine 的设计
  • §2.4 subMsg 错误日志位置
  • §2.5 withdraw + timeout 资产无法退还

可一并修复:§2.6 多处切片越界保护、§2.7 重复导入;以及 §3 中的若干工程细节。

修完上述五点后,从代码层面这个 PR 已经具备合并条件;考虑到改动量大、新增 91 commits,也建议在合并前再让另一位熟悉 chain33 共识/account 的 reviewer 至少 sign-off 一次跨链 deposit/withdraw 路径的资金安全审查。

…e, gas estimation, and defensive checks

- evm: add 1.0 lower bound to quickEstimateGasValue preventing OOG on low gasmultiple config
- lightclient: fix nil dereference when setWithdrawStickyUTXO fails, eliminate variable shadowing
- lightclient: replace shared lock with dedicated commitKeyMu + sync.Once in initCommitKey
- lightclient: wrap subMsg error log in else branch to avoid noise on valid events
- lightclient: add P2WPKH-only comment and slice bounds checks (tx.TxIn, out.PkScript)
- rgbx: reject withdraw+timeout confirm in checkTx with ErrWithdrawConfirmTimeoutNotAllowed
- rgbx: remove duplicate lighttypes import, fix TyUnknowAction typo, remove unused tlog
- rgbx: unify confirmHash log field naming in Exec_Confirm and related methods
- tss: remove unused singleInputSignTimeout, crypto import, and duplicate commitTxKey field

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Copy link
Copy Markdown
Collaborator Author

@bysomeone bysomeone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

感谢详细的 review,以下逐条回复:

§2.1 quickEstimateGasValue 下限

已添加 multiple < 1.0 下限保护,防止运维误配导致 Gas 估算不足。注释已更新,新增 below 1.0 clampedexactly 1.0 两个回归测试用例。

§2.2 processWithdrawRequest nil 解引用

已修复:消除变量遮蔽(统一使用 lastUTXO),错误日志改用 lastUTXO.OutPoint.String(),添加 len(lockedUTXOs) == 0 防御检查。

§2.3 initCommitKey 锁与持锁 goroutine

保留了先加锁再启动 goroutine 的阻塞语义(确保 getCommitKey() 调用者在 key 就绪前阻塞),但改用专用 commitKeyMu sync.RWMutex 替代共享 n.lockgetBestBlock/setBestBlock 不再受影响。同时使用 sync.Once 消除原来的裸读 data race,并添加注释说明设计意图。

§2.4 subMsg 错误日志

已加 else 分支,正常订阅推送不再输出 Error 日志。

§2.5 withdraw + timeout 资产退还

提现是代理自动提交的跨链操作,发起后不支持链上退款。处理方式:

  • checkConfirm 中显式拒绝 withdraw + timeout,返回专用错误 ErrWithdrawConfirmTimeoutNotAllowed
  • Exec_Confirm 保持不变(timeout 直接返回 ExecOk,checkTx 已拦截)
  • 新增测试用例:Timeout=true + ActionType=TyWithDrawAsset → ErrWithdrawConfirmTimeoutNotAllowed

§2.6 切片下标边界

3 处已全部添加长度检查:

  • btcwallet.go: tx.TxIn[0].Witness → 先检查 len(tx.TxIn) > 0
  • validate_proof.go hasDepositCommitment: 同上,长度为 0 返回 false
  • rgbx.go createConfirmPayload: out.PkScript[0] → 先检查 len(out.PkScript) > 0

§2.7 重复导入

已移除裸 lighttypes 导入,统一使用 ltypes 别名。

§3 一般性建议(额外处理)

  • tlog 未使用 → 已移除
  • TyUnknowAction 拼写 → 修正为 TyUnknownAction
  • tssService.commitTxKey 字段重复 → 已移除,统一走 client.getCommitKey()
  • singleInputSignTimeout 未使用 → 已移除
  • analyzeTransaction P2WPKH 限制 → 已添加注释说明仅支持 P2WPKH
  • confirmHash/confirmTx 日志字段混用 → 统一为 confirmHash

关于 §3 中未处理项

  • pullPendingTx 通道满阻塞和 subMsg subChan 满阻塞 → 当前属于预期设计(背压信号),暂不修改
  • MaxUtxoRescanTime 单位和 Jenkinsfile 空格 → 属于代码风格问题,后续统一处理

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants