feat(transport): WebRTC DataChannel pubsub via Cloudflare Realtime SFU#2048
Draft
spomichter wants to merge 1 commit into
Draft
feat(transport): WebRTC DataChannel pubsub via Cloudflare Realtime SFU#2048spomichter wants to merge 1 commit into
spomichter wants to merge 1 commit into
Conversation
…e SFU Implements a new pubsub transport backed by WebRTC DataChannels over Cloudflare's Realtime SFU. Two new classes in dimos/protocol/pubsub/impl/webrtcpubsub.py: - CloudflareSession: manages the WebRTC PeerConnection lifecycle. Opens two CF sessions (publisher + subscriber) so a single process can do loopback pubsub. Runs aiortc on a dedicated background asyncio thread with its own ThreadPoolExecutor (so we don't leak asyncio_N worker threads). Uses negotiated=True placeholder DCs with id=100 during transport establishment to avoid stream-id collisions with CF-assigned ids. - WebRTCPubSub: bytes-on-the-wire pubsub facade matching the LCMPubSubBase / BytesSharedMemory interface (string topics, bytes payloads). Lazily creates pub/sub DataChannel pairs on first publish/subscribe per topic. Also adds: - WebRTCTransport in dimos/core/transport.py (mirrors LCMTransport pattern, no encoding - bytes only). - WebRTC benchmark testcase in dimos/protocol/pubsub/benchmark/testdata.py, gated on aiortc + CF_TELEOP_APP_ID / CF_TELEOP_APP_SECRET env vars. - Integration test in dimos/protocol/pubsub/impl/test_webrtcpubsub.py covering basic pub/sub, latency, and throughput (all live tests skip without CF credentials). - aiortc + httpx as new 'webrtc' optional extra in pyproject.toml. Live benchmark (us-east-2 -> CF edge): - 64-256B: ~10K msgs/s, 0% loss - 1KiB: ~7K msgs/s, 0% loss - >= 64KiB: dropped (above SCTP message size) - Median single-RTT: ~2.5 ms
spomichter
force-pushed
the
feat/webrtc-transport
branch
from
e56fd1f to
212450b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
DimOS has LCM, SHM, DDS, and ROS transports for local/LAN communication but no transport for internet-scale real-time data (NAT traversal, global edge routing). The hosted teleop system currently requires a dedicated
HostedTeleopModuleto bridge WebRTC DataChannels to DimOS streams — adding unnecessary complexity.Solution
Add a WebRTC DataChannel transport backed by Cloudflare Realtime SFU, following the same pubsub abstraction as
LCMTransportandSHMTransport.Architecture
CloudflareSession— manages two CF SFU sessions (pub + sub) on a dedicated background asyncio thread. Per-topic DataChannel pairs created lazily. Thread-safe.WebRTCPubSub— thin pubsub facade (same interface as LCM, BytesSharedMemory)WebRTCTransport— DimOS Transport[T] implementationBenchmark Results (us-east-2 EC2 → CF edge)
Breaking Changes
None — new optional transport. Requires
pip install dimos[webrtc]for aiortc.How to Test
Contributor License Agreement