 |
+  |
+  |
+
 |
+  |
+ + |
+
+ 
+
+
This guide takes you from a fresh Untold Engine checkout to a working game
+project. You will run the demo, create an Xcode project, export assets into the
+engine's .untold runtime format, and load those assets from a GameScene.
You can create projects and export assets in two ways:
+After your project is created, both workflows lead to the same place: an Xcode
+project with a GameData folder that contains the assets your game loads at
+runtime.
Clone the repository and launch the demo:
+ +You can create a project with either Untold Engine Studio or the CLI. If you are +new to Untold Engine, start with the Editor. If you prefer terminal workflows or +want repeatable project setup, use the CLI.
+Use Untold Engine Studio for a visual workflow. It is a standalone editor for +creating projects, preparing assets, composing scenes, and generating scene files +used inside your game.
+ +
To set up a project: +1. Click on "New". +2. Provide a Project name +3. Provide a Bundle name +4. Select the Target Platform +5. Provide an output path
+Untold Engine Studio will create an Xcode project ready to be used with Untold Engine.
+Use untoldengine-create to generate a ready-to-run Xcode project with Untold Engine wired in.
Install it from the repository:
+ +Now create an Xcode project. The example below uses --platform visionos to
+create a Vision Pro project.
cd ~/Projects
+untoldengine-create create VisionGame --platform visionos
+open VisionGame/VisionGame.xcodeproj
+If you want to create a project for other platforms, you can use the flags below:
+# visionOS (Apple Vision Pro)
+untoldengine-create create MyGame --platform visionos
+
+# macOS (default)
+untoldengine-create create MyGame --platform macos
+
+# iOS with ARKit
+untoldengine-create create MyGame --platform ios-ar
+
+# iOS
+untoldengine-create create MyGame --platform ios
+Dependency behavior by platform:
+visionos: UntoldEngineXR + UntoldEngineARios-ar: UntoldEngineARios and macos: UntoldEngine.untoldUntold Engine uses .untold as its native runtime asset format. USDZ/USD remains
+the authoring format — you model assets in your DCC tool, export to USDZ, then
+convert to .untold before loading them in the engine.
The .untold format is a binary container optimised for fast runtime parsing with
+no ModelIO dependency. It supports runtime mesh data, PBR materials, texture references,
+transforms, bounds, and exported animation clips.
++Note: The exporter requires Blender.
+
You can convert assets with either Untold Engine Studio or the CLI. If you are +new to Untold Engine, start with the Editor. If you prefer terminal workflows or +need repeatable asset export commands, use the CLI.
+To convert a USDZ file into the .untold format using the editor:
.untold model under the Model CategoryAt this point, head over to your Xcode project. You will also notice that your .untold model is under Sources/<ProjectName>/GameData/Models.
Use the export-untold script to convert a single USDZ asset:
./scripts/export-untold \
+ --input /path/to/your/model/robot/robot.usdz \
+ --output /path/to/your/project/GameData/Models/robot/robot.untold \
+ --ConvertOrientation \
+ --source-orientation blender-native
+For animation assets, use the --animation flag:
./scripts/export-untold \
+ --input /path/to/your/animation/robot/robot.usdz \
+ --output /path/to/your/project/GameData/Animations/robot/robot.untold \
+ --ConvertOrientation \
+ --source-orientation blender-native \
+ --animation
+For large scenes that need tile-based streaming, use export-untold-tiles to
+partition the scene and generate a manifest JSON:
./scripts/export-untold-tiles \
+ --input /path/to/your/model/dungeon/dungeon.usdz \
+ --output-dir /path/to/your/project/GameData/StreamModels/dungeon/tile_exports \
+ --tile-size-x 25 \
+ --tile-size-z 25 \
+ --generate-hlod \
+ --generate-lod
+For the full list of options, validation flags, and expected output layout see +Using The Exporter. For optional asset optimization +workflows, see Optimizations.
+Once in your Xcode project, head over to the init function in Sources/
Use setEntityMeshAsync to load an .untold file as an always-resident asset.
+This is the right choice for props, characters, and any object that should stay
+in memory for the lifetime of the scene.
//...After configureEngineSystems()
+
+let entity = createEntity()
+setEntityName(entityId: entity, name: "robot")
+
+setEntityMeshAsync(entityId: entity, filename: "robot", withExtension: "untold") { success in
+ if success {
+ translateBy(entityId: entity, position: simd_float3(0.0, 0.0, 0.0))
+ setEntityKinetics(entityId: entity)
+ }
+ setSceneReady(success)
+}
+setEntityMeshAsync is non-blocking. The completion block fires on the main thread
+once the mesh is parsed and uploaded to GPU memory.
Use setEntityStreamScene to load a large scene that streams tiles in and out of
+GPU memory based on camera proximity. Pass either a local manifest path or a remote
+https:// URL — the engine handles downloading and caching automatically.
//..After configureEngineSystems()
+
+
+let sceneRoot = createEntity()
+setEntityName(entityId: sceneRoot, name: "dungeon")
+
+// Local manifest
+setEntityStreamScene(entityId: sceneRoot, manifest: "dungeon", withExtension: "json") { success in
+ setSceneReady(success)
+}
+To streame a remote scene, you use the same function setEntityStreamedScene() but provide a url to your manifest json file.
// Remote manifest (downloaded and cached on demand)
+if let url = URL(string: "https://cdn.example.com/dungeon/dungeon.json") {
+ setEntityStreamScene(entityId: sceneRoot, url: url) { success in
+ setSceneReady(success)
+ }
+}
+setEntityStreamScene registers lightweight stub entities for every tile in the
+manifest, all parented under sceneRoot (no geometry is parsed at this point).
+GeometryStreamingSystem then loads and unloads tile geometry as the camera moves.
+See Tile-Based Streaming for the full streaming
+architecture.
++Legacy overloads —
+loadTiledScene(manifest:)andloadTiledScene(url:)remain +available for backwards compatibility. They create an internal root entity automatically.
Retrieve a named entity with findEntity(name:) inside the completion block or
+after setSceneReady:
setEntityMeshAsync(entityId: entity, filename: "stadium", withExtension: "untold") { success in
+ if let player = findEntity(name: "player") {
+ rotateTo(entityId: player, angle: 0, axis: simd_float3(0.0, 1.0, 0.0))
+ setEntityKinetics(entityId: player)
+ }
+ setSceneReady(success)
+}
+Create a camera and directional light manually in your scene setup, then position +the camera after assets load:
+let gameCamera = createEntity()
+setEntityName(entityId: gameCamera, name: "Main Camera")
+createGameCamera(entityId: gameCamera)
+CameraSystem.shared.activeCamera = gameCamera
+
+let light = createEntity()
+setEntityName(entityId: light, name: "Directional Light")
+createDirLight(entityId: light)
+After loading:
+ +A complete GameScene using the patterns above:
final class GameScene {
+
+ init() {
+ // Camera and light
+ let gameCamera = createEntity()
+ setEntityName(entityId: gameCamera, name: "Main Camera")
+ createGameCamera(entityId: gameCamera)
+ CameraSystem.shared.activeCamera = gameCamera
+
+ let light = createEntity()
+ setEntityName(entityId: light, name: "Directional Light")
+ createDirLight(entityId: light)
+
+ // Load a single always-resident asset
+ let stadium = createEntity()
+
+ setEntityMeshAsync(entityId: stadium, filename: "stadium", withExtension: "untold") { success in
+ if let player = findEntity(name: "player") {
+ rotateTo(entityId: player, angle: 0, axis: simd_float3(0.0, 1.0, 0.0))
+ setEntityAnimations(entityId: player, filename: "running", withExtension: "untold", name: "running")
+ setEntityAnimations(entityId: player, filename: "idle", withExtension: "untold", name: "idle")
+ setEntityKinetics(entityId: player)
+ }
+
+ moveCameraTo(entityId: findGameCamera(), 0.0, 3.0, 10.0)
+ ambientIntensity = 0.4
+ setSceneReady(success)
+ }
+ }
+}
+For a large streaming scene, replace the setEntityMeshAsync call with setEntityStreamScene:
Untold Engine supports optional asset optimization workflows that reduce runtime
+memory, file size, and streaming cost. These steps are usually applied after
+exporting assets to .untold.
For exporter commands, flags, and expected output layout, see +Using The Exporter.
+ASTC is a GPU-native block-compression format supported on all Apple Silicon and A-series devices. Converting textures to ASTC reduces texture memory by 4-8x compared to uncompressed RGBA8 and eliminates CPU-side decode. The GPU receives the compressed blocks directly.
+ASTC compression is a post-export step run with texbake.py, separate from the exporters. This keeps the exporter's Blender Python environment free of extra dependencies.
Install astcenc. The tool is resolved in this order:
ASTCENC_BIN=/path/to/astcencTools/astcenc/astcenc beside the repo rootastcenc on PATH# 1. Export the asset
+./scripts/export-untold \
+ --input GameData/Models/robot/robot.usdz \
+ --output GameData/Models/robot/robot.untold
+
+# 2. Bake all textures in the Textures/ directory to .utex
+python3 scripts/texbake.py --dir GameData/Models/robot/Textures/
+
+# 3. Patch the .untold file to reference the .utex files
+python3 scripts/texbake.py --patch-refs GameData/Models/robot/robot.untold
+Tile exports produce many .untold files. Pass the tile output directory to --patch-refs and all .untold files inside are patched in one go:
# 1. Export tiles
+./scripts/export-untold-tiles \
+ --input GameData/Models/dungeon/dungeon.usdz \
+ --output-dir GameData/Models/dungeon/tile_exports
+
+# 2. Bake all textures
+python3 scripts/texbake.py --dir GameData/Models/dungeon/tile_exports/Textures/
+
+# 3. Patch all .untold files in the tile directory
+python3 scripts/texbake.py --patch-refs GameData/Models/dungeon/tile_exports/
+.utex using astcenc:.untold binary to point each texture reference at the new .utex file and sets the textureFormat field to the correct ASTC variant..utex when present and falls back to PNG/JPEG otherwise.For cases where filename-based slot detection is insufficient, pass --input and --slot directly:
python3 scripts/texbake.py \
+ --input GameData/Models/robot/Textures/surface_data.png \
+ --slot roughness
+Available slots: base_color, normal, roughness, metallic, occlusion, orm, emissive, opacity, data.
Pass --compress-geometry to export-untold or export-untold-tiles to compress the vertex and index chunks of the output .untold file with LZ4.
Install the Python LZ4 package:
+ +./scripts/export-untold \
+ --input GameData/Models/robot/robot.usdz \
+ --output GameData/Models/robot/robot.untold \
+ --compress-geometry
+./scripts/export-untold-tiles \
+ --input GameData/Models/dungeon/dungeon.usdz \
+ --output-dir GameData/Models/dungeon/tile_exports \
+ --tile-size-x 25 \
+ --tile-size-y 10000 \
+ --tile-size-z 25 \
+ --compress-geometry
+vertex_data and index_data chunks are compressed. Metadata chunks (string table, entity table, mesh table, material table, texture table) are always stored uncompressed.lz4.block, not lz4.frame), which matches Apple's COMPRESSION_LZ4_RAW algorithm used by the runtime decompressor.Compression is compatible with all other flags including --validate, --generate-hlod, --generate-lod, and the ASTC texture bake workflow.
The Spatial Debugger provides visual overlays that help diagnose +spatial systems in the engine, including:
+It renders wireframe octree leaf bounds and can color them based on +runtime state to help identify issues such as:
+This tool is intended for debugging large scenes and spatial performance +issues.
+The Spatial Debugger renders octree leaf bounds as wireframe boxes.
+Each leaf represents a spatial region managed by the engine's octree +system.
+Leaf coloring can be configured to visualize:
+import UntoldEngine
+
+// Optional: tighten world bounds to your scene for better visualization.
+OctreeSystem.shared.worldBounds = AABB(
+ min: simd_float3(-40, -5, -40),
+ max: simd_float3(40, 25, 40)
+)
+
+// Enable octree debug rendering.
+setOctreeLeafBoundsDebug(
+ enabled: true,
+ maxLeafNodeCount: 0, // 0 = unlimited
+ occupiedOnly: true, // draw only leaves containing entries
+ colorMode: .culling
+)
+Disable the spatial debugger:
+ +Draws octree leaf bounds in a single color.
+Useful for verifying:
+setOctreeLeafBoundsDebug(
+ enabled: true,
+ maxLeafNodeCount: 0,
+ occupiedOnly: true,
+ colorMode: .plain
+)
+Colors leaves based on asset residency and streaming state.
+Useful for diagnosing:
+setOctreeLeafBoundsDebug(
+ enabled: true,
+ maxLeafNodeCount: 0,
+ occupiedOnly: true,
+ colorMode: .residency
+)
+Color meanings:
+Color Meaning
+Green assets resident + Yellow loading/unloading + Red unloaded + Orange mixed residency states within the leaf + White no residency signal found
+Residency information is derived from:
+StreamingComponentLODComponentIf these components are missing, the leaf falls back to white.
+Colors leaves based on runtime visibility.
+Useful for diagnosing:
+setOctreeLeafBoundsDebug(
+ enabled: true,
+ maxLeafNodeCount: 0,
+ occupiedOnly: true,
+ colorMode: .culling
+)
+Color meanings:
+Color Meaning
+Green entity visible this frame
+ Blue entity culled this frame
+ Gray entity hidden (RenderComponent.isVisible == false)
+ Orange leaf contains mixed visibility states
+ White no culling signal found
Culling colors are evaluated per leaf each frame, using:
+visibleEntityIdsRenderComponent.isVisibleBecause visibility updates every frame, colors may change as the camera +moves.
+To visualize the full octree structure including empty regions:
+setOctreeLeafBoundsDebug(
+ enabled: true,
+ maxLeafNodeCount: 0,
+ occupiedOnly: false,
+ colorMode: .residency
+)
+This can help diagnose:
+setOctreeLeafBoundsDebug(
+ enabled: Bool,
+ maxLeafNodeCount: Int,
+ occupiedOnly: Bool,
+ colorMode: SpatialDebugColorMode
+)
+Parameters:
+Parameter Description
+enabled Master toggle for octree visualization
+ maxLeafNodeCount Maximum leaves drawn per frame (0 = unlimited)
+ occupiedOnly When true, only leaves containing entries are drawn
+ colorMode Controls how leaf bounds are colored
Available color modes:
+.plain
+.residency
+.culling
+Disables all spatial debugging overlays.
+ +The spatial debugger:
+This ensures the visualization remains readable without interfering with +scene rendering.
+Default behavior:
+When enabled, the renderer periodically prints a status line:
+This provides quick feedback that the system is active and indicates:
+The engine also provides an LOD visualizer to display which LOD +level each renderable is currently using.
+Enable it with:
+ +This mode colors renderables by their active LOD level to help diagnose:
+When diagnosing spatial performance issues, a typical workflow is:
+Together these tools provide a full picture of how the engine is +managing spatial data.
+ + + + + + + + + + + + + +This page collects common Untold Engine setup patterns in one place. Use it when +you want a concrete example first, then open the focused API pages for details.
+For a broader first project walkthrough, see Getting Started.
+Untold Engine uses .untold as its preferred runtime asset format. Keep USD/USDZ
+as your authoring format, then export it before loading it in the engine.
Convert one asset:
+./scripts/export-untold \
+ --input GameData/Models/robot/robot.usdz \
+ --output GameData/Models/robot/robot.untold \
+ --ConvertOrientation \
+ --source-orientation blender-native
+Export an animation clip:
+./scripts/export-untold \
+ --input GameData/Models/robot/running.usdz \
+ --output GameData/Models/robot/running.untold \
+ --ConvertOrientation \
+ --source-orientation blender-native \
+ --animation
+Export a large streamed scene:
+./scripts/export-untold-tiles \
+ --input GameData/Models/city/city.usdz \
+ --output-dir GameData/Models/city/tile_exports \
+ --tile-size-x 25 \
+ --tile-size-z 25 \
+ --generate-hlod \
+ --generate-lod
+For exporter options and expected output layout, see +Using The Exporter. For texture compression and LZ4 +compression, see Optimizations.
+Entities are lightweight IDs. Systems add behavior by attaching components or by +calling helper APIs that register the right components for you.
+ +For direct entity/component lifecycle APIs, see +Using the Registration System.
+Use setEntityMesh when you want a small asset loaded immediately on the calling
+thread. This is useful for simple examples, tiny props, tests, and editor-style
+setup.
let crate = createEntity()
+setEntityName(entityId: crate, name: "crate")
+
+setEntityMesh(entityId: crate, filename: "crate", withExtension: "untold")
+translateTo(entityId: crate, position: simd_float3(0.0, 0.0, 0.0))
+For most game scene setup, prefer setEntityMeshAsync so loading does not stall
+the render loop.
Use setEntityMeshAsync when a model is large enough that loading it on the
+calling thread could stall the engine. The asset still becomes always-resident
+after it loads, but parsing and GPU upload happen asynchronously so the render
+loop can keep running.
setSceneReady(false)
+
+let robot = createEntity()
+setEntityName(entityId: robot, name: "robot")
+
+setEntityMeshAsync(entityId: robot, filename: "robot", withExtension: "untold") { success in
+ if success {
+ translateTo(entityId: robot, position: simd_float3(0.0, 0.0, 0.0))
+ rotateTo(entityId: robot, angle: 0.0, axis: simd_float3(0.0, 1.0, 0.0))
+ }
+
+ setSceneReady(success)
+}
+setEntityMeshAsync calls the completion block after parsing and GPU upload
+finish. Apply dependent transforms, physics setup, animation setup, and batching
+inside the completion block when they depend on the loaded mesh.
For loading behavior and progress APIs, see +Using Async Loading.
+Most examples need a game camera and at least one light.
+let camera = createEntity()
+setEntityName(entityId: camera, name: "Main Camera")
+createGameCamera(entityId: camera)
+CameraSystem.shared.activeCamera = camera
+moveCameraTo(entityId: camera, 0.0, 3.0, 10.0)
+
+let sun = createEntity()
+setEntityName(entityId: sun, name: "Sun")
+createDirLight(entityId: sun)
+
+ambientIntensity = 0.4
+For camera controls and path following, see +Using the Camera System. For light types, see +Using the Lighting System.
+Load the rigged mesh, then register one or more exported animation clips on the
+same entity. Use changeAnimation to choose the active clip.
let player = createEntity()
+setEntityName(entityId: player, name: "player")
+
+setEntityMeshAsync(entityId: player, filename: "redplayer", withExtension: "untold") { success in
+ guard success else {
+ setSceneReady(false)
+ return
+ }
+
+ setEntityAnimations(entityId: player, filename: "running", withExtension: "untold", name: "running")
+ setEntityAnimations(entityId: player, filename: "idle", withExtension: "untold", name: "idle")
+
+ changeAnimation(entityId: player, name: "idle")
+ setSceneReady(true)
+}
+Switch animations later:
+ +Pause or resume the current animation:
+pauseAnimationComponent(entityId: player, isPaused: true)
+pauseAnimationComponent(entityId: player, isPaused: false)
+For the full animation workflow, see +Using the Animation System.
+Call setEntityKinetics after creating or loading the entity. Then configure mass,
+gravity, or forces as needed.
let ball = createEntity()
+setEntityName(entityId: ball, name: "ball")
+
+setEntityMeshAsync(entityId: ball, filename: "ball", withExtension: "untold") { success in
+ guard success else {
+ setSceneReady(false)
+ return
+ }
+
+ translateTo(entityId: ball, position: simd_float3(0.0, 2.0, 0.0))
+ setEntityKinetics(entityId: ball)
+ setMass(entityId: ball, mass: 1.0)
+ setGravityScale(entityId: ball, gravityScale: 1.0)
+
+ setSceneReady(true)
+}
+Apply a force when a gameplay event happens:
+ +For physics properties and steering helpers, see +Using the Physics System and +Using the Steering System.
+Use setEntityStreamScene for large worlds, cities, terrain, and remote streamed
+scenes. The manifest is generated by export-untold-tiles.
setSceneReady(false)
+
+let sceneRoot = createEntity()
+setEntityName(entityId: sceneRoot, name: "city")
+
+setEntityStreamScene(entityId: sceneRoot, manifest: "city", withExtension: "json") { success in
+ if success {
+ moveCameraTo(entityId: findGameCamera(), 0.0, 4.0, 12.0)
+ ambientIntensity = 0.4
+ }
+
+ setSceneReady(success)
+}
+Load a remote manifest:
+let sceneRoot = createEntity()
+setEntityName(entityId: sceneRoot, name: "city")
+
+if let url = URL(string: "https://cdn.example.com/city/city.json") {
+ setEntityStreamScene(entityId: sceneRoot, url: url) { success in
+ setSceneReady(success)
+ }
+}
+Do not attach StreamingComponent manually for app-level streaming. The tiled
+scene pipeline creates and manages the streaming components internally.
For the public streaming API, see +Using Geometry Streaming. For the deeper +architecture, see Tile-Based Streaming.
+This example creates a camera, light, animated player, and physics-enabled ball.
+final class GameScene {
+
+ private var pendingLoads = 2
+ private var sceneFailed = false
+
+ init() {
+ setSceneReady(false)
+ setupCameraAndLight()
+ loadPlayer()
+ loadBall()
+ }
+
+ private func setupCameraAndLight() {
+ let camera = createEntity()
+ setEntityName(entityId: camera, name: "Main Camera")
+ createGameCamera(entityId: camera)
+ CameraSystem.shared.activeCamera = camera
+ moveCameraTo(entityId: camera, 0.0, 3.0, 10.0)
+
+ let sun = createEntity()
+ setEntityName(entityId: sun, name: "Sun")
+ createDirLight(entityId: sun)
+
+ ambientIntensity = 0.4
+ }
+
+ private func loadPlayer() {
+ let player = createEntity()
+ setEntityName(entityId: player, name: "player")
+
+ setEntityMeshAsync(entityId: player, filename: "redplayer", withExtension: "untold") { success in
+ if success {
+ translateTo(entityId: player, position: simd_float3(0.0, 0.0, 0.0))
+ setEntityAnimations(entityId: player, filename: "idle", withExtension: "untold", name: "idle")
+ setEntityAnimations(entityId: player, filename: "running", withExtension: "untold", name: "running")
+ changeAnimation(entityId: player, name: "idle")
+ setEntityKinetics(entityId: player)
+ }
+
+ self.finishLoad(success)
+ }
+ }
+
+ private func loadBall() {
+ let ball = createEntity()
+ setEntityName(entityId: ball, name: "ball")
+
+ setEntityMeshAsync(entityId: ball, filename: "ball", withExtension: "untold") { success in
+ if success {
+ translateTo(entityId: ball, position: simd_float3(2.0, 1.0, 0.0))
+ setEntityKinetics(entityId: ball)
+ setMass(entityId: ball, mass: 1.0)
+ setGravityScale(entityId: ball, gravityScale: 1.0)
+ }
+
+ self.finishLoad(success)
+ }
+ }
+
+ private func finishLoad(_ success: Bool) {
+ if !success {
+ sceneFailed = true
+ }
+
+ pendingLoads -= 1
+
+ if pendingLoads == 0 {
+ setSceneReady(!sceneFailed)
+ }
+ }
+}
+The Untold Engine simplifies adding animations to your rigged models, allowing for lifelike movement and dynamic interactions. This guide will show you how to set up and play animations for a rigged model.
+Start by creating an entity to represent your animated model.
+ +Load your rigged model's .untold runtime asset and link it to the entity. This step ensures the entity is visually represented in the scene.
++++++Note: If your model renders with the wrong orientation, set the flip parameter to false.
+
Load the animation data for your model by providing the exported animation .untold file and a name to reference the animation later.
setEntityAnimations(entityId: redPlayer, filename: "running", withExtension: "untold", name: "running")
+Trigger the animation by referencing its name. This will set the animation to play on the entity.
+ +To pause the current animation, simply call the following function. The animation component will be paused for the current entity.
+ +Once the animation is set up:
+UntoldEngine loads meshes asynchronously so scene setup does not stall the render loop. Use native .untold assets for runtime geometry; legacy USD/USDZ runtime paths are still supported.
setEntityMeshAsync is the primary async asset-loading API for always-resident assets:
let entity = createEntity()
+
+setEntityMeshAsync(
+ entityId: entity,
+ filename: "robot",
+ withExtension: "untold"
+) { success in
+ guard success else { return }
+ translateTo(entityId: entity, position: simd_float3(0, 0, 0))
+}
+The completion Bool is a success flag:
true: the asset loaded and registered successfullyfalse: loading failed and the engine fell back to the default placeholder meshIt does not indicate whether the asset used an out-of-core path.
+Use scene readiness when your own setup performs multiple dependent mutations:
+setSceneReady(false)
+
+let entity = createEntity()
+setEntityMeshAsync(entityId: entity, filename: "robot", withExtension: "untold") { success in
+ if success {
+ setEntityKinetics(entityId: entity)
+ translateTo(entityId: entity, position: simd_float3(0, 0, 0))
+ }
+ setSceneReady(success)
+}
+For ordinary setEntityMeshAsync(...) and setEntityStreamScene(...) flows, the engine already uses internal loading gates. setSceneReady(...) is mainly for your own multi-step scene setup.
| Use case | +API | +
|---|---|
| Single always-resident asset | +setEntityMeshAsync(...) |
+
| Large streamed world | +setEntityStreamScene(...) |
+
Use setEntityMeshAsync for props, characters, gameplay objects, HUD meshes, and any asset that should stay resident.
setEntityMeshAsync(
+ entityId: entity,
+ filename: "stadium",
+ withExtension: "untold"
+) { success in
+ if success {
+ setEntityStaticBatchComponent(entityId: entity)
+ }
+}
+Use setEntityStreamScene for geometry that should stream by camera distance:
let sceneRoot = createEntity()
+setEntityName(entityId: sceneRoot, name: "dungeon")
+
+setEntityStreamScene(entityId: sceneRoot, manifest: "dungeon", withExtension: "json") { success in
+ setSceneReady(success)
+}
+Or load a remote manifest directly:
+let sceneRoot = createEntity()
+setEntityName(entityId: sceneRoot, name: "dungeon")
+
+if let url = URL(string: "https://cdn.example.com/dungeon/dungeon.json") {
+ setEntityStreamScene(entityId: sceneRoot, url: url) { success in
+ setSceneReady(success)
+ }
+}
+++Legacy overloads —
+loadTiledScene(manifest:)andloadTiledScene(url:)remain available for backwards compatibility.
This is the public streaming workflow. Do not build app-level streaming logic around StreamingComponent or enableStreaming(...); those are internal to the tile/OCC pipeline.
streamingPolicysetEntityMeshAsync accepts a streamingPolicy parameter to control how geometry
+is uploaded to the GPU. For standalone assets, two values are relevant:
.auto — default; the engine chooses full upload or incremental based on asset size.immediate — always uploads in a single pass; use for props that must appear fully
+ formed on first frame (player characters, weapons, HUD objects)setEntityMeshAsync(
+ entityId: entity,
+ filename: "small_prop",
+ withExtension: "untold",
+ streamingPolicy: .immediate
+)
+.outOfCore is reserved for the engine's internal tile streaming pipeline. Passing it
+directly on a standalone entity is unsupported — StreamingComponent stubs created
+outside of a TileComponent hierarchy are not managed by GeometryStreamingSystem.
+Use setEntityStreamScene(...) instead when you need distance-based streaming.
Task {
+ let isLoading = await AssetLoadingState.shared.isLoadingAny()
+ print("Loading: \(isLoading)")
+}
+Task {
+ let count = await AssetLoadingState.shared.loadingCount()
+ print("Loading \(count) asset(s)")
+}
+Task {
+ let (current, total) = await AssetLoadingState.shared.totalProgress()
+ let percentage = total > 0 ? Float(current) / Float(total) * 100.0 : 0.0
+ print("Progress: \(percentage)% (\(current)/\(total))")
+}
+Task {
+ if let progress = await AssetLoadingState.shared.getProgress(for: entity) {
+ print("\(progress.filename): \(progress.currentMesh)/\(progress.totalMeshes)")
+ }
+}
+.untold is the preferred runtime format for static geometry.--animation can be loaded as .untold assets.setEntityStreamScene(...) automatically aligns texture streaming distances to the manifest radii and enables the full tile/HLOD/LOD/OCC streaming pipeline.This document explains how to move, rotate, and control cameras using the APIs in CameraSystem.swift.
For gameplay, always use the game camera (not the editor/scene camera). Call findGameCamera() and make it active:
If no game camera exists, findGameCamera() creates one and sets it up with default values.
Use absolute or relative movement:
+// Absolute position
+moveCameraTo(entityId: camera, 0.0, 3.0, 7.0)
+
+// Relative movement in camera local space
+cameraMoveBy(entityId: camera, delta: simd_float3(0.0, 0.0, -1.0), space: .local)
+
+// Relative movement in world space
+cameraMoveBy(entityId: camera, delta: simd_float3(1.0, 0.0, 0.0), space: .world)
+Use rotateCamera for pitch/yaw rotation, or cameraLookAt to aim at a target.
// Rotate by pitch/yaw (radians), with optional sensitivity
+rotateCamera(entityId: camera, pitch: 0.02, yaw: 0.01, sensitivity: 1.0)
+
+// Look-at orientation
+cameraLookAt(
+ entityId: camera,
+ eye: simd_float3(0.0, 3.0, 7.0),
+ target: simd_float3(0.0, 0.0, 0.0),
+ up: simd_float3(0.0, 1.0, 0.0)
+)
+Follow a target entity with a fixed offset. You can optionally smooth the motion.
+let target = findEntity(name: "player") ?? createEntity()
+let offset = simd_float3(0.0, 2.0, 6.0)
+
+// Instant follow
+cameraFollow(entityId: camera, targetEntity: target, offset: offset)
+
+// Smoothed follow
+cameraFollow(entityId: camera, targetEntity: target, offset: offset, smoothFactor: 6.0, deltaTime: deltaTime)
+cameraFollowDeadZone only moves the camera when the target leaves a box around it. This is useful for platformers and shoulder cameras.
let deadZone = simd_float3(1.0, 0.5, 1.0)
+cameraFollowDeadZone(
+ entityId: camera,
+ targetEntity: target,
+ offset: offset,
+ deadZoneExtents: deadZone,
+ smoothFactor: 6.0,
+ deltaTime: deltaTime
+)
+The camera path system moves the active camera through a sequence of waypoints with smooth interpolation.
+let waypoints = [
+ CameraWaypoint(
+ position: simd_float3(0, 5, 10),
+ rotation: simd_quatf(angle: 0, axis: simd_float3(0, 1, 0)),
+ segmentDuration: 2.0
+ ),
+ CameraWaypoint(
+ position: simd_float3(10, 5, 10),
+ rotation: simd_quatf(angle: Float.pi / 4, axis: simd_float3(0, 1, 0)),
+ segmentDuration: 2.0
+ )
+]
+
+startCameraPath(waypoints: waypoints, mode: .once)
+You can also build waypoints that look at a target:
+let waypoint = CameraWaypoint(
+ position: simd_float3(0, 5, 10),
+ lookAt: simd_float3(0, 0, 0),
+ up: simd_float3(0, 1, 0),
+ segmentDuration: 2.0
+)
+Call updateCameraPath(deltaTime:) from your main update loop:
startCameraPath(waypoints: waypoints, mode: .loop)
+
+let settings = CameraPathSettings(startImmediately: true) {
+ print("Camera path completed")
+}
+startCameraPath(waypoints: waypoints, mode: .once, settings: settings)
+startCameraPath and updateCameraPath operate on CameraSystem.shared.activeCamera.segmentDuration is the time to move from the current waypoint to the next.findGameCamera() and set it active before path playback or follow logic.The Gaussian System in the Untold Engine is responsible for rendering Gaussian Splatting models. It enables you to visualize high-quality 3D reconstructions created from photogrammetry or neural rendering techniques, providing a modern approach to displaying complex 3D scenes.
+Start by creating an entity that represents your Gaussian Splat object.
+To display a Gaussian Splat model, load its .ply file and link it to the entity using setEntityGaussian.
+ +Parameters:
+++Note: The Gaussian System renders point cloud data stored in the .ply format. Ensure your Gaussian Splat file is properly formatted and contains the necessary attributes (position, color, opacity, scale, rotation).
+
Once everything is set up:
+UntoldEngine streams large worlds through a manifest-driven tiled scene pipeline.
+The public rule is simple:
+| Use case | +API | +
|---|---|
| Streamed world geometry (manifest-driven) | +setEntityStreamScene(entityId:manifest:withExtension:completion:) |
+
| Handcrafted streaming zones (no manifest) | +StreamingRegionManager — register StreamingRegion AABB + asset lists directly |
+
| Always-resident assets | +setEntityMeshAsync(entityId:filename:withExtension:completion:) |
+
GeometryStreamingSystem manages the runtime once a streamed scene is loaded. It is not a public component-authoring workflow for standalone entities.
++For handcrafted zone streaming without a manifest (e.g. dungeon rooms, level sectors), use
+StreamingRegionManager.shared. See the StreamingRegionManager architecture doc for the full API.
let sceneRoot = createEntity()
+setEntityName(entityId: sceneRoot, name: "city")
+
+setEntityStreamScene(entityId: sceneRoot, manifest: "city", withExtension: "json") { success in
+ setSceneReady(success)
+}
+let sceneRoot = createEntity()
+setEntityName(entityId: sceneRoot, name: "city")
+
+if let url = URL(string: "https://cdn.example.com/city/city.json") {
+ setEntityStreamScene(entityId: sceneRoot, url: url) { success in
+ setSceneReady(success)
+ }
+}
+++Legacy overloads —
+loadTiledScene(manifest:)andloadTiledScene(url:)remain available for backwards compatibility. They create an internal root entity automatically. PrefersetEntityStreamScene(entityId:...)when you need a stable handle to the scene.
Remote manifests are downloaded and cached locally. Tile, HLOD, and per-tile LOD URLs are resolved relative to the manifest URL and fetched on demand.
+The engine uses multiple geometry layers:
+loadTile()StreamingComponent entities created internally inside large tilesStreamingComponent is internal to the tile-owned OCC path. External callers should not attach it manually or rely on enableStreaming(...).
These are the important fields for geometry streaming:
+| Field | +Meaning | +
|---|---|
streaming_radius |
+Full tile display zone | +
unload_radius |
+Tile teardown threshold | +
prefetch_radius |
+Background parse threshold before the tile becomes visible | +
priority |
+Tile load ordering when many tiles compete | +
hlod_levels |
+Optional far proxy meshes | +
lod_levels |
+Optional per-tile intermediate LOD meshes | +
file_size_bytes |
+Parse-budget hint used by the runtime gate | +
If prefetch_radius is omitted, the engine computes it automatically from the gap between streaming_radius and unload_radius.
Each update tick, GeometryStreamingSystem:
maxQueryRadius.maxConcurrentTileLoads tiles, subject to tileParseMemoryBudgetMB.maxConcurrentLoads.Important defaults:
+maxConcurrentTileLoads = 2maxConcurrentLoads = 3maxConcurrentLODLoads = 4maxConcurrentHLODLoads = 4updateInterval = 0.1burstTickInterval = 0.016GeometryStreamingSystem.shared.maxConcurrentTileLoads = 2
+GeometryStreamingSystem.shared.maxConcurrentLoads = 3
+GeometryStreamingSystem.shared.enableFrustumGate = true
+GeometryStreamingSystem.shared.tileFrustumGatePadding = 20.0
+GeometryStreamingSystem.shared.maxQueryRadius = 500.0
+Use maxQueryRadius large enough to cover the farthest unload_radius in the scene, or out-of-range tiles may not be discovered for teardown.
setEntityStreamScene(...) automatically aligns texture distance bands to the manifest radii.BatchingSystem automatically. OCC sub-mesh uploads join batching incrementally through normal residency events.GeometryStreamingSystem.shared.tileFrustumGatePaddingenableFrustumGate = truestreaming_radius and unload_radiusprefetch_radiusmaxConcurrentTileLoadssetEntityStreamScene(...)StreamingComponent entities to stream; tile ownership is enforcedThe Input System in the Untold Engine allows you to detect user inputs, such as keystrokes and mouse movements, to control entities and interact with the game. This guide will explain how to use the Input System effectively.
+To detect if a specific key is pressed, use the keyState object from the Input System.
+Example: Detecting the 'W' Key
+func init(){
+// Make sure that you have enabled keyevents in your init function:
+InputSystem.shared.registerKeyboardEvents()
+}
+
+// Then in the handleInput callback, you can do this:
+
+func handleInput() {
+ // Skip logic if not in game mode
+ if gameMode == false { return }
+
+ let inputSystem = InputSystem.shared
+
+ // Handle input here
+ if inputSystem.keyState.wPressed{
+ Logger.log(message: "w pressed")
+ }
+}
+let inputSystem = InputSystem.shared
+
+if inputSystem.keyState.aPressed == true {
+ // Move left
+}
+
+if inputSystem.keyState.sPressed == true {
+ // Move backward
+}
+
+if inputSystem.keyState.dPressed == true {
+ // Move right
+}
+Here’s an example function that moves a car entity based on keyboard inputs:
+func moveCar(entityId: EntityID, dt: Float) {
+
+ let inputSystem = InputSystem.shared
+
+ // Ensure we are in game mode
+ if gameMode == false {
+ return
+ }
+
+ var position = simd_float3(0.0, 0.0, 0.0)
+
+ // Move forward
+ if inputSystem.keyState.wPressed == true {
+ position.z += 1.0 * dt
+ }
+
+ // Move backward
+ if inputSystem.keyState.sPressed == true {
+ position.z -= 1.0 * dt
+ }
+
+ // Move left
+ if inputSystem.keyState.aPressed == true {
+ position.x -= 1.0 * dt
+ }
+
+ // Move right
+ if inputSystem.keyState.dPressed == true {
+ position.x += 1.0 * dt
+ }
+
+ // Apply the translation to the entity
+ translateTo(entityId: entityId, position: position)
+}
+To detect if a specific button is pressed, use the gameControllerState object from the Input System.
+Example: Detecting the 'A' button
+func init(){
+// Make sure that you have enabled game controller events in your init function:
+ InputSystem.shared.registerGameControllerEvents()
+}
+
+// Then in the handleInput callback, you can do this:
+
+func handleInput() {
+ // Skip logic if not in game mode
+ if gameMode == false { return }
+ let inputSystem = InputSystem.shared
+
+ // Handle input here
+ if inputSystem.gameControllerState.aPressed {
+ Logger.log(message: "Pressed A key")
+ }
+}
+UntoldEngine now has two distinct optimization workflows:
+| Workflow | +Use for | +
|---|---|
| Entity-level LOD + manual batching | +Always-resident props, structures, authored gameplay objects | +
| Manifest-driven tile streaming + automatic batching | +Large worlds, terrain, cities, remote streamed scenes | +
For normal entities that should stay resident, combine LODComponent with StaticBatchComponent:
private func setupLODWithBatching() {
+ var loadedCount = 0
+ let totalTrees = 20
+
+ for i in 0 ..< totalTrees {
+ let tree = createEntity()
+ setEntityName(entityId: tree, name: "Tree_\(i)")
+ setEntityLodComponent(entityId: tree)
+
+ let x = Float(i % 5) * 10.0
+ let z = Float(i / 5) * 10.0
+
+ addLODLevels(entityId: tree, levels: [
+ (0, "tree_LOD0", "untold", 50.0, 0.0),
+ (1, "tree_LOD1", "untold", 100.0, 0.0),
+ (2, "tree_LOD2", "untold", 200.0, 0.0),
+ ]) { success in
+ if success {
+ translateTo(entityId: tree, position: simd_float3(x, 0, z))
+ setEntityStaticBatchComponent(entityId: tree)
+ }
+
+ loadedCount += 1
+ if loadedCount == totalTrees {
+ enableBatching(true)
+ generateBatches()
+ }
+ }
+ }
+}
+This is still the correct pattern when all meshes are present up front and stay resident.
+For large worlds, do not build a manual LOD + enableStreaming(...) stack on standalone entities. Use the tiled-scene pipeline:
let sceneRoot = createEntity()
+setEntityName(entityId: sceneRoot, name: "city")
+
+setEntityStreamScene(entityId: sceneRoot, manifest: "city", withExtension: "json") { success in
+ setSceneReady(success)
+}
+In this workflow:
+lod_levels supply intermediate representationshlod_levels cover the far fieldYou do not call generateBatches() per tile. The runtime hands new resident tile geometry directly to BatchingSystem.
.untold for static runtime geometry whenever possible.StreamingComponent as internal to the tiled streaming architecture.The Untold Engine provides a flexible LOD system for optimizing rendering performance by displaying different mesh details based on camera distance.
+The LOD system allows you to: +- Add multiple levels of detail to any entity +- Automatically switch between LOD levels based on distance +- Customize distance thresholds for each LOD level +- Configure LOD behavior (bias, hysteresis, fade transitions)
+++Choose Your Path: You can set up LOD via the Editor (no code required) or programmatically in Swift.
+
Click the trash icon next to any LOD level to remove it.
+// Create entity
+let tree = createEntity()
+
+// Add LOD component
+setEntityLodComponent(entityId: tree)
+
+// Add LOD levels (from highest to lowest detail)
+addLODLevel(entityId: tree, lodIndex: 0, fileName: "tree_LOD0", withExtension: "untold", maxDistance: 50.0)
+addLODLevel(entityId: tree, lodIndex: 1, fileName: "tree_LOD1", withExtension: "untold", maxDistance: 100.0)
+addLODLevel(entityId: tree, lodIndex: 2, fileName: "tree_LOD2", withExtension: "untold", maxDistance: 200.0)
+addLODLevel(entityId: tree, lodIndex: 3, fileName: "tree_LOD3", withExtension: "untold", maxDistance: 400.0)
+How it works: +- LOD0 (highest detail) renders when camera is < 50 units away +- LOD1 renders between 50-100 units +- LOD2 renders between 100-200 units +- LOD3 (lowest detail) renders beyond 200 units
+Use addLODLevels to load all LOD levels with a single completion handler. This is especially important when combining LOD with static batching:
let tree = createEntity()
+setEntityLodComponent(entityId: tree)
+
+// Load all LOD levels and wait for completion
+addLODLevels(entityId: tree, levels: [
+ (0, "tree_LOD0", "untold", 50.0, 0.0),
+ (1, "tree_LOD1", "untold", 100.0, 0.0),
+ (2, "tree_LOD2", "untold", 200.0, 0.0),
+ (3, "tree_LOD3", "untold", 400.0, 0.0)
+]) { success in
+ if success {
+ print("All LOD levels loaded")
+
+ // Apply transforms AFTER mesh is loaded
+ translateTo(entityId: tree, position: simd_float3(10, 0, 5))
+
+ // Apply static batching if necessary
+ }
+}
+++Important: When using LOD with async loading, apply transforms (
+translateTo,rotateTo,scaleTo) inside the completion handler. Transforms applied before the mesh loads may not take effect.
You can also load an initial mesh synchronously before adding LOD levels:
+let tree = createEntity()
+
+// Load initial mesh synchronously (shows immediately)
+setEntityMesh(entityId: tree, filename: "tree_LOD0", withExtension: "untold")
+
+// Add LOD component
+setEntityLodComponent(entityId: tree)
+
+// Add LOD levels (will replace initial mesh when ready)
+addLODLevel(entityId: tree, lodIndex: 0, fileName: "tree_LOD0", withExtension: "untold", maxDistance: 50.0)
+addLODLevel(entityId: tree, lodIndex: 1, fileName: "tree_LOD1", withExtension: "untold", maxDistance: 100.0)
+addLODLevel(entityId: tree, lodIndex: 2, fileName: "tree_LOD2", withExtension: "untold", maxDistance: 200.0)
+addLODLevel(entityId: tree, lodIndex: 3, fileName: "tree_LOD3", withExtension: "untold", maxDistance: 400.0)
+Since addLODLevel loads meshes asynchronously, use the completion handler when you need to perform actions after loading completes:
let tree = createEntity()
+setEntityLodComponent(entityId: tree)
+
+// Chain completion handlers for sequential loading
+addLODLevel(entityId: tree, lodIndex: 0, fileName: "tree_LOD0", withExtension: "untold", maxDistance: 50.0) { success in
+ if success {
+ print("LOD0 loaded")
+ // Now it's safe to use the mesh data
+ }
+}
+LOD files should be organized in subdirectories:
+GameData/
+└── Models/
+ ├── tree_LOD0/
+ │ └── tree_LOD0.untold
+ ├── tree_LOD1/
+ │ └── tree_LOD1.untold
+ ├── tree_LOD2/
+ │ └── tree_LOD2.untold
+ └── tree_LOD3/
+ └── tree_LOD3.untold
+Note: Each LOD file should be in its own folder with the same name as the file (without extension).
+setEntityLodComponent(entityId:)Registers an LOD component on an entity. Call this before adding LOD levels.
+ +addLODLevel(entityId:lodIndex:fileName:withExtension:maxDistance:completion:)Adds a single LOD level to an entity.
+Parameters:
+- entityId: The entity to add LOD to
+- lodIndex: LOD level index (0 = highest detail)
+- fileName: Name of the mesh file (without extension)
+- withExtension: File extension (e.g., "untold")
+- maxDistance: Maximum camera distance for this LOD
+- completion: Optional callback when loading completes
addLODLevel(
+ entityId: tree,
+ lodIndex: 0,
+ fileName: "tree_LOD0",
+ withExtension: "untold",
+ maxDistance: 50.0
+) { success in
+ if success {
+ print("LOD0 loaded successfully")
+ }
+}
+addLODLevels(entityId:levels:completion:)Adds multiple LOD levels with a single completion handler. Useful when you need to wait for all LOD levels to load.
+Parameters:
+- entityId: The entity to add LOD levels to
+- levels: Array of tuples: (lodIndex, fileName, withExtension, maxDistance, screenPercentage)
+- completion: Called when ALL levels finish loading (true only if all succeeded)
addLODLevels(entityId: tree, levels: [
+ (0, "tree_LOD0", "untold", 50.0, 0.0),
+ (1, "tree_LOD1", "untold", 100.0, 0.0),
+ (2, "tree_LOD2", "untold", 200.0, 0.0)
+]) { success in
+ if success {
+ print("All LODs loaded")
+ }
+}
+removeLODLevel(entityId:lodIndex:)Removes a specific LOD level from an entity.
+ +replaceLODLevel(entityId:lodIndex:fileName:withExtension:maxDistance:completion:)Replaces an existing LOD level with a new mesh.
+replaceLODLevel(
+ entityId: tree,
+ lodIndex: 1,
+ fileName: "tree_LOD1_new",
+ withExtension: "untold",
+ maxDistance: 100.0
+)
+getLODLevelCount(entityId:) -> IntReturns the number of LOD levels for an entity.
+ +Adjust distances based on your scene scale:
+// Small scene (indoor environment)
+addLODLevel(entityId: prop, lodIndex: 0, fileName: "prop_LOD0", withExtension: "untold", maxDistance: 10.0)
+addLODLevel(entityId: prop, lodIndex: 1, fileName: "prop_LOD1", withExtension: "untold", maxDistance: 20.0)
+
+// Large scene (outdoor landscape)
+addLODLevel(entityId: mountain, lodIndex: 0, fileName: "mountain_LOD0", withExtension: "untold", maxDistance: 500.0)
+addLODLevel(entityId: mountain, lodIndex: 1, fileName: "mountain_LOD1", withExtension: "untold", maxDistance: 1000.0)
+addLODLevel(entityId: mountain, lodIndex: 2, fileName: "mountain_LOD2", withExtension: "untold", maxDistance: 2000.0)
+Configure global LOD behavior:
+// Adjust LOD bias (higher = switch to lower detail sooner)
+LODConfig.shared.lodBias = 1.5 // Performance mode
+LODConfig.shared.lodBias = 0.75 // Quality mode
+
+// Adjust hysteresis to prevent flickering
+LODConfig.shared.hysteresis = 10.0
+
+// Enable fade transitions between LODs - Not yet implemented
+LODConfig.shared.enableFadeTransitions = true
+LODConfig.shared.fadeTransitionTime = 0.5 // seconds
+Force a specific LOD level (useful for debugging):
+if let lodComponent = scene.get(component: LODComponent.self, for: tree) {
+ lodComponent.forcedLOD = 2 // Always show LOD2
+ // lodComponent.forcedLOD = nil // Resume automatic LOD selection
+}
+// Create entity with LOD component
+let rock = createEntity()
+setEntityLodComponent(entityId: rock)
+
+// Add LODs dynamically based on performance
+let lodFiles = ["rock_LOD0", "rock_LOD1", "rock_LOD2"]
+let distances: [Float] = [30.0, 60.0, 120.0]
+
+for (index, fileName) in lodFiles.enumerated() {
+ addLODLevel(
+ entityId: rock,
+ lodIndex: index,
+ fileName: fileName,
+ withExtension: "untold",
+ maxDistance: distances[index]
+ )
+}
+
+// Check LOD count
+let lodCount = getLODLevelCount(entityId: rock)
+print("Rock has \(lodCount) LOD levels")
+
+// Remove highest detail LOD on low-end hardware
+if isLowEndDevice {
+ removeLODLevel(entityId: rock, lodIndex: 0)
+}
+Base distances on object importance and size: +- Hero objects: Longer high-detail distance +- Background objects: Shorter high-detail distance +- Large objects: Visible from farther away, need more LODs
+setEntityMeshAsync) for better performanceforcedLOD during development to preview each LOD levelhasComponent(entityId: tree, componentType: LODComponent.self)CameraComponent and is activeLODConfig.shared.hysteresis valueLODConfig.shared.enableFadeTransitions = true - not yet implementedGameData/Models/ pathimport UntoldEngine
+
+// Create multiple trees with LODs
+var trees: [EntityID] = []
+
+for i in 0..<10 {
+ let tree = createEntity()
+ setEntityName(entityId: tree, name: "Tree_\(i)")
+
+ // Position trees
+ translateTo(entityId: tree, position: simd_float3(Float(i * 10), 0, 0))
+
+ // Add LOD component
+ setEntityLodComponent(entityId: tree)
+
+ // Add 4 LOD levels
+ addLODLevel(entityId: tree, lodIndex: 0, fileName: "tree_LOD0", withExtension: "untold", maxDistance: 50.0)
+ addLODLevel(entityId: tree, lodIndex: 1, fileName: "tree_LOD1", withExtension: "untold", maxDistance: 100.0)
+ addLODLevel(entityId: tree, lodIndex: 2, fileName: "tree_LOD2", withExtension: "untold", maxDistance: 200.0)
+ addLODLevel(entityId: tree, lodIndex: 3, fileName: "tree_LOD3", withExtension: "untold", maxDistance: 400.0)
+
+ trees.append(tree)
+}
+
+// Configure LOD system for this scene
+LODConfig.shared.lodBias = 1.2 // Slightly favor performance
+LODConfig.shared.hysteresis = 8.0 // Prevent flickering
+
+print("Created \(trees.count) trees with LOD support")
+When combining LOD with static batching, ensure transforms and batching setup happen after meshes are loaded:
+import UntoldEngine
+
+private func setupLODWithBatching() {
+ var loadedCount = 0
+ let totalTrees = 20
+
+ for i in 0..<totalTrees {
+ let tree = createEntity()
+ setEntityName(entityId: tree, name: "Tree_\(i)")
+
+ // Capture position for the closure
+ let x = Float(i % 5) * 10.0
+ let z = Float(i / 5) * 10.0
+
+ // Add LOD component BEFORE loading levels
+ setEntityLodComponent(entityId: tree)
+
+ // Load all LOD levels with completion handler
+ addLODLevels(entityId: tree, levels: [
+ (0, "tree_LOD0", "untold", 50.0, 0.0),
+ (1, "tree_LOD1", "untold", 100.0, 0.0),
+ (2, "tree_LOD2", "untold", 200.0, 0.0)
+ ]) { success in
+ if success {
+ // Apply transform AFTER mesh is loaded
+ translateTo(entityId: tree, position: simd_float3(x, 0, z))
+
+ // Mark for batching
+ setEntityStaticBatchComponent(entityId: tree)
+ }
+
+ // Track completion
+ loadedCount += 1
+ if loadedCount == totalTrees {
+ // All trees loaded - generate batches
+ enableBatching(true)
+ generateBatches()
+ print("\(totalTrees) trees configured with LOD + Batching")
+ }
+ }
+ }
+}
+Key points:
+1. Call setEntityLodComponent() before loading LOD levels
+2. Apply transforms (translateTo) inside the completion handler
+3. Call setEntityStaticBatchComponent() after mesh is loaded
+4. Call generateBatches() only after all entities are ready
The Lighting System lets you add illumination to your scenes using common real-time light types. Under the hood it wires up the required ECS components, provides an editor-friendly visual handle, and tags the light so the renderer can pick it up.
+Use for sunlight or distant key lights. Orientation (rotation) defines its direction.
+ +Omni light that radiates equally in all directions from a position.
+ +Cone-shaped light with a position and direction.
+ +Rect/area emitter used to mimic panels/windows; position and orientation matter.
+ + + + + + + + + + + + + + +The engine uses a PBR (Physically Based Rendering) material model. Each entity's mesh can contain one or more submeshes, and each submesh holds its own Material. You can read and write individual material properties at runtime using the functions below.
All material functions accept optional meshIndex and submeshIndex parameters (both default to 0) so you can target a specific submesh when an entity contains more than one.
++Note: Every update function automatically refreshes static batching for the affected entity, so you do not need to do this manually.
+
The base color is stored as a simd_float4 (RGBA). The .w component doubles as the opacity channel.
let color = getMaterialBaseColor(entityId: entity)
+// color.x = red, color.y = green, color.z = blue, color.w = alpha
+This converts the SwiftUI Color to RGBA internally. If the alpha is below 1.0, the material automatically switches to .blend alpha mode.
Controls how rough or smooth a surface appears. A value of 0.0 is perfectly smooth (mirror-like reflections) and 1.0 is fully rough (diffuse).
++When a roughness texture is present, the scalar value acts as a modulator (multiplied with the texture sample in the shader). If you remove the texture, the scalar value is used directly.
+
Controls how metallic a surface appears. 0.0 is fully dielectric (plastic, wood, etc.) and 1.0 is fully metallic.
++Like roughness, the scalar value modulates the metallic texture when one is present.
+
Controls self-illumination. The value is a simd_float3 (RGB) representing the emitted light color and intensity. A value of .zero means no emission.
++Spelling note: The API currently uses
+getMaterialEmmissive/updateMaterialEmmisive(with double-m). Use these exact names when calling the functions.
Determines how the renderer handles transparency for this material.
+MaterialAlphaMode).opaque — Fully opaque. Alpha channel is ignored..mask — Binary transparency. Pixels with alpha below the cutoff are discarded; the rest are fully opaque. Useful for foliage, fences, etc..blend — Smooth alpha blending. Pixels are composited based on their alpha value.Used only when the alpha mode is .mask. Pixels with alpha below this threshold are discarded. The value is clamped to 0.0 ... 1.0. Default is 0.5.
A convenience layer over the base color's alpha channel (.w). The value is clamped to 0.0 ... 1.0. Setting opacity below 1.0 automatically switches the alpha mode to .blend.
By default this applies to every submesh on the entity. To target a single submesh instead:
+ +Or specify exact indices:
+ +getMaterialBaseColor(entityId:meshIndex:submeshIndex:) → simd_float4updateMaterialColor(entityId:color:meshIndex:submeshIndex:) — sets base color from SwiftUI ColorgetMaterialRoughness(entityId:meshIndex:submeshIndex:) → FloatupdateMaterialRoughness(entityId:roughness:meshIndex:submeshIndex:)getMaterialMetallic(entityId:meshIndex:submeshIndex:) → FloatupdateMaterialMetallic(entityId:metallic:meshIndex:submeshIndex:)getMaterialEmmissive(entityId:meshIndex:submeshIndex:) → simd_float3updateMaterialEmmisive(entityId:emmissive:meshIndex:submeshIndex:)getMaterialAlphaMode(entityId:meshIndex:submeshIndex:) → MaterialAlphaModeupdateMaterialAlphaMode(entityId:mode:meshIndex:submeshIndex:)getMaterialAlphaCutoff(entityId:meshIndex:submeshIndex:) → FloatupdateMaterialAlphaCutoff(entityId:cutoff:meshIndex:submeshIndex:)getMaterialOpacity(entityId:meshIndex:submeshIndex:) → FloatupdateMaterialOpacity(entityId:opacity:applyToAllSubmeshes:)updateMaterialOpacity(entityId:opacity:meshIndex:submeshIndex:)The physics system in the Untold Engine enables realistic simulations such as gravity, forces, and dynamic interactions. While collision support is still under development, this guide will walk you through adding physics to your entities.
+Start by creating an entity that represents the object you want to add physics to.
+Next, load your model’s mesh file and link it to the entity. This step visually represents your entity in the scene.
+Activate the physics simulation for your entity using the setEntityKinetics function. This function prepares the entity for movement and dynamic interaction.
+You can customize the entity’s physics behavior by defining its mass and gravity scale:
+You can apply a custom force to the entity for dynamic movement. This is useful for simulating actions like jumps or pushes.
+ +++Note: Forces are applied per frame. To avoid unintended behavior, only apply forces when necessary.
+
For advanced movement behaviors, leverage the Steering System to steer entities toward or away from targets. This system automatically calculates the required forces.
+Example: Steering Toward a Position
+steerTo(entityId: redPlayer, targetPosition: simd_float3(0.0, 0.0, 5.0), maxSpeed: 2.0, deltaTime: deltaTime)
+The Steering System includes other useful behaviors, such as:
+These functions simplify complex movement patterns, making them easy to implement.
+Once you've set up physics, run the project to see it in action:
+The engine provides a post-processing system through the PostFX namespace. You can enable and configure individual effects, or apply a full preset that sets a curated combination of color grading and SSAO values in a single call.
The simplest way to set up post-effects is to apply one of the built-in presets:
+ +That single call configures color grading and SSAO together. No scene wiring or callback setup is needed — it works from anywhere in your game code.
+| Preset | +Description | +
|---|---|
.neutral |
+All effects disabled, default values restored | +
.cinematic |
+Slightly underexposed, desaturated, strong SSAO — moody interior feel | +
.highContrast |
+Boosted exposure and saturation, punchy SSAO — vivid outdoor scenes | +
.softAO |
+Subtle color grading, wide-radius soft ambient occlusion | +
.archviz |
+Bright and airy, clean slightly warm whites, precise SSAO for edge detail — architectural visualization | +
Presets can be swapped at any point during gameplay — for example when transitioning between areas:
+// Entering a dark dungeon
+PostFX.apply(.cinematic)
+
+// Entering a bright outdoor area
+PostFX.apply(.highContrast)
+
+// Reset everything to defaults
+PostFX.apply(.neutral)
+If the built-in presets do not fit your scene, create a PostFXPreset directly and apply it:
let sunset = PostFXPreset(
+ name: "Sunset",
+ colorGrading: true,
+ exposure: 0.1,
+ saturation: 1.3,
+ temperature: 0.4
+)
+
+PostFX.apply(sunset)
+All parameters have defaults (matching .neutral), so you only need to specify the values you want to change.
| Parameter | +Type | +Default | +Description | +
|---|---|---|---|
name |
+String |
+— | +Identifier for the preset | +
colorGrading |
+Bool |
+false |
+Enables color grading pass | +
exposure |
+Float |
+0.0 |
+EV adjustment (-2.0 to 2.0) | +
brightness |
+Float |
+0.0 |
+Additive brightness (-1.0 to 1.0) | +
contrast |
+Float |
+1.0 |
+Contrast multiplier (0.5 to 2.0) | +
saturation |
+Float |
+1.0 |
+Saturation multiplier (0.0 to 2.0) | +
temperature |
+Float |
+0.0 |
+Color temperature (-1.0 cool to +1.0 warm) | +
tint |
+Float |
+0.0 |
+Green/magenta tint (-1.0 to 1.0) | +
ssao |
+Bool |
+false |
+Enables screen-space ambient occlusion | +
ssaoRadius |
+Float |
+0.5 |
+Sample radius in world units (0.1 to 2.0) | +
ssaoBias |
+Float |
+0.025 |
+Self-occlusion bias (0.01 to 0.1) | +
ssaoIntensity |
+Float |
+0.0 |
+Final SSAO multiplier (0.5 to 2.0) | +
For fine-grained control outside of presets, you can enable or disable individual effects:
+PostFX.setEnabled(.colorGrading, true)
+PostFX.setEnabled(.vignette, true)
+PostFX.setEnabled(.chromaticAberration, false)
+And read their current state:
+ +| Effect | +Description | +
|---|---|
.colorGrading |
+Exposure, brightness, contrast, saturation, temperature, tint | +
.colorCorrection |
+Lift/gamma/gain per-channel color correction | +
.bloomThreshold |
+Bright-pass filter for bloom | +
.bloomComposite |
+Bloom blend pass | +
.vignette |
+Screen-edge darkening | +
.chromaticAberration |
+RGB channel fringing | +
.depthOfField |
+Focus blur | +
Each effect exposes its parameters through a shared singleton. Import UntoldEngine and write directly:
// Color grading
+ColorGradingParams.shared.exposure = -0.2
+ColorGradingParams.shared.contrast = 1.15
+ColorGradingParams.shared.saturation = 0.9
+ColorGradingParams.shared.temperature = -0.1
+
+// Bloom
+BloomThresholdParams.shared.threshold = 0.6
+BloomThresholdParams.shared.intensity = 0.8
+BloomThresholdParams.shared.enabled = true
+
+// Vignette
+VignetteParams.shared.intensity = 0.5
+VignetteParams.shared.radius = 0.8
+VignetteParams.shared.enabled = true
+
+// SSAO
+SSAOParams.shared.radius = 0.8
+SSAOParams.shared.intensity = 0.75
+SSAOParams.shared.enabled = true
+UntoldEngine profiling has two layers that are meant to be used together:
+Use structured metrics as the source of truth, then enable category logs only when you need extra context.
+Enable the profiler at runtime:
+ +Or via environment variable:
+ +Enable periodic frame stats logging:
+setEngineStatsLogging(
+ enabled: true,
+ profile: .compact, // or .verbose
+ intervalSeconds: 1.0
+)
+Read profiler snapshots programmatically:
+let metrics = EngineProfiler.shared.snapshot()
+print("CPU mean: \(metrics.cpuFrame.meanMs) ms")
+print("GPU mean: \(metrics.gpuCommandBuffer.meanMs) ms")
+
+let frameStats = getEngineStatsSnapshot()
+print("Frame: \(frameStats.frameIndex)")
+print("Update: \(frameStats.timing.updateMs) ms")
+print("Render: \(frameStats.timing.renderTotalMs) ms")
+High-volume instrumentation categories are disabled by default:
+OOCTimingOOCStatusAssetLoaderEnable them when diagnosing OOC/loader behavior:
+// Keep structured profiler metrics on
+enableEngineMetrics = true
+setEngineStatsLogging(enabled: true, profile: .compact, intervalSeconds: 1.0)
+
+// Add focused trace logs
+Logger.enable(category: .oocStatus) // OutOfCore lifecycle/status
+Logger.enable(category: .oocTiming) // OOC timing detail
+Logger.enable(category: .assetLoader) // progressive loader parse/upload
+Disable after capture:
+Logger.disable(category: .oocTiming)
+Logger.disable(category: .oocStatus)
+Logger.disable(category: .assetLoader)
+When metrics are enabled, the engine emits signpost scopes:
+FrameUpdateRenderPrepEncodeSubmitTo inspect timeline data:
+com.untoldengine.profilingenableEngineMetrics = true (or UNTOLD_METRICS=1)EngineProfiler is available in all configs, but disabled by default until enabled at runtime.EngineStats collection is compiled in debug by default (ENGINE_STATS_ENABLED).EngineStats, build with -DENGINE_STATS_ENABLED.If ENGINE_STATS_ENABLED is not compiled in, getEngineStatsSnapshot() returns default values and setEngineStatsLogging(...) is effectively a no-op.
Logger.log(...) debug/info trace paths.#if DEBUG
+let metricsLogger = MetricsDebugLogger()
+metricsLogger.logIfNeeded() // throttle-prints approximately once per second
+#endif
+Profiler hooks are already integrated into:
+UntoldEngine.swift (runFrame)RenderingSystem.swift (UpdateRenderingSystem)UntoldEngineXR.swift (executeXRSystemPass)UntoldEngineAR.swift (draw)The Registration System in the Untold Engine is an integral part of its Entity-Component-System (ECS) architecture. It provides core functionalities to manage entities and components, such as:
+Entities represent objects in the scene. Use the createEntity() function to create a new entity.
+ +Components define the behavior or attributes of an entity. Use registerComponent to add a component to an entity.
+ +Example: +When you load a mesh for rendering, the system automatically registers the required components:
+ +This function:
+.untold file.To remove an entity and its components from the scene, use destroyEntity.
+ +This ensures the entity is properly removed from all systems.
+Use destroyAllEntities(completion:) when you need to clear the world before loading new content.
destroyAllEntities {
+ // Safe point: pending destroys have been finalized.
+ // Load new content here (.untold, deserializeScene, etc).
+}
+Important behavior:
+destroyAllEntities is a deferred operation. Entities are marked for destroy first.finalizePendingDestroys()).completion block runs only after that finalization step has finished.This prevents race conditions where new entities are created while old entities are still pending destroy.
+Example: clear world, then load a new .untold asset
destroyAllEntities {
+ let entity = createEntity()
+ setEntityMeshAsync(entityId: entity, filename: "office", withExtension: "untold")
+}
+Example: playSceneAt pattern
public func playSceneAt(url: URL, completion: (() -> Void)? = nil) {
+ guard let scene = loadGameScene(from: url) else {
+ completion?()
+ return
+ }
+
+ destroyAllEntities {
+ deserializeScene(sceneData: scene) {
+ completion?()
+ }
+
+ // Early camera rebind during async mesh loading window.
+ CameraSystem.shared.activeCamera = findGameCamera()
+ }
+}
+The Rendering System in the Untold Engine is responsible for displaying your models on the screen. It supports advanced features such as Physically Based Rendering (PBR) for realistic visuals and multiple types of lights to illuminate your scenes.
+Start by creating an entity that represents your 3D object.
+To display a model, load its .untold runtime asset and link it to the entity using setEntityMesh.
Parameters:
+.untold file without the extension."untold" for runtime assets.++Note: If PBR textures (e.g., albedo, normal, roughness, metallic maps) are included, the rendering system will automatically use the appropriate PBR shader to render the model with realistic lighting and material properties.
+
Once everything is set up:
+.untold asset references the correct PBR textures, and verify their paths during the loading process.The Untold Engine includes a Scene Graph data structure, designed to manage hierarchical transformations efficiently. This enables parent-child relationships between entities, where a child's transformation (position, rotation, scale) is relative to its parent. For example, a car's wheels (children) move and rotate relative to the car body (parent).
+Parent-child relationships are useful when you want multiple entities to move or transform together. When a parent entity changes its position, rotation, or scale, its child entities inherit those changes automatically. This is ideal for scenarios like:
+To assign a parent to an entity, use the setParent function. This function establishes a hierarchical relationship between the specified entities.
+// Create child and parent entities
+let childEntity = createEntity()
+let parentEntity = createEntity()
+
+// Set parent-child relationship
+setParent(childId: childEntity, parentId: parentEntity)
+The child’s transformation is expressed relative to the parent.
+Independent Local Transformations:
+Spatial input in Untold Engine follows a simple pipeline:
+That separation keeps the system flexible: the OS-facing code stays in UntoldEngineXR, while gesture classification stays in +the recognizer.
+From XRSpatialInputState, you can read:
+So your game logic can stay focused on behavior (select, move, rotate, scale), not event parsing.
+You must enable XR event ingestion:
+InputSystem.shared.registerXREvents()
+If you skip this, the callback still receives OS events, but the engine ignores them.
+In your handleInput():
+For object manipulation, use SpatialManipulationSystem for robust pinch-driven transforms, then layer custom behavior on top +when needed.
+This example shows how to drag and rotate a mesh using the engine:
+func handleInput() {
+ if gameMode == false { return }
+
+ let state = InputSystem.shared.xrSpatialInputState
+
+ if state.spatialTapActive, let entityId = state.pickedEntityId {
+ Logger.log(message: "Tapped entity: \(entityId)")
+ }
+
+ // Handles drag-based translate + twist rotation on picked entity
+ SpatialManipulationSystem.shared.processPinchTransformLifecycle(from: state)
+}
+processPinchTransformLifecycle handles:
This lifecycle model prevents stuck manipulation sessions.
+If ray picking hits a child mesh and you want to manipulate the parent +actor:
+var state = InputSystem.shared.xrSpatialInputState
+
+if let picked = state.pickedEntityId,
+ let parent = getEntityParent(entityId: picked) {
+ state.pickedEntityId = parent
+}
+
+SpatialManipulationSystem.shared.processPinchTransformLifecycle(from: state)
+This is useful when:
+Do not early-return only because pickedEntityId == nil before calling
+lifecycle processing.
End/cancel phases must still propagate to properly close manipulation +sessions.\ +Failing to do so can leave the engine in an inconsistent transform +state.
+Use these APIs to control whether an entity can be selected by spatial tap/ray picking and what hit representation it uses.
+setEntityPickParticipation(entityId: entityId, enabled: false) // visible, not pickable
+setEntityPickHitRepresentationMode(entityId: entityId, mode: .bounds) // pick using bounds
+setEntityPickHitRepresentationMode(entityId: entityId, mode: .mesh) // pick using mesh (default)
+Available APIs:
+setEntityPickParticipation(entityId:enabled:)getEntityPickParticipation(entityId:)setEntityPickHitRepresentationMode(entityId:mode:)getEntityPickHitRepresentationMode(entityId:)Hit representation modes:
+.none\
+ Never pickable..bounds\
+ Pick using bounds intersection..mesh\
+ Pick using mesh-capable path (default behavior).Behavior rules:
+.mesh.enabled == false means the entity is never returned by picking, regardless of mode.mode == .none also means the entity is never returned by picking.It is strongly recommended to use the Spatial Helper functions instead +of raw gesture access.
+Raw access is useful when:
+Vision Pro air-tap gesture.
+let state = InputSystem.shared.xrSpatialInputState
+if state.spatialTapActive, let entityId = state.pickedEntityId {
+ // selectEntity(entityId)
+}
+Use this to:
+Single-hand pinch detected.
+ +This does not imply dragging yet --- only that a pinch is currently +held.
+World-space position of pinch.
+ +Useful for:
+Drag delta while pinch is active.
+let state = InputSystem.shared.xrSpatialInputState
+if state.spatialPinchActive {
+ let dragDelta = InputSystem.shared.getPinchDragDelta()
+ // app-defined translation/scaling response
+}
+Common use cases:
+For stable translation (no per-frame delta accumulation), use the +anchored lifecycle helper:
+func handleInput() {
+ let state = InputSystem.shared.xrSpatialInputState
+
+ SpatialManipulationSystem.shared.processAnchoredPinchDragLifecycle(
+ from: state,
+ entityId: sceneRootEntity
+ )
+}
+This helper:
+Use this when moving large roots (buildings/scenes) where incremental +delta jitter can become visible.
+For translating the entire scene root (rather than a single entity), use the anchored scene drag lifecycle:
+func handleInput() {
+ let state = InputSystem.shared.xrSpatialInputState
+
+ SpatialManipulationSystem.shared.processAnchoredSceneDragLifecycle(from: state)
+}
+This helper:
+translateSceneTo, keeping static batches intactYou can adjust movement speed with the sensitivity parameter (defaults to 1.0):
To manually end the drag (e.g. on a mode change), call:
+ +Use this when panning an entire scene — for example, sliding a map, architectural model, or level layout in world space.
+For rotating the entire scene root around world up (+Y) while preserving static batching, use the anchored scene rotate lifecycle. This requires a two-hand pinch + twist gesture (spatialRotateActive with both hands pinching):
func handleInput() {
+ let state = InputSystem.shared.xrSpatialInputState
+
+ SpatialManipulationSystem.shared.processAnchoredSceneRotateLifecycle(from: state)
+}
+This helper:
+rotateSceneToYaw, keeping static batches intactYou can adjust rotation speed with the sensitivity parameter (defaults to 1.0):
SpatialManipulationSystem.shared.processAnchoredSceneRotateLifecycle(from: state, sensitivity: 0.5)
+To manually end rotation (e.g. on a mode change), call:
+ +Use this when aligning or calibrating an already-loaded large scene in place without rebatching.
+To avoid drag/rotate gesture fighting, use the unified scene-root manipulation lifecycle:
+func handleInput() {
+ let state = InputSystem.shared.xrSpatialInputState
+
+ SpatialManipulationSystem.shared.processAnchoredSceneManipulationLifecycle(
+ from: state,
+ dragSensitivity: 1.0,
+ rotateSensitivity: 0.5
+ )
+}
+Arbitration rules:
+manipulationClassificationFrames, default 3) so the second hand has time to arrivespatialRotateActive + both hands pinching) routes to scene rotatedrag or rotate) until the gesture ends/release happensYou can tune the deferral window (set to 0 to commit immediately):
+ +To manually end the unified lifecycle (e.g. on a mode change), call:
+ +Use this as the default scene-root helper when your app supports both panning and rotation.
+All three scene-level gestures can live in the same input loop — they gate on different input conditions so they don't conflict:
+func handleInput() {
+ let state = InputSystem.shared.xrSpatialInputState
+
+ // Single-hand pinch + drag → pan the scene
+ SpatialManipulationSystem.shared.processAnchoredSceneDragLifecycle(from: state)
+
+ // Two-hand pinch + twist → rotate the scene (yaw)
+ SpatialManipulationSystem.shared.processAnchoredSceneRotateLifecycle(from: state)
+
+ // Two-hand pinch + spread/pinch → zoom an entity
+ SpatialManipulationSystem.shared.applyTwoHandZoomIfNeeded(from: state)
+}
+For context-based entity vs. scene rotation — route two-hand twist to entity rotate when something is picked, and to scene rotate otherwise:
+func handleInput() {
+ let state = InputSystem.shared.xrSpatialInputState
+
+ // Scene-level drag (always active)
+ SpatialManipulationSystem.shared.processAnchoredSceneDragLifecycle(from: state)
+
+ if state.pickedEntityId != nil {
+ // Entity is picked → two-hand twist rotates the entity
+ SpatialManipulationSystem.shared.applyTwoHandRotateIfNeeded(from: state)
+ } else {
+ // Nothing picked → two-hand twist rotates the scene
+ SpatialManipulationSystem.shared.processAnchoredSceneRotateLifecycle(from: state)
+ }
+
+ SpatialManipulationSystem.shared.applyTwoHandZoomIfNeeded(from: state)
+}
+Apply the built-in zoom response:
+let state = InputSystem.shared.xrSpatialInputState
+
+SpatialManipulationSystem.shared.applyTwoHandZoomIfNeeded(
+ from: state,
+ sensitivity: 1.0
+)
+By default, the helper scales the parent of the picked entity when available.
+If you want to choose the exact target, pass entityId:
let state = InputSystem.shared.xrSpatialInputState
+
+if let picked = state.pickedEntityId {
+ // Scale exactly what was hit
+ SpatialManipulationSystem.shared.applyTwoHandZoomIfNeeded(
+ from: state,
+ entityId: picked,
+ sensitivity: 1.0
+ )
+
+ // Or scale its parent explicitly
+ if let parent = getEntityParent(entityId: picked) {
+ SpatialManipulationSystem.shared.applyTwoHandZoomIfNeeded(
+ from: state,
+ entityId: parent,
+ sensitivity: 1.0
+ )
+ }
+}
+Use setXRTwoHandRotateAxisMode to control how the rotation axis is derived:
Available modes:
+.cameraForward: rotates around camera-forward axis (screen-style twist).dynamic: derives axis from actual two-hand motion.dynamicSnapped: dynamic axis snapped to dominant world axis (x, y, or z)Apply the built-in rotate response:
+let state = InputSystem.shared.xrSpatialInputState
+
+SpatialManipulationSystem.shared.applyTwoHandRotateIfNeeded(
+ from: state,
+ sensitivity: 1.5
+)
+By default, the helper rotates the parent of the picked entity when available.
+If you want to choose the exact target, pass entityId:
let state = InputSystem.shared.xrSpatialInputState
+
+if let picked = state.pickedEntityId {
+ // Rotate exactly what was hit
+ SpatialManipulationSystem.shared.applyTwoHandRotateIfNeeded(
+ from: state,
+ entityId: picked,
+ sensitivity: 1.5
+ )
+
+ // Or rotate its parent explicitly
+ if let parent = getEntityParent(entityId: picked) {
+ SpatialManipulationSystem.shared.applyTwoHandRotateIfNeeded(
+ from: state,
+ entityId: parent,
+ sensitivity: 1.5
+ )
+ }
+}
+To get the distance to an entity use the following:
+// Get distance to hit-entity
+let state = InputSystem.shared.xrSpatialInputState
+if state.spatialTapActive, let entityId = state.pickedEntityId {
+ // get distance
+ let distance = state.pickedEntityDistance
+ print("Object distance: \(distance) meters")
+}
+To retrieve the exact world-space position where the user taps on the ground, use pickRealSurfacePosition. This is useful for calibration workflows where you need to anchor a point on the ground and scale a model relative to it.
let state = InputSystem.shared.xrSpatialInputState
+
+if state.spatialTapActive{
+ if let hit = pickRealSurfacePosition(
+ rayOrigin: state.rayOriginWorld,
+ rayDirection: state.rayDirectionWorld,
+ filter: .horizontalAny
+ ) {
+ Logger.log(message: "Surface type: \(hit.surfaceKind)", vector: hit.worldPosition)
+ }
+}
+Use these helpers from SpatialManipulationSystem.shared:
processPinchTransformLifecycle(from:)\
+ Recommended default. Handles translation + twist rotation lifecycle
+ safely.
applyPinchDragIfNeeded(from:entityId:sensitivity:)\
+ Lower-level translation helper if you want full control.
processAnchoredSceneDragLifecycle(from:sensitivity:)\
+ Anchored drag for the entire scene root. Applies absolute
+ displacement via translateSceneTo.
endAnchoredSceneDrag()\
+ Manually ends an in-progress anchored scene drag session.
processAnchoredSceneRotateLifecycle(from:sensitivity:)\
+ Anchored rotate for the entire scene root using two-hand pinch + twist.
+ Applies absolute yaw via rotateSceneToYaw.
endAnchoredSceneRotate()\
+ Manually ends an in-progress anchored scene rotate session.
processAnchoredSceneManipulationLifecycle(from:dragSensitivity:rotateSensitivity:)\
+ Unified scene-root helper with drag/rotate arbitration to prevent
+ gesture-fighting. Uses a deferral window (manipulationClassificationFrames) before
+ committing to drag so the second hand has time to arrive for rotate.
endAnchoredSceneManipulation()\
+ Ends any in-progress unified scene manipulation (drag, rotate, or pending classification).
applyTwoHandZoomIfNeeded(from:sensitivity:)\
+ Provides zoom delta signal. You must define what zoom means in your
+ app.
UntoldEngine supports two batching modes in practice:
+| Mode | +Use for | +
|---|---|
| Manual batch generation | +Always-resident static content | +
| Runtime cell-based batching | +Tiled streaming scenes | +
Mark loaded entities as static, enable batching, then build the initial artifacts.
+let cube1 = createEntity()
+setEntityMesh(entityId: cube1, filename: "cube", withExtension: "untold")
+translateTo(entityId: cube1, position: simd_float3(0, 0, 0))
+setEntityStaticBatchComponent(entityId: cube1)
+
+let cube2 = createEntity()
+setEntityMesh(entityId: cube2, filename: "cube", withExtension: "untold")
+translateTo(entityId: cube2, position: simd_float3(2, 0, 0))
+setEntityStaticBatchComponent(entityId: cube2)
+
+enableBatching(true)
+generateBatches()
+For async loading, mark entities static in the completion block:
+let building = createEntity()
+
+setEntityMeshAsync(entityId: building, filename: "office_building", withExtension: "untold") { success in
+ guard success else { return }
+ setEntityStaticBatchComponent(entityId: building)
+ enableBatching(true)
+ generateBatches()
+}
+For tiled scenes, the flow is different:
+let sceneRoot = createEntity()
+setEntityName(entityId: sceneRoot, name: "city")
+
+setEntityStreamScene(entityId: sceneRoot, manifest: "city", withExtension: "json") { success in
+ setSceneReady(success)
+}
+In this mode:
+registerTiledScene(...) enables batching automaticallynotifyTileEntitiesResident(_:)You do not call generateBatches() every time a tile loads. The batching system rebuilds dirty cells incrementally based on residency changes.
For tiled/streamed scenes, the engine manages static batching automatically. When a tile finishes loading, the engine assigns a StaticBatchComponent to all of its entities and schedules an incremental batch rebuild for only the spatial cells affected by that tile. This happens internally on a background queue via the engine's tick() loop.
Do not call generateBatches() for streamed scenes. That function performs a full global rebuild — it queries every entity in the scene simultaneously, merges entities from different tiles into shared batch groups, and allocates all GPU buffers synchronously on the render thread. This overrides the engine's incremental system and causes a noticeable stall.
For streamed scenes, only call enableBatching(true) after the scene loads. The engine handles the rest:
setEntityStreamScene(entityId: sceneRoot, manifest: "city", withExtension: "json") { success in
+ enableBatching(true)
+ setSceneReady(success)
+}
+For non-streamed scenes (single .untold), call setEntityStaticBatchComponent, generateBatches(), and enableBatching(true) as normal. The same applies to any operation that mutates material state (color, opacity) — wrap it with enableBatching(false) before and generateBatches() + enableBatching(true) after, but only for non-streamed scenes:
// Non-streamed only — do not use this pattern in tiled/streamed scenes
+enableBatching(false)
+setEntityColor(entityId: prop, color: simd_float4(1, 0, 0, 1))
+generateBatches()
+enableBatching(true)
+setEntityStaticBatchComponent(entityId:)Marks an entity hierarchy as eligible for batching.
+ +removeEntityStaticBatchComponent(entityId:)Removes static batching tags from the entity hierarchy.
+ +enableBatching(_:)Globally enables or disables runtime batching.
+ +generateBatches()Builds batch artifacts for the currently marked static entities. This is mainly for always-resident/manual workflows.
+ +clearSceneBatches()Clears all generated batch artifacts.
+ +TileLODTagComponent lets batching treat per-tile LODs and HLODs as distinct LOD groups even though they are not entity-level LODComponent assets.For architectural details, see Batching System.
+ + + + + + + + + + + + + +The Steering System in the Untold Engine enables entities to move dynamically and intelligently within the scene. It provides both low-level steering behaviors (e.g., seek, flee, arrive) for granular control and high-level behaviors (e.g., steerTo, steerAway, followPath) that integrate seamlessly with the Physics System.
+The Steering System is essential for creating dynamic and realistic movement for entities, such as:
+The high-level behaviors are recommended because they are designed to work closely with the Physics System, simplifying implementation while maintaining smooth motion.
+Examples:
+steerAvoidObstacles(entityId: entity, obstacles: obstacleEntities, avoidanceRadius: 2.0, maxSpeed: 5.0, deltaTime: 0.016)
+UntoldEngine ships two user-facing exporter commands in the scripts/ folder at the repo root:
export-untoldexport-untold-tilesThese wrappers launch Blender in background mode and run the Python exporters for you. Users should run the shell wrappers, not the raw Blender commands.
+Blender must be installed.
+The wrappers resolve Blender in this order:
+--blender /path/to/BlenderBLENDER_BIN=/path/to/Blender/Applications/Blender.app/Contents/MacOS/Blenderblender on PATHIf Blender cannot be found, the wrapper prints an install message and exits.
+Use export-untold (found in scripts/) to convert one USD or USDZ asset into one .untold runtime file.
Basic usage:
+ +Common options:
+--input <path>: required source .usd, .usda, .usdc, or .usdz--output <path>: required destination .untold--file-type <tile|lod|hlod|shared>: optional, defaults to tile--mesh-name <name>: optional, export only one mesh from a multi-mesh asset--ConvertOrientation: optional, convert the export into engine space--source-orientation <blender-native|engine-oriented>: optional, defaults to blender-native--validate: optional, also writes <name>.validation.json--blender <path>: optional wrapper-level Blender overrideExample:
+./scripts/export-untold \
+ --input GameData/Models/robot/robot.usdz \
+ --output GameData/Models/robot/robot.untold \
+ --ConvertOrientation \
+ --source-orientation blender-native \
+ --validate
+Expected output:
+robot.untoldTextures/... beside the .untold file if the asset uses texturesrobot.validation.json only when --validate is passedUse export-untold-tiles (found in scripts/) to partition a USD or USDZ scene into tile payloads and generate a manifest JSON file.
Basic usage:
+./scripts/export-untold-tiles \
+ --input /path/scene.usdz \
+ --output-dir /path/tile_exports \
+ --tile-size-x 25 \
+ --tile-size-y 10000 \
+ --tile-size-z 25
+Common options:
+--input <path>: required source .usd, .usda, .usdc, or .usdz--output-dir <path>: required destination directory for tile payloads--tile-size-x <number>: optional tile width in world units--tile-size-y <number>: optional tile height in world units, defaults to 10000--tile-size-z <number>: optional tile depth in world units--auto-tile-size: optional automatic tile sizing--generate-hlod: optional HLOD generation--generate-lod: optional per-tile LOD generation--dry-run: optional planning pass without writing payload files--write-manifest-in-dry-run: optional manifest write during dry run--visible-only: optional export only visible meshes--all-meshes: optional include hidden meshes--debug-aabb-only: optional emit debug AABB payloads instead of geometry--quadtree: optional partition tiles using a quad-tree instead of a uniform grid--floor-count <number>: optional number of vertical floors to split each tile into--blender <path>: optional wrapper-level Blender overrideExample:
+./scripts/export-untold-tiles \
+ --input GameData/Models/dungeon/dungeon.usdz \
+ --output-dir GameData/Models/dungeon/tile_exports \
+ --tile-size-x 25 \
+ --tile-size-y 10000 \
+ --tile-size-z 25 \
+ --generate-hlod \
+ --generate-lod
+Dry-run example:
+./scripts/export-untold-tiles \
+ --input GameData/Models/dungeon/dungeon.usdz \
+ --output-dir GameData/Models/dungeon/tile_exports \
+ --tile-size-x 25 \
+ --tile-size-y 10000 \
+ --tile-size-z 25 \
+ --dry-run \
+ --write-manifest-in-dry-run
+Expected output layout:
+dungeon.json beside the tile payload directorytile_exports/tile_*.untold.untold files in tile_exports/tile_exports/Textures/... for staged texturesThe manifest stores relative runtime paths so it remains portable across machines, repos, and app bundles.
+After exporting assets, use Optimizations for optional +workflows such as ASTC texture compression and LZ4 geometry compression.
+Single asset:
+ +Tiled scene:
+let sceneRoot = createEntity()
+setEntityName(entityId: sceneRoot, name: "dungeon")
+setEntityStreamScene(entityId: sceneRoot, manifest: "dungeon", withExtension: "json")
+The manifest should live next to the tile payload directory. Tile, HLOD, LOD, and shared-bucket payloads are resolved relative to the manifest file.
+.untold tile payloads participate in the current tiled streaming architecture, including tile-level load/unload, remote download + cache, per-tile LOD/HLOD, and large-tile OCC sub-mesh streaming when the runtime classifies a tile into the OOC path.scripts/ are implementation details. The recommended user entry points are the shell wrappers in the same folder.UntoldEngine includes a thread-safe logger with log-level filtering, per-category toggles, and a sink API for routing log events to custom destinations (e.g. an in-editor console).
+Log level controls the minimum severity that is emitted. Set it once at startup:
+Logger.logLevel = .debug // emit everything
+Logger.logLevel = .info // emit info, warnings, and errors
+Logger.logLevel = .warning // emit warnings and errors only
+Logger.logLevel = .error // emit errors only
+Logger.logLevel = .none // suppress all output
+| Level | +Value | +What emits | +
|---|---|---|
.none |
+0 | +nothing | +
.error |
+1 | +errors | +
.warning |
+2 | +warnings + errors | +
.info |
+3 | +info + warnings + errors | +
.debug |
+4 | +everything | +
.test |
+5 | +everything (used in unit tests) | +
The default level is .debug.
Requires logLevel >= .info. Suppressed if the category is disabled.
Requires logLevel >= .warning. Always emits regardless of category state.
Logger.logError(message: "Failed to load texture: \(name)", category: LogCategory.general.rawValue)
+Requires logLevel >= .error. Always emits regardless of category state.
++Note: Messages are lazily evaluated (
+@autoclosure), so string interpolation cost is skipped when the log would be suppressed.
Categories let you silence or focus specific subsystems without changing the global log level.
+| Category | +Raw value | +Default state | +
|---|---|---|
.general |
+"General" |
+enabled | +
.ecs |
+"ECS" |
+enabled | +
.engineStats |
+"EngineStats" |
+enabled | +
.integration |
+"Integration" |
+enabled | +
.xrCamera |
+"XRCamera" |
+disabled | +
.oocTiming |
+"OOCTiming" |
+disabled | +
.oocStatus |
+"OOCStatus" |
+disabled | +
.assetLoader |
+"AssetLoader" |
+disabled | +
High-volume categories (xrCamera, oocTiming, oocStatus, assetLoader) are off by default to avoid log spam during normal operation.
// Enable a category
+Logger.enable(category: .oocStatus)
+
+// Disable a category
+Logger.disable(category: .xrCamera)
+
+// Toggle with a Bool
+Logger.set(category: .assetLoader, enabled: true)
+
+// Check current state
+if Logger.isEnabled(category: .ecs) { ... }
+
+// Reset all overrides back to defaults
+Logger.resetCategoryToggles()
+// Turn on verbose streaming traces for a debug session
+Logger.enable(category: .oocStatus)
+Logger.enable(category: .oocTiming)
+Logger.enable(category: .assetLoader)
+
+// ... reproduce the issue ...
+
+// Clean up after capture
+Logger.disable(category: .oocStatus)
+Logger.disable(category: .oocTiming)
+Logger.disable(category: .assetLoader)
+Implement LoggerSink to route events to a custom destination such as an editor console or file:
final class ConsoleSink: LoggerSink {
+ func didLog(_ event: LogEvent) {
+ print("[\(event.category)] \(event.message)")
+ }
+}
+
+let sink = ConsoleSink()
+Logger.addSink(sink)
+LogEvent exposes level, message, category, file, function, line, and timestamp.
Sinks are held weakly — the logger will not extend their lifetime.
+++Sink delivery and backlog replay are available on macOS (
+AppKit) builds only.
Logger.log(...) respects both logLevel and category state.Logger.logWarning(...) and Logger.logError(...) respect logLevel only — they are never suppressed by category.resetCategoryToggles() to restore defaults without restarting.The Transform System is a core part of the Untold Engine, responsible for managing the position, rotation, and scale of entities. It provides both local transformations (relative to a parent entity) and world transformations (absolute in the scene).
+You can retrieve an entity’s position, orientation, or axis vectors using the provided functions.
+Retrieves the entity’s position relative to its parent.
+ +Retrieves the entity’s absolute position in the scene.
+ +Retrieves the entity’s orientation matrix relative to its parent.
+ +Retrieves the entity’s absolute orientation matrix.
+ +Retrieve the entity’s forward, right, or up axis:
+let forward = getForwardAxisVector(entityId: entity)
+let right = getRightAxisVector(entityId: entity)
+let up = getUpAxisVector(entityId: entity)
+Modify an entity’s transform by translating or rotating it.
+Move the entity to a new position:
+ +Move the entity by an offset relative to its current position:
+ +Rotate the entity to a specific angle around an axis:
+ +Apply an incremental rotation to the entity:
+ +Directly set the entity’s rotation matrix:
+ +These functions move the scene root without modifying individual entity transforms. Because per-entity transforms stay untouched, static batches remain intact and no rebatching is needed.
+Use these when you need to pan or reposition the entire world — for example, sliding a map or architectural model during a spatial drag gesture.
+These functions rotate the scene root around world up (+Y) without modifying individual entity transforms. Static batches remain intact and no rebatching is required.
Use these when you need to align or calibrate a large loaded scene in-place (for example, Vision Pro room alignment) while keeping batching and culling efficient.
+Use these helpers when converting between scene-local/entity space and visual world space:
+let visual = sceneLocalToVisualWorld(simd_float3(1, 0, 0))
+let local = visualWorldToSceneLocal(visual)
+To get an entity's visual world position (with scene-root transform applied):
+ +To return scene root to identity quickly:
+ +This resets position/rotation/scale and refreshes root matrices immediately.
+translateSceneTo / translateSceneBy instead of moving every entity individually.rotateSceneToYaw / rotateSceneByYaw to rotate large scenes around world up.The untoldengine-create CLI tool scaffolds ready-to-run Xcode projects with UntoldEngine pre-configured. Instead of setting up package dependencies and boilerplate by hand, you run one command and get a fully wired project for your target platform.
The install script (scripts/install-untoldengine-create.sh) builds the CLI from source and places it in /usr/local/bin so it is available globally in your shell.
Clone the repository and run the install script from the repo root:
+git clone https://github.com/untoldengine/UntoldEngine.git
+cd UntoldEngine
+./scripts/install-untoldengine-create.sh
+The script will:
+untoldengine-create in release mode using Swift Package Manager./usr/local/bin (prompts for admin privileges if needed).PATH.If the final verification step warns that untoldengine-create is not found in PATH, add /usr/local/bin to your shell profile:
Then reload your shell:
+ +Run from the parent directory — the CLI creates the project folder for you:
+ +| Flag | +Target | +
|---|---|
--platform macos |
+macOS (default) | +
--platform ios |
+iOS | +
--platform ios-ar |
+iOS with ARKit | +
--platform visionos |
+visionOS / Apple Vision Pro | +
--platform multi |
+macOS + iOS + visionOS | +
# macOS project (default)
+untoldengine-create create MyGame --platform macos
+
+# iOS project
+untoldengine-create create MyGame --platform ios --bundle-id com.company.mygame
+
+# iOS with ARKit
+untoldengine-create create ARGame --platform ios-ar --bundle-id com.company.argame
+
+# visionOS / Apple Vision Pro
+untoldengine-create create VisionGame --platform visionos
+
+# Multi-platform (macOS, iOS, visionOS) — Team ID required for signing
+untoldengine-create create CrossGame --platform multi --team-id ABCD1234EF
+| Option | +Description | +Default | +
|---|---|---|
--platform |
+Target platform | +macos |
+
--bundle-id |
+Bundle identifier | +— | +
--output |
+Output directory | +current directory | +
--macos-version |
+macOS deployment target (13, 14, 15) |
+15 |
+
--ios-version |
+iOS deployment target (16, 17, 18) |
+17 |
+
--visionos-version |
+visionOS deployment target (1, 2) |
+2 |
+
--team-id |
+Apple Developer Team ID | +— | +
--optimization |
+Optimization level (none, speed, size) |
+none |
+
--debug / --no-debug |
+Include debug information | +yes | +
The update command refreshes only the GameData folder in an existing project, leaving your custom code untouched:
untoldengine-create update MyGame --asset-path ~/GameAssets
+
+# Or point to an absolute project path
+untoldengine-create update ~/Projects/MyGame --asset-path ~/GameAssets
+MyGame/
+├── Package.swift
+├── README.md
+└── Sources/
+ └── MyGame/
+ ├── AppDelegate.swift
+ ├── GameScene.swift
+ ├── GameViewController.swift
+ ├── Base.lproj/
+ │ └── Main.storyboard
+ ├── Info.plist
+ └── GameData/
+ ├── Scenes/
+ ├── Scripts/
+ ├── Models/
+ ├── Textures/
+ └── Shaders/
+The starter GameScene.swift shows how to load a mesh, enable geometry streaming, and enable static batching.
++Note: The demo references
+city.usdz. Place that file inGameData/Models/before running.
The generated Package.swift pulls in only the engine modules needed for your platform:
| Platform | +Engine modules | +
|---|---|
macos / ios |
+UntoldEngine |
+
ios-ar |
+UntoldEngineAR |
+
visionos |
+UntoldEngineXR + UntoldEngineAR |
+
multi |
+UntoldEngine + UntoldEngineXR + UntoldEngineAR |
+
After create finishes, open the generated Xcode project:
Select your scheme and press Run.
+ + + + + + + + + + + + + +.untold).untold is the native runtime asset container for UntoldEngine tile streaming.
It is not an interchange format and it is not intended to preserve full USD
+semantics. USD/USDZ remains the authoring/import format. The .untold container is the
+runtime package consumed by the engine for:
V1 is intentionally narrow:
+The design goals are:
+asset_remote_streaming.md)Float32UInt32.maxFloat32s, column-major, matching simd_float4x4min.xyz followed by max.xyzUInt64 offsets from the start of the fileUInt64 offsets from the start of the uncompressed chunk payloadThe Swift types in
+Sources/UntoldEngine/AssetFormat/UntoldFormat.swift
+are the logical schema. They are not the authoritative on-disk memory layout.
+The exporter and loader must write/read fields explicitly.
Each .untold file is laid out as:
FileHeaderChunkTableRecommended chunk payload order:
+STRING_TABLEENTITY_TABLEMESH_TABLEMATERIAL_TABLETEXTURE_TABLEVERTEX_DATAINDEX_DATAThis order allows the runtime to read metadata first and defer heavy geometry reads.
+The header is written field-by-field in this exact order:
+magic[8] UInt8 x 8
+formatVersion UInt32
+fileType UInt32
+flags UInt32
+headerSize UInt32
+chunkCount UInt32
+meshCount UInt32
+materialCount UInt32
+textureRefCount UInt32
+entityCount UInt32
+vertexLayout UInt32
+reserved0 UInt32
+worldBounds.min Float32 x 3
+worldBounds.max Float32 x 3
+rootTransform Float32 x 16
+contentHash UInt8 x 32
+reserved1 UInt8 x 32
+Rules:
+magic must be exactly 8 bytes, recommended value: "UNTOLD\0\0"headerSize is the serialized byte size of the header, not MemoryLayoutcontentHash is exactly 32 bytesreserved1 is exactly 32 bytesEach chunk entry is written in this exact order:
+chunkType UInt32
+compressionType UInt32
+fileOffset UInt64
+compressedSize UInt64
+uncompressedSize UInt64
+elementCount UInt32
+reserved0 UInt32
+Rules:
+fileOffset must be 16-byte alignednone, compressedSize == uncompressedSizeelementCount is used for record-table chunksThe string table payload is a raw byte blob containing:
+0x00 terminator after each stringRules:
+UInt32 byte offsets into this chunkUInt32.max means “no string”Each entity record is serialized in this exact order:
+entityId UInt32
+parentEntityId UInt32
+nameOffset UInt32
+firstMeshRecordIndex UInt32
+meshRecordCount UInt32
+flags UInt32
+localBounds.min Float32 x 3
+localBounds.max Float32 x 3
+worldBounds.min Float32 x 3
+worldBounds.max Float32 x 3
+localTransform Float32 x 16
+Rules:
+parentEntityId == UInt32.max means no parentfirstMeshRecordIndex + meshRecordCount must stay within mesh table boundsEach mesh record is serialized in this exact order:
+entityId UInt32
+meshNameOffset UInt32
+materialIndex UInt32
+indexType UInt32
+vertexCount UInt32
+indexCount UInt32
+vertexStrideBytes UInt32
+flags UInt32
+vertexDataOffset UInt64
+indexDataOffset UInt64
+vertexDataSizeBytes UInt64
+indexDataSizeBytes UInt64
+estimatedGPUBytes UInt64
+reserved0 UInt64
+localBounds.min Float32 x 3
+localBounds.max Float32 x 3
+Rules:
+vertexDataOffset is relative to the start of VERTEX_DATAindexDataOffset is relative to the start of INDEX_DATAvertexDataSizeBytes == vertexCount * vertexStrideBytesindexDataSizeBytes == indexCount * indexElementSizematerialIndex == UInt32.max means no materialEach material record is serialized in this exact order:
+nameOffset UInt32
+flags UInt32
+baseColorFactor Float32 x 4
+emissiveFactor Float32 x 3
+normalScale Float32
+metallicFactor Float32
+roughnessFactor Float32
+occlusionStrength Float32
+alphaCutoff Float32
+baseColorTextureIndex UInt32
+normalTextureIndex UInt32
+metallicTextureIndex UInt32
+roughnessTextureIndex UInt32
+emissiveTextureIndex UInt32
+occlusionTextureIndex UInt32
+reserved0 UInt32 x 2
+Rules:
+TEXTURE_TABLEUInt32.maxflags holds alpha mode, double-sided, transparent, and similar runtime bitsEach texture record is serialized in this exact order:
+nameOffset UInt32
+uriOffset UInt32
+textureFormat UInt32
+flags UInt32
+width UInt32
+height UInt32
+mipCount UInt32
+reserved0 UInt32
+Rules:
+uriOffset points into the string tabletextureFormat describes the cooked/runtime texture formatV1 supports one layout only:
+UNT_VERTEX_LAYOUT_PBR_STATIC_V1Serialized layout:
+position.x Float32
+position.y Float32
+position.z Float32
+normalPacked UInt32
+tangentPacked UInt32
+uv0.u UInt16
+uv0.v UInt16
+uv1.u UInt16
+uv1.v UInt16
+color0.r UInt8
+color0.g UInt8
+color0.b UInt8
+color0.a UInt8
+Total size: 32 bytes
+Rules:
+uv0 is requireduv1 may be zeroed if unusedcolor0 defaults to 255,255,255,255 if unusednormalPacked and tangentPacked use signed normalized 10:10:10:2 packing.
For normalPacked:
For tangentPacked:
Recommended tangent handedness mapping:
++1 for non-negative handedness-1 for negative handednessExporter rules:
+[-1, 1]normalPacked.w = 0tangentPacked.w = +1 or -1Runtime rule:
+cross(normal, tangent.xyz) * tangentSignindexType = uint16 means 2 bytes per indexindexType = uint32 means 4 bytes per indexuint16 when vertexCount <= 65535Supported compression types:
+nonelz4zstdRules:
+The loader must reject files when:
+vertexStrideBytes does not match the declared vertex layoutindexDataSizeBytes does not match indexCount * indexElementSizeDo not serialize .untold files using MemoryLayout<T> or direct struct dumps.
Both the exporter and the loader must use explicit field-by-field helpers:
+writeUInt32LEwriteUInt64LEwriteFloat32LEwriteBytesreadUInt32LEreadUInt64LEreadFloat32LEreadBytesThis keeps the binary stable even if the Swift type layout changes.
+ + + + + + + + + + + + + +AssetProfiler is a lightweight classification layer that runs between Mesh.parseAssetAsync() and ECS entity creation. It analyzes a parsed asset's composition — geometry bytes, texture bytes, mesh count — and recommends an AssetLoadingPolicy that selects the most appropriate residency strategy for each memory domain independently.
Its job is to answer: given this asset and this platform's memory budget, should geometry stream or load eagerly, and should textures stream or load eagerly?
+setEntityMeshAsync(.auto)
+ │
+ ├─ Mesh.parseAssetAsync() ← CPU-only parse, no GPU allocation
+ │ └─ ProgressiveAssetData ← MDLMesh objects in CPU RAM
+ │
+ ├─ AssetProfiler.profile() ← analyze ProgressiveAssetData
+ │ └─ AssetProfile ← geo bytes, tex bytes, mesh count, character
+ │
+ ├─ AssetProfiler.classifyPolicy() ← compare profile against platform budget
+ │ └─ AssetLoadingPolicy ← geometryPolicy + texturePolicy
+ │
+ └─ Routing decision
+ geometryPolicy == .streaming → out-of-core stubs path
+ geometryPolicy == .eager → immediate GPU upload path
+The profiler runs entirely on the CPU in the async registration task. No GPU resources are allocated during classification.
+struct AssetProfile {
+ let totalFileBytes: Int
+ let estimatedGeometryBytes: Int // vertex + index bytes, summed across all MDLMesh objects
+ let estimatedTextureBytes: Int // estimated GPU footprint after decompression
+ let meshCount: Int
+ let materialCount: Int
+ let largestSingleMeshBytes: Int
+ let isEffectivelyMonolithic: Bool // meshCount <= 2
+ let assetCharacter: AssetCharacter
+}
+| Value | +Meaning | +
|---|---|
.textureDominated |
+Textures > 75% of combined estimate. Few or small meshes with large maps. | +
.geometryDominated |
+Geometry > 75% of combined estimate. Many or large meshes with minimal textures. | +
.mixed |
+Neither domain exceeds 75%. Both contribute meaningfully. | +
.monolithic |
+≤ 2 meshes. Streaming still prevents OOM at registration, but the mesh loads in one step rather than incrementally. | +
Geometry bytes are estimated using the same formula as CPUMeshEntry.estimatedGPUBytes — this keeps the profiler and the budget manager consistent:
vertexBytes = vertexCount × vertexDescriptor.stride (stride default: 48 bytes)
+indexBytes = vertexCount × 3 × 4 (~3 indices/vertex, 4 bytes each)
+meshBytes = vertexBytes + indexBytes
+Summed across all MDLMesh leaves in ProgressiveAssetData.topLevelObjects.
Texture bytes are estimated without decompressing any texture data. The profiler scans MDLMaterial semantic slots for texture URL references and uses file sizes as a proxy:
For regular file URLs (external textures): +
+For USDZ-embedded textures (bracket-notation paths like file:///scene.usdz[0/tex.png]):
+Individual zip entries cannot be statted without decompressing. A file-level heuristic is used instead:
+
packedTextureBytes = max(0, fileSizeBytes − (geometryBytes / 10))
+textureBytes = packedTextureBytes × 3
+If no texture URLs are found, estimatedTextureBytes is 0 and texturePolicy defaults to .eager.
struct AssetLoadingPolicy {
+ var geometryPolicy: GeometryResidencyPolicy // .eager or .streaming
+ var texturePolicy: TextureResidencyPolicy // .eager or .streaming
+ var source: PolicySource // .auto or .userForced
+}
+The two policies are independent. A texture-dominated asset with 3 meshes and 150 MB of maps gets geometry: .eager, texture: .streaming. A geometry-dominated city with 400 small meshes gets geometry: .streaming, texture: .eager.
| Preset | +Geometry | +Texture | +
|---|---|---|
.fullLoad |
+.eager |
+.eager |
+
.geometryStreaming |
+.streaming |
+.eager |
+
.textureStreaming |
+.eager |
+.streaming |
+
.combinedStreaming |
+.streaming |
+.streaming |
+
if isMonolithic:
+ streaming if geometryBytes / budget > 0.30
+ else eager
+
+if meshCount >= 50:
+ streaming ← many meshes spike GPU allocation regardless of total size
+
+if geometryBytes / budget > 0.30:
+ streaming
+
+else:
+ eager
+The same 200 MB asset routes differently depending on the device:
+| Device | +Budget | +Geo fraction | +Geometry policy | +
|---|---|---|---|
| macOS | +1 GB | +20% | +.eager — fits comfortably |
+
| iOS high-end | +512 MB | +39% | +.streaming — too large |
+
| iOS low-end | +256 MB | +78% | +.streaming — far too large |
+
| visionOS | +512 MB | +39% | +.streaming — too large |
+
Fixed thresholds (the old fileSizeThresholdBytes = 50 MB) applied the same cutoff on all platforms. A 200 MB asset would always trigger streaming, even on a macOS workstation with 1 GB of headroom.
For every .auto classification the profiler emits two log lines:
[AssetProfiler] 'dungeon3' (2.1 MB) → mixed | geo ~2.9 MB, tex ~6.2 MB | budget: 1024 MB | meshes: 410
+[AssetProfiler] Policy → geometry: streaming, texture: eager (source: auto)
+Line 1 — profile snapshot: filename, file size, asset character, estimated geometry bytes, estimated texture bytes, platform budget, mesh count.
+Line 2 — the chosen policy for each domain and whether it was auto-selected or user-forced.
+If geometry streaming is selected, the out-of-core log also captures the reason:
+[OutOfCore] 'dungeon3': mixed asset, geo ~2.9 MB on 1024 MB budget → out-of-core stub registration (410 stubs)
+Texture estimation for USDZ-embedded textures is approximate. The (fileSizeBytes − geometryBytes/10) × 3 heuristic overestimates for geometry-heavy assets and underestimates for assets with highly compressed textures (e.g. ASTC at 8:1). It is biased toward overestimation intentionally.
Monolithic assets stream but do not incrementally load. An asset with 1 mesh and 400 MB of geometry enters the streaming path (to prevent OOM at registration) but loads its full geometry in a single upload step. There is no incremental benefit beyond registration-time safety.
+External texture files must be accessible at classification time. If textures are on a remote URL or behind a slow filesystem, the stat() calls in estimateTextureBytes will block the registration task. This is only a concern for non-local assets.
Material semantics scanned are limited to the seven standard PBR slots. Custom material properties outside those semantics are not counted. If an asset uses non-standard semantic names, its texture estimate will be lower than the true cost.
+| System | +Relationship | +
|---|---|
ProgressiveAssetLoader |
+Provides ProgressiveAssetData that AssetProfiler.profile() analyzes |
+
MemoryBudgetManager |
+Provides meshBudget; all thresholds are fractions of this value |
+
GeometryStreamingSystem |
+Activated when geometryPolicy == .streaming; manages GPU residency per entity |
+
TextureStreamingSystem |
+Runs on all entities with RenderComponent regardless of texture policy; the policy makes the intent explicit for future per-entity gating |
+
RegistrationSystem |
+Calls AssetProfiler in the .auto branch of setEntityMeshAsync; maps the result to useOutOfCore |
+
UntoldEngine supports streaming scene geometry directly from remote HTTPS servers (e.g., a CDN). Plain http:// URLs are rejected at the engine boundary to prevent unencrypted asset transmission. When a scene manifest URL points to a remote HTTPS server, the engine downloads the manifest, resolves all tile/HLOD/LOD asset URLs relative to that server, and downloads each asset on demand as the camera approaches. Downloaded assets are stored in a persistent disk cache so subsequent sessions never re-download unchanged files.
This layer sits below the tile streaming system but above the local asset loading pipeline. The rest of the engine — GeometryStreamingSystem, TileComponent, UntoldReader — is unaware of whether an asset came from disk or the network; RemoteAssetDownloader resolves a remote URL to a local cached path before the file is opened.
RemoteAssetDownloader (actor)Sources/UntoldEngine/Systems/RemoteAssetDownloader.swift
The single download actor for all remote assets. Responsibilities:
+URLSession and commits them to AssetDiskCache.localURL(for:) throws DownloadError.insecureScheme immediately for any non-HTTPS URL. Plain http:// is never sent to the network.If-None-Match: <etag>. A 304 Not Modified response returns the cached path instantly without re-downloading..untold file, immediately fetches all textures referenced in its texture table in the background so they are cache-resident before the tile is parsed.Public API:
+ +Returns a local file:// URL pointing to the cached asset. Throws on permanent failure after all retries are exhausted.
URLSession configuration:
+| Parameter | +Value | +
|---|---|
timeoutIntervalForRequest |
+30 s | +
timeoutIntervalForResource |
+300 s | +
| Max retry attempts | +3 (delays: 1 s, 2 s, 4 s) | +
| Retry delay formula | +2^attempt seconds |
+
AssetDiskCache (actor)Sources/UntoldEngine/Systems/AssetDiskCache.swift
A persistent LRU cache that maps remote URLs to local files on disk.
+SHA256(url.absoluteString). Files are stored at <cacheDir>/<hash>.<ext>.<hash>.meta (JSON). Retrieved by RemoteAssetDownloader for conditional GET on subsequent requests.lastAccess timestamp (oldest first) until usage falls to 75% of budget.storeAtRelativePath(_:data:), so NativeFormatLoader can resolve texture URIs by the same relative path they have inside the .untold file.Cache parameters:
+| Parameter | +Default | +
|---|---|
| Cache directory | +Library/Caches/UntoldAssetCache/ |
+
| Budget | +500 MB | +
| Eviction target | +75% of budget | +
| Cache key | +SHA256(url.absoluteString) |
+
| ETag storage | +<hash>.meta |
+
| Write strategy | +tmp file → atomic rename | +
The engine resolves asset URLs lazily at load time. The helper resolveAssetURL(_:label:) is called by the tile loading path before opening any file:
func resolveAssetURL(_ url: URL, label: String) async -> URL? {
+ if url.scheme?.lowercased() == "http" {
+ // Plain HTTP is rejected — only HTTPS is permitted.
+ let error = RemoteAssetDownloader.DownloadError.insecureScheme("http")
+ Logger.logError(message: "[TileStreaming] Remote download failed for \(label): \(error)")
+ return nil
+ }
+ guard url.scheme?.lowercased() == "https" else {
+ return url // Local file:// path — pass through unchanged
+ }
+ do {
+ return try await RemoteAssetDownloader.shared.localURL(for: url)
+ } catch {
+ Logger.logError(message: "[TileStreaming] Remote download failed for \(label): \(error)")
+ return nil
+ }
+}
+Local file:// paths bypass the downloader entirely. Only https:// URLs go through the remote download cache. Plain http:// URLs are explicitly rejected with a logged error and a nil return, which causes the tile load to mark the tile .failed and enter exponential backoff.
When setEntityStreamScene(entityId:url:) is called with a remote manifest URL, tile asset URLs are resolved relative to the manifest directory:
Manifest: https://cdn.example.com/dungeon3/dungeon3.json
+Tile path: tiles/tile_0_0.untold
+→ Tile URL: https://cdn.example.com/dungeon3/tiles/tile_0_0.untold
+
+HLOD path: hlods/tile_0_0_hlod.untold
+→ HLOD URL: https://cdn.example.com/dungeon3/hlods/tile_0_0_hlod.untold
+The same construction applies to HLOD and per-tile LOD URLs. All of these are stored in their respective TileComponent fields as absolute https:// URLs. resolveAssetURL translates them to cached local paths at load time.
let sceneRoot = createEntity()
+setEntityName(entityId: sceneRoot, name: "scene")
+
+if let manifestURL = URL(string: "https://cdn.example.com/scene/scene.json") {
+ setEntityStreamScene(entityId: sceneRoot, url: manifestURL)
+}
+ │
+ ├─ URL has https scheme → download manifest
+ │ └─ RemoteAssetDownloader.localURL(for: manifestURL)
+ │ ├─ AssetDiskCache.localURL(for:) → cache miss
+ │ ├─ URLSession GET with optional If-None-Match header
+ │ ├─ 200 OK → AssetDiskCache.store(data:for:etag:) atomically
+ │ └─ Return local file:// path
+ │
+ └─ Decode JSON at local path → TileManifest
+ └─ registerTiledScene(manifest:baseURL:)
+ ├─ Construct tile URLs relative to manifest URL
+ ├─ Register lightweight TileComponent stubs (no geometry)
+ └─ Each stub stores tileURL as https:// in TileComponent
+GeometryStreamingSystem: camera enters prefetchRadius for tile_0_0
+ │
+ └─ loadTile(entityId:)
+ ├─ setEntityMeshAsync(entityId: meshEntity, ...)
+ │
+ └─ Inside async Task:
+ ├─ resolveAssetURL(tileURL) ← tileURL is https://
+ │ └─ RemoteAssetDownloader.localURL(for: tileURL)
+ │ ├─ Cache hit? → return local path immediately
+ │ └─ Cache miss? → download, store, return local path
+ │
+ ├─ Parse asset at local path
+ │ ├─ .untold → UntoldReader (no ModelIO dependency)
+ │ └─ .usdz/.usdc → ModelIO via NativeFormatLoader
+ │
+ └─ Upload geometry to Metal GPU buffers
+.untold assets)After a .untold file is downloaded, RemoteAssetDownloader immediately pre-fetches all referenced textures in the background:
RemoteAssetDownloader downloads tile_0_0.untold
+ │
+ └─ downloadTextures(in: untoldData, remoteBaseURL: tileDirectoryURL)
+ ├─ Parse texture table URIs from .untold header
+ └─ For each texture URI (e.g. "Textures/wall_albedo.png"):
+ ├─ Check AssetDiskCache for relative path → skip if present
+ ├─ Construct: remoteBaseURL + "/" + uri
+ ├─ Download → AssetDiskCache.storeAtRelativePath(uri, data:)
+ └─ (Failures skipped; geometry still loads, just untextured)
+Textures are stored at their relative URI under the cache root. When NativeFormatLoader later resolves texture paths, it finds them at the same relative path without knowing they came from a CDN.
On the second session or after an initial warm-up pass, all tiles are cached locally:
+resolveAssetURL(tileURL)
+ └─ RemoteAssetDownloader.localURL(for: tileURL)
+ └─ AssetDiskCache.localURL(for: tileURL) → hit
+ └─ Return file:// path instantly (zero network I/O)
+On subsequent sessions the manifest is revalidated cheaply:
+GET /scene/scene.json
+Headers: If-None-Match: "abc123"
+
+← 304 Not Modified
+ → Return cached manifest path, no re-download
+ → All tile URLs resolved from unchanged local manifest
+If the server returns 200 with a new ETag, the manifest and its ETag sidecar are updated atomically.
+When accumulated downloads exceed the 500 MB budget:
+AssetDiskCache.evictToLimit()
+ ├─ Sort entries by lastAccess ascending (oldest first)
+ └─ Delete files (and their .meta sidecars) until usage ≤ 375 MB (75%)
+Evicted files are re-downloaded transparently on next access. The cache directory persists across app launches; only explicit clearCache() or OS-driven cache purges remove it entirely.
| Attempt | +Delay before next attempt | +
|---|---|
| 0 | +— (immediate first try) | +
| 1 | +1 s | +
| 2 | +2 s | +
| 3 (final) | +4 s | +
After 3 failed attempts, localURL(for:) throws. The tile load marks state .failed and enters the tile-level exponential backoff (5 s → 10 s → 20 s → 60 s max) before retrying.
Treated as a cache hit. No file write occurs; localURL(for:) returns the existing cached path.
Counted as a failure, subject to the retry policy above. A 404 is retried like any other error — the manifest may be temporarily unavailable or behind an eventual-consistent CDN.
+Individual texture download failures are silently skipped. Geometry still loads and renders, just without that texture (Metal will bind a default 1×1 white fallback). This prevents a single bad texture URL from blocking tile geometry.
+If 10 tiles all reference the same shared texture and all request it simultaneously, only one URLSession task fires. The other 9 suspend and resume with the cached result once the first download completes.
Remote streaming is transparent to GeometryStreamingSystem and TileComponent. From their perspective:
TileComponent.tileURL holds the canonical URL for the tile (https:// for remote scenes, file:// for local scenes).loadTile() calls setEntityMeshAsync, which calls resolveAssetURL to get a local path before parsing..unloaded → .parsing → .parsed → .unloading) is identical regardless of whether the asset was local or remote.The only difference in behavior is a latency bump on first load when a tile is not yet cached. The prefetch radius absorbs most of this: the tile begins downloading when the camera is effectivePrefetchRadius away, giving the download time to complete before the camera reaches streamingRadius.
For a typical scene with streamingRadius = 80 m and unloadRadius = 120 m, effectivePrefetchRadius auto-computes to 100 m. At walking speed (~1.5 m/s) this is ~13 seconds of headroom — well above the download + parse time for a 15–20 MB tile over a 10 Mbps connection (~12–16 s worst case).
Sources/DemoGame/DemoState.swift registers two remote scenes:
let remoteScenes: [RemoteSceneOption] = [
+ .init(
+ id: "dungeon",
+ title: "Dungeon",
+ manifestURL: URL(string: "https://cdn.example.net/dungeon3/dungeon3.json")!
+ ),
+ .init(
+ id: "city",
+ title: "City",
+ manifestURL: URL(string: "https://cdn.example.net/city/city.json")!
+ ),
+]
+GameScene.loadTileScene(url:) creates a root entity and passes it with the manifest URL to setEntityStreamScene(entityId:url:):
func loadTileScene(sceneID: String, url: URL, completion: @escaping @Sendable (Bool) -> Void) {
+ clearSceneBatches()
+ GeometryStreamingSystem.shared.enabled = true
+
+ let sceneRoot = createEntity()
+ setEntityName(entityId: sceneRoot, name: sceneID)
+
+ setEntityStreamScene(entityId: sceneRoot, url: url) { success in
+ completion(success)
+ }
+}
+The HUD (DemoHUD.swift) presents scene options; on selection it calls onLoadTiledScene which routes to loadTileScene(url:).
| Concern | +Mechanism | +
|---|---|
RemoteAssetDownloader actor isolation |
+Swift actor — safe to call from any Task or thread |
+
AssetDiskCache actor isolation |
+Swift actor — all reads and writes are serialised | +
| Single-flight gate | +inFlightDownloads: [URL: Task<URL, Error>] dictionary, actor-protected |
+
| Atomic cache writes | +Temp file + FileManager.moveItem (atomic on same volume) |
+
| ECS mutations | +Main thread only, via withWorldMutationGate |
+
| Tile completion guard | +scene.exists(entityId) checked before every ECS write in upload completion closures |
+
| Parameter | +Value | +Location | +
|---|---|---|
| Permitted URL schemes | +https:// only — http:// is rejected |
+RemoteAssetDownloader, resolveAssetURL |
+
| Max download retries | +3 | +RemoteAssetDownloader |
+
| Retry delay formula | +2^attempt seconds |
+RemoteAssetDownloader |
+
| Request timeout | +30 s | +URLSession configuration |
+
| Resource timeout | +300 s | +URLSession configuration |
+
| Disk cache budget | +500 MB | +AssetDiskCache |
+
| Cache eviction target | +75% of budget | +AssetDiskCache |
+
| Cache key | +SHA256(url.absoluteString) |
+AssetDiskCache |
+
| ETag revalidation | +Yes (conditional GET) | +RemoteAssetDownloader |
+
| Texture pre-fetch | +Yes (post-download, async) | +RemoteAssetDownloader |
+
| Single-flight dedup | +Yes (actor-isolated dictionary) | +RemoteAssetDownloader |
+
tilebasedstreaming.md — tile lifecycle, manifest schema, HLOD, LOD bandsgeometryStreamingSystem.md — mesh-level OCC streaming, memory pressure, evictionassetFormat.md — .untold binary format specificationprogressiveAssetLoader.md — CPU heap management for out-of-core assetsThe goal is simple: instead of issuing 100 separate draw calls (one per entity), merge entities that share the same material into a single combined GPU buffer and issue one draw call per material group. This is called static batching.
+The 3D world is partitioned into a 3D grid of cells. The cell size is calibrated at scene load time: when a tile manifest is present, the cell size is set to 2 × tileSize so that cell boundaries align with tile boundaries. When no manifest is present, the default of 32 world units is used.
Every entity is assigned to a cell based on the world-space center of its bounding box:
+ +Why align to tile boundaries? Batching 100 entities scattered across a huge world into one mesh is wasteful — you'd rebuild everything when anything changes. Cells localize the damage. When cell boundaries align with tile boundaries, loading or unloading a tile only touches the cells that tile occupies — no cross-tile batch rebuilds.
+When your 100 entities load, each one that has a StaticBatchComponent gets registered:
resolveBatchCandidate): the entity must have a RenderComponent, WorldTransformComponent, no skeleton/animation, no transparency, no gizmo/light component, and its mesh must already be resident in memory. The LOD index is derived from LODComponent.currentLOD (entity-level LOD), then TileLODTagComponent.levelIndex (per-tile LOD/HLOD children), defaulting to 0. isLODBatch on the resulting BatchGroup is true if any member entity has either component.cellToEntities[cellId].renderableUnbatched.Every frame, tick() runs through this pipeline:
Any entities that changed (LOD switch, mesh evicted/streamed in) are removed from their old cell and re-registered in their current cell. This marks the affected cells dirty.
+Tile-loaded entities bypass the quiescence delay. When a fullLoad tile (or LOD/HLOD load) finishes, GeometryStreamingSystem calls BatchingSystem.notifyTileEntitiesResident(_:) with the set of render-ready entity IDs. This single call directly registers the entities in pendingEntityAdditions, marks them as tile-parsed (for quiescence bypass), and resolves their cell membership — replacing the former two-step queueResidencyEventsForRenderDescendants + notifyTileParsedEntities pairing and avoiding the per-entity event storm through SystemEventBus. Their cells are immediately promoted to batchPending in the same tick. See Tile-Local Batch Promotion below.
Stale entity purge on LOD/HLOD teardown. When unloadLODLevel or unloadHLOD destroys child entities, it first calls BatchingSystem.cancelPendingEntities(_:) with the render descendant IDs. This removes them from pendingEntityAdditions, pendingEntityRemovals, newlyResidentEntities, and tileParsedEntityIds before the entities are destroyed — preventing "entity is missing" errors on the next tick() and avoiding wasted batch rebuilds for entities that no longer exist.
The system checks which cells currently contain visible entities and records cellLastVisibleFrame[cellId]. This drives visibility gating — the system won't waste CPU rebuilding cells you can't see.
batchPendingFor each dirty cell in state renderableUnbatched or streaming:
+- Is it visible (or recently visible within 120 frames)?
+- Has it been stable for at least quiescenceFramesBeforeBatchBuild frames (default: 1)?
If yes → state becomes batchPending.
Cells flagged by tile promotion skip the quiescence check and advance directly to batchPending in the same tick.
rebuildDirtyCells)This is the core build loop:
+Apply completed background artifacts first — results from previous frames' async builds are swapped in (up to maxArtifactAppliesPerTick = 4 per frame).
Gather batchPending cells and build rebuild candidates. For each:
runtimeIneligibleCells and stays unbatched.Otherwise it becomes a CellRebuildCandidate.
Sort candidates by priority:
+Oldest dirty-since-frame first
+Apply per-tick budgets: up to 8 cells, 120K verts, 220K indices, 6MB total per tick. Once budgets are exhausted, remaining cells defer to next frame.
+Snapshot build inputs under a world mutation gate: for each selected cell, group its entities' meshes by BatchBuildKey = (cellId, materialHash, lodIndex). This produces CellBuildInput.
Dispatch background builds on artifactBuildQueue (a .utility DispatchQueue). The heavy work — actually merging vertex data — happens off the main thread.
buildPreparedArtifact)For each CellBuildInput, on the background thread:
createBatchGroup:worldTransform.space * mesh.localSpace).MTLBuffers for the merged position/normal/UV/tangent/index data.PreparedCellArtifact containing [BatchGroup].So 100 entities all sharing the same wood-plank material → 1 BatchGroup with 1 merged MTLBuffer and one tight world-space AABB.
+++What is an artifact? +An artifact is the output package produced by a build job: + the input is
+CellBuildInput(a snapshot of which entities are in a cell and how they're grouped by material), and the artifact (PreparedCellArtifact) is the finished result — the merged MTLBuffers, entity-to-batch mappings, vertex/index counts, and build time. Everything needed to install the batch into the live scene. +
Back on the main thread (next frame or same frame if sync mode):
+BatchGroups to batchGroups.entityToBatch[entityId] so the renderer knows each entity is now represented by a batch.renderableBatched.The renderer uses cluster-level frustum culling to determine which batch groups to submit. Each BatchGroup carries a precomputed world-space AABB (boundingBox) covering all geometry in the group. The render passes test each group's AABB directly against the current-frame frustum — one AABB test per batch group, not one per entity. Groups whose AABB is fully outside the frustum are skipped without any entity-level traversal.
For batch groups that survive the AABB test, each BatchGroup is one draw call with its merged buffer. 100 entities sharing one material = 1 draw call, submitted only when the group's spatial bounds are within the frustum.
Per-entity batching membership (entityToBatch) is still maintained and used by non-batched rendering paths and by tests.
Old GPU buffers aren't freed immediately. They go into retiringBatchArtifacts with a retireAfterFrame = currentFrame + 3. After 3 frames, the system drops the Swift reference, allowing ARC to release the MTLBuffers — guaranteeing the GPU has finished with them.
unloaded
+ ↓ (entity becomes resident)
+streaming
+ ↓ (quiescence + visibility check pass)
+ ↓ (or: tile-local promotion — bypasses quiescence)
+renderableUnbatched
+ ↓ (promoted, budget available)
+batchPending
+ ↓ (build dispatched + applied)
+renderableBatched
+ ↓ (entity removed or LOD change)
+retiring → unloaded
+When a fullLoad tile (one that completes GPU upload in a single step, occCount == 0), per-tile LOD level, or HLOD mesh finishes loading, GeometryStreamingSystem hands off the set of render-ready entity IDs to the batching system:
This single call combines what was previously a per-entity AssetResidencyChangedEvent storm + a separate notifyTileParsedEntities call. The entities are registered directly in the batching system's pending additions and marked for quiescence bypass, avoiding hundreds of individual events through SystemEventBus.
In the next tick(), those entities are processed differently from ordinary streaming arrivals:
deferBatchBuild = false — the extra one-frame deferral applied to newly-resident streaming entities is suppressed.renderableUnbatched → batchPending in the same tick, bypassing the quiescenceFramesBeforeBatchBuild wait.The result: a tile that finishes parsing on frame N will have its cells in batchPending on frame N+1 and a completed batch ready to submit one or two frames later (depending on background build time), rather than waiting for the quiescence window first.
This is safe because a tile's geometry arrives atomically — all entities are registered at once with no further churn expected. The quiescence delay exists to absorb incremental arrivals (OCC stub uploads); it is unnecessary and harmful for the fullLoad tile path.
+OCC tiles are unchanged. For tiles using the out-of-core upload path, individual mesh stubs still arrive one at a time via the normal handleResidencyChange flow. The quiescence delay is preserved for these so the batch doesn't rebuild for each individual stub upload.
Every BatchGroup stores a precomputed world-space AABB:
This is computed during createBatchGroup as the min/max of all transformed vertex positions. It represents the tightest world-space bounding box over all geometry in the group.
The AABB serves two purposes:
+1. Cluster-level frustum culling — the render pass tests this AABB against currentFrameFrustum before encoding the draw call, skipping entire groups that are outside the view.
+2. Future use — HLOD transitions, occlusion culling, and GPU-driven rendering will all use this AABB as the cluster's spatial identity.
Suppose your 100 entities break down as: +- 60 entities: wood material, LOD 0, all in cell (0,0,0) +- 30 entities: stone material, LOD 0, cell (0,0,0) +- 10 entities: glass material (transparent) → excluded from batching
+Result: +- 2 BatchGroups for cell (0,0,0): one wood, one stone +- Each BatchGroup has its own world-space AABB covering all vertices in the group +- 2 draw calls instead of 90 (the 10 transparent ones draw individually) +- Both groups are frustum-culled by AABB before encoding — if the whole cell is off-screen, 0 draw calls are issued +- On a LOD change (say 20 wood entities switch to LOD 1), cell (0,0,0) is marked dirty → rebuild fires next eligible tick → now 3 BatchGroups (wood LOD0, wood LOD1, stone LOD0)
+ + + + + + + + + + + + + +Imagine a large tile has been parsed through setEntityStreamScene(...) and classified into the out-of-core path. The tile now contains many internally created OCC child stubs — building_A, building_B, streetlamp_01, car_01, etc. Each stub has a StreamingComponent that stores:
assetFilename / assetExtension — where the mesh or source asset came fromstreamingRadius — how close the camera must be to load itunloadRadius — how far before it gets unloadedpriority — which buildings load first when slots are contestedstate — .unloaded, .loading, .loaded, or .unloadingupdate(cameraPosition:deltaTime:)The engine calls this every frame. Here's what happens:
+The system normally does real work every 0.1 seconds (updateInterval). Between ticks, it's a no-op. This prevents wasting CPU every single frame.
When lastPendingLoadBacklog > 0 (candidates are queued but all slots are busy), the effective interval drops to burstTickInterval (default 16 ms). This prevents a 100 ms stall between slot pickups during active loading. The tick rate returns to 100 ms once the backlog drains.
OS pressure bypass — if a pendingPressureRelief flag is set (fired by the OS pressure callback on a background queue), the throttle check is bypassed entirely for that call. This guarantees eviction runs within one frame (≤ 11 ms at 90 fps) rather than waiting up to 100 ms for the next normal tick. Without this, a .critical signal arriving right after a tick would sit unprocessed for the full throttle interval — longer than visionOS's kill window.
Instead of checking every OCC child stub in the scene, it asks the OctreeSystem:
++"Give me every entity within 500m of the camera."
+
This is the key performance trick — only nearby entities are evaluated.
+For each entity the octree returns, the system calculates the distance from camera to the entity's bounding box center, then:
+| State | +Condition | +Action | +
|---|---|---|
.unloaded |
+distance ≤ streamingRadius |
+→ add to load candidates | +
.loaded |
+distance > unloadRadius |
+→ add to unload candidates | +
.loaded |
+still in range | +→ stamp lastVisibleFrame (keep alive) |
+
.loading / .unloading |
+— | +skip, already in progress | +
The octree query only covers nearby space. But what if building_Z was loaded and the player sprinted far away — it might not be in the octree result anymore. So the system also checks its loadedStreamingEntities tracking set for any loaded entity not in the octree result, and adds those to unload candidates if they're too far.
Unload candidates are sorted farthest-first (most wasteful memory first). Up to maxUnloadsPerUpdate = 12 are processed per tick to avoid frame spikes.
unloadMesh() does:
+1. Sets state → .unloading
+2. Notifies BatchingSystem the entity is retiring
+3. Cancels any in-flight load task
+4. Calls MeshResourceManager.shared.release(entityId:) — decrements reference count on the cached mesh
+5. Clears render.mesh = [] — the GPU buffers are not destroyed (cache still owns them)
+6. Clears LOD level meshes if applicable
+7. Unregisters from MemoryBudgetManager
+8. Sets state → .unloaded
+9. Fires an AssetResidencyChangedEvent(isResident: false)
Load candidates are sorted by priority then distance (high priority + closest first). Only maxConcurrentLoads = 3 can be active simultaneously.
Before dispatching, the scheduler applies four guards in order:
+isTileOwned) — the entity must be a descendant of a TileComponent entity. Non-tile-owned entities are rejected immediately and their state is never mutated. StreamingComponent is an internal, tile-subordinate mechanism; it is not valid on standalone entities. See StreamingComponent Ownership Model below.CPUMeshEntry is not yet stored in ProgressiveAssetLoader are skipped. This prevents pre-streaming stubs from holding slots while registration is still running.isPrewarmActive returns false.evictLRU is called first.When all near-band candidates share one assetRootEntityId, the near-band concurrency limit expands from nearBandMaxConcurrentLoads to maxConcurrentLoads. All sub-meshes of one USDZ are treated as a single burst rather than being serialized one-at-a-time.
loadMesh() does:
+1. Reserves a slot in activeLoads (thread-safe via NSLock)
+2. Sets state → .loading
+3. Notifies BatchingSystem streaming started
+4. Spawns a Swift Task (runs off the main thread)
Inside the async task:
+- If the entity has a LODComponent → calls reloadLODEntity() which loads all LOD levels
+- Otherwise → calls loadMeshAsync() which goes to MeshResourceManager (cache-first, file fallback)
+- After loading, back on the main thread via withWorldMutationGate:
+ - Assigns render.mesh with fresh copies of uniform buffers (critical — prevents entities sharing GPU state from overwriting each other)
+ - Sets state → .loaded
+ - Fires AssetResidencyChangedEvent(isResident: true)
+ - Records load in MemoryBudgetManager
StreamingComponent is an internal, tile-subordinate component. It is not a public API for external callers.
TileComponent entity may have an active StreamingComponent. loadMesh() enforces this with the isTileOwned() guard, which walks the ScenegraphComponent.parent chain and returns false for any entity that has no TileComponent ancestor.StreamingComponent stubs are created internally by setEntityMeshAsync for the out-of-core (OCC) path when a tile file is large enough to be split into sub-mesh stubs. External callers never attach StreamingComponent directly.enableStreaming() is internal; it is not part of the public API.| Use case | +API | +
|---|---|
| Streamable geometry exported by the Blender pipeline (terrain, city blocks, large scenes) | +setEntityStreamScene(entityId:url:completion:) with a local or remote (https://) manifest URL |
+
| Handcrafted streaming zones (dungeon rooms, level sectors, indoor areas) | +StreamingRegionManager — register explicit StreamingRegion AABB + asset lists; no manifest required |
+
| Always-resident objects (characters, props, HUD elements) | +setEntityMeshAsync(entityId:filename:withExtension:completion:) |
+
setEntityStreamScene is the preferred public entry point for streamable scene geometry. It accepts a root EntityID and a local file:// path or remote https:// URL. For remote URLs it downloads and caches the manifest via RemoteAssetDownloader before decoding. It then calls the internal registerTiledScene() and hands off all streaming lifecycle management to GeometryStreamingSystem. The backwards-compatible loadTiledScene(manifest:) / loadTiledScene(url:) overloads remain available and create an internal root entity automatically. See asset_remote_streaming.md for the full remote download lifecycle.
isTileOwned(entityId:) — private helper in GeometryStreamingSystem+MeshStreaming.swift
+ Walks ScenegraphComponent.parent chain upward.
+ Returns true only if a TileComponent is found somewhere in the ancestry.
+ Returns false for any standalone entity (no parent chain, or chain ends without a tile).
+reloadLODEntity() (lines 313–415)For LOD entities (e.g., a skyscraper with 3 detail levels), it:
+1. Loads all LOD levels concurrently from cache/disk
+2. Calculates current camera-to-entity distance
+3. Picks the appropriate LOD level (highest detail that fits distance)
+4. Sets renderComponent.mesh to that LOD's mesh data
+5. Marks lodComponent.currentLOD
The engine uses two independent memory pressure signals and responds to them in priority order:
+| Pressure signal | +Method | +Meaning | +
|---|---|---|
| Combined | +shouldEvict() |
+Geometry pool ≥ 85% of geometryBudget OR texture pool ≥ 85% of textureBudget |
+
| Geometry only | +shouldEvictGeometry() |
+Mesh allocations alone ≥ 85% of geometryBudget |
+
Why two signals? TextureStreamingSystem upgrades visible textures to higher resolutions after meshes load. Those upgrades increase totalTextureMemory in MemoryBudgetManager. If the load gate used the combined signal, texture upgrades on already-loaded meshes would silently prevent new mesh loads — even when the geometry-only footprint is well within budget. The split pools (geometryBudget + textureBudget) ensure each domain has an independent ceiling so neither can starve the other.
Before considering geometry eviction, the system sheds texture quality on the farthest loaded entities:
+if combined pressure is high AND geometry pressure is NOT high:
+ TextureStreamingSystem.shedTextureMemory(maxEntities: 4)
+ → no geometry eviction; texture relief only
+shedTextureMemory forces the farthest entities in the upgradedEntities set to minimumTextureDimension immediately, bypassing the normal distance-band schedule. A distant wall dropping from 1024 px to 256 px is far less noticeable than a missing mesh.
Only triggered when geometry memory itself hits the high-water mark:
+if geometry pressure is high:
+ TextureStreamingSystem.shedTextureMemory(maxEntities: 8) ← try texture relief first
+ evictLRU(cameraPosition:) ← then fall back to geometry eviction
+evictLRU:
+1. First evicts unused cached meshes (MeshResourceManager.evictUnused())
+2. Collects all loaded streaming entities
+3. Sorts by value score (far + large = first to go; see value-score eviction in the out-of-core walkthrough)
+4. Unloads them one by one until geometry-only pressure clears (loop breaks on shouldEvictGeometry(), not the combined signal)
+5. Skips entities that are both visible and within visibleEvictionProtectionRadius (30 m default)
+6. Accepts an optional maxEvictions cap (default Int.max). The OS pressure path passes 16 per call — this bounds single-frame work during a burst. Any remaining candidates spill to subsequent ticks.
The sizeFactor in the eviction score is normalized against geometryBudget (not the combined budget), so a mesh consuming 80% of the geometry pool scores correctly rather than appearing to consume only ~48% of a combined total.
In addition to the per-tick budget checks above, MemoryBudgetManager subscribes to OS memory pressure events via DispatchSource.makeMemoryPressureSource:
| OS signal | +Response | +maxEntities |
+
|---|---|---|
.warning |
+Texture shed | +8 | +
.critical |
+Texture shed + double geometry eviction pass (capped at 16 per pass) + CPU heap release | +20 | +
The OS callback fires on a background queue and sets a pendingPressureRelief flag on GeometryStreamingSystem. The flag is drained at the start of the next update() tick on the main thread, so all eviction work stays on the same thread as the rest of the streaming system. This prevents the OS from silently escalating to .critical and terminating the process — on visionOS in particular, the window between .warning and process kill can be under a second.
CPU heap release on critical pressure — evictLRU only frees GPU Metal buffers tracked by MemoryBudgetManager. The OS measures total process memory, which includes ProgressiveAssetLoader.rootAssetRefs (the live MDLAsset tree and all child CPUMeshEntry vertex/index buffers). For a 500-building scene this CPU heap can reach hundreds of megabytes. On .critical, after the two geometry eviction passes, GeometryStreamingSystem calls ProgressiveAssetLoader.shared.releaseWarmAsset(rootEntityId:) on every warm root. This frees the CPU heap immediately. The rehydration context (asset URL + loading policy) is retained, so a cold re-stream from disk is transparent when the camera re-approaches.
Player spawns at corner of city block
+│
+├─ Frame 1 tick: Octree finds 8 nearby buildings
+│ ├─ 5 are unloaded + within streamingRadius → load candidates
+│ └─ 3 are loading already → skip
+│
+├─ Up to 3 async loads fire simultaneously
+│ ├─ building_A: cache miss → read from USDZ file
+│ ├─ building_B: cache hit → instant
+│ └─ building_C: cache miss → read from USDZ file
+│
+├─ Player walks forward → building_K enters range
+│ └─ Queued in load candidates (backlog until a slot frees)
+│
+├─ Player runs past old buildings → building_A now > unloadRadius
+│ └─ render.mesh cleared, reference released, memory freed
+│
+└─ Memory pressure → LRU eviction kicks in
+ └─ building_E (not visible, oldest lastVisibleFrame) → evicted
+The key design decisions here are:
+- Octree spatial query prevents O(n) entity iteration every tick
+- Concurrency cap (3) prevents GPU/IO saturation during fast movement
+- Adaptive tick rate — 16 ms during backlog, 100 ms steady-state — prevents stalls between slot pickups without wasting CPU when idle
+- Single-root burst detection — when all near-band candidates are sub-meshes of one asset, concurrency expands to the global cap so the asset loads in parallel rather than one mesh at a time
+- Background texture prewarm — loadTextures() runs at registration time so the first-upload path is a no-op and lock wait ≈ 0
+- Prewarm-active deferral — dispatch is held until the prewarm releases the texture lock, keeping all slots free for the burst
+- Narrowed texture lock scope — the per-asset lock covers only ensureTexturesLoaded; makeMeshesFromCPUBuffers runs outside the lock so all slots upload in parallel
+- CPU-entry readiness guard — stubs registered before their CPU data is ready are skipped rather than wasting a slot
+- Unload-before-load ordering ensures you free memory before consuming more
+- Cache ownership means unloading just clears references, actual GPU memory is reused if the same mesh comes back into range
+- Geometry-only load gate prevents texture upgrades from blocking mesh loads — each domain is budgeted independently
+- Texture relief before geometry eviction means a drop in distant texture resolution is always preferred over a missing mesh
+- Split geometry/texture pools (geometryBudget + textureBudget) give each domain an independent ceiling and high-water mark — a texture-heavy scene cannot crowd out geometry loads and vice versa
+- Runtime device budget probing — geometryBudget and textureBudget are derived at init from MTLDevice.recommendedMaxWorkingSetSize (macOS) or os_proc_available_memory() (visionOS/iOS) rather than hardcoded platform defaults; budgets adapt to actual device headroom
+- SceneRootTransform consistency — all distance calculations (GeometryStreamingSystem, LODSystem, inline LOD upload helpers) pass camera position through SceneRootTransform.shared.effectiveCameraPosition() so XR physical-head movement and scene-root translations are applied uniformly; raw cameraComponent.localPosition is never used directly for distance math
+- Camera sync always runs — syncStreamingCameraPosition() executes every frame regardless of the loading flag; decoupling it from the loading guard prevents the streaming camera from freezing while an asset load is in flight
+- OS memory pressure subscription — DispatchSource.makeMemoryPressureSource fires proactive texture shedding and geometry eviction before the OS escalates to process termination; the response runs on the next update() tick to stay single-threaded
+- evictLRU per-call cap — the maxEvictions parameter (default Int.max) bounds single-frame eviction work; the OS pressure path uses 16 per pass so a .critical burst doesn't spike one frame; remaining candidates spill to subsequent ticks
+- CPU heap release on critical pressure — on .critical, after geometry eviction, ProgressiveAssetLoader.releaseWarmAsset() is called for every warm root, freeing the MDLAsset CPU heap the OS measures; rehydration context survives so cold re-stream from disk is transparent
The tile-level streaming layer sits above the mesh-level OOC streaming. For full documentation, architecture diagrams, and tuning guidance see docs/Architecture/tilebasedstreaming.md. This section summarises the key design decisions that interlock with the mesh-level system documented above.
update() (tile layer, in order).unloaded stubs within effectivePrefetchRadius (frustum-gated, budget-gated, up to maxConcurrentTileLoads). Tiles with LOD levels are gated: the full tile only loads when the camera is within the finest LOD switch distance.unloadRadius, .parsed outside query radius, .parsing outside query radius).hlodSwitchDistance and tileComp.state. Uses hlodHysteresisFactor (default 0.90) to prevent thrashing at boundaries. Capped by maxConcurrentHLODLoads (default 4).maxQueryRadius.lodHysteresisFactor, default 0.90). Skips when HLOD is resident (avoids dual representation). Loads target; unloads others; unloads all when tile is .parsed. Capped by maxConcurrentLODLoads (default 4).maxQueryRadius.Tile stubs carry a TileComponent (no StreamingComponent, no RenderComponent):
| Field | +Purpose | +
|---|---|
tileURL |
+Absolute URL of the tile asset — file:// for local scenes, https:// for remote CDN scenes |
+
fileSizeBytes |
+Pre-computed file size for the memory budget gate | +
streamingRadius |
+Visual display threshold — tile is rendered once parsed | +
prefetchRadius |
+Background load threshold (> streamingRadius); auto = midpoint of stream/unload gap |
+
unloadRadius |
+Distance beyond which teardown is scheduled | +
priority |
+Load-order priority when multiple tiles are candidates | +
tileId |
+Debug identifier matching the manifest tile_id |
+
state |
+.unloaded → .parsing → .parsed → .unloading |
+
pendingUnloadSince |
+CFAbsoluteTime when tile first exceeded unloadRadius; 0 = in range |
+
loadTask |
+The in-flight Swift Task (cancelled on teardown) |
+
meshEntityId |
+The dedicated mesh-child entity ID; stored so the asset-loading timeout guard can force-close AssetLoadingGate if loadTextures() hangs |
+
hlodURL |
+URL of the HLOD proxy USDC, if present in the manifest | +
hlodEntityId |
+ECS entity holding the HLOD mesh; .invalid when unloaded |
+
hlodState |
+HLOD lifecycle: .unloaded → .loading → .loaded → .unloading |
+
hlodSwitchDistance |
+Camera distance beyond which the HLOD is shown | +
hlodLoadTask |
+In-flight HLOD load Task (cancelled before hlodState = .unloading) |
+
lodLevels |
+[TileLODLevel] — per-tile intermediate LOD entries parsed from manifest |
+
parseStartTime |
+CFAbsoluteTime set when tile CPU work begins (after remote fetch). The tile-parse watchdog uses this; stays 0 during remote download so the 60 s clock does not run on network latency |
+
lastHLODTransitionTime |
+CFAbsoluteTime of the most recent HLOD load/unload; used by secondaryRepresentationMinDwellSeconds guard to prevent faster-than-1 s HLOD flip-flop |
+
lastLODTransitionTime |
+CFAbsoluteTime of the most recent per-tile LOD load/unload; same dwell guard as HLOD |
+
update())After the mesh-level scan, a second pass handles tile stubs:
+.unloaded stub within effectivePrefetchRadius + 1.0 (not streamingRadius — tiles start loading before the camera enters the display zone):tileStreamingFrustum, padded by tileFrustumGatePadding = 20 m).shouldEvictGeometry(), runs texture shedding and evictLRU (capped at 8) before dispatch.maxConcurrentTileLoads (default 2) dispatched via loadTile(), subject to the tileParseMemoryBudgetMB (200 MB) in-flight gate.update())Three sub-passes each tick, capped at maxTileUnloadsPerUpdate (default 2) total teardowns:
unloadRadius — differentiates by state:.parsing (not yet visible) — cancelled immediately, no grace delay..parsed (visible geometry) — starts or checks the unload grace period (unloadGracePeriod = 3 s). Tile only tears down after being out of range for the full grace period; the timer resets if the camera returns inside unloadRadius..parsed tiles outside the octree query radius — same grace period logic..parsing tiles outside the octree query radius — cancelled immediately to prevent ghost geometry flashes on fast movement or teleports.effectivePrefetchRadius (auto: midpoint of stream/unload gap) so the parse completes before the camera reaches the visual zone, eliminating blank-screen pops on tile entry..parsed tile teardown stops rapid load/unload cycles at tile boundaries, which was the primary cause of flickering in large scenes.maxTileUnloadsPerUpdate = 2 — spreading GPU buffer releases across frames prevents a single-frame blank when many tiles leave range simultaneously.maxConcurrentTileLoads = 2 — two concurrent parses balance throughput for large scenes against RAM spike risk. Each parse calls MDLAsset(url:) on a full USDC file; the tileParseMemoryBudgetMB gate serialises naturally when a large tile saturates the budget.blockRenderLoop: false on all tile/LOD/HLOD loads — setEntityMeshAsync is called with blockRenderLoop: false so that AssetLoadingGate.isLoadingAny is not held true during the (potentially multi-second) parse. Without this, concurrent parses freeze visibleEntityIds updates and stall the render loop.lodHysteresisFactor (default 0.90) and hlodHysteresisFactor (default 0.90) add a 10% inner band so the camera must move meaningfully past a switch boundary before the current representation is unloaded. Without hysteresis, frame-to-frame distance jitter causes rapid load/unload cycles that freeze the engine.cancelPendingEntities before entity destruction — when unloadLODLevel or unloadHLOD tears down child entities, it first calls BatchingSystem.shared.cancelPendingEntities(_:) with the render descendant IDs, purging them from all pending batching queues. This prevents "entity is missing" errors when the batching tick tries to process additions for entities that were destroyed between event queuing and tick processing.notifyTileEntitiesResident replaces the event storm — tile/LOD/HLOD load completions call BatchingSystem.shared.notifyTileEntitiesResident(_:) instead of the former two-step queueResidencyEventsForRenderDescendants + notifyTileParsedEntities pairing. The single call directly registers entities in the batching system's pending additions and marks them for quiescence bypass, avoiding hundreds of individual AssetResidencyChangedEvent objects through SystemEventBus..auto streaming policy — tiles use the same admission gate as regular assets; unexpectedly large tiles are gracefully rejected and retried rather than crashing.tc.state == .parsing. If unloadTile ran mid-parse (state is .unloading), result is discarded and the pre-created child entity is cleaned up — stub never enters a "geometry missing" zombie state.defer slot release — releaseActiveTileLoad in defer frees the concurrency slot on all exit paths (success, failure, cancelled-state early return).removeTileComponent deregisters from streaming system — cancels in-flight loadTask and calls GeometryStreamingSystem.shared.unregisterTileEntity(entityId) to atomically remove the entity from all four tile tracking sets (loadedTileEntities, loadingTileEntities, activeTileLoads, meshEntityToTileEntity).reset() clears tile tracking sets — called by setEntityStreamScene() (via registerTiledScene) to reset interiorZone and firstRangeTimestamps so stale scene-level state from the previous scene does not persist into the new scene's streaming passes.hlodState = .unloading is set before hlodLoadTask.cancel(). The load-completion callback checks hlodState before marking .loaded; setting it first ensures the callback always discards a racing in-flight result.level.state = .unloading is set before level.loadTask?.cancel().loadTile's completion callback fires (state transitioning to .parsed), both unloadHLOD(entityId:) and unloadAllLODLevels(entityId:) are called. The full tile has taken over all distance bands; intermediate representations are no longer needed.loadedLODEntities tracking set — mirrors loadedTileEntities for the LOD layer. Allows reset() to cancel all in-flight LOD tasks and setEntityStreamScene() second-call safety to clear stale LOD entity IDs.AssetLoadingGate timeout — meshEntityId is stored in TileComponent so the timeout guard can call AssetLoadingState.shared.finishLoading(entityId: meshEntityId) without an O(n) map scan if loadTextures() hangs and the gate would otherwise remain open permanently.UntoldEngine has two separate LOD mechanisms that operate at different granularities. Understanding the distinction is important before reading either system's details.
+| + | Entity-level LOD (this document) | +Per-tile LOD (tile streaming) | +
|---|---|---|
| Unit | +Individual mesh entity (LODComponent) |
+Whole tile USDC file (TileLODLevel) |
+
| Control | +LODSystem — runs every frame |
+GeometryStreamingSystem.update() — runs per tick |
+
| Switch trigger | +Camera distance vs LODLevel.maxDistance |
+Camera distance vs TileLODLevel.switchDistance (with hysteresis) |
+
| Hysteresis | +5-unit inner band on finer-LOD transitions only | +lodHysteresisFactor (default 0.90 = 10% band) on active level |
+
| Meshes in memory | +All LOD levels GPU-resident simultaneously | +Only the active LOD level is loaded | +
| Use case | +Individual detailed objects (buildings, props) | +Tile-granularity intermediate representations for large scenes | +
| Content pipeline | +Separate OBJ/USDZ per LOD level, wired via LODComponent |
+Separate USDC per tile LOD, listed in manifest lod_levels array |
+
| Debug tagging | +LODComponent.currentLOD read by LOD debug renderer |
+TileLODTagComponent.levelIndex placed on render descendants |
+
Per-tile LOD documentation: docs/Architecture/tilebasedstreaming.md.
Each building entity has a LODComponent with an array of LODLevel entries sorted by detail:
Building Entity
+ LODComponent.lodLevels[0] → LOD0: highDetailMesh, maxDistance: 50.0
+ LODComponent.lodLevels[1] → LOD1: medDetailMesh, maxDistance: 100.0
+ LODComponent.lodLevels[2] → LOD2: lowDetailMesh, maxDistance: 200.0
+Each LODLevel tracks its own residencyState (.resident, .loading, .notResident, .unknown) and a url so the streaming system knows where to reload it from.
LODSystem.update()The system runs once per frame. Here's what happens for all 500 buildings:
+Step 1 — Get camera position +
+Step 2 — Query all LOD entities +
+Returns all 500 building entities in one shot. +Step 3 — For each building: updateEntityLOD()
This is the core loop. Three sub-steps per building:
+calculateDistance()Takes the building's local AABB bounding box, finds its center, transforms it to world space via WorldTransformComponent.space, then computes the straight-line distance to the camera.
selectLODLevel()Applies lodBias to the distance (default 1.0, so no change), then walks through lodLevels in order, comparing against each level's maxDistance:
adjustedDistance = distance * lodBias
+
+If adjustedDistance ≤ 50 → desiredLOD = 0 (high detail)
+If adjustedDistance ≤ 100 → desiredLOD = 1 (medium)
+If adjustedDistance ≤ 200 → desiredLOD = 2 (low)
+Beyond all thresholds → desiredLOD = 2 (lowest available)
+Hysteresis: When switching to a finer LOD (e.g. camera approaching, LOD2→LOD1), the threshold is tightened by 5.0 units. This prevents flickering when the camera hovers right at a boundary. Switching to coarser LODs has no penalty — it happens immediately.
resolveActualLOD()The desired LOD may not be resident yet (e.g. it's still streaming in). So:
+Is desiredLOD mesh resident?
+ YES → use it (isUsingFallback = false)
+ NO → findFallbackLOD():
+ Try coarser LODs first (LOD2, LOD3...)
+ Then try finer LODs (LOD0...)
+ If nothing → stay at currentLOD
+This means a building 30m away that wants LOD0 but hasn't finished loading will temporarily show LOD1 or LOD2 — always something visible, never a pop-in hole.
+applyLOD()This is the write step. Runs inside withWorldMutationGate to be thread-safe.
If newLOD == currentLOD and no transition in progress → skip (no-op)
+
+Otherwise:
+ - Update lodComponent.currentLOD = newLOD
+ - Copy lodLevels[newLOD].mesh → renderComponent.mesh
+ - Generate meshAssetID: "<url>_LOD<index>"
+ - Store in lodComponent.activeMeshAssetID (used by batching system)
+ - If LOD actually changed → emit EntityLODChangedEvent to SystemEventBus
+The renderComponent.mesh swap is the handoff to the renderer — whatever mesh array sits there is what gets drawn next frame.
Given a camera standing near one end of the block:
+| Distance | +Buildings | +Desired LOD | +Typical Outcome | +
|---|---|---|---|
| 0–50m | +~20 | +LOD0 | +High detail meshes | +
| 50–100m | +~80 | +LOD1 | +Medium meshes | +
| 100–200m | +~150 | +LOD2 | +Low meshes | +
| 200m+ | +~250 | +LOD2 (last) | +Lowest available | +
The system processes all 500 in sequence, but buildings where LOD hasn't changed are early-exited with no mutation (the newLOD == previousLODIndex guard). In a stable scene, the vast majority of buildings hit that fast path.
activeMeshAssetID is the bridge to the batching system. When a LOD switches, the new asset ID tells the batcher to move that entity into a different batch group.EntityLODChangedEvent on SystemEventBus is how downstream systems (geometry streaming, batching) learn that a switch happened — they react to the event rather than polling.transitionProgress, previousLOD) but enableFadeTransitions defaults to false, so currently all switches are instant.Prior to this integration, LOD and OOC were mutually exclusive: assets that qualified for out-of-core streaming would bypass LOD group detection, causing each LOD0/LOD1/LOD2 object to become an independent stub entity.
How it works now:
+setEntityMeshAsync runs LOD group detection before the OOC branching decision. When the asset both qualifies for OOC streaming and contains LOD groups, the LOD+OOC path runs:
Registration — one entity per LOD group (not one per MDLObject). Each entity gets a LODComponent with stub LODLevels (empty mesh, .notResident) and a StreamingComponent(.unloaded).
CPU registry — ProgressiveAssetLoader.cpuLODRegistry[groupEntityId][lodIndex] stores a CPUMeshEntry for every LOD level. The MDLAsset is retained so CPU buffers remain valid.
GPU upload — when GeometryStreamingSystem picks up the entity (.unloaded → in streaming range), it calls uploadActiveLODFromCPU. This uploads all LOD levels in one pass from the CPU registry, marks each LODLevel.residencyState = .resident, and sets renderComponent.mesh to the level appropriate for the current camera distance.
LOD switching after load — LODSystem.applyLOD continues to work as normal: it reads from lodComponent.lodLevels[n].mesh (now populated) and swaps renderComponent.mesh. No additional streaming requests are needed — all levels are already GPU-resident.
Cold re-hydration — if releaseWarmAsset was called on the root and a group entity re-enters streaming range, rehydrateColdAsset re-parses the USDZ, re-runs LOD detection, and rebuilds cpuLODRegistry entries before uploadActiveLODFromCPU runs.
Result: the caller sets any MeshStreamingPolicy and LOD assets always get proper LODComponent wiring. The mutual exclusivity that required users to choose between OOC and LOD is eliminated.
MeshResourceManager is a singleton that acts as the shared GPU memory layer for mesh assets. It caches entire USDZ files, tracks which entities use which meshes via reference counting, and evicts unused geometry under memory pressure.
MeshResourceManager is a singleton (shared) that manages two key dictionaries:
resources: URL -> MeshResource (the cache — one entry per USDZ file)
+entityToMesh: EntityID -> (URL, name) (which entity uses which mesh)
+A MeshResource holds all meshes from a single USDZ file, keyed by asset name:
city_block_A.usdz -> MeshResource {
+ meshesByName: { "building_01": [...], "window_01": [...], "door_01": [...] }
+ refCountByName: { "building_01": 12, "window_01": 48, "door_01": 12 }
+ totalMemorySize: 42_000_000 // bytes on GPU
+ lastAccessFrame: 1042
+}
+Say entity e001 needs "building_01" from city_block_A.usdz:
Step 1 — Cache check (getCachedMesh): nothing cached yet → miss.
Step 2 — Single-flight gate (waitForExistingLoadOrBecomeLoader):
+- First caller wins: it is designated the loader and gets true.
+- If 10 other entities request the same file simultaneously, they all queue up as waiters inside inFlightLoadWaiters[url]. They suspend via CheckedContinuation — no busy-waiting.
Step 3 — Parse & upload (Mesh.loadSceneMeshesAsync):
+- The entire USDZ is parsed once. Every mesh in the file is uploaded to GPU buffers.
+- Result is a [[Mesh]] — one inner array per named asset.
Step 4 — Build the dictionary and cache it: +
meshesByName["building_01"] = [mesh1, mesh2, ...] // LODs or submeshes
+meshesByName["window_01"] = [mesh3, ...]
+meshesByName["door_01"] = [mesh4, ...]
+totalMemorySize = 42 MB
+resources[city_block_A.usdz].
+Step 5 — Wake waiters (finishInFlightLoad):
+- All 10 suspended callers are resumed with false (they don't load — file is already cached).
+- Each then calls getCachedMesh and gets their mesh instantly.
When the streaming system decides entity e001 will render "building_01":
This:
+1. Releases any previous mesh the entity held (safety cleanup).
+2. Records entityToMesh[e001] = (city_block_A.usdz, "building_01").
+3. Increments refCountByName["building_01"] from 0 → 1.
After 500 entities are assigned across the city block:
+refCountByName: { "building_01": 45, "window_01": 180, "streetlight": 60, ... }
+totalRefCount = 500
+isInUse = true ← cannot be evicted
+When an entity scrolls out of view and is culled:
+ +This removes entityToMesh[e099] and decrements refCountByName["building_01"] from 45 → 44.
Three eviction strategies exist:
+| Method | +When to Use | +
|---|---|
evict(url:) |
+Force-remove one specific file (only if refCount == 0) |
+
evictUnused() |
+Sweep all files with zero references | +
evictToFreeMemory(targetBytes:) |
+LRU — evict oldest-accessed files first until targetBytes freed |
+
For a city block, evictToFreeMemory is most useful. Say GPU budget is exceeded by 80 MB:
candidates = resources where totalRefCount == 0
+ sorted by lastAccessFrame ascending (oldest first)
+
+Evict city_block_C.usdz → frees 38 MB (last seen frame 200)
+Evict city_block_D.usdz → frees 44 MB (last seen frame 310)
+Total freed: 82 MB ✓
+For each evicted mesh, mesh.cleanUp() is called to free the Metal GPU buffers.
lastAccessFrame is updated every time a mesh is accessed (cache hit) or retained, so actively-used files naturally survive LRU pressure.
All state is protected by a concurrent DispatchQueue with the readers-writers pattern:
+- Reads use accessQueue.sync — concurrent reads are fine.
+- Writes use accessQueue.sync(flags: .barrier) — exclusive access, blocks concurrent reads.
This makes the manager safe to call from multiple streaming tasks loading different USDZ files in parallel.
+Frame 0: Scene starts loading
+ → 5 USDZ files queued
+ → Each file: one task becomes loader, others wait
+ → All 5 files parsed, all meshes cached in GPU memory
+
+Frame 1–N: Entities stream in/out of view
+ → retain() as entities enter view frustum
+ → release() as entities leave
+ → refCounts track exactly how many entities use each mesh
+
+Memory pressure detected:
+ → evictToFreeMemory() walks LRU list
+ → Only files with refCount==0 are eligible
+ → GPU buffers freed via cleanUp()
+ → Files still in view are untouched
+The key insight: one disk parse per USDZ file, no matter how many entities share its meshes. Reference counting prevents premature eviction, and LRU ensures the least-recently-seen geometry is freed first under memory pressure.
+MeshResourceManager is used by three systems:
Owns the full lifecycle:
+- Updates currentFrame each tick to keep LRU timestamps fresh.
+- Calls loadMesh + retain when an entity streams into view.
+- Calls release when an entity streams out of view.
+- Calls evictUnused() to free GPU memory.
+- Reads getStats() for diagnostics and memory budgeting.
Calls cacheLoadedMeshes(url:meshArrays:) when entities are registered into the scene, so meshes are already in the cache before the streaming system requests them. Also calls release when entities are unregistered.
Reads getStats() only, likely for a debug overlay or performance HUD.
UntoldEngine still uses an out-of-core (OOC) geometry path, but it is now part of the tile streaming architecture, not a general public workflow for arbitrary standalone entities.
+The public entry point for streamed geometry is:
+ +Inside that pipeline, large tiles may be classified as OOC during setEntityMeshAsync(streamingPolicy: .auto).
| System | +Responsibility | +
|---|---|
RegistrationSystem |
+Parses the tile payload and chooses fullLoad vs OOC registration |
+
ProgressiveAssetLoader |
+Stores CPU-resident CPUMeshEntry records and warm/cold rehydration context |
+
GeometryStreamingSystem |
+Uploads and evicts tile-owned OCC stubs by distance and budget | +
MeshResourceManager still serves disk/cache-backed mesh loads for non-OOC paths, but OOC residency is fundamentally driven by ProgressiveAssetLoader.
When a large tile is routed to the OOC path:
+StreamingComponent(state: .unloaded).ProgressiveAssetLoader.GeometryStreamingSystem uploads those stubs incrementally as the camera approaches.Those StreamingComponent stubs are valid only when they are descendants of a TileComponent entity. GeometryStreamingSystem.loadMesh(...) enforces that ownership rule.
The runtime still exposes MeshStreamingPolicy, but the architecture has moved:
setEntityMeshAsync(..., streamingPolicy: .auto) is fine for normal always-resident assetssetEntityMeshAsync(..., streamingPolicy: .immediate) forces full uploadsetEntityStreamScene(...) is the supported public streaming pathStreamingComponent and enableStreaming(...) are internal tile/OOC mechanismsThat means the old pattern:
+ +is no longer the recommended app-level workflow. The engine now expects streamed geometry to come from a tiled scene manifest.
+loadTile(entityId:) resolves the tile URL, then calls:
setEntityMeshAsync(
+ entityId: meshEntityId,
+ filename: ...,
+ withExtension: ...,
+ streamingPolicy: .auto,
+ blockRenderLoop: false
+)
+The asset admission/classification stage decides whether this tile becomes:
+For an OOC tile, registerProgressiveStubEntity(...) creates one ECS stub per mesh leaf:
StreamingComponent starts as .unloadedEach stub stores a CPUMeshEntry in ProgressiveAssetLoader, including:
MDLObjectThis is the warm CPU copy used for re-upload.
+On streaming ticks, GeometryStreamingSystem evaluates tile-owned OCC stubs:
nearBandMaxConcurrentLoadsmaxConcurrentLoadsSuccessful upload transitions the stub from .loading to .loaded, increments the parent tile's OCC-ready counters, and emits normal residency events for batching.
When the camera moves away or geometry pressure rises:
+unloadMesh(entityId:) clears RenderComponent.meshMemoryBudgetManager unregisters the GPU allocationThis is why a normal OOC re-approach can re-upload without a disk read.
+On critical pressure, the engine may release warm CPU state:
+ +This frees the retained MDLAsset tree and CPU mesh buffers but preserves the rehydration context. A later re-approach reparses the root asset once and rebuilds the CPU registry.
OOC is just one representation inside the broader tile system:
+That is why the runtime documentation should describe OOC as an implementation detail of tile streaming rather than a separate public scene-loading mode.
+ProgressiveAssetLoader is the CPU residency layer.GeometryStreamingSystem is the GPU residency scheduler.setEntityStreamScene(...) is the supported public streaming API surface.ProgressiveAssetLoader is a CPU registry — its sole responsibility is storing CPUMeshEntry records for out-of-core stub entities and serving them to GeometryStreamingSystem on demand.
++Note: This document describes the current architecture. The earlier tick-based progressive loader (per-frame job queue,
+PendingObjectItem,enqueue(job),tick()processing N meshes per frame) was replaced by the out-of-core stub system.tick()is retained as a no-op for call-site compatibility only.
When setEntityMeshAsync routes an asset through the out-of-core path it stores CPU-side geometry in one of two registries depending on whether the asset contains LOD groups:
cpuMeshRegistry ([EntityID: CPUMeshEntry]) — one entry per stub entity (regular OOC assets)cpuLODRegistry ([EntityID: [Int: CPUMeshEntry]]) — one entry per LOD level per LOD group entity (LOD+OOC assets)Both registries store CPUMeshEntry records:
struct CPUMeshEntry {
+ let object: MDLObject // MDLMesh with CPU-heap vertex/index data
+ let vertexDescriptor: MDLVertexDescriptor
+ let textureLoader: TextureLoader
+ let device: MTLDevice
+ let url: URL
+ let filename: String
+ let withExtension: String
+ let uniqueAssetName: String // "Hull_A#42" — stable across load cycles
+ let estimatedGPUBytes: Int // vertex + index bytes; used for pre-emptive budget reservation
+}
+Entries are keyed by child entity ID. GeometryStreamingSystem retrieves them via retrieveCPUMesh(for:) when an entity enters streaming range, copies the MDL buffers into Metal-backed buffers, and registers a RenderComponent. The CPU entry is never removed on unload — re-approaching an evicted entity re-uploads from RAM with no disk I/O.
When a USDZ asset contains LOD groups (top-level objects named Tree_LOD0, Tree_LOD1, etc.) and qualifies for OOC streaming, setEntityMeshAsync takes the LOD+OOC path instead of the per-stub path:
LODComponent with stub LODLevels — empty mesh arrays, residencyState: .notResidentCPUMeshEntry is stored in cpuLODRegistry[groupEntityId][lodIndex] for each LOD level// LOD+OOC: per-level CPU entries, keyed by (group entity ID, LOD index)
+cpuLODRegistry[treeEntityId] = [
+ 0: CPUMeshEntry(object: tree_LOD0_MDLObject, uniqueAssetName: "Tree_LOD0", ...),
+ 1: CPUMeshEntry(object: tree_LOD1_MDLObject, uniqueAssetName: "Tree_LOD1", ...),
+ 2: CPUMeshEntry(object: tree_LOD2_MDLObject, uniqueAssetName: "Tree_LOD2", ...),
+]
+GeometryStreamingSystem detects the LOD+OOC path via hasCPULODData(for:) and calls uploadActiveLODFromCPU instead of uploadFromCPUEntry. This uploads all LOD levels from the CPU registry in one pass, then sets the render component to the level appropriate for the current camera distance. Subsequent LOD switches are handled by LODSystem which swaps renderComponent.mesh from the already-resident lodComponent.lodLevels array — no additional streaming needed until the entity is evicted and re-enters range.
MDLMeshBufferDataAllocator (used by parseAssetAsync) backs all CPU buffers via the MDLAsset container. If the asset is released, all child MDLMesh CPU pointers become dangling.
ProgressiveAssetLoader solves this with rootAssetRefs:
storeAsset(_:for:) pins the MDLAsset to the root entity ID. It stays alive until removeOutOfCoreAsset(rootEntityId:) is called at entity destruction time.
storeAsset(_:for:) immediately fires a background Task at .userInitiated priority to call loadTextures() before any mesh enters streaming range:
func storeAsset(_ asset: MDLAsset, for rootEntityId: EntityID) {
+ // Pin the asset and create the per-asset texture lock.
+ lock.lock()
+ rootAssetRefs[rootEntityId] = asset
+ assetTextureLocks[rootEntityId] = NSLock()
+ lock.unlock()
+ // Kick off background prewarm immediately.
+ prewarmTexturesAsync(for: rootEntityId)
+}
+The prewarm task acquires the per-asset texture lock, calls ensureTexturesLoaded, and releases the lock — all off the critical path. By the time the first mesh enters streaming range, loadTextures() has typically already completed, so the first-upload path sees a no-op ensureTexturesLoaded call and zero lock wait.
activePrewarmRoots tracks which roots have an in-flight prewarm task. GeometryStreamingSystem queries isPrewarmActive(for:) in the dispatch loop and defers uploading entities for that root until the prewarm completes. This prevents the first batch of uploads from blocking on the texture lock for the full remaining prewarm duration.
MDLAsset is not thread-safe. Two GeometryStreamingSystem tasks uploading different meshes from the same asset concurrently can race during loadTextures(). ProgressiveAssetLoader prevents this with a per-asset NSLock:
storeAsset creates the lock alongside the asset reference. Every upload task brackets only ensureTexturesLoaded with the lock — the lock is released before makeMeshesFromCPUBuffers:
ProgressiveAssetLoader.shared.acquireAssetTextureLock(for: rootId)
+ProgressiveAssetLoader.shared.ensureTexturesLoaded(for: rootId)
+ProgressiveAssetLoader.shared.releaseAssetTextureLock(for: rootId)
+// makeMeshesFromCPUBuffers runs without the lock — MDLAsset is read-only after loadTextures()
+After loadTextures() completes the MDLAsset is in a stable read-only state. Concurrent makeMeshesFromCPUBuffers calls from the same asset are safe without the lock, so all three upload slots can proceed in parallel once the prewarm is done.
Only the ensureTexturesLoaded call is serialized per asset. Meshes from different assets upload concurrently without any contention.
loadTextures()Large assets skip asset.loadTextures() at parse time to avoid the OOM risk of decompressing all textures before the app is interactive. The call is deferred via ensureTexturesLoaded:
func ensureTexturesLoaded(for rootEntityId: EntityID) {
+ // Must be called while per-asset texture lock is held.
+ // Calls asset.loadTextures() exactly once per asset lifetime.
+}
+assetTexturesLoaded: Set<EntityID> ensures the call happens exactly once. In normal operation the prewarm task wins the race and marks the asset loaded before any upload task reaches ensureTexturesLoaded, making the upload-path call a no-op.
| Method | +Purpose | +
|---|---|
storeCPUMesh(_:for:) |
+Store a CPUMeshEntry keyed by child entity ID (regular OOC) |
+
retrieveCPUMesh(for:) |
+Fetch the entry for GeometryStreamingSystem upload (regular OOC) |
+
removeCPUMesh(for:) |
+Remove a single entry (rarely needed; prefer removeOutOfCoreAsset) |
+
storeCPULODMesh(_:for:lodIndex:) |
+Store a CPUMeshEntry for one LOD level of a LOD group entity |
+
retrieveCPULODMesh(for:lodIndex:) |
+Fetch the entry for a specific LOD level | +
retrieveAllCPULODMeshes(for:) |
+Fetch all LOD-level entries for a group entity | +
hasCPULODData(for:) |
+Returns true if the entity was registered via the LOD+OOC path |
+
removeCPULODEntry(for:) |
+Remove all LOD entries for a group entity | +
storeAsset(_:for:) |
+Pin an MDLAsset, create its per-asset texture lock, and kick off background prewarm |
+
isPrewarmActive(for:) |
+Returns true while the background prewarm task holds the texture lock for this root |
+
registerChildren(_:for:) |
+Associate child entity IDs with a root for bulk cleanup | +
acquireAssetTextureLock(for:) |
+Lock before calling ensureTexturesLoaded |
+
releaseAssetTextureLock(for:) |
+Unlock immediately after ensureTexturesLoaded — before GPU upload work |
+
ensureTexturesLoaded(for:) |
+Call loadTextures() exactly once per asset (must hold texture lock) |
+
removeOutOfCoreAsset(rootEntityId:) |
+Release all CPU entries (both registries) + MDLAsset for a destroyed root entity | +
cancelAll() |
+Release everything — use on scene reset or test teardown | +
tick() |
+No-op stub; retained for call-site compatibility | +
setEntityMeshAsync (out-of-core path — regular OOC)
+ │
+ ├─ parseAssetAsync() → MDLAsset in CPU RAM (no GPU spike)
+ ├─ registerProgressiveStubEntity() → N ECS stubs, StreamingComponent(.unloaded)
+ ├─ storeCPUMesh(entry, for: childId) × N → cpuMeshRegistry
+ ├─ storeAsset(asset, for: rootId) → rootAssetRefs, assetTextureLocks
+ │ └─ prewarmTexturesAsync() → background Task: acquireLock / loadTextures() / releaseLock
+ ├─ registerChildren(childIds, for: rootId)
+ └─ completion(true) → GeometryStreamingSystem picks up stubs automatically (always running)
+
+setEntityMeshAsync (out-of-core path — LOD+OOC)
+ │
+ ├─ parseAssetAsync() → MDLAsset in CPU RAM
+ ├─ detectImportedLODGroups() → N LOD groups detected
+ ├─ (per group) createEntity + LODComponent(stubs) + StreamingComponent(.unloaded)
+ ├─ storeCPULODMesh(entry, for: groupId, lodIndex:) × (N groups × L levels) → cpuLODRegistry
+ ├─ storeAsset(asset, for: rootId)
+ │ └─ prewarmTexturesAsync() → background Task: acquireLock / loadTextures() / releaseLock
+ ├─ registerChildren(groupEntityIds, for: rootId)
+ └─ completion(true)
+
+GeometryStreamingSystem (adaptive tick: 16 ms during backlog, 100 ms steady-state)
+ │
+ ├─ isPrewarmActive(rootId)? → YES → defer all entities for this root (slots stay free)
+ │
+ ├─ entity within streamingRadius && state == .unloaded
+ │ ├─ hasCPULODData? → YES → uploadActiveLODFromCPU()
+ │ │ ├─ retrieveAllCPULODMeshes(for: entityId)
+ │ │ ├─ acquireAssetTextureLock / ensureTexturesLoaded / releaseAssetTextureLock
+ │ │ ├─ makeMeshesFromCPUBuffers() × L levels ← lock released; parallel uploads safe
+ │ │ ├─ LODComponent.lodLevels[i].residencyState = .resident for each uploaded level
+ │ │ └─ registerRenderComponent() at distance-appropriate LOD
+ │ │
+ │ └─ hasCPULODData? → NO → retrieveCPUMesh / uploadFromCPUEntry (regular OOC)
+ │ ├─ acquireAssetTextureLock / ensureTexturesLoaded / releaseAssetTextureLock
+ │ ├─ makeMeshesFromCPUBuffers() ← lock released; parallel uploads safe
+ │ └─ registerRenderComponent()
+ │
+ └─ entity beyond unloadRadius && state == .loaded
+ └─ render.mesh = [] (cpu entries kept — re-upload from RAM on re-approach)
+
+destroyAllEntities / scene reset
+ └─ removeOutOfCoreAsset(rootEntityId:) → frees both CPU registries + MDLAsset
+CPU RAM: all leaf meshes' MDLMesh vertex/index data — always resident
+GPU RAM: only entities within streamingRadius — uploaded on demand
+Disk: read exactly once at parse time
+This trades a modest CPU-RAM footprint for predictable GPU memory usage and zero-latency re-uploads after eviction.
+Call removeOutOfCoreAsset(rootEntityId:) when destroying a root entity to free its CPU-heap geometry and texture-lock state:
destroyAllEntities does not call this automatically — you must call it explicitly if you are managing entity lifetimes outside the engine's destruction path.
For full teardown (scene resets, tests):
+ + + + + + + + + + + + + + +The rendering system's job is to take the current set of visible entities and turn them into pixels on screen every frame. It does this in two distinct phases: pre-render compute (GPU culling and sorting) followed by a render graph — a dependency-ordered DAG of passes that each write into shared textures until the final image lands on the drawable.
+The entry point is UpdateRenderingSystem(in view: MTKView), called once per frame from the MTKView draw loop.
Before any rendering begins, the system needs to know which entities are visible. This is managed through a triple-buffer called tripleVisibleEntities:
The key insight here is that visibleEntityIds is not rebuilt from scratch each frame. It is the result of the previous frame's GPU frustum cull — a compute pass that ran last frame and wrote its output into the triple-buffer. The current frame reads that result and uses it immediately.
Why triple-buffered? The GPU may still be consuming last frame's cull output while the CPU is already preparing the next frame. Three slots prevent read/write races across overlapping frames.
+While loading: When AssetLoadingGate.shared.isLoadingAny is true, the snapshot step is skipped entirely. The last-known-good visibleEntityIds is reused. This prevents reading from ECS storage while asset loading is mutating it on a background thread.
The engine allows at most 3 command buffers in flight at once (matching the triple-buffer count). The semaphore blocks the CPU if the GPU is still consuming all three slots.
+acquireUniformFrameSlot() returns the index into the per-frame uniform buffer ring. Because the CPU writes entity transforms and camera matrices into these buffers while the GPU reads them, each in-flight frame needs its own slot to avoid corruption.
Before any uniforms are uploaded, dirty transforms are propagated down the scene graph. An entity whose parent moved needs its WorldTransformComponent updated before the model matrix is sent to the GPU. This runs lazily — only if something was marked dirty since the last frame.
These three compute dispatches run before any render encoder is opened. They prepare data that the render passes will consume.
+performFrustumCulling(commandBuffer:)A compute shader tests every entity's axis-aligned bounding box (EntityAABB) against the camera's 6 frustum planes. Entities outside the frustum are excluded.
The result is written into tripleVisibleEntities — for the next frame. So culling is always one frame behind rendering. This is an intentional latency trade-off: GPU-driven culling is far faster than CPU culling, and one frame of lag is imperceptible.
In addition to writing the GPU visibility result, executeFrustumCulling stores the current-frame frustum in the module-level variable currentFrameFrustum. This frustum is the padded, CPU-side version built from the view-projection matrix. It is read later in the same frame by the batched render passes for cluster-level AABB culling of BatchGroups (see G-Buffer Passes and Shadow Passes).
For XR, a reduce-scan variant runs the test against both eyes simultaneously.
+executeGaussianDepth(commandBuffer)For entities carrying a GaussianComponent (3D Gaussian splat data), a compute pass calculates the camera-space depth of each splat. This depth value is used as the sort key in the next step.
executeBitonicSort(commandBuffer)A GPU bitonic sort reorders the Gaussian splats back-to-front by depth. Gaussian splats must be composited in this order for correct alpha blending. The sort runs entirely on the GPU and its output feeds directly into the Gaussian render pass later in the graph.
+buildGameModeGraph()Rather than hard-coding a linear sequence of passes, the engine constructs a directed acyclic graph (DAG) of RenderPass nodes each frame:
struct RenderPass {
+ let id: String
+ var dependencies: [String]
+ var execute: (MTLCommandBuffer) -> Void
+}
+Each pass declares which other passes must complete before it can run. buildGameModeGraph() assembles these nodes into a dictionary and returns it. Nothing executes yet — this is purely declarative.
The full graph for a typical frame looks like this:
+environment/grid
+ └── shadow
+ └── batchedShadow
+ └── model ──────────────────────────── gaussian
+ └── batchedModel │
+ ├── ssao │
+ └── lightPass │
+ └── transparency │
+ └── spatialDebug
+ └── [post-processing chain]
+ └── precomp ◄── (gaussian joins here)
+ └── look
+ └── outputTransform
+The graph always starts with a background pass whose type depends on the platform and rendering mode:
+| Context | +Pass | +Purpose | +
|---|---|---|
| macOS/iOS with HDR sky | +environment |
+Renders the IBL skybox cubemap | +
| macOS/iOS without HDR | +grid |
+Renders the editor debug grid | +
| XR passthrough (mixed) | +(none) | +Camera feed is the background | +
| XR full immersion | +environment |
+Skybox inside the headset | +
This pass has no dependencies — it is always the root of the graph.
+Both passes render scene geometry from the directional light's point of view into a shadow map depth texture. No color is written — only depth. The renderer checks entityToBatch and routes each entity to the appropriate pass:
+- Regular entities → shadowExecution
+- Batched entities → batchedShadowExecution
batchedShadowExecution uses cluster-level frustum culling: it calls visibleBatchGroupsSnapshot() which tests each BatchGroup's precomputed world-space AABB against currentFrameFrustum. Only groups whose AABB intersects the frustum are submitted. This replaces the previous entity→batchId derivation and operates at batch-group granularity — one AABB test per group instead of one per entity.
The shadow map produced here is consumed later by lightPass.
This is the core of the deferred rendering pipeline. Entities do not produce a shaded color here — they write raw surface data into multiple render targets (the G-Buffer):
+modelExecution iterates visibleEntityIds. For each entity that is not batched:
+- Binds vertex/index buffers
+- Uploads the model matrix, normal matrix, and camera uniforms into the current in-flight frame slot
+- Issues a draw call per mesh submesh
batchedModelExecution uses cluster-level frustum culling: it calls visibleBatchGroupsSnapshot() which tests each BatchGroup's precomputed world-space AABB against currentFrameFrustum using isAABBInFrustum. The result — groups whose AABB intersects the frustum — is cached for the frame and shared with batchedShadowExecution. Each surviving group is then submitted as a single draw call with its merged vertex and index buffers.
ssaoOptimizedExecution reads the G-Buffer normals and depth and produces a screen-space ambient occlusion texture. Blurring is handled internally — no separate blur nodes appear in the graph.
lightExecution is where the entity first appears fully lit. It reads all four G-Buffer textures plus the shadow map and SSAO texture and combines them into a single HDR scene color texture using the deferred lighting algorithm. This is a full-screen quad pass — geometry is never touched again after the G-Buffer step.
++Why deferred? Deferred rendering means the lighting cost scales with the number of lit pixels, not the number of geometry draw calls × number of lights. Complex scenes with many overlapping objects benefit greatly because each pixel is only shaded once, regardless of how many triangles projected onto it.
+
Transparent materials cannot go through the G-Buffer — they require alpha blending which deferred rendering cannot express per-fragment. These entities are rendered forward in a separate pass on top of the deferred lit scene color. They depend on lightPass being complete so they composite correctly against the opaque scene.
Draws wireframe AABB overlays for debug purposes. Runs last in the geometry chain so it draws on top of everything.
+Renders the back-to-front-sorted Gaussian splats using the indices produced by the bitonic sort. This pass depends on "model" because it needs the depth buffer that was populated during the G-Buffer model pass — splats use that depth to correctly composite against solid geometry.
+Note that Gaussian does not depend on lightPass, transparency, or the post-processing chain. It runs in parallel with those in the dependency graph and merges back at precomp.
spatialDebug → depthOfField → chromatic → bloomThreshold
+ → blur_hor_1 → blur_ver_1 → blur_hor_2 → blur_ver_2
+ → bloomComposite → vignette
+postProcessingEffects() builds this chain dynamically inside buildGameModeGraph(). Each effect reads from the previous pass's output texture and writes to its own.
Fast path: If every effect (BloomThresholdParams, VignetteParams, ChromaticAberrationParams, DepthOfFieldParams) is disabled, the entire chain is replaced by a single bypass pass that points the post-process descriptor at the deferred output texture directly. This avoids allocating ~142 MB of intermediate render targets that would be unused.
The number of blur iterations is driven by BloomThresholdParams.shared.enabled — when bloom is on, two horizontal/vertical pairs are dispatched; when off, zero. The loop that generates blur nodes in the graph is:
So the graph topology literally changes based on whether bloom is enabled.
+This is the convergence point of the two parallel tracks. The post-processed scene color and the Gaussian splat render both arrive here and are composited into a single texture. Neither track can be finalized without the other.
+Applies lift/gamma/gain color correction and optional LUT-based grading to the composited image.
+Tone maps the HDR scene color into the display's color space (SDR or EDR depending on the target). This is the terminal node of the graph — its output texture is what gets presented to the drawable.
+With the graph assembled, the engine sorts and executes it:
+let sortedPasses = try! topologicalSortGraph(graph: graph)
+executeGraph(graph, sortedPasses, commandBuffer)
+topologicalSortGraph performs a depth-first search over the dependency edges and returns a [String] of pass IDs in a valid execution order — every pass appears after all its dependencies.
executeGraph iterates that list and calls each pass's execute closure, encoding Metal render or compute commands into the shared commandBuffer. All passes share one command buffer, so Metal can pipeline them efficiently on the GPU.
After the render graph finishes, the depth texture produced during the G-Buffer model pass is downsampled into a hierarchical Z-buffer mip pyramid. This feeds next frame's occlusion culling — a coarse depth mip level can quickly reject large occluded objects before the fine cull.
+This is intentionally scheduled here, after the render graph and before commit(), so the HZB is built from the freshest depth available and ready for the next frame's culling compute dispatch.
The completion handler fires on the GPU thread when the command buffer finishes executing:
+commandBufferSemaphore.signal() — frees one slot, allowing the CPU to encode the next frameneedsFinalizeDestroys = true — deferred ECS entity removal can proceed safely now that the GPU is done with this frame's dataMemoryBudgetManager.shared.markUsed(entityIds:) — records which entities were rendered so the memory budget manager knows what to keep resident and what to evict[CPU] snapshotVisibleEntities (from last frame's cull)
+[CPU] wait on semaphore / acquire uniform slot
+[CPU] propagate dirty transforms
+ │
+ ▼
+[GPU compute] frustumCulling → writes next frame's visibleEntityIds
+[GPU compute] gaussianDepth → depth per splat
+[GPU compute] bitonicSort → sort splats back-to-front
+ │
+ ▼
+[CPU] buildGameModeGraph() → construct render pass DAG
+[CPU] topologicalSortGraph() → linearize pass order
+ │
+ ▼
+[GPU render] environment/grid
+[GPU render] shadow + batchedShadow (depth from light POV)
+[GPU render] model + batchedModel (entity → G-Buffer)
+[GPU render] ssao (occlusion from G-Buffer)
+[GPU render] lightPass (entity appears fully lit)
+[GPU render] transparency (forward-rendered alphas)
+[GPU render] spatialDebug (debug overlays)
+[GPU render] gaussian (sorted splats)
+[GPU render] post-processing chain (DOF, bloom, vignette)
+[GPU render] precomp (merge scene + splats)
+[GPU render] look (color grading)
+[GPU render] outputTransform (tone map → drawable)
+[GPU compute] buildHZB (depth pyramid for next frame)
+ │
+ ▼
+[CPU] present drawable + commit
+[GPU→CPU callback] signal semaphore, mark memory used
+A fixed sequence of if statements works fine until the graph needs to change — when post-processing is disabled, when XR changes the base pass, or when the number of bloom blur iterations varies based on settings. A render graph makes these variations declarative: each pass states what it needs, and the topology sorts itself. Adding a new pass means adding one RenderPass node with its dependency list — the rest of the system adapts automatically.
It also makes the dependency structure explicit and auditable. If a pass reads a texture produced by another pass, that relationship is encoded as a graph edge rather than buried in execution order assumptions.
+ + + + + + + + + + + + + +This document describes how the current streaming architecture manages geometry across three layers:
+| Layer | +Owns | +
|---|---|
GeometryStreamingSystem |
+Distance-based residency decisions | +
ProgressiveAssetLoader |
+Warm/cold CPU mesh state for tile-owned OOC assets | +
MeshResourceManager |
+Shared GPU mesh cache for non-OOC and disk-backed reload paths | +
For eager loads, MeshResourceManager owns the shared mesh data:
loadMesh(url:meshName:) returns cached or freshly loaded meshesretain(...) increments residency ownership for an entityrelease(entityId:) decrements the reference countevictUnused() frees GPU data when the ref count reaches zeroFor large streamed tiles, ProgressiveAssetLoader owns the CPU source data:
CPUMeshEntry stores the parsed CPU mesh buffersGeometryStreamingSystem uploads those buffers on demandsetEntityStreamScene(...) registers lightweight TileComponent stubs only, parented under the supplied root entity. No geometry is resident yet.
When a tile enters prefetch range, loadTile(entityId:) parses the tile payload.
At that point the runtime chooses one of two outcomes:
+StreamingComponent stubs are registered and backed by CPUMeshEntryFor OOC stubs, GeometryStreamingSystem.loadMesh(...):
ProgressiveAssetLoader when warmFor eager/disk-backed meshes, the load path uses MeshResourceManager:
loadMesh(...)retain(...)copyWithNewUniformBuffers()RenderComponentThe copied uniform buffers are per-entity, but the underlying cached geometry is shared.
+When geometry leaves range or memory pressure rises:
+unloadMesh(...) clears the entity's live mesh referenceMeshResourceManager.release(entityId:) decrements shared cache refs when applicableMeshResourceManager.evictUnused() removes zero-ref cached meshes. This is the first stage of geometry relief before more aggressive runtime eviction.
Under critical memory pressure the engine may call:
+ +That frees the retained MDLAsset tree and child CPU buffers for that streamed root while preserving enough context to reparse later.
The split cache model supports both fast reuse and bounded memory:
+MeshResourceManager is efficient for shared eager meshes and disk-backed reloadsProgressiveAssetLoader avoids reparsing large tiles on every near/far traversalGeometryStreamingSystem arbitrates both with the same distance, frustum, and budget logicWhen reviewing a streamed tile today, think of residency in this order:
+StreamingRegionManager provides a lightweight, region-based geometry streaming API that is an alternative to the tile manifest system (setEntityStreamScene). Rather than consuming a JSON manifest, callers register explicit StreamingRegion values — each describing a world-space AABB and a list of asset references — and the manager loads or unloads them based on camera proximity.
Use this API for handcrafted or procedurally defined streaming zones where a manifest file is not practical. Use setEntityStreamScene for large outdoor/indoor scenes exported by the Blender pipeline.
| Scenario | +Preferred API | +
|---|---|
| Large scene exported by the Blender pipeline (tiles, HLOD, LOD levels) | +setEntityStreamScene(entityId:url:) |
+
| Handcrafted streaming zones (e.g. dungeon rooms, level sectors) | +StreamingRegionManager |
+
| Always-resident objects (characters, props, HUD elements) | +setEntityMeshAsync(entityId:filename:withExtension:) |
+
StreamingRegionpublic struct StreamingRegion: Identifiable {
+ public let id: UUID
+ public let bounds: AABB // World-space AABB that triggers load/unload
+ public let priority: Int // Higher = loaded first when slots compete
+ public let assets: [AssetReference]
+ public var state: StreamingState
+ public var loadedEntities: [EntityID]
+ public var estimatedMemorySize: Int // Bytes; used for memory budget gate
+}
+AssetReference names a single asset by filename and extension:
public struct AssetReference: Equatable, Hashable {
+ public let filename: String
+ public let fileExtension: String
+}
+StreamingState| State | +Meaning | +
|---|---|
.unloaded |
+No geometry loaded for this region | +
.loading |
+Async load task running | +
.loaded |
+All region assets are GPU-resident | +
.unloading |
+Teardown in progress | +
StreamingRegionManager.shared.enabled = true
+StreamingRegionManager.shared.streamingRadius = 100.0 // load within this distance
+StreamingRegionManager.shared.unloadRadius = 150.0 // unload beyond this distance
+StreamingRegionManager.shared.maxConcurrentLoads = 3 // max simultaneous loads
+StreamingRegionManager.shared.checkInterval = 0.5 // seconds between evaluations
+| Property | +Default | +Notes | +
|---|---|---|
streamingRadius |
+100 m | +Camera-to-AABB distance inside which the region loads | +
unloadRadius |
+150 m | +Camera-to-AABB distance beyond which the region unloads | +
maxConcurrentLoads |
+3 | +Hard cap on simultaneous region load tasks | +
checkInterval |
+0.5 s | +Tick rate; skips frames between evaluations | +
These are shared global defaults. Per-region load distance is not currently configurable; all regions use the same radii. For per-region control, use setEntityStreamScene with a manifest.
let region = StreamingRegion(
+ bounds: AABB(min: simd_float3(-50, 0, -50), max: simd_float3(50, 10, 50)),
+ priority: 1,
+ assets: [
+ AssetReference(filename: "dungeon_room_A", withExtension: "usdz"),
+ AssetReference(filename: "dungeon_room_A_props", withExtension: "usdz"),
+ ],
+ estimatedMemorySize: 40_000_000 // 40 MB estimate
+)
+
+StreamingRegionManager.shared.registerRegion(region)
+// Call from your game loop:
+StreamingRegionManager.shared.update(cameraPosition: cameraPos, deltaTime: dt)
+update() throttles internally — real work only runs every checkInterval seconds.
Unregistering cancels any in-flight load task immediately.
+let didLoad = await StreamingRegionManager.shared.forceLoadRegion(id: region.id)
+let didUnload = await StreamingRegionManager.shared.forceUnloadRegion(id: region.id)
+Each tick (every checkInterval seconds):
.unloaded regions whose AABB is within streamingRadius of the camera. Sorted by priority (descending) then distance (ascending)..loaded regions whose AABB is beyond unloadRadius.maxConcurrentLoads candidates — each spawns an async Task.Distance is measured as the closest point on the region AABB to the camera position (AABB distance, not center distance), so an AABB that surrounds the camera has distance 0.
+loadRegion(id:) (internal):
.loading.MemoryBudgetManager.canAccept(sizeBytes:). If the budget is full, attempts evictLRU before proceeding; if still full, marks region .unloaded and returns.AssetReference in region.assets:createEntity() + setEntityMeshAsync(entityId:filename:withExtension:).MemoryBudgetManager for the root entity and all children..loaded; records loadedEntities.AssetResidencyChangedEvent(isResident: true) for each entity (including children) so BatchingSystem and LODSystem see the new geometry.unloadRegion(id:) (internal):
.unloading.AssetResidencyChangedEvent(isResident: false) for all entities (children first) before destroying them — ensures BatchingSystem removes them from pending queues cleanly.MemoryBudgetManager.unregisterMesh(entityId:) for all entities.destroyEntity(entityId:) for each root (cascades to children)..unloaded; clears loadedEntities.let stats = StreamingRegionManager.shared.getStats()
+// stats.totalRegions, loadedRegions, loadingRegions, activeLoads
+// stats.totalRootEntities, totalEntitiesWithChildren
+// stats.regionMemory (actual GPU bytes from MemoryBudgetManager)
+// stats.estimatedMemory (sum of user-supplied estimates)
+// stats.totalEngineMemory (entire engine, from MemoryBudgetManager)
+GeometryStreamingSystemStreamingRegionManager is independent of GeometryStreamingSystem. They can run simultaneously — for example, a tile-streamed outdoor scene (setEntityStreamScene) with handcrafted interior sectors (StreamingRegionManager). Both systems share MemoryBudgetManager, so memory pressure from one is visible to the other.
StreamingRegionManager does not use the octree, frustum gating, prefetch radius, grace-period teardown, or HLOD/LOD systems. It is intentionally simpler. For any of those features, use setEntityStreamScene with a manifest.
tilebasedstreaming.md — tile lifecycle, manifest schema, HLOD, LOD bandsgeometryStreamingSystem.md — mesh-level OCC streaming, memory pressure, evictionstreamingCacheLifecycle.md — how GPU and CPU residency are managed across both systemsTextureStreamingSystem.swift dynamically adjusts the resolution of textures on entities based on their distance from the camera. Instead of keeping every texture at full resolution all the time, it streams textures up or down as the player moves through the scene — saving GPU memory while keeping nearby geometry crisp.
Imagine a USDZ scene with a city block containing 500 buildings. Each building has a RenderComponent with meshes and submeshes, and each submesh has a Material containing up to four PBR textures:
At full resolution, each building's textures might be 2048×2048 or larger. Loading all 500 buildings at full resolution at once would immediately exhaust GPU memory, causing frame drops or crashes.
+The TextureStreamingSystem solves this by managing three quality tiers and promoting/demoting each building's textures as the camera moves.
The system operates with three tiers, controlled by two distance thresholds:
+| Tier | +Condition | +Max Dimension | +
|---|---|---|
| Full | +distance <= upgradeRadius (default 12m) |
+Native source resolution (nil cap) | +
| Medium | +upgradeRadius < distance <= downgradeRadius (default 20m) |
+maxTextureDimension (1024px on macOS, 768px on visionOS) |
+
| Minimum | +distance > downgradeRadius |
+minimumTextureDimension (256px on macOS, 192px on visionOS) |
+
On first import, TextureLoader caps all textures at minimumTextureDimension (256px). The streaming system then only upgrades as the camera approaches — it never immediately downgrades a freshly-loaded entity.
Every frame, the game loop calls:
+ +Throttle: The system only does real work every updateInterval seconds (default 0.2s). This prevents spending every frame scanning all entities.
Concurrency cap: At most maxConcurrentOps (default 3) async streaming operations run simultaneously. If all slots are busy, the tick exits early.
Frame N arrives
+ └─ timeSinceLastUpdate += deltaTime
+ └─ if < 0.2s → return (skip this frame)
+ └─ availableSlots = maxConcurrentOps − activeOps.count
+ └─ if 0 slots → return
+ └─ Priority-0 burst pass: drain priorityEntities (tile-freshly-loaded)
+ └─ Priority-1 pass: visible entities
+ └─ Priority-2 pass: upgraded-but-not-visible entities
+When a tile finishes loading, GeometryStreamingSystem calls notifyEntitiesReady(_:) to register the tile's render-descendant entity IDs as priority candidates:
On the next update tick, these entities are processed before the normal visible-entity pass. This ensures newly-streamed-in tile geometry gets its texture tier evaluated immediately — without waiting for the entity to appear in the frustum-culled visible set.
+Entities tagged with TileLODTagComponent (HLOD and per-tile LOD proxy meshes) are skipped in this pass — they are transient geometry whose textures do not need progressive streaming.
The priorityEntities set is drained every tick and is also cleared by reset().
The system gets the current list of visible entity IDs (from the scene's frustum culling or visibility tracking) and iterates them first:
+For each visible entity:
+ 1. Calculate distance from camera to entity's world-space bounding box center
+ 2. Determine desired tier via desiredMaxDimension(distance:)
+ 3. Build work items — which textures actually need to change
+ 4. If work items exist → scheduleResolutionChange(...)
+ 5. Decrement available slots
+City block example: The camera is standing on the sidewalk in front of Building #42. Buildings #42 and #43 are within 12m (full tier). Buildings #44–#60 are within 20m (medium tier). The remaining 440 buildings are beyond 20m (minimum tier).
+On this tick, the three available slots might be assigned to: +- Building #42: upgrade base color from 1024px → full resolution (2048px) +- Building #43: upgrade roughness from 1024px → full resolution +- Building #57: already at 1024px medium — no change needed, slot freed
+After handling visible entities, the system checks its upgradedEntities set — entities whose textures are currently above the minimum tier. Even if they're off-screen now (the camera rotated away), they may still be nearby and deserve high-res textures so there's no quality drop when the camera rotates back.
For each entity in upgradedEntities that is NOT in the visible set:
+ 1. Calculate actual distance (do not assume minimum)
+ 2. Determine desired tier
+ 3. Build work items
+ 4. Schedule if needed
+City block example: The camera rotated 90°, so Building #42 left the frustum. It is still 3m away. The system sees it in upgradedEntities, computes distance = 3m, desired tier = full — no downgrade is needed. It stays tracked.
If the camera then walks 20m away, Building #42's distance becomes 20m > 12m, so the system schedules a downgrade from full → 256px minimum.
+buildWorkItems(entityId:targetMaxDimension:) inspects every texture slot on the entity's meshes and filters to only the ones that actually need to change:
For each mesh → submesh → material:
+ For each texture type (baseColor, roughness, metallic, normal):
+ currentMax = max(currentTexture.width, currentTexture.height)
+ desiredMax = min(targetMaxDimension, sourceMaxDimension)
+
+ if currentMax == desiredMax → skip (already correct)
+ if upgrading but source is no bigger than current → skip
+
+ else → emit StreamWorkItem(slot, direction, targetMaxDimension)
+Each StreamWorkItem carries:
+- The mesh/submesh index to know where to write back
+- The direction (.upgrade or .downgrade)
+- The target max dimension (nil means full source resolution)
+- The texture source: either an MDLTexture object or a URL on disk
scheduleResolutionChange(...) is where the real work happens — but critically it happens off the main thread:
1. reserveOp(entityId) — mark entity as busy, return false if already active
+2. Initialize MTLCommandQueue and MTKTextureLoader once (reused across ticks)
+3. Spawn a Swift Task (async, off main thread)
+Inside the Task, for each work item:
loadSourceTexture(source, isSRGB:, loader:)
+ └─ MTKTextureLoader loads the original MDLTexture or URL from disk
+ └─ options: shaderRead | pixelFormatView, generateMipmaps: true, SRGB flag
+ └─ Returns a full-resolution MTLTexture
+
+resampleTextureIfNeeded(sourceTexture, targetMaxDimension:, commandQueue:)
+ └─ if targetMaxDimension == nil → return texture as-is (full res)
+ └─ else → GPU downsample to targetMaxDimension
+resampleTextureIfNeeded(currentTexture, targetMaxDimension:, commandQueue:)
+ └─ GPU downsample the already-loaded texture to targetMaxDimension
+ └─ No disk I/O needed — current texture is the source
+downsampleTexture)1. Compute target dimensions preserving aspect ratio
+2. Allocate new MTLTexture (private storage, mipmapped)
+3. Encode MPSImageBilinearScale → bilinear downsample
+4. Encode BlitCommandEncoder.generateMipmaps(for:)
+5. commit() and await completion via CheckedContinuation
+Using MPSImageBilinearScale means the downsampled texture is high quality (bilinear filtering by the GPU shader), and the full mip chain is generated immediately so the renderer can use the appropriate mip level right away.
After all textures in the task are loaded/resampled, execution returns to the main thread via await MainActor.run { withWorldMutationGate { ... } }:
For each LoadedTexture:
+ 1. textureViewMatchingSRGB(texture, wantSRGB:)
+ └─ creates a MTLTextureView with sRGB or linear pixel format
+ without copying pixel data (zero cost)
+
+ 2. updateMaterial(entityId:meshIndex:submeshIndex:) { material in
+ // Three-tier level: .full (nil cap), .capped (medium), .minimum
+ let streamLevel: TextureStreamingLevel = item.targetMaxDimension == nil
+ ? .full
+ : (item.targetMaxDimension! <= capturedMinimumDim ? .minimum : .capped)
+ material.baseColor.texture = item.texture
+ material.baseColorStreamingLevel = streamLevel
+ // (same for roughness, metallic, normal)
+ }
+
+ 3. BatchingSystem.shared.updateBatchMaterialInPlace(for: entityId) { batchMaterial in
+ // Mirror the same three-tier level into the batch group's representative
+ // material so the new texture is visible on the next frame with zero batch churn
+ }
+The withWorldMutationGate wrapper ensures the ECS is not mutated mid-render. The BatchingSystem update ensures batched draw calls reflect the new texture without rebuilding the batch.
After applying, the entity's membership in upgradedEntities is updated: if any texture is still above the minimum tier, the entity stays tracked.
textureViewMatchingSRGB handles a subtle correctness issue: after GPU resampling, the output texture may have a linear pixel format even if the original was sRGB (e.g., rgba8Unorm instead of rgba8Unorm_srgb). Rather than re-encoding with the correct format, the system creates a MTLTextureView — a zero-copy reinterpretation of the same underlying memory with the correct format. This costs nothing in GPU memory.
| Event | +Action | +
|---|---|
| Scene loads | +All 500 buildings loaded at 256px (minimum tier) by TextureLoader | +
| Camera 30m away from Building #42 | +distance > 20m → already at minimum, no work | +
| Camera walks to 18m away | +distance 18m → desired = 1024px medium; upgrade scheduled | +
| Upgrade task runs | +Loads MDLTexture from source → 1024px; applied to ECS + batch | +
Building #42 added to upgradedEntities |
+(1024px > minimum) | +
| Camera walks to 10m away | +distance 10m ≤ 12m → desired = nil (full); upgrade scheduled | +
| Upgrade task runs | +Loads MDLTexture → full 2048px, no GPU resample needed; applied | +
| Camera walks away to 15m | +distance 15m > 12m × (1 + 0.15) → downgrade to 1024px; scheduled | +
| Downgrade task runs | +GPU resamples 2048px → 1024px; applied | +
| Camera walks away to 25m | +distance 25m > 20m × (1 + 0.15) → downgrade to 256px minimum; scheduled | +
| Downgrade task runs | +GPU resamples 1024px → 256px; applied; entity removed from tracking | +
At no point are more than 3 buildings being streamed simultaneously, keeping GPU command submission predictable.
+| Thread | +What happens there | +
|---|---|
| Main / game loop | +update() called; distance math; buildWorkItems; reserveOp; resource init |
+
| Swift Task (async) | +Disk I/O (MTKTextureLoader); GPU encode + await (MPSImageBilinearScale) |
+
| MainActor | +ECS mutation (updateMaterial); batch update; upgradedEntities bookkeeping |
+
activeOps and upgradedEntities are protected by NSLock. The command queue and texture loader are initialized once on the main thread before any Task is spawned, then captured as local constants — no concurrent access to instance state from async tasks.
shedTextureMemoryTextureStreamingSystem exposes a public method for on-demand texture downgrade under memory pressure:
@discardableResult
+public func shedTextureMemory(cameraPosition: simd_float3, maxEntities: Int = 4) -> Int
+This is called by GeometryStreamingSystem — not on a timer, but reactively whenever combined GPU memory (mesh + texture) hits the 85% high-water mark. It bypasses the normal distance-band schedule and forces immediate action.
What it does:
+1. Snapshots upgradedEntities — the set of entities currently holding textures above minimumTextureDimension
+2. Calculates the camera distance for each
+3. Sorts farthest-first — the least visually valuable textures at their current resolution get downgraded first
+4. Schedules up to maxEntities force-downgrades to minimumTextureDimension, skipping any entity already in an active op
+5. Returns the number of entities scheduled
Why farthest-first? A distant entity's 1024 px texture dropping to 256 px is nearly invisible. A nearby entity's texture downgrading would be immediately obvious. This ordering gives the maximum memory relief for the minimum perceptible quality loss.
+Relationship to the update loop: Normal update() ticks also schedule downgrades for out-of-range entities, but only as slots become available and on the 0.2 s timer. shedTextureMemory is a burst — it fills up to maxEntities slots immediately, regardless of the timer, to respond to pressure before the next geometry load attempt.
| Caller | +maxEntities |
+Condition | +
|---|---|---|
GeometryStreamingSystem.update() |
+4 | +Combined pressure high, geometry pressure low — texture relief only, no geometry eviction | +
GeometryStreamingSystem.update() |
+8 | +Geometry pressure also high — shed texture first, then evict geometry | +
OS .warning pressure callback |
+8 | +MemoryBudgetManager.onMemoryPressureWarning fires — proactive shed before OS escalates |
+
OS .critical pressure callback |
+20 | +MemoryBudgetManager.onMemoryPressureCritical fires — aggressive shed + double geometry eviction pass (16 evictions each) + CPU heap release via ProgressiveAssetLoader.releaseWarmAsset() |
+
The larger batch size (8) when geometry is also under pressure reflects that more aggressive texture shedding is needed before the costlier geometry eviction path runs. The OS pressure rows bypass the normal per-tick budget check entirely — they fire out-of-band whenever the OS signals memory pressure, and the actual shedding runs on the next GeometryStreamingSystem.update() tick (deferred via a flag to stay on the main thread).
Apply a built-in profile at scene init instead of setting every property individually:
+TextureStreamingSystem.shared.apply(.detailed) // close-inspection / high-detail assets
+TextureStreamingSystem.shared.apply(.superdetailed) // hero assets / showroom inspection
+TextureStreamingSystem.shared.apply(.openWorld) // large outdoor scenes
+TextureStreamingSystem.shared.apply(.balanced) // general-purpose default
+TextureStreamingSystem.shared.apply(.tiled) // tile-based streaming (see alignToManifest)
+Individual properties can be overridden after applying a profile:
+TextureStreamingSystem.shared.apply(.detailed)
+TextureStreamingSystem.shared.upgradeRadius = 3.0 // widen full-res zone
+| Profile | +upgradeRadius |
+downgradeRadius |
+minDim |
+maxConcurrentOps |
+Best for | +
|---|---|---|---|---|---|
.detailed |
+2.5 m | +6.0 m | +512 px | +6 | +Vehicles, products, characters, props, interiors | +
.superdetailed |
+4.0 m | +8.0 m | +1024 px | +6 | +Showroom vehicles, product configurators, hero assets | +
.openWorld |
+15.0 m | +60.0 m | +256 px | +3 | +Cities, landscapes, terrain | +
.balanced |
+12.0 m | +20.0 m | +platform default | +3 | +Mixed / unknown scene type | +
.tiled |
+30.0 m* | +70.0 m* | +256 px | +6 | +Tile-based streaming scenes | +
* Placeholder defaults only — immediately overridden by alignToManifest (see below).
Detailed rationale: close-inspection content keeps nearby surfaces large on screen, whether the subject is a car, product, character, prop, or room. The minimum tier is raised to 512 px (from the engine default of 256 px) because low-resolution mips look visibly compressed at a few metres. maxConcurrentOps = 6 is safe here because these streaming ops are GPU-bound (no cold disk I/O on the warm path). .archviz remains available as a deprecated compatibility alias for .detailed.
Superdetailed rationale: hero assets may be inspected from a few metres away while still filling a large portion of the view. This profile keeps full-resolution textures through 4 m, uses a 2048 px medium tier, and only drops to 1024 px beyond 8 m. Use compressed formats such as ASTC for large source textures before enabling this profile on memory-constrained devices.
+Open-world rationale: tiers are spread across a city-block scale. The minimum tier stays at 256 px because objects beyond 60 m occupy very few pixels. Keeping maxConcurrentOps = 3 avoids GPU memory spikes when hundreds of entities enter range simultaneously.
Tiled rationale: tile streaming scenes have manifest-defined streaming and unload radii that vary per-scene. The .tiled profile sets concurrency (maxConcurrentOps = 6) and quality (minimumTextureDimension = 256, maxTextureDimension = 1024) for tile-scale geometry, then alignToManifest overrides the radii with values derived from the manifest's streaming_defaults.
When setEntityStreamScene() decodes a tile manifest, it calls:
TextureStreamingSystem.shared.alignToManifest(
+ streamingRadius: manifest.streamingDefaults.streamingRadius,
+ unloadRadius: manifest.streamingDefaults.unloadRadius
+)
+This applies the .tiled profile (concurrency, dimensions) then derives the texture tier radii from the manifest geometry streaming bands:
Why 0.70 × streamingRadius? The streaming radius is the distance at which tile geometry loads. Upgrading to full-res at 70% of that radius means the camera has already moved well inside the loaded zone before the texture upgrade fires — reducing the chance of a visible resolution pop the moment a tile appears.
+Why downgradeRadius = unloadRadius? Tile geometry unloads at unloadRadius. Degrading textures to minimum at the same distance means the GPU texture memory is already at minimum cost exactly when the geometry is about to be evicted, preventing a spike of full/medium-res textures on geometry that is about to disappear.
Example (city.json: streaming_radius = 38.5m, unload_radius = 57.8m):
+- upgradeRadius = 38.5 × 0.70 = 26.97m — full res within ~one tile diagonal
+- downgradeRadius = 57.8m — minimum tier at the tile unload boundary
GeometryStreamingSystem informs the texture system about tile lifecycle events:
// Tile finished loading — schedule priority texture evaluation
+TextureStreamingSystem.shared.notifyEntitiesReady(tileRenderIds)
+
+// Tile unloading — cancel any in-flight or queued ops for its entities
+TextureStreamingSystem.shared.cancelEntities(renderIds)
+cancelEntities removes the entity IDs from upgradedEntities, activeOps, and priorityEntities, ensuring no stale texture upgrade lands on a destroyed entity.
TextureStreamingSystem.shared.upgradeRadius = 4.0 // meters: go full-res inside this
+TextureStreamingSystem.shared.downgradeRadius = 12.0 // meters: go minimum beyond this
+TextureStreamingSystem.shared.maxTextureDimension = 1024
+TextureStreamingSystem.shared.minimumTextureDimension = 256
+TextureStreamingSystem.shared.updateInterval = 0.2 // seconds between evaluations
+TextureStreamingSystem.shared.maxConcurrentOps = 3 // parallel streaming tasks
+TextureStreamingSystem.shared.hysteresisFraction = 0.15 // dead-band fraction at tier boundaries
+TextureStreamingSystem.shared.verboseLogging = true // log each up/downgrade
+Without hysteresis, an entity hovering exactly at a tier boundary (e.g. downgradeRadius = 20 m) oscillates between tiers on alternate streaming ticks, causing mip-map flicker on distant meshes.
hysteresisFraction (default 0.15) applies an asymmetric dead band at each tier boundary:
| Transition | +Triggers at | +
|---|---|
Upgrade to full (boundary: upgradeRadius) |
+distance < upgradeRadius × (1 − h) |
+
| Downgrade from full | +distance > upgradeRadius × (1 + h) |
+
Upgrade to medium (boundary: downgradeRadius) |
+distance < downgradeRadius × (1 − h) |
+
| Downgrade to minimum | +distance > downgradeRadius × (1 + h) |
+
At defaults (upgradeRadius = 12 m, downgradeRadius = 20 m, h = 0.15):
+- Full ↔ medium transition: upgrade at < 10.2 m, downgrade at > 13.8 m
+- Medium ↔ minimum transition: upgrade at < 17 m, downgrade at > 23 m
shedTextureMemory always bypasses hysteresis (passes Float.greatestFiniteMagnitude as distance) so memory-pressure downgrades are never suppressed.
The engine ships a native ASTC texture loader (NativeTexFormat.swift, NativeTextureLoader.swift) that decodes ASTC-compressed textures stored inside .untold binary asset files without going through ModelIO or MTKTextureLoader.
How it interacts with texture streaming:
+.untold files are decoded by NativeTextureLoader at load time and handed to the same TextureLoader GPU cache used by the USDZ/USDC path. From the streaming system's perspective, they are ordinary MTLTexture objects.MPSImageBilinearScale, just as it does for PNG or JPEG source textures.ProgressiveAssetLoader CPU heap pressure..superdetailed and .detailed streaming profiles note ASTC as the recommended source format for high-resolution hero assets on memory-constrained devices.Format support: The engine targets MTLPixelFormatASTC_*_LDR block sizes (4×4, 6×6, 8×8). HDR ASTC is not currently supported. Use the Blender export pipeline's ASTC conversion step to produce compatible .untold files.
TextureLoader.defaultMaxTextureDimension (set in Mesh.swift) is aligned to TextureStreamingSystem.platformDefaultMinimumTextureDimension:
This ensures every freshly loaded entity starts at the streaming system's minimum tier. The streaming system then only upgrades as the camera approaches — it never issues an immediate downgrade on a newly-loaded entity (which would have been visible as a resolution pop on the first frame the entity appeared).
+TextureLoader (the private helper class in Mesh.swift) maintains a per-instance GPU texture cache keyed by TextureCacheKey(id: String, isSRGB: Bool). Two possible key strategies are used depending on what information ModelIO provides:
When property.stringValue contains a parseable USDZ bracket path
+(e.g. "file:///scene.usdz[0/floor_albedo.png]"), the key is:
+usdz-embedded://0/floor_albedo.png
This is unique per physical embedded file. Two materials that reference the same embedded texture file correctly share one GPU texture via this key.
+When bracket notation is absent, the key is the MDLTexture object's memory address:
+mdl-obj-<hex-pointer>
This is used because the name-based fallback (usdz-embedded://GameData/embedded_Basecolor_map) is not safe as a cache key: multiple genuinely different materials can map to the same synthetic name when their MDLTexture objects have an empty .name property. Using a shared cache entry for different physical textures causes some meshes to display the wrong texture on first load.
Sharing still works correctly: two code paths that hold a reference to the same MDLTexture object (same pointer) get the same cache key and share one GPU texture, which is the intended deduplication.
+Both values use the same strategy: bracket URL when available, object-identity URL otherwise.
+| Field | +Value | +Used by | +
|---|---|---|
cacheKeyURL |
+Bracket URL or object-identity URL | +GPU textureCache lookup only |
+
outputURL → material.baseColorURL |
+Bracket URL or object-identity URL | +BatchingSystem.getMaterialHash, TextureStreamingSystem source reference |
+
Why both use object identity when bracket notation is absent:
+The name-based fallback (usdz-embedded://scene.usdz/embedded_Basecolor_map) is the same string for every unnamed texture from the same USDZ, regardless of its actual pixel content. BatchingSystem.normalizeTextureURL then strips the asset-scope host, collapsing all unnamed textures to the same token (usdz-embedded://embedded_Basecolor_map). This causes getMaterialHash to produce the same hash for entities with genuinely different textures, grouping them into one batch and rendering all of them with the first entity's GPU texture — the wrong texture on every other entity.
Using object identity for outputURL as well means:
+- Same MDLTexture pointer → same physical texture → same material.baseColorURL → same batch hash → share a batch group ✓
+- Different MDLTexture pointers → different physical textures → different material.baseColorURL → different batch hash → separate batch groups ✓
The MDLAsset is kept alive in ProgressiveAssetLoader.rootAssetRefs for the entity's lifetime, so MDLTexture pointers are stable across warm eviction/re-upload cycles.
Set textureCacheLoggingEnabled = true before loading to trace every cache hit/miss:
Each log line contains: HIT/MISS, cache key, key source (bracket / obj-identity(unnamed) / obj-identity(named-no-bracket)), MDLTexture pointer, texture name, map type, and isSRGB flag.
Before the fix: you would see HIT entries where the same key is reused across different MDLTexture object identities for base-color textures — the collision.
After the fix: each unnamed/no-bracket texture gets its own obj-identity key; HIT entries are only seen when the same MDLTexture object is referenced by multiple materials (correct sharing).
UntoldEngine implements a multi-tier proximity-based geometry streaming system for large outdoor and indoor scenes on Apple platforms (macOS, visionOS). The system streams geometry in and out of GPU memory based on camera distance, using a spatial octree for efficient runtime range queries. Tile spatial partitioning in the manifest can follow either a uniform grid (v3) or a quadtree floor layout (v4) — see Manifest Versions and Quadtree Partitioning.
+Tier 1 — Tile streaming (TileComponent): coarse-grained. Each tile is a whole USDC file covering a bounded region of the world. Tiles load and unload as the camera moves through the scene.
Tier 2 — OCC mesh streaming (StreamingComponent): fine-grained. Inside a loaded tile, individual mesh stubs upload to the GPU incrementally, governed by distance bands and memory budgets.
HLOD (Hierarchical Level of Detail): a coarse proxy mesh shown in place of an unloaded tile. Loaded when the tile leaves range, unloaded when the full tile finishes parsing. Provides continuity for distant geometry without holding the full tile in GPU memory.
+Per-tile LOD levels (TileLODLevel): intermediate mesh representations that bridge the gap between the full tile (streamingRadius) and the HLOD switch distance. Finer than HLOD but coarser than the full tile; shown while the tile itself is unloaded and the camera is at mid-range.
Hysteresis: both HLOD and per-tile LOD transitions use a hysteresis band to prevent thrashing when the camera hovers near a switch boundary. A loaded representation is not unloaded until the camera moves meaningfully past the switch threshold (controlled by hlodHysteresisFactor and lodHysteresisFactor, both default 0.90 = 10% inner band). Without hysteresis, frame-to-frame distance jitter near a boundary causes rapid load/unload cycles that freeze the engine.
camera distance → close mid-range far very far
+ ──────────────────────────────────────────────────
+full tile ████████████
+per-tile LOD 0 ██████
+per-tile LOD 1 ██████
+HLOD ████████████████████
+nothing visible nothing (if no HLOD)
+| Band | +Representation | +Controlled by | +
|---|---|---|
< streamingRadius |
+Full tile (all submeshes) | +TileComponent.state == .parsed |
+
streamingRadius … LOD[n].switchDistance |
+Per-tile LOD n (coarser each step) | +TileLODLevel.state |
+
> last LOD switchDistance … hlodSwitchDistance |
+HLOD proxy mesh | +TileComponent.hlodState |
+
> hlodSwitchDistance |
+Nothing (tile + HLOD both unloaded) | +— | +
A scene is described by a manifest file listing tiles.
+| Field | +Description | +
|---|---|
version |
+Integer schema version (3 = uniform grid, 4 = quadtree floor) |
+
partitioning_mode |
+(v4 only) "uniform_grid" or "quadtree_floor" — describes how tiles were partitioned by the export pipeline |
+
streaming_defaults |
+Scene-wide fallback radii and priority used when a tile omits its own values | +
tiles |
+Array of tile entries (see below) | +
shared_bucket |
+(optional) A single always-resident tile for geometry that spans many tiles | +
tile_size |
+(optional) Tile footprint in world units, used to align batch cell size with tile boundaries | +
interior_zone |
+(v4 only) Union AABB of all ExteriorShell tiles. Interior tiles are only loaded while the camera is inside this volume |
+
The streaming_defaults block sets scene-wide fallback values for all per-tile fields. An optional shared_bucket entry holds geometry that spans many tiles and should always be resident (loaded as soon as the camera enters the scene).
| Field | +Description | +
|---|---|
tile_id |
+Human-readable name (e.g. "tile_3_2") |
+
path_relative_to_manifest |
+Path to the USDC file, relative to the manifest | +
file_size_bytes |
+Pre-computed file size used by the memory budget gate | +
bounds.min / bounds.max |
+World-space AABB used for octree insertion and frustum tests | +
center |
+World-space center (used for distance calculations) | +
streaming_radius |
+(optional) Per-tile load threshold; falls back to streaming_defaults |
+
unload_radius |
+(optional) Per-tile unload threshold; falls back to streaming_defaults |
+
prefetch_radius |
+(optional) Per-tile prefetch start; falls back to streaming_defaults, then auto |
+
priority |
+(optional) Load order when multiple tiles are candidates | +
hlod_levels |
+(optional) Array of HLOD proxy entries; see HLOD | +
lod_levels |
+(optional) Array of per-tile intermediate LOD entries; see Per-tile LOD Levels | +
floor_id |
+(v4 only, optional) Floor index within a building; 0 = ground floor |
+
quadtree_node_id |
+(v4 only, optional) Quadtree node identifier written by the export script (e.g. "F02Q100"); used for debug logging only, not required for streaming |
+
semantic_tier |
+(v4 only, optional) One of "ExteriorShell", "StructuralInterior", "RoomContents", "FineProps". The streaming_radius already encodes the correct load distance for the tier; no additional runtime logic is required |
+
interior |
+(v4 only, optional) When true, this tile contains interior-only geometry and is gated on the camera being inside interior_zone |
+
switch_distance is the camera distance beyond which the HLOD is shown in place of the tile. Typically a single entry per tile. The HLOD file is a coarse merged mesh exported by the content pipeline.
"lod_levels": [
+ { "path": "tiles/tile_0_0_lod1.usdc", "switch_distance": 80.0 },
+ { "path": "tiles/tile_0_0_lod2.usdc", "switch_distance": 150.0 }
+]
+Entries must be sorted ascending by switch_distance (smallest = finest = closest). switch_distance is the camera distance beyond which this LOD is preferred over the finer level (or the full tile). The engine sorts entries on load, so unsorted manifests are corrected at runtime, but sorted manifests are the canonical contract.
TileComponent — attached to every tile stub entity created by setEntityStreamScene(). Carries all metadata needed for the streaming bootstrap and teardown lifecycle. Key fields added for HLOD and LOD:hlodURL, hlodEntityId, hlodState, hlodSwitchDistance, hlodLoadTask — HLOD lifecyclelodLevels: [TileLODLevel] — per-tile intermediate LOD entriesmeshEntityId — the dedicated mesh-child entity ID, stored so the timeout guard can force-close AssetLoadingGate if loadTextures() hangsTileLODLevel — one instance per LOD entry in the manifest. Carries url, switchDistance, entityId, state (HLODAssetState), and loadTask. Mirrors the HLOD lifecycle pattern.TileLODTagComponent — lightweight tag placed on render-descendant mesh entities spawned by the streaming system for per-tile LOD levels and HLODs. Carries a levelIndex used by the LOD debug renderer (colorRenderablesByLOD) and by BatchingSystem.resolveBatchCandidate to derive the batch LOD index for these entities (which have no LODComponent). levelIndex follows lodDebugPalette: 1 = LOD1 (green), 2 = LOD2 (blue), 5 = HLOD (cyan).StreamingComponent — attached to individual OCC mesh stubs created inside a loaded tile. Governs per-mesh load/unload within the second streaming tier.RenderComponent — added to an entity only after its GPU geometry upload completes. Absence means the entity is invisible to culling and rendering.The manifest schema has evolved across two versions:
+"partitioning_mode": "uniform_grid" (or version: 3 without the field). Tiles are laid out in a regular spatial grid. There is no interior_zone and no semantic tier hierarchy. This is the original format produced by the v1 Blender export script.
"partitioning_mode": "quadtree_floor" (or version: 4). Tiles are partitioned by a floor-level quadtree, typically for multi-storey indoor scenes. The export script assigns each tile a quadtree_node_id (e.g. "F02Q100") and a semantic_tier label.
Semantic tiers encode the expected load distance by naming convention — the export pipeline sets streaming_radius to the correct value for each tier, so the runtime treats them identically during streaming. The tiers are:
| Tier | +Description | +
|---|---|
ExteriorShell |
+Outer building shell, always-visible facade geometry | +
StructuralInterior |
+Floors, walls, and structural elements inside the shell | +
RoomContents |
+Furniture and fixtures within individual rooms | +
FineProps |
+Small detail props, only visible at close range | +
Interior zone gating — the manifest's interior_zone is the union AABB of all ExteriorShell tiles. On each streaming tick, the engine checks whether the camera is inside this volume. Tiles with "interior": true are only dispatched for loading while the camera is inside. This prevents the engine from loading room-level geometry when the player is outside the building, regardless of distance.
++The quadtree partitioning is a content-pipeline and manifest-level concept. At runtime the engine uses an octree for spatial range queries (finding tile stubs near the camera). The manifest's
+quadtree_node_idis used for debug logging only and has no effect on streaming logic.
| State | +Meaning | +
|---|---|
.unloaded |
+Stub registered; no geometry in flight | +
.parsing |
+setEntityMeshAsync Task is running; GPU upload in progress |
+
.parsed |
+Tile's child entities exist and are rendering (or uploading via OCC) | +
.failed |
+Last parse attempt failed; exponential backoff before retry (5 s → 10 s → 20 s → max 60 s) | +
.unloading |
+Teardown in progress; blocks re-dispatch for this tick | +
setEntityStreamScene)RemoteAssetDownloader before decoding. Tile asset URLs in the manifest are resolved relative to the manifest's base URL, so remote manifests produce remote tile URLs (e.g. https://cdn.example.com/scene/tiles/tile_0_0.untold). See asset_remote_streaming.md for the full download lifecycle.interiorZone and firstRangeTimestamps on GeometryStreamingSystem so stale scene-level state from a previous scene does not bleed into the new one.TiledSceneComponent, LocalTransformComponent, and ScenegraphComponent.withWorldMutationGate, parented under the root entity. Each stub receives:LocalTransformComponent.boundingBox set to the tile's world-space AABBTileComponent in .unloaded state, with all radii and metadata from the manifestqueryNear finds it immediately)No geometry is parsed or uploaded at this stage. The whole function completes in milliseconds regardless of scene size.
+GeometryStreamingSystem.update(cameraPosition:deltaTime:) runs each frame. It queries the octree for all entities within maxQueryRadius (default 500 m). This is a query ceiling, not a load radius — it just defines the outer bound of the candidate pool. Each entity in the result is then tested against its own per-entity radii (effectivePrefetchRadius, streamingRadius, unloadRadius) to decide what actually happens. maxQueryRadius must be large enough to cover the largest unloadRadius in the scene, or far tiles will never be found for out-of-range teardown.
Camera velocity is computed each tick via exponential smoothing and used to project a predictive position (velocityLookAheadTime = 0.5 s ahead). Tile distances are scored against min(actual, predictive) so tiles in the direction of travel are prioritised before the camera physically arrives.
Tile load pass — for each .unloaded stub:
effectivePrefetchRadius (see Prefetch Radius).tileFrustumGatePadding = 20 m). Tiles fully outside the frustum are skipped this tick.maxConcurrentTileLoads (default 2) are dispatched via loadTile(), subject to the memory budget gate: the total parse memory in flight must stay under tileParseMemoryBudgetMB (200 MB), with a guarantee that at least one tile always loads even if it alone exceeds the budget.Tile unload pass — three sub-passes each tick. All passes use min(actual, predictive) distance, matching the load pass, so a tile the camera is approaching is not torn down mid-parse:
unloadRadius.maxQueryRadius.maxQueryRadius (fast movement or teleport).Both .parsing and .parsed tiles go through the grace period (see Unload Grace Period) before actual teardown (passes 1 and 2). Pass 3 tiles are genuinely beyond the 500 m query radius and are cancelled without a grace period — boundary oscillation cannot occur at that range. At most maxTileUnloadsPerUpdate (default 2) tiles are torn down per tick to spread GPU buffer releases across frames.
loadTile(entityId:)tileComp.state = .parsing; reserves a slot in activeTileLoads.capturedMeshEntityId) inside withWorldMutationGate. This guarantees unloadTile's collectTileDescendants always has at least one child to destroy, regardless of how many submeshes the tile contains.capturedMeshEntityId → tileEntityId in meshEntityToTileEntity for O(1) OCC upload counter updates.Task calling setEntityMeshAsync(entityId: capturedMeshEntityId, streamingPolicy: .auto, blockRenderLoop: false)..auto policy: the admission gate chooses fullLoad (parse + immediate GPU upload) or outOfCore (parse to CPU heap, upload stubs via StreamingComponent) based on tile file size and available RAM.blockRenderLoop: false — tile parses do not hold the AssetLoadingGate open. Without this, concurrent tile parses would keep isLoadingAny == true for their full duration, freezing visibleEntityIds updates and stalling the render loop. LOD and HLOD loads also use blockRenderLoop: false for the same reason.tc.state == .parsing. If unloadTile ran while the parse was in flight, the state will be .unloading. The callback discards the result, destroys the pre-created child entity, and returns without marking the tile loaded..parsing: transitions to .parsed, seeds totalOCCStubs from countOCCDescendants.occCount == 0): all geometry is immediately GPU-resident. The callback:setEntityStaticBatchComponent to tag the entity hierarchy for cell-based static batching.BatchingSystem.shared.notifyTileEntitiesResident(_:) with the set of render descendant IDs. This single call replaces the former two-step queueResidencyEventsForRenderDescendants + notifyTileParsedEntities pairing — it directly registers the entities in the batching system's pending additions and marks them for quiescence bypass, avoiding the per-entity event storm through SystemEventBus. See Tile-Local Batch Promotion.occCount > 0): setEntityStaticBatchComponent is called but residency notifications are not sent — they fire automatically as each OCC stub completes its GPU upload via the normal handleResidencyChange flow. The normal quiescence delay applies to keep the batch from rebuilding after each individual stub upload.failureCount, sets state to .failed (retry backoff).defer { releaseActiveTileLoad } — the concurrency slot is freed on all exit paths.For large tiles using the outOfCore path, setEntityMeshAsync creates child OCC stub entities under capturedMeshEntityId, each with a StreamingComponent in .unloaded state. GeometryStreamingSystem.update() iterates these stubs in a separate pass — uploading them in batches governed by maxConcurrentLoads (3), with a nearBandMaxConcurrentLoads = 1 serial slot for the closest mesh stubs so distance-ordered appearance is preserved.
Each completed OCC upload calls incrementParentTileOCCCount(for:), which increments tileComp.uploadedOCCStubs. The visualState property (TileVisualState) tracks upload progress: .empty → .partial → .usable (≥ 50% uploaded) → .complete.
unloadTile(entityId:)wasParsing = (tileComp.state == .parsing).tileComp.state = .unloading; cancels tileComp.loadTask.wasParsing: removes from loadingTileEntities and bails out. The Task completion callback will find .unloading, discard the result, and dispatch deferred child-entity cleanup — this avoids a concurrent ECS write race since setEntityMeshAsync may still be running..parsed: calls collectTileDescendants(entityId) to walk the child tree, cancelling any in-flight OCC streaming tasks. Calls destroyEntity on all descendants + finalizePendingDestroys(). This releases GPU buffers, removes octree entries, releases MeshResourceManager refs, and unregisters from MemoryBudgetManager.ProgressiveAssetLoader.shared.removeOutOfCoreAsset(rootEntityId:) to free CPU-heap MDLAsset data for out-of-core tiles.totalOCCStubs, uploadedOCCStubs, pendingUnloadSince to 0.tileComp.state = .unloaded; removes from loadedTileEntities.The tile stub entity itself is never destroyed. It stays in the octree as a cheap placeholder so the streaming system reloads the tile on the next approach.
+The prefetch radius decouples "when the tile starts loading" from "when the tile must be visible." Tiles begin parsing as soon as the camera enters effectivePrefetchRadius, which is larger than streamingRadius. By the time the camera reaches streamingRadius, the parse is already complete and the geometry appears without a blank frame.
camera direction →
+─────────────────────────────────────────────────────
+ prefetchRadius (auto: midpoint)
+ │ streamingRadius
+ │ │ unloadRadius
+ ▼ ▼ ▼
+ · · · · · · · ·|· · · · · ·|████████|· · · · · · ·
+ start tile is stop
+ loading visible loading
+effectivePrefetchRadius resolution (in priority order):
+1. Per-tile prefetch_radius field in the manifest
+2. Scene-wide prefetch_radius in streaming_defaults
+3. Auto: streamingRadius + (unloadRadius − streamingRadius) × 0.5
For the typical streamingRadius = 80 m, unloadRadius = 120 m default, auto resolves to 100 m — giving 20 m of prefetch advance at walking speed (~1.5 m/s) that is ~13 seconds of loading headroom, well above the 1–2 s parse time for a 15–20 MB tile.
When a .parsed tile (with visible GPU geometry) first exceeds unloadRadius, pendingUnloadSince is set to the current time. The tile is only torn down once CFAbsoluteTimeGetCurrent() − pendingUnloadSince ≥ unloadGracePeriod (default 3 seconds).
If the camera re-enters unloadRadius before the grace period expires, pendingUnloadSince is reset to 0 and the tile stays loaded with no interruption. This eliminates rapid load/unload oscillation at tile boundaries (the most common cause of flickering at tile edges).
Both .parsing and .parsed tiles honour the grace period. .parsing tiles have no visible geometry, but the grace window lets an in-flight parse complete naturally rather than being cancelled and immediately re-dispatched. Immediate cancellation was a false economy: the cancelled Swift Task still ran to completion before the state could reset, so the tile was re-dispatched on the very next tick, creating a tight load-cancel loop.
pendingUnloadSince is also reset in unloadTile() so the counter is clean for the next load/unload cycle.
MemoryBudgetManager tracks geometry and texture GPU bytes against per-platform budgets (probed at startup).shouldEvictGeometry(). If true, it runs TextureStreamingSystem.shedTextureMemory and evictLRU (capped at 8 evictions) before attempting a tile parse. This prevents a tile's multi-MB commit from pushing RAM over budget.evictionDistanceWeight + GPU size × evictionSizeWeight. Entities within visibleEvictionProtectionRadius (30 m) are protected.DispatchSource.makeMemoryPressureSource) set a flag; eviction is deferred to the next update() tick to stay single-threaded.tripleVisibleEntities.clearAll() — called in finalizePendingDestroys() to clear all triple-buffer slots so the renderer does not read stale entity IDs after a scene reload.createEntity, registerComponent, destroyEntity, finalizePendingDestroys) must run on the main thread.withWorldMutationGate is an activity counter, not a mutex. It does not provide mutual exclusion — it signals that ECS mutations are occurring.scene.exists(entityId) guards before every ECS write in upload completions prevent writes to entities destroyed while an upload was in flight (cooperative cancellation race).loadedTileEntities, loadingTileEntities, activeTileLoads, meshEntityToTileEntity) are protected by stateLock and accessed only through accessor methods.When setEntityStreamScene is called for a second time (replacing a previous scene):
destroyEntity(entityId: oldRoot)), which cascades to all tile stubs. For each destroyed stub, removeTileComponent (the ComponentRegistry cleanup handler for TileComponent) cancels the tile's in-flight loadTask and calls GeometryStreamingSystem.shared.unregisterTileEntity(entityId), removing stale IDs from all tracking sets atomically.setEntityStreamScene resets interiorZone and firstRangeTimestamps so scene-level streaming state is clean for the new scene.Any tile Task that was already in flight and completes after step 1–2 finds scene.exists(entityId) == false and returns early via the guard in its completion closure. The defer-based slot release still fires, so no concurrency slot is leaked.
HLOD fills the gap beyond a tile's unloadRadius by showing a coarse proxy mesh while the full tile is not in GPU memory. This eliminates the visual "pop to nothing" when a tile leaves range.
| State | +Meaning | +
|---|---|
.unloaded |
+No HLOD geometry in GPU memory | +
.loading |
+loadHLOD() task running |
+
.loaded |
+HLOD entity exists and is rendering | +
.unloading |
+Teardown in progress | +
HLOD is loaded when all of the following hold:
+- The tile has an hlodURL in its TileComponent
+- dist >= hlodSwitchDistance (camera is beyond the switch threshold)
+- tileComp.state != .parsed (full tile is not resident — no redundant HLOD)
+- hlodState == .unloaded
+- activeHLODLoadCount() < maxConcurrentHLODLoads (default 4) — prevents simultaneous mass dispatch of 100+ HLOD parses that would OOM-kill the process
HLOD is unloaded (and the load task cancelled) when:
+- The full tile reaches .parsed — full geometry has taken over; HLOD is redundant. No hysteresis here — always unload promptly.
+- dist < hlodSwitchDistance × hlodHysteresisFactor (default 0.90) — camera has moved meaningfully inside the switch distance. The hysteresis band [switchDistance × factor, switchDistance) keeps the HLOD resident while the camera lingers at the boundary, preventing thrashing.
Race fix: hlodState = .unloading is set before hlodLoadTask.cancel(). This prevents the load completion callback from seeing .loading and incorrectly marking the HLOD as .loaded after teardown is already in progress.
Minimum dwell guard: After any HLOD load or unload, tileComp.lastHLODTransitionTime is stamped. The next HLOD transition is suppressed until secondaryRepresentationMinDwellSeconds (default 1.0 s) has elapsed. This prevents the HLOD from flip-flopping faster than once per second when the camera lingers at the switch boundary while hysteresis alone is insufficient.
A second pass after the main HLOD pass tears down HLOD entities for tiles that have drifted entirely outside maxQueryRadius. These tiles are no longer in the octree result and would otherwise remain with stale .loaded HLOD entities indefinitely.
Per-tile LODs fill the mid-distance band between the full tile (streamingRadius) and the HLOD (hlodSwitchDistance). They are intermediate meshes — coarser than the full tile but finer than HLOD — shown while the tile itself is unloaded.
Distinction from HLOD:
+| + | HLOD | +Per-tile LOD | +
|---|---|---|
| Purpose | +Replace unloaded tile at very far distances | +Provide finer intermediate detail at mid distances | +
| Memory concern | +Yes — always-resident coarse mesh | +Yes — one entity per active LOD level | +
| GPU cost concern | +Low (coarse mesh) | +Medium (finer than HLOD, less than full tile) | +
| Distance band | +> hlodSwitchDistance |
+streamingRadius … hlodSwitchDistance |
+
Each TileLODLevel follows the same HLODAssetState state machine as HLOD:
if HLOD is loaded or loading → unload all LOD levels (avoid dual representation)
+if dist >= hlodSwitchDistance → unload all LOD levels (HLOD pass handles this band)
+
+find target index (with hysteresis):
+ for each level i:
+ threshold = (i is currently active) ? switchDistance × lodHysteresisFactor : switchDistance
+ last level where threshold ≤ dist → that level
+ no match (dist < LOD[0].threshold) → no LOD (tile will load or is already parsed)
+
+load target level if .unloaded (capped by maxConcurrentLODLoads)
+unload all other levels that are .loaded or .loading
+The hysteresis ensures the currently active LOD level uses a lowered unload threshold (switchDistance × lodHysteresisFactor, default 0.90), so the camera must move meaningfully inward before the level is swapped. Levels that are not currently active use the full switchDistance.
Minimum dwell guard: LOD transitions are also gated by secondaryRepresentationMinDwellSeconds (default 1.0 s). After any per-tile LOD load or unload, tileComp.lastLODTransitionTime is stamped and the next transition is suppressed until 1.0 s has elapsed. This prevents oscillation between adjacent LOD levels when the camera is stationary near a switch threshold.
When tileComp.state == .parsed (full tile is resident), all LOD levels are unloaded — the full tile has taken over for that distance band.
loadLODLevel(entityId:levelIndex:)setEntityMeshAsync with .immediate policy and blockRenderLoop: false (small proxy mesh, must not stall the render loop).TileLODTagComponent(levelIndex: capturedIndex + 1) for LOD debug visualization, tags the entity for static batching (setEntityStaticBatchComponent), and calls BatchingSystem.shared.notifyTileEntitiesResident(_:) to bypass the quiescence delay.loadedLODEntities tracking set.unloadLODLevel(entityId:levelIndex:)level.state = .unloading before level.loadTask?.cancel() (same race fix as HLOD).BatchingSystem.shared.cancelPendingEntities(_:) with the render descendant IDs — removes them from all pending batching queues before the entities are destroyed, preventing "entity is missing" errors on the next batching tick.AssetLoadingGate for the destroyed entity (idempotent no-op if the Task already closed it).loadedLODEntities when all levels are clear.A pass after the LOD streaming pass tears down LOD entities for tiles outside maxQueryRadius, mirroring the HLOD cleanup pass.
loadTile completionWhen the full tile finishes parsing, unloadAllLODLevels(entityId:) is called alongside unloadHLOD(entityId:). Both intermediate representations are removed as the full tile takes over.
ModelIO's loadTextures() is a blocking call that can hang indefinitely on unsupported image formats inside USDC archives (e.g. a format that stalls the ObjC image decoder with no timeout). When it hangs, AssetLoadingState.finishLoading is never called, AssetLoadingGate.isLoadingAny stays true permanently, and RenderingSystem skips all ECS traversal and culling every frame — the app remains alive but the view is frozen.
ResumeOnceloadTextures() in RegistrationSystem is wrapped with a DispatchQueue + 15-second deadline:
let textureLoadOK = await withCheckedContinuation { cont in
+ let once = ResumeOnce() // NSLock-backed: resumes cont exactly once
+ DispatchQueue.global(qos: .userInitiated).async {
+ assetRef.loadTextures()
+ once.callOnce { cont.resume(returning: true) }
+ }
+ DispatchQueue.global().asyncAfter(deadline: .now() + 15.0) {
+ once.callOnce { cont.resume(returning: false) } // deadline fires
+ }
+}
+If loadTextures() hangs, the deadline fires after 15 s and the async continuation proceeds without textures (geometry is still rendered, just untextured).
TileComponent.meshEntityId stores the dedicated mesh-child entity ID. If the tile is unloaded while a parse is in flight and loadTextures() is hung, the timeout guard retrieves meshEntityId and force-closes the gate:
let hungMeshId = tc.meshEntityId
+tc.meshEntityId = .invalid
+if hungMeshId != .invalid {
+ Task { await AssetLoadingState.shared.finishLoading(entityId: hungMeshId) }
+}
+This unblocks AssetLoadingGate, allowing the render loop to resume its normal ECS traversal.
The loadTextures() 15-second timeout (above) guards against a hung texture decode inside an already-running parse. A separate, coarser watchdog guards against the entire tile parse Task becoming stuck — for example when the OS suspends the Task, disk I/O stalls indefinitely, or a remote download never completes.
On every streaming tick, GeometryStreamingSystem.update() checks every tile currently in .parsing state:
if CFAbsoluteTimeGetCurrent() - tc.parseStartTime > tileParseTimeoutSeconds (default 60 s):
+ cancel loadTask
+ force-close AssetLoadingGate for meshEntityId
+ increment failureCount → tile enters exponential backoff
+ release concurrency slot
+parseStartTime is set after the remote fetch completes, not when loadTile() is first called. For remote tiles, the time waiting for the network does not count against the 60-second budget — only the actual CPU parse time does. This avoids spurious timeouts on slow connections.
Two distinct timeout layers:
+| Mechanism | +Scope | +Deadline | +On trigger | +
|---|---|---|---|
ResumeOnce + DispatchQueue |
+loadTextures() call only |
+15 s | +Proceeds without textures; geometry still renders | +
| Tile-parse watchdog | +Entire tile parse Task | +60 s (tileParseTimeoutSeconds) |
+Cancels task, marks .failed, enters retry backoff |
+
| Property | +Default | +Notes | +
|---|---|---|
maxConcurrentTileLoads |
+2 | +Hard cap on simultaneous tile parses | +
maxConcurrentLODLoads |
+4 | +Hard cap on simultaneous per-tile LOD level loads | +
maxConcurrentHLODLoads |
+4 | +Hard cap on simultaneous HLOD mesh loads | +
lodHysteresisFactor |
+0.90 | +LOD unload threshold multiplier (10% inner band) | +
hlodHysteresisFactor |
+0.90 | +HLOD unload threshold multiplier (10% inner band) | +
tileParseMemoryBudgetMB |
+200 MB | +Total CPU parse memory allowed in flight | +
maxTileUnloadsPerUpdate |
+2 | +Max tile teardowns per streaming tick | +
unloadGracePeriod |
+3.0 s | +Hold time before tearing down a visible tile | +
maxConcurrentLoads (OCC) |
+3 | +Simultaneous mesh-level GPU uploads | +
nearBandMaxConcurrentLoads |
+1 | +Serial slot for closest mesh stubs | +
maxUnloadsPerUpdate (mesh) |
+12 | +Max mesh-level unloads per tick | +
updateInterval |
+100 ms | +Streaming tick rate (steady state) | +
burstTickInterval |
+16 ms | +Tick rate during near-band backlog | +
frustumGatePadding (mesh) |
+5 m | +Frustum pad for mesh-level candidates | +
tileFrustumGatePadding |
+20 m | +Frustum pad for tile-level candidates | +
velocityLookAheadTime |
+0.5 s | +Predictive position look-ahead | +
velocityLookAheadMinSpeed |
+1.5 m/s | +Minimum speed to activate look-ahead | +
visibleEvictionProtectionRadius |
+30 m | +Distance inside which eviction is blocked | +
hlodSwitchDistance |
+manifest switch_distance |
+Camera distance beyond which HLOD is shown | +
LOD switchDistance |
+manifest switch_distance per entry |
+Camera distance beyond which this LOD is preferred | +
loadTextures() timeout |
+15 s | +Deadline before ResumeOnce force-proceeds without textures |
+
tileParseTimeoutSeconds |
+60 s | +Watchdog deadline for an entire tile parse Task; forces tile to .failed and frees concurrency slot — distinct from the per-loadTextures() 15 s timeout |
+
secondaryRepresentationMinDwellSeconds |
+1.0 s | +Minimum time a HLOD or per-tile LOD level must dwell before the next transition is allowed; prevents flip-flopping between representations faster than once per second | +
The XR rendering system is the visionOS counterpart to UpdateRenderingSystem. It drives the same deferred render graph used on macOS, but within a completely different frame lifecycle imposed by CompositorServices — Apple's low-latency compositor for spatial computing.
The entry point is UntoldEngineXR, a class that owns the render loop, the ARKit session, and the spatial input bridge. Everything in this file is compiled only on visionOS (#if os(visionOS)).
On macOS, the MTKView delegate calls draw(in:) on the main thread at display refresh rate. The engine owns the timing.
On visionOS, the compositor owns the timing. It tells you when to render, provides per-eye textures, and requires you to attach a device anchor (head pose) to every frame before presenting. If you miss the deadline or present without an anchor, the compositor either drops your frame or logs a warning. The UntoldEngineXR run loop is specifically structured to satisfy these compositor requirements frame by frame.
At init time, three things happen in parallel:
+ARKit session startup (async Task): Queries world sensing authorization, then launches WorldTrackingProvider and PlaneDetectionProvider. World tracking is what gives you the device anchor — the head pose needed to render correctly in the user's space. If world sensing is denied (e.g., the user blocked it in Settings), the engine still runs with world tracking only so rendering doesn't break, it just has no plane data.
Plane monitor (background Task): A long-running Swift structured concurrency task that consumes the planeDetection.anchorUpdates async stream. Every time the system detects, updates, or removes a real-world surface (floor, wall, table, etc.), it maps the ARKit classification to the engine's RealSurfaceKind enum and forwards it to RealSurfacePlaneStore. Game code queries this store to snap objects to real surfaces.
Renderer creation: UntoldRenderer.createXR(...) initializes the Metal device, command queue, G-Buffer textures, pipeline states, and all other GPU resources at the fixed visionOS viewport size (2048 × 1984 per eye).
runLoop() is called from a dedicated background thread (the compositor render thread) and runs for the lifetime of the XR session:
while true {
+ switch layerRenderer.state {
+ case .paused: layerRenderer.waitUntilRunning()
+ case .running: renderNewFrame()
+ case .invalidated: break // exit
+ }
+}
+The .paused state blocks the thread cheaply until the compositor is ready — this happens when the user puts the app in the background or when the system needs to reclaim resources. The .invalidated state is the clean shutdown signal.
Why a background thread? The compositor render thread must never be blocked by Swift's main actor or UIKit layout passes. Running here ensures that Metal encoding proceeds at compositor frame rate (90 FPS on Vision Pro) without contention.
+renderNewFrame()CompositorServices frames follow a strict protocol. Every call to renderNewFrame() must progress through these phases in order:
queryNextFrame() dequeues the next compositor frame. predictTiming() returns optimalInputTime — the deadline by which you must finish reading input and preparing CPU-side data for the frame. These are not suggestions; missing them causes judder.
Everything between startUpdate and endUpdate is CPU-side frame preparation:
Progressive loading tick: ProgressiveAssetLoader.shared.tick() is dispatched to the main thread via DispatchQueue.main.async. The run loop lives on the compositor thread, but tick() requires @MainActor. This mirrors what UntoldEngine.swift's draw() does on macOS.
Spatial input processing: updateSpatialInputState() drains the queued XRSpatialInputSnapshot events and updates the InputSystem. If assets are loading or the scene isn't ready, input is cleared instead to avoid acting on stale state.
Game update: renderer.updateXR() calls the user's gameUpdate and handleInput callbacks. This is where game logic runs — entity movement, animation state machines, physics steps. It is skipped entirely while AssetLoadingGate.shared.isLoadingAny is true.
The thread sleeps until the compositor says it's the best moment to submit GPU work. Submitting too early wastes GPU time on a stale pose; submitting too late misses the scanline. This one call is what makes visionOS rendering feel low-latency.
+The defer is important: endSubmission() must be called even if rendering fails partway through. If it isn't called, the compositor stalls. The defer guarantees this regardless of how the function exits.
let deviceAnchor = worldTracking.queryDeviceAnchor(atTimestamp: presentationTimeCA)
+drawable.deviceAnchor = anchor
+The device anchor is the head pose — a 4×4 transform from world space to the device. The compositor requires it to be attached to the drawable before presenting; without it, the system can't reproject the frame correctly for the user's eyes.
+The engine queries the anchor at presentation time (the future moment when the frame will appear on screen), not at "now". This is predictive — it compensates for the latency between encoding and display by predicting where the head will be.
+Resilience strategy: ARKit can occasionally return nil for the anchor (e.g., tracking hiccup, recovery). A three-level fallback prevents dropped frames:
+1. Query at predicted presentation time → use it if valid
+2. Query at "now" as a retry → use it if valid
+3. Fall back to lastValidDeviceAnchor — the last anchor that was valid
The engine never skips presenting a drawable once it has been dequeued. Even with no anchor at all, the drawable is presented (the compositor handles it gracefully); skipping would cause a more disruptive compositor error.
+executeXRSystemPass()This is where the actual Metal work happens, split into three parts.
+performFrustumCulling(commandBuffer: commandBuffer)
+executeGaussianDepth(commandBuffer)
+executeBitonicSort(commandBuffer)
+These are the same three compute passes as the macOS path: frustum cull, Gaussian depth, bitonic sort. The key difference is they run once per frame, not once per eye. The culled visibility list and sorted splat indices produced here are reused by both the left and right eye render passes.
+++Why only once? Running culling and sorting twice — once per eye — at 90 FPS would double the compute budget for work that produces nearly identical results (the two eyes are only ~65mm apart). One cull pass with a slightly conservative frustum covers both views.
+
for (viewIndex, view) in drawable.views.enumerated() {
+ // compute view and projection matrices
+ // configure pass descriptor
+ // call renderer.renderXR(...)
+}
+The drawable provides two views (left eye, right eye). For each:
+View matrix construction: +
+originFromDevice is the world-to-device transform from the anchor. deviceFromView is the eye offset relative to the device center (the IPD offset). Multiplying them gives world-to-eye, then inverting gives the view matrix the shaders expect.
+Projection matrix: +
+The compositor provides the exact asymmetric projection for each eye. This accounts for the different FOV angles per eye and the physical lens geometry of the headset — it cannot be constructed manually. +Pass descriptor: Pre-allocated (passDescriptorLeft, passDescriptorRight) and reused every frame to avoid 180 allocations/second (2 eyes × 90 FPS). The color and depth textures are swapped in from drawable.colorTextures[viewIndex] and drawable.depthTextures[viewIndex].
renderer.renderXR(...) calls buildGameModeGraph() + topologicalSortGraph() + executeGraph() — the exact same render graph pipeline as macOS. The only thing that changes is:
+- renderInfo.currentEye = viewIndex — tells uniform uploads which eye's matrices to use
+- The base pass mode: .mixed immersion omits the base pass (camera passthrough is the background), .full immersion renders the skybox
After both eyes are encoded into the same command buffer, the HZB depth pyramid is built from the depth texture of the last eye rendered. In stereo, this mono pyramid is used for both per-eye occlusion tests in the next frame's frustum cull.
+Why after both eyes, not per eye? Building HZB per eye would double the cost and produce two pyramids that next frame's single-dispatch cull can't easily consume. The right eye's depth is a reasonable approximation for the combined scene.
+Note encodePresent vs. macOS's commandBuffer.present(drawable). On visionOS, the present must be encoded into the command buffer as a GPU command so the compositor can precisely time when the drawable lands on screen relative to GPU work completion. It is not a CPU-side call.
The completion handler signals commandBufferSemaphore when the GPU finishes, freeing a slot for the next frame.
configureSpatialEventBridge() registers a closure on layerRenderer.onSpatialEvent. Every time the compositor fires a pinch gesture or spatial tap, the closure:
selectionRay field.active, .ended, .cancelled) to the engine's XRSpatialInteractionPhaseXRSpatialInputSnapshot and enqueues it in InputSystemThe snapshot is processed on the next frame's update phase by spatialGestureRecognizer.updateSpatialInputState(), which converts raw ray/phase sequences into higher-level gesture events (tap, hold, drag) that game code can query.
++The bridge is gated on
+isSceneReady()and!AssetLoadingGate.shared.isLoadingAny. Input events while loading are discarded to prevent game code from acting on uninitialized entities.
[Compositor thread] runLoop() → renderNewFrame()
+ │
+ ├─ queryNextFrame() + predictTiming()
+ ├─ frame.startUpdate()
+ │ ├─ [Main thread async] ProgressiveAssetLoader.tick()
+ │ ├─ updateSpatialInputState() (drain gesture queue)
+ │ └─ renderer.updateXR() (gameUpdate + handleInput)
+ ├─ frame.endUpdate()
+ ├─ wait(until: optimalInputTime) (sleep until compositor deadline)
+ ├─ frame.startSubmission()
+ ├─ queryDrawable() (get per-eye textures)
+ ├─ queryDeviceAnchor() → fallback chain → drawable.deviceAnchor = anchor
+ │
+ └─ executeXRSystemPass()
+ │
+ ├─ [GPU compute] frustumCulling (once, covers both eyes)
+ ├─ [GPU compute] gaussianDepth
+ ├─ [GPU compute] bitonicSort
+ │
+ ├─ for eye in [left, right]:
+ │ ├─ compute view matrix (originFromDevice × deviceFromView)⁻¹
+ │ ├─ compute projection (drawable.computeProjection)
+ │ ├─ configure passDescriptor (color + depth textures)
+ │ └─ renderer.renderXR() → buildGameModeGraph()
+ │ topologicalSortGraph()
+ │ executeGraph()
+ │ [same DAG as macOS]
+ │
+ ├─ [GPU compute] buildHZBDepthPyramid (once, after both eyes)
+ ├─ drawable.encodePresent(commandBuffer)
+ └─ commandBuffer.commit()
+ │
+ └─ [GPU→thread callback] semaphore.signal()
+ │
+ └─ frame.endSubmission()
+| + | macOS (UpdateRenderingSystem) |
+visionOS (executeXRSystemPass) |
+
|---|---|---|
| Frame timing | +MTKView drives at display rate | +CompositorServices dictates via optimalInputTime |
+
| Eyes | +1 | +2 (per-eye loop over drawable.views) |
+
| Compute passes | +Once per frame | +Once per frame (shared across both eyes) | +
| View matrix | +Camera entity transform | +(originFromDevice × deviceFromView)⁻¹ from ARKit anchor |
+
| Projection | +Camera component FOV | +drawable.computeProjection() — asymmetric per-eye |
+
| Present call | +commandBuffer.present(drawable) |
+drawable.encodePresent(commandBuffer:) — encoded as GPU command |
+
| HZB build | +After the single render graph | +After both eyes, once | +
| Base pass | +Environment or grid | +Environment (full immersion) or none (mixed/passthrough) | +
| Game update thread | +Main thread (MTKView delegate) | +Compositor thread, with main-thread dispatch for restricted APIs | +
Thank you for your interest in contributing!
+The vision for Untold Engine is to continually shape a 3D engine that is stable, performant and developer-friendly.
+As maintainer, my focus is on performance, testing, quality control, and API design.
+Contributors are encouraged to expand features, fix bugs, improve documentation, and enhance usability — always keeping the vision in mind.
The Untold Engine is guided by a clear vision: To be a stable, performant, and developer-friendly 3D engine that empowers creativity, removes friction, and makes game development feel effortless.
+To achieve this vision, we follow these principles:
+As the maintainer, my primary focus is to ensure the project stays true to this vision.
+All contributions are welcome, but acceptance will be guided by the project’s vision and the priorities above.
+PRs that align with clarity, performance, or creativity — while keeping the engine stable and simple — are more likely to be accepted.
These guidelines aren’t here to block you, but to make sure every contribution keeps the engine stable, clear, and useful for everyone.
+One feature or bug fix per PR
+ Each Pull Request should focus on a single feature, bug fix, or documentation improvement.
+ This keeps the history clean and makes it easier to track down issues later.
Commit hygiene
+✅ Example:
+- Good: “Add PhysicsSystem with gravity integration”
+- Bad: “Added PhysicsSystem + fixed rendering bug + updated docs”
For new systems or major features, your PR must include:
+This ensures new features are stable, documented, and accessible to all users.
+👉 Note: For small fixes or incremental features, a How-To is not required.
+Your guide must follow this structure:
+Introduction
+Briefly explain the feature and its purpose.
+Describe what problem it solves or what value it adds.
+Why Use It
+Provide real-world examples or scenarios where the feature is useful.
+Explain the benefits of using the feature in these contexts.
+Step-by-Step Implementation
+Break down the setup process into clear, actionable steps.
+Include well-commented code snippets for each step.
+What Happens Behind the Scenes
+Provide technical insights into how the system works internally (if relevant).
+Explain any significant impacts on performance or functionality.
+Tips and Best Practices
+Share advice for effective usage.
+Highlight common pitfalls and how to avoid them.
+Running the Feature
+Explain how to test or interact with the feature after setup.
+To keep communication clear and accessible for everyone:
+This way, conversations stay organized, visible to the community, and future contributors can benefit from past discussions.
+Thank you for contributing to the Untold Engine! Following these guidelines will ensure that your work aligns with the project's goals and provides value to users.
+ + + + + + + + + + + + + +To maintain a consistent code style across the Untold Engine repo, we use SwiftFormat. SwiftFormat is a code formatter for Swift that helps enforce Swift style conventions and keep the codebase clean. If you don't have SwiftFormat installed, see the Installing SwiftFormat section below.
+Navigate to the root directory of Untold Engine and then run the commands below:
+To lint (check) all Swift files without making changes:
+ +Or, using the Makefile:
+ +This command runs the same lint configuration as our GitHub Actions workflow and pre-commit hook, ensuring consistent results locally and in CI.
+To format files:
+ +Alternatively, you can use the Makefile shortcut:
+ +💡 Tip +If the pre-commit hook blocks your commit due to formatting issues, simply run:
+ +then re-stage your changes and try committing again.
+You can bypass the hook temporarily (not recommended) with:
+ +The simplest way to install SwiftFormat is through the command line.
+Format a Single File
+To format a specific Swift file:
+Open the terminal and navigate to your project directory.
+Run the following command:
+To format all Swift files in your project:
+Navigate to your project directory in the terminal.
+Run the following command:
+This will recursively format all Swift files in the current directory and its subdirectories.
+ + + + + + + + + + + + + +To help us identity the purpose of your commits, make sure to use the following tags in your commit messages. The tags will also automatically increment the the current version when pushed to github.
+