Capabilities

USDKit is a new first-party Swift framework introduced in iOS/macOS 27 that brings native USD scene creation, composition, modification, and export capabilities to Apple platform apps, with deep RealityKit and Spatial Preview integration.

visionOS supports fully photo-realistic, spatially-aware 360° immersive environments with true depth, parallax, spatial audio, and motion — going far beyond static panoramas. Developers can now surface these environments on both native visionOS apps and the spatial web.

visionOS 27 expands ARKit object tracking with high-frame-rate tracking, metric-space pose queries, extended Create ML training, spatial accessory support, and brings object tracking to iOS for the first time.

Safari on visionOS now exposes a JavaScript immersive API that lets websites transport users into full 3D environments using the HTML model element, mirroring the Fullscreen API pattern with requestImmersive(). Developers can build inline 3D previews and seamless transitions into spatial environments directly from a webpage.

visionOS 27 introduces ClippingComponent and enhanced ManipulationComponent placement patterns in RealityKit, enabling multiple users in a SharePlay session to simultaneously manipulate, disassemble, and cross-section structured 3D assemblies in shared space.

visionOS 27 introduces Gaussian Splatting support in RealityKit, enabling developers to scan real-world objects and render them as photorealistic 3D Gaussian splats directly in their spatial experiences. This bridges the gap between physical reality and virtual content without requiring manual 3D modeling.

Reality Composer Pro 3 introduces a node-based visual scripting system called Script Graph that lets developers and designers build interactive RealityKit experiences — including full games — without writing any code. Features include drag gesture events, physics integration, prototyped subgraphs, custom events, and live preview directly on Vision Pro.

Reality Composer Pro 3 is a standalone tool (no longer bundled inside Xcode) that introduces live on-device preview, GPU-based Compute Graphs, a Lightmapping system for baked indirect lighting, a prototype/instancing workflow, and an AI Assistant that generates 3D objects and materials on demand for visionOS experiences.

Xcode plugins can now extend Reality Composer Pro 3 by registering custom ECS components, systems, animation actions, and Script Graph nodes so artists can iterate on 3D content directly inside the editor without rebuilding the app.

The FoveatedStreaming framework lets visionOS apps connect to an external PC or cloud endpoint to stream OpenXR content directly to Apple Vision Pro, using eye-tracking-based foveated video compression and the built-in NVIDIA CloudXR runtime for low-latency wireless delivery.

iOS 27 adds soft shadow rendering to RealityKit spotlights and point lights via a configurable lightSize and quality, plus a SurroundingsLightComponent that lets virtual lights illuminate real-world surfaces using the scene understanding mesh.

Reality Composer Pro 3 introduces Animation Graph, Behavior Trees, Script Graph, Navigation Mesh, and Compute Graph — a suite of visual, node-based editors that let you author character animation, autonomous AI routines, interactivity, pathfinding, and particle effects entirely within the editor without writing code.