The FoveatedStreaming framework lets visionOS apps connect to an external PC or cloud endpoint to stream OpenXR content directly to Apple Vision Pro, using eye-tracking-based foveated video compression and the built-in NVIDIA CloudXR runtime for low-latency wireless delivery.
⢠Enables high-fidelity PC-class OpenXR experiences (flight sims, racing games, industrial visualisation) on Apple Vision Pro without any native port ā get streaming in a day
⢠Foveated compression is handled automatically by visionOS using eye tracking, delivering sharp detail where the user looks while keeping bandwidth manageable over Wi-Fi
⢠Full integration with ARKit, SwiftUI, and RealityKit means streamed content can be spatially anchored to real-world objects and composed with native on-device rendering
A minimal visionOS app that creates a FoveatedStreamingSession, connects to a nearby streaming endpoint, and presents the streamed OpenXR content inside a progressive ImmersiveSpace alongside a native SwiftUI control window.
import SwiftUI
import FoveatedStreaming
// MARK: - App entry point
@main
struct FoveatedStreamingApp: App {
@State private var session = FoveatedStreamingSession()
var body: some Scene {
// Native SwiftUI control window
WindowGroup("Controls", id: "controls") {
ControlView(session: session)
}
// Immersive space that hosts the streamed content
ImmersiveSpace(id: "stream") {
StreamingImmersiveView(session: session)
}
.immersionStyle(selection: .constant(.progressive), in: .progressive)
}
}
// MARK: - Control window
struct ControlView: View {
let session: FoveatedStreamingSession
@Environment(\.openImmersiveSpace) private var openImmersiveSpace
@Environment(\.dismissImmersiveSpace) private var dismissImmersiveSpace
@State private var isStreaming = false
@State private var connectionError: String?
var body: some View {
VStack(spacing: 20) {
Text("Foveated Streaming")
.font(.title)
if let error = connectionError {
Text(error)
.foregroundStyle(.red)
.font(.caption)
}
Button(isStreaming ? "Disconnect" : "Connect to Endpoint") {
Task {
if isStreaming {
session.disconnect()
await dismissImmersiveSpace()
isStreaming = false
} else {
await startStreaming()
}
}
}
.buttonStyle(.borderedProminent)
}
.padding(40)
}
private func startStreaming() async {
do {
// connect() surfaces the system endpoint picker and QR-code
// pairing UI automatically; it returns once streaming begins.
try await session.connect()
await openImmersiveSpace(id: "stream")
isStreaming = true
} catch {
connectionError = error.localizedDescription
}
}
}
// MARK: - Immersive space content
struct StreamingImmersiveView: View {
let session: FoveatedStreamingSession
var body: some View {
// Passing the session to the ImmersiveSpace view hierarchy
// causes visionOS to composite the foveated stream into the space.
FoveatedStreamingView(session: session)
.onAppear {
// Example: send an opaque message to the OpenXR host
// once the immersive space is visible.
let payload = Data("{ \"event\": \"immersiveSpaceReady\" }".utf8)
session.sendMessage(payload, channel: "app-events")
}
}
}USDKit is a new first-party Swift framework introduced in iOS/macOS 27 that brings native USD scene creation, composition, modification, and export capabilities to Apple platform apps, with deep RealityKit and Spatial Preview integration.
iOS 27 adds sectioned queries, codable model attributes, ResultsObserver for non-SwiftUI change observation, and HistoryObserver for reacting to persistent history changes in SwiftData.
LiveCommunicationKit is the modern replacement for CXProvider that delivers rich, native conversation UIs integrated with the Lock Screen, Dynamic Island, Phone app Recents, and Siri. It provides a unified lifecycle model for audio and video conversations with a single delegate-driven action pipeline.
In-depth guide
RealityKit & USDKit in iOS 27 āFoveatedStreaming is a visionOS-only framework ā it does not exist on iOS or iPadOS. The NVIDIA CloudXR runtime and the Foveated Streaming Protocol (TCP/JSON pairing over Bonjour) must be implemented on the Windows host; Apple provides open-source reference code on GitHub. Pairing requires scanning a QR code displayed by the endpoint. When the headset is removed it goes to sleep and severs all connections ā the host must stay discoverable so users can reconnect. Depth buffer and alpha channel support on the OpenXR side are strongly recommended for proper occlusion and passthrough blending.
Requires Apple Vision Pro; streaming endpoint must run Windows with NVIDIA CloudXR SDK; Wi-Fi or local network required; no additional Apple Silicon constraint but GPU-capable PC recommended on the host side
iOS 27 expands the App Intents framework with RelevantEntities for contextual content surfacing, EntityCollection for high-performance batch parameter handling, SyncableEntity for cross-device Siri continuity, native Duration/PersonNameComponents parameter types, and @UnionValue for multi-type parameters ā plus extended intent execution time limits.