The FoveatedStreaming framework lets visionOS apps connect to an external PC or cloud endpoint to stream OpenXR content directly to Apple Vision Pro, using eye-tracking-based foveated video compression and the built-in NVIDIA CloudXR runtime for low-latency wireless delivery.
โข Enables high-fidelity PC-class OpenXR experiences (flight sims, racing games, industrial visualisation) on Apple Vision Pro without any native port โ get streaming in a day
โข Foveated compression is handled automatically by visionOS using eye tracking, delivering sharp detail where the user looks while keeping bandwidth manageable over Wi-Fi
โข Full integration with ARKit, SwiftUI, and RealityKit means streamed content can be spatially anchored to real-world objects and composed with native on-device rendering
A minimal visionOS app that creates a FoveatedStreamingSession, connects to a nearby streaming endpoint, and presents the streamed OpenXR content inside a progressive ImmersiveSpace alongside a native SwiftUI control window.
import SwiftUI
import FoveatedStreaming
// MARK: - App entry point
@main
struct FoveatedStreamingApp: App {
@State private var session = FoveatedStreamingSession()
var body: some Scene {
// Native SwiftUI control window
WindowGroup("Controls", id: "controls") {
ControlView(session: session)
}
// Immersive space that hosts the streamed content
ImmersiveSpace(id: "stream") {
StreamingImmersiveView(session: session)
}
.immersionStyle(selection: .constant(.progressive), in: .progressive)
}
}
// MARK: - Control window
struct ControlView: View {
let session: FoveatedStreamingSession
@Environment(\.openImmersiveSpace) private var openImmersiveSpace
@Environment(\.dismissImmersiveSpace) private var dismissImmersiveSpace
@State private var isStreaming = false
@State private var connectionError: String?
var body: some View {
VStack(spacing: 20) {
Text("Foveated Streaming")
.font(.title)
if let error = connectionError {
Text(error)
.foregroundStyle(.red)
.font(.caption)
}
Button(isStreaming ? "Disconnect" : "Connect to Endpoint") {
Task {
if isStreaming {
session.disconnect()
await dismissImmersiveSpace()
isStreaming = false
} else {
await startStreaming()
}
}
}
.buttonStyle(.borderedProminent)
}
.padding(40)
}
private func startStreaming() async {
do {
// connect() surfaces the system endpoint picker and QR-code
// pairing UI automatically; it returns once streaming begins.
try await session.connect()
await openImmersiveSpace(id: "stream")
isStreaming = true
} catch {
connectionError = error.localizedDescription
}
}
}
// MARK: - Immersive space content
struct StreamingImmersiveView: View {
let session: FoveatedStreamingSession
var body: some View {
// Passing the session to the ImmersiveSpace view hierarchy
// causes visionOS to composite the foveated stream into the space.
FoveatedStreamingView(session: session)
.onAppear {
// Example: send an opaque message to the OpenXR host
// once the immersive space is visible.
let payload = Data("{ \"event\": \"immersiveSpaceReady\" }".utf8)
session.sendMessage(payload, channel: "app-events")
}
}
}LiveCommunicationKit is the modern replacement for CXProvider that delivers rich, native conversation UIs integrated with the Lock Screen, Dynamic Island, Phone app Recents, and Siri. It provides a unified lifecycle model for audio and video conversations with a single delegate-driven action pipeline.
USDKit is a new first-party Swift framework introduced in iOS/macOS 27 that brings native USD scene creation, composition, modification, and export capabilities to Apple platform apps, with deep RealityKit and Spatial Preview integration.
iOS 27 adds sectioned queries, codable model attributes, ResultsObserver for non-SwiftUI change observation, and HistoryObserver for reacting to persistent history changes in SwiftData.
In-depth guide
RealityKit & USDKit in iOS 27 โFoveatedStreaming is a visionOS-only framework โ it does not exist on iOS or iPadOS. The NVIDIA CloudXR runtime and the Foveated Streaming Protocol (TCP/JSON pairing over Bonjour) must be implemented on the Windows host; Apple provides open-source reference code on GitHub. Pairing requires scanning a QR code displayed by the endpoint. When the headset is removed it goes to sleep and severs all connections โ the host must stay discoverable so users can reconnect. Depth buffer and alpha channel support on the OpenXR side are strongly recommended for proper occlusion and passthrough blending.
Requires Apple Vision Pro; streaming endpoint must run Windows with NVIDIA CloudXR SDK; Wi-Fi or local network required; no additional Apple Silicon constraint but GPU-capable PC recommended on the host side
The NowPlaying framework introduces a first-class Swift API for surfacing app media in system-wide now-playing surfaces โ Lock Screen, Control Center, Dynamic Island, StandBy, CarPlay, Apple Watch, and Apple TV โ via a declarative MediaSessionRepresentable protocol. It also supports remote media sessions (for controlling external speakers/TVs) and Media Sharing Extensions for routing media to third-party devices.