iOS 27 introduces on-device AI-generated subtitles that automatically transcribe or translate audio/subtitles during video playback, plus a new subtitle style preview API that lets users audition caption styles without leaving the player.
⢠Generated subtitles are created live, on-device via Speech-to-Text or translation models ā no server round-trips, no extra code required to activate them
⢠Subtitle style preview (new AVPlayerLayer API) lets users instantly see how each caption style looks while watching, dramatically improving accessibility
⢠Both features work across HLS live streams, VOD, and file-based content, meaning virtually every video app benefits automatically
Shows how to fetch system caption style profiles and use the new AVPlayerLayer API to display a live style preview during playback, then commit the chosen style.
import SwiftUI
import AVFoundation
import AVKit
struct SubtitleStylePreviewView: View {
private let player: AVPlayer
private let playerLayer: AVPlayerLayer
// Each style in the system has a profile ID
@State private var styleProfiles: [(id: String, name: String)] = []
@State private var selectedProfileID: String? = nil
init() {
let url = URL(string: "https://devstreaming-cdn.apple.com/videos/wwdc/2023/101/4/3CB5A0B4-B344-4173-98E6-C5A6C0B0A14D/cmaf/hvc/2160p_16800/hvc_2160p_16800.m3u8")!
self.player = AVPlayer(url: url)
self.playerLayer = AVPlayerLayer(player: player)
}
var body: some View {
VStack(spacing: 0) {
PlayerLayerView(playerLayer: playerLayer)
.frame(maxWidth: .infinity)
.aspectRatio(16/9, contentMode: .fit)
ScrollView(.horizontal, showsIndicators: false) {
HStack(spacing: 12) {
ForEach(styleProfiles, id: \.id) { profile in
Button(profile.name) {
previewStyle(profileID: profile.id)
}
.padding(.horizontal, 14)
.padding(.vertical, 8)
.background(
selectedProfileID == profile.id
? Color.accentColor
: Color.secondary.opacity(0.2)
)
.foregroundStyle(
selectedProfileID == profile.id ? .white : .primary
)
.clipShape(Capsule())
}
}
.padding()
}
HStack {
Button("Cancel") {
// Removes preview, restores active subtitles
playerLayer.stopSubtitleStylePreview()
selectedProfileID = nil
}
Spacer()
Button("Apply Style") {
if let id = selectedProfileID {
// Persist the chosen style system-wide
AVCaptionStyleProfile.setCurrentProfileID(id)
}
playerLayer.stopSubtitleStylePreview()
}
.disabled(selectedProfileID == nil)
.buttonStyle(.borderedProminent)
}
.padding()
}
.onAppear {
loadStyleProfiles()
player.play()
}
}
private func loadStyleProfiles() {
// Fetch all system caption style profile IDs and their display names
let profiles = AVCaptionStyleProfile.profileIDs.compactMap { id -> (id: String, name: String)? in
guard let name = AVCaptionStyleProfile.localizedName(forProfileID: id) else { return nil }
return (id: id, name: name)
}
styleProfiles = profiles
}
private func previewStyle(profileID: String) {
selectedProfileID = profileID
// Show live styled preview on the player layer.
// nil text ā system shows localized placeholder string.
// position offsets the preview text downward to avoid our HStack controls.
playerLayer.showSubtitleStylePreview(
forProfileID: profileID,
text: nil,
position: CGPoint(x: 0, y: -60)
)
}
}
// MARK: - UIViewRepresentable wrapper for AVPlayerLayer
struct PlayerLayerView: UIViewRepresentable {
let playerLayer: AVPlayerLayer
func makeUIView(context: Context) -> UIView {
let view = UIView()
view.backgroundColor = .black
playerLayer.videoGravity = .resizeAspect
view.layer.addSublayer(playerLayer)
return view
}
func updateUIView(_ uiView: UIView, context: Context) {
playerLayer.frame = uiView.bounds
}
}Foundation Models is a new Apple framework introduced in iOS 27 that gives developers on-device access to the same Apple Intelligence language model powering system features, enabling text generation, structured output, and tool-calling entirely on-device without a network connection.
iOS 27 opens the Foundation Models framework to third-party LLM providers via a new public LanguageModel protocol, enabling anyone to integrate custom, server-based, or open-source models using the same Swift API as Apple's on-device system model.
App Schemas let developers describe their app's content and actions using pre-defined domain schemas (like the Calendar domain) so Siri can understand, search, and act on app data without custom NLP. Entities conforming to IndexedEntity are donated to Spotlight's semantic index, enabling natural-language queries over app content.
In-depth guide
iOS 27 On-Device AI & Apple Intelligence āGenerated subtitles appear automatically ā no opt-in code needed, but you must provide subtitle selection UI (AVPlayerViewController handles this for free). The style preview API on AVPlayerLayer requires you to fetch profile IDs first, call stopSubtitleStylePreview() when done, and pass nil text to show localized system placeholder text. Passing a position offset is important to avoid overlapping your own controls.
On-device AI generation requires Apple Intelligence-capable devices; English speech transcription and multi-language translation from English subtitles supported at launch
Visual Intelligence brings iOS 17's Visual Look Up capabilities to a new developer-facing API surface in iOS 27, letting apps pipe live camera frames or static images through on-device scene understanding to extract subjects, text, barcodes, and rich semantic labels without any cloud round-trip.