A new on-device framework that analyzes audio files for musical properties including beat/rhythm, key signature, structure (sections/segments/phrases), pace, instrument activity, and loudness — all without requiring ML or signal processing expertise.
• Enables rich music-driven experiences like beat-synced video editing, DJ apps, and reactive game audio without any server-side processing or ML knowledge
• All analysis runs entirely on-device, keeping user audio private and working offline
• Pre-computed results are fully Codable, so you can bundle analysis data with your app or export it for later use
Loads a local audio file, runs Music Understanding analysis, and displays the detected key signature and BPM alongside a list of beat timestamps.
import SwiftUI
import MusicUnderstanding
import AVFoundation
struct BeatKeyAnalyzerView: View {
@State private var bpm: Double? = nil
@State private var keyLabel: String = "—"
@State private var beatTimes: [String] = []
@State private var isAnalyzing = false
@State private var showFilePicker = false
@State private var selectedURL: URL? = nil
var body: some View {
NavigationStack {
VStack(spacing: 20) {
Button("Select Audio File") {
showFilePicker = true
}
.buttonStyle(.borderedProminent)
if isAnalyzing {
ProgressView("Analyzing…")
} else {
VStack(alignment: .leading, spacing: 12) {
Label("Key: \(keyLabel)", systemImage: "music.note")
.font(.title2.bold())
Label("BPM: \(bpm.map { String(format: "%.1f", $0) } ?? "—")", systemImage: "metronome")
.font(.title2.bold())
Divider()
Text("First 10 Beat Timestamps")
.font(.headline)
ForEach(beatTimes, id: \.self) { t in
Text(t)
.font(.caption)
.foregroundStyle(.secondary)
}
}
.padding()
.frame(maxWidth: .infinity, alignment: .leading)
.background(.regularMaterial, in: RoundedRectangle(cornerRadius: 16))
.padding(.horizontal)
}
Spacer()
}
.navigationTitle("Music Understanding")
.fileImporter(
isPresented: $showFilePicker,
allowedContentTypes: [.audio]
) { result in
if case .success(let url) = result {
selectedURL = url
Task { await analyzeAudio(url: url) }
}
}
}
}
func analyzeAudio(url: URL) async {
isAnalyzing = true
defer { isAnalyzing = false }
let asset = AVURLAsset(
url: url,
options: [AVURLAssetPreferPreciseDurationAndTimingKey: true]
)
do {
let session = try MusicUnderstandingSession(asset: asset)
let results = try await session.analyze(for: [.rhythm, .key])
// Key
if let keyResult = results.key,
let firstRange = keyResult.ranges.first {
let sig = firstRange.value
keyLabel = "\(sig.tonic) \(sig.mode == .major ? "Major" : "Minor")"
}
// Rhythm
if let rhythmResult = results.rhythm {
bpm = rhythmResult.beatsPerMinute
beatTimes = rhythmResult.beats.prefix(10).map { beat in
let seconds = beat.seconds
return String(format: "%.3f s", seconds)
}
}
} catch {
keyLabel = "Error: \(error.localizedDescription)"
}
}
}
#Preview {
BeatKeyAnalyzerView()
}Foundation Models is a new Apple framework introduced in iOS 27 that gives developers on-device access to the same Apple Intelligence language model powering system features, enabling text generation, structured output, and tool-calling entirely on-device without a network connection.
Visual Intelligence brings iOS 17's Visual Look Up capabilities to a new developer-facing API surface in iOS 27, letting apps pipe live camera frames or static images through on-device scene understanding to extract subjects, text, barcodes, and rich semantic labels without any cloud round-trip.
iOS 27 opens the Foundation Models framework to third-party LLM providers via a new public LanguageModel protocol, allowing anyone to integrate server-based or local models (e.g. Claude, Gemini, CoreAI, MLX) using the same API as Apple's on-device system model.
In-depth guide
iOS 27 On-Device AI & Apple Intelligence →Set AVURLAssetPreferPreciseDurationAndTimingKey to true on your AVURLAsset for accurate timing results. The beatsPerMinute property on RhythmResult is optional — it returns nil until at least two beats have been detected. When using the targeted analyze(for:) API, all unrequested result fields will be nil.
Runs on-device; no Apple Intelligence hardware required, but performance may vary on older devices
iOS 27 opens the Foundation Models framework to third-party LLM providers via a new public LanguageModel protocol, enabling anyone to integrate custom, server-based, or open-source models using the same Swift API as Apple's on-device system model.