macOS 27 ships a pre-installed `fm` command-line tool and a pip-installable Python SDK that expose the on-device Apple Foundation Model outside of Swift apps, enabling shell scripts, automation pipelines, and Python-based ML evaluation workflows.
โข Developers can prototype and test Foundation Models prompts directly from a terminal or Jupyter Notebook without writing Swift or rebuilding an Xcode project.
โข The Python SDK integrates with the rich Python ML/data science ecosystem (Pandas, Jupyter, etc.) making it easy to build evaluation pipelines that quantify prompt quality.
โข The fm CLI's structured-output support (via `fm schema` + `fm respond --schema`) lets shell scripts consume model results as JSON, unlocking AI-powered automation without any API keys or cloud costs.
Shows how to use the fm CLI to classify files as draft vs. final via structured JSON output in a shell script, then demonstrates the equivalent Python SDK session call for prototyping the same prompt in a notebook.
// NOTE: fm CLI is a macOS shell tool; the Swift snippet below shows
// how you would accomplish the same structured-output task using the
// FoundationModels Swift framework on iOS/macOS 27+, mirroring what
// the fm CLI does under the hood.
import FoundationModels
import Foundation
// 1. Define the structured output type (mirrors `fm schema object` in the CLI)
struct FileSortResult: Codable, Generable {
@Guide(description: "Files identified as final versions")
var finalFiles: [String]
@Guide(description: "Files identified as drafts or work-in-progress")
var draftFiles: [String]
}
// 2. Prepare a list of file names to classify
let fileNames = [
"logo_v1.png", "logo_final.png", "hero_draft2.jpg",
"hero_FINAL.jpg", "icon_wip.png", "icon_approved.png"
]
let fileList = fileNames.joined(separator: "\n")
// 3. Create a session with instructions
let session = LanguageModelSession(
instructions: "You are a file organizer. Classify each filename as either a final deliverable or a draft/work-in-progress based on its name."
)
// 4. Prompt the model and request structured output
let prompt = "Classify these files:\n\(fileList)"
do {
let response = try await session.respond(
to: prompt,
generating: FileSortResult.self
)
let result = response.value
print("โ
Final files: \(result.finalFiles)")
print("๐๏ธ Draft files: \(result.draftFiles)")
} catch {
print("Error: \(error)")
}
// --- Equivalent fm CLI one-liner (run in macOS 27 Terminal) ---
// SCHEMA=$(fm schema object '{"finalFiles":["string"],"draftFiles":["string"]}')
// fm respond "Classify these files: logo_v1.png logo_final.png hero_draft2.jpg" --schema "$SCHEMA"Foundation Models is a new Apple framework introduced in iOS 27 that gives developers on-device access to the same Apple Intelligence language model powering system features, enabling text generation, structured output, and tool-calling entirely on-device without a network connection.
App Schemas let developers describe their app's content and actions using pre-defined domain schemas (like the Calendar domain) so Siri can understand, search, and act on app data without custom NLP. Entities conforming to IndexedEntity are donated to Spotlight's semantic index, enabling natural-language queries over app content.
iOS 27 introduces GenerateIterativeSegmentationRequest in the Vision framework, letting users interactively isolate any object in an image by providing a point, bounding box, lasso, or scribble as a seed, then iteratively refine the resulting mask.
In-depth guide
iOS 27 On-Device AI & Apple Intelligence โThe fm CLI and Python SDK are macOS-only โ they do not run on iOS/iPadOS devices. The Python SDK must be installed via pip into a Python 3.10+ environment and requires Xcode to be present. Private Cloud Compute usage is rate-limited. The Python SDK mirrors the Swift FoundationModels API surface but is a separate package, so Swift-only features may lag behind.
Apple Silicon Mac required for both the fm CLI and Python SDK. The on-device model is always available; the Private Cloud Compute model is subject to usage limits.
Visual Intelligence brings iOS 17's Visual Look Up capabilities to a new developer-facing API surface in iOS 27, letting apps pipe live camera frames or static images through on-device scene understanding to extract subjects, text, barcodes, and rich semantic labels without any cloud round-trip.