macOS 27 ships a pre-installed `fm` command-line tool and a pip-installable Python SDK that expose the on-device Apple Foundation Model outside of Swift apps, enabling shell scripts, automation pipelines, and Python-based ML evaluation workflows.
โข Developers can prototype and test Foundation Models prompts directly from a terminal or Jupyter Notebook without writing Swift or rebuilding an Xcode project.
โข The Python SDK integrates with the rich Python ML/data science ecosystem (Pandas, Jupyter, etc.) making it easy to build evaluation pipelines that quantify prompt quality.
โข The fm CLI's structured-output support (via `fm schema` + `fm respond --schema`) lets shell scripts consume model results as JSON, unlocking AI-powered automation without any API keys or cloud costs.
Shows how to use the fm CLI to classify files as draft vs. final via structured JSON output in a shell script, then demonstrates the equivalent Python SDK session call for prototyping the same prompt in a notebook.
// NOTE: fm CLI is a macOS shell tool; the Swift snippet below shows
// how you would accomplish the same structured-output task using the
// FoundationModels Swift framework on iOS/macOS 27+, mirroring what
// the fm CLI does under the hood.
import FoundationModels
import Foundation
// 1. Define the structured output type (mirrors `fm schema object` in the CLI)
struct FileSortResult: Codable, Generable {
@Guide(description: "Files identified as final versions")
var finalFiles: [String]
@Guide(description: "Files identified as drafts or work-in-progress")
var draftFiles: [String]
}
// 2. Prepare a list of file names to classify
let fileNames = [
"logo_v1.png", "logo_final.png", "hero_draft2.jpg",
"hero_FINAL.jpg", "icon_wip.png", "icon_approved.png"
]
let fileList = fileNames.joined(separator: "\n")
// 3. Create a session with instructions
let session = LanguageModelSession(
instructions: "You are a file organizer. Classify each filename as either a final deliverable or a draft/work-in-progress based on its name."
)
// 4. Prompt the model and request structured output
let prompt = "Classify these files:\n\(fileList)"
do {
let response = try await session.respond(
to: prompt,
generating: FileSortResult.self
)
let result = response.value
print("โ
Final files: \(result.finalFiles)")
print("๐๏ธ Draft files: \(result.draftFiles)")
} catch {
print("Error: \(error)")
}
// --- Equivalent fm CLI one-liner (run in macOS 27 Terminal) ---
// SCHEMA=$(fm schema object '{"finalFiles":["string"],"draftFiles":["string"]}')
// fm respond "Classify these files: logo_v1.png logo_final.png hero_draft2.jpg" --schema "$SCHEMA"Foundation Models is a new Apple framework introduced in iOS 27 that gives developers on-device access to the same Apple Intelligence language model powering system features, enabling text generation, structured output, and tool-calling entirely on-device without a network connection.
iOS 27 opens the Foundation Models framework to third-party LLM providers via a new public LanguageModel protocol, allowing anyone to integrate server-based or local models (e.g. Claude, Gemini, CoreAI, MLX) using the same API as Apple's on-device system model.
iOS 27 introduces GenerateIterativeSegmentationRequest in the Vision framework, letting users interactively isolate any object in an image by providing a point, bounding box, lasso, or scribble as a seed, then iteratively refine the resulting mask.
In-depth guide
iOS 27 On-Device AI & Apple Intelligence โThe fm CLI and Python SDK are macOS-only โ they do not run on iOS/iPadOS devices. The Python SDK must be installed via pip into a Python 3.10+ environment and requires Xcode to be present. Private Cloud Compute usage is rate-limited. The Python SDK mirrors the Swift FoundationModels API surface but is a separate package, so Swift-only features may lag behind.
Apple Silicon Mac required for both the fm CLI and Python SDK. The on-device model is always available; the Private Cloud Compute model is subject to usage limits.
Visual Intelligence brings iOS 17's Visual Look Up capabilities to a new developer-facing API surface in iOS 27, letting apps pipe live camera frames or static images through on-device scene understanding to extract subjects, text, barcodes, and rich semantic labels without any cloud round-trip.