Orchard Loop

Apple Intelligence used to improve iOS, iPadOS, visionOS, tvOS accessibility features

From Apple:

Apple today previewed a suite of accessibility updates that use Apple Intelligence to bring new capabilities to features users rely on every day, including VoiceOver, Magnifier, Voice Control, and Accessibility Reader. Apple also announced on-device generated subtitles for uncaptioned video content coming to the Apple ecosystem, as well as a new feature for Apple Vision Pro users to control compatible wheelchairs with their eyes.

There’s a lot to unpack in Apple’s press release today. While some got an early briefing, we can’t yet test these features. Apple says they’re all coming later this year, presumably in iOS 27.

Some of the key highlights:

  • Apple Intelligence is coming to VoiceOver and Magnifier, allowing users to get more detailed descriptions of images system-wide
  • Users will also be able to use Live Recognition (debuted in iOS 26) to ask a question about what’s in the camera viewfinder and get a more detailed response
  • Magnifier will support voice control (eg. to “Zoom in”) and can be opened via the Action button
  • Voice Control gains support for natural language control, allowing users to “say what they see” - a great improvement for usability over the experience today that is far more prescriptive
  • Accessibility Reader doesn’t miss out on Apple Intelligence support, with on-demand summaries and built-in translations coming
  • Videos will soon be able to display automatically generated subtitles where captions or subtitles aren’t available
  • Users in supported power wheelchairs will soon be able to control their drive system using the Vision Pro eye-tracking system - what a great use case!
  • visionOS gains support for Vehicle Motion Cues that are already available in iOS today
  • Touch Accommodations will provide a new way to “personalize setup in iOS and iPadOS”
  • Apple has improved pairing, hand-off and setup for Made for iPhone hearing aids
  • tvOS will support larger text, allowing the text size to be increased - this might be one to watch as a tvOS developer
  • A new API will be added to support adding a human interpreter to an ongoing FaceTime video call

Notably there were no mentions of new watchOS accessibility features this year. Hopefully we still see some announced at WWDC.