Apple announces new AI and ML-based eye tracking features and music haptics accessibility features

Apple announced several new accessibility-focused features for its iPhone and iPad devices on Wednesday. The company regularly introduces new accessibility features to make the devices easier for those with physical disabilities to use. This year, the tech giant is introducing a new eye-tracking feature that will allow users to control their device with just eye movements. In addition, Music Haptics will allow users to experience music through vibrations, and Vocal Shortcuts will allow them to perform tasks with custom sounds.

Functions were declared using a post in the company’s editorial office. Apple’s senior director of global accessibility policy and initiatives, Sarah Herlinger, said: “Every year we break new ground when it comes to accessibility. These new features will impact the lives of a wide range of users, providing new ways to communicate, control their devices and move around the world.”

First, the Eye Tracking feature offers users a built-in option to control the iPhone and iPad with just eye movements. Powered by artificial intelligence (AI), this feature uses the front-facing camera that can be calibrated with the user’s eyes, and the device’s machine learning (ML) features track eyes to allow people with physical disabilities to easily navigate the phone . The company says it does not have access to user data.

Music Haptics is another new feature that offers a unique way for hearing impaired users to experience music. The iPhone feature uses the Taptic Engine to reproduce taps, textures and vibrations to match the sound of the music. Apple says the feature can play millions of songs in the Apple Music catalog. It will also be available as an API for developers to integrate it into their music apps.

Next, Vocal Shortcuts for iPhone and iPad users is designed to help people suffering from speech-related disabilities. It allows users to set custom utterances that can be understood by Siri to launch shortcuts and complete tasks. Additionally, a new feature called Vehicle Motion Cues adds animated dots around the edges of the screen to reduce the sensory conflict between what one sees and feels. Citing research, Apple said this conflict is one of the main causes behind motion sickness, and the feature could reduce those symptoms.

Additionally, CarPlay also gets voice control, sound recognition and color filters to help users with various disabilities. Apple’s latest product line, the Vision Pro, also gets a system-wide live caption feature for the hearing impaired.