This AI app is helping visually impaired people find and navigate objects around them

What’s happened? For people who can’t rely on sight, a new AI tool is stepping in as an extra set of senses. Researchers at Penn State have developed an app called NaviSense that can locate objects in real time using voice commands, then guide users toward them using spatial audio cues and vibration feedback from the phone.

• The app tracks the position of objects around you using a camera sensor input.
• It translates spatial information into sound cues to guide users towards the object.
• Since NaviSense works in real-time, it requires no long setup or manual marking of objects.
• Designed to help visually impaired users navigate indoor and outdoor spaces more safely.

This is important because: Previous systems made for the visually impaired required preloaded object models, which made them slow and limited. NaviSense overcomes this limitation by connecting to an external server powered by AI models to recognize objects in real-time based on voice command, without pre-training the environment.

It utilizes the smartphone camera, spatial audio, and vibration to guide you toward an object, telling users if it’s left, right, up, or down, and giving a “bullseye” confirmation when their hand is in the right spot. The app also tracks hand movement through phone motion, asks follow-up questions if your request is unclear, and adapts instantly to new spaces like kitchens, sidewalks, or stores.

Why should I care? Even if you’re not visually impaired, this shows how AI can bridge gaps and make everyday life more accessible. In a world where technology too often feels flashy, this one feels deeply human.

• NaviSense could give visually impaired individuals far more independence and confidence while navigating unfamiliar spaces.
• The technology could be built into smartphones or wearables, making it more accessible in everyday situations.

OK, what’s next? Vijaykrishnan Narayanan, an Evan Pugh University Professor working on NaviSense, said, “This technology is quite close to commercial release, and we’re working to make it even more accessible.” AI often sounds futuristic or abstract, but this is the kind of real-world impact that hits home.