Northeastern students in Seattle and Vancouver build AI tools to support fitness and mobility
More than 200 computer science students on over 30 teams participated in the event sponsored by Qualcomm and Microsoft.
Sometimes the best way to push the limits of AI is to push the limits of the human body.
That’s exactly what two student teams at Northeastern University did, creating innovative artificial intelligence tools designed to support human movement — either during exercise or while using a wheelchair.
“I have taken a class about computer vision, but I didn’t really know how it could be used to help people with mobility difficulties,” said Tianyu Fang, who collaborated with fellow graduate students on the Vancouver campus to design an AI-powered wheelchair navigation assistant.
“This was a way to apply what I’ve learned in class to real-life problems,” Fang said.
Their product, called Sightmate, uses live video feeds to detect obstacles in the path of a wheelchair. Mounted on a laptop attached to the chair, the system converts visual data into text, which is then translated into an audible message alerting the user.
Sightmate runs locally on the user’s computer without an internet connection, but does use an external YOLOv8 Nano camera that can capture what is in the wheelchair’s surrounding environment with a verbal command.
Image-to-text software converts to an audio message to alert the user if someone, or something, is in the way.
“If nothing is in front of you, then the system will notify you that it’s clear, good to go,” Fang said.
Real-time exercise feedback
Meanwhile, on Northeastern’s Seattle campus, a second group of students developed a different kind of movement assistant — one designed to provide real-time exercise feedback without costly and bulky fitness mirrors.
The project, called FitMirror, uses a projector to demonstrate the correct way to do an exercise.
“We had a conversation about how expensive and bulky fitness mirrors can be,” Xinlei Li said. “We wanted to build something more accessible, combining computer vision to track the movements.”
The platform uses a large language model to analyze a user’s body posture and can recognize how the person is moving in relation to a projected image.
“It’s like having a private instructor by your side to instruct you,” said Seattle team member Ruixin Shi.
Like a coach, FitMirror tells users if they’re not squatting deeply enough, Shi said, and gives encouragement when they’re doing the exercise correctly.
Editor’s Picks
The tiny ticks that cause Lyme seem to have superpowers that make them hard to kill. But you can protect yourself by following these steps
Trialled on Tower Bridge and ready for toxic zones — Northeastern students show off their market-ready engineering products in London
Barbara Lee, Mills College graduate and longtime U.S. representative, elected mayor of Oakland
Living tissues may form like avalanches, Northeastern researchers say — a discovery that could aid new treatments
Northeastern announces speakers for 2025 global campus commencements, and college and school ceremonies
Dual-campus student competition
These projects were developed as part of a dual-campus student competition that was held simultaneously on Northeastern’s Vancouver and Seattle campuses. More than 200 computer science graduate students on over 30 teams participated.
The event was sponsored by Qualcomm and Microsoft, which provided each team with a Snapdragon X Elite Copilot+ laptop. A key rule of the challenge: solutions had to rely solely on the AI capabilities of the laptops, with no use of external devices for processing.
Behind both winning entries was intense teamwork and problem-solving. Teams submitted project proposals in advance, divided up the workload, and came together to integrate everything on a tight timeline.
The biggest challenge? Making it all work on a single device.
“We only had one, and there was limited time to test,” Shi said. “If I’m testing my part, then my partner won’t be able to test theirs. So there was a lot of time waiting for our turn.”
The same last-minute hustle kept the Vancouver team up late.
“The last several hours we combined what we had developed individually and made sure it works as we designed,” Fang said.
The next step for the Sightmate team? Adding a searchable map feature.
“We didn’t implement this because we would need a third-party app,” Fang said, “but it’s really easy to achieve.”
