Advertisement

New Student-Developed Technology Could Be A Game-Changer For People With Disabilities

• Thomas Pryor and Navid Azodi of the University of Washington, $10,000 Lemelson-MIT “Use it!” Undergraduate Winners for their invention SignAloud, gloves that translate sign language into text and speech. CREDIT: PHOTO COURTESY OF CONRADO TAPADO | UW COMOTION
• Thomas Pryor and Navid Azodi of the University of Washington, $10,000 Lemelson-MIT “Use it!” Undergraduate Winners for their invention SignAloud, gloves that translate sign language into text and speech. CREDIT: PHOTO COURTESY OF CONRADO TAPADO | UW COMOTION

Wearable tech now goes way beyond your FitBit.

Researchers have floated possibilities in this space that could end up transforming health care, like using Google Glass to give doctors more treatment information in real time. And health care wearables promise to be a billion-dollar industry. Just this month, IBM announced the latest in a string of health and tech partnerships, a project with pharmaceutical giant Pfizer to us wearable sensors to help study and treat patients with Parkinson’s disease.

But last week, two new wearable technologies gained a spotlight when their creators were awarded the Lemelson-MIT student prize, a yearly cash award to a handful of collegiate inventors. The twist? These promising technologies weren’t developed by major corporations or experienced PhDs — some were developed by people who can’t even legally drink alcohol.

ThinkProgress spoke with a few of the winners about exactly how their inventions work, and how they hope they will transform the ability to communicate with two different populations: children with autism and deaf users of American Sign Language.

A.I. that provides helps kids with autism interpret emotions

Nonverbal cues like eye contact, gestures, and recognizing facial expressions and emotions are crucial to helping us connect, but are often a struggle for children with autism. Researchers speculate that can cause a vicious cycle and send them further down a path of isolation.

Advertisement

One of the most successful ways to fix this is intensive training, in which a therapist and child use flashcards to painstakingly build an association between facial cues and emotions. The real world, however, is far more complicated than a clinician’s office, and people’s faces vary far beyond what could reasonably be shown on flashcards. Plus, there simply aren’t enough therapists to treat every child with autism — which, in the U.S., is now about one in 68 kids.

That’s where 20-year-old Catalin Voss’ invention, Autism Glass, comes in. Voss, who is part of a team at Stanford and has been working as a software developer since age 15, first developed a face-recognition A.I. with a partner as a freshman. Building on that software, he loaded it onto Google Glass — “because, why not?” — and realized it could be a powerful therapeutic tool.

Catalin Voss, Lemelson-MIT “Cure it!” Graduate Winner for his invention the Autism Glass Project. CREDIT: Photo courtesy of Stanford University
Catalin Voss, Lemelson-MIT “Cure it!” Graduate Winner for his invention the Autism Glass Project. CREDIT: Photo courtesy of Stanford University

Voss has a cousin with autism, so he was already familiar with flashcard therapy’s successes and limitations. Using a similar concept of associated learning, his A.I. automatically recognizes people’s emotions, then gives kids with autism a heads up display identifying the emotion — in real time, in their real world.

Voila, adaptive flashcards that can transcend the doctors office.

“That brings it into your home and your life and hopefully unlocks these children’s ability to integrate it into their actual situation,” Voss told ThinkProgress over the phone.

Advertisement

The software is light enough to run on a smartphone, which is completely locked and shielded to protect privacy. In their current clinical study, children are only allowed to use the Glass in their homes, and the study app — also on the smartphone — contains an “e-consent” form that any new person being filmed must sign before researchers can review.

Although the current iteration of the software just identifies the emotions of surrounding people, Voss imagines future iterations could be even more adaptable to the surrounding situation. For example, he said, if a child gets focused on a rant about a subject and doesn’t realize that they’ve lost the interest of their audience, the software could recognize this and give a behavioral prompt.

“What the system should do is recognize that I’m monotonically speaking, while you’re starting to make less and less eye contact, and the appropriate social cue might be: ‘Pause, ask a question.’ That’s where we’re trying to go. What we’re doing now is basic emotion recognition.”

And, although the software currently uses donated Google Glass — which Voss says the kids in their study loved, because they were their “cool super hero super power glasses” — all it really requires is a outward facing camera, a prompting mechanism such as a video display or an earpiece, and a smartphone. If proven successful in trials, the therapy can make necessary behavioral therapy available to far more children.

Gloves that translate ASL into spoken words

For some, however, verbal language is the challenge. Millions of deaf Americans use American Sign Language (ASL) to communicate — but although ASL is a complex language in its own right, it can present a language barrier for hearing people, many of whom have never learned it.

Advertisement

Another winning team, sophomores Thomas Pryor and Navid Azodi, want to change that with SignAloud, a pair of lightweight gloves that instantly translate ASL into spoken words.

The gloves are loaded with sensors that calibrate to people’s hands as they put them on, then send data on hand position and movement over bluetooth to a computer, where they are matched with ASL words, translated, and then spoken aloud through a speaker.

For future iterations, the pair told ThinkProgress they are hoping to use their prize money to invest back in the gloves — they made two pairs of the winning gloves on a $100 dollar budget. And, as they continue to develop the product, they hope to work more closely with the deaf and American Sign Language community, and to work towards being able to best capture the nuance of ASL.

“Our technical challenge is to stay true to the integrity and syntax of American Sign Language — if we’re going to do it we’re going to do it right,” Azodi said, adding that their ultimate goal is to create a device that will have a positive social impact.

The next generation of game-changing inventions

This year there were seven Lemelson-MIT student prize winners, most of which have buzzy, investment-ready names like SignAloud and Sension (Voss’ first software iteration).

Heather Hava of the University of Colorado Boulder, and SmartPot (SPOT), a smart growth chamber that can work semi-autonomously to grow food in space and other harsh climates CREDIT: Photo courtesy of Alex Pilnick | CU Media
Heather Hava of the University of Colorado Boulder, and SmartPot (SPOT), a smart growth chamber that can work semi-autonomously to grow food in space and other harsh climates CREDIT: Photo courtesy of Alex Pilnick | CU Media

“Highlight,” developed by a team of Columbia undergraduates, could save the lives of health care workers around the world by showing gaps in disinfectant coverage. An MIT graduate student invented a new car transmission that will reduce emissions. And, if that doesn’t work and Earth’s population is forced to pull a Wall-E and abandon ship, another graduate student invented a growth chamber — complete with robots — that will help astronauts grow fruits and vegetables in space, which she calls “SmartPot.”

Each invention is incredible, and has the potential to transform some aspect of the world. And, many of the winners started exploring innovation — and science and technology — far before college.

Voss, who is from Heidelberg, said that while he was always interested in “all things tech” from a young age, he was fortunate to be at a high school with a robotics and computer science club. Pryor and Azodi initially bonded through a “knack for tinkering with things.” Indeed, a general bent towards experimentation — as well as involvement in high school robotics and similar clubs — is a common theme across many of the winners.

The next world’s next generation will face daunting challenges — from a changing climate to expanding the bounds of medicine and therapy. Supporting and encouraging exploration is the best way to make sure we’re ready to meet those challenges — and if anything, this years Lemelson-MIT winners provide hope that we’re on our way.