The BYU MATRIX Lab is working on a groundbreaking project: using AI to assist in American Sign Language translation.
“Matrix” stands for machine translation research and interlingual experimentation.
“I hope that my research is able to improve the quality of machine translation,” Kolton Baldwin, a Matrix lab member, said.
The lab focuses on enabling global communication, particularly for low-resource languages.
“Ideally they'll be able to not just consume media, but also, in the future through automated signing, and we want them to be able to communicate in any setting as well,” Eric Ringger, matrix lab advisor, said.
Nathan Browne, a PhD student, explained that every sign in American Sign Language has five phonological features.
“You have handshape and then palm orientation, the location of each of your hands, the way they move, whether it's bouncing, or in a circular motion. And then you also have facial expressions, which is really important,” Browne said.
Browne is working on specialist AI models that can analyze each feature individually.
“All of those feed to one model that takes their outputs as input and does the translation that way,” Browne said.
As the work continues, the lab is looking for more researchers to help.
“Any undergraduate student who wants to do research in machine translation, who cares about the languages of the world and would like to come work with us, we invite you to come talk to us,” Ringger said.
Visit matrix.byu.edu to learn more or get involved with the lab.