With NSF grant, SLU engineer brings STEM to the visually impaired

ST. LOUIS – As educational materials increasingly become digitized, the inability to read the graphs and simulations that illustrate math, engineering and science concepts can be a barrier to STEM fields for the visually impaired.

With the help of a million dollar National Science Foundation (NSF) CAREER award, Jenna Gorlewicz, Ph.D., assistant professor of mechanical engineering at Saint Louis University, aims to bridge this divide through robotics and touchscreen technology.

“STEM consists of highly visual content,” Gorlewicz said. “We are concerned that as educational content moves to the digital space, those with disabilities like visual impairments are not being given the equal access they deserve and need. This exacerbates the fact that students with disabilities are not well represented in STEM fields and professions, and therefore, limits the inclusion and diversity in STEM at large, which is a huge disservice to our field.

“Our work is focused on raising the bar for digital accessibility, specifically for visual impairments, but in ways that may be of benefit to others, as well.”

The NSF award will allow Gorlewicz and her team to create graphics and simulations that can be felt and heard using currently available, low-cost systems such as touchscreens.

“The hard copy raised dots are wonderful but we can’t replicate them in the electronic space in a refreshable, dynamic way,” Gorlewicz said. “Because we have voiceover technology, text isn’t as big of a problem. But, there is no analog for graphics. Right now, there are only alt tags, which offer brief descriptions to images online and in some cases, auditory feedback. This isn’t enough information to understand a complex chart, and we can do better.”

The researchers’ focus is not to recreate content, but to make existing content accessible using sound, vibrations and force feedback. The team will develop new software and hardware platforms and will work with students and their teachers to incorporate feedback throughout the process. They’ll also leverage existing educational content, such as PhET Interactive Simulations, a library of free interactive math and science simulations developed at the University of Colorado Boulder, to ensure that they are working with content that has broad reach and meaning.

“In previous studies, we explored how to represent the fundamental building blocks of a graph, like lines and points, using a small number of vibrations and sounds” Gorlewicz said.

“In this grant, we’re building off of our earlier work, marking up more detailed images with sonification and vibration. We are answering questions like: What do you do with a complex diagram of an animal cell or a scatter plot with a ton of data? How do we separate out the information so that it is “readable” through multiple senses?”

The grant focuses on K-12 education, and Gorlewicz says that middle school is where complex graphs really start to come into play.

The team will conduct empirical studies with students and their teachers, focusing on graph literacy and incorporating users’ feedback into the technology design.

“Users have contributed many of the ideas we’ve adopted, like that the sound for a bar graph could involve the volume going up and down as your finger moves along the bar. Students and teachers had ideas for how we could incorporate this technology in periodic tables in chemistry, in basketball plays in P.E. and for getting from point a to point b, like a bus route.”

In addition to helping those with visual disabilities, the project will extend the research to students with other needs as well, including those with cognitive disabilities.

For Gorlewicz, this work is a perfect combination of her enthusiasm for engineering and robotics and her drive to cultivate a sense of higher purpose by working with people.

“It was around my second year of Ph.D. work in medical robotics and I was finding that I missed the people connection,” Gorlewicz said. “I was working in a school system in the Nashville area with two students who were blind, and their teacher. One of the students said, ‘It’s so cool you’re an engineer. I could never be an engineer, though. Math is full of graphs and I can’t see them.’

“Here was a motivated student, very capable, but she felt she couldn’t participate because of access.”

This was a turning point for Gorlewicz, who saw a pathway where she could use her engineering skills to open up access to the sciences for those who felt locked out of the field.

In addition to developing new technology, the grant will support the training of students at many levels.

“It’s such an honor to receive this grant,” Gorlewicz said. “It’s more than an award. It’s going to fund several Ph.D.’s and SLU undergrads, as well as college-age students with visual impairments who will help conduct the research. We’re going to be training students at all levels, in universal design of technologies from the onset, which is so important.”

###

The SLU team is partnering on the project with the Special School District of St. Louis County, Missouri School for the Blind, Francis Howell School District, Hollinger Consulting and Lighthouse for the Blind. The grant number is NSF 1845490.

Founded in 1818, Saint Louis University is one of the nation’s oldest and most prestigious Catholic institutions. Rooted in Jesuit values and its pioneering history as the first university west of the Mississippi River, SLU offers nearly 13,000 students a rigorous, transformative education of the whole person. At the core of the University’s diverse community of scholars is SLU’s service-focused mission, which challenges and prepares students to make the world a better, more just place. For more information, visit slu.edu.

The grantee is responsible for assuring that every publication of material (including web pages) based on or developed under an NSF award, except scientific articles or papers appearing in scientific, technical or professional journals, contains a disclaimer that specifies that the opinions, findings, and conclusions or recommendations expressed are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.