Mantis shrimp inspires new breed of light sensors

Inspired by the eyes of mantis shrimp, researchers have developed a new kind of optical sensor that is small enough to fit on a smartphone but is capable of hyperspectral and polarimetric imaging.

“Lots of artificial intelligence (AI) programs can make use of data-rich hyperspectral and polarimetric images, but the equipment necessary for capturing those images is currently somewhat bulky,” says Michael Kudenov, co-corresponding author of a paper on the work and an associate professor of electrical and computer engineering at North Carolina State University. “Our work here makes smaller, more user friendly devices possible. And that would allow us to better bring those AI capabilities to bear in fields from astronomy to biomedicine.”

In the context of this research, hyperspectral imaging refers to technologies that can break down the visible wavelengths of light into more narrow bands. The human eye can’t distinguish between these slight variations in color, but computers can – making hyperspectral imaging valuable for tasks such as determining the chemical composition of objects in the image.

Polarimetry refers to the measurement of polarization in light, which is data that can be used to determine the surface geometry of an object in the image. For example, is the surface rough or smooth? And what is the angle of the surface relative to the light source?

Light is famously tricky to describe, since it is both a particle and a wave. If a wave of light is moving from Point A to Point B, the path between those two points is the direction of the light. If you think of the light as a particle, it is moving in a straight line from Point A to Point B. But the light is also an electromagnetic field that fluctuates like a wave. If you picture that wave as wiggling up and down or side to side as it travels from Point A to Point B, polarization is a measurement of the orientation of that wave along the path.

While there are larger devices that are capable of capturing hyperspectral and polarimetric images, smartphone-sized imaging technologies have run into significant challenges.

For example, the design of cell phone camera technologies results in very slight errors in the alignment of the different wavelengths of light in the final image. The result is not a big deal for taking family photos, but is problematic for scientific image analysis. And the problem is exacerbated when a camera can capture more colors, as is the case with hyperspectral technologies.

The creators of the new light sensors were inspired by the eyes of mantis shrimp, which are exceptionally good at accurately capturing subtle gradations of color. So, the researchers created an organic electronic sensor that mimics the mantis shrimp’s eye. It’s called the Stomatopod Inspired Multispectral and POLarization sensitive (SIMPOL) sensor. And, yes, stomatopod is the fancy name for mantis shrimp.

The researchers developed a prototype SIMPOL sensor that can simultaneously register four spectral channels and three polarization channels. By comparison, the charge-coupled devices used in smartphones have only three spectral imaging sensors, which detect red, green, and blue; and only two polarization channels. In addition, the SIMPOL prototype can measure the four color channels and three polarization channels at one point, whereas CCDs rely on imaging sensors spread across several points.

While only a proof of concept, the researchers used modeling simulations to determine that the design could be used to create detectors capable of sensing up to 15 spatially registered spectral channels.

“SIMPOL’s color channels can discern spectral features 10 times narrower than typical imaging sensors; in other words, it is 10 times more precise,” Kudenov says.

“Our work demonstrates that it is possible to create small, efficient sensors that can simultaneously capture hyperspectral and polarimetric images,” says Brendan O’Connor, co-corresponding author of the paper and an associate professor of mechanical and aerospace engineering at NC State. “I think this opens the door to a new breed of organic electronic sensing technologies.”

###

The paper, “Mantis shrimp-inspired organic photodetector for simultaneous hyperspectral and polarimetric imaging,” appears in the journal Science Advances. First author of the paper is Ali Altaqui, a postdoctoral researcher at NC State. Co-corresponding author of the paper is Brendan O’Connor, an associate professor of mechanical and aerospace engineering at NC State. The paper was co-authored by Pratik Sen, a former Ph.D. student at NC State; Harry Schrickx, a Ph.D. student at NC State; Michael Escuti, a professor of electrical and computer engineering at NC State; the late Robert Kolbas, a former professor of electrical and computer engineering at NC State; Jeromy Rech and Wei You of the University of North Carolina at Chapel Hill; and Jin-Woo Lee and Bumjoon J. Kim of the Korea Advanced Institute of Science and Technology.

The work was done with support from the National Science Foundation under grants 1809753 and 1639429; and from the National Research Foundation of Korea, under grant NRF-2017M3A7B8065584.