Rice-led project aims to transfer visual perceptions from the sighted to the blind
Credit: J. Robinson/Rice University
A Rice University-led team of neuroengineers is embarking on an ambitious four-year project to develop headset technology that can directly link the human brain and machines without the need for surgery. As a proof of concept, the team plans to transmit visual images perceived by one individual into the minds of blind patients.
“In four years we hope to demonstrate direct, brain-to-brain communication at the speed of thought and without brain surgery,” said Rice’s Jacob Robinson, the lead investigator on the $18 million project, which was announced today as part of the Defense Advanced Research Projects Agency’s (DARPA) Next-Generation Nonsurgical Neurotechnology (N3) program.
Sharing visual images between two brains may sound like science fiction, but Robinson said a number of recent technological breakthroughs make the idea feasible. Just how feasible is the question DARPA hopes to address with a series of N3 awards to the Rice-led team and five others that have proposed different technological solutions for the broader challenge of connecting brains and machines.
“Speed is key,” said Robinson, an associate professor of electrical and computer engineering and of bioengineering in Rice’s Brown School of Engineering. “We have to decode neural activity in one person’s visual cortex and recreate it in another person’s mind in less than one-twentieth of a second. The technology to do that, without surgery, doesn’t yet exist. That’s what we’ll be creating.”
Because surgery is a nonstarter, all the N3 teams plan to use some combination of light, ultrasound or electromagnetic energy to read and write brain activity. Rice’s “magnetic, optical and acoustic neural access device,” or MOANA, will test techniques that employ all three. The MOANA team includes 15 co-investigators from Rice, Baylor College of Medicine, the Jan and Dan Duncan Neurological Research Institute at Texas Children’s Hospital, Duke University, Columbia University and Yale’s John B. Pierce Laboratory.
Robinson said a big differentiator between N3-funded teams is how they plan to deal with the 50-millisecond latency threshold as well as DARPA’s requirements for spatial resolution. The agency is seeking devices that can read from and write to a minimum of 16 locations in a volume of the brain about the size of a pea.
Robinson said MOANA’s decoding and encoding technologies will each employ viral vector gene delivery, a technology that’s in clinical trials for treating macular degeneration, as well as some cancers and neurological conditions. Genetic payloads, which differ for decoding and encoding, will be delivered with the help of ultrasound to select groups of neurons in the 16 target areas of the brain.
To “read” neural activity, the MOANA team will reprogram neurons to make synthetic proteins called “calcium-dependent indicators” that are designed to absorb light when a neuron is active, or firing.
Rice co-investigator Ashok Veeraraghavan said red and infrared wavelengths of light can penetrate the skull, and MOANA’s device will utilize this. The optical subsystem will consist of light emitters and detectors that are arrayed around the target area on a skull cap.
“Most of this light scatters off the scalp and skull, but a small fraction can make it into the brain, and this tiny fraction of photons contain information that is critical to decoding a visual perception,” said Veeraraghavan, an associate professor of electrical and computer engineering and of computer science. “Our aim is to capture and interpret the information contained in photons that pass through the skull twice, first on their way to the visual cortex and again after they are reflected back to the detector.”
MOANA’s photodetectors will be both ultrafast and ultrasensitive. The former is important for ignoring light that scatters off the skull and instead capturing only those photons that have had enough time to travel all the way to the target area of the brain and back.
“By utilizing ultrasensitive, single-photon counting detectors, the tiny signal from brain tissue can be selectively sensed,” Veeraraghavan said.
Veeraraghavan, Robinson and MOANA collaborators Kenneth Shepard and Andreas Hielscher from Columbia Engineering plan to use the detectors to develop a technology called “time-of-flight enhanced functional diffuse optical tomography,” or ToFF-DOT. Like a CT scanner, ToFF-DOT constructs a real-time 3D image of what’s inside the body, but whereas a CT scan uses X-rays, ToFF-DOT uses visible light.
Robinson said neurons in the 16 target regions of the visual cortex are expected to show up darker than normal on ToFF-DOT scans when they are firing and their calcium-dependent indicator proteins are absorbing light. Interpreting the dynamic changes from dark to light in the target areas is what MOANA will do to “read” neural activity.
Robinson said three years of work, first in cell cultures and then animals, will precede any work with human patients. But he said the MOANA team will coordinate its efforts with Baylor Department of Neurosurgery’s Daniel Yoshor and Michael Beauchamp, who are conducting clinical trials to restore sight to blind patients using an experimental prosthetic that directly stimulates the visual cortex with surgically implanted electrodes.
“There may be patients who prefer a visual prosthetic that doesn’t require brain surgery,” Robinson said. “If our work in cells and animal models goes well, MOANA could be approved for clinical tests as a nonsurgical alternative. It would require gene therapy, but not brain surgery.”
In the brain receiving an image, MOANA would “write” information to neurons that are reprogrammed to fire in response to magnetic signals. The gene therapy payload delivered to these neurons will create proteins that tether either naturally occurring or synthetic iron nanoparticles to ion channels inside the neurons. The release of calcium through these ion channels is what “fires” a neuron, causing it to actively transmit an electrical impulse.
“We plan to use magnetic fields to heat the iron, which in turn will open the channel and fire the neuron,” Robinson said. “But it’s not enough to do that every second or two. Our system must respond in milliseconds for the receiver and perceiver to experience the perception close enough in time that it seems simultaneous.”
Human thought involves the coordinated firing of many neurons, sometimes in different regions of the brain. Rice co-investigator Caleb Kemere said the quality of communication that can be achieved with 16 channels of information is an open question.
“We know that the circuits of the brain that are involved are very dense,” said Kemere, an associate professor of electrical and computer engineering and of bioengineering who has previously studied neural circuits using invasive technologies. “It’s possible or even likely that early 16-channel demonstrations may deliver somewhat muddied perceptions, but this is an exciting path towards a more noninvasive future. The timing, density and performance of the systems we are developing will be orders of magnitude more sophisticated than anything currently available.”
###
Other MOANA co-investigators include Rice Bioengineering’s Gang Bao and Omid Veiseh, Columbia Engineering’s Elisa Konofagou and Columbia Neuroscience’s Vincent Ferrera, Yale Pierce Laboratory’s Vincent Pieribone, Duke Psychiatry’s Angel Peterchev and Stefan Goetz, Baylor Neuroscience’s Andreas Tolias, Matt McGinley and Benjamin Arenkiel. McGinley and Arenkiel are also Duncan Institute investigators.
Robinson, Kemere, Veeraraghavan, Veiseh, Beauchamp and Tolias are also faculty investigators in Rice’s Neuroengineering Initiative, a strategic initiative launched in 2018 to improve lives by advancing the world’s understanding of the brain.
IMAGES are available for download at:
https:/
CAPTION: Rice University neuroengineers are leading an ambitious DARPA-funded project to develop MOANA, a nonsurgical device capable of both decoding neural activity in one person’s visual cortex and recreating it in another’s in less than one twentieth of a second. (Image courtesy of J. Robinson/Rice University)
Animated version:
https:/
Still version:
https:/
CAPTION: The Defense Advanced Research Projects Agency has funded an ambitious four-year project to develop headset technology that can directly link two human brains without the need for surgery. Led by neuroengineers at Rice University, the team is creating MOANA, a nonsurgical device capable of both decoding neural activity in one person’s visual cortex and recreating it in another’s in less than one-twentieth of a second. The above image and animation depict the planned components of the MOANA headset. (Image and animation courtesy of J. Robinson/Rice University)
https:/
CAPTION: Rice University MOANA engineering researchers (from left) Ashok Veeraraghavan, Jacob Robinson and Caleb Kemere. (Photo by M. Williams/Rice University)
https:/
CAPTION: Rice’s “magnetic, optical and acoustic neural access” (MOANA) device will test techniques that use light (bottom panel) to read brain activity and electromagnetic energy to write that activity into another person’s brain in less than 50 milliseconds. (Image courtesy of J. Robinson/Rice University)
https:/
CAPTION: Rice University’s MOANA team includes (from left) Charles Sebesta, Josh Chen, Jacob Robinson, Amanda Wickens and Gillaume Duret. (Photo by Jeff Fitlow/Rice University)
https:/
CAPTION: Rice University’s MOANA team includes (from left) Yongyi Zhao, Ankit Raghuram and Akshat Dave. (Photo by Jeff Fitlow/Rice University)
Links and resources:
George R. Brown School of Engineering: engineering.rice.edu
Rice Neuroengineering Initiative: neuroengineering.rice.edu
Robinson Lab: robinsonlab.com
Rice Department of Bioengineering: bioengineering.rice.edu
Rice Department of Electrical and Computer Engineering: ece.rice.edu
Related research stories from Rice:
Wearable hospital lab: NSF awards $10M for bioimaging — Feb. 27, 2018
http://news.
Veeraraghavan wins NSF CAREER Award — March 20, 2017
https:/
Rice team developing flat microscope for the brain — July 12, 2017
http://news.
No lens? No problem for FlatCam — Nov. 23, 2015
http://news.
This release can be found online at news.rice.edu.
Follow Rice News and Media Relations via Twitter @RiceUNews.
Located on a 300-acre forested campus in Houston, Rice University is consistently ranked among the nation’s top 20 universities by U.S. News & World Report. Rice has highly respected schools of Architecture, Business, Continuing Studies, Engineering, Humanities, Music, Natural Sciences and Social Sciences and is home to the Baker Institute for Public Policy. With 3,962 undergraduates and 3,027 graduate students, Rice’s undergraduate student-to-faculty ratio is just under 6-to-1. Its residential college system builds close-knit communities and lifelong friendships, just one reason why Rice is ranked No. 1 for lots of race/class interaction and No. 2 for quality of life by the Princeton Review. Rice is also rated as a best value among private universities by Kiplinger’s Personal Finance.
DARPA Distribution Statement “A” (Approved for Public Release, Distribution Unlimited)
Media Contact
David Ruth
[email protected]