Could an artificial neural network connected to the brain of a living animal improve its performance on a task, such as the ability to find food? A strength of biologically based intelligence is that it performs well in novel situations by applying principles learned through experience in other contexts. Artificial intelligence (AI), on the other hand, can rapidly process huge quantities of information and thus detect correlations among disparate kinds of data, making connections that a biological neural network might miss.
To test how an artificial and natural system interact, Chenguang Li, a doctoral student in biophysics, linked neurons in the brains of the soil-dwelling nematode worm Caenorhabditis elegans to a reinforcement learning agent designed to handle inputs of recorded data. Reinforcement learning is a form of machine learning in which an algorithm adapts its behavior in response to what it has learned during repeated attempts to achieve a particular outcome. Without human supervision, the agent learns through trial and error how to optimally perform a task.
Li and her advisers—professor of ophthalmology Gabriel Kreiman, an expert in biological and computer vision as well as AI and algorithm design, and Gund professor of neurosciences and of molecular and cellular biology Sharad Ramanathan (see “How to Make a Mammal,” January-February, page 40)—chose a nematode for the experiment because its simple nervous system contains just 302 neurons. Then they trained the AI to assist the animal on specific motor tasks, culminating in a search for food that involved navigating around barriers. The research could one day lead to improved treatments for diseases that affect motor control, such as Parkinson’s.
To allow the AI to modify the worm’s behavior, they used an optogenetic interface, in which neurons in genetically modified worms could be excited or inhibited using blue or green light. They first experimented with a neuron that prompts the worms to turn around, training their agent with video data of worm behavior. Then they gave the agent control of the light, and feedback in the form of video footage from a camera that monitored the worms’ responses—enabling the AI to analyze the effect of the light on the animals’ behavior and movement.
They then repeated the experiment in worms with a gene that encodes a neuron that stimulates the opposite type of behavior (suppressing turning); and then with a neuron whose effect on behavior had not been previously characterized. In all instances, the AI had no prior information about how each neuron behaved yet learned how to integrate its limited ability to intervene in an animal’s observed behavior, successfully guiding it to a target area with food. Control animals—those without an agent or subjected to random light—rarely reached the food.
To understand how the agent was learning, the researchers analyzed what it was “seeing” in the images coming from the camera. They found that it had learned to integrate visual data about the worm’s head and body angles—and what those might signal about its intentions—with the effects of the light.
In a final set of experiments, they pushed the system even further, placing obstacles in the worms’ way, and slightly offsetting the food’s location from the target area specified by the AI. “We wanted to see,” says Li, “is this just a robot?” Would the worm be guided to the target area by the AI, or would it go to the food? “What happened was that once the animal got close enough,” even if the agent was directing it elsewhere, she says, “the animal would listen to its own nervous system, and decide to go to the food.” This showed that “the worms were integrating all the information that they had, just like a real nervous system that doesn’t just listen to one part of what it is hearing. The AI’s role was “more like GPS” than mind control.
Li says reinforcement learning systems might someday be modified to help people with Parkinson’s disease by optimizing treatments that temporarily suppress tremors and restore motor control. On a recent visit to a local hospital, she observed patients receiving 60 to 90 minutes of deep brain electrical stimulus. Neurologists administering such treatments must use enough current to mitigate symptoms and run numerous tests to keep the stimulus below the threshold that causes side effects. “But so much data is lost” in this process, says Li. “It is no longer used once observed by the neurologist.” A reinforcement learning system could retain and use that highly individualized data to minimize patient discomfort and side effects while maximizing the control of symptoms, she says. “The hope is that, in optimizing brain stimulation using reinforcement learning, it might be possible to reduce other medications, and make life easier for these patients.”
