Pioneering Neuroscience: Harvard and Google DeepMind Team Up with Virtual Rodents and AI

by | Jun 15, 2024

On a crisp Tuesday morning, the scientific community was abuzz with excitement as Harvard University and Google DeepMind unveiled a groundbreaking study published in the prestigious journal Nature. This pioneering research demonstrates how AI deep reinforcement learning can be harnessed to create highly realistic virtual rodents, potentially revolutionizing behavioral neuroscience and impacting fields such as medicine, robotics, and artificial intelligence.

The study, spearheaded by Bence P. Ölveczky, Ph.D., a professor of organismic and evolutionary biology for brain science at Harvard, was conducted with a team of experts including Josh Merel, Jesse D. Marshall, Leonard Hasenclever, Ugne Klibaite, Amanda Gellis, Yuval Tassa, Greg Wayne, Diego Aldarondo, and Matthew Botvinick. “Animals have exquisite control of their bodies, allowing them to perform a diverse range of behaviors,” Ölveczky noted. “How such control is implemented by the brain, however, remains unclear.”

Designing a virtual mammal in silico involves several intricate components. For this study, the AI model incorporated a vision encoder, a proprioceptive encoder, a core module trained by backpropagation, and a policy module consisting of one or more long short-term memory (LSTM) recurrent neural networks. “We used deep reinforcement learning to train the virtual agent to imitate the behavior of freely moving rats, allowing us to compare neural activity recorded in real rats to the network activity of a virtual rodent mimicking their behavior,” the researchers elaborated. Deep reinforcement learning fuses deep neural networks (DNN) with reinforcement learning to enable an agent to learn behaviors based on the outcomes of actions. DNNs consist of multiple layers—input, output, and hidden layers—that process and transmit data. Reinforcement learning, on the other hand, simulates learning by trial and error, refining actions based on feedback from the environment. This method is especially useful for complex, real-world scenarios where decisions made today affect future outcomes. Applications range from robotics and autonomous vehicles to personalized medicine and recommendation engines.

The researchers meticulously developed a virtual rat body using a physics-based engine for model-based control, alongside actual measurements from laboratory rats. The virtual rodents were then assigned a series of tasks, such as jumping, foraging, escaping, and double-touching. “We found that neural activity in the sensorimotor striatum and motor cortex was better predicted by the virtual rodent’s network activity than by any features of the real rat’s movements, consistent with both regions implementing inverse dynamics,” the scientists reported. What makes this development particularly exciting is the ability to comprehensively monitor neural activity, behavior, sensory inputs, as well as the model’s training goals, variance sources, and connectivity. “These results demonstrate how physical simulation of biomechanically realistic virtual animals can help interpret the structure of neural activity across behavior and relate it to theoretical principles of motor control,” concluded the researchers.

This pioneering study holds significant potential for advancing various disciplines. Behavioral neuroscience, or psychobiology, delves into the neural and biological foundations of behavior in both humans and animals, merging elements from physics, biology, chemistry, mathematics, and psychology. This interdisciplinary approach is invaluable for fields such as developmental psychology, cognitive psychology, psychiatry, neuroendocrinology, audiology, biochemistry, drug discovery, biotechnology, healthcare, medicine, assistive technology, pharmaceuticals, speech-language pathology, and veterinary sciences. The ability to create realistic virtual rodents could lead to more accurate models for studying human diseases and developing treatments, enhance AI and robotics, and even provide insights into animal behavior that could improve veterinary practices.

The creation of a virtual rodent using AI deep reinforcement learning marks a monumental step forward in both artificial intelligence and neuroscience. The ability to simulate realistic animal behavior and neural activity opens new avenues for research and practical applications across multiple industries. For instance, in healthcare, such models could lead to breakthroughs in understanding neurological diseases and developing new treatment modalities. In robotics, these insights can be translated into more sophisticated and adaptive machines. Moreover, the interdisciplinary nature of behavioral neuroscience means that advances in this field can ripple outward, influencing areas as diverse as developmental psychology and drug discovery. The ability to monitor and predict neural activity with such precision could lead to more effective therapies and interventions for a range of conditions, from mental health disorders to motor impairments.

As this research progresses, the potential for further advancements is immense. Future studies could focus on refining the virtual rodent model, making it even more accurate and applicable to human biology. Researchers could also explore the use of virtual models for other animals, expanding the scope of behavioral neuroscience and providing new insights into the animal kingdom. In the realm of artificial intelligence, the integration of deep reinforcement learning with other AI techniques could lead to more sophisticated and capable systems. These advancements could revolutionize fields like autonomous vehicles, where understanding and predicting behavior is crucial for safety and efficiency. Furthermore, as the technology becomes more accessible, it could democratize research, allowing smaller institutions and independent researchers to contribute to the field. This could accelerate the pace of discovery and innovation, leading to a cascade of new findings and applications.

The creation of a virtual rodent using AI deep reinforcement learning is a groundbreaking achievement with far-reaching implications. As we continue to explore and refine this technology, the future of behavioral neuroscience, artificial intelligence, and a host of other fields looks incredibly promising.