How Neuroscientists Are Learning To Decode Your Brain

On June 12th, 2014, millions of soccer fans from around the world watched the opening ceremonies of the World Cup in Sao Paulo, Brazil. A time-honored tradition of the ceremony is the first kick, which officially starts the intense period of competition to follow.

Immediately, viewers could see that there was something that made this season’s kickoff unlike any other: Juliano Pinto, a paraplegic man, tapped the ball with a robotic foot attached to a full body exoskeleton whose movements were controlled entirely by his brain. The construction of the suit was the result of a collaborative, international effort by the Walk Again project involving^(a) over one hundred researchers under the leadership of neuroscientist Miguel Nicolelis.¹

Since the dawn of the industrial age, humankind has imagined a world in which we could leverage our most powerful biological tool, the brain, to control the technologies we create. Pinto’s kick serves as a powerful demonstration of how far our understanding of the brain – and what we can do with that knowledge – has advanced in recent years. At the frontier of these efforts is computational neuroscience, a field that is unlocking the secrets of how our brain works and developing exciting new applications for this knowledge.

Order within Chaos

We now have several ways to record what’s happening inside the brain. Scientists can drop electrodes directly into neurons in animals to conduct single-cell spike recording or measure voltage changes on the scalps of humans with electro-encephalograms (EEGs). They use gigantic magnets to detect fine changes in the blood flow feeding regions of the brain, in a procedure known as functional Magnetic Resonance Imaging (fMRI). There is even a technique, called electrocorticography (ECoG), that involves placing electrodes directly on the surface of a human brain during surgery.

While the technology behind these methods is diverse, what unites them is that they all produce information that is quite noisy and seemingly chaotic. For example, take a glance at an EEG trace and you’ll see a squiggly line that bounces around the screen like a flag blowing in the wind. However, while flags may seem to move aimlessly in the breeze, there is a reason behind each fluctuation that can be understood through the study of aerodynamics.

Likewise, while brain activity may seem random, this chaos is merely an illusion. Embedded in a sea of noise are the subtle and intricate signals that control human movements, thoughts, and emotions. Armed with modern-day computers, sophisticated statistical algorithms, and machine learning techniques, scientists can now sift through the signals coming from the brain and begin to decipher their meaning.

Modeling the Brain

For decades, scientists have been looking for ways to understand the electrical activity in the brain. In 1952, Alan Lloyd Hodgkin and Andrew Huxley attempted to work out the mathematics behind the action potential, an event in which a neuron fires an electrical pulse to communicate with other neurons.^(b) This dialogue among neurons is what makes muscle movement and other biological functions possible. The result of their efforts was a mathematical model of the neuron, which won them the Nobel Prize in 1963.²

In the years since, neuroscientists, physicists, mathematicians, and computer scientists have built upon these methods to ask new questions of the human brain and empower it to do things never before thought possible. Researchers now have a range of software packages, innovative techniques, and data tools at their disposal to parse neural signals and create computational models. All of these tools rely on computers to set the language of mathematics into motion, allowing scientists to answer a wide range of complex questions that, in the past, would have taken years to model.^(c)

We now have computational models that explain how cats process the visual landscape, how birds understand songs, even how rats use their whiskers to explore the world around them. In 2013, Nima Mesgarani and Eddie Chang, neurosurgeons studying language at the University of California, San Francisco, created a neural model that predicted which of two voices a person was paying attention to at any one time.³ Other researchers have developed models to explain how humans interpret images,⁴ understand stories,⁵ parse sounds, ⁶ and feel emotions. These approaches offer a direct window into human behavior and create new possibilities for linking the brain with the outside world.

Encoding & Decoding Thought

Most neuroscience models take sensory experiences of the world as input and map the brain’s response to this information as output. For example, sound waves are fed into a model that predicts the resulting activity in auditory areas of the brain. These are called encoding models because they attempt to capture how the brain “encodes” information from the outside world.

While encoding models have proven quite useful, they ultimately raise a more provocative question: Could we create a model that takes as its input a pattern of brain activity and outputs information about what that brain was experiencing at the time? For example, could we look at a pattern of activity in the auditory region of the brain and decipher what song someone was listening to at the time that activity was recorded?

Scientists are beginning to discover that the answer is yes, and that these decoding models have a wide array of applications ranging from “brain reading” to neuro-prosthetic devices like the exoskeleton Juliano Pinto wore at the World Cup ceremony.⁷

One example of this futuristic-sounding research comes from the lab of neuroscientist Jack Gallant at UC Berkley. Gallant and his team recorded the fMRI activity of people who were watching movie clips.⁸ As the subjects watched a variety of scenes, blood flowed to regions of the brain that correspond with the different ideas, objects, and emotions they were seeing on the screen. The researchers divided the fMRI scans into voxels, which are tiny 3D cubes that reflect the amount of blood flowing to a given brain area. They created an encoding model that predicts the voxel activity resulting from a particular scene, then “reversed” the model, making it possible to decode what scene is being watched from a particular pattern of brain activity.

Decoding methods are far from perfect and the features they decode are often quite noisy and unclear, but they are proof that in many cases the information is there; we only need to find a method that is clever enough to extract it.

Decoding techniques can do more than just reveal what we’re thinking – they have a wide array of potential applications in daily life. One exciting possibility is the creation of brain-computer interfaces (BCIs), a broad class of devices and algorithms that attempt to facilitate interactions between brain activity and the outside world. The recently created Center for Neural Engineering and Prostheses (CNEP) at UC Berkeley & UC San Francisco aims to design BCI technology that allows people to control robotic arms and legs through thought. Researchers at the center are also developing technology that detects and corrects maladaptive patterns of brain activity associated with mental health issues like depression, anxiety, and addiction.^(d)

While decoding models have many potentially valuable applications, they could one day be used for less noble purposes. It will be some time before technology will allow anyone to decipher a person’s specific thoughts or read their mind without permission. If and when such a reality arrives, however, it will likely upturn many long-held beliefs about privacy, free speech, and ethics.

A Brave New World

We have come a long way since Hodgkin and Huxley huddled around a tiny neuron trying to predict its spiking activity. The field of computational neuroscience has moved from individual neurons, to networks of neurons, to entire neural systems, and then to brain-wide activity in just a matter of years. As more sophisticated approaches to brain encoding and decoding become a reality, they will continue to unlock new doors in our understanding of the mind and may ultimately change the way we think about our connection with the world around us.

In the coming years, computational neuroscience will harness the complexity of the brain in ways that were once the stuff of science fiction. Innovations will also introduce ethical and moral challenges that the human race has never before faced. Regardless, one thing is certain: our brains will be hacked, recorded, understood, and empowered like never before.