163 — Semantic attributes are encoded in human electrocorticographic signals during visual object recognition

Here’s some really awesome research that came out of JHU/APL just a few desks down from where I work! (Actually some of it came from my own desk because I eventually stole Mark Chevillet’s desk.)

Electrocorticography (ECoG) is very similar to conventional noninvasive EEG — but with the important distinction that electrodes are placed (yes, invasively) inside the skull, embedded in the neuropil. ECoG is understandably less common than EEG monitoring, but it provides much higher resolution and lower-noise datasets than EEG. And so only in cases where patients have the clinical requirement of electrode-array implantation (most often epilepsy patients) is ECoG a viable method for human study.

In this study, nine patients (admittedly a small $n$, but that’s about as good as it gets!) of similar age volunteered to have their BlackRock neuroport electrodes monitored while they performed a task.

Line drawings from this study were shown to the subjects and they were asked to name the object as quickly as possible. The electrode arrays were recorded at 1000Hz.

These data were then used as inputs to a classifier model which tried to reverse-engineer which of the classes of objects (manmade versus animate, large versus small, etc) the signal was in response to — effectively, reading the participant’s mind.

Either as a result of the location of the array or some other unknown variable, some patients’ minds were very unreadable, with only slightly better-than-chance results. Other patients’ minds, however, were very readable, with up to high-80s or even low-90s Mean Rank Accuracy (MRA).

The paper makes some general categorizations on the primary “semantic dimensions” of the topics: Namely, for each word, the four axes “manmade”, “large”, “manipulable”, and “edible” were often enough to encode both brain position and semantic meaning. Large, manmade objects (such as “airplane”) trended toward medial cortex electrodes whereas small, natural objects elicit responses in lateral electrodes.

A controversial paragraph: This research is particularly interesting because semantic decoding is only really feasible in human brains. Sure, animals have mental representations of objects — but they do not have a language that is both embedded in a known brain area (humans have Wernicke’s and Broca’s areas) as well as decodable to a human researcher (even if dolphin calls were well-localized and understood, we still don’t know what they’re saying).