2024 Theses Doctoral

Parsing the geometry of distributed representations

Alleman, Matteo

The progression of neuroscience relies on the discovery of structure in the brain. From the discovery of neurons to the structure of the potassium channel, and, in recent years, the repeated observation of remarkable geometric structure in the distributed activity of neural populations. What this population-level structure does is not written on it for anyone to read, generally speaking; many statistical and theoretical tools have had to be developed for interpretation.

In these chapters, I benefit from and contribute to the growing set of tools for parsing geometries. First, my collaborators and I studied the representation of syntax in (at the time) state-of-the-art language models. Second, we sought to understand why certain geometries emerge in artificial networks. Third, we model the geometry of working memory representations to try and find why 'swap errors' occur. Finally, we offer a new framework and method for discovering discrete structure in continuous representations.