Todd Decker

This is a follow-up to some things discussed in our last group episode on artificial intelligence. Since that conversation I’ve been digging more into the subject and wanted to share some ideas about it. I’ve been interested in artificial intelligence for a number of years. Part of that interest is because of its practical usefulness, which we’re really seeing explode now, with ChatGPT in particular. But I’m also interested in artificial intelligence as a model that could give us insights about human intelligence.

I have to say that the performance of these most recent models, like ChatGPT-3 and especially ChatGPT-4, is something that has really surprised me. If someone had told me a couple years ago that in 2022 & 2023 a deep learning model would be able to perform as well as these do I wouldn’t have believed it. I’m typically predisposed to doubt or at least be very critical about the capabilities of artificial intelligence. But in this case I think I was wrong and I’m happy to have been wrong about that. I don’t mean to swing too far to the other extreme and get too exuberant about it and overstate the capabilities of these models. But just a little bit of excess excitement might be excusable for the moment.

One claim that would be too extreme would be that these deep learning models are actually self-conscious already. Now I have no philosophical reason to suppose that an artificial device could not be self-conscious. I just don’t think we’re there yet. Another, less extreme claim, but one that would still go too far would be that deep learning models actually replicate the way human speech is produced in the brain. I think the implementations are still distinct. But that being said, I think there are enough similarities to be useful and interesting.

For comparison, there are insights we can gain into sight and hearing from cameras and audio recorders. Obviously they are not the same as our own sense organs but there are some similar principles that can help us think about how our senses work. The comparisons work both at the level of physical mechanisms and at the level of data processing. For example, I think there are some interesting insights about human senses from perceptual coding. Perceptual coding is a method used in digital signal processing that leverages the limitations and characteristics of the human sensory systems (auditory and visual) to provide data compression. For example, in audio, certain sounds are inaudible if they’re masked by louder sounds at a similar frequency. Similarly, in an image, subtle color differences in areas with high spatial detail are less noticeable than in smooth areas. Perceptual coding takes advantage of this by selectively removing the less noticeable information to reduce the data size, without significantly impacting perceived quality. This is done with MP3s and JPEGs. Extending this comparison to large language models, I’d propose that models like ChatGPT might be to human language production what cameras, JPEGs, audio recorders, and MP3s are to sight and sound. They aren’t the same but there are some parallels. ChatGPT is not the same as a human brain any more than a camera is an eye or an audio recorder is an ear. But, more modestly, ChatGPT may have some interesting similarities to human language production.

The most significant developments in this technology are so recent that the most useful reading material I’ve had to go to on the subject is peer-reviewed literature from the past year. Even there a lot of the research was done with GPT-2, which was a much less advanced model than we have available today. So it will be interesting to see what studies come out in the next year and beyond. The papers I want to focus on are 3 research papers from 2022 that present the results of experiments and 2 (just slightly older) perspective papers that offer some broad reflections and theoretical considerations.

In what follows, I’ll proceed in three parts: (1) philosophical background, (2) an overview of neural networks: biological and artificial, and (3) recent scientific literature.

Philosophical Background

Most of the philosophy I’d like to discuss is from the 20th century, in which there was considerable philosophical interest in language in what has been called the “linguistic turn”. But first something from the 18th century.

Something that stood out to me in all the research articles was the issue of interpretability. Artificial neural networks have been shown to have remarkable parallels to brain patterns in human brain production. That’s nice because the brain is so complex and little understood. The only problem is that ANNs themselves are also extremely complex and opaque to human comprehension. This challenges a notion going back to the 18th-century Italian philosopher Giambattista Vico: the Verum factum principle.

The phrase “verum factum” means “the true is the made,” which refers to the notion that truth is verified through creation or invention. In other words, we can only know with certainty that which we have created ourselves, because we understand its origins, structure, and purpose. Vico developed this principle as a critique of the Cartesian method of knowing, which, in Vico’s view, emphasized the abstract and ignored the concrete, humanistic dimensions of knowledge. By asserting that true knowledge comes from what humans create, Vico highlighted the role of human agency, creativity, and historical development in the creation of knowledge.

However, applying the verum factum principle to complex human creations like modern industrial economies, social organizations, big data, and artificial neural networks poses some interesting challenges. These creations certainly reflect human ingenuity and creativity, but they also possess a complexity that can make them difficult to fully comprehend, even for those directly involved in their creation. Artificial neural networks are inspired by our understanding of the human brain, but their function, especially in deep learning models, can be incredibly complex. It’s often said that these networks function as “black boxes,” as the pathway to a certain output given a certain input can be labyrinthine and largely inexplicable to humans, including their creators. So, while the verum factum principle encapsulates the role of human agency and creativity in the construction of knowledge, artificial neural networks illustrate that our creations can reach a level of complexity that challenges our ability to fully comprehend them.

Now turning to the 20th century I think four philosophers are especially relevant to the subject. These are Martin Heidegger, Ludwig Wittgenstein, Ferdinand de Saussure, and Hubert Dreyfus. Of these four Hubert Dreyfus was the one who most directly commented on artificial intelligence. But Dreyfus was also using ideas from Heidegger in his analysis of AI.

Let’s start with Dreyfus and Heidegger. Dreyfus’s main arguments were outlined in his influential 1972 book, What Computers Can’t Do. The core of his critique lies in what he sees as AI’s misguided reliance on formal symbolic reasoning and the assumption that all knowledge can be explicitly encoded. Dreyfus argued that human intelligence and understanding aren’t primarily about manipulating symbolic representations, as early AI research assumed. Instead, he believed that much of human knowledge is tacit, implicit, and tied to our embodied experience of “being in the world”, an important Heideggerian concept. These are aspects that computers, at least during that time, couldn’t easily replicate.

Dreyfus drew heavily on the philosophy of Martin Heidegger to make his arguments. Heidegger’s existential phenomenology, as expressed in his 1927 book Being and Time describes human existence (“Dasein”) as being-in-the-world—a complex, pre-reflective involvement with our surroundings. This contrasts with the traditional view of humans as subjects who perceive and act upon separate objects. According to Heidegger, we don’t usually encounter things in the world by intellectually representing them to ourselves; instead, we deal with them more directly.

Dreyfus related this to AI by arguing that human expertise often works in a similar way. When we become skilled at something, we don’t typically follow explicit rules or representations—we just act. This aligns with Heidegger’s notion of ‘ready-to-hand’—the way we normally deal with tools or equipment, not by observing them as separate objects (‘present-at-hand’), but by using them directly and transparently in our activities.

Another philosopher relevant to this topic is Ludwig Wittgenstein. He was one of the most influential philosophers of the 20th century. He is considered to have had two major phases that were quite different from each other.  His early work, primarily represented in Tractatus Logico-Philosophicus, proposed that language is a logical structure that represents the structure of reality. But in his later work, chiefly Philosophical Investigations, Wittgenstein advanced a very different view.

In Philosophical Investigations, Wittgenstein introduces the concept of language as a form of social activity, what he called “language games.” He argues that language does not have a single, universal function (as he had previously believed) but is instead used in many different ways for many different purposes.

Language, Wittgenstein claims, should be seen as a myriad of language games embedded in what he called ‘forms of life’, which are shared human practices or cultural activities. Different language games have different rules, and they can vary widely from commands, to questions, to descriptions, to expressions of feelings, and more. These language games are not separate from our life but constitute our life.

Wittgenstein also introduced the idea of ‘family resemblances’ to discuss the way words and concepts gain their meanings not from having one thing in common, but from a series of overlapping similarities, just like members of a family might resemble each other.

He also challenged the idea that every word needs to have a corresponding object in the world. He argued that trying to find a definitive reference for each word leads to philosophical confusions and that words acquire meaning through their use in specific language games, not through a one-to-one correspondence with objects in the world. So, for Wittgenstein, the meaning of a word is not something that is attached to it, like an object to a label. Instead, the meaning of a word is its use within the language game. This was a notion similar to a theory of language called structuralism. 

The leading figure of structuralism was the Swiss linguist Ferdinand de Saussure. His ideas laid the groundwork for much of the development in linguistics in the 20th century and provided the basis for structuralism. In his course in general linguistics, compiled from notes taken by his students, Saussure proposed a radical shift in the understanding of language. He proposed that language should be studied synchronically (as a whole system at a particular point in time) rather than diachronically (as a historical or evolutionary development). According to Saussure, language is a system of signs, each sign being a combination of a concept (the ‘signified’) and a sound-image (the ‘signifier’). Importantly, he emphasized that the relationship between the signifier and the signified is arbitrary – there is no inherent or natural reason why a particular sound-image should relate to a particular concept.

Regarding the creation of meaning, Saussure proposed that signs do not derive their meaning from a connection to a real object or idea in the world. Instead, the meaning of a sign comes from its place within the overall system of language and its differences from other signs; It’s within the structure of the language. That is, signs are defined not positively, by their content, but negatively, by their relations with other signs. For example, the word “cat” doesn’t mean what it does because of some inherent ‘cat-ness’ of the sound. Instead, it gains meaning because it’s different from “bat,” “cap,” “car,” etc. Moreover, it signifies a particular type of animal, different from a “dog” or a “rat”. Thus, a sign’s meaning is not about a direct link to a thing in the world but is about differential relations within the language system.

Saussure’s ideas about language and the generation of meaning can be interestingly compared to the techniques used in modern natural language processing (NLP) models, such as word2vec and to cosine similarity. For example, in word2vec, an algorithm developed by researchers at Google, words are understood in relation to other words.

Word2vec is a neural network model that learns to represent words as high-dimensional vectors (hence “word to vector”) based on their usage in large amounts of text data. Each word is assigned a position in a multi-dimensional space such that words used in similar contexts are positioned closer together. This spatial arrangement creates ‘semantic’ relationships: words with similar meanings are located near each other, and the differences between word vectors can capture meaningful relationships.

A measure of the similarity between two vectors is called cosine similarity. In the context of NLP, it’s often used to measure the semantic similarity between two words (or word vectors). If the word vectors are close in the multi-dimensional space (meaning the angle between them is small), their cosine similarity will be high, indicating that the words are used in similar contexts and likely have similar meanings. There are some interesting parallels between Saussure’s linguistics and AI language models. Both approaches stress that words do not have meaning in isolation but gain their meaning through their relations to other words within the system; language for Saussure and the trained model for word2vec.

Neural Networks: Biological and Artificial

Recall Hubert Dreyfus’s critique of formal symbolic reasoning in artificial intelligence and the assumption that all knowledge can be explicitly encoded. His critique is most relevant to traditional programming in which explicit program instructions are given. In machine learning, however, and in artificial neural networks program rules are developed in response to data. To whatever degree this is similar to the human mind, biology is at least the inspiration for artificial neural networks.

What is the structure of biological neural networks (BNNs)? In the brain connections between neurons are called synapses. Synapses are the tiny gaps at the junctions between neurons  in the brain where communication occurs. They play a vital role in the transmission of information in the brain. Each neuron can be connected to many others through synapses, forming a complex network of communicating cells.

Neurons communicate across the synapse using chemicals called neurotransmitters. When an electrical signal (an action potential) reaches the end of a neuron (the presynaptic neuron), it triggers the release of neurotransmitters into the synapse. These chemicals cross the synapse and bind to receptors on the receiving neuron (the postsynaptic neuron), which can result in a new electrical signal in that neuron. This is how neurons interact with each other and transmit information around the brain.

Synapses form during development and continue to form throughout life as part of learning and memory processes. The creation of new synapses is called synaptogenesis. This happens when a neuron extends a structure called an axon toward another neuron. When the axon of one neuron comes into close enough proximity with the dendrite of another neuron, a synapse can be formed.

The strength of synapses in the brain can change, a phenomenon known as synaptic plasticity. This is thought to be the basis of learning and memory. When two neurons are activated together frequently, the synapse between them can become stronger, a concept known as long-term potentiation (LTP). This is often summarized by the phrase “neurons that fire together, wire together”.

On the other hand, if two neurons aren’t activated together for a while, or the activation is uncorrelated, the synapse between them can become weaker, a process known as long-term depression (LTD).

Multiple factors contribute to these changes in synaptic strength, including the amount of neurotransmitter released, the sensitivity of the postsynaptic neuron, and structural changes such as the growth or retraction of synaptic connections. By adjusting the strength of synaptic connections, the brain can adapt to new experiences, form new memories, and continually rewire itself. This is a dynamic and ongoing process that underlies the brain’s remarkable plasticity.

How then do biological neural networks compare to artificial neural networks? In an artificial neural network, each connection between artificial neurons (also called nodes or units) has an associated weight. These weights play a role somewhat analogous to the strength of synaptic connections in a biological brain. A weight in an ANN determines the influence or importance of an input to the artificial neuron. When the network is being trained, these weights are iteratively adjusted in response to the input the network receives and the error in the network’s output. The goal of the training is to minimize this error, usually defined by a loss function.

The process of adjusting weights in an ANN is a bit like the changes in synaptic strength observed in biological neurons through processes like long-term potentiation (LTP) and long-term depression (LTD). In both cases, the changes are driven by the activity in the network (biological or artificial) and serve to improve the network’s performance – either in terms of survival and behavior for a biological organism, or in terms of prediction or classification accuracy for an ANN.

Of courses there are still multiple differences between biological neural networks and artificial neural networks. ANNs usually involve much simpler learning rules and lack many of the complex dynamics found in biological brains, such as the various types of neurons and synapses, detailed temporal dynamics, and biochemical processes. The biological synaptic plasticity is a much richer and more complex process than the adjustment of weights in an ANN. Also, in most ANNs, once training is complete, the weights remain static, while in biological brains, synaptic strength is continually adapting throughout an organism’s life. Biological and artificial neural networks share computational principles but they certainly don’t implement these computations in the same way.  Brains and computers are simply very different physical things, right down to the materials that compose them.

Artificial neural networks have been in development for several decades. But it is in very recent years that we’ve seen some especially remarkable advances, to which we’ll turn now.

Recent Scientific Literature

I’d like to share 5 papers that I’ve found useful on this subject. Three are research papers with experimental data and two are perspective papers that offer some broad reflections and theoretical considerations.

The 3 research papers are:

“Brains and algorithms partially converge in natural language processing”, published in Communications Biology in 2022 by Caucheteux & King.

“Shared computational principles for language processing in humans and deep language models”, published in Nature Neuroscience in 2022 by Goldstein et al.

“Explaining neural activity in human listeners with deep learning via natural language processing of narrative text”, published in Scientific Reports in 2022 by Russo et al.

And the 2 perspective articles are:

“Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks”, published in Neuron in 2020 by Hasson et al.

“A deep learning framework for neuroscience”, published in Nature Neuroscience in 2019 by Richards et al.

In each of the three research papers human participants read or listened to certain passages while their brain signals for specific brain regions were measured. Deep learning models were trained on this data to predict the brain signals that would result from the text. Researchers looked for instances of high correlation between actual brain patterns and the brain patterns predicted by the model and mapped where in the brain these signals occurred at various points in time before and after word onset. In particular, they noted whether the brain regions activated corresponded to those regions that would be expected from neuroscience to activate in the various stages of language processing.

In the first article, “Brains and algorithms partially converge in natural language processing”, published in Communications Biology in 2022 by Caucheteux & King the researchers used deep learning models to predict brain responses to certain sentences. Then the actual brain responses of human subjects were used as training data for the models. They used a variety of models that they classified as visual, lexical, and compositional. Then they evaluated how well these different types of models matched brain responses in different brain regions. The brain responses in the human subjects were measured using functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG).

Regarding the 3 different types of models:

Visual models are deep learning models that are primarily used for tasks involving images or videos. They are trained to recognize patterns in visual data, which can then be used to perform tasks such as image classification, object detection, image generation, and more. The most common types of visual deep learning models are convolutional neural networks (CNNs). CNNs are specifically designed to process pixel data and have their architecture inspired by the human visual cortex. 

Lexical models are models that focus on the processing of words or “lexemes” in a language. They work with individual words or groups of words (n-grams), treating them as atomic units. Lexical models can learn word representations (often called “embeddings”) that capture the semantic meanings of words, and their relationships with each other. They are often used in natural language processing (NLP) tasks such as text classification, sentiment analysis, and named entity recognition. Examples of lexical models include traditional word2vec or GloVe models, which map words into a high-dimensional vector space.

Compositional models, also called “sequential” or “recurrent” models, handle sequences of data where the order of the data points is important, such as sentences, time-series data, etc. They are designed to process one part of the sequence at a time and maintain a kind of memory (in the form of hidden states) of what has been seen so far. This allows them to capture patterns over time and use this information to make predictions about future data points in the sequence. Examples include causal language transformers (CLTs) like GPT.

Interestingly enough, the accuracy of the different types of models was observed to vary with time from the word onset. And the moments of high correlation of each model type corresponded with the activation of certain brain regions.

In early visual responses – less than 150 ms, when subjects would first see a word – brain activations were in the primary visual cortex and correlated best with activations in visual models, convolutional neural networks (CNNs).

At around 200 ms these brain activations were conveyed to the posterior fusiform gyrus. At the same time lexical models like Word2Vec started to correlate better than CNNs. This tracks with the hypothesis that the fusiform gyrus is responsible for orthographic and morphemic computations.

Around 400 ms brain activations were present in a broad fronto-temporo-parietal network that peaked in the left temporal gyrus. At this point lexical models like Word2Vec also correlated with the entire language network. These word representations were then sustained for several seconds, suggesting a widespread distribution of meaning in the brain.

Around 500-600 ms there were complex recurrent dynamics dominated by both visual and lexical representations.

After 800 ms, brain activations were present in the prefrontal, parietal, and temporal lobes. At the same time compositional models like causal language transformers (CLTs) correlated better than lexical models. The team speculated that these late responses might be due to the complexity of the sentences used in this study, potentially delaying compositional computations.

The researchers concluded from their experiment that the results show that deep learning algorithms partially converge toward brain-like solutions.

In “Shared computational principles for language processing in humans and deep language models”, published in Nature Neuroscience in 2022 by Goldstein et al. the researchers compared the responses of human participants and autoregressive deep language models (DLMs) to the text of a 30-minute podcast.

The authors note that human language has traditionally been explained by psycholinguistic approaches using interpretable models that combine symbolic elements, such as nouns, verbs, adjectives, and adverbs, with rule-based operations. This is similar to the kind of traditional programming that Hubert Dreyfus argued would not be viable for AI. In contrast, autoregressive Deep Language Models (DLMs) learn language from real-world textual examples, with minimal or no explicit prior knowledge about language structure. They do not parse words into parts of speech or apply explicit syntactic transformations. Instead, these models learn to encode a sequence of words into a numerical vector, termed a contextual embedding, from which the model decodes the next word. Autoregressive DLMs, such as GPT-2, have demonstrated effectiveness in capturing the structure of language. But the open question is whether the core computational principles of these models relate to how the human brain processes language. The authors present their experimental findings as evidence that human brains process incoming speech in a manner similar to an autoregressive DLM.

In the first experimental setup, participants proceeded word by word through a 30-minute transcribed podcast, providing a prediction of each upcoming word. Both the human participants and GPT-2 were able to predict words well above chance. And there was high overlap in the accuracy of the predictions of human subjects and GPT-2 for individual words, i.e. words that human subjects predicted well GPT-2 also predicted well. This experiment was determined to demonstrate that listeners can accurately predict upcoming words when explicitly instructed to do so, and that human predictions and autoregressive DLM predictions are matched in this context. Next the researchers wanted to determine if the human brain, like an autoregressive DLM, is continuously engaged in spontaneous next-word prediction without such explicit instruction. And whether neural signals actually contain information about the words being predicted.

In the next experimental setup, the researchers used electrocorticography (ECoG) to measure neural responses of human participants before and after word-onset. Subjects engaged in free listening, without being given any explicit instruction to predict upcoming words. The goal was to see if our brains engage in such prediction all the time as simply a natural part of language comprehension.

The results from human subjects in this experiment were also compared to models. The first model used was a static word embedding model, GloVe. The model was used to localize electrodes containing reliable responses to single words in the narrative. The words were aligned with neural signals and then the model would be trained to predict neural signals from word embeddings. A series of coefficients corresponding to features of the word embedding was learned using linear regression to predict the neural signal across words from the assigned embeddings. “The model was evaluated by computing the correlation between the reconstructed signal and the actual signal” for the word.

In the results of this experiment there was indeed found to be a neural signal before word onset. But what the model also enabled the researchers to do was ascertain some kind of semantic content from that signal, since the model had been trained to predict certain neural signals for given words. What was observed was that “the neural responses before word onset contained information about human predictions regarding the identity of the next word. Crucially, the encoding was high for both correct and incorrect predictions. This demonstrated that pre-word-onset neural activity contains information about what listeners actually predicted, irrespective of what they subsequently perceived.” Of course, sometimes the subject’s predictions were wrong. So what happened in those cases? “The neural responses after word onset contained information about the words that were actually perceived.” So “the encoding before word onset was aligned with the content of the predicted words” and “ the encoding after word onset was aligned with the content of the perceived words.” This all aligns with what we would expect under a predictive processing (PP) model of the brain.

The next level of analysis was to replace the static embedding model (GloVe) with a contextual embedding model (GPT-2) to determine if this would improve the ability to predict the neural signals to each word. It did; an indication that contextual embedding is a closer approximation to the computational principles underlying human language. And the improved correlation from contextual embedding was found to be localized to specific brain regions. “Encoding based on contextual embeddings resulted in statistically significant correlations” in electrodes that “were not significantly predicted by static embedding. The additional electrodes revealed by contextual embedding were mainly located in higher-order language areas with long processing timescales along the inferior frontal gyrus, temporal pole, posterior superior temporal gyrus, parietal lobe and angular gyrus.” The authors concluded from this that “the brain is coding for the semantic relationship among words contained in static embeddings while also being tuned to the unique contextual relationship between the specific word and the preceding words in the sequence.”

The authors submit that DLMs provide a new modeling framework that drastically departs from classical psycholinguistic models. They are not designed to learn a concise set of interpretable syntactic rules to be implemented in novel situations, nor do they rely on part of speech concepts or other linguistic terms. Instead, they learn from surface-level linguistic behavior to predict and generate the contextually appropriate linguistic outputs. And they propose that their experiments provide compelling behavioral and neural evidence for shared computational principles between the way the human brain and autoregressive DLMs process natural language.

In “Explaining neural activity in human listeners with deep learning via natural language processing of narrative text”, published in Scientific Reports in 2022 by Russo et al. human participants listened to a short story, both forward and backward. Their brain responses were measured by functional MRI. Text versions of the same story were tokenized and submitted to GPT-2. Both the brain signal data and GPT-2 outputs were fed into a general linear model to encode the fMRI signals.

The 2 outputs researchers looked at from GPT-2 were surprisal and saliency. Surprisal is a measure of the information content associated with an event, in terms of its unexpectedness or rarity. The more unlikely an event, the higher its surprisal. It is defined mathematically as the negative logarithm of the probability of the event. Saliency refers to the quality by which an object stands out relative to its neighbors. In a text it’s the importance or prominence of certain words, phrases, or topics, a measure of how much a particular text element stands out relative to others in the same context.

What they found in their results was that the surprisal from GPT-2 correlated with the neural signals in the superior and middle temporal gyri, in the anterior and posterior cingulate cortices, and in the left prefrontal cortex. Saliency from GPT-2 correlated with the neural signals for longer segments in the left superior and middle temporal gyri.

The authors proposed that their results corroborated the idea that word-level prediction is accurately indexed by the surprisal metric and that the neural activation observed from the saliency scores suggests the co-occurrence of a weighing mechanism operating on the context words. This was something previously hypothesized as necessary to language comprehension.

The involvement of areas in the middle and the superior temporal gyrus aligns with previous studies supporting that core aspects of language comprehension, such as maintaining intermediate representations active in working memory and predicting upcoming words, do not necessarily engage areas in the executive control network but are instead performed by language-selective brain areas that, in this case, are the ones relatively early in the processing hierarchy.

I found the following comment in the discussion section of the paper quite interesting: “In general, considering that the architecture of artificial neural networks was originally inspired by the same principles of biological neural networks, it might be not at all surprising that some specific dynamics observed in the former are somehow reflected in the functioning of the latter.” I think that’s an interesting point. The whole idea of artificial neural networks came from biological neural networks. We were basically trying to do something similar to what neurons do. We don’t know exhaustively how biological neural networks work but we do know that they work very well. When we are finally able to make artificial networks that work quite well it’s perhaps to be expected that they would have similar characteristics as biological neural networks.

The other two papers were perspective papers. These didn’t present the results of experiments but discussed what I thought were some interesting ideas relating to the whole interchange between language processing in the human brain and in deep learning models.

In “Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks”, published in Neuron in 2020 by Hasson et al. the authors covered several topics. One thing they addressed that I found interesting was a challenge to three basic assumptions of cognitive psychology. These assumptions are:

1. The brain’s computational resources are limited and the underlying neural code must be optimized for particular functions. They attribute this to Noam Chomsky and Jerry Fodor.

2. The brain’s inputs are ambiguous and too impoverished for learning without built-in knowledge. They attribute this to Noam Chomsky.

3. Shallow, externally supervised and self-supervised methods are not sufficient for learning. They attribute this to Steven Pinker.

In response to the first assumption the authors argue that the brain’s computational resources are actually not scarce. “Each cubic millimeter of cortex contains hundreds of thousands of neurons with millions of adjustable synaptic weights, and BNNs utilize complex circuit motifs hierarchically organized across many poorly understood cortical areas. Thus, relative to BNNs, ANNs are simplistic and minuscule.” Artificial neural networks are indeed trained on huge amounts of data. GPT-4 is essentially trained on the whole internet. Human children don’t learn to talk by reading the whole internet; that’s true. But the human brain is also a lot more complex than even the most sophisticated artificial neural networks; so far at least. So if GPT-4 is able to perform so impressively with a structure that’s less sophisticated than the human brain we can expect that the human brain’s computational resources are hardly scarce.

In response to the second assumption the authors argue that the brain’s input is not impoverished. Noam Chomsky, arguably the most important linguist of the 20th century, argued for what he called the “poverty of the stimulus,” meaning that the linguistic input children receive is often incomplete, ungrammatical, or otherwise imperfect. But they still manage to learn their native language effectively. How? Chomsky proposed that there is a “Language Acquisition Device” (LAD) within the human brain. This hypothetical module is thought to be equipped with knowledge of a “Universal Grammar,” which encapsulates the structural rules common to all human languages. But Hasson et al. argue that there is no poverty of the stimulus because deep learning models can produce direct fit with reliable interpretations using dense and broad sampling for the parameter space. The model is casting a very wide net. They state: “One of our main insights is that dense sampling changes the nature of the problem and exposes the power of direct-fit interpolation-based learning… The unexpected power of ANNs to discover unintuitive structure in the world suggests that our attempts to intuitively quantify the statistical structure in the world may fall short. How confident are we that multimodal inputs are in fact not so rich?” By the way, I was sharing a draft of this with a friend who shared another recent paper with me by UC Berkeley professor Steven Piantadosi, titled “Modern language models refute Chomsky’s approach to language”. I’m not going to get into that now but just thought I’d mention it.

In response to the third assumption the authors argue that shallow self-supervision and external-supervision are sufficient for learning. The authors cite Pinker’s book The Language Instinct: How the Mind Creates Language as an example of the view that they are challenging. Pinker’s views are very similar to Chomsky’s. Pinker argues that language learning is not just about imitation or conditioning. Instead, he believes that the human brain has an inherent structure for understanding language, which is why children are able to learn languages so rapidly and effortlessly, often making grammatical leaps that aren’t explicitly taught or present in their environment. But Hasson et al. argue that humans have a great deal of external supervision from our environment, both social and physical. They refer to the importance of embodiment to predictive processing, referring to the ideas of Andy Clark and Karl Friston, among others.

Another subject the authors address is the issue of interpretability. This goes back to the Verum factum principle from Vico. Scientific models, including those in neuroscience, are often evaluated based on two desirable features: (1) interpretability and (2) generalization. We want explanations to have good predictive power but we also want to be able to understand them. Not just verify that they work. If it’s an equation we like to be able to look at an equation and be able to intuit how it works. And this means that the equation can’t be too long or have too many parameters. However, interpretability and generalization are often in conflict. Models with good interpretability may have strong explanatory appeal but poor predictive power, and vice versa.

The authors suggest that the brain is an exceptionally over-parameterized modeling organ. Interpretability in the brain is intractable for the same reason interpretability of deep learning models is intractable. They work with a huge number of parameters. There’s quantification occurring but it’s not like a concise equation that you can look at in grasp intellectually. The authors propose that neural computation relies on brute-force direct fitting, which uses over-parameterized optimization algorithms to enhance predictive power, i.e. generalization, without explicitly modeling the underlying generative structure of the world.

One thing that’s really nice about this paper (and I highly recommend it by the way, it’s a delightful read) is its 3 “boxes” that touch on some key concepts. One box covers the biomimicry of biological neural networks by artificial neural networks. The authors state that artificial neural networks (ANNs) are learning models that draw inspiration from the biological neural networks (BNNs) present in living brains, but that ANNs are a highly abstracted version of BNNs. Some biological nervous systems include functional specialized system-level components like the hippocampus, striatum, thalamus, and hypothalamus, elements not included in contemporary ANNs. ANNs are also disembodied and do not closely interact with the environment in a closed-loop manner. While the authors concede that ANNs are indeed highly simplified models of BNNs, they propose that there exist some essential similarities: they both belong to the same group of over-parameterized, direct-fit models that depend on dense sampling for learning task-relevant structures in data. And, crucially, ANNs are currently the only models that achieve human-like behavioral performance in many domains and can offer unanticipated insights into both the strengths and limitations of the direct-fit approach. Like BNNs, ANNs are founded on a collection of connected nodes known as artificial neurons or units that loosely resemble neurons in a biological nervous system. Each connection, akin to synapses in BNNs, links one artificial neuron to another, and the strength of these connections can be adjusted through learning. The connections between artificial neurons have weights that are adjusted during the learning process based on supervised feedback or reward signals. The weight amplifies or reduces the strength of a connection. And much like BNNs, ANNs are sometimes organized into layers.

Another “box” addresses embodiment. This is something the philosopher Andy Clark has addressed a lot in his work. Not to mention, going further back, the philosopher Maruice Merleau-Ponty. At present, ANNs are disembodied and unable to actively sample or modify their world. The brain does not operate with strictly defined training and test regimes as found in machine learning. Objective functions in BNNs must satisfy certain body-imposed constraints to behave adaptively when interacting with the world. The authors suggest that adding a body to current ANNs, capable of actively sampling and interacting with the world, along with ways to directly interact with other networks, could increase the network’s learning capacity and reduce the gaps between BNNs and ANNs. Interestingly enough, they cite Wittgenstein’s “Philosophical Investigations” when addressing the way social others direct our learning processes.

One other topic in the paper that I found interesting was a discussion of “System 1” and “System 2”. This model was made most famous by Daniel Kahneman in his 2011 book Thinking Fast and Slow. The authors cite Jonathan St B. T. Evans’s 1984 paper “Heuristic and analytic processes in reasoning”. And there are earlier precedents for the general idea going back further in history. System 1 represents fast, automatic, and intuitive thinking, what Evans called heuristic processes. And System 2 represents slow, effortful, and deliberate thinking, what Evans called analytic processes. Hasson et al. propose that we can understand System 1 to be a kind of substrate from which System 2 can arise. System 2 is where things get really interesting. That’s where we find some of the most impressive capacities of the human mind. But they maintain that we have to start with System 1 and build from there. They state: “Although the human mind inspires us to touch the stars, it is grounded in the mindless billions of direct-fit parameters of System 1.” They see artificial neural networks as having the most relevance toward explaining System 1 processes. And the thing is we seem to be continually finding that System 1 includes more than we might have thought. “Every day, new ANN architectures are developed using direct-fit procedures to learn and perform more complex cognitive functions, such as driving, translating languages, learning calculus, or making a restaurant reservation–functions that were historically assumed to be under the jurisdiction of System 2.” 

In “A deep learning framework for neuroscience”, published in Nature Neuroscience in 2019 by Richards et al. the authors focus on three key features of artificial neural network design – (1) objective functions, (2) learning rules, and (3) architectures – and address how these design components can impact neuroscience.

The authors observe that when the traditional framework for systems neuroscience was formulated, they could only collect data from a small selection of neurons. Under this framework, a scientist observes neural activity, formulates a theory of what individual neurons compute, and then constructs a circuit-level theory of how these neurons integrate their operations. However, the question arises as to whether this traditional framework can scale up to accommodate recordings from thousands of neurons and all of the behaviors that one might want to explain. It’s arguable that the classical approach hasn’t seen as much success when applied to large neural circuits that perform a variety of functions, such as the neocortex or hippocampus. These limitations of the classical framework suggest that new methodologies are necessary to capitalize on experimental advancements.

At their fundamental level, ANNs model neural computation using simplified units that loosely emulate the integration and activation properties of real neurons. The specific computations performed by ANNs are not designed but learned. When setting up ANNs, scientists don’t shape the specific computations performed by the network. Instead, they establish the three components mentioned previously: objective functions, learning rules, and architecture. Objective functions measure the network’s performance on a task, and learning involves finding synaptic weights that maximize or minimize this objective function. These are often referred to as ‘loss’ or ‘cost’ functions. Learning rules offer a guide for updating the synaptic weights. And architectures dictate the arrangement of units in the network and determine the flow of information, as well as the computations the network can or cannot learn.

Richards et al. make an observation about interpretability similar to that made by Hasson et al. The computations that emerge in large-scale ANNs trained on high-dimensional datasets can be hard to interpret. An ANN can be constructed with a few lines of code, and for each unit in an ANN, the equations determining their responses to stimuli or relationships to behavior can be specified. But after training, a network is characterized by millions of weights that collectively encode what the network has learned, and it is difficult to envision how such a system could be described with only a few parameters, let alone in words. They suggest that we think about this in the following way. Theories can have a compact explanation that can be expressed in relatively few words that can then be used to develop more complex, non-compact models. They give the theory of evolution by natural selection as a comparative example. The underlying principle is fairly simple and comprehensible, even if the actual mechanics that emerge from it are very complex. For systems neuroscience we can start with these three relatively simple and comprehensible principles: objective functions, learning rules, and architecture. Then even though the system that emerges from that is too complex to comprehend at least the underlying principles are comprehensible and give some degree of intuitive understanding.

Conclusion

Something that I find exciting about all this is that it’s an interesting interface between philosophy of mind, neuroscience, and programming. I think that some of the most interesting problems out there are philosophical problems. Even many scientific problems transition into philosophical problems eventually. But our philosophy needs periodic grounding in the world of empirical observations. What we might call armchair philosophy runs the danger of getting untethered from reality. In the philosophy of mind we can speculate about a lot of things that don’t work out very well in neuroscience. That’s not to say that philosophy of mind has to be entirely bounded by neuroscience. Just because human minds work in a certain way doesn’t mean that minds of any kind would have to be constrained in the same way. There could be many different ways for minds to work. But if we’re theorizing about ways other types of minds might work we don’t, at present, have ways to verify that they actually would work. With theories about human minds we can at least try to verify them. Even that’s kind of challenging though because the brain is so complex and difficult to observe directly at high resolution.

Still, there’s a lot about our brains that we do know that we can take into account in our theories of the mind. We know that our brains have neurons and that neurons make synaptic connections. And we know that those synaptic connections can strengthen or weaken. We can at least account for that in our theories. Artificial neural networks patterned after biological neural networks are useful tools to model our brains. We can’t go into every synaptic cleft in the brain to sample its flux of neurotransmitters. Or record the firing frequency of every neuron in the brain. That would be great but we just don’t have that capability. With artificial neural networks, as imperfect approximations as they are, we at least have recorded information for billions of parameters, even if their sheer quantity defies comprehension. And we can try out different configurations to see how well they work.

Another subtopic that’s interested me for a while is the possibility of what I call a general theory of mind. “General” in the sense of applying beyond just the special case of human minds, a theory of the human mind being a “special” theory of mind. What other kinds of minds might there be? What are all the different ways that a mind can work? AI might give us the ability to simulate and test more general and exotic possibilities and to extract the general principles they all hold in common.

I think the recent success of these large language models is quite exciting. Maybe a little bit frightening. But I’m mostly excited to see what we can learn.