How many angels can dance on the head of a pin?

The question “How many angels can dance on the head of a pin?” is typically used as a mocking retort to questions that are thought to be of little usefulness. It’s especially used in reference to philosopher theologians of the Middle Ages like the Scholastics or to theology in general. It’s not a question any of the Scholastics ever actually asked. But medieval philosophy did have plenty of talk about angels, and for good reason. They used angels as subjects for thought experiments to explore concepts like cognition and identity in the most generalized way possible, in the way modern philosophers talk about brains in a vat, brains separated from the body and sent to another planet, philosophical zombies, or people living in a black-and-whiteworld. Their topics are just as relevant today as we develop technologies like artificial intelligence and deepen our understanding of the brain and the mind.

The question “How many angels can dance on the head of a pin?” is typically used as a mocking retort to the sorts of philosophical and especially theological questions that are thought to be of little usefulness and a general waste of effort. It’s especially used in reference to philosopher theologians of the Middle Ages like Scholastics including Thomas Aquinas and Duns Scotus. While the material conditions of people in the world were those of misery and squalor (so it is supposed) these folks were sitting in their ivory towers thinking about useless questions instead of doing science and inventing things. As you might guess, I don’t share this perspective and as a kind of subversion of the retort I’d like to appropriate it. The question “How many angels can dance on the head of a pin?” is not a question anyone actually ever asked. It’s a straight up caricature. But Medieval Scholastic philosophy did have plenty of talk about angels. Why? And could angels have any modern intellectual relevance?

In what follows I propose that angels were used in Medieval Scholastic philosophy as subjects for thought experiments. In such thought experiments the properties of angels were not those primarily of angels as described in biblical texts but more of an idealized notion serviceable to philosophical exercises. I propose that these sorts of philosophical angels can still be used to explore questions we find interesting today in thought experiments pertaining to cognition and consciousness. With an understanding of thought experiments as idealizations which transcend particularity in order to achieve generality I’ll go through various stages of generalization of consciousness from its particular human form to its most general form. This process moves into the complete abstraction of consciousness from any of its particular physical instantiations in order to explore its most essential features, though with our present knowledge this can only be the outline of a conceptual scaffolding since we don’t currently know what the essential features of consciousness actually are, or what it even is. And this will lead back to speculation about the nature of angels and what their nature might be.

When Medieval philosophers like Aquinas and Scotus talked about angels they did not get into historical scholarship about the way angels were understood at the times that the biblical texts were written. Or the way angels were understood during the Second Temple period with the writing of pseudepigraphal texts like the Book of Enoch. This is a fascinating subject and on this subject I’d recommend the work of Michael S. Heiser in his books Angels and The Unseen Realm. In Biblical texts angels are messengers. That’s what the Greek angelos (ἄγγελος) means. Also the Hebrew malak (מֲלְאָךְ). There’s not much information given about their metaphysical nature. But it is their metaphysical nature that is most interesting to the Medieval philosophers.

The most important attribute of angels for philosophical purposes is that they are non-corporeal. Angels do not have physical bodies. So they are very unlike us humans. Yet they are also like us humans in a very significant aspect: they are conscious beings. They’re not only conscious but also self-conscious, intelligent, and rational, again like humans. In our regular experience we only know of one kind being that is self-conscious, intelligent, and rational: human beings. And we have good reason to believe that these attributes are essentially connected to our physical bodies and especially our brains. The idea that a being could be self-conscious, intelligent, and rational without a physical body or even just a physical brain conflicts with our regular experience. But that’s why it’s interesting.

An excellent resource on the use of angels in Medieval philosophy is Martin Lenz and Isabel Iribarren’s Angels in Medieval Philosophical Inquiry: Their Function and Significance, which is a collection of essays on the subject. In particular I recommend the chapter by Dominik Perler titled Thought Experiments: The Methodological Function of Angels in Late Medieval Epistemology. In his essay Perler works with a definition of the thought experiment from The Routledge Encyclopedia of Philosophy, given by Timothy Gooding, whom he quotes: “A thought experiment is an idealization which transcends the particularity and the accidents of worldly human activities in order to achieve the generality and rigour of a demonstrative procedure.” I think this is an excellent definition for a thought experiment. If you want to get at the essence of a concept a thought experiment is a way of looking at it in the most generalized way possible. There are certain concepts like language, rationality, and possibly self-consciousness (of the most reflective sort) that we only find in human beings. How can we think about these concepts in their general form when we only have one kind of example from actual experience? We have to use our imaginations. And so we make thought experiments.

Perler notes that thought experiments take different forms in different ages. I would say that they make use of the images that are readily available in the culture. “Today, of course, philosophers hardly speak about angels. They prefer talking about brains in the vat, brains separated from the body and sent to another planet, zombies, or people living in a black-and-whiteworld.” We take our ideas from the culture: from religion, myth, literature, and film. But are these necessarily fictional? Perler makes an interesting distinction: “Of course, one needs to make a crucial distinction when talking about thought experiments. They can be understood either as scenarios involving purely fictitious entities (e.g., brains in the vat), or as scenarios appealing to entities that have real existence or could in principle have real existence, but are considered under ideal conditions (e.g., the scientist Mary who has all knowledge about colours). Since medieval authors took angels to be real entities, endowed with real causal power and interacting with other real entities, they were certainly not interested in thought experiments in the first sense [fictitious entities]. They were rather focusing on thought experiments in the second sense, analyzing angels as real creatures that transcend the material world and therefore enable us to examine cognitive activities in its purest and most ideal form, which is not subject to material constraints.”

I mentioned before that no Medieval person really asked “How many angels can dance on the head of a pin?”. But we can find examples of something kind of close. In the Summa Theologiae Thomas Aquinas asked the question: “Whether several angels can be at the same time in the same place?” (Summa Theologiae, First Part, Question 52, Article 3) This he answered in the negative: “I answer that, There are not two angels in the same place.” I’d actually answer the question differently I think. But what’s important is that the question makes sense to ask. We assume here that angels are immaterial. So there are questions that arise regarding the relation of immaterial things to space. Does something immaterial take up space? Could it take up space under certain circumstances? If the specific question about angels seems too remote, think about other immaterial things. First, consider mathematical sentences like 1+1=2. Does that take up space? It would seem not to. It’s just not a spatial kind of thing. Second, consider a field as understood in physics? A field is “a physical quantity, represented by a scalar, vector, or tensor, that has a value for each point in space and time.” Some fields probably have actual physical existence. In quantum field theory certain fields are understood to be the most basic building block of physical reality that give rise to everything else. But other fields are more abstract, especially since we can invent all kinds of fields to suit our purposes. For example, we can imagine a temperature field where every point in a given volume has a certain scalar temperature value. This kind of field would obviously be spatial in nature since it is defined over a given region of space. But it’s not exactly a physical thing either. It’s just a set of numbers. These two kinds of immaterial things have very different relations to space. For any kind of immaterial thing, including angels, it’s reasonable to ask which kind of relation to space it has. How many mathematical sentences, such as 1+1=2, can fit on the head of a pin? In the asking of that question we can see that mathematical sentences just aren’t the kinds of things that take up space or subsist in space at all. That’s an interesting feature of mathematical sentences.

Let’s look at another Medieval example. In his article Dominik Perler looks at the work of Duns Scotus and William of Ockham on the subject of perception. They framed the issue as one of how angels could have cognition of things in the world. Physical beings perceive the world through their physical senses. But nonphysical beings wouldn’t have sense organs of this sort. They would need other sorts of cognitive devices “to make things cognitively present to them.” Both Scotus and Ockham held that “every cognition requires a cognitive act and a cognitive object.” But they had different views about what that cognitive object would be. For Scotus, “the appropriate object for an intellectual act is the essence of a thing.” For Ockham, “the appropriate object for an intellectual act is the individual thing with its individual qualities.” Both Scotus and Ockham then think about how cognitive objects become cognitively accessible. And they use angels in their thought experiments to make this as general as possible. They don’t even refer to sense perception. Rather, there’s simply something that makes the cognitive object cognitively accessible. That something could be sense perception but it doesn’t have to be. So it’s thoroughly general. For me, as a physical being with physical senses, before I cognize something like a chair I perceive it with my senses. But after I perceive it with my senses there’s an intellectual act by which I cognize it. For Scotus the object of this intellectual act is the essence of a chair. For Ockham the object of this intellectual act is the individual object, which happens to be a chair. These are two very different ways of understanding cognition. And by using angels, nonphysical beings, in their thought experiments they bracket everything that comes before the intellectual act because it’s that intellectual act specifically that interests them. It doesn’t matter if what makes the cognitive object cognitively accessible is sight, smell, echolocation, electroreceptors, or what have you. This can apply to humans, bats, sharks, computers, or angels. That’s why it’s fully generalized. They are just interested in what happens in the intellect.

So how does the cognitive object become cognitively accessible to the intellect? For Scotus, the cognitive object is the essence of a thing and “to make the essence of a thing cognitively accessible, the intellect needs a cognitive device: the intelligible species.” For Ockham, the cognitive object is the individual thing and “to make an individual thing cognitively accessible, the intellect simply needs a causal relationship with that thing.” For Scotus, “the intelligible species determines the content of an intellectual act”. For Ockham, “the individual thing itself determines the content of an intellectual act.” What does this look like in practice? I expect that Ockham’s view will seem most plausible up front. Going back to the chair example, the essence of a chair sounds kind of fictional. Isn’t it just the thing itself we’re dealing with that we then understand to be a chair? Nevertheless, I’d say that in the history of philosophy these two views are pretty evenly represented. Just to comment on it briefly, it’s arguably that our cognition always has some intentionality to it. It’s always directed towards something. We don’t just cognize the thing but also what we can do with it. A chair, for example, is not just an elevated platform with four legs and a backing. A chair is something to sit on. And it’s arguable that that usefulness is the first thing we cognize.

Perler comments that for both Scotus and Ockham, “their detailed analysis was motivated by an interest in the general structure of cognition… They both wanted to know how cognition works in principle, i.e. what kinds of entities and relations are required in any cognitive process.” The philosophy of mind understandably tends to be very human-focused. But that leaves out many other ways of cognizing, both real and imagined, from bats to sharks to computers. By using angels in their thought experiments Scotus and Ockham were able to think about these issues in a way that transcended the particularity of all these cases and think about cognition as such. I think that Medieval people had an easier time thinking about human attributes in this more general way because more of them believed in a universe populated by beings of an unseen realm that had many human-like attributes. Part of the “discarded image” as C.S. Lewis called it. The anthropologist Marshall Sahlins called these kinds of beings “metapersons”, something he proposed recovering and studying in a “new science of the enchanted universe” (see The New Science of the Enchanted Universe: An Anthropology of Most of Humanity). This worldview of a heavily populated unseen realm led Medievals to come up with useful concepts that they otherwise might not have. This is something Joseph Koterski has argued, with the case of Boethius in particular (see Natural Law and Human Nature). In his writings about these various kinds of human-like beings Boethius had to come up with a technical term and stipulate an appropriate definition to refer to all of them, both human and non-human. He used the term “person”, which he defined as an individual substance of a rational nature. A wonderfully general definition that preserves the essential core.

Could we make similar use of angels in modern thought experiments? I think they could be used in a similar way as used by the Medievals, i.e. to transcend the particularity and accidents of human beings and explore concepts in their most general form. One topic that interests me and where I can see application of this is to consciousness, or more especially self-consciousness. This is a distinction that Roger Scruton has made (see The Soul of the World, p. 40). Consciousness is the state of being aware of and able to perceive one’s environment, thoughts, and feelings. This is something that many animals would seem to have.  Self-consciousness is a higher level of awareness where an individual is not only conscious but also aware of themselves as a distinct entity with their own thoughts, feelings, and identity. It involves reflection on one’s own mental states and actions. Humans are certainly self-conscious. As humans we know this because we have direct, first-person access to this higher level of awareness and reflection. And we assume, rightly and non-solipsistically, that this is common to other humans as well. We can also talk to each other and infer from our conversations that we share this self-conscious nature. Other animals might be self-conscious in this way as well. Or they might not be. We don’t really know because we are not of the same species and we can’t talk with them. No other animals have language. It’s possible we Homo sapiens are unique in our self-consciousness.

Let’s consider a progression of abstraction from human consciousness to hypothetical consciousness in other physical beings, to general consciousness as such, abstracted from all particular instances.

We start with human beings. We know a lot about human physiology, psychology, and neurology. All these contribute to the human experience, including our experience of self-consciousness. We know our mental operations are highly if not entirely dependent on our brains. At a very basic level our brains are composed of neurons and the synaptic relations between neurons. Our brains are also further organized into structures whose activity we can observe under various scenarios and activities. We’ve amassed a good deal of knowledge about our brains but there is still a great deal we don’t know. And ultimately we don’t have an answer to what David Chalmers has called the “easy problem of consciousness”: how all our mental experiences like perception, memory, and attention correlate to specific brain mechanisms and processes. We don’t have a complete “map” of the brain. We can’t read people’s minds by looking at their brains. So we still have to speak in rather general terms.

I think the next step removed from human beings would be other species that are closely related to us in the genus Homo. Unfortunately all other species of our genus are extinct so we can’t observe them or talk with them. It’s reasonable that it may have been possible to talk to some of them. Neanderthals seem to have been very similar to humans. They may have had very similar brains, similar enough to have similar capabilities but different enough to be interesting and to make some generalizations about our common features. That’s no longer possible, but that was a very close step removed from human beings that would have been useful for generalization.

To move another step from humans we may have to look beyond our planet and our evolutionary relatives. This would be an organic alien lifeform. We may never encounter such a being but I put it before an artificial lifeform, which we may encounter much sooner, simply because we can imagine such beings being, for lack of a better word, organic: organisms composed of organic molecules with water-based biochemistry, probably composed of cells, with genetic information encoded in molecules, and evolutionary history. Basically, not computers or machines. We can only speculate what such beings might be like. And I suspect we’ll have more insight into the nature of their consciousness after we develop artificial consciousness, for reasons I’ll explain. What might their brains be like? Would they have the basic cellular unit (like neurons) with a complex structure built up on the relations between them?

The next level removed would be beyond any organic physical entity, human or alien, to artificial entities: artificial intelligence or artificial consciousness. The consciousness of artificial consciousness would be considered artificial because it would not be naturally occurring but instead be a product of human engineering and design. What I’m talking about here is not just artificial intelligence of the sort that started showing up everywhere in 2023, when it seemed like almost everything was getting AI capability of some kind. These are large language models (LLMs) like ChatGPT. A large language model’s artificial intelligence processes and generates human-like text based on patterns learned from large sets of text data. But LLMs lack actual awareness and subjective experience. Artificial consciousness, on the other hand, would entail a system having self-awareness, subjective experiences, and a sense of understanding its existence and environment.

We may well encounter artificial consciousness before we encounter alien life. So even though alien life might be more similar to us, being organic, if we don’t encounter it we won’t be able to learn much from it or make comparisons to the particular physical features that give rise to their consciousness. Nevertheless we may be able to infer certain features that a self-conscious alien species would have to have from features held in common between human consciousness and artificial consciousness.

What might be analogous to the brain in artificial consciousness? We can imagine that an artificial consciousness would have physical similarities to modern computers. They would probably have processing units and memory systems. Although existing artificial intelligence in the form of large language models may not be conscious, artificial consciousness may end up having similar features like a neural network structure consisting of layers of interconnected nodes (like neurons) that process data through weighted connections.

What kinds of features would be held in common between all sorts of conscious entities: humans, Neanderthals, aliens, and artificial consciousness? Right now we can’t know. But we can speculate. I suspect that there will be some kind of structure common to all. This might not be the case, which would be very confusing indeed. But let’s suppose there would be some kind of common structure. I would further speculate that it would have the form composed of basic objects plus the relations between objects. For example, in human brains the basic object is the neuron. And the brain is organized as a system of relations between neurons, the synapses. What constitutes the object and the relations might be rather different from one entity to the next. But I suspect that basic structure will apply to all conscious beings. The definition of the structure will be very complex. It’s not just that there are objects and relations between objects. Many structures would meet that description without being conscious. For example, a three-dimensional matrix of neurons in which every neuron was simply connected synaptically to its nearest neighbor wouldn’t be much of a structure. Consciousness-producing structures are much more complex. In all these cases so far each entity is physical and has some consciousness-imparting structure. In Aristotelian terms these entities are composites of matter and form, a notion called hylemorphism. In each case there is a material substratum for the consciousness-producing form or structure.

This brings me to the final level of abstraction where we pull away the material substratum leaving only the consciousness-producing structure itself; the form without the matter. This moves beyond all physical conscious entities to the completely abstract and nonphysical. The features that I’ve speculated are held in common between conscious entities have the features of a mathematical structure: a set of objects having some additional features such as relations between objects. Fully abstracted, the actual objects themselves don’t matter. These instead become open slots. This is how Verity Harte describes structures generally. Structures are the sorts of things that make available open slots that can be filled by particular objects. When the slot is filled by a physical entity, like a neuron, the structure has a particular physical instantiation. But when the slot is empty the structure is abstracted from physical instantiation. Here the structure is at its most generalized state.

The highest levels of abstraction bring up some interesting philosophical issues. Let’s start with artificial consciousness. Here the question of the number of angels dancing on the head of a pin, or the matter of space, comes up again. The volume of an adult human brain is around 1300 cubic centimeters. That’s not even counting the rest of the body that’s needed for the brain to survive and operate. Our consciousness requires space to operate. Artificial consciousness would also take up space. Maybe a single artificial conscious entity would require even more space than a human body or human brain. But let’s hope that it could be less. Could there be a kind of corollary to Moore’s Law for consciousness. Moore’s Law is the trend in the semiconductor industry for the number of transistors in integrated circuits to double every two years. Since individual transistors are being made to be progressively smaller, more can fit in a given area. Could an artificial consciousness fit in the volume of a modern smartphone? Or maybe even smaller? Could multiple artificial consciousnesses fit on the head of the pin? That might give renewed relevance to the never-actually-asked question of angels dancing on the head of a pin. How many artificial consciousnesses could operate in a space the size of the head of a pin?

Artificial consciousness also brings up questions about embodiment. In humans this is a question of the way our minds relate not just to our brains but to our whole bodies and even to our environments. Hubert Dreyfus was an important contributor to philosophical work on this question, notably in his book What Computers Can’t Do. Dreyfus was drawing on the thought of Martin Heidegger. Maurice Merlau-Ponty would also be relevant to the subject. Dreyfus referred to Heideggerian concepts such as being-in-the-world, ready-to-hand, present-to-hand, and mood to explain the necessarily embodied nature of consciousness. Dreyfus argued that human understanding is fundamentally embodied and situated in a social and physical context, that much of human expertise is tacit and non-reflective. We tend to be very brain-centered in our thinking about human beings. But we are much more than just our brains. If a human brain could be separated from the rest of the body and survive it’s hard to speak for that quality of life, if it could even qualify as a life. Notably, even Dreyfus wasn’t arguing that artificial intelligence or artificial consciousness weren’t possible. Just that they would have to have those sets of features he identified. Would an artificial consciousness then require not only processing and memory but also motility and some form of embodied existence in the world?

Then there’s the highest, nonphysical level of abstraction. An abstract, nonphysical entity that has all the essential structures of consciousness. What is the nature of such an abstraction? Could such an abstract entity actually be conscious? Or would it have mere hypothetical consciousness, in the event that the open slots happen to be filled by physical objects? Does it even make sense to think of such nonphysical, abstract entities as existing at all? This last question is the basic question of platonism? Do abstract objects in general have any existence when we’re not thinking about them or using them in some way?

We can imagine the case where this nonphysical abstraction only becomes conscious when it is physically instantiated. It’s embodied in some way, whether that body by organic or computer. This would be like the relationship of the concept of a machine to the machine itself. We might have the concept of a machine like an engine but this concept doesn’t actually perform work. For that we actually need to build the physical machine. But even with the abstract concept of a machine we could say true and false things about it. Any concept for a steam engine, for example, has to have features for heat addition, expansion, heat rejection, and compression. Any concept lacking these features could be said to be nonfunctional, even in the abstract. In the abstract neither the concept for the functional design nor the nonfunctional design produce any work. But if we were to build them, one would work and the other would not. A philosophical angel might be the same way. It’s not actually conscious in the abstract but the abstract structure would be conscious if it were physically instantiated. So we can still refer to it, like Scotus and Ockham, and talk about cognition and consciousness, and how they would work in any kind of physical instantiation, be it human, Neanderthal, alien, or computer.

Or we can imagine the more extravagant case where this nonphysical abstraction actually is conscious, even when unembodied. That’s difficult to imagine. And there are a number of objections we can make. Maybe not the most obvious of these objections but one I think is important is the matter of temporality. Consciousness seems to be essentially temporal in nature. It’s a process. Our brains aren’t static but change from moment to moment. And our brain states are constantly changing. So how would a nonphysical abstraction do the things that conscious entities do across time? We think of abstractions as being atemporal. They don’t ever change. That’s true in a sense. They don’t change with respect to actual time. But some abstract objects could have multiple elements that are organized in a way like a process. For example, an algorithm with multiple steps is a kind of process. The steps in an algorithm aren’t moments in time but they have an ordering and sequence similar to temporal processes. To take another example, a piece of music, when played, is played in time. But we can also look at a whole page of music at once. In the abstract the piece of music is no longer temporal, but it still has order and sequence.

Another objection is that it seems like a nonphysical abstraction just isn’t enough. It needs something more. Stephen Hawking put this thought quite poetically in A Brief History of Time: “Even if there is only one possible unified theory, it is just a set of rules and equations. What is it that breathes fire into the equations and makes a universe for them to describe? The usual approach of science of constructing a mathematical model cannot answer the questions of why there should be a universe for the model to describe. Why does the universe go to all the bother of existing?” Of philosophical angels we might ask, what breathes fire into them? Do they need physical instantiation to be conscious?

In the foregoing I’ve also operated under an assumption that consciousness is a structure of some kind. And that we find these kinds of structures instantiated in multiple ways that constitute individual conscious beings. This is reasonable. We always find them together. We don’t observe human consciousness without human brains. But what if that assumption is not correct? Maybe consciousness is not the structure but the structure is a condition for consciousness. And consciousness itself is something more basic. Maybe consciousness itself is a kind of monad, a fundamental, indivisible unit that possesses perception and self-awareness. A theory of such monads was developed in the thought of Gottfried Wilhelm Leibniz. There are some modern theories based on the idea that consciousness might be a basic component of the universe, rather than something reducible to anything else. Not a structure but a unit. See David Chalmers as one prominent philosopher in this area. That’s a very different picture.

In the monad view the complex structures associated with brains and computer systems would not have some structural essence identical to consciousness. Instead they might produce conditions under which it can appear. As an analogy, a generator built to create an electric current has a complex structure. But this structure is not the electric current. The electric current is something else, and something much simpler. The generator produces the conditions under which an electric current can appear. But neither the generator nor its structure are the electric current itself.

In all of this my thoughts about angels have been instrumental. Not so much a question about what actual angels might be like, but about the utility of a certain concept of angels for philosophical exercises. But what of actual angels? Could any of these ideas about angels be correct about actual angels? Granted, that question will only be interesting if you think angels might actually exist. Well, I do. So it is an interesting question for me. I like to think these kinds of metaphysical speculations are on the right track. But I definitely won’t make any strong claims to that effect. As in scripture, the angels will just have to tell us themselves what they are like, if they feel so inclined.

But to be honest what got me onto this whole subject and thinking about it was just that derisive line about angels dancing on the head of a pin. I enjoy reading Medieval philosophy and I just don’t find the characterization of their thought as endless circling around useless questions to be at all accurate. Their thinking was deep and touched on the most foundational issues of knowledge and existence. And when they did talk about angels they were talking about timeless issues on the subjects of cognition, individuation, and language. So one response to the derisive line could be that it’s not really the “own” you think it is. Medieval philosophers didn’t actually ask how many angels could dance on the head of a pin. But even if they had, I’m sure it would have been interesting. 

Human Language and Artificial Neural Networks

The recent developments in AI are quite impressive. If someone had told me a couple years ago about the capabilities of something like ChatGPT I wouldn’t have believed them. AI certainly has enormous practical benefit. But since artificial neural networks were inspired by biological neural networks they can also be useful models for them. In this episode I share some recent studies investigating the behavior of the brain using AI models and evaluating their possible underlying computational similarities.

This is a follow-up to some things discussed in our last group episode on artificial intelligence. Since that conversation I’ve been digging more into the subject and wanted to share some ideas about it. I’ve been interested in artificial intelligence for a number of years. Part of that interest is because of its practical usefulness, which we’re really seeing explode now, with ChatGPT in particular. But I’m also interested in artificial intelligence as a model that could give us insights about human intelligence.

I have to say that the performance of these most recent models, like ChatGPT-3 and especially ChatGPT-4, is something that has really surprised me. If someone had told me a couple years ago that in 2022 & 2023 a deep learning model would be able to perform as well as these do I wouldn’t have believed it. I’m typically predisposed to doubt or at least be very critical about the capabilities of artificial intelligence. But in this case I think I was wrong and I’m happy to have been wrong about that. I don’t mean to swing too far to the other extreme and get too exuberant about it and overstate the capabilities of these models. But just a little bit of excess excitement might be excusable for the moment.

One claim that would be too extreme would be that these deep learning models are actually self-conscious already. Now I have no philosophical reason to suppose that an artificial device could not be self-conscious. I just don’t think we’re there yet. Another, less extreme claim, but one that would still go too far would be that deep learning models actually replicate the way human speech is produced in the brain. I think the implementations are still distinct. But that being said, I think there are enough similarities to be useful and interesting.

For comparison, there are insights we can gain into sight and hearing from cameras and audio recorders. Obviously they are not the same as our own sense organs but there are some similar principles that can help us think about how our senses work. The comparisons work both at the level of physical mechanisms and at the level of data processing. For example, I think there are some interesting insights about human senses from perceptual coding. Perceptual coding is a method used in digital signal processing that leverages the limitations and characteristics of the human sensory systems (auditory and visual) to provide data compression. For example, in audio, certain sounds are inaudible if they’re masked by louder sounds at a similar frequency. Similarly, in an image, subtle color differences in areas with high spatial detail are less noticeable than in smooth areas. Perceptual coding takes advantage of this by selectively removing the less noticeable information to reduce the data size, without significantly impacting perceived quality. This is done with MP3s and JPEGs. Extending this comparison to large language models, I’d propose that models like ChatGPT might be to human language production what cameras, JPEGs, audio recorders, and MP3s are to sight and sound. They aren’t the same but there are some parallels. ChatGPT is not the same as a human brain any more than a camera is an eye or an audio recorder is an ear. But, more modestly, ChatGPT may have some interesting similarities to human language production.

The most significant developments in this technology are so recent that the most useful reading material I’ve had to go to on the subject is peer-reviewed literature from the past year. Even there a lot of the research was done with GPT-2, which was a much less advanced model than we have available today. So it will be interesting to see what studies come out in the next year and beyond. The papers I want to focus on are 3 research papers from 2022 that present the results of experiments and 2 (just slightly older) perspective papers that offer some broad reflections and theoretical considerations.

In what follows, I’ll proceed in three parts: (1) philosophical background, (2) an overview of neural networks: biological and artificial, and (3) recent scientific literature.

Philosophical Background

Most of the philosophy I’d like to discuss is from the 20th century, in which there was considerable philosophical interest in language in what has been called the “linguistic turn”. But first something from the 18th century.

Something that stood out to me in all the research articles was the issue of interpretability. Artificial neural networks have been shown to have remarkable parallels to brain patterns in human brain production. That’s nice because the brain is so complex and little understood. The only problem is that ANNs themselves are also extremely complex and opaque to human comprehension. This challenges a notion going back to the 18th-century Italian philosopher Giambattista Vico: the Verum factum principle.

The phrase “verum factum” means “the true is the made,” which refers to the notion that truth is verified through creation or invention. In other words, we can only know with certainty that which we have created ourselves, because we understand its origins, structure, and purpose. Vico developed this principle as a critique of the Cartesian method of knowing, which, in Vico’s view, emphasized the abstract and ignored the concrete, humanistic dimensions of knowledge. By asserting that true knowledge comes from what humans create, Vico highlighted the role of human agency, creativity, and historical development in the creation of knowledge.

However, applying the verum factum principle to complex human creations like modern industrial economies, social organizations, big data, and artificial neural networks poses some interesting challenges. These creations certainly reflect human ingenuity and creativity, but they also possess a complexity that can make them difficult to fully comprehend, even for those directly involved in their creation. Artificial neural networks are inspired by our understanding of the human brain, but their function, especially in deep learning models, can be incredibly complex. It’s often said that these networks function as “black boxes,” as the pathway to a certain output given a certain input can be labyrinthine and largely inexplicable to humans, including their creators. So, while the verum factum principle encapsulates the role of human agency and creativity in the construction of knowledge, artificial neural networks illustrate that our creations can reach a level of complexity that challenges our ability to fully comprehend them.

Now turning to the 20th century I think four philosophers are especially relevant to the subject. These are Martin Heidegger, Ludwig Wittgenstein, Ferdinand de Saussure, and Hubert Dreyfus. Of these four Hubert Dreyfus was the one who most directly commented on artificial intelligence. But Dreyfus was also using ideas from Heidegger in his analysis of AI.

Let’s start with Dreyfus and Heidegger. Dreyfus’s main arguments were outlined in his influential 1972 book, What Computers Can’t Do. The core of his critique lies in what he sees as AI’s misguided reliance on formal symbolic reasoning and the assumption that all knowledge can be explicitly encoded. Dreyfus argued that human intelligence and understanding aren’t primarily about manipulating symbolic representations, as early AI research assumed. Instead, he believed that much of human knowledge is tacit, implicit, and tied to our embodied experience of “being in the world”, an important Heideggerian concept. These are aspects that computers, at least during that time, couldn’t easily replicate.

Dreyfus drew heavily on the philosophy of Martin Heidegger to make his arguments. Heidegger’s existential phenomenology, as expressed in his 1927 book Being and Time describes human existence (“Dasein”) as being-in-the-world—a complex, pre-reflective involvement with our surroundings. This contrasts with the traditional view of humans as subjects who perceive and act upon separate objects. According to Heidegger, we don’t usually encounter things in the world by intellectually representing them to ourselves; instead, we deal with them more directly.

Dreyfus related this to AI by arguing that human expertise often works in a similar way. When we become skilled at something, we don’t typically follow explicit rules or representations—we just act. This aligns with Heidegger’s notion of ‘ready-to-hand’—the way we normally deal with tools or equipment, not by observing them as separate objects (‘present-at-hand’), but by using them directly and transparently in our activities.

Another philosopher relevant to this topic is Ludwig Wittgenstein. He was one of the most influential philosophers of the 20th century. He is considered to have had two major phases that were quite different from each other.  His early work, primarily represented in Tractatus Logico-Philosophicus, proposed that language is a logical structure that represents the structure of reality. But in his later work, chiefly Philosophical Investigations, Wittgenstein advanced a very different view.

In Philosophical Investigations, Wittgenstein introduces the concept of language as a form of social activity, what he called “language games.” He argues that language does not have a single, universal function (as he had previously believed) but is instead used in many different ways for many different purposes.

Language, Wittgenstein claims, should be seen as a myriad of language games embedded in what he called ‘forms of life’, which are shared human practices or cultural activities. Different language games have different rules, and they can vary widely from commands, to questions, to descriptions, to expressions of feelings, and more. These language games are not separate from our life but constitute our life.

Wittgenstein also introduced the idea of ‘family resemblances’ to discuss the way words and concepts gain their meanings not from having one thing in common, but from a series of overlapping similarities, just like members of a family might resemble each other.

He also challenged the idea that every word needs to have a corresponding object in the world. He argued that trying to find a definitive reference for each word leads to philosophical confusions and that words acquire meaning through their use in specific language games, not through a one-to-one correspondence with objects in the world. So, for Wittgenstein, the meaning of a word is not something that is attached to it, like an object to a label. Instead, the meaning of a word is its use within the language game. This was a notion similar to a theory of language called structuralism. 

The leading figure of structuralism was the Swiss linguist Ferdinand de Saussure. His ideas laid the groundwork for much of the development in linguistics in the 20th century and provided the basis for structuralism. In his course in general linguistics, compiled from notes taken by his students, Saussure proposed a radical shift in the understanding of language. He proposed that language should be studied synchronically (as a whole system at a particular point in time) rather than diachronically (as a historical or evolutionary development). According to Saussure, language is a system of signs, each sign being a combination of a concept (the ‘signified’) and a sound-image (the ‘signifier’). Importantly, he emphasized that the relationship between the signifier and the signified is arbitrary – there is no inherent or natural reason why a particular sound-image should relate to a particular concept.

Regarding the creation of meaning, Saussure proposed that signs do not derive their meaning from a connection to a real object or idea in the world. Instead, the meaning of a sign comes from its place within the overall system of language and its differences from other signs; It’s within the structure of the language. That is, signs are defined not positively, by their content, but negatively, by their relations with other signs. For example, the word “cat” doesn’t mean what it does because of some inherent ‘cat-ness’ of the sound. Instead, it gains meaning because it’s different from “bat,” “cap,” “car,” etc. Moreover, it signifies a particular type of animal, different from a “dog” or a “rat”. Thus, a sign’s meaning is not about a direct link to a thing in the world but is about differential relations within the language system.

Saussure’s ideas about language and the generation of meaning can be interestingly compared to the techniques used in modern natural language processing (NLP) models, such as word2vec and to cosine similarity. For example, in word2vec, an algorithm developed by researchers at Google, words are understood in relation to other words.

Word2vec is a neural network model that learns to represent words as high-dimensional vectors (hence “word to vector”) based on their usage in large amounts of text data. Each word is assigned a position in a multi-dimensional space such that words used in similar contexts are positioned closer together. This spatial arrangement creates ‘semantic’ relationships: words with similar meanings are located near each other, and the differences between word vectors can capture meaningful relationships.

A measure of the similarity between two vectors is called cosine similarity. In the context of NLP, it’s often used to measure the semantic similarity between two words (or word vectors). If the word vectors are close in the multi-dimensional space (meaning the angle between them is small), their cosine similarity will be high, indicating that the words are used in similar contexts and likely have similar meanings. There are some interesting parallels between Saussure’s linguistics and AI language models. Both approaches stress that words do not have meaning in isolation but gain their meaning through their relations to other words within the system; language for Saussure and the trained model for word2vec.

Neural Networks: Biological and Artificial

Recall Hubert Dreyfus’s critique of formal symbolic reasoning in artificial intelligence and the assumption that all knowledge can be explicitly encoded. His critique is most relevant to traditional programming in which explicit program instructions are given. In machine learning, however, and in artificial neural networks program rules are developed in response to data. To whatever degree this is similar to the human mind, biology is at least the inspiration for artificial neural networks.

What is the structure of biological neural networks (BNNs)? In the brain connections between neurons are called synapses. Synapses are the tiny gaps at the junctions between neurons  in the brain where communication occurs. They play a vital role in the transmission of information in the brain. Each neuron can be connected to many others through synapses, forming a complex network of communicating cells.

Neurons communicate across the synapse using chemicals called neurotransmitters. When an electrical signal (an action potential) reaches the end of a neuron (the presynaptic neuron), it triggers the release of neurotransmitters into the synapse. These chemicals cross the synapse and bind to receptors on the receiving neuron (the postsynaptic neuron), which can result in a new electrical signal in that neuron. This is how neurons interact with each other and transmit information around the brain.

Synapses form during development and continue to form throughout life as part of learning and memory processes. The creation of new synapses is called synaptogenesis. This happens when a neuron extends a structure called an axon toward another neuron. When the axon of one neuron comes into close enough proximity with the dendrite of another neuron, a synapse can be formed.

The strength of synapses in the brain can change, a phenomenon known as synaptic plasticity. This is thought to be the basis of learning and memory. When two neurons are activated together frequently, the synapse between them can become stronger, a concept known as long-term potentiation (LTP). This is often summarized by the phrase “neurons that fire together, wire together”.

On the other hand, if two neurons aren’t activated together for a while, or the activation is uncorrelated, the synapse between them can become weaker, a process known as long-term depression (LTD).

Multiple factors contribute to these changes in synaptic strength, including the amount of neurotransmitter released, the sensitivity of the postsynaptic neuron, and structural changes such as the growth or retraction of synaptic connections. By adjusting the strength of synaptic connections, the brain can adapt to new experiences, form new memories, and continually rewire itself. This is a dynamic and ongoing process that underlies the brain’s remarkable plasticity.

How then do biological neural networks compare to artificial neural networks? In an artificial neural network, each connection between artificial neurons (also called nodes or units) has an associated weight. These weights play a role somewhat analogous to the strength of synaptic connections in a biological brain. A weight in an ANN determines the influence or importance of an input to the artificial neuron. When the network is being trained, these weights are iteratively adjusted in response to the input the network receives and the error in the network’s output. The goal of the training is to minimize this error, usually defined by a loss function.

The process of adjusting weights in an ANN is a bit like the changes in synaptic strength observed in biological neurons through processes like long-term potentiation (LTP) and long-term depression (LTD). In both cases, the changes are driven by the activity in the network (biological or artificial) and serve to improve the network’s performance – either in terms of survival and behavior for a biological organism, or in terms of prediction or classification accuracy for an ANN.

Of courses there are still multiple differences between biological neural networks and artificial neural networks. ANNs usually involve much simpler learning rules and lack many of the complex dynamics found in biological brains, such as the various types of neurons and synapses, detailed temporal dynamics, and biochemical processes. The biological synaptic plasticity is a much richer and more complex process than the adjustment of weights in an ANN. Also, in most ANNs, once training is complete, the weights remain static, while in biological brains, synaptic strength is continually adapting throughout an organism’s life. Biological and artificial neural networks share computational principles but they certainly don’t implement these computations in the same way.  Brains and computers are simply very different physical things, right down to the materials that compose them.

Artificial neural networks have been in development for several decades. But it is in very recent years that we’ve seen some especially remarkable advances, to which we’ll turn now.

Recent Scientific Literature

I’d like to share 5 papers that I’ve found useful on this subject. Three are research papers with experimental data and two are perspective papers that offer some broad reflections and theoretical considerations.

The 3 research papers are:

“Brains and algorithms partially converge in natural language processing”, published in Communications Biology in 2022 by Caucheteux & King.

“Shared computational principles for language processing in humans and deep language models”, published in Nature Neuroscience in 2022 by Goldstein et al.

“Explaining neural activity in human listeners with deep learning via natural language processing of narrative text”, published in Scientific Reports in 2022 by Russo et al.

And the 2 perspective articles are:

“Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks”, published in Neuron in 2020 by Hasson et al.

“A deep learning framework for neuroscience”, published in Nature Neuroscience in 2019 by Richards et al.

In each of the three research papers human participants read or listened to certain passages while their brain signals for specific brain regions were measured. Deep learning models were trained on this data to predict the brain signals that would result from the text. Researchers looked for instances of high correlation between actual brain patterns and the brain patterns predicted by the model and mapped where in the brain these signals occurred at various points in time before and after word onset. In particular, they noted whether the brain regions activated corresponded to those regions that would be expected from neuroscience to activate in the various stages of language processing.

In the first article, “Brains and algorithms partially converge in natural language processing”, published in Communications Biology in 2022 by Caucheteux & King the researchers used deep learning models to predict brain responses to certain sentences. Then the actual brain responses of human subjects were used as training data for the models. They used a variety of models that they classified as visual, lexical, and compositional. Then they evaluated how well these different types of models matched brain responses in different brain regions. The brain responses in the human subjects were measured using functional magnetic resonance imaging (fMRI) and magnetoencephalography (MEG).

Regarding the 3 different types of models:

Visual models are deep learning models that are primarily used for tasks involving images or videos. They are trained to recognize patterns in visual data, which can then be used to perform tasks such as image classification, object detection, image generation, and more. The most common types of visual deep learning models are convolutional neural networks (CNNs). CNNs are specifically designed to process pixel data and have their architecture inspired by the human visual cortex. 

Lexical models are models that focus on the processing of words or “lexemes” in a language. They work with individual words or groups of words (n-grams), treating them as atomic units. Lexical models can learn word representations (often called “embeddings”) that capture the semantic meanings of words, and their relationships with each other. They are often used in natural language processing (NLP) tasks such as text classification, sentiment analysis, and named entity recognition. Examples of lexical models include traditional word2vec or GloVe models, which map words into a high-dimensional vector space.

Compositional models, also called “sequential” or “recurrent” models, handle sequences of data where the order of the data points is important, such as sentences, time-series data, etc. They are designed to process one part of the sequence at a time and maintain a kind of memory (in the form of hidden states) of what has been seen so far. This allows them to capture patterns over time and use this information to make predictions about future data points in the sequence. Examples include causal language transformers (CLTs) like GPT.

Interestingly enough, the accuracy of the different types of models was observed to vary with time from the word onset. And the moments of high correlation of each model type corresponded with the activation of certain brain regions.

In early visual responses – less than 150 ms, when subjects would first see a word – brain activations were in the primary visual cortex and correlated best with activations in visual models, convolutional neural networks (CNNs).

At around 200 ms these brain activations were conveyed to the posterior fusiform gyrus. At the same time lexical models like Word2Vec started to correlate better than CNNs. This tracks with the hypothesis that the fusiform gyrus is responsible for orthographic and morphemic computations.

Around 400 ms brain activations were present in a broad fronto-temporo-parietal network that peaked in the left temporal gyrus. At this point lexical models like Word2Vec also correlated with the entire language network. These word representations were then sustained for several seconds, suggesting a widespread distribution of meaning in the brain.

Around 500-600 ms there were complex recurrent dynamics dominated by both visual and lexical representations.

After 800 ms, brain activations were present in the prefrontal, parietal, and temporal lobes. At the same time compositional models like causal language transformers (CLTs) correlated better than lexical models. The team speculated that these late responses might be due to the complexity of the sentences used in this study, potentially delaying compositional computations.

The researchers concluded from their experiment that the results show that deep learning algorithms partially converge toward brain-like solutions.

In “Shared computational principles for language processing in humans and deep language models”, published in Nature Neuroscience in 2022 by Goldstein et al. the researchers compared the responses of human participants and autoregressive deep language models (DLMs) to the text of a 30-minute podcast.

The authors note that human language has traditionally been explained by psycholinguistic approaches using interpretable models that combine symbolic elements, such as nouns, verbs, adjectives, and adverbs, with rule-based operations. This is similar to the kind of traditional programming that Hubert Dreyfus argued would not be viable for AI. In contrast, autoregressive Deep Language Models (DLMs) learn language from real-world textual examples, with minimal or no explicit prior knowledge about language structure. They do not parse words into parts of speech or apply explicit syntactic transformations. Instead, these models learn to encode a sequence of words into a numerical vector, termed a contextual embedding, from which the model decodes the next word. Autoregressive DLMs, such as GPT-2, have demonstrated effectiveness in capturing the structure of language. But the open question is whether the core computational principles of these models relate to how the human brain processes language. The authors present their experimental findings as evidence that human brains process incoming speech in a manner similar to an autoregressive DLM.

In the first experimental setup, participants proceeded word by word through a 30-minute transcribed podcast, providing a prediction of each upcoming word. Both the human participants and GPT-2 were able to predict words well above chance. And there was high overlap in the accuracy of the predictions of human subjects and GPT-2 for individual words, i.e. words that human subjects predicted well GPT-2 also predicted well. This experiment was determined to demonstrate that listeners can accurately predict upcoming words when explicitly instructed to do so, and that human predictions and autoregressive DLM predictions are matched in this context. Next the researchers wanted to determine if the human brain, like an autoregressive DLM, is continuously engaged in spontaneous next-word prediction without such explicit instruction. And whether neural signals actually contain information about the words being predicted.

In the next experimental setup, the researchers used electrocorticography (ECoG) to measure neural responses of human participants before and after word-onset. Subjects engaged in free listening, without being given any explicit instruction to predict upcoming words. The goal was to see if our brains engage in such prediction all the time as simply a natural part of language comprehension.

The results from human subjects in this experiment were also compared to models. The first model used was a static word embedding model, GloVe. The model was used to localize electrodes containing reliable responses to single words in the narrative. The words were aligned with neural signals and then the model would be trained to predict neural signals from word embeddings. A series of coefficients corresponding to features of the word embedding was learned using linear regression to predict the neural signal across words from the assigned embeddings. “The model was evaluated by computing the correlation between the reconstructed signal and the actual signal” for the word.

In the results of this experiment there was indeed found to be a neural signal before word onset. But what the model also enabled the researchers to do was ascertain some kind of semantic content from that signal, since the model had been trained to predict certain neural signals for given words. What was observed was that “the neural responses before word onset contained information about human predictions regarding the identity of the next word. Crucially, the encoding was high for both correct and incorrect predictions. This demonstrated that pre-word-onset neural activity contains information about what listeners actually predicted, irrespective of what they subsequently perceived.” Of course, sometimes the subject’s predictions were wrong. So what happened in those cases? “The neural responses after word onset contained information about the words that were actually perceived.” So “the encoding before word onset was aligned with the content of the predicted words” and “ the encoding after word onset was aligned with the content of the perceived words.” This all aligns with what we would expect under a predictive processing (PP) model of the brain.

The next level of analysis was to replace the static embedding model (GloVe) with a contextual embedding model (GPT-2) to determine if this would improve the ability to predict the neural signals to each word. It did; an indication that contextual embedding is a closer approximation to the computational principles underlying human language. And the improved correlation from contextual embedding was found to be localized to specific brain regions. “Encoding based on contextual embeddings resulted in statistically significant correlations” in electrodes that “were not significantly predicted by static embedding. The additional electrodes revealed by contextual embedding were mainly located in higher-order language areas with long processing timescales along the inferior frontal gyrus, temporal pole, posterior superior temporal gyrus, parietal lobe and angular gyrus.” The authors concluded from this that “the brain is coding for the semantic relationship among words contained in static embeddings while also being tuned to the unique contextual relationship between the specific word and the preceding words in the sequence.”

The authors submit that DLMs provide a new modeling framework that drastically departs from classical psycholinguistic models. They are not designed to learn a concise set of interpretable syntactic rules to be implemented in novel situations, nor do they rely on part of speech concepts or other linguistic terms. Instead, they learn from surface-level linguistic behavior to predict and generate the contextually appropriate linguistic outputs. And they propose that their experiments provide compelling behavioral and neural evidence for shared computational principles between the way the human brain and autoregressive DLMs process natural language.

In “Explaining neural activity in human listeners with deep learning via natural language processing of narrative text”, published in Scientific Reports in 2022 by Russo et al. human participants listened to a short story, both forward and backward. Their brain responses were measured by functional MRI. Text versions of the same story were tokenized and submitted to GPT-2. Both the brain signal data and GPT-2 outputs were fed into a general linear model to encode the fMRI signals.

The 2 outputs researchers looked at from GPT-2 were surprisal and saliency. Surprisal is a measure of the information content associated with an event, in terms of its unexpectedness or rarity. The more unlikely an event, the higher its surprisal. It is defined mathematically as the negative logarithm of the probability of the event. Saliency refers to the quality by which an object stands out relative to its neighbors. In a text it’s the importance or prominence of certain words, phrases, or topics, a measure of how much a particular text element stands out relative to others in the same context.

What they found in their results was that the surprisal from GPT-2 correlated with the neural signals in the superior and middle temporal gyri, in the anterior and posterior cingulate cortices, and in the left prefrontal cortex. Saliency from GPT-2 correlated with the neural signals for longer segments in the left superior and middle temporal gyri.

The authors proposed that their results corroborated the idea that word-level prediction is accurately indexed by the surprisal metric and that the neural activation observed from the saliency scores suggests the co-occurrence of a weighing mechanism operating on the context words. This was something previously hypothesized as necessary to language comprehension.

The involvement of areas in the middle and the superior temporal gyrus aligns with previous studies supporting that core aspects of language comprehension, such as maintaining intermediate representations active in working memory and predicting upcoming words, do not necessarily engage areas in the executive control network but are instead performed by language-selective brain areas that, in this case, are the ones relatively early in the processing hierarchy.

I found the following comment in the discussion section of the paper quite interesting: “In general, considering that the architecture of artificial neural networks was originally inspired by the same principles of biological neural networks, it might be not at all surprising that some specific dynamics observed in the former are somehow reflected in the functioning of the latter.” I think that’s an interesting point. The whole idea of artificial neural networks came from biological neural networks. We were basically trying to do something similar to what neurons do. We don’t know exhaustively how biological neural networks work but we do know that they work very well. When we are finally able to make artificial networks that work quite well it’s perhaps to be expected that they would have similar characteristics as biological neural networks.

The other two papers were perspective papers. These didn’t present the results of experiments but discussed what I thought were some interesting ideas relating to the whole interchange between language processing in the human brain and in deep learning models.

In “Direct Fit to Nature: An Evolutionary Perspective on Biological and Artificial Neural Networks”, published in Neuron in 2020 by Hasson et al. the authors covered several topics. One thing they addressed that I found interesting was a challenge to three basic assumptions of cognitive psychology. These assumptions are:

1. The brain’s computational resources are limited and the underlying neural code must be optimized for particular functions. They attribute this to Noam Chomsky and Jerry Fodor.

2. The brain’s inputs are ambiguous and too impoverished for learning without built-in knowledge. They attribute this to Noam Chomsky.

3. Shallow, externally supervised and self-supervised methods are not sufficient for learning. They attribute this to Steven Pinker.

In response to the first assumption the authors argue that the brain’s computational resources are actually not scarce. “Each cubic millimeter of cortex contains hundreds of thousands of neurons with millions of adjustable synaptic weights, and BNNs utilize complex circuit motifs hierarchically organized across many poorly understood cortical areas. Thus, relative to BNNs, ANNs are simplistic and minuscule.” Artificial neural networks are indeed trained on huge amounts of data. GPT-4 is essentially trained on the whole internet. Human children don’t learn to talk by reading the whole internet; that’s true. But the human brain is also a lot more complex than even the most sophisticated artificial neural networks; so far at least. So if GPT-4 is able to perform so impressively with a structure that’s less sophisticated than the human brain we can expect that the human brain’s computational resources are hardly scarce.

In response to the second assumption the authors argue that the brain’s input is not impoverished. Noam Chomsky, arguably the most important linguist of the 20th century, argued for what he called the “poverty of the stimulus,” meaning that the linguistic input children receive is often incomplete, ungrammatical, or otherwise imperfect. But they still manage to learn their native language effectively. How? Chomsky proposed that there is a “Language Acquisition Device” (LAD) within the human brain. This hypothetical module is thought to be equipped with knowledge of a “Universal Grammar,” which encapsulates the structural rules common to all human languages. But Hasson et al. argue that there is no poverty of the stimulus because deep learning models can produce direct fit with reliable interpretations using dense and broad sampling for the parameter space. The model is casting a very wide net. They state: “One of our main insights is that dense sampling changes the nature of the problem and exposes the power of direct-fit interpolation-based learning… The unexpected power of ANNs to discover unintuitive structure in the world suggests that our attempts to intuitively quantify the statistical structure in the world may fall short. How confident are we that multimodal inputs are in fact not so rich?” By the way, I was sharing a draft of this with a friend who shared another recent paper with me by UC Berkeley professor Steven Piantadosi, titled “Modern language models refute Chomsky’s approach to language”. I’m not going to get into that now but just thought I’d mention it.

In response to the third assumption the authors argue that shallow self-supervision and external-supervision are sufficient for learning. The authors cite Pinker’s book The Language Instinct: How the Mind Creates Language as an example of the view that they are challenging. Pinker’s views are very similar to Chomsky’s. Pinker argues that language learning is not just about imitation or conditioning. Instead, he believes that the human brain has an inherent structure for understanding language, which is why children are able to learn languages so rapidly and effortlessly, often making grammatical leaps that aren’t explicitly taught or present in their environment. But Hasson et al. argue that humans have a great deal of external supervision from our environment, both social and physical. They refer to the importance of embodiment to predictive processing, referring to the ideas of Andy Clark and Karl Friston, among others.

Another subject the authors address is the issue of interpretability. This goes back to the Verum factum principle from Vico. Scientific models, including those in neuroscience, are often evaluated based on two desirable features: (1) interpretability and (2) generalization. We want explanations to have good predictive power but we also want to be able to understand them. Not just verify that they work. If it’s an equation we like to be able to look at an equation and be able to intuit how it works. And this means that the equation can’t be too long or have too many parameters. However, interpretability and generalization are often in conflict. Models with good interpretability may have strong explanatory appeal but poor predictive power, and vice versa.

The authors suggest that the brain is an exceptionally over-parameterized modeling organ. Interpretability in the brain is intractable for the same reason interpretability of deep learning models is intractable. They work with a huge number of parameters. There’s quantification occurring but it’s not like a concise equation that you can look at in grasp intellectually. The authors propose that neural computation relies on brute-force direct fitting, which uses over-parameterized optimization algorithms to enhance predictive power, i.e. generalization, without explicitly modeling the underlying generative structure of the world.

One thing that’s really nice about this paper (and I highly recommend it by the way, it’s a delightful read) is its 3 “boxes” that touch on some key concepts. One box covers the biomimicry of biological neural networks by artificial neural networks. The authors state that artificial neural networks (ANNs) are learning models that draw inspiration from the biological neural networks (BNNs) present in living brains, but that ANNs are a highly abstracted version of BNNs. Some biological nervous systems include functional specialized system-level components like the hippocampus, striatum, thalamus, and hypothalamus, elements not included in contemporary ANNs. ANNs are also disembodied and do not closely interact with the environment in a closed-loop manner. While the authors concede that ANNs are indeed highly simplified models of BNNs, they propose that there exist some essential similarities: they both belong to the same group of over-parameterized, direct-fit models that depend on dense sampling for learning task-relevant structures in data. And, crucially, ANNs are currently the only models that achieve human-like behavioral performance in many domains and can offer unanticipated insights into both the strengths and limitations of the direct-fit approach. Like BNNs, ANNs are founded on a collection of connected nodes known as artificial neurons or units that loosely resemble neurons in a biological nervous system. Each connection, akin to synapses in BNNs, links one artificial neuron to another, and the strength of these connections can be adjusted through learning. The connections between artificial neurons have weights that are adjusted during the learning process based on supervised feedback or reward signals. The weight amplifies or reduces the strength of a connection. And much like BNNs, ANNs are sometimes organized into layers.

Another “box” addresses embodiment. This is something the philosopher Andy Clark has addressed a lot in his work. Not to mention, going further back, the philosopher Maruice Merleau-Ponty. At present, ANNs are disembodied and unable to actively sample or modify their world. The brain does not operate with strictly defined training and test regimes as found in machine learning. Objective functions in BNNs must satisfy certain body-imposed constraints to behave adaptively when interacting with the world. The authors suggest that adding a body to current ANNs, capable of actively sampling and interacting with the world, along with ways to directly interact with other networks, could increase the network’s learning capacity and reduce the gaps between BNNs and ANNs. Interestingly enough, they cite Wittgenstein’s “Philosophical Investigations” when addressing the way social others direct our learning processes.

One other topic in the paper that I found interesting was a discussion of “System 1” and “System 2”. This model was made most famous by Daniel Kahneman in his 2011 book Thinking Fast and Slow. The authors cite Jonathan St B. T. Evans’s 1984 paper “Heuristic and analytic processes in reasoning”. And there are earlier precedents for the general idea going back further in history. System 1 represents fast, automatic, and intuitive thinking, what Evans called heuristic processes. And System 2 represents slow, effortful, and deliberate thinking, what Evans called analytic processes. Hasson et al. propose that we can understand System 1 to be a kind of substrate from which System 2 can arise. System 2 is where things get really interesting. That’s where we find some of the most impressive capacities of the human mind. But they maintain that we have to start with System 1 and build from there. They state: “Although the human mind inspires us to touch the stars, it is grounded in the mindless billions of direct-fit parameters of System 1.” They see artificial neural networks as having the most relevance toward explaining System 1 processes. And the thing is we seem to be continually finding that System 1 includes more than we might have thought. “Every day, new ANN architectures are developed using direct-fit procedures to learn and perform more complex cognitive functions, such as driving, translating languages, learning calculus, or making a restaurant reservation–functions that were historically assumed to be under the jurisdiction of System 2.” 

In “A deep learning framework for neuroscience”, published in Nature Neuroscience in 2019 by Richards et al. the authors focus on three key features of artificial neural network design – (1) objective functions, (2) learning rules, and (3) architectures – and address how these design components can impact neuroscience.

The authors observe that when the traditional framework for systems neuroscience was formulated, they could only collect data from a small selection of neurons. Under this framework, a scientist observes neural activity, formulates a theory of what individual neurons compute, and then constructs a circuit-level theory of how these neurons integrate their operations. However, the question arises as to whether this traditional framework can scale up to accommodate recordings from thousands of neurons and all of the behaviors that one might want to explain. It’s arguable that the classical approach hasn’t seen as much success when applied to large neural circuits that perform a variety of functions, such as the neocortex or hippocampus. These limitations of the classical framework suggest that new methodologies are necessary to capitalize on experimental advancements.

At their fundamental level, ANNs model neural computation using simplified units that loosely emulate the integration and activation properties of real neurons. The specific computations performed by ANNs are not designed but learned. When setting up ANNs, scientists don’t shape the specific computations performed by the network. Instead, they establish the three components mentioned previously: objective functions, learning rules, and architecture. Objective functions measure the network’s performance on a task, and learning involves finding synaptic weights that maximize or minimize this objective function. These are often referred to as ‘loss’ or ‘cost’ functions. Learning rules offer a guide for updating the synaptic weights. And architectures dictate the arrangement of units in the network and determine the flow of information, as well as the computations the network can or cannot learn.

Richards et al. make an observation about interpretability similar to that made by Hasson et al. The computations that emerge in large-scale ANNs trained on high-dimensional datasets can be hard to interpret. An ANN can be constructed with a few lines of code, and for each unit in an ANN, the equations determining their responses to stimuli or relationships to behavior can be specified. But after training, a network is characterized by millions of weights that collectively encode what the network has learned, and it is difficult to envision how such a system could be described with only a few parameters, let alone in words. They suggest that we think about this in the following way. Theories can have a compact explanation that can be expressed in relatively few words that can then be used to develop more complex, non-compact models. They give the theory of evolution by natural selection as a comparative example. The underlying principle is fairly simple and comprehensible, even if the actual mechanics that emerge from it are very complex. For systems neuroscience we can start with these three relatively simple and comprehensible principles: objective functions, learning rules, and architecture. Then even though the system that emerges from that is too complex to comprehend at least the underlying principles are comprehensible and give some degree of intuitive understanding.

Conclusion

Something that I find exciting about all this is that it’s an interesting interface between philosophy of mind, neuroscience, and programming. I think that some of the most interesting problems out there are philosophical problems. Even many scientific problems transition into philosophical problems eventually. But our philosophy needs periodic grounding in the world of empirical observations. What we might call armchair philosophy runs the danger of getting untethered from reality. In the philosophy of mind we can speculate about a lot of things that don’t work out very well in neuroscience. That’s not to say that philosophy of mind has to be entirely bounded by neuroscience. Just because human minds work in a certain way doesn’t mean that minds of any kind would have to be constrained in the same way. There could be many different ways for minds to work. But if we’re theorizing about ways other types of minds might work we don’t, at present, have ways to verify that they actually would work. With theories about human minds we can at least try to verify them. Even that’s kind of challenging though because the brain is so complex and difficult to observe directly at high resolution.

Still, there’s a lot about our brains that we do know that we can take into account in our theories of the mind. We know that our brains have neurons and that neurons make synaptic connections. And we know that those synaptic connections can strengthen or weaken. We can at least account for that in our theories. Artificial neural networks patterned after biological neural networks are useful tools to model our brains. We can’t go into every synaptic cleft in the brain to sample its flux of neurotransmitters. Or record the firing frequency of every neuron in the brain. That would be great but we just don’t have that capability. With artificial neural networks, as imperfect approximations as they are, we at least have recorded information for billions of parameters, even if their sheer quantity defies comprehension. And we can try out different configurations to see how well they work.

Another subtopic that’s interested me for a while is the possibility of what I call a general theory of mind. “General” in the sense of applying beyond just the special case of human minds, a theory of the human mind being a “special” theory of mind. What other kinds of minds might there be? What are all the different ways that a mind can work? AI might give us the ability to simulate and test more general and exotic possibilities and to extract the general principles they all hold in common.

I think the recent success of these large language models is quite exciting. Maybe a little bit frightening. But I’m mostly excited to see what we can learn.

The AI Revolution

Jared, Mike, and Todd talk about ChatGPT and the AI revolution, particularly the remarkable developments of the last year and 6 months (2022-2023). We talk briefly about how neural networks and large language models work. Then we get into the possible social, economic, aesthetic, and even existential implications for humanity. We talk about the possibility of this being a major positive breakthrough of the first order, with revolutionary implications for human prosperity and standard of living. But we also talk about the risks, ranging from an eclipsing of human distinctiveness and creativity to a threat to humanity’s very survival. Overall, we have a hunch that something big is happening and we want to talk about it!