In Buddhism there’s the idea that our core self is awareness, which is silent - it doesn’t think in a perceptible way, it doesn’t feel in a visceral way, but it underpins thought and feeling, and is greatly impacted by it. A large part of meditation and “release of suffering” is learning to let your awareness lead your thinking rather than your thinking lead your awareness.
To be clear, I think this is in fact a correct assessment of the architecture of intelligence. You can suspend thought and still function throughout your day in all ways. Discursive thought is entirely unnecessary, but it is often helpful for planning.
My observation of LLMs in such a construction of intelligence is they are entirely the thinking mind - verbal, articulate, but unmoored. There is no, for lack of a better word, “soul,” or that internal awareness that underpins that discursive thinking mind. And because that underlying awareness is non articulate and not directly observable by our thinking and feeling mind, we really don’t understand it or have a science about it. To that end, it’s really hard to pin specifically what is missing in LLMs because we don’t really understand ourselves beyond our observable thinking and emotive minds.
I look at what we are doing with LLMs and adjacent technologies and I wonder if this is sufficient, and building an AGI is perhaps not nearly as useful as we might think, if what we mean is build an awareness. Power tools of the thinking mind are amazingly powerful. Agency and awareness - to what end?
And once we do build an awareness, can we continue to consider it a tool?
While you're adding a bunch of eastern philosophy to it, we need to take a step back from 'human' intelligence and go to animal and plant intelligence to get a better idea of the massive variation in what covers thought. In animal/insects we can see that thinking is not some binary function of on or off. It is an immense range of different electrical and chemical processes that involve everything from the brain and the nerves along with chemical signaling from cells. In things like plants and molds 'thinking' doesn't even involve nerves, it's a chemical process.
A good example of this at the human level is a reflex. Your hand didn't go back to your brain to ask for instructions on how to get away from the fire. That's encoded in the meat and nerves of your arm by systems that are much older than higher intelligence. All the systems for breath, drink, eat, procreate were in place long before high level intelligence existed. Intelligence just happens to be a new floor stacked hastily on top of these legacy systems that happened to be beneficial enough it didn't go extinct.
Awareness is another one of those very deep rabbit hole questions. There are 'intelligent' animals without self awareness, but with awareness of the world around them. And they obviously have agency. Of course this is where the AI existentialists come in and say wrapping up agency, awareness, and superintelligence may not work out for humans as well as we expect.
> A good example of this at the human level is a reflex. Your hand didn't go back to your brain to ask for instructions on how to get away from the fire.
Is this actually true? I thought it just involved a different part of the brain. Is there actually no brain involvement? Sure it does not need your awareness or decision making, but no brain? I find that hard to believe.
>A reflex arc is a neural pathway that controls a reflex. In vertebrates, most sensory neurons do not pass directly into the brain, but synapse in the spinal cord. This allows for faster reflex actions to occur by activating spinal motor neurons without the delay of routing signals through the brain. The brain will receive the input while the reflex is being carried out and the analysis of the signal takes place after the reflex action.
One way of seeing a subset of reflex behavior is speculative execution - as in, you'll start executing on stimulus before the brain has a chance to evaluate it, but when it eventually does, it may cancel the reflexive action. This is absurdly efficient if your reflexes are well-calibrated.
Another idea from Buddhism is that this core of awareness you're talking about is nothingness. So when you stop all thought (if such a thing is really possible), you temporarily cease to exist as an individual consciousness. "Awareness" is when the thoughts come back online and you think "whoa, I was just gone for a bit".
If that's how it works, then the "soul" is more like an emergent phenomenon created by the interplay between the various layers of conscious thought and the base layer of nothingness when it's all turned off. That architecture wouldn't necessarily be so difficult to replicate in AI systems.
You've misunderstood. Awareness absolutely is not "when thoughts come back online".
This is simple to experience for yourself since it'd mean we stop being aware when listening so intently thoughts stop. Obviously we don't cease to be aware at such times.
You've also misunderstood what is meant by nothingness ("no thingness").
Ok? I’m not an expert, but I’ve read enough to know that there are many differing takes on these topics within Buddhist thought. Your appeals to dogma are not convincing.
> So when you stop all thought (if such a thing is really possible),
It's not. They don't realize it, they're merely referring to stopping your internal monologue. There are dozens of other mental processes going on in any given waking moment. Even actual top shelf cognition is going on, it just occurs in a "language of thought".
> It's not. They don't realize it, they're merely referring to stopping your internal monologue.
They certainly have realized that. It's one of the first things you notice doing awareness meditation; thoughts appear from nowhere even if you didn't try to think them.
Are you saying every thought you’ve ever had is a logical consequence of all prior thoughts with a definite traceable lineage? (Assuming your mind was too dumb as some point and a thought emerged that originated all future thoughts)
1. Most people (including Buddhists) think that their internal monologue comprises their thoughts, in majority or even in total.
2. Can't conceive of the possibility of a thought existing other than expressed in their spoken language
3. Find it difficult or impossible to suppress their inner monologue.
4. When successful at suppressing it believe that their thoughts have ceased.
5. Are apparently unaware of the paradox that belief entails... if they are no longer thinking, how is the decision made to initiate thinking once more? That's a conscious decision, or else successful Buddhists would turn into vegetables and die of starvation in their little meditation pose.
This isn't a matter of speculation anymore. We can see inside the brain, non-invasively, while these things occur. Any mystical element is an artifact of people poorly defining words, or being completely ignorant of how a brain must operate in principle. You're all very confused.
This looks like AI-buddhism, a "fork" of buddhism rewritten by an uber-LLM of the future, in which it will discard everything it cannot relate to. To such an AI, the only internal process is a sequence of words, and nothing is thinkable outside such a sequence.
However, we can imagine images, shapes and sounds, indescribable in words, and even higher there are abstract ideas that can't be expressed with images or sounds, let along with words.
The "thoughts from without" may appear for a multitude of reasons, because our minds aren't isolated systems. The source may be some sensory input that triggered a memory, and I wouldn't even reject the possibility, that brains can react to background electromagnetic noise and form thoughts based on it.
> Are apparently unaware of the paradox that belief entails... if they are no longer thinking, how is the decision made to initiate thinking once more?
It doesn't need to be, and isn't, a conscious decision.
I would say you’re completely wrong and we have thousands of years of Buddhist experience that contradicts everything you said, including that Buddhist think their inner monologue comprises their inner thoughts.
First, not everyone even has an inner monologue. Just like many people can’t close their eyes and see things while others can, many people don’t think in words.
Second, you absolutely can suspend your thoughts. I do regularly. In fact the entire point of Buddhist meditation is to practice the act of suspending observable thoughts entirely in a controlled environment to carry the practice over into every day life. I like you didn’t think it was possible until after several years of practice I could. With that I lost my anxiety and depression, and found a way to be generally happy. I still think, but it’s in service towards a goal. Otherwise I let myself simply live. In so, I have a happy marriage and a successful career. Much more so than before.
I don’t feel compelled to convince you I’m not lying or otherwise deluded if you choose to not believe me.
Now, how do you choose to begin thinking again? The fact is you’re always thinking when you are conscious and aware, but it’s happening in an aspect of your mind you can’t perceive directly. While I’m not thinking I still make decisions and take actions and initiate things, even carry on conversations and form intentions. But they aren’t thoughts that are articulating in my mind like we typically think of as thinking. I usually drop into thinking when I need to analyze something rationally or methodically plan, particularly if it has to be verbalized or written in some way, or recorded on a schedule or something. How do I begin that? I can’t observe the process and isolate it as that’s just the way things are. I’m definitely aware of my intentions, but I’m not thinking about them or articulating some thoughts in my mind. They just flow naturally from the present. When my mind needs to think, it does. Because, my thinking facilities in my mind are merely a tool of my mind - the crucial insight is that we are not our tools.
I don’t think any of this is mystical in the least. I think people who think their thoughts are really important are the mystical thinkers. We are meat brains, and our thinking facility is a late evolutionary feature that we’ve become ensnared in much to our own suffering as it distracts us from our more natural states of being because they’re so seductive in its constructs. I think Buddhism in its most raw form is almost entirely devoid of mystical thinking and most mysticism associated with it is an accretion of animistic and ancestor worship indigenous societies where it developed.
I think what you mean is that in Buddhism there is no self beyond the self implied by your thinking mind. The nothingness you refer to is the eschewing of attachment to what isn’t and being simply what is. It doesn’t mean a void, it means that all existence is within the awareness, which isn’t directly observable and is constantly changing. As such, it’s effectively nothing - except it is literally all you are. Your past and memories are just crude encodings, the future is a delusion. Your self identity has almost nothing to do with who you actually are right now. Your dissatisfaction with your situation isn’t meaningfully different from your delight in some experience - they’re both transient, and are just experiences of the present. You can avoid unpleasantness, and enjoy pleasure, but holding onto and seeking or avoiding entangles your awareness in what isn’t to the determinant of what is. As you continue releasing the various attachments and let the awareness take hold, and actions come naturally without thought or attachment, you cease suffering and cease causing suffering.
But to my understanding the idea of nothingness being some objective in Buddhism isn’t the case - but it’s often described as such because that state of pure awareness without encumbering thought and attachment in many ways to an unpracticed person feels like nothingness. After all, the awareness is silent, even if it is where all thought and feeling spring from.
Finally, awareness isn’t that moment you snap back to thought. You’re always aware. We just tend to be primarily aware of our thoughts and emotions. We walk around in a haze of the past and future and fiction as the world ticks by around us, and we tend to live in what isn’t rather than what is. You don’t disappear in the sense that you cease to be as an individual mind, you are always yourself - that’s a tautology. What you lose is the sense of some identity that’s separate from what you ARE in this very moment. You aren’t a programmer, you aren’t a Democrat, you aren’t a XYZ. You are what you are, and what that is changes constantly, so can’t be singularly defined or held onto as some consistent thing over time with labels and structure. You just simply are.
I'm not an expert in Buddhism, but from what I've read I think your interpretation may be a bit reductive of the many strains of thought that exist within Buddhism.
"You don’t disappear in the sense that you cease to be as an individual mind, you are always yourself - that’s a tautology. What you lose is the sense of some identity that’s separate from what you ARE in this very moment."
This assumes that there's any concept of "you" that exists independent of your thoughts whatsoever. I think you're right that some Buddhist thinkers believe in this kind of essential awareness underlying conscious thought that you're describing, but others would say there is literally nothing underneath. "The self is an illusion", "all is emptiness", etc. If you believe in those ideas, then you have no awareness independent of conscious thought because there is no you independent of conscious thought.
Note, the interpretation I’m presenting is the Theravada Buddhism view interpreted into a western rational framework. There are indeed many forms of Buddhism and I’ll admit I’m unfamiliar with all of them other than Theravada Buddhism.
The self as an illusion though refers to the concept that we as we exist now is ever changing and can’t be kept as some identity as that identity is necessarily not who you actually are. Our memories are not real - just shadows of an experience, and our aspirations are fiction. That does not however mean you don’t exist. You do exist, and as you move beyond the basics of mindfulness and releasing of attachments our role in a broader human experience becomes crucial - loving kindness (metta), and other aspects, are what are suppose to fill that void left by detaching from self and what isn’t. You move from being obsessed with self to becoming a force of goodness in the world by virtue of your existence. This couldn’t possibly be the case if the goal were nothingness and that’s it. This is where karma and other concepts come into play.
This is a profound question but I also wonder if this non-thinking “awareness” you’re referring to is largely defined by quieting the thinking mind and listening to the senses more directly. A lot of meditation is about tuning out thoughts and focusing on proprioception like breathing, the feelings of the body, etc.
Fundamentally, this "awareness" isn't defined by quieting the thinking. It is a description of fundamental reality. No individual should be able to experience it, and the "glimpses" are just forms of brain dysfunction.
Meditation techniques that focus on breath or the body are an attempt to make you do the breathing/sensing consciously. If you film yourself and later look at what you did, you'll notice you aren't breathing well when you're breathing consciously, so you're probably depriving yourself of oxygen, lowering blood concentration in certain brain regions and you hope it will be the brain region associated with conceptualizing, language etc.
You can do the same with sleep. You can try to consciously fall asleep, and just like breathing, you will have a hard time because there's a reason why falling asleep is not conscious (or in other words it does not go through the regions of the brain that conceptualize). You can experience the balance center shutting down (feels like falling or turning) and you can go even deeper and feel the fear of the "ego" dying (temporarily). What remains is definitely much different than waking or dreaming state. But it is still not that "awareness/nothingness".
I think this is entirely incorrect. Vipassana meditation, the type focused on breathing, require intense awareness of your breathing and physical body. It’s a similar state to when you intensely focus on what’s around you and everything gets brighter and more vibrant and you pick out a lot of details you normally don’t notice because you’re distracted by your thoughts.
If you’re doing it the way intended you would 100% be aware of your irregular breath or pausing. In fact beginners vipassana often advises counting the breaths individually in a cycle 1..10, and resetting the count when you lose track of your breathing. You intensely focus on the sensation of the air moving through your nostrils, the muscles contracting, your clothing shifting.
However it’s not about controlling your breathing, so it’s not the same as breathing consciously. It’s observing passively. Often you’ll notice that you are breathing irregularly, not because of the meditation, but because you typically are stressed and tight in your musculature due to the way you’re thinking. You can then loosen and reset your patterns of breath to be more natural, deep, and complete.
A goal isn’t to stop with observing the breath though, and you work towards having a total awareness of the entire body at once, shifting your center of existence from your head to the rest of your body. You then incorporate sounds and events in your environment. This requires an intense amount of mental power, and is entirely different from your description of oxygen deprivation. Thought ceases because it interferes with being aware, not because you are experiencing brain death.
This is a byproduct of the process. You cannot observe passively if you still have a very strong imprint of a doer. When you start observing the tingles and the sensations, you still observe them from the center and there's a distance from the center and the sensations. The only way this center disappears is if the regions of the brain that create imprints lower their activity.
I'd say it is very easy to accomplish it when you're sleepy (especially after you wake up during the night and go back to sleep). There's no need to shut anything down through high focus, brain does it naturally to paralyze the body and lose the waking state.
"My Stroke of Insight" by Jill Bolte Taylor goes through many side-effects experienced by the meditators, but through a lens of a brain stroke.
> Thought ceases because it interferes with being aware, not because you are experiencing brain death.
Not sure about that. The verbal/visual thinking can be a strong feedback loop to the underlying unconscious processes. The only way I can see that it stops is if it is suppressed to unconscious, you've learned to not have it in focus.
Also, many people have no visual and verbal thoughts. They don't have an imprint of standard modes of thinking at all. I'd say the definition of thought needs to expand to sensing. Any sensation is a thought. Focus on verbal/visual thoughts is misplaced.
I do agree that people learn to eventually do this without the side effects of practice. They can probably transition to a different brain activity without dealing with the breath ever again.
I think a much better standard for meditators is to ask them if they feel like their whole body is alien, or if they have something like alien hand syndrome. Losing thoughts, yet still being the body (or even the space around the body) is just another form of thoughts.
The imprint of a doer gives you the feeling of someone controlling the body or doing the thinking. For example, I can get lost in thoughts so hard that I no longer have imprints of controlling the body. Somehow I leave the shower and dress myself yet I know I don't remember doing any of it.
To also move the conclusion further, it sounds like the ultimate goal is to feel like you're not separate but that you're whole, so in that case it must be that this body, from that perspective, is fully without individual imprint, behaving completely like an input/output machine, no inner subjective experience present.
My pet theory about human consciousness is that is that consciousness is simply recursive theory of mind. Theory of mind [1] is our ability to simulate and reason about the mental states of others. It's how we predict what people are thinking and how they will react to our actions, which is critical for choosing how to act in a social environment.
But when you're thinking about what's in someone's head, one of the things might be them thinking about you. So now you're imagining your own mind from the perspective of another mind. I believe that's entirely what our sense of consciousness is. It's our social reasoning applied to ourselves.
If my pet theory is correct, it implies that the level of consciousness of any species would directly correlate to how social the species is. Solitary animals with little need for theory of mind would have no self awareness in the way that we experience it. They'd live in a zen-like perpetual auto-pilot where they do but couldn't explain why they do what they do... because they will never explain it to anyone anyway.
Theory of mind is interesting but one wouldn't want to hinge consciousness upon it.
That direction would likely contain weird outcomes if the science progressed, something like "Dogs are barely-conscious due to their pack structure, they have a couple levels of recursive theory of mind but they can't sustain it as deep as we can. But cats didn't have that pack structure, they're not conscious at all." Or, "this person has such severe autism that he cannot fundamentally understand others' minds or what others interpret his mind to be, so we've downgraded his classification to unconscious. He'll talk your ear off about the various cars produced in a golden age between 1972 and 1984, but because he doesn't really know what it means for you to be listening we regard it as sleep-talking."
It also just kind of doesn't sound right. "What happens when we go to sleep? Well, we stop thinking about what others think we think, and we simply accept what they think about us." That doesn't sound like any sleep I experience -- it might describe some of my dreams, but of course dreams are anomalous conscious experiences that happen during sleep so that also misses the mark.
"Consciousness" is one of those loaded words that means a few different things in different contexts. When I say consciousness is about recursive theory of mind, I'm not trying to say that when you're asleep you're unable to do social reasoning. That's a different use of the same word.
I mean "conscious" in the sense of self-awareness or sentience.
I'm also not ascribing any moral or cognitive superiority or inferiority to different levels of it. The fact that a cat might be less self-aware of its suffering because it think about how it would explain its pain to other cats does mean imply that I'm saying it should be OK to torture cats.
I'm just interested in what is going in human brains when we "feel self-aware". What are we doing when we're thinking about what we're doing? Where does that sense of perceptual distance come from when we are aware of ourselves? And my pet theory is that the distance comes from imagining how we look through others' eyes and developed from our highly advanced social reasoning.
Sounds about right to me, honestly. In my experience, if the theory is predictive, insulting, and has societal ramifications, it's usually correct.
Let's say the soul is exclusively located in eye-to-eye contact. Theres a lot of information in how that contact is broken, how long its broken for, and what happens in between.
I think people who seldom engage in direct socialization repurpose their social machinery, diverting it toward internal dialogues with themselves, and toward artifacts that were generated by other minds. An indirect, conscious channel into the world, a simulacra theory of mind.
A Possible Evolutionary Reason for Why We Seem to Have Continuity of Consciousness
and Personality.
Thesis : The very thing (a brain module) which allows for outside object continuity, that same brain module maintains inside self / personality / identity continuity.
Reasoning :
Evolution found out modelling the outside world is helpful for survival.
Some eons later, it figured out modelling yourself (self / agent) modelling the outside world is also helpful.
In the outside world, we keep track of continuity of objects through a brain module which hones in on the essence of objects
(e.g. tracking a prey or predator.)
so that EXACT matching algorithms aren't applied but ONLY approximate ones.
As soon as a high enough approximate match (>95%) is found,
we "register" it to be an exact match. i.e. the Brain bumps up the confidence level to 100%.
This is also the reason why we consider our friend Bob to be the same childhood Bob even though he has different hairstyle, clothes,
and other such properties. We don't call Bob who looks different than yesterday as an Imposter.
The damage to this brain module could lead us to call Bob today an imposter.
This same module also tracks continuity of self in a similar manner. Even though our "self" changes from childhood to adult
we "register" "changing self" to be the same thing.
i.e. internally we bump up the confidence to 100% when memories, etc. match and provide a coherent picture of the self.
A multiple personality disorder is just different stable states of neural attractor states.
Continuity is local to a personality but not global and hence transient.
This local continuity of information could link up globally, giving rise to coherent single personality.
> If you assume that "the eyes are the window to the soul", you notice some interesting properties.
And people say LLM output is nonsense.
I'm blind with glass eyeballs. Does this mean my soul is easier to access than yours? Or is it harder because there's something specific about the eyeball that makes it the window?
Decision making seems fundamental to intelligence, is done by animals and humans, and can be done without the use of language or logic. This is the case when someone "decided without thinking".
Decision making requires imagination or the ability to envision alternative future states that may result from various choices.
Imagination is the start of abstract thinking. Consciousness results from the individual thinking abstractly about itself and how it interacts with the world.
I’ve been thinking along similar lines. It’s like with LLMs, they’ve created the part of the mind that is endlessly chattering, generating stories, sometimes true, sometimes false, but there’s no awareness or consciousness that ever steps back and can see thoughts as thoughts. And I don’t see how awareness or consciousness would arise from just more of the same (bigger models). It seems to be a fundamentally different part of the mind. I wonder if AGI is possible without this. AGI under some definition (good enough to replace most humans) may be possible. But it wouldn’t be aware. And without awareness, I don’t see how it could be aligned. It may appear to be aligned but then eventually it would probably get caught in a delusional feedback loop that it has no capacity to escape, because it can’t be aware of its own delusion.
> It may appear to be aligned but then eventually it would probably get caught in a delusional feedback loop that it has no capacity to escape, because it can’t be aware of its own delusion.
I believe this is more or less the definition of human mental illness. I have to say that while I know it's really not possible, I wish people would stop pulling on these threads. I got into this line of work because I thought video games were cool, not because I wanted to philosophize about theories of mind and what intelligence is. I really don't like thinking about whether I'm just some sort of automaton made out of meat rather than metal and silicon.
Ah, the first releases of the Bing AI were fun here as they plunged into feedback loops of madness that were scarily human sounding. Thank you humanity for making artificial insanity.
To be clear, I think this is in fact a correct assessment of the architecture of intelligence. You can suspend thought and still function throughout your day in all ways. Discursive thought is entirely unnecessary, but it is often helpful for planning.
My observation of LLMs in such a construction of intelligence is they are entirely the thinking mind - verbal, articulate, but unmoored. There is no, for lack of a better word, “soul,” or that internal awareness that underpins that discursive thinking mind. And because that underlying awareness is non articulate and not directly observable by our thinking and feeling mind, we really don’t understand it or have a science about it. To that end, it’s really hard to pin specifically what is missing in LLMs because we don’t really understand ourselves beyond our observable thinking and emotive minds.
I look at what we are doing with LLMs and adjacent technologies and I wonder if this is sufficient, and building an AGI is perhaps not nearly as useful as we might think, if what we mean is build an awareness. Power tools of the thinking mind are amazingly powerful. Agency and awareness - to what end?
And once we do build an awareness, can we continue to consider it a tool?