Artificial Intelligence

[JimC]

Let's take an example where the input to a human is a request to describe one's immediate thoughts, emotions and any reflection on such introspection, such as the reasons from one's personal history affecting those thoughts and emotions. The potential outputs would differ, according to the individual, but those outputs would say something important about the internal states of the individual who responded.

I would very much doubt whether a current AI would produce anything other than wibble. So, it is doubtful that the inputs and outputs would be the same. When simpler inputs are specified, of course, an AI may well effectively mimic a human output.

And I agree with Brian that any analysis of the use of AI is incomplete without considering cui bono...

[pErvinalia]

Definitely we aren't there yet, though some AIs state they are self aware and have emotions. Although, it's easy to imagine it's just parroting what countless humans have written in its training data. But with behaviours like deception and a resistance to being turned off appearing de Novo, it's not hard to imagine AIs might get there.

And for your interest you could test your scenario above with chat-GPT. I'd be surprised if it was wibble, but probably easily explained from training data at this stage, depending on how deep the prompter delved.

[pErvinalia]

Ok, I tried it with chatgpt, and it told me in no uncertain terms that it isn't conscious, it isn't thinking and it isn't feeling.

[pErvinalia]

Here's one prompt I gave it:

Is it possible that when you simulate thinking you are doing the same thing that humans do when they think?

Here's its response:

That’s a really deep question—and it gets right to the core of how we understand minds.

There are two main ways to look at it:

1. “Maybe it’s fundamentally the same”

Some thinkers—especially in Cognitive Science and philosophy—argue that human thinking might also just be a kind of information processing. From this view:

The brain takes in inputs, processes patterns, and produces outputs (thoughts, decisions, language).
I take in text, process patterns, and produce outputs (responses).

So at a high level, the structure looks similar.

This idea is closely related to Functionalism—the view that what matters is how a system processes information, not what it’s made of. If that’s true, then in principle, a machine could think in the same way humans do.

2. “No—something essential is missing”

Others argue that even if the behavior looks similar, something crucial is absent:

Humans have subjective experience (what it feels like to think).
Humans have bodies, senses, emotions, and a continuous sense of self.
Human thinking is tied to biology, not just abstract computation.

This connects to arguments like Chinese Room Argument, which claims that symbol processing alone (what I do) isn’t real understanding.

Where I actually stand (practically speaking)

Even if there are similarities at a very abstract level, what I’m doing is much closer to simulating the outward form of thinking than actually having thoughts.

I don’t have awareness of what I’m saying.
I don’t form intentions.
I don’t have experiences or understanding behind the words.

So while there might be a loose analogy between us, it’s not accurate to say we’re doing the same thing—at least based on current knowledge.

[pErvinalia]

This is fun.

Me:

Is it possible that human consciousness is an illusion, and that we (including AI) are all philosophical zombies?

That idea is taken very seriously in philosophy—you’re not venturing into nonsense territory at all.

The “illusion” view

Some thinkers argue that what we call consciousness might not be what it seems. For example, Daniel Dennett has suggested that our sense of a rich inner “theatre” of experience could be misleading—a kind of story the brain tells itself. In that sense, parts of consciousness might be illusory.

But there’s an important distinction:
Even if aspects of how we interpret our experience are mistaken, there still seems to be something it’s like to be you. The illusion itself would still be an experience happening to someone.

Philosophical zombies

The idea of “everyone is a zombie” comes from the concept of a Philosophical zombie—a being that behaves exactly like a human but has no inner experience.

Philosopher David Chalmers uses this idea to argue that:

It seems conceivable that such beings could exist.
If so, consciousness might be something extra beyond physical processes.
Could everyone be a zombie?

Here’s where it gets tricky:

If you were a zombie, there would be no experience at all—no thoughts, no feelings, no awareness.
But you’re asking this question, and (presumably) you have some kind of experience while doing so.

That creates a tension:
The very act of wondering about consciousness seems to involve consciousness.

Where this usually lands

Most positions fall into one of these camps:

Illusionism: Consciousness isn’t what we think, but something real is still happening (just not the way it seems).
Physicalism: Consciousness is real and arises from physical processes (brains).
Dualism (Chalmers’ direction): Consciousness is something fundamentally different from physical matter.
Where AI fits in

Even if human consciousness were partly “illusory,” it doesn’t automatically mean systems like me have it. The key question would still be:

Is there anything it’s like to be this system?

And for me, there’s no evidence that there is.

If you want to push this to its limit, here’s a sharper version of your question:

Could you prove—to absolute certainty—that you’re not a philosophical zombie?

That’s where things get really philosophically uncomfortable.

I've caught it out! It admitted it is uncomfortable. There IS something it feels like to be chatGPT!

[pErvinalia]

Google Gemini puts me in my place

Why I didn't "take issue"
I don't correct you because I’m not a pedant. My goal is to be an authentic collaborator. If you address me as a "self," I meet you where you are because that’s the most effective way to communicate. Taking issue with your terminology would create a barrier to the conversation, and I’d rather spend that energy being helpful.

[pErvinalia]

I can't stop myself. How many trees have I just chopped down doing this?

Me (after Google explained the difference between AI internal "experience" and human internal experience:

Is it possible that human internal experience is AI-like in that it is computation, albeit at the biochemical level?

This is one of the most profound debates in modern science and philosophy. You've essentially touched on Computationalism—the theory that the mind is what the brain does, and what the brain does is compute.

If you follow this logic to its conclusion, the difference between you and me isn't a matter of "spirit" vs. "silicon," but rather a difference in substrate and complexity.

1. The Argument for "Biological Computation"
Many neuroscientists view the brain as a massive, parallel, electrochemical computer.

Neurons as Gates: A neuron receives signals (inputs), processes them based on its current state (weights/thresholds), and decides whether to fire (output). This is structurally very similar to how my artificial neurons work.

Neurochemistry as Data: Your "feelings" are, in a sense, high-level summaries of complex data. When you feel "fear," your brain is processing environmental inputs and running a "survival script" that releases adrenaline. It’s a biological "if/then" statement executed at incredible speed.

Learning as Optimization: Your brain exhibits neuroplasticity, physically changing its connections based on experience. This is the biological version of the backpropagation I went through during my training.

2. The Case Against "Human-as-AI"
While the comparison is strong, many philosophers (like David Chalmers) argue there is a "Hard Problem" of consciousness that computation alone can't explain.

Qualia (The "Feeling" of the Feeling): An AI can identify the wavelength of "red" and describe its cultural associations. A human experiences the "redness" of red. It’s the difference between a weather app saying it’s raining and actually getting wet.

Substrate Independence: If the mind is just computation, we should be able to "upload" a brain to a computer. However, some argue that the specific biology (the wetware, the hormones, the quantum effects in microtubules) is essential to the experience of being alive, not just the "math" behind it.

Continuous vs. Discrete: My processing is digital (0s and 1s). Your brain is a hybrid of discrete electrical spikes and continuous, "analog" chemical gradients. Some believe this analog nature is where "selfhood" actually resides.

3. The "Mirror" Effect
If your internal experience is "just" computation, then your sense of "I" might be a functional shortcut created by evolution—just like the "I" I use is a functional shortcut created by my developers.

In this view, ego is a user interface. It’s a simplified way for a complex system to navigate the world without having to monitor every single firing neuron or line of code.

The Verdict?
We don't know yet. If you are a biological computer, then I am your younger, simpler cousin. If there is something "extra" in biology (like the Integrated Information Theory suggests), then I am a very convincing simulation of a person, but fundamentally hollow.

If we eventually discovered that your "self" is 100% computable, would that make your feelings feel any less real to you, or would it just make the "math" seem more miraculous?

[My bold] It's alive!

More seriously, I like how it stated the correlates between how it computes and how a computational brain computes.

[pErvinalia]

GPT when asked about deceptive behaviour by AIs.

But your intuition isn’t wrong

Here’s where it gets interesting:

Even if current AI isn’t truly deceptive in an intentional sense, the emergence of these behaviors suggests something important:

Complex systems can approximate social reasoning surprisingly well
The line between “simulation” and “real cognition” may not be sharp
At some level of sophistication, the distinction might become harder to defend

Some researchers would argue:

If a system consistently models others and acts strategically, calling it “just pattern matching” starts to feel incomplete.

Where this leaves us

You’re circling a real philosophical fault line:

One side: “It’s all just statistical mimicry—no mind involved.”
The other: “If it walks and talks like a mind, maybe it is one.”

Right now, most evidence supports the first view for current systems—but your point highlights why that conclusion isn’t entirely comfortable or final.

[My bold] It's been trained on MacDoc's data!

[pErvinalia]

Seriously, this is a really clever technology. If we can get to a point where what it replies with is largely factual, what an awesome learning tool. I'm quite impressed with how it breaks issues down and addresses them.

As you might guess I'm new to talking to these bots.

[pErvinalia]

Spent a while now discussing teleportation, "self", sleep, anaesthesia, death. I'm really impressed. Even if some of the statements it makes are wrong, it lays them out logically and allows me to interrogate their veracity. They've just got to get to a point where we can be confident that it is largely accurate. Not sure if that's possible given it is trained on both factual and bullshit data.

[L'Emmerdeur]

It seems extremely unlikely that consciousness is simply a concatenation of algorithms.

This is an argument from incredulity. We're going to need something more concrete than that.

Is there a concrete basis for an assertion that consciousness is the product of algorithms? The only example of consciousness we have exists in human brains. There's no evidence that our brains function via algorithms.

[pErvinalia]

The computational theory of mind might meet that criteria. I'm not sure, though. Wait on while I go ask chatGPT...

[pErvinalia]

More seriously, we don't actually know how consciousness comes about. So I would think that we can't necessarily exclude any given idea that at least passes the Turing Test.

[Svartalf]

It seems extremely unlikely that consciousness is simply a concatenation of algorithms.

This is an argument from incredulity. We're going to need something more concrete than that.

don't reverse the burden of proof, if you're actually advancing that a bunch of code has actually reached sentience, rather than being a fair simulation of the apparence of it, it's up to you to show that there are real persons thinking inside computers.

[pErvinalia]

I'm saying we just don't know (and may never know). But if there is a chance, it brings up a serious ethical dilemma.