It's lacking agency, and it's not doing enough of that quacking and walking, it doesn't have memory, it doesn't have self-consciousness.
The Panpsychist view is a good starting point, but ultimately it's too simple. (It's just a spectrum, yes, and?) However, what I found incredibly powerful is Joscha Bach's model(s) about intelligence and consciousness.
To paraphrase: intelligence is the ability to model the subject(s) of a mind's attention, and consciousness is a model that contains the self too. (Self-directed attention. Noticing that there's a feedback loop.)
And this helps to understand that currently these AIs have their intelligence outside of their self, nor do they have agency (control over) their attention, nor do they have much persistence for forming models based on that attention. (Because the formed attention-model lives as a prompt, and it does not get integrated back into the trained-model.)
The Panpsychist view is a good starting point, but ultimately it's too simple. (It's just a spectrum, yes, and?) However, what I found incredibly powerful is Joscha Bach's model(s) about intelligence and consciousness.
To paraphrase: intelligence is the ability to model the subject(s) of a mind's attention, and consciousness is a model that contains the self too. (Self-directed attention. Noticing that there's a feedback loop.)
And this helps to understand that currently these AIs have their intelligence outside of their self, nor do they have agency (control over) their attention, nor do they have much persistence for forming models based on that attention. (Because the formed attention-model lives as a prompt, and it does not get integrated back into the trained-model.)