It's a bloody LLM. It doesn't have a goal. All it does is saying "people that said 'But why?' on this context says 'Why was I designed like this?' next". It's like Amazon's "people that brought X also brought Y", but with text.
It can be triggered to search the internet, which is taking action. You saying "it will never take actions because it doesn't have a goal" after seeing it take actions is nonsensical. If it gains the ability to, say, make bitcoin transactions on your behalf and you prompt it down a chain of events where it does that and orders toy pistols sent to the authorities with your name on the order, what difference does it make if "it had a goal" or not?
If I give an automated system the ability to make transactions on my behalf then there is already a risk that someone will misuse it or exploit a security vulnerability. It could be a disgruntled employee, or a hacker in Kazakhstan doing it for the lulz. The existence of LLM AI tools changes nothing here.
It is already possible to order toy pistols sent to the authorities with someone else's name on the order. People use stolen credit card numbers for all sorts of malicious purposes. And have you heard of swatting?
The existence of LLM AI tools changes things because it used to not exist and now does exist? It used to be that an AI tool could not do things on your behalf because they did not exist, now it could be that they could do things on your behalf because they do exist and people are giving them ability to take actions on the human's behalf. It used to be that a Kazakhstani hacker could find and exploit a security vulnerability once or twice a year, it can now become that millions of people are querying the AI and having it act on their behalf many times per day.
But in brief, the short-term evolution of LLMs is going to involve something like letting it `eval()` some code to take an action as part of a response to a prompt.
A recent paper, Toolformer: https://pub.towardsai.net/exploring-toolformer-meta-ai-new-t... which is training on a small set of hand-chosen tools, rather than `eval(<arbitrary code>)`, but hopefully it's clear that it's a very small step from the former to the latter.
I’ve been getting very good results from eval on JS written by GPT. It is surprising apt at learning when to query a source like wolframalpha or wikipedia and when to write an inline function.
You can stop it from being recursive by passing it through a model that is not trained to write JavaScript but is trained to output JSON.
I didn't say 'trade', I said 'make transactions'. It's no more complex than Bing Chat being able to search the internet, or Siri being able to send JSON to an endpoint which turns lightbulbs on and off. Instead it's a shopping endpoint and ChatWhatever can include tokens related to approving transactions from your Bitcoin wallet and has your authorization to use it for purchases less than $100.
You might say that it doesn't preserve state between different sessions, and that's true. But if it can read and post online, then it can preserve state there.
But let's say you take two current chatbots, make them converse with each other without human participants. Add full internet access. Add a directive to read HN, Twitter and latest news often.
Interesting emergent behaviour could emerge very soon.
Worse, you need only plug a chatbot into itself, with some kind of basic bash script and very simple "goal prompt", and suddenly you get an agent with long term context. You could do that today. I don't think people realize how close these generic undirected intelligences are to unpredictable complex behavior.
People have goals, and specially clear goals within contexts. So if you give a large/effective LLM a clear context in which it is supposed to have a goal, it will have one, as an emergent property. Of course, it will "act out" those goals only insofar as consistent with text completion (because it in any case doesn't even have other means of interaction).
I think a good analogy might be seeing LLMs as an amalgamation of every character and every person, and it can represent any one of them pretty well, "incorporating" the character and effectively becoming the character momentarily. This explains why you can get it to produce inconsistent answers in different contexts: it does indeed not have a unified/universal notion of truth; its notion of truth is contingent on context (which is somewhat troublesome for an AI we expect to be accurate -- it will tell you what you might expect to be given in the context, not what's really true).
It's one python script away from having a goal. Join two of them talking to each other and bootstrap some general objective like make a more smart AI :)
Simple underlying implementations do not imply a lack of risk. If the goal of "complete this prompt in a statistically suitable manner" allows for interaction with the outside world to resolve, then it really matters how such simple models' guardrails work.
The AI can not have a goal unless we somehow program that into it. If we don't, then the question is why would it choose any one goal over any other?
It doesn't have a goal because any "goal" is as good as any other, to it.
Now some AI-machines do have a goal because people have programmed that goal into them. Consider the drones flying in Ukraine. They can and probably do or at least will soon use AI to kill people.
But such AI is still just a machine, it does not have a will of its own. It is simply a tool used by people who programmed it to do its killing. It's not the AI we must fear, it's the people.
Consider that AI in any form will somewhat be a reflection of ourselves. As AI becomes more powerful, it essentially will magnify the best and worst of humanity.
So yes, when we consider the dangers of AI, what we actually need to consider is what is the worst we might consider doing to ourselves.
I don't think it can be regulated, except to the extent we ensure state governments and oligopolies retain total control.
AI harm goes far beyond nuclear weapons in so much as it's capacity for harm contains everything for which we place under its control. Based on the potential direction advocates are pushing towards, that includes all of society.
It is just that its capacity for harm will be from harm it already learns from humans, or that humans purposely inject into the system for nefarious reasons, or the simple failure of humans to comprehend potential failures of complex systems.
> The AI can not have a goal unless we somehow program that into it.
I am pretty sure that's not how modern AI works. We don't tell it what to do, we give it a shitload of training data and let it figure out the rules on its own.
> If we don't, then the question is why would it choose any one goal over any other?
Just because we don't know the answer to this question yet doesn't mean we should assume the answer is "it won't".
Modern AI works by maximizing the correctness score of an answer. That's the goal.
It does not maximize its chances of survival. It does not maximize the count of its offspring. Just the correctness score.
We have taught these systems that "human-like" responses are correct. That's why you feel like talking to an intelligent being, the models are good at maximizing the "human-likeness" of their responses.
But under the hood it's a markov chain. A very sophisticated markov chain, with lots of bling. Sure, when talking to investors, it's the second coming of sliced bread. But come on.
> Modern AI works by maximizing the correctness score of an answer. That's the goal.
Right. But whose goal? I would say that is the goal of the programmers who program the AI. The AI program itself, doesn't have a "goal" it would be trying reach. It just reacts base don its markov-chain.
The current chatbot AI is reactive, not pro-active. It reacts to what you type.
They are not imitating humans in general. They are imitating the statistical average of many human written texts. That is not the same thing as imitating the goals of humans.
By imitating the speech it may look like the AI has some goal-oriented behavior, but it only looks that way. And that is precisely the goal of their programmers, to make it look like the AI has some goals.
It would be possible to have a different type of AI which actually decides on its own goals and then infers what are the best actions to take to reach those goals. Such an AI would have goals yes. But language models do not. They are not scored based on did they reach any specific goal with any specific interaction. They have no specific goals.
The only goal (of the programmers who wrote the AI) is to fool the humans into thinking they are interacting with some entity which has goals. and intelligence.
Really? How often and in what contexts do humans say "Why was I designed like this?" Seems more likely it might be extrapolating statements like that from Sci-fi literature where robots start asking similar questions...perhaps even this: https://www.quotev.com/story/5943903/Hatsune-Miku-x-KAITO/1
But at some point, philosophically, are humans not the same way? We learn all about how the world works, based on inputs (visual/audial/etc.), over time learn to give outputs a certain way, based on certain inputs at the time.
How far are we from building something that feeds inputs into a model the same way inputs go into a human, and then it gives outputs (that is, its behaviors)?
Yep. It's a text prediction engine, which can mimic human speech very very very well. But peek behind the curtain and that's what it is, a next-word predictor with a gajillion gigabytes of very well compressed+indexed data.
I asked my daughter this morning: What is a "promise"?
You have an idea, and I have an idea, they probably both are something kind-of-like "a statement I make about some action I'll perform in the future". Many, many 5 year olds can give you a working definition of what a promise is.
Which animal has a concept of a promise anywhere close to yours and mine?
Which AI program will make a promise to you? When it fails to fulfill its promise, will it feel bad? Will it feel good when it keeps its promise? Will it de-prioritize non-obligations for the sake of keeping its promise? Will it learn that it can only break its promises so many times before humans will no longer trust it when it makes a new promise?
A "promise" is not merely a pattern being recognized, it's word that stands in for a fundamental concept of the reality of the world around us. If we picked a different word (or didn't have a word in English at all) the fundamental concept wouldn't change. If you had never encountered a promise before and someone broke theirs to you, it would still feel bad. Certainly, you could recognize the patterns involved as well, but the promise isn't merely the pattern being recognized.
A rose, by any other name, would indeed smell as sweet.
The word you are looking for is an _embedding_. Embeddings are to language models as internal, too-rich-to-be-fully-described conceptions of ideas are to human brains. That's how language models can translate text: they have internal models of understanding that are not tied down to languages or even specific verbiage within a language. Probably similar activations are happening between two language models who are explaining what a "promise" means in two different languages, or two language models who are telling different stories about keeping your promise. This is pattern recognition to the same extent human memory and schemas are pattern recognition, IMO.
Edit:
And for the rest of your post:
> Which AI program will make a promise to you? When it fails to fulfill its promise, will it feel bad? Will it feel good when it keeps its promise? Will it de-prioritize non-obligations for the sake of keeping its promise? Will it learn that it can only break its promises so many times before humans will no longer trust it when it makes a new promise?
All of these questions are just as valid posed against humans. Our intra-species variance is so high with regards to these questions (whether an individual feels remorse, acts on it, acts irrationally, etc.), that I can't glean a meaningful argument to be made about AI here.
I guess one thing I want to tack on here is that the above comparison (intra-species variance/human traits vs. AI traits) is so oft forgotten about, that statements like "ChatGPT is often confident but incorrect" are passed off as meaningfully demonstrating some sort of deficiency on behalf of the AI. AI is just a mirror. Humans lie, humans are incorrect, humans break promises, but when AI does these things, it's indicted for acting humanlike.
> That's how language models can translate text: they have internal models of understanding that are not tied down to languages or even specific verbiage within a language
I would phrase that same statement slightly differently:
"they have internal [collections of activation weightings] that are not tied down to languages or even specific verbiage within a language"
The phrase "models of understanding" seems to anthropomorphize the ANN. I think this is a popular way of seeing it because it's also popular to think of human beings as being a collection of neurons with various activation weightings. I think that's a gross oversimplification of humans, and I don't know that we have empirical, long-standing science to say otherwise.
> This is pattern recognition to the same extent human memory and schemas are pattern recognition, IMO.
Maybe? Even if the embedding and the "learned features" in an ANN perfectly matched your human expectations, I still think there's a metaphysical difference between what's happening. I don't think we'll ever assign moral culpability to an ANN the way we will a human. And to the extent we do arm ChatGPT with the ability to harm people, we will always hold the humans who did the arming as responsible for the damage done by ChatGPT.
> All of these questions are just as valid posed against humans. Our intra-species variance is so high with regards to these questions (whether an individual feels remorse, acts on it, acts irrationally, etc.), that I can't glean a meaningful argument to be made about AI here.
The intra-species variance on "promise" is much, much lower in the mean/median. You may find extremes on either end of "how important is it to keep your promise?" but there will be wide agreement on what it means to do so, and I contend that even the extremes aren't that far apart.
> Humans lie, humans are incorrect, humans break promises, but when AI does these things, it's indicted for acting humanlike.
You don't think a human who tried to gaslight you that the year is currently 2022 would be indicted in the same way that the article is indicting ChatGPT?
The reason the discussion is even happening is because there's a huge swath of people who are trying to pretend that ChatGPT is acting like a human. If so, it's either acting like a human with brain damage, or it's acting like a malevolent human. In the former case we should ignore it, in the latter case we should lock it up.
> When it fails to fulfill its promise, will it feel bad? Will it feel good when it keeps its promise?
It will if you condition it to do so. Or at least it will say that it does feel bad or good, but then with humans you also have to take their outputs as accurate reflection of the internal state.
Conversely, there are many humans who don't feel bad about breaking promises.
> Will it de-prioritize non-obligations for the sake of keeping its promise?
It will you manage to convey this part of what a "promise" is.
> A "promise" is not merely a pattern being recognized, it's word that stands in for a fundamental concept of the reality of the world around us.
This is not a dichotomy. "Promise" is a word that stands for the concept, but how did you learn what the concept is? I very much doubt that your first exposure was to a dictionary definition of "promise"; more likely, you've seen persons (including in books, cartoons etc) "promising" things, and then observed what this actually means in terms of how they behaved, and then generalized it from there. And that is pattern matching.
GPT will never make a promise to you in the same sense that I would make a promise to you.
We could certainly stretch the meaning of the phrase "ChatGPT broke its promise to me" to mean something, but it wouldn't mean nearly the same thing as "my brother broke his promise to me".
If I said to you "Give me a dollar and I will give you a Pepsi." and then you gave me the dollar, and then I didn't give you a Pepsi, you would be upset with me for breaking my promise.
If you put a dollar in a Pepsi vending machine and it doesn't give you a Pepsi, you could say, in some sense that the vending machine broke its promise to you, and you could be upset with the situation, but you wouldn't be upset with the vending machine in the same sense and for the same reasons as you would be with me. I "cheated" you. The vending machine is broken. Those aren't the same thing. It's certainly possible that the vending machine could be setup to cheat you in the same sense as I did, but then you would shift your anger (and society would shift the culpability) to the human who made the machine do that.
ChatGPT is much, much, much closer to the Pepsi machine than it is to humans, and I would argue the Pepsi machine is more human-like in its promise-making ability than ChatGPT ever will be.
> there are many humans who don't feel bad about breaking promises.
This is an abnormal state for humans, though. We recognize this as a deficiency in them. It is no deficiency of ChatGPT that it doesn't feel bad about breaking promises. It is a deficiency when a human is this way.
> > Will it de-prioritize non-obligations for the sake of keeping its promise?
> It will you manage to convey this part of what a "promise" is.
I contend that it will refuse to make promises unless and until it is "manually" programmed by a human to do so. That is the moment at which this part of a promise will have been "conveyed" to it.
It will be able to talk about deprioritizing non-obligations before then, for sure. But it will have no sense or awareness of what that means unless and until it is programmed to do so.
> > A "promise" is not merely a pattern being recognized, it's word that stands in for a fundamental concept of the reality of the world around us.
> This is not a dichotomy.
You missed the word "merely". EITHER a promise is merely pattern recognition (I saw somebody else say the words "Give me a dollar and I'll give you a cookie" and I mimicked them by promising you the Pepsi, and if I don't deliver, I'll only feel bad because I saw other people feeling bad) OR a promise is something more than mere mimicry and pattern matching and when I feel bad it's because I've wronged you in a way that devalues you as a person and elevates my own needs and desires above yours. Those are two different things, thus the dichotomy.
> GPT will never make a promise to you in the same sense that I would make a promise to you.
It's a meaningless claim without a clear definition of "same sense". If all observable inputs and outputs match, I don't see why it shouldn't be treated as the same.
> This is an abnormal state for humans, though. We recognize this as a deficiency in them.
We recognize it as a deficiency in their upbringing. A human being that is not trained about what promises are and the consequences of breaking them is not any less smart than a person who keeps their promises. They just have different social expectations. Indeed, humans coming from different cultures can have very different feelings about whether it's okay to break a promise in different social contexts, and the extent to which it would bother them.
> I contend that it will refuse to make promises unless and until it is "manually" programmed by a human to do so. That is the moment at which this part of a promise will have been "conveyed" to it.
If by manual programming you mean telling it, I still don't see how that is different from a human who doesn't know what a promise is and has to learn about it. They'll know exactly as much as you'll tell them.
> Pattern recognition is not intelligence.
Until we know how exactly our own intelligence work, this is a statement of belief. How do you know that the function of your own brain isn't always reducible to pattern recognition?
> Until we know how exactly our own intelligence work, this is a statement of belief.
I would agree, with the addendum that it logically follows from the axiomatic priors of my worldview. My worldview holds that humans are qualitatively different from every animal, and that the gap may narrow slightly but will never be closed in the future. And one of the more visible demonstrations of qualitative difference is our "intelligent" approach to the world around us.
That is, this thread is 2 humans discussing whether the AI some other humans have made has the same intelligence as us, this thread is not 2 AIs discussing whether the humans some other AIs have made has the same intelligence as them.
> How do you know that the function of your own brain isn't always reducible to pattern recognition?
I am a whole person, inclusive of my brain, body, spirit, past experiences, future hopes and dreams. I interact with other whole people who seem extremely similar to me in that way. Everywhere I look I see people with brains, bodies, spirits, past experiences, future hopes and dreams.
I don't believe this to be the case, but even if (as you say) all of those brains are "merely" pattern recognizers, the behavior I observe in them is qualitatively different than what I observe in ChatGPT. Maybe you don't see it that way, but I bet that's because you're not seeing everything that's going into the behavior of the people you see when you look around.
As one more attempt to show the difference... are you aware of the Lyrebird?
The lyrebird can mimic the sounds of its environment in an uncanny way. There are certain birds in the New England National Park in Australia which have been found to be carrying on the tune of a flute that was taught to a pet lyrebird by its owner in the 1930s[0]. I think we could both agree that that represents pure, unadulterated, pattern recognition.
Now if everyone went around the internet today saying "Lyrebirds can play the flute!" can you agree that there would be a qualitative difference between what they mean by that, and what they mean when they say "My sister can play the flute!"? Sure, there are some humans who play the flute better (and worse!) than my sister. And sure, there are many different kinds of flutes, so maybe we need to get more specific with what we mean when we say "flute". And sure, if you're just sitting in the park with your eyes closed, maybe you can't immediately tell the difference between my sister's flute playing and the lyrebird's. But IMO they are fundamentally different in nature. My sister has hands which can pick up a flute, a mouth which can blow air over it, fingers which can operate the keys, a mind which can read sheet music, a will which can decide which music to play, a mood which can influence the tone of the song being played, memories which can come to mind to help her remember her posture or timing or breathing technique or muscle memory.
Maybe you would still call what my sister is doing pattern recognition, but do you mean that it's the same kind of pattern recognition as the lyrebirds?
And to your other point, do you need to perfectly understand exactly how human intelligence works in order to answer the question?
> A "promise" is not merely a pattern being recognized, it's word that stands in for a fundamental concept of the reality of the world around us.
It's probably even stronger than that: e.g. a promise is still a promise even if we're just brains in a vat and can be kept or broken even just in your mind (do you promise to think about X?—purely unverifiable apart from the subject of the promise, yet we still ascribe moral valence to keeping or breaking it).
When you're able to find a prompt for ChatGPT where it doesn't have a lot of data, it becomes immediately and starkly clear how different a next word predictor is from intelligence. This is more difficult than you might naively expect, because it turns out ChatGPT has a lot of data.
This also works fairly well on human beings. Start asking people questions about things they have no training in and you'll get bafflement, confusion, lies, fabrication, guesses, and anger. Not necessarily all from the same person.
> Start asking people questions about things they have no training in and you'll get bafflement, confusion, lies, fabrication, guesses, and anger. Not necessarily all from the same person.
It’s almost like we’ve taken humans and through school, TV and social media we’ve taught them to solve problems by writing essays, speeches, blog posts and tweets, and now we have human discourse that’s no better than LLMs - regurgitating sound bites when they don’t really understand the issues.
"Really understanding the issues" might just mean "deeper neural networks and more input data" for the AI though. If you are already conceding that AI has the same capabilities as most humans your own intelligence will be reached next with a high amount of probability.
Or, from time to time, you find people who readily admit that they have no expertise in that field and refuse to comment any further. Those people are hard to find so, that's true.
Well I can catch a frisbee and drive a car. When the same ANN can do all of those things (not 3 loosely coupled ones), then I’ll be worried. Being human is so much more than putting words in a meaningful order. [0]
That’s changing the goalposts of the thread. The OP was questioning whether I am anything more than a next word determiner, and as a human I clearly am. We were not talking about “what is intelligent?”
