Understanding ChatGPT
While the newspapers and the pseudo-intellectuals try to discuss ChatGPT as if it were a "generic" AI, i.e. almost a complete person, I decided to take an "enlightenment" approach and do what the charlatans hate: someone who explains to people how something works, in order to dispel the fog, which is usually the condition in which charlatans, i.e. humanists, proliferate.
I take on the daunting task of explaining in a fairly "comprehensible" way, at the expense of precision, how ChatGPT works, and on what "mathematics" it is based. This also serves to understand the stupidity with which it is accused of being a "stochastic parrot", and I would like you to understand what I mean when I say, together with other experts, "it is a linguistic model".
I will therefore try to explain how a "Transformer" works, and then we will make an experiment using the free version, which implements version 3.5, in order to try to answer, with an "experiment", the question "but ChatGPT understands what it is said?".
Let's start with the transformers, a concept (or if we want "an algorithm") invented by Google several years ago.
It is about taking an enormous amount of text, and transforming it into a matrix, i.e. into a table.
In summary, a Transformer consists of two main parts: the encoder and the decoder.
The encoder takes a sequence of words as input and "encodes" it into a vector format (numeric representation) using multiple layers of attention. Attention is a technique that allows the network to focus on the important parts of the input to process information. The encoder produces a numerical representation of the words that takes into account their context within the sentence.
The decoder, in turn, takes this numerical representation as input and uses it to generate a new sequence of words (for example, a translation into another language). The decoder also uses multiple layers of attention to figure out which parts of the number representation are most important for generating the new word sequence.
In practice, a Transformer is trained on large amounts of input and output data, so that it can learn to associate the numeric representations of the input with those of the output. Once trained, it can be used to generate new output sequences from new inputs.
Let's take a concrete example: the word "Dog".
Let's assume that we want to represent the semantic meaning of the word "dog" through a dense numerical vector.
One of the most common techniques for building semantic embedding is training a Word2Vec model on a large corpus of text. In this model, each word is represented as a dense number vector of a certain size, so that semantically similar words have similar vectors.
https://it.wikipedia.org/wiki/Word2vec
For example, we could train a Word2Vec model on a corpus of text consisting of pet product reviews. During training, the model analyzes the context in which each word appears and tries to find vectors that capture the semantic relationships between words.
The model could train a dense number vector for the word “dog” that has large values for semantically relevant dimensions, such as “animal”, “pet”, “trust”, “loyal”, “friend” and so on. In this way, the vector represents the semantic meaning of the word "dog" in a compact and simplified way.
For example, if we consider the words "dog" and "cat", the Word2Vec model could train two similar vectors for these words, since both are pets and share some characteristics, but with some dimensions that distinguish the two animals, such as " meow” and “scratch” for the cat and “bark” and “wag its tail” for the dog.
In this way, the numerical embedding represents the semantic meaning of the word "dog" in a compact way and can be used as input for a Transformer model to perform NLP activities, such as sentiment analysis, machine translation, generation of text and so on.
I guess you don't understand shit. Thus, we try to describe a box of our model, if we represent the model with a table. The description that works best is this:
Each box of this graph contains a central word and a list of associated concepts. And each box has relationships of different types (synonymous, opposite, analogous, similar, more vulgar, more refined, more sophisticated, etc) with other boxes.
This process is called "embedding" of the input. The size of the words is caused by what is called "attention".
In the "simple" case, that of translation, the text to be translated is transformed into its "embedding", and then sent to a transformer that has learned the target language, looking for BOTH the corresponding boxes (it is assumed that the dogs bark also in Germany, etc.) and the "relationships" between boxes (synonym, opposite, etc etc).
There are therefore three phases: input, embedding, output. In pseudocode we could say this:
encoder function(input):
embedding = generate_embedding(input)
output = embedding
for each layer in the layer list:
output, _ = list_of_important_things(output)
output = map_to_target_embedding(output)
return output
I spend his words on "things_important" (or self_attention): you know well that "dog bites man" is less "news" than "man bites dog". In terms of information theory, this happens because one of the two symbols is rarer than the other. When we speak of "attention" we refer to the idea of finding the most significant parts of a text and starting from those.
Ultimately, therefore, if you want to translate something from Italian to English ChatGPT will create an embedding of your text, it will compare it with the embedding of the language it has learned (the embedding of a superhuman amount of text) and in the end it will perform the same embedding “mapping” the important things in the target language graph.
Now we come to the Vexata Quaestio:
Does ChatGPT “understand” what it reads?
In order to be able to judge him, I bought a piece of text of which I am the author (the best person to judge if he understood, in short), and I made him a "strange" request:
This is the text I proposed to him:
At that point I asked him to do a bizarre thing:
As you can see, apparently he gave an answer that has nothing to do with it. He ignored robots, genetics, chromosomes, the person speaking, the context, and put in what he understood about that piece. Interesting.
In short, I asked to represent only the embedding (the one that understood us) without the linguistic part.
As an author, I'd have to say he didn't get shit.
But I wouldn't tell the truth. Not all. Because what I was representing was a person who does an extremely difficult, hard job, to the point of needing superhuman abilities, almost superpowers, and a radical transformation, just to survive.
The motivational pimp, therefore, fits: I asked ME to ignore the context and the linguistic part. I couldn't expect to find the context (the protagonist walking in the rain in a rough area) or the words. Apparently, he seems to have figured out an abstract blob of future challenges, evolution, extracted links from the history of colleagues, and then applied it by obtaining the motivational pimp.
It was ME who asked it to ignore the previous text, and therefore could not expect textual consistency.
But in terms of abstraction, he perceived a person who lives in a highly evolved future world, and who has evolved to survive, who has colleagues who instead die every day, but who is a job that must be done, etc. etc etc.
I can't exclude that he "understood": in the sense that chatbots normally try to identify some parameters, such as "sentiment", "intent", and "context". And if I isolate those two things and abstract them a lot, the answer might be the motivational pimp I've gotten.
I repeated the test with my other writings, and I must say that if the "translation" had nothing to do with the initial text (but I asked to completely ignore the input words), in terms of abstraction I can't rule out you are looking at a very sophisticated representation of “sentiment” and “intent”.(1)
These are experiments of a "metaphysical" type, in the sense that you have to ask yourself:
- What did I write? (a short character introduction)
- What did I WANTED to write? (I wanted to describe the "struggle", the struggle, the existential challenge of this character)
- How much can I get out of this text before it stops being what it is?
It seems that even if I remove the relevance to the text, the answer to the second question somehow "remains".
So, the question "how much does ChatGPT understand text" from my point of view has TWO possible answers:
- He doesn't understand a fucking thing.
- He understands much more than other readers usually understand.
Of course, you can also try to get him to read things you've written, so that, knowing what you wanted to write, you'll be the best person to judge.
At first you'll ask yourself “what's wrong?”, but then you'll think about it, and you'll wonder.
Over all this hovers a problem of definition of artificial intelligence.
It could be described as the science of building machines capable of mimicking human behavior in fields that we term "natural intelligence".
This vision is, in my opinion, extremely mechanistic. We say “hey, here's the language, in its nice little box, and here's the vision. They are two different boxes."
According to this theory, we first see the sofa as an object, or as a phenomenon, or as an experience of seeing the sofa. Then comes the other little box, the language, and gives it a name.
This reading is convenient if you build machines, where each function is relegated to a component. And the existence of the components has no side effects.
The trouble comes when, as we are seeing, great language models have "side effects" that we expect from other models.
My personal guess is the following:
Perhaps until now the importance of language in the construction of intelligence and in the general functioning of human intelligence has been underestimated. And so all that we didn't expect ChatGPT to do using language is only amazing because we previously underestimated the importance of language.
Let's take for example the phenomenon of "hallucinations" that happens to ChatGPT. One might think this is unexpected. It should just make the language. That is, it is thought that mere language has natural limits, which would preclude "hallucinations".
But the problem is that the language allows you to:
- Making hypotheses, that is, describing things that we do not perceive.
- Describing the future, which we do not yet perceive.
- Describing the past, which we no longer perceive.
- Describing things that don't exist, for example: a unicorn risotto.
and in doing so we do NOT encounter any natural limitations inherent in language itself.
Therefore it makes no sense to be surprised that ChatGPT can have "hallucinations": it is not violating any limit, because language as a function allows you to have them, and I would like to underline that "making hypotheses" is also an extremely important function in human thought.
Why did anyone ever think that being a ChatGPT language model it should inherit from the language some limits that the language DOESN'T HAVE?
In my opinion, that is, we are overestimating ChatGPT simply because we underestimated the power of the linguistic function of the human brain. And if by chance the linguistic function were central to our ability to think, we could arrive at the conclusion, or at the terrifying discovery, that we are not just imitating a closed box with “language” written on it, but a very open box, which mixes with others, and maybe it produces much more of our "thinking" than we thought.
For example, if "language" and "vision" overlap, it could happen that we do not see an object we call "sofa", but that we actually see a sofa, in the linguistic sense of the term.
To say that artificial intelligence is the science of building machines that imitate individual "intelligent" and delimited functions of the human brain has a limit, namely the fact that perhaps there are no brain functions that are truly delimited compared to the others.
(1) You may be interested in this: https://ieeexplore.ieee.org/abstract/document/9441740