The greatest thing since bitcoin. GPT-4: what is it to expect?

To appreciate the scope of GPT-4, we must first consider GPT-3, its predecessor. GPT-3 was the most powerful text generation algorithm available till today.

The algorithm presented by OpenAI in May 2020 writes poetry and prose, news, posts, and descriptions, can answer questions on the reading, translate, solve examples, write music, and program. To do this, he will only need a verbal description of the problem and a couple of examples that GPT-3 needs only to better understand the specific context.

For more information about the possibilities of ML architecture, read these AI-related posts’  compilation.

Such a short introduction became possible because 500 GB of text data were “fed” for GPT-3 pretraining, or rather, the entire English-language Wikipedia, data from the Common Crawl open library (it scans the Internet every month since 2011), datasets with books and several billion web pages. The training of the algorithm cost the company, founded by Elon Musk and Sam Altman, about $5 million.

What’s in GPT-3

GPT-3 is based on the same Transformer architecture (developed by Google) as previous versions of the algorithm, but with a larger number of parameters and on a much larger dataset. The algorithm now has 175 billion parameters – this is the most prominent language model. Parameters are variables that the neural network optimizes during its training. There were 1.5 billion of them in GPT-2, and 17 billion in the second most powerful language model, Microsoft Turing-NLG.

GPT-4 is expected to be better at multitasking; it will be a version of machine learning that will bring the results even closer to humans. The GPT-3 was built using hundreds of millions of pounds, but the GPT-4 is expected to be even more expensive and the GPT-4 is estimated to be five hundred times the size. In comparison, GPT-4 will have as many parameters as synapses in the brain.

What is the difference between GPT-3 and GPT-4?

From a technical point of view, GPT-4 will have about 100 trillion parameters – about 500 times more than GPT-3. On top of that, the input will allow more characters to be used (roughly counted as words), so much longer text blocks will be consumed and generated.

For practical use, GPT-3 allowed users to enter natural language text but still required some skill to design ads in a way that would work. GPT-4 will be much better at detecting user intent.

Concerns of creators and developers

Programs like GPT-4 create frightening opportunities for abuse. If computers can produce significant texts that are indistinguishable from humans, how can we tell humans from machines? The document, posted by the creators of GPT-3 and the future GPT-4, describes in detail the capabilities of the algorithm and some dangers associated with them: fakes, spam, phishing, forgery of academic papers, abuse of legal and state information.

There are other problems as well. Since the algorithm is learned from texts on the internet, likely, that GPT-4 can also adopt the prejudices that exist in society. While we cannot know for sure that the text generated by the robot will not be racist or sexist. In addition, GPT-3 does not learn how to distinguish fact from fiction. It is not yet clear whether the GPT-4 will be capable of this.

There are still reasons to doubt that GPT-4 will represent the meaning of the words it uses. After all, GPT-3 “knowledge” is based only on textual descriptions of processes, phenomena, events, objects, etc. While for example, children learn a language in a completely different way: they compare words with concepts that they received not only when reading a text but and through exploration of the world.

The meanings of words in GPT-4 cannot be based on perceptions of the world. It lacks the intentions, goals, beliefs, and desires that govern people’s use of language. His utterances have no “purpose,” and he does not “think” before he speaks. But is it that important at this stage? After all, his complex and structured internal ideas about language and the world allow us to compose sentences in a way that often seems natural to us.

Finally

There are many predictions about what GPT-4 will look like and what it will mean for the future. In general, the main differences are as follows:

• GPT-4 will have more parameters – it will train on more data to make it even more powerful.
• Their productivity will be closer to humans.
• GPT-4 will be more robust to human error.

If you are in rush to create some powerful project with GPT technologies, it’s high time to get in touch with expert ML practitioners.