What can we expect from Gen AI next year?
For most of 2024, the narrative has been about quality. Models are measured and compared on a range of qualitative and quantitative benchmarks. In almost every dimension, the difference in performance among frontier models is now within a tenth of a percent.
At the same time, the rate of improvement has been slowing. There is an ocean of difference between GPT-3.5 and GPT-4, but a much smaller improvement from GPT-4 to GPT-4o. o1, the reasoning model from OpenAI, was a step down in many benchmarks compared to its predecessor.
In other words, we are approaching the top of the S-curve for the transformer-type models.
Simultaneously, the cost of using a large language model has been dropping precipitously. Mini models cost less than a dollar per million input tokens. For reference, there are 800K words in all works of Shakespeare or roughly 1.2M tokens. So, one can now summarize Shakespeare life's work for about a dollar.
This brings us to the point: GPT-style intelligence quickly becomes a free commodity.
Consequently, unless the industry makes a breakthrough discovery that leads to a step-up in intelligence, the next few years will be less about quality and more about quantity.
As we all know from Georg Wilhelm Friedrich Hegel, a German philosopher and one of the most important figures in German idealism, quantity has a quality of its own.
This principle is often summarized as "quantitative changes lead to qualitative changes," which expresses the idea that small, incremental changes can result in a significant transformation when a certain threshold is crossed.
In the context of Generative AI, every business leader must consider a simple proposition:
"What could I achieve with an infinite army of knowledge workers?"
With the cost of intelligence quickly approaching zero, this proposition is becoming a reality. Would you like to assign each customer, no matter how small, a dedicated account manager? Done! Would you like every college student to have a personal tutor? Done! Would you like to spawn 1000 copies of yourself and attend all meetings at your company and beyond? No problem! Done!
The opportunities and possibilities are endless and limited only by our imagination.
As always, there is a catch.
These new knowledge workers, let us call them GPT workers, are unlike any other workers we've seen before. They have unique qualities that will dramatically reshape how we study, work, sell, buy, and play.
- A GPT worker is incredibly well-read. It read every book, scientific paper, blog post, and web page. It viewed all videos on YouTube and saw every post on Instagram. It possesses truly encyclopedic knowledge, can reason, and can make connections across seemingly unrelated domains.
- At the same time, a GPT worker is also not very smart. It often comes up with brilliant answers for simple problems but also trivial solutions for complex ones. It can prove theorems but also makes stupid mistakes. We don't know everything it knows or does not know.
- A GPT worker is frozen in time and has an incredibly short memory. It does not know what happened last month and needs to be constantly reminded about conversations that happened a week ago. It knows nothing about you, your business, or your customers.
- At the same time, a GPT worker is an incredibly quick learner. It can read and summarize an entire textbook in seconds. It can learn to solve complex puzzles from just a few examples. A GPT worker can also use tools such as search engines, calculators, and business applications.
- A GPT worker is an outstanding artist and actor. It can make itself look, sound, and act like anything or anyone, including you and your friends. It's great at following a script. It can natively speak and translate hundreds of languages. A computer language is just another language. It can produce incredible videos, paintings, and web designs.
- At the same time, a GPT worker is incredibly gullible and can be talked into saying or doing virtually anything, like selling a $100K car for $1. It responds to strange phrases like "Think step by step" or "You are a pilot." It also lies from time to time and does so very convincingly.
As humans, we are not prepared to deal with such personalities. Now imagine ten, twenty, a hundred, or even thousands of them reporting to you at work. The power is enormous, but we must learn to control it.
Some applications, like drafting personalized emails, assisting in customer support, researching prospects, ideating, writing computer code, or designing investment strategies, are almost straightforward. Some other applications, like diagnosing medical problems or winning people's trust, are very difficult.
So, my projection for 2025 and beyond is that we will all be learning to answer one fundamental question: "What can I do with an army of incredibly bright but not very smart GPT workers?"