We are in the Great Acceleration – a singularity, not in the mainstream S-Kurzweil sense of rising robots, but in the one described by Foucault: a period of time in which change is so widespread and so fundamental that one cannot properly discern what the other side will be like those changes.
We have already gone through the singularities:
- The rise of agriculture (which created a surplus of resources and gave us an academic and merchant class).
- The invention of the printing press (which democratized knowledge and made it less malleable, giving us the idea of a source of truth beyond our senses).
- The steam engine (which allows machines to perform physical tasks).
- Computer software (which allows us to give instructions to machines to follow).
- Internet and smartphones (which interactively connect us all to each other).
This singularity, in its simplest form, is that we invented a new kind of software.
An old kind of software
The old kind of software—the kind that’s on your phones and computers right now—has changed our lives in ways that would make them almost unrecognizable to someone from the 1970s. Humanity has had 50 years to adapt to software because it started slowly with academics, then hobbyists, with dial-up modems and corporate e-mail. But even with half a century to adapt, our civilization is struggling with its consequences.
The software you’re familiar with today—the stuff that sends messages, or adds numbers, or makes a reservation on your calendar, or even starts a video call—is deterministic. This means it does what you expect. When the result is unexpected, it is called an error.
From deterministic software to artificial intelligence
Earlier examples of “thinking machines” included cybernetics (feedback loops like autopilot) and expert systems (decision trees for doctors). But they were still predictable and understandable. They just followed a lot of rules.
In the 1980s, we tried a different approach. We structured the software to behave like a brain, giving it “neurons”. And then we let it configure itself based on the example. In 1980, a young researcher named Yann LeCun tried it on image classification.
He is now the head of AI at Meta.
Then the artificial intelligence went into a kind of hibernation. Progress was being made, but it was slow and happening in the corridors of the academy. Deep learning technologies, TensorFlow and others have emerged, mainly to power search engines, recommendations and advertising. But artificial intelligence was a thing that happened behind the scenes, in advertising services, maps and voice recognition.
In 2017, some researchers published a seminal paper called “Attention is all you need.” At the time, the authors worked at Google, but many have since moved to companies like OpenAI. The document described a much simpler way to allow software to configure itself by paying attention to the parts of the language that matter most.
An early use for this was translation. If you feed the algorithm enough English and French text, it can figure out how to translate from one to the other by understanding the relationship between words in each language. But the basic approach allowed us to train the software on text downloaded from the Internet.
Since then, progress has been quite rapid. In 2021, we discovered how to create an “instruction model” that used a process called Supervised Fine Tuning (SFT) to make the AI follow instructions. In 2022, we had people rate responses to our instructions (called Modified Supervised Fine-Tuning), and in late 2022 we added something called Reinforcement Learning based on human feedback, which gave us GPT-3.5 and ChatGPT. AI can now give feedback to other AIs.
Anyway, by 2024, people are the input on which things are trained and provide feedback on the quality of the output that is used to improve it.
When it’s unexpected it’s a feature, not a bug
The result is a new kind of software. To make it work, we first collect a bunch of data and use it to train a massive mathematical model. Then we input a query into the model and it predicts the answer we want (many people don’t realize this once the AI is trained, the same input produces the same output – the one it thinks is “best” – every time). But we want creativity, so we add a nuisance, called temperature, that tells the AI how much randomness it should inject into its responses.
We cannot predict in advance what the model will do. And we intentionally introduce randomness to get different answers every time. The whole point of this new software is to be unpredictable. Being non-deterministic. He does unexpected things.
In the past, you put something into an app and it followed a set of instructions that people wrote and the expected result appeared. Now, you put something into the AI and it follows a series of instructions that it wrote, and the unexpected result appeared on the other side. And the unexpected result is not a bug, but a feature.
Incredibly fast adoption
We are adopting this second type of software much faster than the first, for several reasons
- Creates own user manual: While we all get excited about good results, we often overlook how well it can respond to simple inputs. This is the first software with no learning curve – literally anyone who can type or speak will be told how to use it. It is the first software that creates its own documentation.
- Anyone can try it: Thanks to ubiquitous connectivity via mobile phones and broadband and the SaaS model of hosted software, many people have access. You no longer need to buy and install software. Anyone with a browser can try it out.
- Hardware is everywhere: Gaming GPUs, Apple’s M-series chips, and cloud computing make massive computing resources trivially easy to deploy.
- Costs have fallen. A lots of: Some algorithmic advances have reduced the cost of artificial intelligence by several orders of magnitude. The cost of classifying a billion images dropped from $10,000 in 2021 to $0.03 in 2023 – 450 times cheaper per day.
- We live online: People are online for an average of six hours a day, and most of that interaction (email, chat rooms, texting, blogging) is text-based. In the online world, a human is largely indistinguishable from an algorithm, so there are many easy ways to connect AI output to the feeds and screens people use. COVID-19 accelerated remote work, and thus the introduction of text and algorithms into our lives.
What non-deterministic software can do
Non-deterministic software can do many things, some of which we are just beginning to understand.
- It is generative. It can create new things. We see this in images (Stable Diffusion, Dall-e) and music (Google MusicLM), and even in finance, genomics and resource discovery. But the place that’s getting the most attention is chatbots like those from OpenAI, Google, Perplexity and others.
- He’s good at creativity, but that’s it he makes things up. This means we give him “fun” jobs like art, prose, and music for which he has no “right answer.” It also means a deluge of misinformation and an epistemic crisis for humanity.
- It still is it needs a lot of human input filter the output into something usable. In fact, many of the steps in creating a conversational AI involve humans giving it examples of good answers or rating the answers it gives.
- Because it is often wrong, we must be able to blame someone. The man who decides what to do with his result is responsible for the consequences.
- It can understand in ways we didn’t think should be possible. We do not understand why this is so.
The pendulum and the democratization of IT
While it is, by definition, difficult to predict the other side of the singularity, we can make some educated guesses about how information technology (IT) will change. The IT industry has undergone two major changes over the past century:
- A constant pendulum, it swings from the centralization of mainframes to the distributed nature of web clients.
- It’s a gradual democratization of resources, from the days when computing was rare, precious, and guarded by IT to the era when developers, and then job holders themselves, could allocate resources as needed.
This diagram shows that change:
There’s another layer that’s happening thanks to AI: User-Controlled Computing. We’re already seeing no-code and low-code tools like Unqork, Bubble, Webflow, Zapier and others making it easier for users to build apps, but what’s far more interesting is when AI prompts users to run code. We see this in OpenAI’s ChatGPT code translator, which will write and run data processing applications.
There will likely be another swing of the pendulum towards the edge in the coming years as companies like Apple enter the fray (which have built powerful AI processing into their domestic chipsets in anticipation of this day). Here’s what the next layer of computing looks like:
Construction of the second beam
Another prediction we can make about IT in the non-deterministic age is that companies will have two sets.
- One will be deterministic, performing predictable tasks.
- One will be non-deterministic, producing unexpected results.
Perhaps most interestingly, the second (non-deterministic) stack will be able to write code that the first (deterministic) stack will be able to run – soon, better than humans can.
The coming decade will see a rush to build a second string in every organization. Each company will be judged on the value of its corpus, proprietary information and the real-time updates it uses to get the best results from its AI. Each set will have different hardware requirements, architectures, management, user interfaces, and cost structures.
We cannot predict how AI will reshape humanity. But we can make an educated guess as to how it will change IT businesses, and those who adapt quickly will be best prepared to take advantage of what comes next.
Alistair Croll He is the author of several books on technology, business and society, including the bestseller Lean Analytics. He is the founder and co-chair of FWD50, the world’s leading public sector innovation conference, and has served as visiting executive director at Harvard Business School, where he helped develop the data science and critical thinking curriculum. He is the chairman of the Data Universe 2024 conference.
Meet the author at Data Universe
Join author, Alistair Croll, at Data Universe April 10-11, 2024 in NYC, where he will chair the inaugural launch of a brand-agnostic new data and AI conference designed to for the entire global data and artificial intelligence community.
Bringing EVERYTHING together – Data Universe welcomes data professionals of all skill levels and roles, as well as business people, executives and industry partners to engage with the most current and relevant expert insights in data, analytics, ML and AI researched across industries, to help you to evolve along with the rapidly changing norms, tools, techniques and expectations that are changing the future of business and society. Join us at the North Javits Center in New York City this April to be a part of the future of data and artificial intelligence.
INFORMS is happy to be a strategic partner of Data Universe 2024 and will hold four sessions during the conference.
Featured image credit: Growtika/Unsplash