Nvidia CEO describes new AI way of software development

Over the past few weeks, Nvidia CEO Jensen Huang has openly said that with upcoming advances in AI, humans will not need to know how to program.

At Nvidia’s annual GPU Technology Conference (GTC) this week, Huang doubled down on his words and explained in his keynote how the company is remaking software development during his keynote

Many have talked about AI being used to generate code, but Nvidia is putting it to work. Huang spoke more broadly – about cleaning up the entire development pipeline with new ways of writing programs.

“How do we build software in the future? You’re unlikely to write it from scratch or write a whole bunch of Python code or something like that. It is very likely that you will assemble a team of AIs,” Huang said.

What is the plan?

Nvidia is creating an AI-specific software development cycle that relies on chatbots and copilots to build apps.

“There will probably be a super AI that will take the mission you give it and break it down into an execution plan,” Huang said.

The programming language is English. Users can type the program they want into the ChatGPT-style interface, and the output will be an app, Huang said.

The shift to a new model of software creation comes as computers with artificial intelligence capable of reasoning gain a foothold. Current software development cycles are highly dependent on the logical nature of the CPU.

The new computer will “help you create a new type of application for the future. Not the one you wrote completely from scratch,” Huang said.

Nvidia announced the concept of an “artificial intelligence foundry” as a so-called building kit for generating applications. Users can just specify the type of app they want, and the artificial intelligence foundry — based on Nvidia’s hardware and software — will churn out the app.

What is Backend?

Nvidia’s development workflow relies on using conversational and automated interfaces to write, package, and deploy software. The goal is to end the manual work involved in traditional CI/CD pipelines.

“We’re going to invent a new way to receive and manage software,” Huang said.

Nvidia hopes to automate software creation and code generation through copilots, AI interfaces, containers and microservices. Nvidia’s interface automates dependency configuration and performs relevant fine-tuning.

The most important component is NIMs (Nvidia Inference Microservices), which is more like an API for AI, which was announced at GTC. NIMs help users build applications by accessing the right data, big language models, programming tools, and dependencies.

All pre-trained proprietary and large open source language models are stored in a container built on top of Kubernetes. The container — which is more of a black box — also includes a cloud-native set optimized for GPUs, Nvidia’s CUDA parallel programming language, the CuDNN neural network, and other tools like TensorRT, which improves inference performance.

Different NIMs work together to generate and execute code inside a black box, which then delivers the end results to users.

“That NIM might be able to understand SAP — the language of SAP is ABAP, it might be able to understand ServiceNow and go get some information from their platforms,” ​​Huang said.

Instead of coding, users can speak plain English.

“This is a piece of software in the future that has a really simple API. And that API is called human,” Huang said.

Nvidia’s stack uses industry-standard APIs for speech, text, images, and video. Nvidia announced new AI software called AI Enterprise 5.0 that includes NeMo Retriever, which can retrieve information, and Triton Inference Server, which serves information.

The Nvidia software suite retrieves structured and unstructured data from the database and converts it into conversational data.

“Essentially you take structured data or unstructured data, learn its meaning, encode its meaning. Now this is becoming an AI database,” Huang said. “Once you create it, you can talk to it.”

SAP, ServiceNow, Cohesity and Snowflake are among some of the customers using Nvidia’s NIMs to create copilots, chatbots and virtual assistants for users to interact with in plain English.

Nvidia’s AI Stack Origins

Nvidia’s proprietary AI software suite, called CUDA, began in 2006 as a programming model for high-performance computing.

In 2012, CUDA made “first contact” with AlexNet, a neural network for image recognition. It was the first trained on an Nvidia GPU.

“Recognizing the importance of this computing model, we invented an entirely new type of computer that we call DGX-1 — 170 teraflops in this supercomputer, eight GPUs connected together for the first time. I delivered the first DGX-1 to a San Francisco-based startup called OpenAI,” Huang said.

In 2022, Nvidia GPUs were used to revive ChatGPT.

It will be the world of artificial intelligence

To be sure, Huang’s big plans are more relevant to programming for AI systems, which can be targeted and specific to customer requirements. This differs from the conventional CI/CD model.

But the market demands that coders quickly be upskilling for AI — the number of AI-related tech jobs is on the rise in a market where IT jobs are declining, according to research from the University of Maryland.

Nvidia dominates the AI ​​market and has a fundamental approach — if you want to use Nvidia GPUs, you need to know how their development model works.

“The first is to have AI technology, the second is to help you modify, and the third is the fine-tuning infrastructure,” Huang said.

Nvidia unveiled a new GPU called Blackwell at the show, which company executives said delivers 20 petaflops of AI performance on a single GPU. A set of 576 Blackwell GPUs could train models with trillions of parameters.

Challenges

Nvidia wants customers to use its expensive hardware and software, which requires the use of CUDA and creates a barrier to entry.

Nvidia H100 GPU instances on cloud providers like CoreWeave and Lambda Labs cost twice as much per hour as older A100 models.

Google floated the idea of ​​its AI-based Duet service that creates customized programs, like financial tools, just by talking to AI. Nvidia’s main competitors, AMD, Intel and Cerebras, use an open source software approach. Companies support open models that include Llama 2 and Mixtral, but chip makers don’t provide tools where users can write applications just by talking to a computer.