Nvidia is doubling down on what could be the next big battleground in artificial intelligence, inference computing, with the company estimating that its AI chip revenue opportunity could reach at least $1 trillion through 2027.
The projection, shared by CEO Jensen Huang at the company’s annual GTC developer conference in San Jose, marks a sharp jump from the $500 billion opportunity Nvidia had outlined earlier for its Blackwell and Rubin chips, reported Reuters.
The announcement underscores a broader shift in the AI ecosystem, where the focus is moving from training models to deploying them at scale.
From Training To Real-Time AI: A Strategic Shift
For the past few years, Nvidia has dominated the AI training segment, powering the development of large language models and complex AI systems. However, the next phase of growth lies in inference, the process of running these models in real time to answer queries and perform tasks.
Huang described this transition as a turning point for the industry. Speaking at the conference, he noted that the demand for inference computing is accelerating rapidly, with businesses now focused on serving millions of users interacting with AI tools daily.
This shift is significant because inference requires a different kind of computing architecture, one that balances speed, efficiency and cost, rather than just raw processing power.
New Chips, New Strategy
To strengthen its position, Nvidia unveiled a new central processing unit (CPU) alongside an AI system built on technology from Groq, a chip startup from which the company licensed technology for $17 billion in December.
The strategy reflects Nvidia’s attempt to broaden its hardware ecosystem beyond graphics processing units (GPUs), which have traditionally been its core strength.
Huang explained that inference workloads will increasingly be divided into two stages. The first stage, known as “prefill”, involves converting user inputs into tokens, the language that AI models understand. This will be handled by Nvidia’s upcoming Vera Rubin chips.
The second stage, called “decode”, generates the actual response. This is where Groq’s specialised chips come into play, enabling faster and more efficient outputs.
Together, this two-step architecture aims to optimise how AI systems operate at scale, especially as demand continues to surge.
Rising Competition From CPUs And Custom Chips
The growing importance of inference is also intensifying competition.
While Nvidia’s GPUs have dominated AI training, inference is opening the door for alternatives such as CPUs and custom-built processors. Companies like Google are already developing their own chips, while Intel continues to push its CPU capabilities as viable solutions for AI deployment.
Huang acknowledged this changing landscape, noting that Nvidia is already seeing strong demand for standalone CPUs. He indicated that this segment is expected to become a multi-billion-dollar business for the company.
This diversification reflects Nvidia’s recognition that future AI infrastructure will not rely on a single type of chip but rather a combination of specialised processors working together.
Investors Watch Growth Sustainability
Nvidia’s aggressive push into inference comes at a time when investors have begun to question whether the company’s rapid growth can be sustained.
After a remarkable rally that saw Nvidia become the first company to reach a $5 trillion valuation last October, concerns have emerged about whether heavy reinvestment into AI infrastructure will deliver proportional returns.
The company’s latest projections appear aimed at addressing those concerns. One of the key themes emerging from Nvidia’s announcements is the transition of AI from experimentation to large-scale deployment.
Major technology companies such as OpenAI, Anthropic, and Meta, which have already invested heavily in training AI models, are now focusing on serving hundreds of millions of users. This shift is driving demand for inference capabilities that can operate efficiently at scale.
