SEOUL, July 26 — The world of artificial intelligence is driven not just by clever algorithms, but by the physical hardware that feeds data to processors. A quiet but critical piece of that infrastructure, High Bandwidth Memory, is evolving at a pace that directly shapes how powerful AI models can become. And the next generation, HBM4, is already casting its shadow ahead of a 2026 arrival.
High Bandwidth Memory is a specialised type of DRAM that stacks multiple memory dies vertically, connecting them through tiny channels called through-silicon vias. This stacking gives it a far wider data pipeline than traditional graphics memory.
The technology has been climbing steadily: HBM2, launched in 2016, offered up to 256 GB/s per stack. HBM2E, which appeared in 2019, pushed that to 460 GB/s. HBM3, arriving in 2022, reached 819 GB/s.
And the current generation, HBM3E, which went into mass production in March 2024 at SK Hynix, now delivers over 1 TB/s. Those first HBM3E units shipped for Nvidia’s H200 GPU.
Nvidia’s appetite for bandwidth
The AI boom has turned memory bandwidth into a strategic resource. In 2023, Nvidia consumed over 90% of all HBM produced. The company’s reliance on these high-speed stacks skyrocketed as training large language models demanded ever-larger data flows.
SK Hynix, the lead supplier, has secured a deal with Nvidia for HBM4, giving the chipmaker priority allocation. That matters because demand is expected to outstrip supply until at least 2027.
Competing suppliers Samsung and Micron are also developing HBM4, but they trail SK Hynix in yield and qualification cycles. HBM4, expected in 2026, will bring a new 12-layer design and deliver up to 1.6 TB/s per stack. To put that in perspective, training a model like GPT-5 — estimated at 10 trillion parameters — would require memory bandwidth on that scale just to reduce communication overhead between processors.
What to watch next
The memory race is far from over. SK Hynix’s early lead in HBM4 gives it a strong position, but Samsung and Micron are working to close the gap. For now, the encouraging part is that each new generation of HBM pushes the boundaries of what AI hardware can do.
Patients and researchers watching the field with cautious hope — whether in healthcare, climate modelling, or language processing — can take comfort that the underlying technology is evolving deliberately, not with hype but with measurable improvements in speed and capacity. As with any fast-moving technology, it pays to stay informed, but for AI systems that touch our lives, the foundation is getting stronger.
Sources
{“@context”: “https://schema.org”, “@type”: “NewsArticle”, “headline”: “HBM Memory Bottleneck Slows AI Supercomputers: HBM2 to HBM4”, “description”: “SEOUL, July 26 — The world of artificial intelligence is driven not just by clever algorithms, but by the physical hardware that feeds data to…”, “datePublished”: “2026-07-26T09:30:00+00:00”, “author”: {“@type”: “Person”, “name”: “Muhammad Asim”}, “publisher”: {“@type”: “Organization”, “name”: “InfoPulse Today”, “url”: “https://newsfeed.infopulsetoday.com/”}, “citation”: [{“@type”: “CreativeWork”, “url”: “https://en.wikipedia.org/wiki/High_Bandwidth_Memory”}]}
