The AI explosion is driving demand for high-performance semiconductor memory that supports AI training and inference. In particular, HBM1, for which SK hynix holds the largest market share, has emerged as a leading AI memory solution. Featuring stacked layers of DRAM chips, HBM is contributing to increased demand in the DRAM market which is predicted to grow by nearly 65% year-on-year, reaching KRW 117 trillion (86 billion USD) in 2024.
1High Bandwidth Memory (HBM): A high-value, high-performance product that possesses much higher data processing speeds compared to existing DRAMs by vertically connecting multiple DRAMs with through-silicon via (TSV).
In light of this rapidly changing business environment, the newsroom invited newly appointed SK hynix executives to a roundtable to discuss the competitiveness underlying the company’s status as the global No. 1 AI memory provider and the company’s future direction.
For this final New Leadership Spotlight article, seven company vice-presidents who previously participated in the series took part in the roundtable: head of HBM Process Integration (PI) Unoh Kwon; head of Material Development Deoksin Kil; head of HBM Sales & Marketing (S&M) Kitae Kim; head of Advanced Package Development Hoyoung Son; head of NAND Advanced Process Integration (PI) Haesoon Oh; head of Performance and Reliability (PnR) for the 321-layer NAND flash Donghun Lee; and head of Global Revolutionary Technology Center Jaeyun Yi. The roundtable was moderated by Jeongho Won, who is head of Global PR.
The roundtable participants: (from left) Unoh Kwon (HBM PI), Kitae Kim (HBM Sales & Marketing), Donghun Lee (PnR for the 321-layer NAND flash), Haesoon Oh (NAND Advanced PI), Deoksin Kil (Material Development), Hoyoung Son (Advanced Package Development), and Jaeyun Yi (Global RTC)
How SK hynix Gained a Competitive Edge in the AI Era
- Long-term R&D and investment in HBM even before the market opened up
- Deep understanding and collaboration with customers
- Preparations for a future of heterogeneous convergence that breaks barriers between fields
SK hynix has taken the lead in the AI memory market through its groundbreaking technology. In March 2024, the company became the first in the world to mass-produce HBM3E, the fifth generation of HBM. SK hynix is also strengthening its position by bringing forward the planned mass production of its next-generation HBM4 to 2025, and securing advanced technology through global investments and cooperation with partners. To kick off the roundtable, the panelists discussed the background of SK hynix’s leadership position.
Kwon claimed SK hynix’s early investment and research in AI memory were key to growth
Kwon: “Our long-term investment and research in AI memory even before the market opened up have been the foundation for the company’s growth. Based on this, SK hynix develops high-performance memory products for various industries including HBM, which is essential for AI infrastructure. Our competitive advantages include proactively securing these types of technologies as well as our mass production experience.”
Kim: “SK hynix’s strength is that the company always collaborates with its customers on the front line and meets their needs based on a thorough understanding of them. We are highly trusted thanks to our timely delivery and experience in the mass production of HBM.”
Son: “HBM’s success is due to the company’s more flexible approach to collaborating not only with customers but also across departments. In order to meet the needs of a more diverse market, we need to take our collaboration with customers to the next level and prepare for heterogeneous integration which blurs the lines between memory and systems, as well as between semiconductor front-end and back-end processes.”
Lee highlighted the importance of high-capacity SSDs for AI
Another point of discussion was high-capacity NAND flash solutions.
Lee: “High-capacity SSD products based on NAND flash are essential for storing large amounts of data required for AI systems. SK hynix was the first in the industry to secure multi-plug technology, which reliably enables NAND flash’s data storage area even at ultra-high density, and All Peri. Under Cell (PUC) technology, which increases production efficiency by reducing product size. Both of these breakthroughs enhanced our competitiveness in this area.”
Oh: “SK hynix’s world-class ability to achieve cost-effectiveness along with its timely product development also allows the company to meet customer needs in the NAND flash field.”
Memory: The Key of AI Infrastructure Set for a Bigger Future Role
When asked why AI memory has been in such high demand, the executives unanimously pointed out that high-performance solutions such as HBM, CXL®2, eSSDs3, PIM4, and others are solving data bottlenecks in traditional memory and enhancing AI system’s operating speeds. They added that demand for high-performance, high-capacity memory will only continue to grow as AI applications expand.
2Compute Express Link® (CXL®): A technology that consolidates the interface between memory and other devices such as logic chip and storage into one to increase efficiency and enable high bandwidth and high capacity.
3Enterprise solid-state drive (eSSD): An SSD is a storage solution which combines NAND flash and a controller. Among the various types of SSDs available, eSSDs are designed specifically for enterprises.
4Processing-In-Memory (PIM): A new generation of intelligent memory that embeds the computational functions of a processor in memory.
Kim: “Memory will be utilized much more as generative AI technologies are being widely used in public services such as education and healthcare, as well as in B2C markets to analyze key consumer segments, develop products and services, run marketing campaigns, and even manage supply chains.”
Kil believes the convergence of various industries with AI will boost growth in the semiconductor market
Kil: “Generative AI technology itself is important, but it is much more crucial to see how many applications it can be implemented in and how it is delivered to consumers. In the future, the convergence of the on-device5, autonomous driving, and robotics industries with AI will drive changes and growth in the semiconductor market.”
5On-device AI: AI technology that implements AI functions on the device itself instead of proceeding with computation in a physically separated server. With on-device AI, AI functions become more responsive and more customized AI services can be provided as smart devices are capable of independently collecting and computing information.
Lee: “As AI technology becomes more advanced, we can envision a future where the metaverse becomes so sophisticated that it’s close to the real world. As big tech companies try to get a head start in this space, we could see an increase in memory demand.”
Oh stated that the proliferation of AI servers is boosting demand for NAND flash products such as eSSDs
The executives also discussed about how the expanding AI industry is opening up new memory markets.
Oh: “NAND flash hasn’t received much attention in the AI industry, but products such as eSSDs have started to gain traction as demand for large-scale AI servers has increased. New markets are opening up in various fields, resulting in numerous memory products gaining in popularity.”
Son: “Initially, it was difficult to predict that the AI memory market would develop in the way it has. Now, the industry is focusing on securing tailored memory solutions for customers considering various future possibilities.”
Yi: “There is also growing interest in emerging memories that go beyond the limits of conventional memories to obtain differentiated technologies. In particular, MRAM6, RRAM7, and PCM8, which combine the high-speed performance of conventional DRAM with the high-capacity characteristics of NAND flash, are increasingly in the spotlight.”
6Magnetic RAM (MRAM): Memory which uses the phenomenon that the resistance of an element changes depending on the direction of the magnetic spin motion to distinguish whether data is stored or not.
7Resistive RAM (RRAM): Memory that stores data based on the resistance difference created by applying voltage to the filaments in the device.
8Phase-change memory (PCM): A memory that stores data using the phenomenon of changing the phase when a voltage is applied to a specific material.
SK hynix’s Top Priorities in Response to Future Market Changes
The executives also had various opinions on the issues SK hynix should pay attention to in order to stay ahead of future industry and technology changes. First of all, they underlined the importance of strengthening relationships with customers and global cooperation to maintain the company’s technological superiority.
Kim called for the company to continue improving in both the front-end and back-end processes
Kim: “Even as AI services diversify, the core specifications required for memory will continue to be ‘speed’ and ‘capacity’ as long as there are no changes to the current structure of the Von Neumann9 architecture which separates logic and memory chips.” In order for SK hynix to maintain its edge in AI memory technology including HBM, it must not only improve its design, device, and product competitiveness in the front-end process, but also continue to develop its outstanding packaging technology such as its unrivaled high-level stacking in the back-end process.
9Von Neumann architecture: A program-embedded computer structure typically featuring three levels consisting of the main memory unit, a central processing unit, and an input/output unit. Most computers today follow this basic structure, but its bottleneck limits the ability to design high-speed computers.
“Our big tech customers are accelerating their new product release cycles to gain AI market leadership. We are already discussing plans for 2025 to ensure we can deliver next-generation HBM products in a timely manner.”
Kwon: “The upcoming HBM4 will be the first HBM solution to introduce a logic chip’s manufacturing process in memory. In addition to realizing specifications beyond what our customers want, the newly introduced process will lead to collaborations with related industries and create new opportunities.”
Son claims that advanced packaging technology is key to developments in conventional memory and heterogenous integration
Son: “To improve the performance of conventional memory and realize heterogeneous integration such as that between memory and logic chips, we need to elevate advanced packaging technology. To do this, we need to strengthen collaboration with global industry, academia, and research institutes in different fields on various subjects ranging from conventional memory to the convergence of semiconductors.”
In addition, the executives emphasized the importance of enhancing product quality through material development, improving high-performance NAND technology for AI, and conducting R&D for next-generation memory.
Kil: “We need to keep a close eye on the emerging AI and semiconductor markets and create a development environment that can flexibly respond to rapidly changing situations. Through innovation in semiconductor materials, we can simplify processes and improve defect rate control and UPH10. As for the packaging field, which is becoming increasingly crucial, it is vital to develop materials that enhance product performance and quality by boosting yield and heat dissipation properties.”
10Unit Per Hour (UPH): The amount of products manufactured per hour on a production line.
Lee: “The ability to store exponentially growing data in a limited space with as much data and as little power as possible is becoming vital. Therefore, high-capacity, low-power eSSDs are essential, while NAND flash for eSSDs needs to offer high performance, quality, and reliability when implementing 300 or more layers of ultra high-density storage.”
Oh: “The use of QLC11-based eSSDs in AI servers has recently grown as QLC technology is specialized for high capacity and complemented by rapid read and write speeds, while it also reduces customers’ total cost of ownership12. We will endeavor to develop timely products to target relevant markets.”
11Quadruple Level Cell (QLC): A form of NAND flash memory that can store up to 4 bits of data per memory cell.
12Total cost of ownership (TCO): The total costs of an asset that includes such as the initial investment, power, facility operations, and maintenance.
Yi revealed that SK hynix will focus on strengthening R&D in future technologies
Yi: “The global environment is changing so rapidly that it is difficult to predict whether the changes we anticipate will occur in the distant future or will soon become a reality. We are focusing on emerging memories such as SOM13, spin memory14, and synaptic memory15 which offer ultra-high speed, high capacity, and low power, as well as MRAM, RRAM, and PCM. Looking ahead, we will continue to strengthen R&D on various future technologies.”
13Selector-Only Memory (SOM): A memory that implements the capacitor (storage area) and transistor (control area), known as a selector, of conventional memory into a single device. This can help overcome the micro-processing limitations of conventional PCM.
14Spin memory: A memory that applies the spin motion characteristics of electrons to distinguish whether data is stored or not.
15Synaptic memory: Memory based on artificial neural network devices which solves limitations found with serial processing in the existing computing structure, such as data bottlenecks, by realizing a highly efficient computing structure similar to the human brain.
<Other articles from this series>
