High Bandwidth Memory (HBM) has evolved once again to its latest incarnation since its debut in 2013. SK hynix began mass-producing HBM3, the 4th generation of HBM, after completing the customer evaluation process seven months after its development. This has allowed the company to maintain its leadership position in the market, which it has held since producing the product’s previous version, HBM2E.
We met the team behind HBM3’s development and mass production, including Wangsu Kim (IPM Planning), Myeong-Jae Park (HBM Design), Kwi Wook Kim (Graphic&HBM Development PI), Jinwoo Park (WLP PKG Product) and Jinwon Huh (HBM/Graphics Enablement). We also took a closer look at the development process and heard the behind-the-scenes story.
The industry’s fastest processing, largest storage and enhanced reliability with error correction code
HBM is a premium memory product that secures more data transmission paths (I/O) than existing DRAM products, which also translates into a high bandwidth. The advantage is that the product processes data very quickly and is also able to send large amounts of data simultaneously.
HBM’s processing speed has increased with each generational upgrade. The per-pin data rate, which was 1Gbps in the first generation HBM, shot up to 6.4Gbps in the fourth generation of HBM. This improvement means that HBM3 is capable of processing 819GB of data per second, equivalent to processing 163 full-HD (5GB) movies in a second and almost twice as fast as HBM2E’s 3.6Gbps capability.
Storage has also increased with each generation. Whilst HBM2E’s maximum storage capacity was 16GB, HBM3 will be available in both a 24GB storage capacity, currently the largest in the industry, and a 16GB product. On-die Error Correction Code (ECC), which self-corrects data errors delivered to DRAM cells, will also improve product reliability.
The ‘Through Silicon Via’ (TSV) technology was critical for SK hynix to achieve all these metrics. TSV technology punches thousands of micro holes on DRAM chips and connects several chips via the electrodes running through these holes to deliver data. This technology enables a faster signal transmission speed, which makes it easier to secure density compared to the existing method of transferring data by connecting chips with external wires.
A key distinguisher between HBM3 and HBM2E is that the product specification (SPEC) itself has been upgraded in the former. An entirely new specification had to be set after it became impossible to improve the processing speed and storage capacity with that of HBM2.
▲Wangsu Kim, Professional Leader at IPM Planning
Wangsu Kim, in charge of HBM3 product planning, explained, “In order to enhance HBM2E’s performance at HBM2’s specification, we needed to increase either the product size or power consumption. Therefore, as we developed HBM3, we set and implemented a new specification that boosted performance whilst maintaining the same conditions as the existing one.”
“We also see the potential to further improve performance within the same specification, in much the same way as we derived HBM2E from HBM2,” he added.
The key to SK hynix’s leadership of the HBM market is ‘airtight, prior preparation’ and the ‘developers’ strong bond’
What was SK hynix’s secret weapon to maintaining its top tier position in the HBM market? The answer, according to the development team members, is ‘thorough preparation ahead of time’ and a ‘team vibe emphasizing co-operation during the development process”.
“The development team got involved right from the product design stage, and we were able to secure the necessary elements and technologies ahead of time,” said Kwi Wook Kim, who oversaw the development of HBM3 devices. “We were able to set our target really high because everyone helped each other regardless of roles, even as we faced challenges during the development process.”
▲Kwi Wook Kim, Professional Leader at Graphic&HBM Development PI
“There was one time when it took us a few months to settle an argument over the micro- unit spacing between cells. After some heated debates during our meetings, we often met up after work to drink together to relieve the tension (laughs). We all did our best in our respective areas, and it was a matter of seeing whether these efforts would translate to the best, most efficient results. All of these processes and the time that we spent spurred us on to develop the best possible HBM3 product.”
The co-operation between the product planning and architecture design teams was particularly important during the process of structuring the HBM3 specification and registering it as an international standard with JEDEC (Joint Electron Device Engineering Council).
“When we developed HBM3, we had to set the technology trend as a ‘first mover’ without the benefit of any prior products as a reference. Therefore, we had to set the product’s own specification whilst simultaneously standardizing it with JEDEC,” said Myeong-Jae Park, Professional Leader at HBM Design.
▲Myeong-Jae Park, Professional Leader at HBM Design
“The hardest part of the process was setting the specification while broadening the design scope. At one point, we were unable to implement the specification that we had negotiated with JEDEC during the development process, so we had to re-set it and persuade JEDEC to accept the changes. We braced ourselves for every possibility as we prepared for the JEDEC negotiations and a marathon meeting with the product design department. We even wrote manuals scripting out various possibilities during the negotiation process down to the smallest detail. We even mentally prepared ourselves for the worst-case scenario, that we would have to start from scratch, but fortunately the negotiations with JEDEC were successful.”
Developing the product as per the design also proved challenging, with many trials and errors along a path previously untrodden. Since HBM3 has the narrowest pitch (minimum center-to-center distance between interconnect lines), it was the most difficult to develop from a technical standpoint among SK hynix’s products. A defect previously unseen by the packaging department in previous pitches also made the development process more challenging.
“It is very time and energy-consuming to investigate what is causing these defects by changing minor motional mechanisms and figures individually, via a lot of trial and error, at the experiment level,” said Jinwoo Park, Professional Leader at WLP PKG Product, who was in charge of developing the package. “Various teams, including design, device, quality assurance, and GSM, also helped in the experiment process and provided a lot of assistance, which was a big help,” he added.
▲Jinwoo Park, Professional Leader at WLP PKG Product
“I used to get stressed as the development deadline approached but I had not yet identified which part was causing the defect, even after hundreds of experiments. Whenever I felt stuck, I would take a quick break and a colleague from another development team left a cup of coffee on my desk with a note that said, “Hang in there”. My colleagues’ encouragement helped me to overcome the challenges when I was ready to give up. To pay it forward, I often came up with new ideas for experiments whilst reviewing data that could help other development teams.”
SK hynix secures a competitive advantage thanks to the world’s first HBM3 mass-production… “The next goal is to maintain the No.1 position”
HBM is installed as a System in Package (SiP) in the client’s products, and it takes a long time for the client to check the product’s operations and performance during the validation stage. However, once the checking process is complete, it opens the door to nabbing an advantageous position at the early market stage and gaining a head start over the competition. SK hynix’s HBM2E products were able to secure a huge market share during the initial stages of their market entry thanks to this distinctive market characteristic.
“SK hynix once again gained a competitive advantage by developing and mass-producing HBM3 alongside HBM2E for the first time in the world,” said Kwi Wook Kim. “Although the market situation will certainly also play a role, this pre-emption effect could last for a considerable period of time,” he predicted.
▲Jinwon Huh, Professional Leader at HBM/Graphics Enablement
Although it would be easy to rest on their laurels, the HBM3 developers are already preparing to take the next step.
“It is essential for us to co-operate with clients such as SoC (System on Chip) companies before completing development of these HBM products. The quality control departments’ continuous feedback, along with those from the development team, is the key to achieving the best performance and quality,” said Jinwon Huh, Professional Leader at HBM/Graphics Enablement, who oversees HBM3 validation.
“In order to maintain our competitiveness in the HBM market, we will do our best to resolve any issues occurring during product certification quickly, and further contribute to expanding the market for HBM products.”
SK hynix is also planning to share its successful experience as a ‘first mover’ of the HBM market at the corporate level.
“The most important asset that we have from developing HBM3 is the experience of being a ‘first mover’ leading the market,” said Wangsu Kim. “We hope this experience plays a big role in the success of SK hynix’s future development by spreading a positive company culture.”
“The AI era is also approaching much faster than expected. The technology is already impacting various sectors, as evidenced by the emergence of real-time translation services and various curation services. Implementing these technologies requires massive amounts of data and increasingly larger AI model sizes. We expect that clients will increasingly demand technology that enables saving data more efficiently and quickly. HBM is the solution that can meet this demand at the highest technology level currently in existence.”
“HBM3 has nearly double HBM2E’s bandwidth. This means that once HBM3 is commercialized, service quality will improve by more than two times the current metric. SK hynix will also ramp up efforts to develop memory solutions with even better performance. The belief that memory solutions can be improved rapidly and continuously opens a plethora of opportunities for IT companies. These technologies will also increase the quality of life, and we want to change the world by developing them.”
