Technology

[The World Shaped by Semiconductors] Virtual Reality in Glasses

By December 5, 2019 December 6th, 2019 No Comments

As soon as I open my eyes in the morning, I put on my glasses to search for today’s weather forecast. As the virtual reality (VR) function is equipped to glasses nowadays, I can experience the outside weather without going outside. Sunny and cool autumn weather makes me feel refreshed. After taking a shower, I choose some clothes to wear. In the old days, I had to take out clothes from the wardrobe and see what I look like in front of a mirror; but now, I can have a look at myself in the clothes in a virtual dressing room with my glasses. I choose a neat style recommended by my Artificial Intelligence (AI) secretary. In the subway on my way to work, I sit in the corner and enjoy the last week’s live concert of my favorite singer. As I can access the VR I want, with my glasses anytime and anywhere, my long commutes are not boring anymore.

To the Virtual World by Just Wearing Glasses… “Smart Glasses” with a VR function, When Would They Be Commercialized?

A daily life with virtual reality, everyone has imagined this at least once. But this is a technology of the future that is not yet feasible in reality. Nowadays, it is easy to play simple virtual games or to experience VR video content through a VR Head Mount Display (HMD) connected to a PC, a smartphone or a console. However, the “smart glasses,” which enable VR at any time or anywhere, can yet only be seen in films.

Nonetheless, it is not right to consider smart glasses as Science Fiction (SF) which will be only possible in the remote future. As IT technologies are being rapidly developed with the arrival of the 4th Industrial Revolution, the technologies required for VR are now being released one by one. As global IT companies are actively entering the VR industry these days, it is expected that smart glasses in movies will be realized in reality in the near future.

Image Download

The volume of the related market is expected to grow rapidly as well. Digital Capital, a British market research institution specialized in VR and augmented reality (AR), predicted that the volume of the global VR market will increase quickly from USD 3.9 billion in 2016 to USD 105 billion in 2022. Another global market research institution, IDC, also expected that the volume of shipments of the related hardware (HMD) will increase from 9.6 million units in 2017 to 59.2 million units in 2021.

Today, Facebook is the most active company in developing smart glasses. Since the acquisition of an HMD-developing startup named Oculus in 2014, Facebook has been expanding its VR-related product lineups, jumping up to be a market leader in the VR HMD-based hardware platform development. Recently, it started to develop a next-generation AR smart glasses code-named Orion, aiming to commercialize them in 2025. Mark Zuckerberg, CEO of Facebook predicted the future of smart glasses at the annual developer conference in 2017, saying that smart glasses will replace smartphones in five to seven years.

(Credit: Vive Newsroom)

 

HTC and Sony are also considered as the market leaders as well. HTC has been showing notable movements recently in the high-end VR HMD market, while Sony has been presenting VR HMD based on its own gaming platform PlayStation. Especially, without considering cost-effectiveness, Vive Pro launched by HTC in 2018 is one of the best VR HMDs with the highest specification ever, as it boasts a resolution of 2880 x 1600 pixels, a field of view (FOV) of 110 degrees, and a refresh rate of 90Hz. In South Korea, Samsung Electronics is joining the related market by a smartphone-based VR HMD called “Gear VR” through collaboration with Oculus.

Along with the VR HMD, technological developments regarding Eye Glassed-type Display (EGD) based on the AR and mixed reality (MR) are being made continuously. Some of the most recognizable companies are Magic Leap and Microsoft.

Magic Leap succeeded in developing a “Photonic Lightfield” technology, which controls the direction of light reaching the retina, and unveiled its MR EGD named “Magic Leap One” in 2018, which makes virtual objects look more realistic. Microsoft also raised expectations of the market by unveiling its next-generation AR EGD, HoloLens 2, at CES this year.

Nonetheless, Magic Leap One: Creator Edition is aimed mostly at developers, as you can see from its name. Microsoft is also currently focusing on the AR EGD market for enterprise rather than for consumers, suggesting that commercialization of AR glasses would require more time.

Current Status of “HMD,” a Prototype of Smart Glasses

In general, for implementation of VR, convex lenses are placed in front of a user’s eyes, so that focal points can be adjusted to give a three-dimensional effect and a sense of distance to images delivered by an exterior display. In other words, this method utilizes a type of optical illusion, making the brain think the virtual images as real. With currently available technologies, the user’s vision should be fully covered, as a sense of immersion can be raised when the visual information in the reality is blocked from the user’s view. Furthermore, there should be some space for the display device, as well as enough space between lenses and the display to adjust the focal distance. This is the reason why all the commercialized VR HMDs so far are adopting a design element of a goggle shape.

This goggle-shaped VR HMD is far too heavy and uncomfortable to wear for long in daily life. To evolve from this to the shape of glasses, the first step should be the weight reduction of peripheral devices, including sensors and batteries. Along with this, development of other related technologies is required as well, such as optical technology and display technology.

Image Download

Nevertheless, current optical and display technologies have clear limitations. To allow a user to be immersed in VR without any blocking of exterior visual information, the FOV of HMD should be able to cover 220 degrees, the maximum FOV of human eyes. However, even high-end products of the major HMD manufacturers in the current market only cover around 110 degrees, which is less than the average FOV of human eyes, i.e., 120 degrees.

The current level of image resolution is not good enough for allowing users to sense the released images as reality. To maximize the sense of immersion, a high-resolution over at least 4K (3840 x 2160 pixels) is required. Especially, in the case of a VR HMD which shows images with a wider FOV than previous displays, the Screen Door Effect, where fine black lines between the pixels make you feel like you’re looking through a mesh screen, should be considered as well. This means that, in 360-degree VR, the resolution sensed by users can be much lower than that of the original videos. To solve this problem, it is expected that high-resolution videos over at least 8K (7680 x 4320 pixels) are required.

Image Download

Securing a stable frequency of display (refresh rate) is crucial to minimize 3D motion sickness that can occur when using VR HMD for a long time. While the industry defines the refresh rate for the optimal VR technology as 120Hz, current products based on Organic Light Emitting Diodes (OLED) have refresh rates of around 90Hz.

“Development of Semiconductor Technology,” a Prerequisite for Commercialization of Smart Glasses

The development of semiconductor technology is one of the most important factors for the commercialization of smart glasses. Even with the great image resolution and display technology, the commercialization of smart glasses is still far-off without semiconductor performance supporting and implementing them. For this reason, in the semiconductor industry, diverse efforts are now being made to secure technologies to lead the VR sector.

(Credit: Qualcomm Website)

 

In the Application Processor (AP) sector, Qualcomm stands unchallenged. Qualcomm’s Snapdragon 835 was adopted by both of two high-end Standalone VR HMD products in the current market, Facebook’s Oculus Quest and HTC’s Vive Focus. Meanwhile, Intel is actively focusing on VR systems for professionals, based on PC-powered VR HMD, with its high-performance CPU “Core X Series.”

AMD is targeting the global VR system market in the GPU field, with its LiquidVR, a solution exclusively for VR, and AMD Radeon VR Ready Premium / Creator, its GPU certification programs. It also acquired a wireless VR chip manufacturing startup named Nitero in 2017 to secure technologies.

Technological development of memory semiconductors is also a prerequisite of the realization of smart glasses. It is because memory semiconductors are used during the entire process of providing a realistic VR environment for users; from producing and saving VR images and securing storage space within a device, to releasing images on a display. Especially, the development of high-performance graphic DRAM is essential for receiving large amounts of data of 8K images in real time, to show this to users in a stable way.

In general, one piece of Full HD (2K) image accounts for around 9.5Mbyte. It increases up to 16 times for 8K (UHD). In addition, VR images consume around four times as much data as ordinary UHD images, as VR images should be displayed in an enlarged form, in a short distance from eyes. In other words, a huge amount of data which is at least 64 times larger than ordinary Full HD images should be processed in real-time for VR images.

Image Download

To meet these conditions, how much more should the graphic DRAM’s performance for VR supporting system be developed? According to SK hynix’s own estimation, 8K images can be processed stably without any delay through a graphic DRAM with a data transfer rate of over 20Gbps for videos in high resolution over 8K with more than 90 frames per second. It means that a high-speed graphic DRAM like GDDR6, the latest one released, is required. For memory semiconductor adopted within the HMD device itself, a design with efficiency and low power consumption is necessary as well, so that data can be processed rapidly in a subminiature chip equipped inside the glass frame.

Seoyeong Jeong, Technical Leader of Graphic/Auto team at DRAM Product Planning department of SK hynix said, “As VR technologies are likely to continue to be sophisticated and used in a wider variety of applications, it is expected that the volume of data to be processed will increase exponentially. To respond to this trend of the future market, SK hynix is making continuous investment in the high-bandwidth, low-power memory semiconductor.”