One of the long-held desires of human beings is to save the view of wonderful scenery and objects for the future. For this, in prehistoric era, human beings painted murals in caves. Later, they painted figures or landscapes on a piece of cloth or paper and passed them onto future generations. After the invention of cameras, photographs replaced paintings for such purposes. Film cameras of the early stages evolved into digital cameras, and digital cameras evolved further to realize clearer images with smaller modules. Gradually, cameras became small enough to be mounted on mobile phones, as well as tiny drones and eyeglasses and began to adopt various functions such as augmented reality (AR). Some people are now seeing the full potential of the future and looking forward to one day achieving small chips implanted in our eyes to replace cameras. The story we are going to tell you now is about “Smart Eye”, something we might see in the future, thanks to the development of semiconductor technology.
“Smart Eye”, an All-round Eye with Various Functions beyond Vision
“Smart Eye”, a kind of artificial eye, is expected to be able to perform a variety of tasks including simple photo taking. Improving vision by allowing to see objects far away, or recognizing very small text or objects that were not previously visible to human eyes, are two of the most basic steps. For Smart Eye, it might also be possible to capture the moment you are looking by simply blinking your eyes or sharing your vision with your friend far away from you in real-time through a network connection.
Since it is a kind of electronic device, it can be equipped with various additional functions like smartphones by utilizing the embedded semiconductor chip. In particular, when combined with AR technology Smart Eye has the potential to enrich our daily lives. For example, you might be able to display translated texts right next to foreign text in your vision at anytime, anywhere. Additionally, when you see something unfamiliar, you might be able to utilize an encyclopedia function to automatically display information about that object, right next to it.
Useful information for daily life such as time and weather can be displayed in a virtual space in your field of vision, while a stored schedule or documents can be loaded and checked with just a blink of your eyes. After you leave your home, a virtual arrow pointing to your destination will show you the way to go. Navigating stores, along with detailed information won’t be a scene of sci-fi movies anymore. All of these are pleasant changes that “Smart Eye” might bring to our daily lives.
Figure. 1 Bionic Vision Technologies’ Bionic Eye System (Source: Official website of BIONIC VISION TECHNOLOGIES)
In the industry, “Smart Glasses”, considered to be just one step before Smart Eye, is already performing many functions as mentioned above. Furthermore, an Australian company named Bionic Vision Technologies had attempted to deliver camera images to the brain through an implanted chip behind the eye; the chip is connected to the eyeglass-shaped camera, and in the end the company was able to achieve some visible business results with this innovative attempt. In the near future, when these technologies become more sophisticated, this imaginary “Smart Eye” might be realized sooner than we expect now.
Will Numerous Ongoing Research Projects of “Artificial Eyeballs” Bring Complete Replacement for the Human Eyes?
When would Smart Eye be realized? To find this out, let’s first look at how far the research on artificial eyes has come in this journey so far.
In the current medical field, research on artificial organs has been actively carried out, contributing to the development of various artificial organs. Some meaningful results have been seen, as some of these artificial organs have been successfully transplanted to patients. Artificial eyes which might completely replace the human eyes one day, however, are still in an incomplete stage. This is because human eyes are one of the most sophisticated organs of human bodies.
An eye is composed of cornea and lens that refract light rays from the outside, iris that controls the amount of light passing through, pupil where light rays enter, retina where light images are formed, and optic nerve that transmits visual information to the brain. Currently, the researches on artificial eyes that uses heterologous tissues or patients’ cells have reached the stage of artificially fabricating each part of human eyes. This is the stage before making the entire eye artificially.
The latest technology in terms of cornea is held by Newcastle University in the UK which succeeded in making the artificial cornea by utilizing 3D bio-printing technology in 2018. In more detail, the research team produced Mixed Stem Cells composed of cornea, alginate, and collagen, and used a 3D bio-printer to print them into a form of a human cornea, which is the first artificial cornea in the world.
Among the attempts to build artificial eyes using electronic devices, “Argus II” developed in 2013 by an American company Second Sight is one of the most well-knowns. After obtaining approval from the US Food and Drug Administration (FDA) in 2013, Argus II was proven through clinical trials that it can partly improve human eyesight. However, there is a clear limitation as it can be applied only to some patients with certain eye diseases.
Figure. 2 Artificial eye developed by a research team at University of Minnesota (Source: Official website of University of Minnesota)
The most recent and noteworthy achievement made in this field was made by the research team at University of Minnesota. In 2018, the team succeeded in printing a receptor (a type of retina) that can receive light upon a transparent hemisphere shaped like an eyeball. With this technology, the receptor can convert light signals into electrical signals via photodiode and semiconductor. Although this “eye” has succeeded in receiving and recognizing light signals with help of semiconductor technologies, further improvement is still expected to produce commercially available artificial eyes.
The Role of Semiconductors in Realization of Smart Eye
Smart eye requires semiconductors, just like any other electronic devices. Specifically, artificial eyes require logic semiconductors to control and operate the system, as well as memory semiconductors to store data necessary for the system implementation. In the case of artificial eyes, the key mechanism is to convert light signals into electrical signals and transmit them to the optic nerve. This is the reason why the role of CMOS Image Sensor (CIS), which is frequently mounted on cameras of electronic devices to perform similar tasks, is crucial.
Figure. 3 The Working Mechanisms of CIS and Human Eyes
Like the retina of human eyes, CIS is a semiconductor that converts color and brightness of light into electrical signals to deliver them to the central processing unit. Inserted into various types of cameras, it serves as an eye for electronic devices.
Nevertheless, the current CIS technology has not reached the level of human eyes in terms of major features such as resolution, three-dimensionality, and sensitivity. While the resolution of human eyes is 576 megapixels (MP), the highest resolution CIS can currently realize is only 108Mp. When brightness of surrounding environment changes in a sudden, CIS is also likely to suffer from latency accepting visual information.
There are some areas where CIS is superior to human eyes, however. The human eyes’ field of view is approximately 110 to 120 degrees left and right, and 150 degrees up and down, whereas CIS can cover a full 360 degree view. It also has the potential to go beyond the limits of human eyes with certain functions, such as the telephoto function which allows seeing objects far away. Once CIS technologies reach the level of human eyes in terms of resolution and sensitivity, it can not only just replace human eyes, but also present various advanced functions that cannot be delivered through naked eyes.
Ho-young Cho, Technical Leader (TL) at CIS Marketing Strategy of SK hynix said, “While human eyes’ main purpose is to recognize rather than display the collected visual information, CIS is designed for securing visual information for output. If CIS can recognize at the same level as human eyes do in the future, it will also function as a displaying device that outputs the collected information.”
Cho continued, “Unlike human eyes, CIS is designed as individual modules for various purposes. As a result, CIS is detachable, and users can equip different CIS depending on various situations. Such flexibility will make our daily lives more convenient with no doubt.”
1BioINwatch 19-51: Core Technology for Bio Artificial Organs, “Organoids”, Biotech Policy Research Center (2019)
22017 Technology Impact Assessment Result Report: The Future of Bio-Artificial Organs, Korea Institute of Science & Technology Evaluation and Planning, Ministry of Science and ICT (2018)
32017 Promising Future Technology Program Organ Bio, National Research Foundation of Korea (2017)
43D Printing Technology for Biomedical Applications, So Hyeon Park, Sang Gu Yim, Seung Yun Yang, and Se Hyun Kim, No 1, Volume 18 (2015), Prospectives of Industrial Chemistry (2015)
5Series 19: Artificial Organs for New Life, “Scientific Technology Told by Great Scholars”, The Korean Academy of Science and Technology, Young Ha Kim, Free Academy
6Medical Device Item Market Report: Intraocular Lens, Kyung-hwa Ko, Korea Health Industry Development Institute (2013)
73D bioprinting of a corneal stroma equivalent, Abigail Isaacson, Stephen Swioklo, Che J Connon, Experimental Eye Research(2018)
8Research Brief: Researchers 3D print prototype for ‘bionic eye’, University of Minesota(https://twin-cities.umn.edu/research-brief-researchers-3d-print-prototype-bionic-eye)
9Korea Institute of Science and Technology (KIST) website, https://www.kist.re.kr
10Korea Institute of S&T Evaluation and Planning (KISTEP) website, https://www.kistep.re.kr
