{"id":12397,"date":"2026-08-18T08:59:58","date_gmt":"2026-08-17T23:59:58","guid":{"rendered":"https:\/\/news.skhynix.com\/en\/?p=12397"},"modified":"2026-08-18T08:59:58","modified_gmt":"2026-08-17T23:59:58","slug":"ai-ecosystem-series-ep1","status":"publish","type":"post","link":"https:\/\/news.skhynix.com\/en\/ai-ecosystem-series-ep1\/","title":{"rendered":"[AI Ecosystem] The paradigm shift in AI computing"},"content":{"rendered":"
\n

As AI workloads evolve toward inference and agentic AI, the criteria for system performance are also changing. Computing power alone is no longer enough. Where data is stored, how quickly it moves, and how efficiently it is processed now have a direct impact on the performance and efficiency of AI systems.
\nThis article series explores the transformation of the AI ecosystem, where software, data center infrastructure, semiconductors, and memory technologies work together. It also examines why memory semiconductors have become a foundational technology for the AI era and outlines SK hynix\u2019s vision for building the next-generation AI memory ecosystem.<\/p>\n

<\/div>\n

\u2460 The paradigm shift in AI computing \u2013 Professor Jinwoo Shin, KAIST<\/strong>
\n\u2461 The real bottleneck: Data, not compute
\n\u2462 Redesigning infrastructure: Why architecture determines performance
\n\u2463 The semiconductor paradigm shift: Memory\u2019s growing role
\n\u2464 Completing AI: Physical intelligence and the role of memory<\/p>\n<\/div>\n

<\/div>\n
\n
\n
<\/div>\n

From training to inference, from AI that answers to AI that acts<\/h3>\n
<\/div>\n<\/div>\n<\/div>\n

Since the launch of ChatGPT in late 2022, AI has spread rapidly across industries and everyday life. Yet one of the most significant changes taking place within the AI industry remains largely unnoticed. The long-held approach of advancing AI by training ever-larger models on ever-growing datasets \u2014 known as Scaling Laws*<\/span> \u2014 is beginning to reach its limits.<\/p>\n

\"\"<\/p>\n

The focus of AI competition is shifting from how much a model has learned to how well it can reason, and ultimately to how effectively it can act on its own. AI is evolving from training-centric systems to reasoning-centric systems, and from tools that simply generate answers to systems capable of autonomous action.<\/p>\n

This transition extends far beyond AI model development. It is also redefining the requirements for the semiconductor and memory technologies that underpin AI infrastructure. This article explores the nature of this paradigm shift.<\/p>\n

* <\/span>Scaling Laws: The principle that AI model performance improves proportionally as more computing resources, data, and model parameters are used.<\/div>\n
<\/div>\n
\n
\n
<\/div>\n

The era of Scaling Laws \u2014 and what comes next<\/h3>\n
<\/div>\n<\/div>\n<\/div>\n

ChatGPT\u2019s success was built on three key pillars: the Transformer*<\/span> architecture, which enabled large-scale neural network training; the pre-training Scaling Laws, which showed that performance improves as models and datasets grow; and post-training*<\/span>, which aligns models with human intent. The scale of this approach is evident in reports that Meta devoted approximately 30 million GPU hours to training Llama 3, underscoring the enormous computing resources required.<\/p>\n

The formula was simple but remarkably effective. It fueled the rapid emergence of frontier models such as ChatGPT, Claude, and Gemini. Since 2024, however, two fundamental limitations have become increasingly apparent.<\/p>\n

* <\/span>Transformer: A neural network architecture introduced by Google in 2017. Built around the self-attention mechanism, it enables models to understand contextual relationships and serves as the foundation for virtually all large language models (LLMs) today.
\n* <\/span>Post-training: The stage following pre-training in which a base LLM is refined using carefully curated examples so that it can follow user instructions and produce desired behaviors. This process enables instruction-following and conversational AI models.<\/div>\n

\"\"<\/p>\n

The first challenge is data scarcity<\/strong>. Ilya Sutskever, one of the pioneers behind ChatGPT, argued that \u201cpre-training as we know it will end\u201d during his acceptance speech for the Test-of-Time Award*<\/span> at NeurIPS 2024, one of the world\u2019s leading conferences on AI and machine learning. While computing resources can continue to grow through better hardware, improved algorithms, and larger computing clusters, the supply of high-quality training data cannot expand in the same way. Sutskever compared internet text to \u201cthe fossil fuel of AI.\u201d Researchers have similarly projected that by around 2028, the total amount of text that can be extracted from the internet will roughly match the amount already used to train AI models.<\/p>\n

* <\/span>Test-of-Time Award: Prestigious award recognizing research that has demonstrated lasting impact over time, typically honoring papers published at least 10 years earlier.<\/div>\n

The second challenge is cost efficiency<\/strong>. Simply doubling a model\u2019s size does not double its performance. Instead, each additional increase in model size, dataset size, or computing power produces a smaller improvement than the one before. If greater gains can be achieved with the same investment through other approaches, there is less reason to devote ever-increasing resources solely to large-scale pre-training.<\/p>\n

Together, these constraints are reshaping the direction of AI development. Rather than continuing to scale models and datasets indefinitely, the industry is increasingly investing more computing resources in the reasoning phase \u2014 when a model generates an answer.<\/p>\n

This does not mean AI progress is slowing. It means the path to progress is changing.<\/p>\n

In the past, the most reliable way to improve performance was to train larger models for longer periods. Today, the emphasis is shifting toward how effectively a model can apply and maximize the knowledge it has already learned during inference.<\/p>\n

Even the same model can produce very different results depending on how a problem is decomposed, how carefully intermediate reasoning steps are carried out and how effectively external tools are used when needed. In other words, the center of AI competition is moving from the training phase to the inference phase.<\/p>\n

<\/div>\n
\n
\n
<\/div>\n

The rise of reasoning: The third Scaling Law<\/h3>\n
<\/div>\n<\/div>\n<\/div>\n

Across several keynote presentations, NVIDIA Founder and CEO Jensen Huang has used the phrase \u201cfrom one to three\u201d to describe a fundamental shift in AI\u2019s development process. AI performance was once explained primarily through a single Scaling Law centered on pre-training, but today, it is increasingly driven by three complementary forms of scaling: pre-training, post-training, and test-time (Inference-time) scaling.<\/p>\n

At the center of this shift is reasoning during inference. Just as people do not immediately blurt out the answer to a difficult math problem but instead work through it step by step, AI models can achieve significantly better results by performing a longer and more sophisticated reasoning process before generating a response. One of the most influential techniques enabling this capability is Chain of Thought*<\/span>.<\/p>\n

Reasoning-first models such as OpenAI\u2019s o1 and o3 series and DeepSeek-R1 exemplify this trend. Among them, o1 clearly demonstrates how inference-time computation can dramatically improve performance. On AIME*<\/span> 2024, a benchmark based on competition-level mathematics, the general-purpose multimodal model GPT-4o scored 13.4%, while o1 achieved 83.3%. On GPQA Diamond*<\/span>, a benchmark consisting of Ph.D.-level scientific questions, o1 scored 78%, surpassing the average human expert accuracy of 69.7%. These results suggest that allocating more computation to the reasoning process itself can substantially improve model performance.<\/p>\n

Another milestone came in September 2025, when a paper on DeepSeek-R1 from Chinese startup DeepSeek was featured on the cover of \u201cNature.\u201d In the paper, the researchers challenged a widely held assumption about how advanced reasoning models are trained. Previously, advanced reasoning capabilities were widely believed to require large volumes of expensive supervised fine-tuning (SFT) data. The DeepSeek research team, however, demonstrated that reinforcement learning alone could enable a model to develop its own chain-of-thought reasoning process. The work suggested that sophisticated reasoning capabilities are no longer exclusive to a handful of major technology companies.<\/p>\n

\"\"<\/p>\n

\u25b2 Image Source: Cover of Nature, Vol. 645, Issue 8081 (18 September 2025)<\/a>.<\/p>\n

This shift is also changing how AI systems perform computation.<\/p>\n

Conventional LLMs are primarily optimized to generate relatively short responses as quickly as possible. By contrast, reasoning models may perform internal reasoning equivalent to hundreds or even thousands of tokens before producing a final answer. Measurements show that for a simple arithmetic problem that a conventional model typically answers using 7-12 output tokens, reasoning models such as o1 and DeepSeek-R1 can consume more than 900 tokens before arriving at the same answer. In other words, the number of output tokens processed by the same GPU can increase by nearly 100 times. Inference is no longer simply the execution stage \u2014 it is becoming one of the primary computational workloads in AI services.<\/p>\n

The economics of AI services are changing as well.<\/p>\n

Pre-training requires a massive upfront investment in compute. Inference costs, however, are incurred every time a user submits a query. As AI systems spend more time reasoning and execute more intermediate steps before generating an answer, the computational demand and infrastructure burden of operating AI services continue to grow.<\/p>\n

The rise of reasoning-centric AI therefore represents far more than the arrival of smarter models. It is transforming the economics of AI services, response latency, user experience, and the way AI infrastructure itself is designed.<\/p>\n

* <\/span>Chain of Thought (CoT): A prompting and generation technique in which a model explicitly works through intermediate reasoning steps before producing a final answer, improving reasoning accuracy.
\n* <\/span>American Invitational Mathematics Examination (AIME): A highly challenging mathematics competition administered by the Mathematical Association of America (MAA). It is widely used as a benchmark for evaluating complex mathematical reasoning in AI models.
\n* <\/span>Graduate-Level Google-Proof Q&A Diamond (GPQA Diamond): A benchmark consisting of Ph.D.-level questions in biology, chemistry, and physics, used to evaluate scientific reasoning capabilities in AI models.<\/div>\n
<\/div>\n
\n
\n
<\/div>\n

Agentic AI: From answering questions to taking action<\/h3>\n
<\/div>\n<\/div>\n<\/div>\n

As AI reasoning capabilities improve, the way people use AI is changing as well. At the center of this shift is agentic AI. Unlike a conversational model that simply responds to a user\u2019s prompt, an agentic AI system can receive a goal, develop its own plan, invoke the tools it needs, evaluate the results, and complete a task autonomously.<\/p>\n

Consider a simple request: \u201cBook a Korean restaurant near Times Square for two people tonight.\u201d Rather than generating a single response, an agentic AI breaks the request into multiple subtasks \u2014 searching for restaurants, comparing options, checking business hours, finding available reservation times, incorporating the user\u2019s preferences, making the reservation, and adding it to a calendar. At each step, it can call external tools such as search engines, maps, reservation application programming interfaces (APIs), and calendar services as needed. Tasks that once required users to search, compare, and enter information manually can instead be carried out by AI.<\/p>\n

\"\"<\/p>\n

This highlights the difference between chatbots and agentic AI. Chatbots are designed primarily to respond to user queries. Agentic AI, by contrast, is designed to accomplish goals. Rather than requiring users to specify every step, it decomposes a task, determines the sequence of actions, gathers the necessary information, evaluates intermediate results, and decides what to do next. In the process, AI expands beyond a standalone language model to work with search engines, code execution environments, databases, enterprise software, and even robotic control systems. This is why agentic AI is gaining traction: AI is evolving from a conversational interface into an active participant in digital work.<\/p>\n

The market has already begun to recognize this trend. In its 2024 Hype Cycle*<\/span> for Artificial Intelligence, Gartner classified AI Agents at the Peak of Inflated Expectations.*<\/span> The same report also identified World Models*<\/span> and Embodied AI*<\/span> as key technologies in the early stages of innovation. Together, these developments suggest that AI capable of taking action \u2014 not simply generating responses \u2014 could reshape industries over the next 5-10 years.<\/p>\n

* <\/span>Gartner Hype Cycle: A framework that visualizes the maturity and adoption of emerging technologies through five stages: Technology Trigger, Peak of Inflated Expectations, Trough of Disillusionment, Slope of Enlightenment, and Plateau of Productivity.
\n* <\/span>Peak of Inflated Expectations: A stage in which interest and expectations surrounding a new technology rise rapidly. While early success stories receive significant attention, the technology\u2019s actual maturity and commercial impact often fall short of market expectations.
\n* <\/span>World Models: AI systems designed to understand how the physical world works, including physical laws and spatial relationships. By learning from text, images, video, audio and motion, they can predict future states and infer what is likely to happen next.
\n* <\/span>Embodied AI: AI integrated into physical systems that can interact with the real world. Applications include general-purpose robots, humanoid robots, autonomous vehicles, factories and logistics centers. By combining machine learning with sensors and computer vision, embodied AI can perceive its surroundings, make decisions and take actions autonomously.<\/div>\n

Agentic AI also changes the nature of computing demand. A single user request may trigger dozens of LLM calls and dozens of tool invocations behind the scenes. While reasoning models increase the amount of computation devoted to thinking through a problem, agentic AI dramatically increases the number of interactions required to complete a task. As reasoning and action converge, the total number of tokens that data centers must process grows substantially.<\/p>\n

\"\"<\/p>\n

The past decade of AI development can be summarized in a single phrase: build bigger models. The next phase, however, will be far more complex. The center of competition is shifting from pre-training to post-training and now to Inference-time computation. At the same time, AI is evolving from a system that provides one-off responses into an autonomous agent capable of carrying out tasks. The standards for evaluating AI are changing as well. Success is no longer measured only by how often a model produces the correct answer, but also by how consistently it reasons, how safely it acts, and how reliably it delivers trustworthy results.<\/p>\n

<\/div>\n
\n
\n
<\/div>\n

How will AI infrastructure keep pace with this evolution?<\/h3>\n
<\/div>\n<\/div>\n<\/div>\n

Reasoning models and agentic AI are fundamentally changing both the volume and the nature of AI computation. Responses are becoming longer, model and tool invocations are increasing, and context must be maintained over much longer interactions. As a result, simply adding more compute is no longer enough to improve AI system performance.<\/p>\n

Part 2 of the series will examine this challenge from a system-level perspective by answering a fundamental question: What is the true bottleneck in AI performance?<\/p>\n

\"\"<\/p>\n

Disclaimer:<\/strong> The opinions expressed in this article are solely those of the author and do not necessarily reflect the official position of SK hynix.<\/span><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"

As AI workloads evolve toward inference and agentic AI, the criteria for system performance are also changing. Computing power alone is no longer enough. Where data is stored, how quickly it moves, and how efficiently it is processed now have<\/p>\n","protected":false},"author":34,"featured_media":12432,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"_migrated_source_id":0,"footnotes":"","_members_access_role":[],"_members_access_error":""},"categories":[5],"tags":[1598,1263,14,32],"class_list":["post-12397","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-tech-and-ai","tag-agentic-ai","tag-ai-infrastructure","tag-ai-memory","tag-semiconductor-industry"],"acf":[],"_links":{"self":[{"href":"https:\/\/news.skhynix.com\/en\/wp-json\/wp\/v2\/posts\/12397","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/news.skhynix.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/news.skhynix.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/news.skhynix.com\/en\/wp-json\/wp\/v2\/users\/34"}],"replies":[{"embeddable":true,"href":"https:\/\/news.skhynix.com\/en\/wp-json\/wp\/v2\/comments?post=12397"}],"version-history":[{"count":20,"href":"https:\/\/news.skhynix.com\/en\/wp-json\/wp\/v2\/posts\/12397\/revisions"}],"predecessor-version":[{"id":12437,"href":"https:\/\/news.skhynix.com\/en\/wp-json\/wp\/v2\/posts\/12397\/revisions\/12437"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/news.skhynix.com\/en\/wp-json\/wp\/v2\/media\/12432"}],"wp:attachment":[{"href":"https:\/\/news.skhynix.com\/en\/wp-json\/wp\/v2\/media?parent=12397"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/news.skhynix.com\/en\/wp-json\/wp\/v2\/categories?post=12397"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/news.skhynix.com\/en\/wp-json\/wp\/v2\/tags?post=12397"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}