📊 Full opportunity report: Engineering Is Automated. Research Is the Residual. on ThorstenMeyerAI.com — validation score, market gap, and execution plan.

Recent advances in AI capabilities have demonstrated that AI systems can now automate the majority of engineering tasks involved in AI research, according to new analysis of benchmark data. While engineering automation appears near complete, research processes still depend heavily on human input, leaving a residual role for researchers. This development has significant implications for the future of AI development and innovation.

Thorsten Meyer’s review of recent benchmark results shows that AI systems have achieved near-saturation on key tasks such as reproducing research experiments (CORE-Bench) and competing in Kaggle-like challenges (MLE-Bench). For example, CORE-Bench, which measures the ability to reproduce research papers, has reached a 95.5% success rate, with some experts declaring it ‘solved.’ Similarly, AI performance on Kaggle competitions has improved to a level where it can match mid-tier human practitioners, reaching 64.4% of competitions at a bronze-medal level. These benchmarks, spanning core research reproduction, competitive machine learning, and kernel design, indicate that AI can automate most engineering aspects of AI R&D, reducing the need for human intervention in these areas.

However, the analysis notes that the automation of AI research—such as formulating novel hypotheses, designing experiments, and interpreting results—remains less developed. While progress is evident, the structural question posed by Clark, whether research itself is just scaled engineering, is still under consideration. The current data suggests that research may be increasingly integrated into engineering workflows, potentially closing the residual gap faster than previously thought.

Implications for AI Development and Research Workforce

This trend signals a potential transformation in AI research, with engineering tasks becoming fully automated and research processes remaining human-centric for now. If research can be integrated into engineering workflows, the pace of AI innovation could accelerate dramatically, reducing costs and increasing productivity. However, it also raises questions about the future role of human researchers, the nature of scientific discovery, and the risk of over-reliance on automated systems. Understanding this shift is crucial for policymakers, institutions, and the AI community as they prepare for a future where AI handles most engineering work but research remains a human endeavor.

Recent Benchmark Progress in AI R&D Skills

Over the past 18 months, multiple independent benchmarks have shown rapid advancements in AI capabilities relevant to research and engineering. CORE-Bench, measuring research reproduction, improved from 21.5% to 95.5%, with some experts declaring it ‘solved.’ The MLE-Bench, assessing performance in Kaggle competitions, rose from 16.9% to 64.4%, reaching a level comparable to mid-tier human practitioners. Additionally, progress in kernel design—such as automated GPU kernel generation—has been documented through numerous research papers and industry applications. These developments collectively suggest that AI is approaching full automation of core engineering tasks in AI R&D, with the measurement limits being reached across multiple domains.

“The pattern across benchmarks indicates that AI is nearing the saturation point in automating core engineering tasks, with the residual research role remaining uncertain.”

— Thorsten Meyer

Unclear Scope of AI-Driven Scientific Discovery

While engineering tasks are nearing full automation, it remains uncertain how much of the research process itself—hypothesis formulation, experimental design, interpretation—can be automated. The structural question of whether research is just scaled engineering is still open, and some experts caution that certain creative or interpretive aspects may resist automation in the near term. Additionally, the pace at which research automation could close this residual gap is not yet fully predictable.

Next Milestones in AI R&D Automation

Researchers and industry observers expect continued rapid progress in automation benchmarks over the next 12 to 24 months. Key areas include further improvements in research reproduction, expansion of automated kernel design, and potential breakthroughs in automating hypothesis generation. Monitoring these developments will help assess whether research automation accelerates beyond current expectations and how the AI community adapts to these shifts.

Key Questions

What does near-complete automation of engineering tasks mean for AI research?

It indicates that most routine engineering tasks—such as reproducing experiments, optimizing models, and designing kernels—can now be handled by AI systems, reducing the need for human effort in these areas.

Is AI capable of conducting full scientific research independently?

Currently, AI can automate many engineering components but remains limited in creative, hypothesis-driven research. The extent of full automation in scientific discovery is still uncertain and under active investigation.

How might this trend affect the AI research workforce?

Automation could reduce the number of researchers needed for engineering tasks, shifting human roles toward higher-level conceptual work, oversight, and interpretation, or potentially leading to a reevaluation of research workflows.

When might AI fully automate all aspects of research?

There is no clear timeline; while progress is rapid, experts caution that some aspects of research—especially creative and interpretive tasks—may remain human-driven for years to come.

Source: ThorstenMeyerAI.com