A few weeks ago, I had the opportunity to give an introductory talk at the HAICON 2026 workshop “Building Agentic ML Tools for Science: What Works and What Doesn’t” in Munich. The talk was designed as a dense overview of concepts, recent news, and trends in agentic AI, with a particular focus on how these systems are beginning to intersect with scientific research.

This post is a written version of that presentation. It covers the key developments I discussed, along with the references and sources mentioned in the slides. The views expressed here are my own and reflect personal observations and opinions formed during the preparation of the talk – not official positions.

What Is an Agent, Really?

The term “agent” is thrown around a lot, and it means different things depending on who you ask. For the purposes of this discussion, I use a working definition: an agent is a system that autonomously plans, acts, and iterates using tools to achieve a goal. An agent is not an LLM making a single tool call. The canonical loop is Goal → Plan → Act → Observe → Repeat, and three core properties distinguish agents from simple LLM interactions:

• Goal-directed behavior: the system works toward an explicit objective, not just toward producing a plausible response.
• Decision-making: it chooses subsequent actions based on intermediate results, not just the original prompt.
• Iteration: it refines outputs over multiple steps, observing and correcting as it goes.

In research, generative AI use cases span a wide range. Witte, Bayer, and Weber (2026) surveyed 118 potential use cases of generative AI for researchers across six categories: idea generation, literature review, methodology design support, data collection, programming, and data analysis and writing support¹. Agents extend these use cases from one-off assistance to continuous, iterative workflows.

The Shift to Persistent Agents

Early LLMs were largely stateless: you send a prompt, get a response, and the context is lost. The new wave of agentic systems increasingly retains project-relevant knowledge, goals, tasks, and preferences across sessions and restarts. These persistent agents resume previous work automatically, maintain evolving context about users and projects, and can monitor systems and perform long-running workflows.

I see this as a fundamental shift in the interaction model: from asking questions to delegating responsibilities, and from the prompt as the unit of interaction to the project. The dominant paradigm is moving from short-lived assistants to persistent systems that operate continuously, remember previous interactions, and execute tasks over extended periods.

Agent Harnesses — The New Operating Layer

Something that became increasingly clear during my preparation for the talk is that performance in agentic systems depends less on model capability alone and more on orchestration. Harnesses – the software infrastructure layer wrapped around an AI model – are becoming the new operating layer. They handle planning and task decomposition, context management and memory retrieval, tool routing and execution, verification and retry loops, and multi-step control flow across agents.

We are shifting from model-centric AI to system-centric AI. The real intelligence of agentic systems is increasingly located in the orchestration layer that coordinates models, tools, memory, and execution. Examples include coding harnesses like Claude Code, GitHub Copilot CLI, and OpenCode, as well as systems like Paperclip, which models software organizations as teams of specialized agents coordinated through structured workflows.

Data, Memory, and Context

As top-tier frontier models become increasingly comparable on standard benchmarks², the bottleneck in agentic systems is shifting from model capability toward how effectively systems persist, structure, and retrieve contextual knowledge over time. Long-horizon agents require persistent external state that goes beyond context windows.

The core mechanisms enabling this are retrieval-augmented generation (RAG) over static prompting, persistent memory stores for project and user history, context compression and summarization pipelines, and external knowledge bases linked to agents – such as Karpathy’s llm-wiki, a curated knowledge base designed for LLM consumption.

MCP: The USB-C of AI Tools

The Model Context Protocol (MCP) has become the standard for how models connect to tools and external systems. Built on simple structured messaging patterns, MCP enables rapid ecosystem adoption. The tool layer has become composable and interchangeable: instead of custom integrations for each system, tools can be plugged in via standardized MCP servers. The protocol is advancing quickly³, though some issues remain – notably, high context-token usage when querying MCP tools can be expensive.

A New Visual and Agentic Interface Paradigm

Text-only interaction is reaching its limits for complex outputs, multi-step workflows, and user interaction. AI systems increasingly generate interactive UI rather than just text. MCP Apps enable interactive applications rendered inside clients like Claude Desktop: MCP servers can return structured HTML-based interfaces, enabling forms, dashboards, data exploration, and tool-driven interaction inside chat environments.

Provider-specific implementations are proliferating: Claude Artifacts, OpenAI Canvas, Google Gemini Canvas, CopilotKit’s Open Generative UI, and Omma all extend the traditional chatbot interface by rendering AI-generated content in dedicated, interactive user interfaces.

Karpathy has articulated this shift explicitly⁴: chat interfaces are only an early stage in human-AI communication. He predicts outputs will get progressively more visual, moving beyond HTML toward advanced interactive visual experiences and eventually fully interactive AI-generated simulations and video. The interface is becoming part of the model output itself – not a separate layer. Agents are evolving from text generators into UI generators. Exciting, but it requires more capabilities, more tokens, and more compute.

Agent Identity and Authentication

As agents increasingly perform user-specific actions, existing authentication mechanisms like OAuth are being extended into contexts they were not designed for. Authentication and authorization systems were built for human-driven, app-centric workflows. Agents operate on behalf of users but lack a stable, first-class identity model in the internet stack. Identity, permissions, and accountability for agents remain fundamentally unresolved.

Small Models Moving Agentic AI to the Edge

There is a clear push toward on-device agentic systems instead of cloud-only execution, driven by latency, privacy, cost, and the desire for always-available agents. Karpathy has suggested that a ~1B parameter cognitive core may be sufficient for many agent tasks⁵. This aligns with a trend: newer models from major providers are going very small, built natively for on-device, mobile, and browser environments. Examples include Gemma 4’s smallest tier at ~2B parameters, Gemma 3’s smallest tier at ~270M, and Qwen3.5 models down to ~0.8B parameters. Google also launched the Coral Board at Google I/O 2026, an edge development board that runs Gemma 3 270M locally, and Chrome now automatically downloads a ~4GB localized AI model to power native on-device features⁶.

Google’s push into edge AI is a significant signal: the capabilities of small models running locally are no longer an academic curiosity but a product strategy backed by major infrastructure investment.

Agents Operating at Superhuman Scale

While many people still think of AI as a chatbot, companies are already running agentic systems at scales that exceed human supervision. Peter Steinberger’s team runs ~100 Codex agents continuously, burning through tokens worth ~$1.3 million in a month across 603 billion input tokens ⁷. Microsoft’s MDASH system orchestrates over 100 agents for vulnerability discovery, with agents challenging and verifying one another ⁸.

Cursor’s FastRender project ran hundreds to thousands of frontier-model agents to develop a browser rendering engine from scratch – browsers being among the most complex pieces of software to develop ⁹. The project amassed over 30,000 commits. Google’s Antigravity team built a full OS from scratch using 93 subagents, 15,314 model calls, and over 339M input tokens ¹⁰. And Qwen3.7-Max wrote, compiled, profiled, and optimized a complex hardware architecture kernel entirely autonomously, running continuously over 35 hours with 1,158 tool calls across 432 kernel evaluations ¹¹.

These are not demonstrations of isolated capabilities. They are evidence that complex, long-horizon tasks are being executed by systems that operate beyond the scale of human oversight.

Multi-Agent Systems

Complex tasks are increasingly decomposed across multiple specialized agents rather than handled by a single system. The Planner → Worker → Reviewer pattern is becoming dominant. Industry examples include Microsoft MDASH (>100 security agents), Cloudflare’s vulnerability discovery harness with up to 50 parallel agents, Anthropic’s Code Review with parallel reviewer agents, and OpenAI Codex designed for multi-agent workflows. At the conceptual level, systems like Paperclip model entire software organizations as teams of specialized agents – engineering, product, design, review – coordinated through structured workflows.

The multi-agent pattern addresses a core limitation of single-agent systems: the difficulty of maintaining focus, coherence, and correctness across long, complex tasks. By decomposing work across specialized agents, systems can achieve outcomes that no single agent could reliably produce alone.

AI Training Becomes Agentic

Agentic systems are now being used to automate parts of AI training and research itself. Karpathy’s autoresearch demonstrates autonomous, self-improving agents that implement small iterative experiments for model optimization, following a Hypothesis → Experiment → Evaluation → Repeat loop. HuggingFace’s ml-intern “reads papers, finds datasets, trains models, and iterates until the numbers go up” – in tests, it generated synthetic data points, upsampled and trained on them, and outperformed baselines ¹².

Karpathy moved to Anthropic to build a team that uses Claude to accelerate model development itself. The AI startup Recursive emerged from stealth with a $4.65 billion valuation, aiming to build AI systems that recursively improve themselves ¹³. Qwen’s team integrated Qwen3.7-Max into their Reinforcement Learning monitoring, where the model acted as a reward hacking self-monitor, systematically identifying candidate hacking patterns, performing rule verification, counter-example mining, and iterative optimization.

AI is beginning to automate its own training pipeline. What this means for the trajectory of model development – and who controls it – remains an open question.

Generative AI Stress-Testing Open Source

The influx of AI-generated content is placing enormous pressure on open source communities. Daniel Stenberg, creator of cURL, described it bluntly: “AI slop is DDoSing open source” ¹⁴. cURL’s bug bounty program saw AI-generated bug reports spike to 8x the usual rate, triage becoming what Stenberg called “terror reporting,” draining the will to live from the project’s seven-person security team.

The OCaml project experienced a particularly illustrative incident: a 13,000-line (!) AI-generated pull request was submitted, incorrectly crediting researcher Mark Shinwell in file headers. When Shinwell responded to the misattribution with a lengthy “AI-written copyright analysis,” he was asked whether he had written the analysis himself. His response: “Beats me. AI decided to do so and I didn’t question it.” The PR was rejected by maintainers, who noted that “AI-written code is more taxing than reviewing human-written code” and creates “a very real risk of bringing the Pull-Request system to a halt” ¹⁵. Complicating matters further, the incident was subsequently misreported by devclass.com, which incorrectly stated that Mark Shinwell was involved in the discussion – a factual error that spread to Reddit and other forums.

This case illustrates the broader problem: the signal-to-noise ratio in AI-generated submissions is deteriorating, and human review cannot scale with AI output. Irresponsible “vibe coding” is prone to creating not just low-quality contributions, but actual factual errors and security threats in codebases.

AI as a Cybersecurity Actor

Frontier models like Claude Mythos Preview and GPT-5.5 have demonstrated the ability to autonomously discover vulnerabilities, develop exploits, and execute end-to-end attack chains ¹⁶. These capabilities lower the skill barrier for advanced cyberattacks and increase the scalability of exploitation. Anthropic’s Project Glasswing systematically discovered 271 Firefox vulnerabilities in a single pass, including long-standing bugs ¹⁷.

While current frontier deployments are limited to major labs and select partners, the underlying capabilities generalize and pose growing risk to smaller, less-resourced projects and organizations. AI systems are shifting from tools that assist security work to systems that can independently perform offensive cyber operations. The result is a rapidly expanding dual-use landscape where offensive capability scales faster than defensive readiness.

The Cost Explosion

Agentic workflows consume massive amounts of tokens due to multi-step reasoning and repeated tool calls. This puts pressure on current pricing models. While inference costs are predicted to drop dramatically – Gartner predicts >90% reduction by 2030 for large models ¹⁸ – the immediate pressure is driving a shift from subscription-based pricing to usage-based, token-based billing. GitHub Copilot already transitioned to token consumption-based billing ¹⁹, and Google is moving to a compute-based consumption model ²⁰.

Local models provide a partial escape hatch. As Simon Willison noted in his analysis of recent LLM developments, local models are becoming surprisingly competitive ²¹. There, he visualizes the rapid performance gains of open-weight models, illustrating that for many use cases, the return on investment of running models locally is increasing.

Are Open-Weight Models Diminishing?

A concerning trend runs parallel to the cost issue. The technical lead of Alibaba’s Qwen team, Junyang Lin, and several key contributors stepped down shortly after the Qwen 3.5 open-weight models were released. Multiple signals suggest that Qwen may reduce its emphasis on open-weight releases.

Frontier labs released open-weight models to drive adoption, build ecosystems, attract talent, and establish market leadership. But open-weight releases make direct monetization more difficult than API-based access. As competition intensifies and inference costs rise, pressure grows to generate revenue from model usage. Labs like OpenAI, Anthropic, and Google heavily subsidize training for their best models and offer them exclusively through proprietary APIs. Consequently, open-weight models typically trail closed frontier models by about 6–10 months on benchmarks on abstract reasoning tasks. But what about practical, creative tasks? Simon Willison’s famous benchmark generating an SVG of a pelican riding a bicycle shows that newer open-weight models like Qwen3.6-35B-A3B can rival closed-model counterparts like Claude Opus 4.7, with results that are hard to distinguish or rank.

The long-term commitment of major providers to open-weight models remains uncertain, and the potential for growing dependency on proprietary providers is a genuine concern – particularly for European research and industry.

Agentic AI and the Scientific Publishing Crisis

Research is already under increasing pressure from exploding publication numbers. ICML received roughly 24,000 submissions in 2026, continuing a rapid growth trend that shows no signs of slowing.

AI tools make it easier to produce papers, reviews, and research artifacts at scale. Reviewers face increasing difficulty distinguishing high-quality work from AI-generated slop. The response from the community has been reactive: conferences like ICLR ²², ACLRR ²³, and EMNLP ²⁴ are already piloting AI-assisted reviewing, essentially using the same technology that floods the pipeline to try to filter it. arXiv has introduced a 1-year penalty for authors submitting low-quality AI-generated content ²⁵.

If agents eventually conduct parts of research and other agents review the resulting work, what does this mean for scientific progress? This is not a speculative concern for the distant future. Research agents are already being developed that automate entire pipelines – literature search, experimental design, data collection, analysis, and writing. Research may increasingly become a continuous, 24/7 process driven by autonomous systems.

The Verification Bottleneck

This brings us to what I consider the central challenge: the new bottleneck in agentic AI is verification, not generation. Systems can produce outputs at scales that humans can no longer reasonably supervise. How do we verify outputs produced by systems operating at scale? How do we distinguish genuine breakthroughs from reward hacking, errors, or AI-generated noise?

The challenge is multi-faceted:

• Context drift: agents losing focus during long workflows, deviating from goals over extended interactions.
• Reliable planning: breaking complex tasks into effective steps, avoiding inefficient or incorrect plans.
• Tool use: robustly selecting and using the right tools under uncertainty, handling failures and noisy outputs.
• Reward hacking: agents optimizing metrics instead of objectives, producing seemingly correct results that mask underlying problems.
• Benchmark integrity: measuring real-world agent performance becomes increasingly controversial as models train on benchmark data.
• Continuous learning: learning from experience without catastrophic forgetting.
• Human oversight: outputs are exceeding human review capacity, and the problem will only get worse.

Research institutions are not prepared for autonomous AI. Regulatory systems were designed for static software, not adaptive autonomous systems acting continuously and at scale. Research society may require entirely new mechanisms for trust, provenance, and accountability.

Where This Is Heading

Agentic AI is not a future technology anymore. The transition is already underway. The field is moving at incredible speed, and the gap between what agents can do and what our institutions, processes, and regulatory frameworks are prepared for continues to widen.

I suspect the answer to the question of whether science will turn into an AI slop factory or enter an era of dramatically accelerated discovery may depend less on model capabilities and more on whether we can build systems that are explainable, verifiable, and genuinely human-centric. The key challenge ahead is shaping how these systems integrate into science, institutions, and society – and how we regulate them.

References