IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation

Why Scientific Ideation Needs Two Objectives

An ideation system can fail in two opposite ways. It may produce ten polished versions of essentially the same proposal, or it may produce ten very different proposals that are vague, obvious, or unsound. Evaluating each idea independently misses this tension.

IDEAgent therefore treats scientific ideation as a quality-diversity search. The objective is not merely to maximize the score of the best idea. Given a fixed number of fresh attempts, the system tries to return a set in which each idea clears quality thresholds and the ideas remain meaningfully distinct from one another.

This changes what counts as progress. A near-duplicate does not become a new discovery because its wording changed, and a surprising idea does not survive if its causal argument is broken. The unit of search is an idea lineage: a seed and all of its repaired or refined descendants count as one attempt.

Six Agents and Compact Search Memory

IDEAgent separates generation, representation, evaluation, and revision across six roles:

• The Ideator proposes a fresh research direction or revises an existing one.
• The Stenographer compresses a draft into its problem, mechanism, value addition, assumptions, and expected effect.
• The Quality Evaluator scores non-obviousness, mechanism clarity, and feasibility.
• The Soundness Panel checks the proposal’s logical and mathematical consistency through multiple independent judgments.
• The Diversity Judge compares the proposal with accepted, historical, and rejected directions.
• The Critic converts the judgments into focused feedback for repair or refinement.

The controller keeps compact summaries rather than repeatedly passing every full proposal back into the model. An active archive stores the current set of accepted ideas. A historical archive remembers qualified ideas that were later replaced. A rejected archive aggregates recurring failure patterns, while a temporary repair queue holds a promising near-miss during revision.

This memory is deliberately selective. The system remembers enough to avoid repeating old mechanisms and failure modes without allowing a growing transcript of previous ideas to dominate later generations.

How an Idea Moves Through the Search

Each fresh seed is compressed and evaluated before the controller decides what happens next.

An invalid or clearly weak idea is rejected. A promising near-miss may receive one repair attempt. An idea that clears the quality and diversity gates enters the active archive, where it can receive targeted refinement if it remains below the system’s softer quality targets.

When the active archive is full, a new proposal must compete with the most similar accepted idea. The higher-quality proposal remains active; the displaced proposal moves to history. Repair and refinement preserve lineage, so small variations cannot inflate the number of discoveries.

For ranking qualified ideas, IDEAgent uses a weighted quality score:

Here, N, S, and C denote non-obviousness, soundness, and mechanism clarity. The weighting gives the greatest emphasis to the conceptual move, while still requiring the proposal to be coherent and sufficiently specified.

Yield: Measuring Useful Breadth

Average quality and average diversity are informative, but neither answers the practical question: how many distinct ideas in the returned set are actually usable?

The paper introduces Yield, the size of the largest subset whose members all satisfy the chosen quality thresholds and whose pairs satisfy a diversity threshold. Reworded duplicates count once. Diverse but unsound ideas do not count at all.

This set-level metric makes the evaluation closer to the intended use of an AI co-scientist. A researcher needs a portfolio of plausible directions, not a high average score produced by one strong idea surrounded by weak or redundant ones.

What the Experiments Show

The study evaluates IDEAgent on 32 topics spanning eight areas of computer science. It compares the framework with stateless generation, single-shot generation, sequential memory, and a NOVA-inspired baseline, using independent LLM judges for downstream evaluation.

Across the reported settings, IDEAgent achieves up to 3.89 times the Yield of the strongest baseline and obtains non-zero Yield on eight times as many topics under the strictest evaluated gate. The advantage is not limited to producing one qualifying idea: the system is more likely to return several distinct, high-quality directions for the same topic.

The revision mechanisms also matter. Of 30 lineages sent for repair, 28 ultimately qualified. Refinement was attempted on 182 of 320 lineages, and the revised idea replaced its parent in 151 cases. Independent judges likewise found improvements in clarity, soundness, and Yield after these interventions.

What the Results Do Not Claim

IDEAgent does not establish that an automatically generated proposal is scientifically correct, important, or ready to execute. Its evaluation depends on model-based judges, explicit thresholds, and a bounded set of background papers. Soundness remains difficult to assess before an idea is implemented and tested.

The contribution is a search and evaluation framework for an earlier stage of research: constructing a broader set of proposals that merit closer human scrutiny. Yield makes that objective measurable, while lineage tracking prevents superficial variations from being counted as independent discoveries.

The Main Takeaway

The central lesson is that scientific ideation should be evaluated as a set-building problem. Good research support requires both depth and breadth: proposals must survive scrutiny, but the system must also continue exploring genuinely different mechanisms.

IDEAgent combines compact memory, explicit diversity comparisons, and bounded repair and refinement to pursue both goals under a fixed budget. It offers a practical quality-diversity lens for building AI systems that assist researchers without reducing ideation to either unconstrained brainstorming or a single-score ranking problem.

This lab note adapts and condenses Varun Gumma’s original article. For the complete method, evaluation protocol, and limitations, see the paper.