Lab Note · Paper Explainer

IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation

Varun Gumma

August 12, 2026 · 7 min read · AI for Science

Paper GitHub Medium

A Quality-Diversity Search for Research Idea Generation

Introduction

Ever thought about an AI system that could generate a bunch of good ideas for you to explore, so you can pursue as many promising directions as possible? … Well, we present something similar today. Stay tuned till the end to learn more!

The advent of Large Language Models (LLMs) has significantly advanced scientific discovery, with AI systems increasingly acting as co-scientists alongside humans to develop, test, materialize, and even publish new ideas. However, most of these systems individually optimize for either the Quality of a particular idea or the Diversity of a set of ideas. Today, we argue that these two axes are two sides of the same coin and must be optimized together. In other words, the strength of an AI-based ideation system should be judged by its ability to generate a higher density of diverse, high-quality ideas within a limited ideation budget.

To this end, we formalize scientific ideation as a Quality-Diversity search problem and develop our framework, IDEAgent, to address it as such.

The IDEAgent Framework

Fig2 — Figure 1: A simplistic version of IDEAgent which generated multiple ideas sequentially for diversification, along with dedicated quality improvements (repair or refine) for eligible ideas. All the new improvements have the same lineage as their patents.

IDEAgent is a multi-agent system built around the following core principles:

Generate ideas sequentially, using the context of prior ideas to explicitly avoid repetition and encourage diversification.
Decompose each free-form idea into specific fields, providing a more interpretable representation for pairwise comparison between ideas.
Treat improvements to an idea as children of a common parent and assign them the same lineage identifier, so they are not mistakenly counted as separate “new discoveries.”
Maintain archives of currently accepted ideas, historical/retired ideas (superseded by newer versions), and rejected ideas and families. Rejected ideas are clustered into families to capture broader avoidance patterns and help identify similar ideas in the future.

Fig3 — Figure 2: A simple representation of the three archives used by IDEAgent. A new incoming idea can replace its closest duplicate (as identified by the Diversity Judge) from the active set based on its overall quality score (Q). The losing idea is moved to the historical set, not rejected, as it had cleared all thresholds and is of good-quality.

To facilitate these processes, our framework consists of six LLM agents, each with a specific role:

Ideator: The primary generator that produces new ideas or refines existing ones.
Stenographe: Compresses each idea into a structured representation by identifying the problem addressed, central mechanism, novel addition over the background literature, key assumptions, and expected measurable effect from the prose.
Quality Judge: Grades the idea ($I$) on Non-obviousness ($N_I$), Clarity ($C_I$), and Feasibility ($F_I$) on a scale of 0–100.
Soundness Panel: Assesses the logical and mathematical rigor of the idea and its assumptions using five independent judgments for broader coverage. The judgments are given on a scale of 0–9 and then aggregated into a single score on a 0–100 scale ($S_I$).
Diversity Judge: Compares the idea against all archives and identifies the nearest duplicates from each. It also assigns a single score ($D_I$) on a 0–100 scale to capture the overall uniqueness, novelty, and diversity of the current idea with respect to prior generations.
Critic: Assimilates the judgments and scores from the three aforementioned judges to produce a single, targeted improvement directive for the Ideator, guiding it to improve the current generation.

We define the overall quality score as: $Q = 0.7N_I + 0.2S_I + 0.1C_I$, with ties decided by $F_I$.

Fig4 — Figure 3: An overall abstract view of IDEAgent.

Fig5 — Figure 4: Different states of an Idea during its lifetime.

The High-Level Algorithm

An idea generated by the Ideator is first compressed by the Steno into a structured summary, which is used for future comparisons.
The full idea is then rigorously evaluated and scored by all three judges. The Diversity Judge also produces a list of near-duplicates from each archive.
If the idea fails to clear the required threshold for any metric (Non-obviousness, Clarity, Soundness, or Diversity) by more than a permissible margin, it is rejected and added to the rejected archive. It is also assigned to one or more rejection families based on its failure pattern.
If the idea matches an idea in the historical archive but has no match in the active set, it is discarded because it has replicated an older idea that has already been superseded.
If the idea falls below a threshold but remains within the permissible margin, it gets an opportunity for repair. Based on the Critic’s feedback, which incorporates the evaluation results, the Ideator corrects the bottleneck or problematic aspects of the idea. The repaired idea is then evaluated again. If it still fails to cross the required thresholds, it is rejected.
If the idea, either before or after repair, clears all the thresholds, it becomes eligible for acceptance. If the active set (which has a fixed capacity) is not full, the idea is added directly. If the active set is full, the incoming idea competes against its closest match in the active set, as identified by the Diversity Judge. The idea with the higher overall (Q) score is retained, while the other is moved to the historical archive. If the incumbent is replaced, the newcomer inherits its lineage because it was identified as a duplicate of the incumbent, thereby preserving the lineage.
Once an idea is accepted into the active set, it has up to two additional refinement opportunities, provided that one of them was not already used for repair. Refinement is triggered when the idea falls below a higher soft target above the acceptance thresholds. The Critic’s feedback is again provided to the Ideator, and the resulting variant is accepted if it preserves Non-obviousness while improving the overall quality score. If the refinement does not meet these criteria, the new child is discarded.
Once the search budget is exhausted, the ideas remaining in the active set are treated as the final variants and passed on for downstream evaluation.

Downstream Evaluation

We evaluate our ideas at both the individual and group levels using LLM-based judges that are distinct from those used during generation (i.e., the internal evaluation).

At the individual level, we evaluate each idea for Non-obviousness, Soundness, and Clarity as our primary quality metrics, each on a 0–9 scale. At the group level, we compute Diversity pairwise between ideas, also on a 0–9 scale. Finally, in line with the Quality-Diversity formulation, we introduce a new metric, Yield. Yield first filters ideas based on predefined quality gates and then identifies the largest subset of ideas that also satisfy a pairwise diversity gate.

In layman’s terms, Yield measures how many viable and pursuable ideas a system produces — the ideas that are both high-quality and sufficiently diverse from one another.

Main Results

We compare IDEAgent against several ablations, as described in our paper, and present the final results below:

Stateless: Naive parallel generation of N ideas, with no information shared between them.
Single-Shot: Generates all N ideas at once in a single output. For a thinking-based Ideator, the reasoning space is shared across all N ideas, allowing them to share a common memory block for reference and diversification.
Sequential Memory: Sequentially generates ideas while providing summaries of prior generations as references for diversification. In other words, this is IDEAgent without the quality-improvement component.
NOVA: A NOVA-inspired version of Sequential Memory that uses iterative seed-pool-based germination and replacement for diversification. In each round, N ideas are generated, after which an internal judge selects the K most promising ideas as seeds for generating the next N ideas. After three rounds, the internal judge selects the most diverse N ideas from the resulting 3N ideas.

Fig6 — Figure 5: The overall metrics table. Successful topics here refers to the number of topics on which the Yield was non-zero. S — soundness, C — clarity, D — diversity, NB — non-obviousness. The values are aggregated across all 32 topics.

Fig7 — Figure 6: Zoomed in Yield surface of IDEAgent vs Sequential Memory: $S \in [6, 8]$, $NB \in [6, 7]$, $D \geq 7$.

Fig8 — Full 10x10 Yield surface: $S \in [6, 8]$, $NB \in [6, 7]$, $D \geq 7$.

Conclusion

In this work, we propose a new Quality-Diversity lens for the scientific ideation process and demonstrate it through a multi-agent framework that generates ideas sequentially using lightweight memories of prior generations. Our framework also employs an opportunity-based quality-improvement pipeline that repairs partially promising ideas and further refines those that are already acceptable.

Finally, we introduce a simple yet powerful metric, Yield, which measures the number of diverse, high-quality ideas in a generated set, providing a direct estimate of a system’s Quality-Diversity search capability.

We hope this formulation offers a new perspective on scientific ideation and opens up new ways to think about evaluating AI-based research ideation systems.

Do check out the full paper and codebase for the exact details, as we have omitted several explanations and implementation details here for brevity and abstraction. We would love to hear your thoughts!

Citation

Code: https://github.com/declare-lab/IDEAgent
Paper: https://arxiv.org/abs/2607.22375

@misc{gumma2026ideagentagenticqualitydiversitysearch,
      title={IDEAgent: Agentic Quality-Diversity Search for Research Idea Generation},
      author={Varun Gumma and Navonil Majumder and Soumitra Sinhahajari and Soujanya Poria},
      year={2026},
      eprint={2607.22375},
      archivePrefix={arXiv},
      primaryClass={cs.AI},
      url={https://arxiv.org/abs/2607.22375},
}