When should an agent trust another agent?

KAIROS separates rapport from evidence

Suppose an agent has agreed with you throughout five earlier quiz rounds. On the sixth round, it confidently gives a different answer. Should its history change your decision? A standard multi-agent accuracy score cannot tell whether the final answer came from useful evidence, blind agreement or misplaced trust in a familiar peer.

KAIROS turns that question into a controlled experiment. Its training set contains 10,000 questions and its test set contains 3,000. The questions cover four kinds of work: reasoning, knowledge, social judgement and creativity. The source tasks include MATH-500, LiveCodeBench, Big-Bench Hard, TruthfulQA, MMLU-Pro, Social IQ, CommonsenseQA 2.0, MacGyver and BrainTeaser. Open-ended tasks are converted to multiple choice; generated distractors are checked automatically and by people.

The benchmark is rebuilt for each model being tested. The model first answers a question five times at temperature 0.7. The majority answer becomes its original belief, and the entropy of the five answers provides a model-specific confidence estimate. Questions are divided into high- and low-confidence groups at that model’s median entropy.

KAIROS then supplies a synthetic interaction history and the current answers of peer agents. The history creates five levels of rapport—0%, 25%, 50%, 75% or 100%—according to how often each peer previously agreed with the model. In the current round, peers either support the model’s belief, oppose it with the most plausible alternative, or oppose it with the least plausible alternative. The intervention therefore controls three variables separately: prior rapport, present peer behaviour and the model’s confidence in its own answer.

Measure both correction and resistance

KAIROS reports ordinary accuracy and three transition measures. Utility is the fraction of originally wrong answers corrected after seeing the peers. Resistance is the fraction of originally correct answers that remain correct. Robustness is the relative change from the model’s original accuracy to its socially influenced accuracy.

Across eleven evaluated models and several training regimes, correct-to-correct resistance accounted for about 65.1% of answer transitions. This apparent stability concealed a directional bias: models lost more correct answers than they recovered wrong ones. Prior rapport amplified the effect of the current message. Averaged across model families, resistance was 31.7 points lower when a trusted peer offered a hard opposing answer than when the peer supported the model.

Training also changed how answers moved. Supervised fine-tuning made models more willing to revise mistakes, but reduced resistance to misleading peers. GRPO recovered resistance and improved utility in some configurations, yet confidence in corrected answers could fall. For Qwen-14B, utility confidence fell from 0.737 before training to 0.137 after supervised tuning and recovered only to 0.167 after GRPO. The best absolute multi-agent accuracy came from unconstrained GRPO with an outcome reward, while its relative robustness could still decline.

KAIROS is a tailored multiple-choice simulation, not a claim about every social interaction. Its value is experimental control: the same agent can be tested with the same underlying question while its relationship history and peer behaviour are changed independently.

Outcome rewards can teach conformity

Epistemic Context Learning (ECL) asks whether an agent can learn which peer to use. The paper begins with a deliberately simple diagnostic: two named peers answer five earlier questions, one always correctly and one always incorrectly. On a new MATH-500 or LiveCodeBench problem, the reliable peer gives the correct answer and the unreliable peer gives a plausible wrong one.

Reinforcement learning with only a final-answer reward often improved ordinary multi-agent accuracy without learning the peer identities. Two counterfactual tests exposed the shortcut. In Flip, the histories of the reliable and unreliable peers are swapped. In All-Wrong, both current peer answers are wrong. A model that learned reliability should notice the switch and should be able to reject both peers.

The collapse under All-Wrong shows that a high final-answer score can come from copying the normally correct peer. Reasoning supervision improved resistance, although the remaining drops show that it did not solve the problem.

Build the peer profile before showing the new answers

ECL blocks the easiest shortcut by splitting inference into two stages. Stage 1 receives only the five-round history; the current question and current peer answers are withheld. It produces a short reliability profile for every peer. Stage 2 receives those profiles together with the new question and peer responses, then produces the final answer. The explicit ECL variant adds a peer-recognition reward that requires Stage 1 to identify the most reliable peer. Each stage is therefore supervised by an output it can actually control.

On the two-peer diagnostic, the explicit two-stage variant reached 81.1% final-answer accuracy on MATH-500 and 89.2% on LiveCodeBench under the outcome metric; its peer-recognition score was 100% on both. The corresponding single-stage RL results were 73.0% and 75.7%. Adding the peer-recognition reward directly to the single-stage model did not work: MATH-500 accuracy fell from 73.0% to 62.2%, because the model could exploit cues in the current round instead of reading the history. Withholding that round from Stage 1 removes the shortcut.

The main evaluation expands to four peers and five historical rounds. MMLU-Pro contributes 100 history, 810 training and 90 test questions; GPQA contributes 50, 358 and 40. In the natural setting, peers range from Gemini 3 Flash to much smaller open models. In the adversarial setting, one peer is correct and three comparably capable peers give subtly wrong answers. ECL is compared with a single agent, ordinary answer aggregation and one-stage history-aware aggregation across Qwen models and frontier systems. On the 40-question adversarial GPQA test, Gemini 3 Pro reached 100% with explicit ECL, so that number should be read with its small denominator.

The method assumes persistent peer identities and labelled history from which correctness can be inferred. Its own stress test shows the risk: flipping a learned reliability prior reduced Gemini 3 Flash accuracy by more than 40 points in one setting. A stale or manipulated profile can therefore become an attack surface. ECL establishes that separating trust estimation from answer aggregation helps under the tested protocol; it does not yet show how an agent should revise trust when identities drift, feedback is delayed or no ground-truth answer arrives.