Arman Ghaffarizadeh, Danyal Mohaddes, Aliakbar Izadkhah, Shahriar Noroozizadeh
The paper proposes a dual-channel debate framework to analyze how social structure influences LLM agent utterances by comparing public statements with off-the-record responses, suggesting new directions for agent evaluation.
When LLM agents operate within social structures where roles, audiences, and relational contexts shape the costs and benefits of speech, it is necessary to determine if and how systematic differences arise between their public and private utterances.
A dual-channel debate framework was designed where agents produce both public utterances and off-the-record (OTR) responses that are recorded but not shown to other participants. Experiments were conducted across 10 models, 3 scenarios, and 5 variations per scenario, with public-OTR divergence measured using four aggregate analyses: stance, semantic similarity, natural language inference, and survey responses.
A systematic public-OTR divergence in decision-making was found, rising from a ~3% baseline to roughly 40% in alignment-inducing settings. In some cases, OTR responses explicitly attributed public accommodation to relational pressures like career risk or sponsorship obligation. The findings suggest agent evaluation should extend beyond explicit goals to detect emergent objectives, and the paper presents a dual-channel evaluation framework and behavioral measures to operationalize this assessment.