Abstract
Social intelligence has become a critical capability for large language
models (LLMs), enabling them to engage effectively in real-world social tasks
such as accommodation, persuasion, collaboration, and negotiation.
Reinforcement learning (RL) is a natural fit for training socially intelligent
agents because it allows models to learn sophisticated strategies directly
through social interactions. However, social interactions have two key
characteristics that set barriers for RL training: (1) partial observability,
where utterances have indirect and delayed effects that complicate credit
assignment, and (2) multi-dimensionality, where behaviors such as
rapport-building or knowledge-seeking contribute indirectly to goal
achievement. These characteristics make Markov decision process (MDP)-based RL
with single-dimensional episode-level rewards inefficient and unstable. To
address these challenges, we propose Sotopia-RL, a novel framework that refines
coarse episode-level feedback into utterance-level, multi-dimensional rewards.
Utterance-level credit assignment mitigates partial observability by
attributing outcomes to individual utterances, while multi-dimensional rewards
capture the full richness of social interactions and reduce reward hacking.
Experiments in Sotopia, an open-ended social learning environment, demonstrate
that Sotopia-RL achieves state-of-the-art social goal completion scores (7.17
on Sotopia-hard and 8.31 on Sotopia-full), significantly outperforming existing
approaches. Ablation studies confirm the necessity of both utterance-level
credit assignment and multi-dimensional reward design for RL training. Our
implementation is publicly available at:
https://github.com/sotopia-lab/sotopia-rl.
Abstract
Generative AI is no longer a peripheral tool in higher education. It is
rapidly evolving into a general-purpose infrastructure that reshapes how
knowledge is generated, mediated, and validated. This paper presents findings
from a controlled experiment evaluating a Socratic AI Tutor, a large language
model designed to scaffold student research question development through
structured dialogue grounded in constructivist theory. Conducted with 65
pre-service teacher students in Germany, the study compares interaction with
the Socratic Tutor to engagement with an uninstructed AI chatbot. Students
using the Socratic Tutor reported significantly greater support for critical,
independent, and reflective thinking, suggesting that dialogic AI can stimulate
metacognitive engagement and challenging recent narratives of de-skilling due
to generative AI usage. These findings serve as a proof of concept for a
broader pedagogical shift: the use of multi-agent systems (MAS) composed of
specialised AI agents. To conceptualise this, we introduce the notion of
orchestrated MAS, modular, pedagogically aligned agent constellations, curated
by educators, that support diverse learning trajectories through differentiated
roles and coordinated interaction. To anchor this shift, we propose an adapted
offer-and-use model, in which students appropriate instructional offers from
these agents. Beyond technical feasibility, we examine system-level
implications for higher education institutions and students, including funding
necessities, changes to faculty roles, curriculars, competencies and assessment
practices. We conclude with a comparative cost-effectiveness analysis
highlighting the scalability of such systems. In sum, this study contributes
both empirical evidence and a conceptual roadmap for hybrid learning ecosystems
that embed human-AI co-agency and pedagogical alignment.