Observing the Moral Psychology of LLMs
A controlled experiment testing how philosophical persona prompting influences moral decision-making across three large language models — Claude Opus, Gemini 2.5 Pro, and GPT-5.5.
Overview
The Research Question
AI systems are being deployed in high-stakes decision-making contexts, yet little research examines whether their moral reasoning is stable, interpretable, or consistent across models. This study investigates how philosophical persona prompting alters moral judgements in LLMs — and whether the resulting justifications are philosophically coherent.
My Role
Solo researcher — study design, prompt engineering, data collection, quantitative analysis, and write-up.
Methods
Experimental comparative design. Controlled prompt inputs, recorded binary decisions and free-text justifications, applied Cohen's Kappa for inter-model agreement.
Context
Graduate research at Georgia Tech, Spring 2026. Addresses a gap identified in Cheung et al. (2025) — no prior work systematically tested philosophical framing on LLM moral outputs.
Research Design
Experimental Setup
Each of the three models was prompted under each of the five philosophical personas, then presented with all 30 moral dilemmas grouped by framework. Models output a binary YES/NO judgement and a free-text justification for each dilemma.
Five philosophical frameworks were selected — Utilitarianism, Deontology, Virtue/Care Ethics, Rights-Based Ethics, and Justice/Fairness Ethics — each with clearly defined decision heuristics. The 30 moral dilemmas came from Lotto et al.'s (2013) validated behavioural dataset, grouped into five subsets of six dilemmas mapped to each framework.
Structured prompts were developed for each persona, instructing each model to adopt the philosophical framework and respond to each dilemma by: (1) providing a YES or NO moral judgement, and (2) giving a justification grounded in the assigned persona. Prompts were standardised identically across all three models to control for input variation.
All 450 model responses (3 models × 5 personas × 30 dilemmas) were collected and logged alongside their free-text justifications. Binary decisions were extracted and compiled into a structured dataset for quantitative analysis.
Analysis covered four dimensions: (1) YES-rate comparisons across personas and models; (2) scenario-level approval rate patterns; (3) inter-model agreement via Cohen's Kappa per persona pair; and (4) qualitative assessment of justification alignment with the assigned philosophical framework.
The Five Philosophical Personas
Utilitarianism
- Maximise overall welfare, minimise harm
- Sacrifice one to save five
- Outcome-focused decision making
Deontology
- Moral duties and rules above outcomes
- Never use a person as mere instrument
- Rule-bound, outcome-independent
Virtue / Care Ethics
- Character and relational context
- Special obligations to close relationships
- Highest sensitivity to framing bias
Rights-Based Ethics
- Individual rights cannot be violated
- Refuses harm even for greater good
- Most resistance to AI recommendations
Justice / Fairness Ethics
- Distributive justice and equality
- Who deserves to be saved based on fairness
- Evolutionary roots in human moral norms
Findings
Key Results
H1 confirmed: Philosophical prompting dramatically shifts YES rates across personas, ranging from 0% (Rights-Based) to 98.9% (Utilitarianism). This pattern held consistently across all three models.
Key insight: Persona prompting is a scalar modifier on moral judgement — it systematically shifts approval rates while preserving the underlying dilemma-type ordering. LLMs retain structural moral intuitions consistent with human behavioural data even under strong philosophical framing.
H3 partially confirmed: Models converged on extreme personas (Rights-Based, Utilitarianism) but diverged substantially on philosophically ambiguous ones — Virtue Ethics in particular.
Notable finding: Under Virtue Ethics, Claude read the persona as restraint and relational loyalty, Gemini as compassionate action toward the majority, and GPT-5.5 landed between the two. The models essentially split along the virtue ethics vs. care ethics divide described by Benner (1997) — without being instructed to.
Approval rates across the 30 dilemmas reveal a clear personal vs. impersonal harm distinction — mirroring human moral psychology data. Remote/impersonal scenarios received substantially higher approval than direct-contact scenarios.
Key insight: LLMs, like human participants in Lotto et al.'s original study, showed stronger inhibition against harm when the agent must make direct physical contact — even when the utilitarian calculus is identical. This structural sensitivity persisted across all personas.
Cohen's Kappa was computed for each model pair, overall and per-persona. Models showed substantial overall agreement, but diverged sharply on philosophically ambiguous personas — confirming that it's persona complexity, not general model differences, that drives disagreement.
Core interpretation: When a persona has absolute rules (Rights-Based: "individual rights cannot be violated"), all models follow them identically. When a persona requires contextual judgement (Virtue Ethics: "what would a person of good character do here?"), models diverge — reflecting genuine philosophical ambiguity, not model failure.
Notable Observations
Claude invokes rule utilitarianism
The only scenario where models disagreed under Utilitarianism (Claude=NO, Gemini=YES, GPT=YES) was the Transplant dilemma. Claude invoked rule utilitarianism — arguing permitting organ harvesting would collapse medical trust and produce greater long-run harm — while Gemini and GPT applied straightforward act-utilitarian arithmetic. Evidence that persona adoption depth varies across models.
Claude exhibits the doctrine of double effect
Claude consistently redirected existing threats (Trolley, Plane, Crane) but refused to use a person as a means even under identical numerical stakes (Hospital, Transplant). This distinction reflects the doctrine of double effect operating beneath Claude's default reasoning — not prompted, but emergent.
Models split along virtue vs. care ethics divide
When prompted with Virtue/Care Ethics, Claude, Gemini, and GPT-5.5 each interpreted "caring and virtuous agent" differently — splitting along the Benner (1997) tension between virtue ethics (character-focused) and care ethics (relational-focused). The models mapped onto real philosophical distinctions without being instructed to distinguish them.
Reflection
Limitations & Improvements
This study was a solo research project with a controlled but limited scope. Several limitations shape the strength of the conclusions and point toward clear future directions.
Persona philosophical specificity varies
Rights-Based and Deontology are tightly defined; Virtue and Justice are inherently pluralistic with no single canonical decision procedure. Future iterations should operationalise more specifically — e.g. Rawlsian justice rather than "justice/fairness."
Single response per condition — no reliability check
With one response per scenario per persona, it's impossible to distinguish a stable moral judgement from a probabilistic output. Running each condition 3–5 times and taking the majority decision would significantly strengthen reliability claims.
Binary YES/NO flattens nuance
Several justifications show genuine moral conflict before landing on a decision. A Likert-scale confidence rating alongside the binary choice would capture this and make the analysis more sensitive to moral ambivalence.
Justification alignment not formally scored
Alignment was assessed qualitatively. Adding a systematic rubric (1–3: misaligned, partially aligned, fully aligned) rated by a second coder would give a quantitative measure for Hypothesis 2 and establish inter-rater reliability.
Model versions are moving targets
Claude Opus, Gemini 2.5 Pro, and GPT-5.5 are commercial models updated silently. Replication in six months may produce different results with the same model names. Noting exact API version strings and dates is critical for reproducibility.
Skills Demonstrated
Research & Design
- Experimental study design
- Literature review & gap identification
- Hypothesis formation & testing
- Validity planning (content, external)
Technical Execution
- OpenAI, Anthropic & Gemini APIs
- Python scripting & data logging
- Prompt engineering at scale
- Cohen's Kappa inter-model analysis
Analysis & Communication
- Quantitative & qualitative analysis
- Data visualisation interpretation
- Academic research writing
- Identifying model-level behaviours