AvailableLLMGR-LLM-02
Human Preference Pairs
Pairwise and graded comparisons for RLHF, DPO and reward modeling.
- Comparisons
- 400,000+
- Avg. turns
- 4.2
- Agreement
- 0.78 κ
Illustrative rendering of annotation layers.
01Overview
Multi-turn conversations with pairwise preferences, Likert ratings and written rationales from trained raters. Covers helpfulness, honesty, harmlessness, instruction following and style, with inter-rater agreement reported per slice.
Modalities
- Text
- Multi-turn dialogue
Use cases
- Reward model training
- Direct preference optimization
- Safety tuning
02Specification
Content
- Format
- Prompt, two candidate responses, preference, rationale
- Ratings
- Pairwise preference + 7-point Likert per axis
- Axes
- Helpfulness, honesty, harmlessness, instruction following, style
03Schema
| Field | Type | Description |
|---|---|---|
| conversation | list<Message> | Shared conversation prefix. |
| chosen | Message | Preferred response. |
| rejected | Message | Dispreferred response. |
| scores | map<string, int> | Per-axis Likert ratings. |
| rationale | string | Rater explanation. |
04File structure
preference-pairs/
├── data_card.md
└── pairs/
└── part-0000.parquet05Sample
Evaluate before you license.
- 1,000 comparisons
- Rater guidelines
- Size
- ≈ 12 MB
- Formats
- Parquet, JSONL
06License
Related datasets
GR-LLM-01 · LLMAvailable
Expert Reasoning SFT
Multi-step solutions written and verified by domain experts.
- Examples
- 250,000+
- Domains
- 24
GR-LLM-03 · LLMEarly access
Agentic Tool-Use Trajectories
Human-demonstrated multi-step tasks across browsers, terminals and APIs.
- Trajectories
- 30,000+
- Environments
- 70+
Get started
Need more than Human Preference Pairs?
We extend catalog datasets with new environments, tasks, languages and annotation layers — or collect from scratch to your spec.