> ## Documentation Index
> Fetch the complete documentation index at: https://docs.simplismart.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# DPO (LLM)

> Specifies how to structure conversational data for DPO-based fine-tuning.

## **OpenAI DPO Format** *(for preference training)*

```json theme={null}
{
  "messages": [
    {"role": "system", "content": "You are a useful and harmless assistant"},
    {"role": "user", "content": "Tell me tomorrow's weather"},
    {"role": "assistant", "content": "Tomorrow's weather will be sunny"}
  ],
  "rejected_response": "I don't know"
}
```

## Example JSONL File

```json theme={null}
{"messages": [{"role": "...", "content": "..."},], "rejected_response": "..."}
{"messages": [{"role": "...", "content": "..."},], "rejected_response": "..."}
{"messages": [{"role": "...", "content": "..."},], "rejected_response": "..."}
```

### **Message fields**

* `messages:` A list of role-based conversational turns that provide context for the model.
* `content:` The text of each message exchanged between roles in the conversation.
* `rejected_response:` The assistant’s response that is considered less helpful or incorrect for training purposes.