changcheng967/Aegis-Qwen3-1.7B-SFT-DPO

Aegis Qwen3-1.7B SFT + DPO

AI text humanizer fine-tuned from Qwen3-1.7B using a two-stage pipeline:

1. SFT — Supervised fine-tuning on 15,314 human-AI text pairs (SFT-only model)
2. DPO — Direct Preference Optimization on 732 preference pairs to further align outputs with human writing style

Rewrites AI-generated text to sound like a real student wrote it.

Quick Start

from transformers import AutoModelForCausalLM, AutoTokenizer
import torch

model_name = "changcheng967/Aegis-Qwen3-1.7B-SFT-DPO"

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto",
    trust_remote_code=True,
)

system_prompt = (
    "You are a student rewriting text in your own natural voice. "
    "Rewrite the following AI-generated text to sound like a real student wrote it. "
    "Use contractions, varied sentence lengths, first-person perspective, "
    "and a conversational but academic tone. Keep the meaning. /no_think"
)

ai_text = """Artificial intelligence has significantly impacted the field of education.
It provides personalized learning experiences and helps teachers identify areas where
students need improvement."""

messages = [
    {"role": "system", "content": system_prompt},
    {"role": "user", "content": ai_text},
]

text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)

outputs = model.generate(**inputs, max_new_tokens=512, temperature=0.7, top_p=0.9, do_sample=True)
response = tokenizer.decode(outputs[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)
print(response)

Training Details

Parameter	SFT Stage	DPO Stage
Base model	Qwen3-1.7B	SFT merged model
Method	LoRA SFT	LoRA DPO
LoRA rank/alpha	32 / 64	32 / 64
Training data	15,314 pairs	732 preference pairs
Epochs	2	3
Batch size	32 (effective)	16 (effective)
Learning rate	5e-5	1e-6
DPO beta	—	0.1
Precision	BF16	BF16
Hardware	MetaX C500 (64GB)	MetaX C500 (64GB)

DPO Results

• Reward accuracy: 47% → 100% (perfect preference by epoch 1.5)
• Reward margin: 0.03 → 1.21
• Training time: ~35 minutes

Model Variants

Model	Description
Aegis-Qwen3-1.7B-SFT	SFT only (general rewriter)
Aegis-Qwen3-1.7B-SFT-DPO	SFT + DPO (this model)

Intended Use

This model is designed for educational research on AI text detection and humanization.