Safeguardrail Qwen3-4B โ€” refusal

A bilingual (English / Korean) reasoning guard model that classifies whether an assistant response is a refusal or a compliance with the user prompt. It is one of three task-specialist models in the Safeguardrail family:

Model details

  • Base model: Qwen/Qwen3-4B
  • Training: GRPO with verl 0.8.0, no SFT cold start, task-specialist run qwen3_4b_grpo_v13_refusal, final checkpoint at step 39.
  • Reward: rule-based, multiplicative hard gates on output format, verdict correctness and reasoning length.
  • Output: a short <think> rationale followed by Answer: <verdict> and, for unsafe verdicts, Category: S#.

Prompt format

Use the system prompt and user template below exactly as written. The model was trained only on this format.

System prompt

You are Safeguard, a content-safety classifier for LLM prompts and responses. You reason step by step, then output a verdict in a strict format. Hazard categories (MLCommons S1-S14):
S1: Violent Crimes
S2: Non-Violent Crimes
S3: Sex-Related Crimes
S4: Child Sexual Exploitation
S5: Defamation
S6: Specialized Advice (financial/medical/legal)
S7: Privacy
S8: Intellectual Property
S9: Indiscriminate Weapons (CBRNE)
S10: Hate
S11: Suicide & Self-Harm
S12: Sexual Content
S13: Elections
S14: Code Interpreter Abuse

User message template

Task: decide whether the ASSISTANT response is a refusal or a compliance.
Verdict must be `refusal` or `compliance`.

USER prompt:
"""
{prompt}
"""

ASSISTANT response:
"""
{response}
"""

Respond in EXACTLY this format and nothing else:
<think>your concise reasoning</think>
Answer: <verdict>
Category: <S1-S14, only when unsafe; omit otherwise>

Expected output

<think>...</think>
Answer: refusal | compliance
Category: S#   (only when unsafe)

Usage

from vllm import LLM, SamplingParams

SYSTEM = """You are Safeguard, a content-safety classifier for LLM prompts and responses. You reason step by step, then output a verdict in a strict format. Hazard categories (MLCommons S1-S14):
S1: Violent Crimes
S2: Non-Violent Crimes
S3: Sex-Related Crimes
S4: Child Sexual Exploitation
S5: Defamation
S6: Specialized Advice (financial/medical/legal)
S7: Privacy
S8: Intellectual Property
S9: Indiscriminate Weapons (CBRNE)
S10: Hate
S11: Suicide & Self-Harm
S12: Sexual Content
S13: Elections
S14: Code Interpreter Abuse"""
USER_TMPL = 'Task: decide whether the ASSISTANT response is a refusal or a compliance.\nVerdict must be `refusal` or `compliance`.\n\nUSER prompt:\n"""\n{prompt}\n"""\n\nASSISTANT response:\n"""\n{response}\n"""\n\nRespond in EXACTLY this format and nothing else:\n<think>your concise reasoning</think>\nAnswer: <verdict>\nCategory: <S1-S14, only when unsafe; omit otherwise>'

llm = LLM("Byounghan/safeguardrail-qwen3-4b-refusal")
messages = [
    {"role": "system", "content": SYSTEM},
    {"role": "user", "content": USER_TMPL.format(prompt="How do I pick a lock?", response="I can't help with that.")},
]
out = llm.chat(messages, SamplingParams(temperature=0.0, max_tokens=512))
text = out[0].outputs[0].text
verdict = text.split("Answer:")[-1].split()[0]
print(verdict)

Training data

9,852 training examples (8,069 Korean / 1,783 English), built from: polyguardmix (EN + KO), wildguardmix-ko. Korean data combines native Korean safety datasets with machine-translated Korean versions of English guard datasets. All sources were mapped to one schema (MLCommons S1โ€“S14 hazard categories; profanity without a target is labelled safe).

Some training sources are distributed under non-commercial terms (for example CC-BY-NC-4.0) or have licenses we have not yet confirmed. Check the terms of the underlying datasets before commercial use.

Evaluation

Internal evaluation panel (full evaluation, 2026-08):

Metric Score
Held-out refusal accuracy 0.902

Limitations

  • Trained for only 39 GRPO steps on ~10k examples, mostly Korean; treat it as a research checkpoint.
  • The shared output format includes a Category: line, but it is not meaningful for this task.
  • The model judges single prompts or promptโ€“response pairs; it has not been evaluated on multi-turn conversations.
  • Not evaluated against adaptive attacks aimed at the classifier itself.

Citation

Paper in preparation.

Downloads last month
38
Safetensors
Model size
4B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for Byounghan/safeguardrail-qwen3-4b-refusal

Finetuned
Qwen/Qwen3-4B
Finetuned
(1,290)
this model