YAML Metadata Error:Invalid content in Eval Result file .eval_results/Inkling-2-Small.yaml

Check out the documentation for more information.

Show details
Task ID "hle" does not match any task in dataset "cais/hle". Available: none
Inkling-2-Small / .eval_results /Inkling-2-Small.yaml
aurick's picture SaylorTwift's picture
SaylorTwift HF Staff
Add evaluation results (#4)
77b7815
Raw History Blame Contribute Delete
2.86 kB
- dataset:
id: datacurve/deep-swe
task_id: deep_swe
value: 72.1
date: "2026-11-13"
source:
url: https://huggingface.co/thinkingmachines/Inkling-2-Small
name: "Inkling-2-Small model card"
notes: "DeepSWE v1.1, mini-swe-agent v2.4.6."
- dataset:
id: harborframework/terminal-bench-science
task_id: terminalbenchscience_0_1
value: 5.7
date: "2026-11-13"
source:
url: https://huggingface.co/thinkingmachines/Inkling-2-Small
name: "Inkling-2-Small model card"
notes: "mini-swe-agent v2.4.6."
- dataset:
id: ScaleAI/SWE-bench_Pro
task_id: SWE_Bench_Pro_hard
value: 86.3
date: "2026-11-13"
source:
url: https://huggingface.co/thinkingmachines/Inkling-2-Small
name: "Inkling-2-Small model card"
notes: "mini-swe-agent v2.4.6."
- dataset:
id: likaixin/ScreenSpot-Pro
task_id: overall
value: 83.8
date: "2026-11-13"
source:
url: https://huggingface.co/thinkingmachines/Inkling-2-Small
name: "Inkling-2-Small model card"
notes: "Images with a long side over 4,096 px downsampled to that limit (aspect ratio kept, ground truth rescaled)."
- dataset:
id: likaixin/ScreenSpot-Pro
task_id: overall
value: 92.0
date: "2026-11-13"
source:
url: https://huggingface.co/thinkingmachines/Inkling-2-Small
name: "Inkling-2-Small model card"
notes: "With Python tool. Images with a long side over 4,096 px downsampled to that limit (aspect ratio kept, ground truth rescaled)."
- dataset:
id: MMMU/MMMU_Pro
task_id: mmmu_pro_standard_10_options
value: 81.6
date: "2026-11-13"
source:
url: https://huggingface.co/thinkingmachines/Inkling-2-Small
name: "Inkling-2-Small model card"
- dataset:
id: cais/hle
task_id: hle
value: 44.1
date: "2026-11-13"
source:
url: https://huggingface.co/thinkingmachines/Inkling-2-Small
name: "Inkling-2-Small model card"
notes: "Text-only subset."
- dataset:
id: Delores-Lin/MDPBench
task_id: overall
value: 81.4
date: "2026-11-13"
source:
url: https://huggingface.co/thinkingmachines/Inkling-2-Small
name: "Inkling-2-Small model card"
- dataset:
id: ScaleAI/SWE-bench_Pro
task_id: SWE_Bench_Pro
value: 97.7
date: "2026-11-13"
source:
url: https://thinkingmachines.ai/model-card/inkling-2-small/
name: "Inkling 2 Small model card (thinkingmachines.ai)"
notes: "SWE-Bench Pro v2 (Full), mini-swe-agent v2.4.6. As reported on the card; higher than the same model's Hard-subset score (86.3), pending confirmation."
- dataset:
id: SWE-bench/SWE-bench_Multimodal
task_id: swe_bench_multimodal_%_resolved
value: 43.1
date: "2026-11-13"
source:
url: https://thinkingmachines.ai/model-card/inkling-2-small/
name: "Inkling 2 Small model card (thinkingmachines.ai)"
notes: "SWE-bench Multimodal v2, mini-swe-agent (Vals harness)."