zli12321 commited on
Commit
cd7d1da
·
verified ·
1 Parent(s): aca966e

Add LHTB 2.0 (Long-Horizon Terminal-Bench) eval result: 12/77 tasks solved

Browse files

Adds the **Long-Horizon Terminal-Bench 2.0 (LHTB 2.0)** result for GLM-5.3 so it appears on the [LHTB benchmark leaderboard](https://huggingface.co/datasets/IntelligenceLab/Long-Horizon-Terminal-Bench) under the `lhtb2_solved` task.

**Score: 12 / 77** tasks solved at reward >= 0.95 (pass@0.95)

| Metric | Value |
|---|---|
| Solved (reward >= 0.95) | 12 / 77 |
| Strict pass (reward = 1.0) | 9 / 77 |
| Mean reward (77 tasks) | 0.542 |
| Agent | terminus-2 (official LHTB Harbor harness) |
| Budget | 3 hours per task, 1 trial per task |

LHTB measures how well agents sustain useful work in a containerized terminal
over hundreds of steps, graded by hidden rebuild-from-artifact verifiers. LHTB 2.0
has 77 tasks: the original 46 plus 31 new ones, 30 of them reconstruction tasks.

Full results, with per-task rewards and trajectories linked from there:
https://github.com/zli12321/LHTB/tree/lhtb-2.0#lhtb-20-results-october-2026

Submitted by the LHTB maintainers; happy to adjust the formatting or withdraw if you prefer.

Files changed (1) hide show
  1. .eval_results/lhtb2_solved.yaml +9 -0
.eval_results/lhtb2_solved.yaml ADDED
@@ -0,0 +1,9 @@
 
 
 
 
 
 
 
 
 
 
1
+ - dataset:
2
+ id: IntelligenceLab/Long-Horizon-Terminal-Bench
3
+ task_id: lhtb2_solved
4
+ value: 12
5
+ date: '2026-09-19'
6
+ source:
7
+ url: https://github.com/zli12321/LHTB/tree/lhtb-2.0#lhtb-20-results-october-2026
8
+ name: LHTB 2.0 results
9
+ notes: "12/77 tasks solved at reward >= 0.95 (pass@0.95); mean reward x100 = 54.2; LHTB 2.0 (77 tasks), terminus-2, 3-hour budget per task, official LHTB Harbor harness"