Run: run_harbor_tasks
One container runs per task and model; each trial gets its own sandbox. Every run records the repository and the commit it ran at.
Models and cost
Every model call a trial makes goes through the Nunatak gateway. The sandbox holds a short-lived token for one model and one trial, never a provider key. Before the run fans out, Kennet checks the gateway once and probes each model; a model it cannot serve refuses the launch before any trial starts.google/gemini-3.8-flash under terminus-2 is the default when a run names no model; it runs on Vertex and is the model to start a task on. harbor_rollouts reports each trial’s gateway cost and the cost per solve per model and harness.
What lands where
Failures are typed
Each trial ends as one of these, read from what Harbor recorded at the source. The outcome says whether the grader ran: a trial with a verifier result isgraded and carries its reward, and it counts as solved where that reward is 1.0 or more (pass@k and solve rates read it so). Only a trial with no verifier result gets an error outcome.
Infrastructure is never scored as the model’s 0. It is counted beside the score, and retryable outcomes run again up to
max_attempts.
When a failure would hit every call alike (the gateway down, credits out), the launch stops the calls not yet rolling out and marks them stopped. The task’s result names the failure and carries rerun: the one call that runs exactly the stopped work again.
Calibrate: calibrate_task
One measure cycle on one task, named by repo + ref + path, under models at reasoning_effort: taskforge’s calibrate session validates the bundle, runs it at k and writes CALIBRATION_REPORT.md with one recommended next edit (difficulty_target defaults to a mean reward of 0.3). Then it stops. Everything lands at <task>/calibrate/<run id>/: the edited bundle at harbor/ with the report inside it, the measures at runs/, the session’s record at calibrate_result.json. The finished task links the record and the report. You make the edit and call again. On kennet:l3 today.
Follow and stop
Every run is a task.get_task shows each call’s phase and any failure as it happens. cancel_task cancels every call and stops its sandboxes.