Basis
Basis
Research that compiles.
Basis
← Back to examples

Technical brief · Reviewed Basis output

Technical Brief Example: Evaluating Coding Agents on Real Repository Tasks

A real 38-page Basis brief on repository-scale coding-agent evaluation, presented as a curated public proof page with the actual manuscript output front and center.

This brief shows Basis holding a long technical argument together across a full manuscript with cover, contents, section structure, bibliography, and a readable public edition.

Document facts

Document
Technical brief
Length
38 pages
Front matter
Cover · contents · main sections
Public edition
Full viewer with cleaned references
Objective
Design a repository-scale evaluation methodology for coding agents that focuses on executable contracts, gated scoring, failure modes, cost controls, and an internal pilot design that an engineering team could actually run.
Outputs
PDF · TeX · bibliography
Viewer
Full public edition of the real manuscript with cleaned front matter and internal artifact references removed.
Introduction page from the Basis coding-agent evaluation brief.

Run summary

What this run had to deliver.

  • Model each evaluation task as an executable contract with pinned repo state, explicit checks, and a controlled execution environment.
  • Separate hard pass-fail gates from diagnostic scoring so failures can be interpreted operationally.
  • Make auditability, cost control, and reviewer handoff part of the methodology instead of afterthoughts.

What persisted after the run

  • Public manuscript PDF
  • LaTeX source manuscript
  • Bibliography and references
  • Figures and diagrams used in the brief
  • Curated public viewer edition

Inside the output

What the document says.

Core move
Treats repository tasks as executable contracts instead of prompt-only exercises.
Scoring model
Uses binary gates plus diagnostic dimensions so reviewers can reason about failure modes.
Handoff surface
Persists as a manuscript, bibliography, integrity checks, and a bundled project export.

What the manuscript actually does

The brief frames each repository task as an executable contract: a frozen repository snapshot, a task specification aligned with real issues, an allowed-tools policy, a resource budget, and explicit acceptance checks.

From there it moves into operating detail rather than staying abstract. The manuscript spells out scoring gates, failure-mode taxonomy, instrumentation, reproducibility hooks, and an internal pilot design that could be used for a real evaluation cycle.

Introduction page from the coding-agent evaluation brief.
Introduction page establishing repository tasks as executable contracts.
Method section from the coding-agent evaluation brief.
Method section formalizing the contract object model and execution surface.
Continuation of the coding-agent evaluation brief.
Continuation of the framework with reproducibility, safety, and scoring detail.

Actual PDF

Read the output directly.

Full public edition of the real manuscript with cleaned front matter and internal artifact references removed.

Public manuscript PDF Open PDF

Human review

What was checked before this became public.

  • The public edition keeps the full manuscript shape, including cover and table of contents.
  • Placeholder author metadata was replaced and internal artifact-path references were removed from the public PDF.
  • Public claims stay tied to the output shape and the visible quality of the manuscript itself.

Where the example comes from

  • Prepared from an audited Basis manuscript reviewed on March 8, 2026.
  • The public edition comes directly from the reviewed run output with light cleanup for public release.

Use this document as the bar for your own run.

Start with a concrete question, explicit constraints, and the artifact package you expect to review at the end.

Create account

Public beta · email verification required · More examples