dbt-labs/docs.getdbt.comOpenBrokejafgen --years 6 fails with No such option: --years.
Test your docs. Audit them against your code.
Doc-test runs your quickstart and proves each fix. Doc-audit finds the pages your code changed underneath.
Try
A partial URL works. So does a tool name. Nothing leaves your browser.
Verified
Watch the replayA saved run. Nothing to install.
Run it fresh with your agent
- Your agent installs FirstRun
- Runs your quickstart clean
- Proves the fix in a second run
- Opens the PR after your yes
$ pip install git+https://github.com/HighnessAtharva/firstrun
$ firstrun https://docs.getdbt.com/guides/duckdb
$ firstrun audit owner/repo # Doc-audit from the terminal
Paste a URL or a Repo and Get Proof Back
Doc-testSee every run
Doc-auditOpen the dbt-core audit
123- 1Stale pagesThe code moved after the page did.
- 2Features with no docsShipped commits that no page explains.
- 3Ship listOne board. Pick a card to see its commits.
Every Number Comes From a Real Run
Doc-test
16pages with a fix proven
8docs that work as written
Upstream
4docs PRs opened
Doc-audit
4repos audited
15stale pages found
neatlogs
$0.02model cost per run
Four upstream docs PRs, all open
dbt-labs/docs.getdbt.comOpen
dbt-labs/docs.getdbt.comOpenBrokeThe repo pins networkx==3.7, which needs Python 3.12.
entireio/cliOpenBrokeAfter a good install, entire version says command not found.
accuknox/helpOpenBrokeThe page names 0.9.0 as latest. The installer already fetches 0.9.58.
Doc-test replaysFix provenDocs work
Open Doc-test →Each Tool Caught a Real Bug in FirstRun
Each finding changed the code.
neatlogs
neatlogs.comRaised an incident and named the cause.
- Incident. 25 failures, 52.8x the baseline.
- Cause. A
cdstep broke the file check. - Fix. Commit 2de852f
- Proof. MkDocs went from a false fail to verified.
A second investigation took v2 from 2 to 3 of 4 quickstarts.
Entire
entire.ioIts review found a fix that never ran.
- Review. Trail 7 saw the
cdguard was never called. - Fix. Commit f87db02 wired it in.
- Record. Each commit carries the session that wrote it.
cfo.ai
cfo.aiFound a budget cap that ends the company.
- Model. 18 months built from 64 measured runs.
- Finding. The old $0.50 cap runs out of cash in month 14.
- Fix. The plan now sets the cap. See the plan
The Business Plan Found a Bug in the Agent
neatlogs measured the cost. cfo.ai modeled the cash. The cap moved into the code.
neatlogs measured
64 runs, one cost spreadMean $0.0167 a run. The worst run cost $0.1014.
drivers.csv, a trace per rowcfo.ai modeled
18 months from the tracesThree plans and six Scenarios. Every cost cell reads the measured drivers.
Open the public modelThe plan found
Cash runs out in month 14A $0.50 cap lets a Scale run cost five times its $0.0997 share.
Read the business planFirstRun fixed
The plan sets the cap
See the change, PR #15firstrun run --tier scalestops a run at $0.028.
Cash on hand, November 2026 to April 2028
Three cfo.ai Scenarios with the same customers. Only the cost cap or the Scale price changes.
The chart reads the cfo.ai figures from data/cfo-model.json.
Show the numbers
Figures read back from cfo.ai on 2026-10-10. Model costs are Sonnet 5.5 list price, which Anthropic did not bill. Customer counts are assumptions until a team pays.
Compare FirstRun with Claude Code, GitHub Actions, GitHub Agents
Five ways to test a quickstart. FirstRun stands out on three rows.
Yes Partly or with setup No
| Feature | FirstRun | Claude Code | GitHub Actions | GitHub Agents | Custom workflow |
|---|---|---|---|---|---|
| Runs the quickstart as a new developer | Every step runs in a fresh Docker container | Runs on your machine. An optional OS sandbox limits files and network | A fresh runner per job, but you script every step | Works in its own Actions-powered VM, on your repo, not a new-user setup | Only as clean as the environment you build |
| Plans steps from a docs URL | The planner writes one step per command | When you prompt it with the page | You write each step | Starts from an issue or task, not a docs page | You write the parser |
| Improvises when a step failsStands out | Classifies the failure, patches, retries up to 3 times per step | Iterates inside one session | The job fails and stops | Iterates in one session and pushes commits to its draft PR | Only what you code |
| Proves the fix | A second clean container must pass the whole quickstart | Only if you ask for it | Only if you script a second run | Your repo's own tests and CI, not a clean-user rerun | Only if you code it |
| Telemetry per runStands out | neatlogs spans for planner, executor and verifier. Measured cost of $0.02 per run | OpenTelemetry metrics and events, such as cost and tokens. Set up through environment variables | Logs and run history. No agent traces | A session log you can read or stream. No cost trace you own | You build it |
| Opens the docs PR | After you approve once | When you ask | Only if you script it | Opens a draft PR and asks you for review | Only if you code it |
| Ease of useStands out | Paste a docs URL, then paste the prompt into your coding agent | One prompt, but you define the test and the clean setup | A YAML file per quickstart | Assign an issue to Copilot | Highest effort |
| Proof in public repos | 4 upstream PRs, all open | Not specific to docs | Not specific to docs | Not specific to docs | Depends on your build |
The Build Happened in Public
Every step is a post on @cultist_dev.
- Sat Oct 10, 10:45
my neatlogs hackathon project has already caught doc issues for a chroma (24K+ github stars). chromadb's deprecated-config error sends you to its own migration guide. that page returns 404.
Read the post on X - Sat Oct 10, 12:30
#neatHack kickoff, 12:30 IST. FirstRun code before kickoff: 0 lines. neatlogs doctor probe: 7/7 PASS. Solo build. Submission opens Monday, 7 PM IST.
Read the post on X - Sat Oct 10, 13:14
#neatHack, day one. all three required tools are up. Entire: trail #1 created. neatlogs: first agent trace, 3 steps, 12s. cfo.ai: Ari read my public profile.
Read the post on X - Sat Oct 10, 13:26
your quickstart is the only part of your product tested by people who leave instead of filing a bug. so for #neatHack i'm building FirstRun. break. fix. prove.
Watch the video on X - Sat Oct 10, 15:15
a quickstart breaks the way an agent breaks. quietly. so the plan for this weekend: every FirstRun run becomes a trace in @neatlogs.
Read the post on X - Sun Oct 11 to Mon Oct 12More posts land here as the weekend goes.
Atharva Shah
Atharva is a DevRel engineer. He owns developer docs at AccuKnox and works with Altimate AI. He built FirstRun alone during the hackathon, in public.