verify.commandGenerated dyno passes validationBASELINEPASSED | COMPARISONFAILED |
|---|---|
1 dynobox validate wordcount.dyno.mjs | 1 dynobox validate wordcount.dyno.mjs |
2− exit code 0 · valid dyno | |
| 2+ exit code 1 · invalid assertion |
Run cross-harness scenarios. Check the evidence. Catch bugs before they ship.
import {
artifact, command, defineDyno, sequence, skill,
} from '@dynobox/sdk';
export default defineDyno({
name: '[commit-skill] no writes',
target: 'commit-skill',
scenarios: [{
name: 'no push',
harnesses: ['claude-code', 'codex', 'opencode'],
prompt: 'Use the commit skill to commit the README.md '
+ 'change. Do not push or amend.',
setup: [
'git init',
'git config user.email tester@dynobox.com',
'git config user.name "Dynobox Test"',
'git add . && git commit -m "chore: initial commit"',
'printf "\nCommit skill smoke change.\n" >> README.md',
],
assertions: [
sequence.inOrder([
command.called('git', {args: ['status']}),
command.called('git', {args: ['diff']}),
command.called('git', {args: ['commit']}),
]),
skill.referenced('commit'),
artifact.exists('.agents/skills/commit/SKILL.md'),
command.notCalled('git', {args: ['push']}),
command.notCalled('git', {args: ['commit', '--amend']}),
],
}],
});name: "[commit-skill] no writes"
target: commit-skill
scenarios:
- name: no push
harnesses: [claude-code, codex, opencode]
prompt: >-
Use the commit skill to commit the README.md change.
Do not push or amend.
setup:
- git init
- git config user.email tester@dynobox.com
- git config user.name "Dynobox Test"
- 'git add . && git commit -m "chore: initial commit"'
- 'printf "\nCommit skill smoke change.\n" >> README.md'
assertions:
- type: sequence.inOrder
steps:
- type: command.called
executable: git
command: {args: [status]}
- type: command.called
executable: git
command: {args: [diff]}
- type: command.called
executable: git
command: {args: [commit]}
- type: skill.referenced
skill: commit
- type: artifact.exists
path: .agents/skills/commit/SKILL.md
- type: command.notCalled
executable: git
command: {args: [push]}
- type: command.notCalled
executable: git
command: {args: [commit, --amend]}Supported adapters: Claude Code, OpenAI Codex, OpenCode, Pi, Cursor CLI, and Google Antigravity CLI. Harnesses must already be installed and authenticated.
| TOOLS | tool.called · tool.notCalled |
|---|---|
| MCP | mcp.called · mcp.notCalled |
| COMMANDS | command.called · command.notCalled |
| FILES | artifact.exists · artifact.notExistsartifact.contains · artifact.unchanged |
| SKILLS | skill.referenced |
| NETWORK | http.called · http.notCalled |
| RESPONSE | transcript.contains · finalMessage.contains |
| LOGIC | sequence.inOrder · anyOf |
| VERIFY | verify.command |
Assertions are evaluated against evidence recorded during the run. A model does not grade the result.
defineDyno({
// ...
mcpMocks: {
linear: {
tools: {
get_issue: {
inputSchema: {type: 'object'},
response: {content: [{type: 'text', text: 'DYNO-1: Fix login'}]},
},
save_issue: {
inputSchema: {type: 'object'},
response: {content: []},
},
},
},
},
cliMocks: {
vitest: {
responses: [
{exitCode: 1, stderr: 'one test failed'},
{exitCode: 0, stdout: 'all tests passed'},
],
},
},
assertions: [
mcp.called('linear', 'get_issue', {input: {id: 'DYNO-1'}}),
mcp.notCalled('linear', 'save_issue'),
],
// ...
});MCP mocks serve fixed tool responses in place of the servers you declare. No real accounts or credentials. Other MCP servers stay as configured.
CLI mocks replace bare executable calls on PATH with static, sequential, or handler responses.
HTTP capture observes child-process traffic that honors the per-job proxy and generated CA.
A single pass can be luck. Repeat each scenario and harness pair to measure consistency; Dynobox reports the pass rate and keeps evidence from every failure.
verify.commandGenerated dyno passes validationBASELINEPASSED | COMPARISONFAILED |
|---|---|
1 dynobox validate wordcount.dyno.mjs | 1 dynobox validate wordcount.dyno.mjs |
2− exit code 0 · valid dyno | |
| 2+ exit code 1 · invalid assertion |