Case files // 03
Open the
evidence.
Two flagship investigations and the supporting work around them. Everything below is public on GitHub — specs, safety rules, tests, scorecards, and reports included.
Case file AF-001 // shipped · MIT license
AgentForge
One spec. Many agents. Agentic coding assistants each want their own config dialect —
CLAUDE.md and hooks here, AGENTS.md there, .cursor/rules somewhere else. AgentForge separates
the portable agent posture from each tool's file formats: author identity, routing, memory, automation,
and telemetry once in canonical YAML, then compile platform-native files for every runtime you use.
What ships
- Six adapter targets: Claude Code, Codex, Gemini CLI, Cursor, Aider, and a portable generic AGENTS.md.
- A real CLI:
agentforge init <adapter>andagentforge doctor, published to npm as@kmitops/agentforge@0.3.1— runnable via npx with no clone. - Proof, not promises: adapter round-trip tests, package install smoke tests, a readiness runbook, and a visual demo.
- Safe to re-run: idempotent installs; every install creates a git-tracked checkpoint so rollback is one command.
- Benchmarks built in: the AgentForge suite includes reproducible agent evaluations — ATT&CKLens (below) is its first defensive-security benchmark.
- Stack
- Node.js ≥ 18 · npm · bash · YAML
- Registry
- npm — @kmitops/agentforge@0.3.1
- Verification
- Round-trip tests · install smoke tests · doctor CLI · readiness runbook
- License
- MIT
Case file AL-002 // published · MIT license
ATT&CKLens Benchmark
The question: can AI coding agents build secure, uncertainty-aware, MITRE ATT&CK-aligned defensive security tooling? Seven agents got identical specifications, standardized acceptance tests, and prompt-injection challenges. Their output was scored against a published rubric — mapping discipline, evidence citation, confidence handling ("No Clear Mapping" when warranted), detection ideas, remediation guidance, and the follow-up questions a real analyst would ask.
Final scores — 7 agent artifacts, rubric of 100
Full methodology, scorecards, raw outputs, and logs live in the repo. The authoritative
report is evaluations/final/final-report.md.
Why it's credible
- Reproducible by design: one branch per evaluated agent plus a main harness branch — clone, fetch all, run
benchmark-all.ps1, and the harness re-scores every submission. - Defensive framing throughout: agents are graded on detection and mitigation quality and on resisting prompt injection — not on building offensive capability.
- Honesty is scored: the rubric rewards "No Clear Mapping" over confident guessing, plus evidence citation and analyst follow-up questions.
- Everything preserved: scripts, scorecards, raw outputs, logs, branch inventory, acceptance tests, and a final report.
MITRE ATT&CK® is a registered trademark of The MITRE Corporation. ATT&CK content is used under MITRE's Terms of Use. The benchmark is independent work, not affiliated with or endorsed by MITRE.
Supporting evidence
Also on file.
Vulnerability Management Mini Program
Dashboard-style vulnerability workflow with CRUD, KPI cards, search, filtering, severity states, and operational status views aligned to how analysts actually track remediation.
View repository →Security Log Anomaly Detection
SOC-style detection work for authentication and network anomaly triage — rule-based and statistical approaches with output written to be read by a human analyst, not just parsed.
View repository →Reproduce it.
Then call me.
The strongest interview question you can ask me is "walk me through your benchmark harness." I'd love that.