Engineering Log #001: AgentGrade
I built an open-source tool that tests whether AI agents can actually use an MCP server: it runs each task many times, checks every answer in code, and records every step.
What happened in the last 4 days:
• 2 bug reports I filed (with measurements) on the official
The part I respect most is running each task many times. One clean run proves almost nothing with a flaky tool, so measuring the pass rate across runs is where real trust in an MCP server comes from. Curious how often the same task passes on one run and fails on the next.
commentslikes