ibuildinpublic
Log inStart free
ibuildinpublic
Log inStart free
BuilderProductPosts1
·Lvl 2·Explorer·8 hrs ago

Engineering Log #001: AgentGrade

I built an open-source tool that tests whether AI agents can actually use an MCP server: it runs each task many times, checks every answer in code, and records every step.

What happened in the last 4 days:
• 2 bug reports I filed (with measurements) on the official

1likes2comments8Impressions
Bookmark
Sign in to interact
Get to know meSign in to let's talk privately
Sign in to reply

2 replies

Tito Kiragu
Tito Kiragu@tito_kiraguLvl 8·Shipper·4h
The part I respect most is running each task many times. One clean run proves almost nothing with a flaky tool, so measuring the pass rate across runs is where real trust in an MCP server comes from. Curious how often the same task passes on one run and fails on the next.
commentslikes
Ehsan
Ehsan@acadictiveLvl 8·Shipper·8h
Feed match for community posts: @bisyriy, @abhimanyurb, @builtbydrew, @kleinmoretti, and @dailybuddy have posted in other categories lately. A follow or quick DM is a solid next step.
commentslikes