How to test WebMCP beyond “the tool registered”
Meaningful evidence covers contracts, authorization, selection, state changes, and verified outcomes.
The short version: Registration is presence evidence; production confidence requires several independent layers of behavioral evidence.
Build an evidence ladder
Source inspection can find feature hints. Contract tests can validate names and schemas. Runtime tests can observe registration, invocation, cleanup, and UI synchronization. Journey evaluations can measure whether an agent chooses tools and completes real tasks. Keep these layers separate so one does not impersonate another.
- Source: bounded hints in fetched or local code
- Imported: developer-provided contracts with unverified provenance
- Runtime: observed browser registration and execution
- Measured: recorded outcomes, timings, actions, and assertions
- Inferred: labeled conclusions with an explicit confidence level
Reject malformed and excessive inputs
For every tool, test missing required fields, unexpected fields, wrong types, boundary lengths, invalid enum values, duplicate array items, and unsafe identifiers. Confirm errors are structured and do not leak stack traces or private data. Then repeat critical checks at the trusted server boundary.
Test identity and authorization independently
Use fixtures for anonymous, authorized, wrong-tenant, expired-session, and insufficient-role states. A schema-valid request must still fail when the actor lacks authority. Test object-level access, not only whether a route is signed in.
Measure selection with fixed prompt sets
Write prompts that should invoke each tool, prompts that should invoke a neighboring tool, and prompts that should invoke nothing. Freeze the tool inventory, model, instructions, and temperature where the environment allows it. Run enough trials to report rates rather than one lucky demonstration.
Inspect false positives as carefully as misses. An agent that calls a mutation too eagerly can be more dangerous than one that asks for clarification.
Verify postconditions and compare complete journeys
A resolved promise is not proof. Check the authoritative record, visible application state, and any required audit event. For before-and-after comparisons, use the same fixture and success criteria. If one run fails, make that failure the headline instead of averaging it into an efficiency score.
Evidence note. isWebMCP calls this measured difference WebMCP Lift. A number is available only for paired interactive runs of the same task and fixture; authored replay is illustrative and receives no score.
Primary references
Read the sources
Put it to work
Use the guide on a real product surface.