Portman vs Specmatic
Pick Specmatic when you want one tool to turn API specs (OpenAPI, AsyncAPI, gRPC, GraphQL and more) into contract tests, mocks for consumers and backward-compatibility checks, including an MCP server for coding agents. Pick Portman when your workflow is built around Postman and Newman and you want the spec turned into an editable Postman collection with contract, content and variation tests.
About Specmatic
Specmatic turns API specifications into executable contracts: it runs contract tests against a provider, serves mocks (service virtualization) from the same specs for consumers, and checks specs for backward-compatible changes. It supports OpenAPI/Swagger, AsyncAPI, GraphQL SDL, gRPC proto, Arazzo and WSDL. The core repository is MIT-licensed and actively maintained, and Specmatic documents an MCP server for agent workflows.
Feature comparison
| Capability | Portman | Specmatic |
|---|---|---|
| Approach | Generates a Postman collection with injected tests from the spec | Executes the spec directly as contract tests and mocks |
| Spec input | OpenAPI 3.0 and 3.1 | OpenAPI/Swagger, AsyncAPI, GraphQL, gRPC, Arazzo, WSDL |
| Output and runtime | Postman collection (JSON), run with Newman | CLI (Maven / Docker image) and Studio UI |
| Test generation | One request per operation from spec examples, plus configured variations | From spec examples, or generated when no examples exist |
| Response schema validation | Yes — JSON Schema test per response | Yes — responses checked against the spec |
| Fuzzing and negative testing | Partial — rule-based fuzzing from required, min/max and length constraints | Yes — generative / resiliency tests |
| Stateful tests and chaining | Configured — assignVariables, overwrites and integration tests | Arazzo listed among supported spec formats |
| CI usage and exit codes | Exit code 1 on any failed assertion | CLI designed for CI/CD |
| Mocking / service virtualization | No | Yes — mocks served from the same spec |
| Backward-compatibility check | No | Yes |
| MCP / AI agent support | No MCP server; Apideck agent skill available | Yes — Specmatic MCP (contract, resiliency, mock, compatibility tools) |
| License and status | Apache 2.0, maintained | MIT (core repo), maintained |
Choose Specmatic if…
- Consumers need mocks generated from the same spec the provider is tested against.
- You want breaking-change detection between spec versions as part of review.
- Your APIs span AsyncAPI, gRPC, GraphQL or SOAP, not only OpenAPI.
- You want coding agents to run contract tests and mocks through an MCP server today.
Choose Portman if…
- Your team lives in Postman and Newman and wants the tests as a Postman collection it can open, edit and debug.
- You want value-level content tests, response-time limits and header checks configured per operation.
- You want to keep a Postman workspace collection in sync with the spec from CI.