A Step CI alternative for OpenAPI contract testing
Step CI is not archived, but its repository has had no commits since August 2024 and its last release (2.8.2) was in June 2024. If your main use is checking a REST API against its OpenAPI spec in CI, Portman generates those tests from the spec and is actively maintained. If you rely on Step CI’s multi-protocol workflows (GraphQL, gRPC, tRPC, SOAP) or load testing, Portman does not cover them.
About Step CI
Step CI is an open-source API quality assurance framework under the Mozilla Public License 2.0. You describe test workflows in YAML, JSON or JavaScript, covering REST, GraphQL, gRPC, tRPC and SOAP, and run them with the stepci CLI; it can generate a workflow from an OpenAPI 3 spec and has fuzzing and load-testing modes. The repository is not archived, but its last commit was on 3 August 2024.
Feature comparison
| Capability | Portman | Step CI |
|---|---|---|
| Approach | Generates a Postman collection with tests from the spec | Hand-written or generated YAML/JSON workflows |
| Spec input | OpenAPI 3.0 and 3.1 | OpenAPI 3.0 and higher (stepci generate) |
| Protocols | HTTP (REST) | REST, GraphQL, gRPC, tRPC, SOAP |
| Output and runtime | Postman collection (JSON), run with Newman | Workflow file run by the stepci CLI |
| Response schema validation | Yes — JSON Schema test per response, generated | Yes — schema check per step, inline or $ref to OpenAPI |
| Fuzzing and negative testing | Partial — fuzzing from required, min/max and length constraints | Naughty-string fuzzing via templates |
| Stateful tests and chaining | Configured — assignVariables and overwrites | Yes — captures between steps |
| Load testing | No | Yes — loadtest mode |
| CI usage and exit codes | Exit code 1 on any failed assertion | CLI and GitHub Actions integration |
| MCP / AI agent support | No MCP server; Apideck agent skill available | Not documented |
| License and status | Apache 2.0, maintained | MPL 2.0; last commit Aug 2024 |
Choose Step CI if…
- You test GraphQL, gRPC, tRPC or SOAP services in the same workflow as REST.
- You use the built-in load-testing mode.
- You prefer hand-authored YAML workflows over tests generated from a spec.
Choose Portman if…
- You want contract tests generated for every operation from your OpenAPI spec, and regenerated when the spec changes.
- You want a tool with recent releases and active maintenance.
- You want the tests as a Postman collection your team can open and debug, optionally synced to a Postman workspace.
Migrating from Step CI
Step CI workflows are hand-written step lists; Portman generates the requests from the OpenAPI spec and you describe only the extra behaviour in a config file. Map each Step CI concept to its Portman equivalent.
- Generate a collection with default contract tests from the same spec you used with stepci generate: npx @apideck/portman -l openapi.yaml -b https://staging.example.com -o tmp/collection.json
- Replace status and schema checks in steps with contractTests in portman-config.json (statusSuccess or statusCode, schemaValidation, contentType, headersPresent).
- Replace checks on specific response values with contentTests.
- Replace captures with assignVariables, and use overwrites to inject the captured values into later requests.
- Replace naughty-string fuzzing with Portman fuzzing inside a variation test, which builds invalid requests from your spec’s required, minimum/maximum and minLength/maxLength constraints.
- Move env values to PORTMAN_* environment variables or an --envFile.
- Run with --runNewman in CI. Portman exits with code 1 on any failed assertion.
- Keep a separate tool for load testing and for non-HTTP protocols; Portman does not cover them.