Variation tests
Add extra Postman requests per operation with changed inputs to test happy-path variants and expected 4xx/5xx error responses.
Variation tests generate additional Postman requests for an operation, each with modified request data and its own tests, so you can verify alternative happy paths and the error responses your API should return. They are configured under tests.variationTests and placed in a separate "Variation testing" folder in the collection.
Happy and unhappy paths
Contract tests cover the happy path using the example values from your spec. Variations cover:
- Happy path variations: for example every supported query parameter or header, each expecting a 2xx.
- Unhappy path variations: invalid input in the body, headers or query params, expecting a specific 4xx or 5xx response.
Typical uses: test API validation responses with invalid body properties, error responses for invalid requests, and invalid authentication responses.
Example
{ "version": 1.0, "tests": { "variationTests": [ { "openApiOperation": "*::/crm/*", "openApiResponse": "401", "variations": [ { "name": "Unauthorized", "overwrites": [ { "overwriteRequestSecurity": { "bearer": { "token": "bogus_token" } } } ], "tests": { "contractTests": [ { "statusCode": { "enabled": true } } ] } } ] }, { "openApiOperationId": "leadsAdd", "openApiResponse": "400::application/*", "variations": [ { "name": "missingParams", "overwrites": [ { "overwriteRequestBody": [ { "key": "first_name", "value": "", "overwrite": true } ] } ], "tests": { "contractTests": [ { "statusCode": { "enabled": true }, "jsonBody": { "enabled": true }, "schemaValidation": { "enabled": true } } ], "contentTests": [ { "responseBodyTests": [ { "key": "resource", "value": "leads" } ] } ] } } ] } ] }}The first entry creates an "Unauthorized" request for every CRM operation with an invalid bearer token and checks for the 401 defined in the spec. The second sends leadsAdd with an empty first_name and checks the 400 response's status, body and schema.
Targeting
| Property | Description |
|---|---|
openApiOperationId | OpenAPI operationId to create a variation for. |
openApiOperationIds | Array of operationIds. |
openApiOperation | Method and path, supports wildcards. |
excludeForOperations | Optional. Operations to skip. |
openApiResponse | Optional. Response code or code::content-type. Supports wildcards for the code (4*, *) and content type (text/*, *). Examples: "404", "200::text/plain", "2*::application/json", "*::*". Defaults to the first response in the spec. If the response code is not in the spec, no variation is generated for that operation. |
openApiRequest | Optional. Request body content type, supports wildcards like application/*. |
Variation properties
variations is an array. Each variation can have:
| Property | Description |
|---|---|
name | Name of the variation, used in the generated request name. |
openApiResponse | Optional. Expected response the variation's contract tests are based on. Falls back to the 2xx/302 response in the spec. |
overwrites | Request overwrites for this variation only. Same options as top-level overwrites, without targeting. |
fuzzing | Fuzzing options that generate requests with unexpected values. |
tests | contractTests, contentTests and extendTests for this variation. |
assignVariables | Variables assigned from this variation's response. |
Variation-level tests and overwrites do not need a target, because they apply to the variation they are defined in.
Content type variations
Variations are also how you test every request or response content type of an operation: create one variation per content type with openApiRequest or openApiResponse. See the testsuite-variation-content-types example.