Skip to content
GitLab

Adds openapi v3.0.3 unit test spec, includes test cases, autogenerates model tests with them

Merged Administrator requested to merge github/fork/spacether/feat_adds_unit_test_spec into master

Created by: spacether

What Is This

This PR adds an auto generated openpai v3.0.3 spec document which includes schemas and tests from Json Schema Draft6 tests That spec is used to auto generate a python client using the python-experimental generator That client includes the auto generated tests that came from the json schema test suite Those tests are run in our CI (in node 4) to verify that the json schema features are working correctly in component schemas for the python-experimental generator.

Why Do This

Because our petstore spec is not the best way to unit test our features. We need a more basic unit test spec which includes json schema and openapi feature tests. This is that spec.

Why Put The tests In components.x-schema-test-examples

There are a number of places one could put the tests case data. They are:

  • component.schemas.SomeComponent.examples: not present in v3.0.3
  • component.schemas.SomeComponent.example: I want to store multiple tests, not just one
  • component.schemas.SomeComponent.x-schema-test-examples: this would work most of the time, but it's possible to have a schema whose value is true or false in v3.1.0 so one could not have that and use SomeComponent.x-schema-test-examples
  • component.examples: there is not any standard field to store whether an example should pass or fail, it's unclear what component schema the example should be testing. Language in the openapi spec and json schema spec leans towards these examples be passing cases NOT failing cases. I want tests cases to be passing and failing
  • component.x-schema-test-examples.SomeComponent = {testName: testCase}: this allows positive and negative test cases, this allows for testing (future) boolean schemas, this allows for storing multiple tests

Details

  • Adds an openapi v3.0.3 test spec built off of json schema draft 6 tests
  • A python file processes draft6 json schema tests and puts them into an openapi spec document
    • Some tests are omitted because openapi excludes them because those features are not supported by openapi
    • Some tests are added because they are openapi specific
  • That openapi spec document is used to generate a sample and auto-gegnerate tests

When writing this PR, I found bugs in swagger-parser that block generating correct code or successfully running some unit tests:

Python-experimental Improvements

  • escapes newline, carriage return, backslash, double quote, tab, form feed, null character in property names + enum values
  • eliminated multiline strings by using the above escaping
  • fixes bug where maxProperties set to 0 was not written as a validation
  • fixed bug where uniqueItems saw 1 == True and 0 == False when they should be different values

Proposed Future Work (out of scope)

  • turn on the oneOf anyOf, allOf, infinite-loop-detection tests
  • test tx and rx of request and response json bodies
  • test serialization and deserialization of parameters(query/path/body/cookie/header)
  • add openapi specific feature tests
  • With this feature, one could use a tool like https://github.com/json-schema-faker/json-schema-faker to generate passing examples, write them into a test spec and then auto-generate tests against schemas + an api.
  • Add a generate command line additional property skippedTests or input file which allows generators to skip specific tests because they would fail

PR checklist

  • Read the contribution guidelines.
  • Pull Request title clearly describes the work in the pull request and Pull Request description provides details about how to validate the work. Missing information here may result in delayed response from the community.
  • Run the following to build the project and update samples: 
    ./mvnw clean package 
./bin/generate-samples.sh
./bin/utils/export_docs_generators.sh
    Commit all changed files. This is important, as CI jobs will verify all generator outputs of your HEAD commit as it would merge with master. These must match the expectations made by your contribution. You may regenerate an individual generator by passing the relevant config(s) as an argument to the script, for example ./bin/generate-samples.sh bin/configs/java*. For Windows users, please run the script in Git BASH.
  • File the PR against the correct branch: master (6.0.1) (patch release), 6.1.x (breaking changes with fallbacks), 7.0.x (breaking changes without fallbacks)
  • If your PR is targeting a particular programming language, @mention the technical committee members, so they are more likely to review the pull request.