Staging instance, all changes can be removed at any time

Skip to content

[POC] OpenAPI and Django REST Framework to specify / implement API v2

Antoine Lambert requested to merge generated-differential-D4629-source into generated-differential-D4629-target

That diff showcases the result of my experiments using OpenAPI and Django REST Framework for the implementation of Software Heritage Web API v2. I mostly focused on how to plug an already defined OpenAPI specification using Django. Once all that plumbing implemented, the idea is to drive API v2 development by its specification.

It is NOT intended to be landed. Its purpose is to discuss about the implementation approach.

Introduction to OpenAPI

The OpenAPI Specification is a specification for machine-readable interface files for describing, producing, consuming, and visualizing RESTful web services.

The current version is 3.0.3 and it enables a fine grained specification for a REST API. It uses an extended subset of JSON Schema Specification Wright Draft 00 (aka Draft 5) to describe the data formats. It also enables to split API specification in multiple files based on the JSON reference specification.

It is language-agnostic. With OpenAPI's declarative resource specification, clients can understand and consume services without knowledge of server implementation.

It is a broadly adopted industry standard for describing modern APIs (see GithHub API description for instance).

It exists a lot of tools that works with OpenAPI, notably to:

  • parse and validate specification
  • validate HTTP requests and responses according to an endpoint specification (parameters, headers, body, ...)
  • generate API client libraries or server stubs (OpenAPI Generator for instance) from a specification
  • generate interactive HTML documentation for a specified REST API (the most famous one is swagger-ui)

OpenAPI and Django REST Framework

I did some reviews of open source projects using OpenAPI with DRF and found the following:

However, all those projects follow the mantra: implement first then generate OpenAPI specification.

From my point of view, this is not the right one to follow and it should rather be: specify with OpenAPI first then connect to implementation. It is notably used by:

I did not found any OpenAPI based project for Django REST Framework using that latter mantra so I tried to implement it in that POC.

Software Heritage API v2 implementation proposal

As explained above, the idea is to be able to specify first then connect to endpoints implementation.

Proceeding like this has several advantages:

  • specification can be validated for correctness using dedicated tools
  • specification can be organized in multiple logical files in order to share and reuse components (schema, parameters, responses, ...)
  • input HTTP requests can be validated according to endpoint specification (parameters format, expected headers, ...)

In order to ease those tasks, I used two interesting Python modules that I found:

  • prance: OpenAPI specification parser, resolver and validator
  • openapi-core: OpenAPI request and response validator compatible with Django

In that diff, the idea was to specify a couple of endpoints related to Software Heritage content objects and connect these specified endpoints to their implementation. I inspired from the preliminary work of @douardda on the subject to write the specification but I splitted the specification into multiple files.

Once the specification written, swh-web will use it to register the API endpoints by following that process at startup:

  1. The specification is parsed and validated using prance
  2. For each specified API path, create a DRF API view wrapping a function implementing the endpoint logic. The name of that function can be found in the path specification under the operationId property.

That diff also adds the following:

  • HTTP requests are validated using openapi-core prior executing endpoint implementation logic
  • HTTP responses are validated in debug mode
  • Full dereferenced OpenAPI specification is made available under the /api/2/schema path
  • Web API v2 interactive HTML documentation (based on swagger-ui) is made available under the /api/2/doc path (see screenshot below)

image

To summarize, OpenAPI is quite pleasant to use and the variety of tools working with it will help us to get a properly specified and documented Web API v2 while being able to change the implementation backend.

Related to #1805

Migrated from D4629 (view on Phabricator)

Merge request reports