API driven services

Abstract

This page provides general guidance on the use, implementation and co-ordination of API-based services within UIS. The Technical Design Authority also has a page detailing specific technical standards for APIs.

This document was last updated in May 2022 and may be cited as version a630d6b.

Introduction

Historically the University have developed services in isolation; services assume that they are the entire world. This has made it hard to ensure consistency between services. For example, the HR system, our student information system, the central directory and our University Card database have differing views of the organisation of the University and present technical challenges in ensuring consistency between them. Developing service in isolation additionally risks duplicating work or having different services all solving the same problem in incompatible ways.

APIs enable services to communicate amongst themselves and can be used to move to a more "single source of truth"-style architecture as opposed to a "re-invent the world"-style one. They help "free" data from being locked in one system and encourage data re-use.

What is an API?

"API" (ay-pee-eye) is an initialism which expands to "Application Programmer's Interface". It is a generic term for technologies which allows services to communicate with each other to exchange information or to automate tasks without human involvement.

The University Lookup service provides an API which allows services to retrieve basic information about people such as their preferred name. As an example, the subject moderation interface (SMI), used as part of the Admissions process, retrieves a display name from Lookup and an avatar image from G Suite@Cambridge to provide a friendlier name than bare CRSid:

Information on applications is retrieved from APIs hosted by CamSIS. CamSIS then retrieves updates made in the SMI by means of a second API.

Unlike many UIS services, APIs are not intended to be used directly by end-users. Rather they are used by service implementers to integrate services together. An API can provide a "single source of truth" for information so that it does not need to be copied repeatedly between services.

Note

By "single source of truth" we mean that there is a single well-documented place to retrieve an item of information for re-use by other systems. This place need not necessarily be where the information originated or the "authoritative source".

Why is an API valuable?

The value of having an API-ecosystem around service development and delivery lies in the ability of services to talk to one another to share information or to automate cross-service tasks.

For those co-ordinating service delivery

Planning to engage with an API ecosystem allows services to spend less time reimplementing the same features over and over again and more time developing the unique features.

Those co-ordinating service delivery can benefit from an API ecosystem but share some responsibility for encouraging it by allowing time for those developing a service to design in APIs. Metaphorically this "lifts everyone's boat" in that those APIs may be used in future to build atop a service rather than replacing or reimplementing it.

For those procuring products from vendors

APIs are vital to enable deep integration of third-party products with existing systems and services. A rich and deep API is a sign that a product has been designed to work well in a varied ecosystem and helps ensure that the product remains integrated as services change over time. Having a well-defined API allows UIS to avoid vendor lock-in and keep an option to exfiltrate data if necessary.

For those developing services

Providing an API for your service will enable it to form part of automated workflows and be integrated into other services offered around the University. An API allows your customers to layer their own functionality atop your service without waiting for you to add functionality.

Relevant Technical standards

Standards and implementation surrounding API technologies are mature at this point. The TDA has gathered many of them into a single standard for APIs in UIS. We strongly recommend that the standards be followed whenever possible for new services.

The API Service

Warning

The API Service is in early alpha. Documentation and procedures are being created in colaboration with an initial set of customers. This section was correct at time of writing but the API Service documentation should be considered as the most up-to-date available.

The API service aims to make it easier for those within the University (and UIS) to make use of APIs and to make it easier for those offering APIs to publish and document them.

For those using APIs

We will primarily discuss use of the API service from the service implementer's perspective but it is worth providing an overview of what the service looks like for those using an API.

The API developer portal provides a catalogue of APIs offered by the service along with documentation and the ability to experiment with the API directly in the browser.

For example, the documentation for the University Card API provides a description of each endpoint along with a "try this API" section:

Any University member may sign in to the developer portal via Raven and create a new development team and application within that team. An application is granted access to one or more APIs. Registration of some APIs is self-service whereas some APIs require manual approval. Each application is given a set of "client credentials" which allow them to authenticate and a set of "scopes" which specify what actions within an API they can perform.

For example, an application may have access to the "card.read" scope which allows reading information on University cards but not the "card" scope which allows reading and modification.

This registration process is transparent to those implementing the APIs.

For those developing APIs

The API service provides common functionality for APIs such as:

  • a self-service developer portal with interactive documentation,
  • rate-limiting and quota enforcement,
  • transparent request/response caching,
  • usage and request latency monitoring,
  • dynamic threat protection,
  • authentication,
  • optional manual approval of applications, and
  • restricting application abilities via OAuth2 scopes.

In order to publish your API via the API service you will need a description of your API as an OpenAPI specification document and to run a backend service which implements your API.

The API service will use your backend service to perform any actions requested by your users. For example, suppose you are creating a "colleges" API which lists the colleges of the University and some information about them. You will need to create a backend service, for example at https://colleges.uis.cam.ac.uk/.

Users of the API access it at https://api.apps.cam.ac.uk/colleges. A request to https://api.apps.cam.ac.uk/colleges/botolphs is forwarded to your backend at https://colleges.uis.cam.ac.uk/botolphs. The request contains authentication information so that you can verify that the request has come via the API service and additional headers indicating the application which made the request, which API "scopes" it is allowed use and where the original request was made. The process is summarised in the following diagram:

Backends do not need to manage application registration, quota management or rate limits; they need only authenticate that an incoming request was initiated by the API service.

For the moment the only form of authentication supported by the API service is a time-limited cryptographically signed JSON Web Token. More authentication mechanisms may be added in the future as service needs become apparent.

How to evaluate an API service

This section provides guidance for those procuring and those developing services. The guidance helps determine if your service offers a useful API which meets the recommended technical standards.

For those procuring services

At the Request for Information (RFI) stage the following questions can be asked to help evaluate whether a particular product meets our API guidelines. These questions necessarily include a number of technical terms and so the service's Technical Lead should be involved in evaluation of responses.

  • Is your API served via HTTP over TLS, also known as HTTPS? Essential answer: yes
  • Does your product offer a REST-style API? Desirable answer: yes
  • Does your API use JSON-formatted request and response bodies? Desirable answer: yes
  • Can all functions within your UI be performed via the API? If not, which ones cannot? Desirable answer: all required functionality should be exposed by the API
  • Is there publicly available API documentation? Desirable answer: yes
  • Does your API have a machine-readable specification? Desirable answer: yes, and it is an OpenAPI version 3 or later specification
  • Alternatively, if there is not publicly available API documentation, may we see a copy under an appropriate Non-Disclosure Agreement? Desirable answer: yes
  • Have other customers integrated your product using the API? If so, can you give examples? Desirable answer: yes
  • Is your API available on the public Internet? Desirable answer: yes
  • How are requests to your API authenticated? Desirable answer: see below
  • Are there any quotas or rate-limits for your product's API? If so, what are they? Desirable answer: any limits are well within expected use

If the answers to these questions do not match the desirable ones, you should engage with the TDA for comment and guidance.

In addition, the following questions should be asked and the Technical Lead should evaluate answers with reference to the TDA API standards:

  • Is UTF-8 used for all textual encoding?
  • How are dates and times represented?
  • Does the API allow to download whole datasets in bulk? If so, in which format?
  • How is application-level authorisation is managed?
  • How is user-level authorisation is managed?
  • Are the APIs versioned? If so, how?
  • What are the API deprecation policies?
  • What are the limits of scalability of the APIs?

It is particularly important to consider the API early if the service is likely to form part of the central University API service. The TDA can help you contact the right people within UIS if this is likely.

The authentication of APIs is a large subject and full of subtleties. In general we recommend that the following authentication schemes be used in decreasing order of preference.

  • A temporary access token passed via HTTP headers. The temporary token is obtained via OAuth2 client credentials grant and may be refreshed via OAuth2 refresh token.
  • A temporary access token passed via HTTP headers in the form of a JWT signed using a public-key cryptography scheme.
  • Mutual Transport Layer Security where the API service and your backend authenticate one-another by means of a set of TLS certificates.
  • A fixed access token passed via HTTP headers.
  • HTTP basic authentication.

If the vendor's preferred authentication scheme doesn't fit into any of the above, we recommend you engage with the TDA to discuss it.

For those implementing services

Warning

The API service described in this document is in early alpha. The service's managers can be contacted by opening an issue on the related Developers' Hub project or by sending an email to gitlab+uis-devops-api-apigee-ops-2126-issue-@developers.cam.ac.uk.

If you are implementing a service which provides an API, you should ask yourself the same questions as in the section above. In particular, you should strive to make your sevice compatible with the central API service. The TDA can help you contact the right people within UIS if the documentation on that site is not sufficient.

As noted in the "Relevant Technical Standards" section above, the TDA has an API standards document. These standards provide a sensible consensus position on what a "modern" API looks like and must be followed. If you are unable to follow them, you should engage with the TDA to explore why.