riomhaire schema REST Gary Leeson

What's wrong with REST

Posted by Riomhaire Research on Monday, April 5, 2021

The Representational State Transfer architecture (REST Reference-1) is a common approach to the API design of micro-services(Reference-2). REST does have some issues which have led to alternative architectures being developed to solve them. The popular GraphQL framework is one of them(Reference-3,7).

This article will not go through all the pros of REST versus GraphQL or other alternatives to REST. Others have produced many good answers on the matter - just Google it. What we intend to describe from a developers perspective is how we can address two of the most common complaints:

  1. Inappropriate Payload
  2. No Payload Schema

There are two main usable patterns of Data APIs: Service to Service and User Interface to Service. It is the latter case where complaints tend to arise from a data service developer’s perspective. Query and Transfer times can be expensive over the internet, and UI developers try to shave off every millisecond they can to produce as responsive a UI as possible. This is understandable and it is as it should be - focused on the end customer. The problem is best described by a simple example: A list of things.

In an application UI, there is often a need to list several things for the customer to look at. This can be a list of messages or of things to buy from a shop for example. The UI may want to display this list in different formats (list, grid, titles, etc) and different parts of the application. In modern frameworks, each of these views may be developed as independent UI components. From a service perspective though they are usually implemented as one endpoint on the same data, and it is common for the data service developer to give the UI all the data for the various perspectives and then some. This saves the data service developer from developing different endpoints for each use case, but this comes at the expense of performance in query time and data transfer costs for the UI Developer. So is there a middle ground?

Mitigation Approach

The service API developer has to weigh the various client users and their demands. Generally, it is good practice to provide at least three levels of data population in conjunction with the usual mix of pagination and search parameters to cover most client use cases. We can call these:

  1. Minimal list
  2. Summary list
  3. Detailed list

The ‘Minimal list’ payload usually only includes the records keys and a very small number of extra data fields. So if we use a shop website selling goods a minimal list may include:

[
  {
    "id":"124242424",
    "title":"Taylor of Old Bond Street Cedarwood Shaving Cream Bowl, 150g",
    "rating-level": 4.5,
    "ratings":1432,
    "price":12.88,
    "currency": "Euro",
    "image":"https://products.com/product/124242424.jpg"
  }
  .....
]

The ‘Summary list’ payload enhances the data with more detailed information:

[
  {
    "id":"124242424",
    "title":"Taylor of Old Bond Street Cedarwood Shaving Cream Bowl, 150g",
    "description":"A classic shaving cream for the ........",
    "rating-level": 4.5,
    "ratings":1432,
    "price":12.88,
    "currency": "Euro",
    "image":"https://products.com/product/124242424.jpg",
    "ingredients":"Aqua, Steric Acid, Myristic Acid, Potassium Hydroxide, Coconut Acid, Glycerin, Allantoin....",
    "instock":3452
  }
  .....
]

Finally, we have the ‘full fat’ ‘Detailed List’ which would include most if not all of the fields for the product and may include several reviews, questions, or answers.

[
  {
    "id":"124242424",
    "title":"Taylor of Old Bond Street Cedarwood Shaving Cream Bowl, 150g",
    "description":"A classic shaving cream for the ........",
    "rating-level": 4.5,
    "ratings":1432,
    "price":12.88,
    "currency": "Euro",
    "image":"https://products.com/product/124242424.jpg",
    "ingredients":"Aqua, Steric Acid, Myristic Acid, Potassium Hydroxide, Coconut Acid, Glycerin, Allantoin....",
    "instock":3452,
    "reviews": [....],
    "questions": [....],
    "firstStocked": 1617644228,
    "lastPurchased": 1617644328,
    "purchaseLeadTime":60,
    "shelfLocation":"w1r11s3b2"
  }
  .....
]

I am sure you get the picture. You could also have other perspectives ‘warehouse’ or ‘purchasing’ depending on the application.

How do we tell the API which data schema to return? - well that is covered in the next section.

Alternative Approaches

REST APIs in conjunction with the HTTP specification provide many different ways that a client could inform the data service of what representation it would like to receive. So let’s examine the main candidates.

None

This is the default position most REST APIs support. You get whatever has been decided on by the microservice team which has a “one size fits all” policy. If the data you need is not there then a change request to add the field or for a separate data endpoint to be provided. The main problem from a REST API point of view is that over time this can lead to payload bloat or a plethora of versioned endpoints to support. For simple APIs or minimal viable products (MVP) this might be fine, but it’s very restrictive in the long term if the API is used by many different clients and scenarios.

Query Parameters

Since we’re using HTTP we could use a query parameter that is populated by the client to indicate the schema or form to be returned by the REST data service. So for the above payload examples, you might have:

  https://somewhere.com/api/v1/products?page=1&schema=minimal
  https://somewhere.com/api/v1/products?page=1&schema=summary
  https://somewhere.com/api/v1/products?page=1&schema=detailed

Naturally, you would have a default schema value. This approach is easy to implement and very easy to debug. The downside of using a query parameter is that you are mixing request and response intentions.

Header Parameter

The next approach is to use a specific request and response HTTP header field(Reference-6) to indicate the schema to use and return. Common suggest header names are ‘Accept-Schema’ for requests and ‘Schema’ for a response. These are in addition to the usual Accept/Content-Type field which is used to define the media type format.

Request

Request: GET /some-resource HTTP/1.1
Accept: application/json
Accept-Schema:urn:company:schema:product

Response

Response: HTTP/1.1 200 OK
Content-Type: application/json
Schema: urn:company:schema:product

Again the intent here is clear and flexible for various scenarios.

Media Type Header

The final option you have is to use vendor-specific media types(Reference-5,6,9). In this variation, you support the schema type as part of the Accept/Content-Type header field - with a suitable default. Of all the approaches mentioned here, the media vendor extension is covered by an actual RFC(Reference-9).

An example where this is used can be seen in the IMSGlobal Names and Role Specification(Reference-4) where the schema representation is given in the content-type header returned:

Content-Type: application/vnd.ims.lti-nrps.v2.membershipcontainer+json

This means the data returned is in the form of JSON which conforms to the schema of type ‘vnd.ims.lti-nrps.v2.membershipcontainer’. The RFC details how the vendor-specific namespace should be defined. For our examples above we could define the Accept/Content-Type to be used as:

application/vnd.company.app.v1.product-list.minimal+json
application/vnd.company.app.v1.product-list.summary+json
application/vnd.company.app.v1.product-list.detailed+json

The intent is clear and flexible and easy to implement for clients and servers.

Conclusion

We hope this article has been useful. If you implement any of these approaches it is good practice to provide clear documentation or API documentation (such as Swagger(Reference-8)) to aid the client make the right decision for them. Things can be improved as well by the REST Service Developer providing the schema definitions in some common definition language which can be used to generate the data structures in whatever language the client chooses. Two common ones we have come across are JSON-Schema(Reference-10) or RelaxNG(Reference-11).

Stay Safe.

References

  1. Representational State Transfer https://en.wikipedia.org/wiki/Representational_state_transfer
  2. Microservices https://en.wikipedia.org/wiki/Microservices
  3. REST vs GraphQL https://www.moesif.com/blog/technical/graphql/REST-vs-GraphQL-APIs-the-good-the-bad-the-ugly/
  4. IMSGlobal Names and Roles https://www.imsglobal.org/spec/lti-nrps/v2p0#membership-container-media-type
  5. Vendor-Specific Media Types https://en.wikipedia.org/wiki/Media_type#Registration_trees
  6. An HTTP Header for Metadata Schema Negotiation https://www.w3.org/2016/11/sdsvoc/SDSVoc16_paper_14
  7. GraphQL https://graphql.org/
  8. Swagger https://swagger.io/
  9. Media Types https://tools.ietf.org/html/rfc6838
  10. JSON schema https://json-schema.org/
  11. RelaxNG https://en.wikipedia.org/wiki/RELAX_NG
CATALOG