Evaluating REST Frameworks Part 1: A Maturity Model

REST has been a very important topic of discussion for a while. This series attempts to provide a comparison of some current frameworks that support the development of systems that should be REST-compliant, regardless of the underlying protocol (typically HTTP). To begin, we need a criterion to perform this comparison and that’s what we’ll build in this first article by

• Explaining what we understand about REST and what is it for.
• Making a list of what is needed to create a REST compliant application.
• Determining which features we may expect in a framework that supports REST development.
• Proposing a maturity model we can use as a tool for evaluation/grading of existing frameworks.

I’m not a maturity model fan, but they can be very handy for evaluation and comparisons since they define a usable criterion and in particular, a grade or level to assign.

A Quick REST Primer

Let’s start with a quick discussion of what REST really means. N.B. The objective of this article is not to provide an in deep explanation of REST. That is better left to Roy T. Fielding’s original dissertation and the O’Reilly book, REST In Practice by Jim Webber, Savas Parastatidis and Ian S Robinson. The Yahoo Rest Discuss group is also the best place to go for any further questions you might have.

REpresentational State Transfer, or REST for short, is an architectural style capable of supporting all the quality attributes required to construct The Web, namely: Anarchical Scalability, Low entry Barrier, Independent Deployment and the like. In general, it is focused on the transfer of large hypermedia content in a networked system. Its creator, Roy T. Fielding, developed it by first analyzing what quality properties were required for the modern web, and then grading several known architectural styles, picking and mixing the ones that better achieved those properties. Although the individual architectural styles can be applied independently, Fielding’s idea was to actually apply them in combination, and that is why he presented them in an incremental, additive way. REST is a combination of all the styles below. Let’s take a look at them in the same fashion as in his dissertation:

1. Client Server (CS). This is an old architectural style. The server provides a Service and the Client consumes it. The CS idea in REST is to be used for decoupling presentation from processing. The client deals with presentation and the server is the one holding the resources and doing any processing.
2. CS + Stateless Communication (SC). Since we need scalability, we need to avoid server side sessions. We do that by keeping all information needed for a transaction, in the message. That is, the messages are self-descriptive and no session information is held on the server.
3. CS+SC+Cache (C$). Since we are making the message bigger, and thus increasing communication cost, we need something to reduce network use and latency. A cache system is a nice addition.
4. CS+SC+C$+Uniform Interface (UIF). The UIF is the implementation of a known principle, called “The Principle of Generality”, but applied to interfaces. The idea is that all components share the same interface definition, a generic one, which follows four specially crafted constraints:
   1. All resources are uniquely identified.
   2. Manipulation of those resources is done using representations through messages.
   3. Those messages are self-descriptive.
   4. Hypermedia is used as the engine of application state. These constraints will provide better transaction visibility and simplifications of the architecture which will in turn lower the entry barrier and promote independent deployment.
5. CS+SC+C$+UIF+Layered System (LS). To support larger deployments and simplification, a layered style is proposed. Each component layer should be abstracted from the system’s complexity by only talking to an immediate layer above or below. Those layers can then in turn talk to other layers, etc. That makes it simpler to compose complex transactions while hiding that complexity from the current layer’s view. This adds overhead of course, but that can be reduced with shared caches.
6. CS+SC+C$+UIF+LS+Code On Demand (CoD). This is an optional style, and presents a way to help the clients. It allows the code that processes some data, to travel with that data from the server to the client, to process the data on the client side. This simplifies the client itself (there is no need for a client to know how to process every data type in the world).

As an architectural style itself, REST must define the elements an architect should use to create a compliant architecture. Here are the components defined by REST:

1. Data Elements: In REST, the main data element is the resource (which may be anything, implemented however you want and whose implementation is totally hidden, encapsulated). Any resource needs to have a unique identifier that is used to reference it. The resource may have one or more representations that are not the resource itself (representations are the ones transmitted to the client). We can have resource metadata and also representation metadata. And of course, there should be control data as well.
2. Connectors. Since REST applications are networked ones, we need elements for communicating between components. These are called connectors. Some are intelligent pieces of software or hardware, some others just plain interfaces to components. Examples are the client and server libraries used for communication, the cache (which represents an interface for accessing resources), resolvers and tunnels.
3. Other Processing Components. In this group we have the actual origin server (the processing node that handles the requests to the server), the user agent (the one that uses the client connector to talk to the server) and intermediary components such as proxies and gateways.

Summarizing REST Application Needs

From what we just described, it is clear that REST applications obey a set of special restrictions. REST is not suitable for all applications. It is specially designed for network applications with a strong need of hypermedia sharing and not much processing. Glenn Block, Senior Project Manager at Microsoft has proposed this list of basic requirements for RESTful Applications (which is directly derived from the constraints above):

1. Must be a client-server application.
2. Must be stateless i.e. it cannot rely on traditional server-side mechanisms like server state to track clients.
3. Must have tolerance for latency. Applications that communicate in real time do not play well with REST.
4. Clients and servers must exchange data via the uniform interface. They cannot be coupled to implementation details or require out of band knowledge. For example, applications that make direct remote procedure calls such as via SOAP Service proxies are not RESTful as they require a WSDL definition file in which is based on the server’s implementation.
5. The server must offer up links to the client which it will use to navigate the system
6. Client and server messages must contain in addition to business data, control data that is used to process the messages.

That list is great. Still, there are a couple more of REST application characteristics we should care about:

7. Look beyond the regular web app where the browser is dumb and everything is done server-side (yes, this is still a RESTful app, but it’s too simple). In those cases, the user plays the part of the client component’s intelligence. In general machine to machine interactions, the client component becomes very important as it plays a role in following a workflow, providing the correct data and understanding changes in data, format and control, dynamically. That poses some important challenges.
8. The application may spawn to several layers and may use third party services.
9. There is a major need in tools (think of libraries, frameworks) to handle the self-descriptive message, which contains not only business data but also metadata, control data and status data.
10. The server needs to provide the workflow or states the client needs to follow/navigate. As per the hypermedia requirement in the Uniform Interface, this has to be done via links (a link becomes a first class citizen).

Now, let’s go one step lower and talk about technology choices to implement such applications. Here is the list:

1. Protocol. To use REST, we would certainly use the web, and since HTTP is the predominant protocol there, we will base our discussion on it (http://www.w3.org/Protocols/rfc2616/rfc2616.html).
2. Resources. To implement resources we can use anything, as their implementation is totally hidden to the clients. A resource can be anything, from a running service to a plain text file. The identifiers assigned to resources in the web are URIs (http://www.ietf.org/rfc/rfc2396.txt).
3. Representations. These are documents that represent a resource. They will need metadata like the Media Type property, which tells me what type the representation it is (note the resource has no media type, the representation does). These representations are usually hypermedia documents, containing links and some status information (say Html, XML or JSON, or any other thing with or without links). We may also have control data (e.g. cache information bits).
4. Resolvers. The client and server components are simply servers and libraries to use HTTP (our defined main protocol). We can also include some cache systems. Resolvers are simply DNS services and the tunnels may be the regular SSL support.
5. We will need to use some proxies and gateways, or at least be aware of them in the design.

Now that we understand what REST is about, what components to use, what restrictions they have, and what can we use to implement them, we realize it would be nice to have a framework to help us with that implementation. What should be in a framework like that? What support should it provide?

The Maturity Model

The next section explores an informal Maturity Model, a plain list of features a framework may provide to support REST development, organized in levels. A framework offering features that match those in higher levels may be considered more mature. Each level of my model defines what a framework needs to provide in order to qualify in that particular level. That is, the expected features in the framework that helps support the development of some REST characteristics.

You may already be familiar with “Leonard Richardson’s REST Maturity Model” (nicely explained here by Martin Fowler). This model was thought to define how RESTFul an application was although applications are not even RESTful until you are at level 3!. We have a slightly different goal here, so this model is different and does not follow Richardson’s structure. Rather than evaluate how RESTFul the framework is, we target but the support level it has towards REST requirements’ achievement. Even at level 1, the developer using the framework may be able to create a RESTful app, although it will take more work and care since the framework does not offer that much help as it would in a higher level.

Each level has an accumulative effect: it will contain also the support offered at lower levels.

Disclaimer: This maturity model is a broad list of expected features and does not provide any detail in measuring terms or formulas. Some frameworks may be much better at supporting a requirement than others, but if they provide at least some kind of support, they will fit into the same level as well.

Level 0. Not a RESTful framework

If a framework is at level zero it provides no support at all for REST in the framework at the current time.

Level 1. Mapping/Routing and HTTP/URI Encapsulation.

A Level 1 framework provides two things:

• A simple way to map the operations and resources to the language constructs (e.g. methods, classes)
• An encapsulation of all the complexities of HTTP and URI management

With these two features, the framework can provide a way to accept HTTP requests and forward them to one of our classes and/or methods. There are several possibilities, like using a matching technique to map particular URI patterns to our methods, for instance. The URI may also be parsed to extract values for arguments passed to our methods, e.g. using a template:

    [method]/[name]/[section]->getMethod(name, section);
GET /book/misserables/chap01 ->getBook("Les Miserables","chap01");

Of course, the framework will handle the HTTP request and response and also all the headers and error codes. Frameworks that expose those elements for greater control of the application also fall into this level. (Please do not be confused: The above example is not enforcing nor accepting RPC as a valid REST interaction, it is just a mapping).

Level 2. Media Type Support and Client Support.

Level 2 frameworks include support for Media Types and something often forgotten: the client.

In terms of Media Type support, the framework should offer the capability to produce the representation of the resource using one known media type. That is, it can send back the response using, for example, XML as a default. Granted, anyone can write XML using a simple string manipulation API but for a framework, support in this context means providing a way to produce any media type from a local data source, like a file or a data object instance. Adding support for multiple media types (being able to return XML, HTML, JSON, etc) means we also need to add support for the content negotiation process. This is, in regular HTTP, the handling of the Accept header and error code 300 – Multiple Choices.

Of course, we are not totally in level 2 if there is no client side support. The framework should provide a way to help clients use a REST application in addition to server side functionality. At this level, supporting clients means:

1. A full client or a set of client libraries to perform HTTP requests.
2. A way to read or process the representation given its media type
3. A way to negotiate the media type

Client support has only recently been added into various REST frameworks and specifications. Version 1 of Jersey (JAX-RS spec) for instance, didn’t include client support at all. Still, that client support is limited as we cannot expect a framework to provide libraries for all different languages out there (clients can use different platforms than that of the server). And that is fine. We can even have server only (for building services) and client only (for consuming them) frameworks for REST.

For this maturity model, we can say those frameworks are partial frameworks that stay in level one. That is because they may be enough for a simple server with one client accessing it, or to just create a service and exposing it, but for larger, layered systems, where one server may play the role of a client consuming other layers’ services, client support is a must: at least we ask for support for the same language the server is using.

A nice discussion of the Client’s importance in REST can be found at Glenn Block’s blog.

Level 3. REST Elements Modeled into Language.

This level may have improved support for HTTP requests, media type handling and clients, all while following the “modeling into language” principle. That principle tries to avoid the Forced Paradigm Anti-pattern, i.e. changing the look and feel of REST to make it look like any regular operation in your language of choice! That breaks REST semantics.

What does the “modeling into language” mean? Well, the REST concepts should be modeled using the language semantics, so the developer is aware of the components, connectors and elements. The language must not obscure the REST concepts or interactions. Or, in Glenn Block’s words : “It [should be] opinionated about REST!”

For instance, taking an object-oriented language as the base (and this is just a suggestion):

1. There should be a resource class that handles what the resource element is capable of. We need to get/send representations of that resource. We need to control the metadata and the representation of the resource. We need to create and modify the resource. Also, the resource should have an identifier which, depending on the complexity of the model, could be a property or an object itself.
2. There should be classes representing gateways, proxies, resolvers and tunnels.
3. There should be methods to execute HTTP operations, methods to perform content negotiation, methods to process cache, etc. This is actually going further than what was available to us in Level 1, where the framework allows the developer to work with all the nuts and bolts of HTTP, but we still need to do some “heavy lifting”. At this level, all that is encapsulated and abstracted in a way that is part of the model.

In modeling, the actual REST world is brought to life using the language modeling capabilities, and not hidden behind normal language operation. For instance, if the framework hides the resource concept or the protocol operations and offers a plain “call this method” representation, it is harming the REST semantics. The worst thing that can be done is to wrap REST into RPC (something all REST fans will tell you is a very bad thing, as RPC has nothing to do with REST), simulating REST interactions with method calls and such.

Level 4. Hypermedia as State Engine plus Semantic Support.

While it is very important for frameworks to reach at least Level 3 in this maturity model, there are further requirements we could ask of them which very few implement. At Level 4 then, a framework should offer support for one of the most forgotten requirements in REST - the use of Hypermedia as the engine of application state (HATEOAS) – and a logical derivation of that support – the provision of a semantic division of information.

HATEOAS is one of the most important aspects of REST’s Uniform Interface (with one of the more unfortunate acronyms in computing). Since REST is oriented to the transfer of large hypermedia documents, it is natural to use hypermedia in the application context. In this particular case, the framework needs at least to offer link management in documents. That is, the insertion/management of links at the server side, and the processing of links on the client side.

Of course, that is a very primitive support. The links are followed by the client to progress in the application workflow, going from one state to the other. Each document may have one or more links, making it difficult for the client to decide which link to follow. Also, the server needs to add those links by either having a complete path/graph already planned ahead, or by making it dynamically as the client goes on following links.

To help the client decide which link to follow, metadata should be used. The metadata management should be also provided by the framework. The client part also needs tools to handle the metadata. One example of metadata could be the current “rel” attribute use to document the link intention or relations. Even forms and XHTML are being proposed to help clients. (Examples of this can be found in REST In Practice Book mentioned before and some work from Ian Robinson).

Since the decision should be taken based on technical and/or business information, the framework needs to provide a semantic division of information. For instance, the document may contain specific business data, some control data and some status data. The client needs some tools to separate those data types and process them. Also, it needs to gather information from all of them to decide which link to follow.

Let’s take an example. Imagine a bank client deciding about what to do with his money. The actual application state is showing his account balance. He can pay services, withdraw money or invest. There are several links to follow, some of them will present some forms to be filled out. The client needs a way to read business information (balance, banking concepts) to decide which option to pursue and to fill in the requested fields. He may also need status information to determine if the planned actions are allowed. To actually follow the links, he may need control information that will drive his workflow by telling it which operations to execute.

Why is this important? True HATEOAS support means the client is able to navigate between states using the self-descriptive message and no out-of-band information. Clients with hard-wired-link-following code are not HATEOAS compliant, as the engine is not hypermedia but the fixed state graph the developer got somewhere else.

This is extremely difficult to do. There is a lot of work trying to clarify concepts about hypermedia, control and states. Mike Amundsen’s blog has a good deal on the hypermedia topics and his book “Building Hypermedia APIs with HTML5 and Node” is well worth reading for more information.

Level 5. CoD and Layering Support.

Ultimately, we come to Level 5. There are still a couple of support types to address, the first of which is Code on Demand. Since in level 4 we need the client to work with the message and provide tools to manage all information types that are embedded in it, we may be asking for a super client due to the many media types out there! That is why some servers may provide code in form of scripts (JavaScript being the most popular language), flash widgets, DSLs and more in charge of processing the media types, so the client may not need to.

The second support type is for Layers. Layering support is hinted at in level 3 via the need to model gateways’ components. However real layer support requires a higher level of abstraction, as it is outside the boundaries of a single server or client; REST applications often require more than one server. Usually development frameworks will offer a server view and a client view and no higher abstraction. A framework should be capable of modeling inter-server interactions and encapsulating intermediary server processing.

Here we can identify two scenarios:

1. One client contacting one server which in turn contacts another server. Say, a client requesting its information from a server that needs a bit of information that is in another server.
2. One client contacting one server which requires some actions from intermediary servers (client is unaware of this). Say, a client sending some form that needs to be translated and validated before getting to the destination server.

In case (a), the first server can simply become a client and use the client libraries to consume the information in the next server. Level 2 offers enough support for this.

In case (b), we may need a way both to model and to inject those intermediary servers into the transaction as well as to route the message based on control and status information; something a Level 5 framework would provide. Arguably, this could be achieved manually with simple URL mapping provided in Level 1, but it gets complicated quickly and is not maintainable.

Lastly, it may be argued that it is at this level when we really need the client support, and maybe not at Level 1. However, client work is required at every other level, so it’s equally valid to say that some client support is needed before a framework gets to this level.

Summary

It’s worth pointing out that a Level 5 framework isn’t necessarily better for your purposes than a Level 1 framework; we’re simply proposing a way to identify its level of support completeness, and what features to look for in a REST framework. This model is not set in stone and some readers may simply decide a level 1 is enough for their needs, which is perfectly fine.

in part two of this series, we’ll check on current and popular REST frameworks and see how they match against this model.

Stay tuned!

Acknowledgements

Thanks to Tomás Chavarría and Glenn Block for proof-reading and improving the text.