About REST and REST APIs on the Web and in Mobile Telecoms

There has been a lot of buzz in the mobile telecoms industry about a new era, characterised by the emergence of RESTful application programming interfaces (APIs). What is REST in the first place, where are we in the process, has it made a difference, why are RESTful APIs supposedly better than the ones which are not full of rest?
In this article, we briefly address the following: What is REST, why are RESTful APIs popular, examples out there of actual REST APIs, and challenges.

When we talk about REST here, we talk about ways and styles of creating distributed computing systems. This could be for example enterprise software systems, or mobile telecoms services, or web-based services.
To recap: REST stands for Representational State Transfer. REST is neither a product nor a tool. You cannot buy REST. It’s a style of how a distributed software system can be architected and was born out of a PhD thesis. The design of a distributed system receives the stamp of approval of experts in REST, if the design meets a number of constraints: then the system design is called RESTful.

Where do we find REST? This style of designing a distributed system is now widely used on the Internet, for web applications and services. Many experts argue the World Wide Web is the most successful and largest-scale implementation of a distributed system following the REST architectural design principles. Well, then it must be important, right?

How simple is REST? Frankly, understanding distributed system design the REST-way is best left to the experts. Understanding a RESTful distributed software implementation is not an afternoon’s job. Designing e.g. a distributed enterprise system in the REST way and then implementing it e.g. in Java or the .NET framework appears to be non-trivial (speak possibly complex, and much of real life is indeed complex). So, in this article we strive at keeping things simple and we will simplify here and there a bit.

The REST style of architecting software systems and HTTP: It’s important to understand that the ideas of REST were developed at the time when the hypertext transfer protocol HTTP/1.1 [1] was created, which is today’s still usual way of shipping around data requests and responses on the Web. When you search on Google, your browser submits a request to a Google server via HTTP. It’s also important to understand where in an overall communications stack we find HTTP. To recap, moving up from the physical transmission layer (your laptop’s LAN or wireless LAN adapter): Physical layer, Data link layer, Network layer (where we find the IP routing), Transport layer (where we find the TCP protocol) to the Application layer (where we find e.g. HTTP as an application protocol). That’s it. Nothing more on top (in contrast to SOAP –based architectures [2]). Lean and simple.

Why is REST called REST? The REST architecture style argues that the world consists mostly of clients and servers (and possibly some more bits in between). Well, that’s not new and we have had client/server applications for decades. However, REST is a recipe of solving a distributed computing problem in a certain way. The way a REST architecture based on HTTP looks at the world is very specific. The (nearly) only thing it sees is different resources (e.g. on a server like html pages, audio and video recordings, database content). Those resources are addressed in a specific way, namely using URIs [3] (imagine a web hyperlink like http://www.computerfirm/laptoprepair). A software client (e.g. a web browser used by a human or another piece of enterprise software) sends a request to a server and asks for a representation (format) of a resource (e.g. laptop repair instructions). The server sends back a representation of the resource (e.g. a video clip, or a pdf document, or an html web page). The response reflects the status of the resource (e.g. the current status of the repair instructions). So in conclusion, looking through the REST glass we find that using HTTP, a client requests a representation of a resource from a server, whereby the client finds the resource through using a web hyperlink as the appropriate address.

Now to the ‘ST’ in REST: Software systems progress through states; an initial state, some intermediate states, a final state (which might be the same as the initial state). The way things progress might be called a Domain Application Protocol. When a head of state visits the Queen, there is a protocol in place: handshake, welcome address, friendly comments from the guest, joint dinner, etc. The Domain Application Protocol for a flight ticket booking system might be (from the system’s perspective): Offer the customer to search for flight numbers and times, Ask to select flights, Ask to enter personal details, Ask to confirm, Ask to provide payment details, Confirm purchase. In essence, such a Domain Application Protocol moves from one state to another driven by the mouse clicks of the customer. In a centralised system the state transitions can be neatly described by a finite state machine. In a distributed system we then need to co-ordinate multiple, distributed finite state machines to ensure an orderly progression of the Domain Application Protocol. In a REST-architected system, nobody has THE ultimate, single, unique, overall view of the global state the distributed system is in. Instead, clients learn about the state of resources when they request a representation of a resource from a server. The state of a remote resource is transferred to the client with every client request. That’s the ‘ST’ in REST. The ‘RE’ highlights that only representations are shipped (e.g. above video clip or a pdf document about the repair instructions), and not the resources themselves.

Making the Domain Application Protocol work and get moving: In REST, the client software learns about the state of a resource on a server. The representation of that state (e.g. an html page) includes further hyperlinks. When the client decides to invoke such a hyperlink, it moves on in its Domain Application Protocol from its current state to a new state. Because of the use of hyperlinks, REST is a perfect style to design a distribute hypermedia system. When the client is a web browser operated by a human user, the client makes decisions on which hyperlink to invoke next – helped by human reasoning. If the client is a pure software client, the client can make similar decisions based on machine-interpretable annotations attached to the usual web hyperlinks.

Is REST married to the web and HTTP? Actually not. A RESTful design doesn’t prescribe the use of HTTP for the sake of building web services. However, REST seems to have gained most adoption in the area of creating web services on the Web together with HTTP.

How to determine whether a system design is RESTful or not? Well, theoretically there are a few constraints to be met, e.g. client server architecture, a uniform interface between them, no session state held on the server, responses from servers must be cacheable in the network, clients don’t need to know whether they talk directly to a server or to an intermediary (e.g. a cache) etc.

The few building blocks of a RESTful web service design include a number of things which have been available on the Internet/web for a good while: addressing of resources (in this case the web service) with URIs (think web addresses), characterisation of resource representations with media types (e.g. MIME types, hypermedia formats like html, a service-specific XML schema, the JSON data format, the Atom syndication format), telling the server what to do using the vocabulary of HTTP (HTTP methods like GET, PUT,…) and progressing the Domain Application Protocol using hypertext (speak hyperlinks and hypermedia controls). Though REST is not a standard, many of the component protocols and languages one uses to implement a system in the REST style are standardised.

Beyond the basic building blocks, a RESTful service implementation may benefit from “outsourcing” some of its functionality to other service providers on the web, e.g. for identity management. A service user (a client in REST) has usually to provide a claim about its identity to the application server (“Are you truly who you claim to be?”). If a 3rd party identity provider, whom the application server trusts, has already authenticated this claim, the app server will happily trust the service user. OpenID is a protocol that performs this authentication service. A similar story applies for authorisation (“What feature of the service are you permitted to use?”) and the relevant protocol is called OAuth.

Benefits of REST: Important is that when a service is architected in a RESTful way particularly by using HTTP, many things fall neatly in place such that the RESTful architecture truly delivers all sorts of benefits.

  • Loose coupling between system software components reduces software maintenance and upgrade costs and enhances the possibilities to easily modify the implementation, on the client side, on the server side, or on both sides. This is e.g. facilitated through the ‘ST’ in REST and the fact that it is hypermedia (hyperlinks) driven. Fair to say, that loose coupling comes at some cost, namely the typical lack of guarantees (as also on the web – guarantees for existence of resources, data consistency in the face of caching etc.).
  • Less network bandwidth is required compared to other implementation approaches (e.g. those based on SOAP over HTTP (extra message header); use of XML (very verbose) in message bodies is not mandatory, though it’s not forbidden.
  • Less reinventing the wheel, more re-use of existing/deployed mechanisms. An example is error handling. An HTTP-based RESTful system makes use of the comprehensive set of HTTP error codes, whereas a SOAP-based system would define its own new error codes which then exist in addition to the HTTP error codes.
  • Performance and scalability of the service is greatly enabled by RESTful HTTP-based web service implementations leveraging the truly large-scale caching infrastructure of the World Wide Web around the globe. This is like a gift to the service provider (as the web and Internet are there anyway). Caching has big benefits for many services, as it improves end user experience by reducing latency and reduces load on parts of the network and application servers: good news for users, (mobile) telecoms operators and the application service providers.
  • Reliability of a service is increased. Because of loose coupling, discovery of state of resources, and possible use of network caches, the overall distributed service implementation gains in reliability.
  • Potentially (or likely) lower initial development and deployment costs and further down lower maintenance and upgrade costs. A cynical (or not at all?) view is that some of the non-RESTful service implementations (e.g. using SOAP and WS-* protocols) are so complex that they require very expensive expert skills and IT companies stand ready to provide those and charge heavily for them e.g. for SOAP-based enterprise integration. In contrast, RESTful service deployments benefit from a large set of web and Internet developers/programmers who don’t need the same level of sophisticated, bespoke enterprise integration skills and deep know-how of e.g. the WS-* stack. Thus, promoters of REST tend to associate a RESTful implementation with light, fast, flexible, and lower cost.

There are also lots of myths flying around which need to be carefully looked at, specifically when it comes to some people promoting REST over SOAP/WS-* or the other way round. For example that a SOAP based design is just bad because of the extra SOAP message header, that SOAP uses message bodies expressed in XML and a RESTful design wouldn’t (though there is no constraint in REST banning the use of XML) etc. Much could be written to surface those myths and provide a more objective positioning between both approaches. Suffice to say: Both have their strengths and weaknesses, with the REST architecture being popular with the web community and SOAP (and its affiliate protocols) being popular with the enterprise IT companies for highly secure and reliable enterprise services and messaging architectures.

Where is REST going? REST has risen over other ways of designing distributed web applications and services at least outside the enterprise software engineering arena. Is has eclipsed the use of SOAP and SOAP Remote Procedure Calls over HTTP for many areas of services for good reasons, though the proponents of REST and those of SOAP (and WSDL, WS-*) will possibly keep fighting for what is better for the years to come. RESTful services seem to have emerged first on the human web, where humans, working with web browsers, interact with web servers. REST can equally be applied to design e.g. distributed enterprise systems, where machines (computer programs) interact with other machines, though it might take some more time for its very serious, large scale adoption for enterprise systems.

Some challenges with RESTful service designs using HTTP are related to security. Overall security can be realised through security steps on multiple levels, e.g. on HTTP level for confidentiality and data integrity and using OpenID for authentication. Use of secure channels by using HTTPS impedes the use of http caching and thus destroys the key benefit of inherent scalability of http-based RESTful services. OpenID is said to suffer from some security vulnerabilities. When it comes to authorisation, OAuth has been the celebrated new kid on the block, though it has created some substantial controversy amongst experts [4]. Other challenges relate to situations where reliable messaging needs to play a key part or where continuous data consistency is paramount (which is not a given in an http-base caching architecture). Finally issues with software developers arise when companies upgrade their web services API and “break it” (then many existing applications don’t work anymore). Breaking the APIs seems to be fashionable.

REST APIs: Despite some issues, RESTful web services and the doors to access them, namely the application programming interfaces (APIs) have flourished dramatically over recent years. An API is a way to programmatically invoke a remote service. Businesses can invoke a remote 3rd party business service (a B2B scenario), consumers can use web applications which are a mix (mashup) of remotely hosted, independent services (a C2B). For instance, service A (the one we own and have built) calls service B (operated by another company) to retrieve certain resources and information. This is done via API calls invoked in service A. A protocol directory [5] lists over 4000 APIs in the category of REST style as of 27 Jul 2012. In that directory, we find e.g. AT&T location based services, Bluevia Mobile Payment API, Amazon S3 online storage service, GSMA OneAPI Zonal Presence (a location service API), Netflix movie rental service, and thousands of others – a pretty fancy mix.

Examples:
The Google+ API. Google offer a REST API to e.g. retrieve a user’s profile from Google+. An example API call may looks like this:
GET https://www.googleapis.com/plus/v1/people/{userId} HTTP/1.1

The API call implies sending the HTTP verb GET to the address indicated by the URI following GET, in this case using secure http (thus not cacheable, therefore still RESTful?). The resource (the user’s profile) is sent back by Google’s server in a certain representation, namely the JSON data format. For more details, see https://developers.google.com/+/api/.

The next example is Facebook’s Graph API. It enables one service to access members and relationships in the Facebook social graph. An example API call might look like this:
GET https://graph.facebook.com/Bob HTTP/1.1

This request to a Facebook server would return a JSON object with public information about Bob and his profile. More details at http://developers.facebook.com/docs/guides/web/.

A further example is the Twitter API which allows retrieving lots of information. An interesting one might be the GET search API, which allows to discover tweets which match a particular query (e.g. show me all tweets on Twitter mentioning “nice article”). The API call may look like this:
GET http://search.twitter.com/search.json?q=nice%20article HTTP/1.1

The call to this API would hopefully return tweets about “nice article”. For more details, see https://dev.twitter.com/docs/api/ .

A further example from mobile telecoms is Bluevia: The send SMS API enables an application to send a text message. The API call may look like this:
POST https://api.bluevia.com/services/REST/SMS/outbound/requests?version=v1 HTTP/1.1

The text message is then appended as the body of the HTTP message. In this case the HTTP verb used is POST, and the text message content can be appended e.g. in XML and JSON format. For more information see https://bluevia.com/en/page/tech.APIs.SMSAPI.

Another example from mobile telecoms is the REST API defined by the GSMA, denoted OneAPI [6]. Actually this is a whole series of different and interesting APIs. Let’s have a look at the Connection Profile API which allows a service or application to discover the connection type of a mobile phone (e.g. 3G, LTE, etc.) and its roaming status. Assume the telephone number is +1-555-555-0100. Then the API call might look like this:
GET http://example.com/terminalstatus/queries/connectionType?address=tel%3A%2B1-555-555-0100 HTTP/1.1

The response from the server comes in this case in the JSON format and could e.g. indicate that the smartphone is currently on an LTE network or on WLAN. Very smart. For more information see http://oneapi.gsma.com/data-connection-profile/.

Finally, let’s look at a REST API from Twilio. The API allows e.g. to make outbound calls from your piece of software to any other party on the globe. An example API call would look a bit like this:

POST https://api.twilio.com/2010-04-01/Accounts/AC30947…/Calls HTTP/1.1


From=+14158675309&To=+14155551212&Url=http://www.myapp.com/mycallhandler.php

This initiates a call from the “From” number to the “To” number and triggers the invocation of a useful script at the indicated Url address when the call actually connects.

So then, producing a new web application which makes use of RESTful web service APIs is pretty straightforward: according to above examples given in raw HTTP format not more than a few lines. In real life though, services are implemented using various programming languages like Python, Java, PHP, Ruby, C#. Life is made easy for developers through particular API libraries which assist in constructing the correct HTTP syntax in those language environments.

Who are the winners? Possibly consumers, some of the startup companies who offer new services and make them accessible via APIs and then sell themselves to the bigger companies, the grown-up startups (like Facebook), businesses who successfully offer REST APIs for serious B2B purposes specifically when their business partners are smaller companies (including “developers”) who are keen to avoid expensive and complex B2B integration, and the companies who deliver the shovels for this ongoing gold rush (like Apigee and others).

The business models vary. The companies offering REST API access to their services may charge their clients (e.g. 3rd party developers) for use of these services on a per usage basis, they may agree on a revenue share deal with developers or offer those APIs for free when the exposure of the API serves a different strategic purpose. There are indeed different ways to make money.

Years after the early appearance of REST APIs, it has dawned upon companies that the exposure of raw APIs (as shown in above HTTP examples) is not good enough and not sufficient to drive an API’s adoption by software and application developers. Equally or more important is winning developers’ imagination, and offering them a platform i) to get going very quickly and easily with using a new API, ii) to reach many consumers, iii) to make money and iv) to associate themselves with a great brand. Money is made where the big waves happen, in places where consumers congregate, where network effects kick in, where consumers say “awesome”, and devoted developers become followers, at least for a good while. That’s today the case with the ecosystems of Apple, Google, and Facebook. Many other players are working hard at cracking the nut and are testing new recipes. In the mobile telecoms arena, the effort of the GSMA around OneAPI [6] (a cross-operator initiative) deserves some praise. Still, the GSMA’s remit is limited and actual deployment of such REST APIs is up to individual service providers and network operators. More movement is to be expected in this space.

Notes:
1) HTTP/1.1: Hypertext transfer protocol version 1.1.
2) SOAP: Simple Object Access Protocol
3) URI: Uniform Resource Identifier
4) http://hueniverse.com/2012/07/oauth-2-0-and-the-road-to-hell/
5) http://www.programmableweb.com/apis/directory/1?protocol=REST
6) http://oneapi.gsma.com/api-list/