Topic 7b: REST

Representational State Transfer: REST

Representational State Transfer, REST, is not so much a technology as an architecture for building web applications. It is a more formal and structured extension to the basic URL-based web service idea. The idea centres around using clear, highly-descriptive URLs to represent each real-world entity that our web application needs to deal with (e.g. a song, a list of all songs by a given artist, a flight, a biography of an actor, etc). For example we could have these URLs:

http://www.hittastic.com/artist/Oasis
http://www.hittastic.com/song/Snow_Patrol/Run
http://www.hittastic.com/biography/Madonna
http://www.solentairways.com/flight/SA101
http://www.solentairways.com/flights/June/1/Southampton/New_York

In REST, these URLs are called resources. REST has the following key principles:

Use of clear, unchanging and descriptive URLs to represent real-world items or entities(as in the example above);
The URL should produce the data expected of it, e.g http://www.solentairways.com/flight/SA101 should return data representing Solent Airways flight SA101
The ability to send different instructions to the URL to make it behave in different ways, i.e. we can send a URL one type of instruction to retrieve data and another type to modify data associated with that URL.
Statelessness

Clean and unchanging URLs

A key principle of REST, illustrated by the examples above, is that of clean, unchanging URLs. Why is this useful? URLs which show the real location on the server, or the server-side technology used (e.g. the fact that it's a PHP script) are prone to continuous change, for example, if the script is moved to a different folder or we switch server-side scripting technology. This causes problems in bookmarking and linking to such pages, and also, if the URLs represent web services, means that developers of client applications have to update their client code to point to the new URL.

With REST, we hide the implementation details with a simple, clean and easily-remembered URL, and define how this URL is mapped to the real, underlying location of the script on the server. For example, rather than

http://www.hittastic.com/track.php?title=Wonderwall&artist=Oasis

we could use:

http://www.hittastic.com/Oasis/Wonderwall

If we changed the underlying URL, i.e. the location of the actual server side script on the server, all we'd need to do is change the mapping of our clean, easily remembered, publicly-visible, "REST-style" URL to the real underlying URL, and clients of the web service could continue to use our web service unchanged with the same publicly-visible URL as before; they wouldn't have to alter their code to reflect the new underlying URL. We could even change the server-side implementation technology (e.g. PHP to ASP) without having to change the publicly-visible URL: once again we would only have to change the mapping from the publicly-visible URL to the underlying URL.

Furthermore, this allows us to easily swap between dynamically and statically generated data. Imagine the URL below points at a static (i.e. not dynamically generated) XML file on the server representing all Oasis hits.

http://www.hittastic.com/artist/Oasis

By changing the server configuration, we could easily change this URL to point to a server side script which dynamically generates the data from a database. So in summary, REST style URLs provide a clean and unchanging interface to data supplied by our server and there is no need to change the URL depending on how the data associated with that URL is generated.

The practical details of how to actually set up REST-style URLs to point to given scripts will be discussed towards the end of this week's notes.

REST and HTTP

With REST, we send different types of messages to the same URL to make it do different things, e.g. retrieve data or change the state of the item represented by the URL. For example if we had the URL:

http://www.solentairways.com/flight/SA101

we could send one type of message (let's call it a "get" message) to to the URL to retrieve the details about flight SA101, and another type of message (let's call it a "put" message) to update the details (e.g. departure time) of flight SA101, and a third type (let's call it a "delete" message) to delete flight SA101.

But what form do these messages take? We could use query string parameters to inform the script of the message type. However, it so happens that HTTP, the protocol used for communication between web clients and servers, already allows us to send different request types to a URL. You've already seen GET and POST requests - but it just so happens that there are PUT and DELETE requests in HTTP too!

HTTP - revision

Recall from last year that HTTP is a set of instructions which allow clients and servers to communicate with each other

e.g. when you enter a URL in a web page, you construct an HTTP request for that page, and the server sends back a response

Recall also that HTTP requests and responses consist of two sections:

the header, which describes the content
the content itself, such as form POST data in the request, or the requested web page in the response

HTTP methods

As mentioned above, HTTP comes with a set of standard request types, or methods, to retrieve and manipulate URLs, which are specified in the HTTP request header. The two you've probably met are:

GET - retrieve data from a URL. Query strings are GET requests.
POST - send data to a URL (typically form data)

However there are other methods which are part of the specification but which are not normally used for standard web transactions. These are PUT and DELETE.

REST and HTTP: detail

REST takes the view that HTTP methods and status codes are under-used and can be exploited in web services. As mentioned above, the idea is that one single web resource (URL) can be used for retrieving, adding, and deleting data associated with a particular item, e.g. a particular song in the HitTastic! database. What we can do is to do different things with the song depending on the type of HTTP method we use to communicate with the URL. In general we:

Use the GET method to retrieve data associated with a URL;
Use the PUT method to modify data associated with a URL;
Use the DELETE method to delete data associated with a URL;
Use the POST method to create a brand new server resource e.g. a new song.

REST and HTTP status codes

REST's association with HTTP does not stop there. REST makes use of different HTTP status codes to communicate the success or otherwise of each operation. Recall that the first line of any HTTP response is a status code which indicates whether the request was successful or not. There are a large number of HTTP status codes including:

200 OK - the page was retrieved successfully
400 Bad Request - the request is in an invalid format
401 Unauthorized - we try to access a page which we don't have rights to view
404 Not Found - the page could not be found
500 Internal Server Error - there was some sort of internal error on the server

REST makes use of these to communicate the success or otherwise of the operation. For example, a 200 OK might be returned if the operation was successful, or a 404 Not Found might be returned if we asked for a resource that did not exist.

Difference in usage of HTTP codes in REST versus normal usage

Imagine we had a URL to look up a given song:

http://hittastic/song/1009

With REST, the URL could return "404 Not Found" to the client if the song with that ID was not on the HitTastic! database, or, if an invalid ID (0 or less) was supplied, the URL could return "400 Bad Request", another standard HTTP error code. Note that this use of error codes differs from the normal usage:

Normally, "404 Not Found" is used to indicate that a given file on the server does not exist
With REST, a script would typically still run (here, that script would look up the song with the ID 1009) and a "404 Not Found" would be generated by the script to indicate to the client that that song does not exist.
So, the error codes are being re-used by REST to indicate an error generated by the script rather than the web server

This illustrates a key principle of REST: reuse the existing standards of the web, rather than invent new ones

A number of examples are shown below.

REST example 1

Imagine we have the URL:

http://hittastic.com/song/1009

to represent the song with the ID of 1009 in the HitTastic! database. (Note - see "Clean and unchanging URLs", above, for more details on why we use a URL like this, rather than something like http://hittastic.com/song.php?id=1009)

We could send:

a GET request to the URL if we wish to retrieve the data about song 1009
a PUT request to the URL if we wish to add or change data relating to song 1009. With a PUT request, we typically send XML data to the server. In this instance it would be XML containing the updated details of the song.
a DELETE request to the URL if we wish to remove song 1009

The REST web service would also make use of HTTP status codes to indicate whether our transaction was successful. For instance:

The REST web service would send back 200 OK if all went well;
The web service might send back 404 Not Found if the song with the ID of 1009 did not exist;
The web service might send back 400 Bad Request if we sent invalid data to the URL, e.g. an ID of 0 (assuming the lowest ID is 1) or, if we were updating or adding data, a year that hasn't happened yet, like 2013
The web service might send back 401 Unauthorized if we try to update data but do not have the rights to do so (e.g. we are not the site administrator, or we supply an invalid password)

REST example 2

Another example: our URL could represent a particular artist in the HitTastic database e.g.

http://hittastic/artist/Oasis

We could send:

a GET request to the URL if we wish to retrieve all Oasis hits
a PUT request to the URL if we wish to add a new Oasis hit. We would send XML to the server containing details of the new hit
a DELETE request to the URL if we wish to remove all Oasis hits

The URL then sends back an HTTP status code to indicate success, or indicate the type of error that occurred,e.g. a 404 Not Found if the artist does not exist.

REST example 3

Our URL could represent a particular song and artist e.g.

http://hittastic/track/Oasis/Wonderwall

We could send:

a GET request to the URL if we wish to retrieve all the details about Wonderwall by Oasis
a PUT request to the URL if we wish to modify its details e.g change the quantity in stock. Again we would typically send these modified detaisls as XML
a DELETE request to the URL if we wish to remove Wonderwall by Oasis from the database

The URL then sends back an HTTP status code to indicate success, or indicate the type of error that occurred, e.g. a 404 Not Found if Wonderwall by Oasis was not in the database.

REST is stateless

Another feature of REST is that we don't transfer state between client and server. In other words, REST web applications do not use session variables server side, and do not transfer session IDs via cookies between server and client and back again. The idea is that the URL, or the content, contains all the information required to maintain state. This allows the development of more loosely-coupled web services and applications which can be connected to any client side application: they do not need to rely on a session variable having been set in some earlier server-side script.

So how is information passed from page to page? A common way is within the URLs themselves. For instance, if a user selects London as their origin and Denver as their destination in a flight booking system, the origin and destination can be written into the URLs of each successive stage of the booking system.

What about more secure information such as usernames and passwords though? It is obviously not a good idea to put those in the URL. Luckily, HTTP comes with its own authentication system known as HTTP authentication. With HTTP authentication, we actually embed the username and password into the HTTP header. This is relatively easily done with PHP. A server-side script can then interpret the HTTP header, extract the username and password from it and use them to authenticate. While this does have the advantage of being able to develop more loosely- coupled web applications as described above (we are not relying on the existence of some "gatekeeper" session variable indicating whether a user is logged in or not) it does have the disadvantage that we have to authenticate with the database every time a script which requires a user to be logged in is accessed, with possible performance issues. This raises an important point, REST does have a number of disadvantages. These are elaborated on below.

How do we set up a clean REST-style URL?

Apache comes with a module called mod_rewrite. mod_rewrite allows you to map one URL to another, e.g. a REST-style URL to the real URL. For example, you can tell mod_rewrite to map the REST-style URL (where T is the title and A is the artist)

http://www.hittastic.com/T/A

to the real URL:

http://www.hittastic.com/search.php?title=T&artist=A

More here, showing examples of how to use mod_rewrite.

Using mod_rewrite

You put mod_rewrite commands in a file called .htaccess and upload it to your public_html directory. mod_rewrite syntax is a big topic, but here is an example based on the mod_rewrite tutorial.

RewriteEngine on
RewriteRule ^home.html$ index.html

This example will rewrite URLs containing home.html so that they load index.html. For example if you type in

http://hittastic.com/home.html

the page

http://hittastic.com/index.html

will be loaded instead. Note that the ^ and $ match the start and end of the input (requested) URL respectively. In other words, it would only match an exact request for home.html, not myhome.html for example.

Thus we can see that the general syntax is :

RewriteRule RequestedURL RewrittenURL

Here is a more complex example:

RewriteEngine on
RewriteRule ^page/([0-9]+)/?$ index.php?page=$1

This example shows the use of a simple regular expression. Regular expressions are specifications that match certain text patterns in the input text, and are a whole topic in themselves. The regular expression here, ^page/([0-9]+)/?$, means the following:

^ matches the beginning of the input text (here, the URL to be rewritten);
([0-9]+) means match any text in the input that consists of one or more (+) of the characters 0-9 ([0-9]). The brackets surroundinding the regular expression indicate that the matching text (i.e. one or more of the characters 0-9) will be saved for later use.
/? means that the sequence of numeric characters 0-9 must be followed by either a slash (/), or nothing.
The $ matches the end of the input text.

Thus, input such as

page/123/
page/456
page/139823/

would match the expression.

After the input URL, the rewritten URL is specified. Here, it is:

index.php?page=$1

The $1 means "substitute this with the results of the first bracketed expression in the input". The bracketed expression here, if you remember, was the sequence of numerical characters. So in other words, a request such as:

page/456

will be rewritten as:

index.php?page=456

A final example shows the use of two bracketed expressions.

RewriteEngine on
RewriteRule ^page/([0-9]+)/([0-9]+)/?$ index.php?page=$1&section=$2

Hopefully you can see that this would match two sequences of digits in the input, separated by a slash. So a URL such as:

page/456/2

would be matched, which would be translated to:

index.php?page=456&section=2

Summary - advantages and disadvantages of REST over POX

Advantages

Use of existing web standards (HTTP) rather than "re-inventing the wheel"
The "everything is a resource" philosophy enables bookmarking, caching, linking
Clean, easily-remembered URLs with well-defined HTTP methods (GET, PUT, DELETE) to operate on them and well-defined HTTP error codes to indicate success (or otherwise)
Security: Firewalls can be set up to block certain HTTP methods, so if you wanted the public to be able to GET an item but not PUT or DELETE it, you could easily set your firewall up to do this

Disadvantages

It can be a pain to write the client code to a REST web service, as PUT and DELETE are little used outside of REST and consequently poorly-supported by many server-side technologies (e.g. PHP)
For this reason a "hybrid" approach is sometimes used, where HTTP error codes are used (easy to implement), but all requests are GET or POST requests (GET to retrieve data, POST to add/modify/delete)
However this does not allow the firewalling advantage of REST described above to be as flexible (what if we wanted to allow modification but not deletion?)
With some applications, the request types do not clearly fall in to the categories GET, PUT, DELETE and POST, which means that basic POX approaches, in which an "action" query string parameter is used to specify the request type, can be more appropriate in certain cases.

Some resources

Building Web Services the REST way
XML-RPC specification
Wikipedia REST page
Roy Fielding's PhD thesis which originally proposed REST