Sound advice - blog

Tales from the homeworld

My current feeds

Fri, 2006-Sep-29

Using TCP Keepalive for Client Failover

I covered recently my foray into using mechanisms that are as standard as possible between client and server to facilitate a fixed-period failover time. A client may have a request outstanding and may be waiting for a response. A client may have subscriptions outstanding to the server. Even a server that transfers its IP or MAC address to its backup during failover does not completely isolate its clients from the failover process. Failover and server restart both cause a loss of the state of the server's TCP/IP stack. When that happens, clients must detect it in order to successfully move their processing to the new server instance.

I had originally pooh poohed TCP/IP keepalive as a limited option. Most (all?) operating systems that support keepalive use system-wide timeout settings, so values can't be tuned based on who you are talking to. I think this might be able to be overcome by solaris zones, however. Also, the failover characteristics of a particular host with respect to the services it talks to are often similar enough that this is not a problem.

I want to keep end-to-end pinging to a minimum, so I only want keepalive to be turned on while a client has requests outstanding. An idle connection should not generate traffic. Interestingly, this seems to be possible by using the socket option. It should be possible to turn the keepalive on when a request is sent, and turn it back off again when the last outstanding response is recieved. In the mean-time the active TCP/IP connection will often be sending data, so keepalives will most often be sent during network lull times while the server is taking time processing.

If I want my four second failover, it should just be a matter of setting the appropriate kernel variables to send requests every second or so and give up after a corresponding number of failures. Combined with IP-level server failover, and subscriptions that are persistent across the failover, this provides a consistent failover experience with a minimum of network load.


Fri, 2006-Sep-29

Common REST Questions

I just came across a blog entry that includes a number of common misconceptions and questions about about REST, here

I posted a response in comments, but I thought I might repeat it here also:

RESTwiki contains a some useful information on how REST models things differently to Object-Orientation. See:

and others. Also, see the rest wikipedia article which sums some aspects of REST up nicely:

The core of prevailing REST philosophy is the rest triangle, where naming of resource is separated from the set of operations that can be performed on resources, and again from the kinds of information representations at those resources. Verbs and content types must be standard if messages are to be self-descripitve, and the requirements of the REST style met. Also, there should be no crossover between the corners of the REST triangle. names should not be found in verbs or content types, except as hyperlinks. Content should not be found in names or verbs. Verbs should not be found in names or content.

REST can be seen a documented-oriented subset of Object-Orientation. It deliberately reduces the expressiveness of Objects down to the capabilities of resources to ensure compatability and interoperability between components of the architecture. Object-Orientation allows too great a scope of variation for internet-scale software systems such as the world-wide-web to develop, and doesn't evolve well as demands on the feature set change. REST is Object-Orientation that works between agencies, between opposing interests. For that you need to make compromises rather than doing things your own way.

Now, to address your example:
Verbs should not be part of the noun-space, so your urls

should not be things you POST to. They should demarcate the "void" state and the "reverse" state of your journal entry. When you GET the void URL it should return the text/plain "true" if the transaction is void and "false" if the transaction is not void. A put of the text/plain "true" will void the transaction, possibling impacting the state demarcated by other resources. Reverse is similar. The URL should be "reversal" rather than "reverse". It should return the url of the reversing transaction, or indicate 404 Not Found to show no reversal. A PUT to the reverse would return 201 Created and further GETs would show the reversal transaction.

Creation in REST is simple. Either the client knows the URL of the resource they want to create and PUT the resource's state to that URL, or the client requests a factory resource add the state it provides to itself. This is designed to either append the state provided or create a new resource to demarcate the new state. POST is more common. The PUT approach requires clients to know something about the namespace that they often shouldn't know outside of some kind of test environment.

On swapping: This is something of an edge case, and this sort of thing comes up less often than you think when you are designing RESTfully from the start. The canonical approach would be to include the position of the resource as part of its content. PUTting over the top of that position would move it. This is messy because it crosses between noun and content spaces. Introducing a SWAP operation is also a problem. HTTP operates on a single resource, so there is no unmunged way to issue a SWAP request. Any such SWAP request would have to assume both of the resources of the unordered list are held by the same server, or that the server of one of these resources was able to operate on the ordered list.

On transactions: The CRUD verb analogy is something of a bane for REST. I prefer cut-and-paste. Interestingly, cut-and-paste on the desktop is quite RESTful. A small number of verbs are able to transfer information in a small number of widely-understood formats from one application to another. The cursor identifies and demarcates the information that will be COPIED (GET) or CUT (GET + DELETE) and the position where the information or state will be PASTED to (PUT to paste over, POST to paste after). The CRUD analogy leaves us wondering how to do transactions, but with the cut-and-paste analogy the answer is obvious: Don't.

In REST, updates are almost universally atomic. You do everything you need to do atomically in a single request, rather than trying to spread it out over several requests and having to add transaction semantics. If you can't see how to do without transactions you are probably applying REST at a lower-level than it is typically applied. In this example, whenever you post a new journal entry you do so as a single operation. POST to a complete representation of the journal entry to a factory resource.

That is not to say that REST can't do transactions. Just POST to a transaction factory resource, perform several POSTS to the transaction that was created, then DELETE (roll-back) or POST a commit marker to the transaction.

How REST maps to objects is up to the implementation. You can evolve your objects independently of the namespace, which is expected to remain stable forever once clients start to use it. The URI space is not a map of your objects, it is a virtual view of the state of your application. Resources are not required or even expected to map directly onto objects. One method of a resource may operate on one object but another may operate on a different object. This is especially the case when state is being created or destroyed.

REST is about modelling the state of your application as resources, then operating on that virtualised state using state transfer methods rather than arbitrary methods with arbitrary parameter lists. REST advocates such as myself will claim this has significant benefits, but I'll refer you to the literature (especially the wikipedia page) rather than list them here.