Ross Tuck
November 7, 2013
In the opening to this series, we discussed what ETags are and demonstrated their most common use case, caching. This time around, we’ll look at a lesser known but perhaps even better feature of ETags: keeping changes safe when writing to the server.
To be, or not to be
Last time, we focused almost exclusively on the If-None-Match header. Now we’re going to examine its more positive cousin, If-Match. These headers are two sides of the same coin: does the client’s ETag “match” the one on the server? The only difference is whether the “if statement” is negated or not. You can imagine these in code as:
// If-Match if ($myETag === $serverETag) { processMyRequest(); }
vs
// If-None-Match if ($myETag !== $serverETag) { processMyRequest(); }
Using these Match headers, we have some control over the server’s workflow. If our condition returns true, then the server should handle our request, be it a GET, POST or any other verb. Otherwise, it rejects it, usually with a special status code.
The If-None-Match header is typically used for caching. By sending it with our version of the ETag, we can prevent the server from regenerating a GET request if it’s a version we already have. This is a win-win for server and client: the server doesn’t waste resources regenerating identical content and the client doesn’t wait to download identical content.
The goal of If-None-Match is to support this type of caching. But what about If-Match? Well, first we need to understand the problem it’s trying to solve.
Conflict Resolution
If you’ve ever used version control software like Git or Subversion, then you’re almost certainly familiar with the concept of a conflict. When two developers edit the same code at the same time and try to commit or merge it, the version control software throws an error and aborts the operation. Why? Because the second set of changes could overwrite or undo the work of the first, making the first developer so angry he might throw a coffee cup or keyboard at the second developer. Thus, most version control systems prevent this from happening.
Most web applications or APIs have this same situation but with a different outcome. Two users open the same record and start editing it at the same time. The first user, Alice, finishes her work more quickly and saves. The second user, Bob, finishes a moment later and clicks save. If the two were using the same form or page, there is a very high chance that Bob has just overwritten Alice’s work completely.
In practice, I’ve seen only a handful of systems that take this problem seriously and try to prevent users from overwriting each other’s work. At a minimum, they rely on users manually checking an “Assigned User” field or directly communicating with each other.
In a more automated system, this is usually done with a version number field that increments every time someone saves a record. The record number is passed along with each edit and if does not match, the change is rejected or a merge tool is displayed.
Each system tends to implement a little differently but it can be tricky when dealing with APIs. Where do you place the version number: custom header or in the body? Both have their disadvantages. How do you ensure that proxies will respect your versioning system correctly? Rolling your own here can be surprisingly difficult.
Luckily, you don’t have to puzzle around too much. HTTP already has a conflict detection system built-in and to use it, we’ll need the If-Match header.
Matchmaker, Matchmaker, Make me a match…
If you recall from the last article, we can fetch the server’s ETag by just doing a GET request to the server:
Request:
GET /record/42 HTTP/1.1 Host: example.org
Response:
HTTP/1.1 200 OK ETag: “72972b69942c42b59bc76bec59436f7c” { “text”: “lorem ipsum” }
Remember, an ETag represents the current “state” of a resource. If the resource changes (i.e. someone updated it), then the server should have a different ETag. That is to say, it can be treated like a version number here. In fact, some systems privately generate their ETags based on a version number in the database.
Now that we have a representation for /record/42 (the json) and an ETag (72972b6…), we can load it into our user interface and allow the user to modify it. For example, we might change the text field from the classic “lorem ipsum” to “foo bar baz” and then send it back to the server.
PUT /record/42 HTTP/1.1 Host: example.org { "text": "foo bar baz” }
However, as discussed, there’s no guarantee someone else hasn’t modified the record since our initial load. If you think about it, we only want to save this record if it’s the same version as we loaded. In other words, if the ETag we have now is the same one that the server still uses.
And that’s how we use the If-Match header:
PUT /record/42 HTTP/1.1 Host: example.org If-Match: "72972b69942c42b59bc76bec59436f7c" { "text": "foo bar baz” }
If it feels more familiar, you could read this logic in PHP pseudo-code as:
$myETag = "72972b69942c42b59bc76bec59436f7c"; $serverETag = generateLatestETagForRecord(42); if ($myETag === $serverETag) { updateRecord(); }
If the two ETags are the same, the PUT operation will go through and the record will be updated. The server’s response will likely be a 2xx success status code and may include the new ETag so that you can immediately send another update if you choose. Updates from users that still have the old ETag will be rejected, protecting the most recent update.
This is the primary use case for If-Match. Whenever we send a method that changes state on the server (POST, PUT, DELETE, etc), we can send an If-Match header to say “Hey, only do this if I’ve got the latest version. I don’t want to ruin someone else’s work.” Contrast this with If-None-Match which is used with methods that don’t change state on the server (GET) and says “Hey, only do this if I don’t have the latest version. I don’t want to make more work for the server.”
Failure Scenarios
When a client does not have the latest ETag but attempts to change state on the server, the server should reject it. It does this by not processing the request (i.e. not performing the change) and by returning a specific HTTP status code: 412 Precondition Failed. This might seem like a vague message but “Precondition” is the generic term for this type of if statement (there are a few others in HTTP), thus the message Precondition Failed is telling the client that the request can not be processed because an “If-*” header has failed (i.e. returned false). Thus you can represent the overall flow as this psuedo code:
if ($myETag === $serverETag) { updateRecord(); } else { throw new PreconditionFailedException(); }
Previously, some would argue for using the 409 Conflict status code here. However, the spec (both the original and the later HTTPbis rewrite) are very clear in preferring 412 here and for good reason: 409 is a broader code, where as 412 is very explicit about the cause.
The keen-eyed reader might note that although we use 412 here, we used a 304 Not Modified for the If-None-Match caching scenario in the previous article. If you’re wondering why this is different, an If-None-Match / 304 combination indicates a cache hit, which is a perfectly valid and working state for the application to be in. However, with the If-Match header, a failure means the request can not go through and there is no cache to fallback on, thus we use an explicit error code, 412 Precondition Failed.
Also note that the error code falls in the 4xx (or “Client Error”) range; it is the client’s responsibility to resolve this by fetching the latest ETag and redoing the update. Having a specific error code makes this much easier to handle, you can detect it and directly tell the user their changes were rejected because the record has since changed. In a more complex application, this could even be the trigger to display a merge tool, showing the differences between the old and new records.
Opt-in, Tune out
There is one important “gotcha” with using If-Match to protect your latest updates: it’s an opt-in feature. If you have 99 clients which use If-Match properly and one client which just sends PUT and DELETE requests willy-nilly, it will simply overwrite the work of the more careful clients.
In practice, if you have control over all your API clients, this probably isn’t an issue because you can enforce this a team requirement. Even if you can’t, it’s only an issue for clients which don’t use If-Match and are sending an overwriting update on a specific record at the same time as someone else: hopefully a very small subset.
If this isn’t an acceptable solution, however, then you may want to start rejecting any state changing request that does not come with a If-Match. Clients would then be required to first fetch the ETag from the resource and then send it when making a change so we can ensure the safety of all changes.
In this case, the server can reject any updates without an If-Match header by returning the 428 Precondition Required status code. If you remember that Precondition is a general term here, this error is very clear: the request must have a precondition before it will be handled. This response is fairly new, it was only released in April 2012 with RFC 6585 (as compared to the original HTTP spec: RFC 2616 from June 1999)! It’s still rarely seen in the wild. Nonetheless, it’s a great addition to your toolbelt.
Alternatives
ETags aren’t the only way to use these features. HTTP also has Last-Modified which can be used in the same manner. Here, the server returns a Last-Modified header which has a datetime as value. The client can then use this datetime just like ETag but with different headers. For caching, you would use If-Modified-Since and to prevent overwriting you would use If-Unmodified-Since. You can see the same symmetry of positive/negative if statements, just as with If-Match and If-None-Match.
At a glance, Last-Modified might seem slightly easier to implement. Many applications already store the last edit date for a record and you don’t need to worry about generating a different timestamp per representation (i.e. if your API returns XML and JSON for the same URL, you must use two different ETags for each version. More on that in a future article).
Keep in mind though, the timestamp format only allows 1 second of precision. If you have machine-to-machine API communication, 1 second can be a very long time indeed. Also, while Last-Modified implies a certain strategy for change tracking, ETags are opaque to the client: you can always begin by using the last edit date as your ETag then change to a more complex versioning strategy later on.
Ideally though, the spec recommends that you support both Last-Modified and ETags. For websites, this is the recommended practice. For APIs, I’ve seen some that support both Last-Modified and ETags but also those that only support one or neither. Whether it’s worth the effort is up to you.
Until Next Time
In this article, we’ve focused on using ETags to ensure the safety of your changes when writing to an API, which is perhaps the killer feature of ETags. In upcoming articles, we’ll take a look at the weak vs strong variants of ETags, using multiple validators at once and tips for implementing ETags on the server side. In the meantime, I hope that you’ve found this information useful and feel free to ask any questions in the comments section.