Developer Notes

API Security : use HMAC to sign requests

Posted 21 Apr 2023

I still come across HTTP APIs that are protected with only an API key in a header. An API key is a way to identify the consumer of an API. It is not sufficient protection for any API.

Many organizations have moved on to more robust delegated authentication like OAuth2. But that's not always necessary or even possible. For example if you're integrating with a platform that's simply not capable of caching security tokens obtained from an identity provider. Acquiring a delegated authentication token on each request can be costly and should be avoided.

There's an interesting alternative though: signing requests with HMAC. Hash-based Message Authentication Code (HMAC) can be used to both validate the contents and authenticity of a request using a pre-shared secret.

Since HMAC actually validates the content of a message it's a lot more valuable than just authentication. It helps prevent tampering with messages. If you include a timestamp, it can also help mitigate replay attacks. If you're dealing with sensitive data like payments or private information, it makes sense to sign messages ánd use an authentication mechanism.

Security considerations

Delegated authentication mechanisms like OAUTH use tokens that expire relatively quickly. Any compromised token will quickly lose it's validity. This may be preferable to using a static pre-shared secret as described in this article.

Signing the request

An effective HMAC implementation should use a signature over multiple fields of the request, including the URL. Using the full URL ensures that the endpoint being called matches your expectations.

The signature is calculated over:

• HTTP Method: GET, POST, etc.
• Full URL as invoked from the API client
• A timestamp, for best compatibility use a Unix timestamp based on UTC
• The request body (assuming it's text based, like JSON)

The following C# code calculates the hash:

using System.Security.Cryptography;
using System.Text;

var timestamp = DateTimeOffset.UtcNow.ToUnixTimeSeconds().ToString();
var stringToHash = $"{method}{scheme}://{host}{path}{timestamp}{body}";

var textBytes = Encoding.UTF8.GetBytes(input);
var keyBytes = Encoding.UTF8.GetBytes(key);

using var hash = new HMACSHA256(keyBytes);
var hashBytes = hash.ComputeHash(textBytes);

return Convert.ToBase64String(hashBytes);

In NodeJS the code is similar:

const crypto = require('crypto');

function generateSignature( apiSecret, method, uri, timestamp, body ) {    
     const hmac = crypto.createHmac( 'SHA256', apiSecret );
 
     hmac.update( `${ method.toUpperCase() }${ uri }${ timestamp }` );
 
     if ( body ) {
         hmac.update( Buffer.from( body ) );
     }
 
     return hmac.digest('base64');
}

For the server to be able to validate the signature, both the timestamp and the signature are added to the request using headers. Unfortunately, there are no standardized headers for HMAC. 

I recommend using headers with a descriptive name like X-Request-Timestamp and X-Request-Signature. There are also implementations that use the Authorization header to transmit the hash.

When you're planning to support multiple clients, each with their own secret it may be easier to use the Authorization header and encode an id for the client and the signature in that header.

Validating the message

To validate the message, first ensure all the expected headers are available in the request. Next, ensure the timestamp of the request is within expected limits. 

It makes no sense for requests to be in the future, so future requests should be denied. It's probably safe to assume network latency is well below a second as well. There may be other factors at play though:

• A difference in clocks between client and server. 
• Cold-start delays, for example on Azure functions

The trade-off here is that the bigger the time difference you allow, the less failures you will have but the bigger the window for replay attacks becomes.

Next, validating the signature comes down to recalculating the hash using the request data an comparing it with the hash that was sent with the request.

Dealing with reverse proxies

As you may have spotted, the full URL of the API endpoint as called by the client is included in the data that calculates the hash. This is a point of concern for multiple reasons:

• Web servers are usually not case sensitive in processing urls, but hash calculation is.
• In an advanced network setup, the API is likely behind some sort of reverse proxy or gateway. This means the API doesn't necessarily know the public URL.
• HTTPS offloading changes the protocol from https to http.

Fortunately, reverse proxies tend to include the original protocol, host and path in the request in headers.

// Handle reverse proxy headers
var host = req.Headers.TryGetValue("X-Forwarded-Host", out var hostHeader) ? hostHeader.FirstOrDefault() : req.Host.Value;
var scheme =  req.Headers.TryGetValue("X-Forwarded-Scheme", out var schemeHeader ) ? schemeHeader.FirstOrDefault() : req.Scheme;
var path = $"{(string.IsNullOrEmpty(_options.BaseUri) ? "" : "/" + _options.BaseUri)}{req.Path.Value}";

//validate hash based on shared secret
var stringToHash = $"{req.Method}{scheme}://{host}{path}{signatureTimeStamp}{body}";
var signatureToMatch = CalculateHash(secret, stringToHash);

These headers work with most reverse proxies like NGINX Ingress and Azure Application Gateway. You'll have to look at the exact headers coming in to your service. In a previous post I explained how to capture the headers coming in to a .NET service.

Add HMAC support to Swagger

Swagger UI provides easy development access to an API. While it supports API keys and authentication, it does not support HMAC. Swagger UI is quite extensible though. The following code adds a request interceptor that uses the secret entered in the authentication dialog to add the timestamp and HMAC signature to the request. Since the interceptor runs in the browser it uses CryptoJS to calculate the hash in the browser.

HMAC support in API Management solutions

When you use an API management (APIM) service in your network, it makes sense to offload the HMAC validation to that service. APIMs are all about API security by managing API secrets for multiple API consumers and verifying access to the API.

Unfortunately, not every APIM supports HMAC request signing in the box. For some, like Azure API Management, it can be implemented in a custom policy. Others, like Kong, do have first class support for HMAC signature validation. 

Gotchas

There's a couple of things that can trip you up when implementing HMAC signing yourself:

• Character encoding; stick to UTF-8 as that works in most platforms.
• Casing of the URL, it's probably best to force lower case urls.

Further reading

• CryptoJS Docs on HMAC
• In depth explanation of message signing by Andrew Hoang