Throttling Your Api In Asp.net

Can someone DOS attack your API and bring down your webservice? Could I hit your API at 100 requests a second and bring down your server? Or can you throttle your users like this?

Sorry, I couldn't resist. How about like this?

If the answer is no, the good news is that it's quite easy to do in ASP.NET. You can use the fully featured WebApiThrottle, or if you don't need something fancy you can build it yourself.

Starting Off
Show me the limit(ing)!
Cleaning up the code
Being nice to your clients
Creating an attribute
How to use it
Update!

Starting Off

Let's start with the basic WebApi project template:

public class ValuesController : ApiController
{
    // GET api/values
    public IEnumerable<string> Get()
    {
        return new string[] { "value1", "value2" };
    }
}

Let's change the method so that it's returning a HttpResponseMessage object instead of an array of strings. This will give us more flexibility and make it easier to return different response codes.

public HttpResponseMessage Get()
{
    return Request.CreateResponse(HttpStatusCode.OK,  new string[] { "value1", "value2" });
}

and let's also change it to use WebApi2 and JSON responses, first off by modifying WebApiConfig:

public static class WebApiConfig
{
    public static void Register(HttpConfiguration config)
    {
        config.MapHttpAttributeRoutes();
        config.Formatters.Remove(config.Formatters.XmlFormatter);
    }
}

and then by adding the right attributes and changing it to return "Hello World":

[HttpGet]
[Route("~/api/helloworld")]
public HttpResponseMessage HelloWorld()
{
        return Request.CreateResponse(HttpStatusCode.OK, "Hello World");
}

Nice. Hitting our API in the browser should now show "Hello World":

Show me the limit(ing)!

Now let's implement a super-basic rate limiter:

[HttpGet]
[Route("~/api/helloworld")]
public HttpResponseMessage HelloWorld()
{
    int? requestCount = (int?)System.Web.HttpRuntime.Cache["throttle"];

    if (!requestCount.HasValue) requestCount = 0;

    requestCount++;

    HttpRuntime.Cache["throttle"] = requestCount;

    if (requestCount > 10) return Request.CreateResponse((HttpStatusCode)429, "Too many requests");

    return Request.CreateResponse(HttpStatusCode.OK, "Hello World");
}

Hitting our API ten times should then return a lovely 429 "Too Many Requests" response:

Don't use that code!

There's a problem here. When can I start making requests again? The answer is... you can't! Not until the web application restarts. Not cool. So we need a way to make sure that after a time limit, the request count resets. How about we make further use of the HttpRuntime.Cache object? Let's set our cache object to expire in 10 seconds:

[HttpGet]
[Route("~/api/helloworld")]
public HttpResponseMessage HelloWorld()
{
    ThrottleInfo throttleInfo = (ThrottleInfo)HttpRuntime.Cache["throttle"];

    if (throttleInfo == null)
        throttleInfo = new ThrottleInfo { 
                    ExpiresAt = DateTime.Now.AddSeconds(10), RequestCount = 0 };

    throttleInfo.RequestCount++;

    HttpRuntime.Cache.Add("throttle", throttleInfo, null, 
        throttleInfo.ExpiresAt, Cache.NoSlidingExpiration, 
                CacheItemPriority.Normal, null);

    if (throttleInfo.RequestCount > 10) 
            return Request.CreateResponse((HttpStatusCode)429, "Too many requests");

    return Request.CreateResponse(HttpStatusCode.OK, "Hello World");
}

private class ThrottleInfo
{
    public DateTime ExpiresAt { get; set; }
    public int RequestCount { get; set; }
}

Hang on... what's with the ThrottleInfo object?

The problem that we have with the Cache object is that there's no way to update a value in it without resetting the timeout window! This means we need to keep track of the expiration date, and whenever we update the RequestCount value, we also need to respecify the original expiration date.

If you hit your API 11 times, and then wait 10 seconds, and then hit it again, you should see your next request is allowed! Cool!

Cleaning up the code

Let's clean up the code a bit shall we?

public class ValuesController : ApiController
{
    [HttpGet]
    [Route("~/api/helloworld")]
    public HttpResponseMessage HelloWorld()
    {
        var throttler = new Throttler("helloworld");

        if (throttler.RequestShouldBeThrottled())
                    return Request.CreateResponse(
                        (HttpStatusCode)429, "Too many requests");

        return Request.CreateResponse(HttpStatusCode.OK, "Hello World");
    }
}

public class Throttler
{
    private int _requestLimit;
    private int _timeoutInSeconds;
    private string _key;

    public Throttler(string key, int requestLimit = 5, int timeoutInSeconds = 10)
    {
        _requestLimit = requestLimit;
        _timeoutInSeconds = timeoutInSeconds;
        _key = key;
    }

    public bool RequestShouldBeThrottled()
    {
        ThrottleInfo throttleInfo = (ThrottleInfo)HttpRuntime.Cache[_key];

        if (throttleInfo == null) throttleInfo = new ThrottleInfo {
                ExpiresAt = DateTime.Now.AddSeconds(_timeoutInSeconds),
                RequestCount = 0
        };

        throttleInfo.RequestCount++;

        HttpRuntime.Cache.Add(_key,
            throttleInfo,
            null,
            throttleInfo.ExpiresAt,
            Cache.NoSlidingExpiration,
            CacheItemPriority.Normal,
            null);

        return (throttleInfo.RequestCount > _requestLimit);
    }

    private class ThrottleInfo
    {
        public DateTime ExpiresAt { get; set; }
        public int RequestCount { get; set; }
    }
}

Oh hang on.. do we really need to depend on the HttpRuntime.Cache object? Not really. It’s not providing much value here except for the expiry of request info. If we want to remove the dependency on the HttpRuntime to make unit testing simpler we can just use a ConcurrentDictionary, which also means in future we could swap it out for a database store or something else. Let’s do that:

public class Throttler
{
    private int _requestLimit;
    private int _timeoutInSeconds;
    private string _key;
    private static ConcurrentDictionary&lt;string, ThrottleInfo&gt; _cache = 
        new ConcurrentDictionary&lt;string, ThrottleInfo&gt;();

    public Throttler(string key, int requestLimit = 5, int timeoutInSeconds = 10)
    {
        _requestLimit = requestLimit;
        _timeoutInSeconds = timeoutInSeconds;
        _key = key;
    }

    public bool RequestShouldBeThrottled()
    {
        ThrottleInfo throttleInfo = _cache.ContainsKey(_key) ? _cache[_key] : null;

        if (throttleInfo == null || throttleInfo.ExpiresAt <= DateTime.Now) 
        {
            throttleInfo = new ThrottleInfo {
                ExpiresAt = DateTime.Now.AddSeconds(_timeoutInSeconds),
                RequestCount = 0};
        };

        throttleInfo.RequestCount++;
        
        _cache[_key] = throttleInfo;

        return (throttleInfo.RequestCount > _requestLimit);
    }

    private class ThrottleInfo
    {
        public DateTime ExpiresAt { get; set; }
        public int RequestCount { get; set; }
    }
}

We now have our lovely self contained Throttler class for throttling our WebApi methods.

So what can we do with our lovely throttle object? Well, we can throttle each method separately:

[HttpGet]
[Route("~/api/helloworld")]
public HttpResponseMessage HelloWorld()
{
    var throttler = new Throttler("helloworld");

    if (throttler.RequestShouldBeThrottled())
            return Request.CreateResponse(
                (HttpStatusCode)429, "Too many requests");

    return Request.CreateResponse(HttpStatusCode.OK, "Hello World");
}

[HttpGet]
[Route("~/api/updatesomething")]
public HttpResponseMessage UpdateSomething()
{
    var throttler = new Throttler("updatesomething");

    if (throttler.RequestShouldBeThrottled())
            return Request.CreateResponse(
                (HttpStatusCode)429, "Too many requests");
            
    // update something here

    return Request.CreateResponse(HttpStatusCode.OK, "Data updated");
}

Or even better, we can throttle based on IP address:

[HttpGet]
[Route("~/api/helloworld")]
public HttpResponseMessage HelloWorld()
{
    var ipAddress = System.Web.HttpContext.Current.Request.ServerVariables["REMOTE_ADDR"];
    var throttler = new Throttler(ipAddress);

Or how about we throttle individual users?

[HttpGet]
[Route("~/api/helloworld")]
public HttpResponseMessage HelloWorld()
{
    var throttler = new Throttler(User.Identity.Name);

Being nice to your clients

But wait.. how are your clients expected to keep track of when they have gone over their limit and when the reset window will be? Well, it's a bit unfair of us to not provide this info to them if we already have it.

Don't worry, just use the standard HTTP headers. Oh wait.

There is no standard

Sigh. Ok, let’s settle on using Github’s headers and also include Vimeo’s “Reset” header too.

[HttpGet]
[Route("~/api/helloworld")]
public HttpResponseMessage HelloWorld()
{
    var throttler = new Throttler(User.Identity.Name);

    HttpResponseMessage response = createResponse("Hello World", throttler);

    return response;
}

private HttpResponseMessage createResponse(object content, Throttler throttler)
{
    HttpResponseMessage response;

    if (throttler.RequestShouldBeThrottled())
        response = Request.CreateResponse((HttpStatusCode)429, "Too many requests");
    else
        response = Request.CreateResponse(HttpStatusCode.OK, content);

    response.Headers.Add("X-RateLimit-Limit", throttler.RequestLimit.ToString());
    response.Headers.Add("X-RateLimit-Remaining", throttler.RequestsRemaining.ToString());
    response.Headers.Add("X-RateLimit-Reset", toUnixTime(throttler.WindowResetDate).ToString());

    return response;
}

private long toUnixTime(DateTime date)
{
    var epoch = new DateTime(1970, 1, 1, 0, 0, 0, DateTimeKind.Utc);
    return Convert.ToInt64((date.ToUniversalTime() - epoch).TotalSeconds);
}

And let’s add the code in to support returning our three new headers:

public class Throttler
{
    public int RequestLimit { get; private set; }
    public int RequestsRemaining { get; private set; }
    public DateTime WindowResetDate { get; private set; }
    private static ConcurrentDictionary&lt;string, ThrottleInfo&gt; _cache = 
        new ConcurrentDictionary&lt;string, ThrottleInfo&gt;();

    private string _key;
    private int _timeoutInSeconds;

    public Throttler(string key, int requestLimit = 5, int timeoutInSeconds = 10)
    {
        RequestLimit = requestLimit;
        _timeoutInSeconds = timeoutInSeconds;
        _key = key;
    }

    public bool RequestShouldBeThrottled()
    {
        ThrottleInfo throttleInfo = _cache.ContainsKey(_key) ? _cache[_key] : null;

        if (throttleInfo == null || throttleInfo.ExpiresAt <= DateTime.Now) 
        {
            throttleInfo = new ThrottleInfo {
                ExpiresAt = DateTime.Now.AddSeconds(_timeoutInSeconds),
                RequestCount = 0};
        };

        WindowResetDate = throttleInfo.ExpiresAt;

        throttleInfo.RequestCount++;

        _cache[ThrottleGroup] = throttleInfo;

        RequestsRemaining = Math.Max(RequestLimit - throttleInfo.RequestCount, 0);

        return (throttleInfo.RequestCount > RequestLimit);
    }

    private class ThrottleInfo
    {
        public DateTime ExpiresAt { get; set; }
        public int RequestCount { get; set; }
    }
}

Creating an attribute

The main downside to this implementation is that we have to create a new instance of our Throttle class and call RequestShouldBeThrottled() each time. This is a bit of a pain and some duplicate code, how about we create our own Attribute that can be simply added to each WebApi method?

Let's create a new filter attribute:

Now we're getting somewhere! We've added three new features:

We can throttle based on the client's IP address
We can throttle based on the client's identity
The request count is only incremented upon a successful request

Now let's update our throttling class to support modifying the group dynamically:

Phew!

How to use it

Basic usage for throttling requests:

[ThrottleFilter()]
[HttpGet]
[Route("~/api/helloworld")]
public HttpResponseMessage HelloWorld()
{
  return Request.CreateResponse(HttpStatusCode.OK, "Hello World");
}

Allow more requests through, say 50 every 5 seconds:

[ThrottleFilter(RequestLimit: 50, TimeoutInSeconds: 5)]
[HttpGet]
[Route("~/api/allow-more")]
public HttpResponseMessage HelloWorld2()
{
    return Request.CreateResponse(HttpStatusCode.OK, "Hello World2");
}

Throttling a group of requests together:

[ThrottleFilter(ThrottleGroup: "updates")]
[HttpPost]
[Route("~/api/name")]
public HttpResponseMessage UpdateName(UpdateNameDTO NameUpdate)
{
    // update name here
    return Request.CreateResponse(HttpStatusCode.OK, "Name updated ok");
}

[ThrottleFilter(ThrottleGroup: "updates")]
[HttpPost]
[Route("~/api/address")]
public HttpResponseMessage UpdateAddress(UpdateAddressDTO AddressUpdate)
{
    // update address here
    return Request.CreateResponse(HttpStatusCode.OK, "Address updated ok");
}

Throttling based on IP address:

[ThrottleFilter(ThrottleGroup: "ipaddress")]
[HttpGet]
[Route("~/api/name")]
public HttpResponseMessage GetName(int id)
{
    return Request.CreateResponse(HttpStatusCode.OK, "John Smith");
}

Throttling based on Identity:

[ThrottleFilter(ThrottleGroup: "identity")]
[HttpGet]
[Route("~/api/name")]
public HttpResponseMessage GetName(int id)
{
    return Request.CreateResponse(HttpStatusCode.OK, "Jane Doe");
}

Enjoy!

Update!

After posting this article on reddit's csharp forum, user ArSoron left a great comment showing a fix for a potential race condition in the code. It turns out using ConcurrentDictionary is not enough, you need to make sure to use the AddOrUpdate method, like so:

public void IncrementRequestCount()
{
    _cache.AddOrUpdate(ThrottleGroup, new ThrottleInfo
    {
            ExpiresAt = DateTime.Now.AddSeconds(_timeoutInSeconds),
            RequestCount = 1
    }, (retrievedKey, throttleInfo) =>
    {
            Interlocked.Increment(ref throttleInfo.RequestCount);
            return throttleInfo;
    });
}

He also points out that ThrottleInfo could be changed to a struct for slightly better performance. Thanks mate!

Table Of Contents