Class CapacityLimiters

java.lang.Object
io.servicetalk.capacity.limiter.api.CapacityLimiters

public final class CapacityLimiters extends Object
A static factory for creating instances of CapacityLimiters.
  • Method Details

    • allowAll

      public static CapacityLimiter allowAll()
      Returns a NO-OP CapacityLimiter that has no logic around acquiring or releasing a permit for a request; thus it allows everything to go through, similar to a non-existing CapacityLimiter.

      This CapacityLimiter allows for situations where partitioned configurations are in use through a resilience filter, and you want to limit one partition but not necessary the other.

      Returns:
      A NO-OP capacity limiter.
    • composite

      public static CapacityLimiter composite(List<CapacityLimiter> providers)
      A composite CapacityLimiter that is composed of multiple CapacityLimiters.

      All capacity limiters need to grant a CapacityLimiter.Ticket for the request to be allowed.

      Parameters:
      providers - The individual CapacityLimiter that form the composite result.
      Returns:
      A CapacityLimiter that is composed by the sum of all the CapacityLimiters passed as arguments.
    • fixedCapacity

      public static FixedCapacityLimiterBuilder fixedCapacity(int capacity)
      A CapacityLimiter that will reject all requests till the current pending request count is equal or less to the passed capacity.

      This CapacityLimiter takes into consideration the Classification of a given request and will variate the effective capacity according to the priority before attempting to grant access to the request. The effective capacity will never be more than the given capacity.

      Requests with priority equal to or greater than 100 will enjoy the full capacity (100%), while requests with priority less than 100 will be mapped to a percentage point of the given capacity and be granted access only if the consumed capacity is less than that percentage.

      Example: With a capacity = 10, and incoming Classification.priority() = 70, then the effective target limit for this request will be 70% of the 10 = 7. If current consumption is less than 7, the request will be permitted.

      Parameters:
      capacity - The fixed capacity value for this limiter.
      Returns:
      A FixedCapacityLimiterBuilder to configure the available parameters.
    • dynamicAIMD

      public static AimdCapacityLimiterBuilder dynamicAIMD()
      AIMD is a request drop based dynamic CapacityLimiter for clients, that adapts its limit based on a configurable range of concurrency and re-evaluates this limit upon a request-drop event (eg. timeout or rejection due to capacity).

      It's not ideal for server-side solutions, due to the slow recover mechanism it offers, which can lead in significant traffic loss during the recovery window.

      The limit translates to a concurrency figure, e.g. how many requests can be in-flight simultaneously and doesn't represent a constant rate (i.e. has no notion of time). Requests per second when that limit is met will be equal to the exit rate of the queue.

      The solution is based on the AIMD feedback control algorithm.

      Returns:
      An AimdCapacityLimiterBuilder to configure the available parameters.
    • dynamicGradient

      public static GradientCapacityLimiterBuilder dynamicGradient()
      Gradient is a dynamic concurrency limit algorithm used for clients or servers.

      Gradient's basic concept is that it tracks two Round Trip Time (RTT) figures, one of long period and another one of a shorter period. These two figures are then compared, and a gradient value is produced, representing the change between the two. That gradient value can in-turn be used to signify load; i.e. a positive gradient can mean that the RTTs are decreasing, whereas a negative value means that RTTs are increasing. This figure can be used to deduce a new limit (lower or higher accordingly) to follow the observed load pattern.

      The algorithm is heavily influenced by the following prior-art:

      1. Envoy Adaptive Concurrency
      2. Netflix Concurrency Limits
      Returns:
      A GradientCapacityLimiterBuilder to configure the available parameters.
      See Also:
    • dynamicGradientOptimizeForThroughput

      public static GradientCapacityLimiterBuilder dynamicGradientOptimizeForThroughput()
      Gradient is a dynamic concurrency limit algorithm used for clients or servers.

      Gradient's basic concept is that it tracks two Round Trip Time (RTT) figures, one of long period and another one of a shorter period. These two figures are then compared, and a gradient value is produced, representing the change between the two. That gradient value can in-turn be used to signify load; i.e. a positive gradient can mean that the RTTs are decreasing, whereas a negative value means that RTTs are increasing. This figure can be used to deduce a new limit (lower or higher accordingly) to follow the observed load pattern.

      The default settings applied on this version, demonstrate aggressive behaviour of the CapacityLimiter, that tries to push the limit higher until a significant gradient change is noticed. It will allow limit increases while latency is changing, favouring throughput overall, so latency sensitive application may not want to use this profile.

      The algorithm is heavily influenced by the following prior-art:

      1. Envoy Adaptive Concurrency
      2. Netflix Concurrency Limits
      Returns:
      A GradientCapacityLimiterBuilder to configure the available parameters.
      See Also:
    • dynamicGradientOptimizeForLatency

      public static GradientCapacityLimiterBuilder dynamicGradientOptimizeForLatency()
      Gradient is a dynamic concurrency limit algorithm used for clients or servers.

      Gradient's basic concept is that it tracks two Round Trip Time (RTT) figures, one of long period and another one of a shorter period. These two figures are then compared, and a gradient value is produced, representing the change between the two. That gradient value can in-turn be used to signify load; i.e. a positive gradient can mean that the RTTs are decreasing, whereas a negative value means that RTTs are increasing. This figure can be used to deduce a new limit (lower or higher accordingly) to follow the observed load pattern.

      The default settings applied from this version, demonstrate cautious behaviour of the CapacityLimiter, that tries to keep the limit lower to avoid increasing the latency of requests. This is a suggested setting for latency sensitive applications, but be aware that it may start throttling much earlier when even small gradients are noticed in the response times.

      The algorithm is heavily influenced by the following prior-art:

      1. Envoy Adaptive Concurrency
      2. Netflix Concurrency Limits
      Returns:
      A GradientCapacityLimiterBuilder to configure the available parameters.
      See Also: