Class GradientCapacityLimiterBuilder

java.lang.Object
io.servicetalk.capacity.limiter.api.GradientCapacityLimiterBuilder

public final class GradientCapacityLimiterBuilder extends Object
Builder for the GradientCapacityLimiter capacity limiter.
  • Method Details

    • name

      Defines a name for this CapacityLimiter.
      Parameters:
      name - the name to be used when building this CapacityLimiter.
      Returns:
      this.
    • limits

      public GradientCapacityLimiterBuilder limits(int initial, int min, int max)
      Define initial, min and max concurrency limits for this CapacityLimiter. The active concurrency will fluctuate between these limits starting from the min and never going beyond max.

      The limit translates to a concurrency figure, e.g. how many requests can be in-flight simultaneously and doesn't represent a constant rate (i.e. has no notion of time).

      The lower the initial or min are, the slower the ramp up will be, and the bigger it is the more aggressive the client will be, keep concurrently issuing min requests to meet this limit.

      Min must always be less than max, and ideally max should be greater by 10x.

      Parameters:
      initial - The initial concurrency allowed, helps with faster start.
      min - The minimum concurrency allowed.
      max - The maximum concurrency allowed.
      Returns:
      this.
    • backoffRatio

      public GradientCapacityLimiterBuilder backoffRatio(float onDrop, float onLimit)
      Defines the backoff ratios when certain conditions are met. Ratios are used to alter the limit of the CapacityLimiter by the provided multiplier on different conditions as identified by their name.

      The formula for the backoff ratio used is: NewLimit = OldLimit * BackoffRatio, always respecting the min and max values.

      Both limits must be between 0 and 1 exclusively.

      Parameters:
      onDrop - The backoff ratio used to bring the limit down by that amount, when a request is dropped either by a server response identified as a rejection, or by a local timeout.
      onLimit - The backoff ratio used to bring the limit down by that amount, when the maximum limit is reached.
      Returns:
      this.
    • headroom

      A function used to positively influence the new limit when the detected gradient is positive (i.e. > 1.0). This can be used to grow the limit faster when the algorithm is being very conservative. Through testing a good headroom function can be the square root of the gradient (i.e. Math.sqrt(double)). The input of the function is the newly calculated gradient, and the previous limit in this order.
      Parameters:
      headroom - An extra padding for the new limit if the algorithm is being very conservative.
      Returns:
      this.
    • limitUpdateInterval

      public GradientCapacityLimiterBuilder limitUpdateInterval(Duration duration)
      How often a new sampled RTT must be collected to update the concurrency limit. This interval is part of the normal CapacityLimiter.tryAcquire(Classification, ContextMap) flow, thus no external Executor is used. The more traffic goes through the more accurate it will be. As a result this is not a hard deadline, but rather an at-least figure.

      There is a hard min duration applied of 50ms that can't be overridden. Any input less than that value will result in a IllegalArgumentException. That minimum interval was determined experimentally to avoid extreme adjustments of the limit without a cool off period.

      Parameters:
      duration - The duration between sampling an RTT value.
      Returns:
      this.
    • minGradient

      public GradientCapacityLimiterBuilder minGradient(float minGradient)
      Defines the min gradient allowance per sampling-interval. This helps push the limit upwards by not allowing quick drops, and it tends to maintain that higher limit.
      Parameters:
      minGradient - The minimum allowed gradient per sampling interval.
      Returns:
      this.
    • maxGradient

      public GradientCapacityLimiterBuilder maxGradient(float maxPositiveGradient)
      Defines the max gradient allowance per sampling-interval. This helps limit how fast the limit can grow, allowing the peer to adjust to the change, before the limit grows out of control causing start-stop reactions (saw-tooth patterns in traffic).
      Parameters:
      maxPositiveGradient - The maximum allowed gradient per sampling interval.
      Returns:
      this.
    • observer

      A observer to consume the current state of this CapacityLimiter when state changes occur. Useful to monitor the limit through logging or metrics, or just debugging.

      It's expected that this GradientCapacityLimiterBuilder.Observer is not going to block the thread that invokes it.

      Parameters:
      observer - The GradientCapacityLimiterBuilder.Observer to inform about the current capacity and consumption of this CapacityLimiter.
      Returns:
      this.
    • suspendLimitIncrease

      public GradientCapacityLimiterBuilder suspendLimitIncrease(BiPredicate<Integer,Double> suspendLimitInc)
      A function to suspend the increase of the limit when that is not consumed (e.g. rate of requests isn't crossing it). This helps prevent the limit growing to infinite, and provides faster reaction when a reduction happens. Additionally, this works as a "blast-radius" concept, effectively limiting spontaneous traffic surges.
      Parameters:
      suspendLimitInc - The BiPredicate that should return true when the limit should halt increasing based on current consumption (first argument as Integer) and current limit (second argument as Double). Helper API to offer blast-radius type predicates.
      Returns:
      this.
    • suspendLimitDecrease

      public GradientCapacityLimiterBuilder suspendLimitDecrease(BiPredicate<Integer,Double> suspendLimitDec)
      A function to suspend the decrease of the limit when that is not consumed (e.g. rate of requests isn't crossing it). When the monitored RTT is considerably noisy (deviations > 500ms) the limit tends to decrease faster, even when there is no significant utilization of it going on. This helps prevent the decrease, until the provided Predicate toggles it.
      Parameters:
      suspendLimitDec - The BiPredicate that should return true when the limit should halt decreasing based on current consumption (first argument as Integer) and current limit (second argument as Double). Helper API to offer occupancy-factor type predicates.
      Returns:
      this.
    • build

      public CapacityLimiter build()
      Builds a Gradient dynamic CapacityLimiter based on config options of this builder.
      Returns:
      A dynamic CapacityLimiter based on the options of this builder.