Class GradientCapacityLimiterBuilder
GradientCapacityLimiter
capacity limiter.-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic interface
A state observer for GradientCapacityLimiter
to monitor internal state changes. -
Method Summary
Modifier and TypeMethodDescriptionbackoffRatio
(float onDrop, float onLimit) Defines the backoff ratios when certain conditions are met.build()
Builds a Gradient dynamicCapacityLimiter
based on config options of this builder.headroom
(BiFunction<Double, Double, Double> headroom) A function used to positively influence the new limit when the detected gradient is positive (i.e.limits
(int initial, int min, int max) limitUpdateInterval
(Duration duration) How often a new sampled RTT must be collected to update the concurrency limit.maxGradient
(float maxPositiveGradient) Defines the max gradient allowance persampling-interval
.minGradient
(float minGradient) Defines the min gradient allowance persampling-interval
.Defines a name for thisCapacityLimiter
.Aobserver
to consume the current state of thisCapacityLimiter
when state changes occur.suspendLimitDecrease
(BiPredicate<Integer, Double> suspendLimitDec) A function to suspend the decrease of the limit when that is not consumed (e.g.suspendLimitIncrease
(BiPredicate<Integer, Double> suspendLimitInc) A function to suspend the increase of the limit when that is not consumed (e.g.
-
Method Details
-
name
Defines a name for thisCapacityLimiter
.- Parameters:
name
- the name to be used when building thisCapacityLimiter
.- Returns:
this
.
-
limits
Defineinitial
,min
andmax
concurrency limits for thisCapacityLimiter
. The active concurrency will fluctuate between these limits starting from themin
and never going beyondmax
.The limit translates to a concurrency figure, e.g. how many requests can be in-flight simultaneously and doesn't represent a constant rate (i.e. has no notion of time).
The lower the
initial
ormin
are, the slower the ramp up will be, and the bigger it is the more aggressive the client will be, keep concurrently issuingmin
requests to meet this limit.Min must always be less than max, and ideally max should be greater by 10x.
- Parameters:
initial
- The initial concurrency allowed, helps with faster start.min
- The minimum concurrency allowed.max
- The maximum concurrency allowed.- Returns:
this
.
-
backoffRatio
Defines the backoff ratios when certain conditions are met. Ratios are used to alter the limit of theCapacityLimiter
by the provided multiplier on different conditions as identified by their name.The formula for the backoff ratio used is:
NewLimit = OldLimit * BackoffRatio
, always respecting themin
andmax
values.Both limits must be between 0 and 1 exclusively.
- Parameters:
onDrop
- The backoff ratio used to bring the limit down by that amount, when a request is dropped either by a server response identified as a rejection, or by a local timeout.onLimit
- The backoff ratio used to bring the limit down by that amount, when the maximum limit is reached.- Returns:
this
.
-
headroom
A function used to positively influence the new limit when the detected gradient is positive (i.e. > 1.0). This can be used to grow the limit faster when the algorithm is being very conservative. Through testing a good headroom function can be the square root of the gradient (i.e.Math.sqrt(double)
). The input of the function is the newly calculated gradient, and the previous limit in this order.- Parameters:
headroom
- An extra padding for the new limit if the algorithm is being very conservative.- Returns:
this
.
-
limitUpdateInterval
How often a new sampled RTT must be collected to update the concurrency limit. This interval is part of the normalCapacityLimiter.tryAcquire(Classification, ContextMap)
flow, thus no externalExecutor
is used. The more traffic goes through the more accurate it will be. As a result this is not a hard deadline, but rather an at-least figure.There is a hard min duration applied of
50ms
that can't be overridden. Any input less than that value will result in aIllegalArgumentException
. That minimum interval was determined experimentally to avoid extreme adjustments of the limit without a cool off period.- Parameters:
duration
- The duration between sampling an RTT value.- Returns:
this
.
-
minGradient
Defines the min gradient allowance persampling-interval
. This helps push the limit upwards by not allowing quick drops, and it tends to maintain that higher limit.- Parameters:
minGradient
- The minimum allowed gradient persampling interval
.- Returns:
this
.
-
maxGradient
Defines the max gradient allowance persampling-interval
. This helps limit how fast the limit can grow, allowing the peer to adjust to the change, before the limit grows out of control causing start-stop reactions (saw-tooth patterns in traffic).- Parameters:
maxPositiveGradient
- The maximum allowed gradient persampling interval
.- Returns:
this
.
-
observer
Aobserver
to consume the current state of thisCapacityLimiter
when state changes occur. Useful to monitor the limit through logging or metrics, or just debugging.It's expected that this
GradientCapacityLimiterBuilder.Observer
is not going to block the thread that invokes it.- Parameters:
observer
- TheGradientCapacityLimiterBuilder.Observer
to inform about the current capacity and consumption of thisCapacityLimiter
.- Returns:
this
.
-
suspendLimitIncrease
public GradientCapacityLimiterBuilder suspendLimitIncrease(BiPredicate<Integer, Double> suspendLimitInc) A function to suspend the increase of the limit when that is not consumed (e.g. rate of requests isn't crossing it). This helps prevent the limit growing to infinite, and provides faster reaction when a reduction happens. Additionally, this works as a "blast-radius" concept, effectively limiting spontaneous traffic surges.- Parameters:
suspendLimitInc
- TheBiPredicate
that should returntrue
when the limit should halt increasing based on current consumption (first argument asInteger
) and current limit (second argument asDouble
). HelperAPI
to offer blast-radius type predicates.- Returns:
this
.
-
suspendLimitDecrease
public GradientCapacityLimiterBuilder suspendLimitDecrease(BiPredicate<Integer, Double> suspendLimitDec) A function to suspend the decrease of the limit when that is not consumed (e.g. rate of requests isn't crossing it). When the monitored RTT is considerably noisy (deviations > 500ms) the limit tends to decrease faster, even when there is no significant utilization of it going on. This helps prevent the decrease, until the providedPredicate
toggles it.- Parameters:
suspendLimitDec
- TheBiPredicate
that should returntrue
when the limit should halt decreasing based on current consumption (first argument asInteger
) and current limit (second argument asDouble
). HelperAPI
to offer occupancy-factor type predicates.- Returns:
this
.
-
build
Builds a Gradient dynamicCapacityLimiter
based on config options of this builder.- Returns:
- A dynamic
CapacityLimiter
based on the options ofthis
builder.
-