Class CapacityLimiters
CapacityLimiter
s.-
Method Summary
Modifier and TypeMethodDescriptionstatic CapacityLimiter
allowAll()
Returns a NO-OPCapacityLimiter
that has no logic around acquiring or releasing a permit for a request; thus it allows everything to go through, similar to a non-existingCapacityLimiter
.static CapacityLimiter
composite
(List<CapacityLimiter> providers) A compositeCapacityLimiter
that is composed of multipleCapacityLimiter
s.static AimdCapacityLimiterBuilder
AIMD is a request drop based dynamicCapacityLimiter
for clients, that adapts its limit based on a configurable range of concurrency and re-evaluates this limit upon arequest-drop event (eg. timeout or rejection due to capacity)
.Gradient is a dynamic concurrency limit algorithm used for clients or servers.Gradient is a dynamic concurrency limit algorithm used for clients or servers.Gradient is a dynamic concurrency limit algorithm used for clients or servers.static FixedCapacityLimiterBuilder
fixedCapacity
(int capacity) ACapacityLimiter
that will reject all requests till the current pending request count is equal or less to the passedcapacity
.
-
Method Details
-
allowAll
Returns a NO-OPCapacityLimiter
that has no logic around acquiring or releasing a permit for a request; thus it allows everything to go through, similar to a non-existingCapacityLimiter
.This
CapacityLimiter
allows for situations where partitioned configurations are in use through a resilience filter, and you want to limit one partition but not necessary the other.- Returns:
- A NO-OP
capacity limiter
.
-
composite
A compositeCapacityLimiter
that is composed of multipleCapacityLimiter
s.All capacity limiters need to grant a
CapacityLimiter.Ticket
for the request to be allowed.- Parameters:
providers
- The individualCapacityLimiter
that form the composite result.- Returns:
- A
CapacityLimiter
that is composed by the sum of all theCapacityLimiter
s passed as arguments.
-
fixedCapacity
ACapacityLimiter
that will reject all requests till the current pending request count is equal or less to the passedcapacity
.This
CapacityLimiter
takes into consideration theClassification
of a given request and will variate the effectivecapacity
according to thepriority
before attempting to grant access to the request. The effectivecapacity
will never be more than the givencapacity
.Requests with
priority
equal to or greater than100
will enjoy the full capacity (100%), while requests withpriority
less than100
will be mapped to a percentage point of the givencapacity
and be granted access only if theconsumed capacity
is less than that percentage.Example: With a
capacity
= 10, and incomingClassification.priority()
= 70, then the effective target limit for this request will be 70% of the 10 = 7. If current consumption is less than 7, the request will be permitted.- Parameters:
capacity
- The fixed capacity value for this limiter.- Returns:
- A
FixedCapacityLimiterBuilder
to configure the available parameters.
-
dynamicAIMD
AIMD is a request drop based dynamicCapacityLimiter
for clients, that adapts its limit based on a configurable range of concurrency and re-evaluates this limit upon arequest-drop event (eg. timeout or rejection due to capacity)
.It's not ideal for server-side solutions, due to the slow recover mechanism it offers, which can lead in significant traffic loss during the recovery window.
The limit translates to a concurrency figure, e.g. how many requests can be in-flight simultaneously and doesn't represent a constant rate (i.e. has no notion of time). Requests per second when that limit is met will be equal to the exit rate of the queue.
The solution is based on the AIMD feedback control algorithm.
- Returns:
- An
AimdCapacityLimiterBuilder
to configure the available parameters.
-
dynamicGradient
Gradient is a dynamic concurrency limit algorithm used for clients or servers.Gradient's basic concept is that it tracks two Round Trip Time (RTT) figures, one of long period and another one of a shorter period. These two figures are then compared, and a gradient value is produced, representing the change between the two. That gradient value can in-turn be used to signify load; i.e. a positive gradient can mean that the RTTs are decreasing, whereas a negative value means that RTTs are increasing. This figure can be used to deduce a new limit (lower or higher accordingly) to follow the observed load pattern.
The algorithm is heavily influenced by the following prior-art:
- Returns:
- A
GradientCapacityLimiterBuilder
to configure the available parameters. - See Also:
-
dynamicGradientOptimizeForThroughput
Gradient is a dynamic concurrency limit algorithm used for clients or servers.Gradient's basic concept is that it tracks two Round Trip Time (RTT) figures, one of long period and another one of a shorter period. These two figures are then compared, and a gradient value is produced, representing the change between the two. That gradient value can in-turn be used to signify load; i.e. a positive gradient can mean that the RTTs are decreasing, whereas a negative value means that RTTs are increasing. This figure can be used to deduce a new limit (lower or higher accordingly) to follow the observed load pattern.
The default settings applied on this version, demonstrate aggressive behaviour of the
CapacityLimiter
, that tries to push the limit higher until a significant gradient change is noticed. It will allow limit increases while latency is changing, favouring throughput overall, so latency sensitive application may not want to use this profile.The algorithm is heavily influenced by the following prior-art:
- Returns:
- A
GradientCapacityLimiterBuilder
to configure the available parameters. - See Also:
-
dynamicGradientOptimizeForLatency
Gradient is a dynamic concurrency limit algorithm used for clients or servers.Gradient's basic concept is that it tracks two Round Trip Time (RTT) figures, one of long period and another one of a shorter period. These two figures are then compared, and a gradient value is produced, representing the change between the two. That gradient value can in-turn be used to signify load; i.e. a positive gradient can mean that the RTTs are decreasing, whereas a negative value means that RTTs are increasing. This figure can be used to deduce a new limit (lower or higher accordingly) to follow the observed load pattern.
The default settings applied from this version, demonstrate cautious behaviour of the
CapacityLimiter
, that tries to keep the limit lower to avoid increasing the latency of requests. This is a suggested setting for latency sensitive applications, but be aware that it may start throttling much earlier when even small gradients are noticed in the response times.The algorithm is heavily influenced by the following prior-art:
- Returns:
- A
GradientCapacityLimiterBuilder
to configure the available parameters. - See Also:
-