Package io.servicetalk.loadbalancer

Interface LoadBalancerBuilder<ResolvedAddress,C extends LoadBalancedConnection>

Type Parameters:

ResolvedAddress - The resolved address type.

C - The type of connection.

All Known Implementing Classes:

DelegatingLoadBalancerBuilder

public interface LoadBalancerBuilder<ResolvedAddress,C extends LoadBalancedConnection>

Builder for LoadBalancerFactory that creates LoadBalancer instances based upon the configuration.

The addresses are provided via the published events that signal the host's status. Instances returned handle ServiceDiscovererEvent.Status.AVAILABLE, ServiceDiscovererEvent.Status.EXPIRED, and ServiceDiscovererEvent.Status.UNAVAILABLE event statuses.

The created instances have the following behaviour:

• Host selection is performed based upon the provided LoadBalancingPolicy. If no policy is provided the default policy is round-robin. While it can be policy specific, most policies will avoid unhealthy hosts.
• Host health is determined by the configured OutlierDetectorConfig. This is inferred from the results of requests and from actively probing for connectivity in accordance with the config.
• Connections are created lazily, without any concurrency control on their creation. This can lead to over-provisioning connections when dealing with a requests surge.
• Existing connections are reused unless a selector passed to LoadBalancer.selectConnection(Predicate, ContextMap) suggests otherwise. This can lead to situations where connections will be used to their maximum capacity (for example in the context of pipelining) before new connections are created.
• Closed connections are automatically pruned.
• When Publisher<ServiceDiscovererEvent> delivers events with ServiceDiscovererEvent.status() of value ServiceDiscovererEvent.Status.UNAVAILABLE, connections are immediately closed (gracefully) for the associated ServiceDiscovererEvent.address(). In case of ServiceDiscovererEvent.Status.EXPIRED, already established connections to ServiceDiscovererEvent.address() continue to be used for requests, but no new connections are created. In case the address' connections are busy, another host is tried. If all hosts are busy based on the selection mechanism of the LoadBalancingPolicy, selection fails with a NoActiveHostException.

Method Summary

Modifier and Type	Method	Description
LoadBalancerBuilder<ResolvedAddress,C>	backgroundExecutor(Executor backgroundExecutor)	Set the background Executor to use for determining time and scheduling background tasks such as those associated with outlier detection.
LoadBalancerFactory<ResolvedAddress,C>	build()	Builds the LoadBalancerFactory configured by this builder.
LoadBalancerBuilder<ResolvedAddress,C>	connectionSelectorPolicy(ConnectionSelectorPolicy<C> connectionSelectorPolicy)	Set the ConnectionSelectorPolicy to use with this load balancer.
LoadBalancerBuilder<ResolvedAddress,C>	loadBalancerObserver(LoadBalancerObserverFactory loadBalancerObserverFactory)	Set the LoadBalancerObserverFactory to use with this load balancer.
LoadBalancerBuilder<ResolvedAddress,C>	loadBalancingPolicy(LoadBalancingPolicy<ResolvedAddress,C> loadBalancingPolicy)	Set the loadBalancingPolicy to use with this load balancer.
default LoadBalancerBuilder<ResolvedAddress,C>	maxRandomSubsetSize(int maxUsed)	Set the maximum number of healthy backends to load balance against using a random-subsetting strategy.
default LoadBalancerBuilder<ResolvedAddress,C>	minConnectionsPerHost(int minConnectionsPerHost)	Set the minimum number of connections an active host should maintain in its pool.
LoadBalancerBuilder<ResolvedAddress,C>	outlierDetectorConfig(OutlierDetectorConfig outlierDetectorConfig)	Set the OutlierDetectorConfig to use with this load balancer.

Method Details

loadBalancingPolicy

LoadBalancerBuilder<ResolvedAddress,C> loadBalancingPolicy(LoadBalancingPolicy<ResolvedAddress,C> loadBalancingPolicy)

Set the loadBalancingPolicy to use with this load balancer.

Parameters:

loadBalancingPolicy - the loadBalancingPolicy to use

Returns:

this

loadBalancerObserver

LoadBalancerBuilder<ResolvedAddress,C> loadBalancerObserver(@Nullable
 LoadBalancerObserverFactory loadBalancerObserverFactory)

Set the LoadBalancerObserverFactory to use with this load balancer.

Parameters:

loadBalancerObserverFactory - the LoadBalancerObserverFactory to use, or null to not use an observer.

Returns:

{code this}

outlierDetectorConfig

LoadBalancerBuilder<ResolvedAddress,C> outlierDetectorConfig(OutlierDetectorConfig outlierDetectorConfig)

Set the OutlierDetectorConfig to use with this load balancer.

The outlier detection system works in conjunction with the load balancing policy to attempt to avoid hosts that have been determined to be unhealthy or slow. The details of the selection process are determined by the LoadBalancingPolicy while the health status is determined by the outlier detection configuration.

Parameters:

outlierDetectorConfig - the OutlierDetectorConfig to use, or null to use the default outlier detection.

Returns:

{code this}

See Also:

• loadBalancingPolicy(LoadBalancingPolicy)
• for common preconfigured policies

connectionSelectorPolicy

LoadBalancerBuilder<ResolvedAddress,C> connectionSelectorPolicy(ConnectionSelectorPolicy<C> connectionSelectorPolicy)

Set the ConnectionSelectorPolicy to use with this load balancer.

Parameters:

connectionSelectorPolicy - the factory of connection selection strategies to use.

Returns:

this

minConnectionsPerHost

default LoadBalancerBuilder<ResolvedAddress,C> minConnectionsPerHost(int minConnectionsPerHost)

Set the minimum number of connections an active host should maintain in its pool.

Setting a minimum number of connections can help reduce the occurrences where connection establishment happens on the request path, thereby reducing tail latencies, particularly for links which are prone idle-connection closure. The tradeoff is that it may require tuning and will create more connection overhead overall.

Note: this is a best-effort setting. There are inherent uncertainties, race conditions, and assumptions that will cause the number of connections to transiently fall below the specified minimum. Currently, these include:

• At startup: implementations do not try to warm never-used pools to prevent unexpected resource usage.
• When warming fails: implementations do not attempt to recover from connection establishment failures to prevent infinite retry loops.

Parameters:

minConnectionsPerHost - the minimum number of connections a host should maintain.

Returns:

this

maxRandomSubsetSize

default LoadBalancerBuilder<ResolvedAddress,C> maxRandomSubsetSize(int maxUsed)

Set the maximum number of healthy backends to load balance against using a random-subsetting strategy. Note: If a backend within the subset is found to be unhealthy, another endpoint will be added until the unhealthy backend recovers, after which the subset will return to its original state.

Parameters:

maxUsed - the maximum number of healthy backends to use.

Returns:

this

backgroundExecutor

LoadBalancerBuilder<ResolvedAddress,C> backgroundExecutor(Executor backgroundExecutor)

Set the background Executor to use for determining time and scheduling background tasks such as those associated with outlier detection.

Parameters:

backgroundExecutor - Executor to use as a time source and for scheduling background tasks.

Returns:

this.

build

LoadBalancerFactory<ResolvedAddress,C> build()

Builds the LoadBalancerFactory configured by this builder.

Returns:

a new instance of LoadBalancerFactory with settings from this builder.