Evolving to asynchronous

This document elaborates how to disable offloading, which is considered an advanced feature.

ServiceTalk provides ways for users start with a simpler blocking programming model and slowly evolve to a completely asynchronous programming model. Motivation and general principles can be found here. This document contains details how to disable offloading for an HTTP client/server.

Client

For a client, there are two distinct ways of turning off offloading:

  1. Disable offloading per request.

  2. Disable offloading for the entire client.

For a client which is used for blocking as well as asynchronous programming models requests, (1) is the suggested mode of operation.

No offloads per request

HTTP clients for all programming models provide methods to make requests with or without HttpExecutionStrategy.

See HttpClient.

Single<HttpResponse> request(HttpRequest request);

Single<HttpResponse> request(HttpExecutionStrategy strategy, HttpRequest request);

In order to disable offloading for a specific request one has to use a specific HttpExecutionStrategy that does not do any offloading. Such a strategy is available out of the box in HttpExecutionStrategies and can be used as below:

// Processing of this request does not have any blocking code
httpClient.request(HttpExecutionStrategies.noOffloadsStrategy(), aRequest);

For a request made using this strategy, ServiceTalk will not perform any offloading and all user calls may be invoked on the EventLoop.

It is assumed that no blocking code exists during the entire processing of this request. If this assumption is violated then it will negatively impact the responsiveness of other work done on that EventLoop.

No offloads per client

If it is certain that no blocking requests will be made using a client without explicitly overriding the strategy, then one can disable offloading for the entire client using the following option on the client builder.

See HttpClientBuilder.

clientBuilder.executionStrategy(HttpExecutionStrategies.noOffloadsStrategy());

This strategy will be used for all requests that do not explicitly specify a strategy.

Beware of inadvertently adding a request which uses blocking code to a client that has offloading disabled.

Server

ServiceTalk is designed in a layered fashion to enable extensibility. Therefore the core is unaware of a specific routing technology (e.g. JAX-RS) and assumes any routing is implemented inside the service provided to the server. This enables us to provide vastly different routing solutions without changing the core of ServiceTalk. However, it does mean that ServiceTalk only knows the programming model of the service that implements routing and not the programming model of the routes for each request. Since, ServiceTalk has to select an execution strategy before invoking the service (router) this means that an individual route can not entirely control all offloading done by the server. Hence, all routers in ServiceTalk have two execution strategies in play for any request processing:

  1. Strategy for the server: All code that constitutes request processing in a server apart from the actual route.

  2. Strategy for a route: Route implementation for a particular request.

Server Strategy

Execution strategy for a server can be specified on the server builder just like the client. In order to disable offloading for the entire server, the following option can be used:

See HttpServerBuilder.

serverBuilder.executionStrategy(HttpExecutionStrategies.noOffloadsStrategy());

This strategy will be used for all requests processed by this server.

Beware of inadvertently adding blocking code to a server that has offloading disabled.

Route strategy

Each route added to any router may define an independent execution strategy which is used to invoke that route. Different routers may provide different ways of specifying that strategy, however, conceptually there are two ways to specify such a strategy:

Implicit route strategy

A router may have the ability to infer an appropriate execution strategy for a route from the programming model the route uses, just like ServiceTalk infers execution strategy per programming model. This auto-inference is only used if no strategy is explicitly specified for that route.

Explicit route strategy

A router may also provide a capability for a route to explicitly define an execution strategy that is used to invoke that route. In presence of such an explicit strategy, auto-inference of the route strategy is disabled. In order to disable offloads, one should use HttpExecutionStrategies.noOffloadsStrategy()

In order to effectively evolve to different programming models, one should ensure that each route defines its own strategy either implicitly (when supported by the router) or explicitly (for overrides or lack of implicit strategy capabilities in the router). Depending on server strategy for a route is dangerous because:

  1. Server strategy may change, adversely impacting the route. If the server strategy does change, and the router doesn’t auto-infer the correct default strategy, then the effective strategy at each route maybe impacted.

  2. Increasing scope of server strategy (to also be the default strategy for all routes) makes it hard to understand how changing that strategy may impact all routes. Typically, server strategy should be chosen based on all code that is executed before executing the actual route for a request.

Making sure that all routes can independently specify their strategy makes it easier to reason about the impacts of changing these strategies. ServiceTalk provided routers provide ways to explicitly specify a route strategy, implicit inference capabilities may vary. Check out Jersey Router to see its capabilities for per route strategies.