Blocking safe by default

This document elaborates how HTTP APIs provide ways to influence offloading. General philosophy of writing blocking code while using ServiceTalk is explained here and is a recommended read.

An optimal execution strategy

As described in Influencing offloading decisions, ServiceTalk tries to calculate an optimal execution strategy based on various inputs. This section describes how this calculation is done for those inputs.

Streaming core

ServiceTalk core is based on the asynchronous streaming programming model and hence exposes different points where user code may interact with the event loop.

(Refer to this document to understand data and control signals terminology).

Client

Considering the request method for StreamingHttpClient

Single<StreamingHttpResponse> request(StreamingHttpRequest request);

we have the following points that require offloads:

  1. Data signals for the response Single.

  2. Data signals for the response payload Publisher.

  3. Control signals for the request payload Publisher.

Server

Considering the handle method for StreamingHttpService

Single<StreamingHttpResponse> handle(HttpServiceContext ctx, StreamingHttpRequest request,
                                     StreamingHttpResponseFactory responseFactory);

we have the following points that require offloads:

  1. Call to handle()

  2. Control signals for the response Single.

  3. Control signals for the response payload Publisher.

  4. Data signals for the request payload Publisher.

Programming models

Programming Paradigms plays an important role in determining an appropriate offloading strategy since they naturally omit some paths that user code can not interact with the event loop. The following matrix elaborates the paths which require offloading with a (✓) and which do not with a (✗).

Client

Programming model response Single (data) response Publisher (data) request Publisher (control)

asynchronous and streaming

asynchronous and aggregated

blocking and streaming

blocking and aggregated

Server

Programming model handle() response Single (control) response Publisher (control) request Publisher (data)

asynchronous and streaming

asynchronous and aggregated

blocking and streaming

blocking and aggregated

User code

Any user code that is added to the processing of client/server is considered suspicious by ServiceTalk and hence any offloading optimizations as described above may get de-optimized. Examples of such code are:

However, these entities do not always contain blocking code. If these entities and the entities they create which are invoked by ServiceTalk do not contain any blocking code, then they can opt-in to the offload optimization by implementing HttpExecutionStrategyInfluencer. If they do, ServiceTalk will include these entities in the process of calculating the optimized offload strategy. ServiceTalk provided filters follow this model themselves, eg: TracingHttpRequesterFilter and RedirectingHttpRequesterFilter.

Evolving to asynchronous

Blocking programming model is easy to start and understand but it may not be the correct tool for all scenarios. In fact, ServiceTalk assumes that applications typically have a mixed profile where different parts follow different Programming Paradigms. One of the goals of ServiceTalk is to help users evolve from a blocking programming model to a completely asynchronous programming model without the need to completely rewrite the entire application. The process for this evolution is elaborated in this document.