Blocking safe by default

By default, users are free to write blocking code when using ServiceTalk.

Background

ServiceTalk uses Netty under the hood which uses the Event loop execution model. A drawback of this construct is that any potentially long-running work (blocking calls, intensive computations, etc) that is done on the event loop negatively impacts the processing of all subsequent tasks. This is the reason why writing blocking code is usually discouraged when using Netty.

Reality

Writing non-blocking code is hard and requires continuous, significant effort to make sure that no code path interacting with the event loop is invoking any blocking code. In practice, even the most careful application owners are occasionally surprised that they have some blocking code in their applications.

ServiceTalk approach

We want ServiceTalk to be an approachable library for all developers. Providing a simple message that "blocking is safe by default" lowers the bar to entry and eases the operational burden of continuously making sure that no blocking code exists in any path. There are no exceptions to this rule for ServiceTalk users, although the bar is higher for ServiceTalk developers.

Completely non-blocking applications?

Although writing completely non-blocking applications is hard in practice, still there are reasons and use cases to reduce thread hops and do all application processing on the event loop. ServiceTalk makes it possible to write such applications but requires explicit opt-ins to avoid inadvertently blocking the event loop.

How is it safe to block?

Simply put, ServiceTalk makes sure that no user code is called on the event loop. In order to do that, it uses an Executor to offload invocation of user code out of the event loop to a different thread.

User code and event loop

Although the outcome is simple that no user code should run on the event loop, implementations of this offloading are often either naive hence sub-optimal or fairly complex. ServiceTalk internally does the heavy lifting to make sure that it does not offload more than what is required. In other words, it reduces offloading when it can determine that no user code can interact with the event loop on a certain path.

Execution Strategy

The primary purpose of an Execution Strategy is to define which interaction paths of a particular transport or protocol layer require offloading. For example, at the HTTP transport layer, four offload paths are available:

  1. Sending data to the transport.

  2. Receiving data from the transport.

  3. Handling transport events.

  4. Closing the transport.

A given application, filter or component may indicate that it requires offloading for none, some or all of these interactions. Protocols other than HTTP will have their own APIs that would otherwise execute user code on an EventLoop thread and define their own ExecutionStrategy to control which of those interaction paths APIs are subject to offloading.

Influencing offloading decisions

ServiceTalk will determine the optimal offloading strategy for handling requests and responses. This optimal strategy is determined based on different inputs as outlined below:

  1. Programming Paradigms.: A certain programming model may reduce the paths user code can interact with the event loop and hence make some offloading redundant.

  2. Presence of user code: ServiceTalk is flexible and allows users to add code at different places. Presence of any user code is deemed as unsafe and hence requires offloading. If users are sure the added code does not block, they can declare so through the constructs provided at the protocol level.

  3. Override execution strategy: ServiceTalk provides a way for users to override the Execution Strategy used for a client/server. This disables all intelligence around finding the optimized strategy and just uses the provided strategy.

Opt-in to run on event loop

ServiceTalk acknowledges that writing completely asynchronous code is hard and is not an "all or nothing" decision. Practically, applications will have a mixed profile such that some paths are completely asynchronous, and some paths have blocking code. This is the reason it provides ways to selectively opt-in for disabling offloading for the same client/server. These selective opt-ins are specific to protocols, for example as elaborated under HTTP. In addition, there is a way to override the strategy for a client/server, but it comes with additional responsibility from the user assuming "you know what you are doing".

ServiceTalk developers

The internal implementation of ServiceTalk generally discourages blocking code and hence introducing blocking should be avoided while contributing to ServiceTalk. Unless there is a valid reason to do it, ServiceTalk internals are always non-blocking.