Async context

ServiceTalk is a fully asynchronous framework and therefore multiple requests may be multiplexed on the same thread. Also depending on the application’s threading model a single request may be processed on different threads. This means that libraries that rely upon ThreadLocal storage such as MDC would not work as expected. To overcome this limitation, we provide an abstraction called AsyncContext which hooks into the internal async machinery to make sure thread local context data is propagated along with the request.

Although AsyncContext may be convenient to use it shouldn’t be over used when traditional argument passing is an option. The intended use case of AsyncContext is to propagate context across method boundaries which do not accommodate for additional parameters. This is typically applicable for infrastructure tasks like tracing, logging, etc.

AsyncContext is designed to provide a static API to retain state associated across asynchronous boundaries. A static API has a few benefits:

Interop with existing APIs depending on ThreadLocal

Some APIs / features assume static state that is coupled with the current thread. This worked well in the 1-request-per-thread model, however it breaks down when we may process a request on different threads due to asynchronous execution and also share threads for processing different requests. For example the OpenTracing APIs and the MDC APIs assume state is stored in some static structure.

API is decoupled from application data APIs

A use case for AsyncContext is to retain distributed tracing state. This state maybe useful across different APIs and maybe burdensome to account for this state in every API (e.g. gRPC and generated code). It also maybe error prone to rely upon users to propagate this state according to control flow which may result inconsistent state. This state is often optional and not always required, and so if it is baked into the protocol APIs this would make them more complex and not be required.

However these benefits don’t come for free. Some of the costs of the static API are as follows:

Complexity of implementation

ServiceTalk is on the hook for making sure the state is saved/restored across asynchronous boundaries. This means that ReactiveStreams operators, Executors, and the offloading strategy must accounted for when relying upon AsyncContext. In order to accomplish this ServiceTalk’s asynchronous control flow needs to account for AsyncContext and provide tools for users if they have existing code outside of ServiceTalk control flow.

Understandability

Although asynchronous function composition is not required to use ServiceTalk but the internals of ServiceTalk uses asynchronous function composition as a core abstraction which can be difficult to reason about relative to traditional imperative programming. On top of this, ServiceTalk provides isolation from the EventLoop threads by offloading to application level threads, which introduces asynchronous boundaries. For request/response protocols the goal is for the AsyncContext to be isolated for each request/response, but folks that need to directly interact with AsyncContext need to understand how modifications are visible (or not) relative to where they are set.

Cost of retention

In order to make state available in a static fashion across API calls the mechanism to do this is to use thread local state. The same thread maybe used to process logically independent asynchronous control flows (e.g. EventLoop can process multiple sockets, and each socket can process multiple requests, and offloading may re-use threads to process different requests). This results in frequently accessing the thread local state to save original value, set correct context, and then restore original value.