ServiceTalk is a fully asynchronous framework and therefore multiple requests may be multiplexed on the same thread.
Also depending on the application’s threading model a single request may be processed on different threads. This means
that libraries that rely upon ThreadLocal storage such as
MDC would not work as
expected. To overcome this limitation, we provide an abstraction called
AsyncContext which hooks into the internal
async machinery to make sure thread local context data is propagated along with the request.
AsyncContext may be convenient to use it shouldn’t be over used when traditional argument passing is an
option. The intended use case of
AsyncContext is to propagate context across method boundaries which do not
accommodate for additional parameters. This is typically applicable for infrastructure tasks like tracing,
AsyncContext is designed to provide a static API to retain state associated across asynchronous boundaries. A static
API has a few benefits:
Interop with existing APIs depending on
Some APIs / features assume static state that is coupled with the current thread. This worked well in the 1-request-per-thread model, however it breaks down when we may process a request on different threads due to asynchronous execution and also share threads for processing different requests. For example the OpenTracing APIs and the MDC APIs assume state is stored in some static structure.
API is decoupled from application data APIs
A use case for
AsyncContext is to retain distributed tracing state. This state maybe useful across different APIs
and maybe burdensome to account for this state in every API (e.g. gRPC and generated code). It also maybe error prone to
rely upon users to propagate this state according to control flow which may result inconsistent state. This state is
often optional and not always required, and so if it is baked into the protocol APIs this would make them more complex
and not be required.
|This maybe seen as a negative in some respects. On the contrary, this does not restrict users from explicitly passing state through their APIs.|
However these benefits don’t come for free. Some of the costs of the static API are as follows:
Complexity of implementation
ServiceTalk is on the hook for making sure the state is saved/restored across asynchronous boundaries. This means
that ReactiveStreams operators, Executors, and the offloading strategy must accounted for when relying upon
AsyncContext. In order to accomplish this ServiceTalk’s asynchronous control flow needs to account for
and provide tools for users if they have existing code outside of ServiceTalk control flow.
Although asynchronous function composition is not required to use ServiceTalk but the internals of ServiceTalk uses
asynchronous function composition as a core abstraction which can be difficult to reason about relative to
traditional imperative programming. On top of this, ServiceTalk provides isolation from the EventLoop
threads by offloading to application level threads, which introduces asynchronous boundaries. For request/response
protocols the goal is for the
AsyncContext to be isolated for each request/response, but folks that need to directly
AsyncContext need to understand how modifications are visible (or not) relative to where they are set.
Cost of retention
In order to make state available in a static fashion across API calls the mechanism to do this is to use thread local state. The same thread maybe used to process logically independent asynchronous control flows (e.g. EventLoop can process multiple sockets, and each socket can process multiple requests, and offloading may re-use threads to process different requests). This results in frequently accessing the thread local state to save original value, set correct context, and then restore original value.