Asynchronous Spring Service

It is not unusual that your web service needs to communicate with another web service in order to serve its clients. In the old days, that would imply that an incoming request to your server would capture one servlet connection, and perform a blocking call to the remote service before it can send a response to the client. It works, but it does not scale very well if you have multiple concurrent clients. However, as of Servlet 3.0 we have support for asynchronous servlets (see my colleague Henrik’s blog post), and as of Servlet 3.1 it also supports non-blocking I/O which means that we can get a significantly increased throughput.


In this blog post, we will implement an asynchronous servlet that talks to a remote server using Spring. For demonstration purposes, GitHub’s repository search API will be used as the remote service. The problem is basically divided into three main classes:

  • A GitHubRepoListService has been implemented using the AsyncRestTemplate to communicate with a remote service.
  • The RepositoryListDtoAdapter is responsible for converting the response extracted from the AsyncRestTemplate from Pojos specific to the GitHub response to other Pojos more suitable for the clients of our service. As such, it also acts as an anti-corruption layer between GitHub and the rest of our service implementation.
  • Lastly, there is the AsyncController that is the entry point for our clients.

Of course, the service would not compile without some glue code and a build script. For reference, the project is available on GitHub.

Remote Server Communication

First, we implement a server that uses a AsyncRestTemplate to communicate with GitHub:

class GitHubRepoListService implements RepoListService {
    private static final String SEARCH_URL = "{query}";

    private AsyncRestTemplate asyncRestTemplate;

    public ListenableFuture search(String query) {
        ListenableFuture> gitHubItems = 
            asyncRestTemplate.getForEntity(SEARCH_URL, GitHubItems.class, query);
        return new RepositoryListDtoAdapter(query, gitHubItems);

The response of the AsyncRestTemplate request is a ListenableFuture that eventually will reference a GitHubItems, which is then wrapped in a custom RepositoryListDtoAdapter that converts it to a ListenableFuture<RepoListDto>, see below.

Asynchronous Object Transformation

If you have not used the AsyncRestTemplate or if you are not used to asynchronous Java programming, you may find that transforming one object to another is more complex than one would expect. It may be tempting to just call gitHubItems.get() to fetch the response entity and then continue from there, but remember that we are dealing with a future which means that the get()method is a blocking call that waits for the future to complete before returning. A better solution is to use the ListenableFutureAdapter which can then do the transformation in an asynchronous way:

class RepositoryListDtoAdapter extends ListenableFutureAdapter> {

    private final String query;

    public RepositoryListDtoAdapter(String query, ListenableFuture> gitHubItems) {
        this.query = query;

    protected RepoListDto adapt(ResponseEntity responseEntity) throws ExecutionException {
        GitHubItems gitHubItems = responseEntity.getBody();
        List repoDtos = 
        return new RepoListDto(query, gitHubItems.totalCount(), repoDtos);

    private static Function toRepositoryDto = item -> {
        GitHubOwner owner = item.owner();
        return new RepoDto(item.fullName(), item.getUrl(), item.description(), 
                           owner.userName(), owner.url(), owner.avatarUrl());


The frontend of our service is a controller that maps incoming requests, extracts the query parameter and delegate to the RepoListService to do the remote call as described earlier. We need to convert the ListableFuture to a DeferredResult before returning it to our clients in order to take advantage of Spring’s asynchronous request processing (see also Considerations). For this reason, we create a ListenableFutureCallback that we add to the ListenableFuture:

class AsyncController {
    private static final Logger log = LoggerFactory.getLogger(AsyncController.class);

    private RepoListService repoListService;

    DeferredResult> async(@RequestParam("q") String query) {
        DeferredResult> deferredResult = new DeferredResult<>();
        ListenableFuture repositoryListDto =;
                new ListenableFutureCallback() {
                    public void onSuccess(RepoListDto result) {
                        ResponseEntity responseEntity = 
                            new ResponseEntity<>(result, HttpStatus.OK);

                    public void onFailure(Throwable t) {
                        log.error("Failed to fetch result from remote service", t);
                        ResponseEntity responseEntity = 
                            new ResponseEntity<>(HttpStatus.SERVICE_UNAVAILABLE);
        return deferredResult;

The ListenableFutureCallback interface has two methods that we need to implement. If all goes well, the onSuccess() method is called. Consequently, the response that is transformed by the RepositoryListDtoAdapter is wrapped in a ResponseEntity together with a success status, which in turn is passed to the DeferredResult instance. If something goes wrong, the onFailure() method is called, an empty response entity with a failure status code is created instead.

Glue Code

Since the project is implemented using Spring Boot, we use a simple Application class to configure Spring:

public class Application {

    public static void main(String[] args) {, args);

    AsyncRestTemplate asyncRestTemplate() {
        return new AsyncRestTemplate();

The rest of the code is basically plumbing to handle the serialization and deserialization of requests and responses. The JSON response from GitHub is mapped to GitHubItems by the AsyncRestTemplate:

@JsonIgnoreProperties(ignoreUnknown = true)
class GitHubItems {

    private int totalCount;

    private List items;

    int totalCount() {
        return totalCount;

    List items() {
        return items;

Where GitHubItem is implemented like:

@JsonIgnoreProperties(ignoreUnknown = true)
class GitHubItem {

    private String fullName;

    private URL url;

    private String description;

    private GitHubOwner owner;

    String fullName() {
        return fullName;

    URL getUrl() {
        return url;

    String description() {
        return description;

    GitHubOwner owner() {
        return owner;

and the GitHubOwner is implemented as:

@JsonIgnoreProperties(ignoreUnknown = true)
class GitHubOwner {

    private String userName;

    private URL url;

    private URL avatarUrl;

    String userName() {
        return userName;

    URL url() {
        return url;

    URL avatarUrl() {
        return avatarUrl;

The response that is sent to the client consists of two Pojos. At the top level, there is a RepoListDto that contains information about the client’s search query, the total number of repositories at GitHub that matched the query and a list of RepoDtos that represents each repository:

public class RepoListDto {

    private final String query;

    private final int nbrOfRepositories;

    private final  List repositories;

    public RepoListDto(String query, int nbrOfRepositories, List repositories) {
        this.query = query;
        this.nbrOfRepositories = nbrOfRepositories;
        this.repositories = repositories;

Each individual GitHub repository is presented as a RepoDto:

public class RepoDto {

    private final String name;
    private final URL url;
    private final String description;

    private final String owner;
    private final URL ownerUrl;
    private final URL ownerAvatar;

    public RepoDto(String name, URL url, String description, 
                   String owner, URL ownerUrl, URL ownerAvatar) { = name;
        this.url = url;
        this.description = description;
        this.owner = owner;
        this.ownerUrl = ownerUrl;
        this.ownerAvatar = ownerAvatar;

Build script

The project was built using Maven and the following pom.xml:



    Demo project of how an asynchronous server can use an AsyncRestTemplate






  • In this blog, I have showed how the AsyncRestTemplate to communicate with an external restful service. However, before using it in production I suggest that you configure the internals of the AsyncRestTemplate such as connection timeout, read timeout, max number of connections, max connections per route etc. The Apache HttpAsyncClient examples, this blog post and this question on Stack Overflow all provide good advice.
  • The asynchronous service will tolerate a much higher load than a synchronous solution would. However, from a client perspective the latency may be an issue since it is first calling your service, which in turn has to call the remote service and wait for its response, before a new response can be created and returned. To mitigate this problem, different cache solutions come to mind. Consider caching the responses from the remote services, caching the Pojos generated by our service and / or using HTTP headers to implement appropriate caching of the responses generated by our service in the HTTP layer.
  • Our asynchronous service is well prepared to handle big load, but is the remote service capable of handling the load you are delegating to it? The suggested cache solution may work if the remote service returns generic answers, but if the remote service returns unique responses for similar requests the cache solution will not do. And what happens if the remote service goes down? One way of handling such requirements, and to protect your server from cascading failures, is to wrap the remote call in a circuit breaker.
  • If you can migrate to Spring 4.1 that was released last week you do no longer need to convert your result to DeferredResult. Simply return the ListenableFuture from your controller (see the list of improvements).


The response as returned from GitHub directly:

$ curl | jq .
  "total_count": 883,
  "incomplete_results": false,
  "items": [
      "id": 6296790,
      "name": "spring boot",
      "full_name": "spring-projects/spring-boot",
      "owner": {
        "login": "spring-projects",
        "id": 317776,
        "avatar_url": "",
        "gravatar_id": "6f8a529bd100f4272a9ff1b8cdfbd26e",
        "url": "",
        "html_url": "",
        "followers_url": "",
        "following_url": "{/other_user}",
        "gists_url": "{/gist_id}",
        "starred_url": "{/owner}{/repo}",
        "subscriptions_url": "",
        "organizations_url": "",
        "repos_url": "",
        "events_url": "{/privacy}",
        "received_events_url": "",
        "type": "Organization",
        "site_admin": false
      "private": false,
      "html_url": "",
      "description": "Spring Boot",
      "fork": false,
      "url": "",
      "forks_url": "",
      "keys_url": "{/key_id}",
      "collaborators_url": "{/collaborator}",
      "teams_url": "",
      "hooks_url": "",
      "issue_events_url": "{/number}",
      "events_url": "",
      "assignees_url": "{/user}",
      "branches_url": "{/branch}",
      "tags_url": "",
      "blobs_url": "{/sha}",
      "git_tags_url": "{/sha}",
      "git_refs_url": "{/sha}",
      "trees_url": "{/sha}",
      "statuses_url": "{sha}",
      "languages_url": "",
      "stargazers_url": "",
      "contributors_url": "",
      "subscribers_url": "",
      "subscription_url": "",
      "commits_url": "{/sha}",
      "git_commits_url": "{/sha}",
      "comments_url": "{/number}",
      "issue_comment_url": "{number}",
      "contents_url": "{+path}",
      "compare_url": "{base}...{head}",
      "merges_url": "",
      "archive_url": "{archive_format}{/ref}",
      "downloads_url": "",
      "issues_url": "{/number}",
      "pulls_url": "{/number}",
      "milestones_url": "{/number}",
      "notifications_url": "{?since,all,participating}",
      "labels_url": "{/name}",
      "releases_url": "{/id}",
      "created_at": "2012-10-19T15:02:57Z",
      "updated_at": "2014-09-05T09:37:27Z",
      "pushed_at": "2014-09-05T02:22:04Z",
      "git_url": "git://",
      "ssh_url": "",
      "clone_url": "",
      "svn_url": "",
      "homepage": "",
      "size": 46263,
      "stargazers_count": 1028,
      "watchers_count": 1028,
      "language": "Java",
      "has_issues": true,
      "has_downloads": true,
      "has_wiki": true,
      "forks_count": 849,
      "mirror_url": null,
      "open_issues_count": 173,
      "forks": 849,
      "open_issues": 173,
      "watchers": 1028,
      "default_branch": "master",
      "score": 78.31211
    // more repositories...

The response that is returned by our service:

curl http://localhost:8080/async?q=spring+boot | jq .
  "query": "spring boot",
  "nbr_of_repositories": 860,
  "repositories": [
      "name": "spring-projects/spring-boot",
      "url": "",
      "description": "Spring Boot",
      "owner": "spring-projects",
      "owner_url": "",
      "owner_avatar": ""
    // more repositories...

Side note, jq is a convenient tool for JSON processing that you can install.


Mattias Severson

Mattias is a senior software engineer specialized in backend architecture and development with experience of cloud based applications and scalable solutions. He is a clean code proponent who appreciates Agile methodologies and pragmatic Test Driven Development. Mattias has experience from many different environments, including everything between big international projects that last for years and solo, single day jobs. He is open-minded and curious about new technologies. Mattias believes in continuous improvement on a personal level as well as in the projects that he is working on. Additionally, Mattias is a frequent speaker at user groups, companies and conferences.

This Post Has 16 Comments

  1. Haytham

    Mattias, thank you for an interesting blog. In the Considerations section you say “However, from a client perspective the latency may be an issue since it is first calling your service, which in turn has to call the remote service and wait for its response, before a new response can be created and returned. ”

    You really lost me there! I thought the whole purpose of the AsyncService was to avoid that. To my understanding the client who called the server via the controller would receive the DeferredResult and call getResult on it and only then block until the result is available.

    1. Mattias Severson

      @Haytham: Sorry for being unclear, let me elaborate. First of all, in this blog by “client” I mean “end user” or “consumer of the service”.
      Now, consider a simplified example where a request from the client to the service takes 200ms and the request from the service to the remote service takes another 100ms during normal operation. If the service only receives a single request it does not matter if the service is single threaded and synchronous or multithreaded and asynchronous, the total response time for the client will be 300ms regardless of the service implementation. However, if we make 100 requests concurrently, the response time of all requests to the single threaded, synchronous service will be 10.2 seconds (200ms + 100 * 100ms) whereas for the asynchronous service it will still be 300ms. Consequently, the asynchronous implementation matters if your service has a high load.
      Secondly, consider another example where the request from the client to the service takes 200ms, but this time the request from the service to the remote service takes 5 seconds. If a single request is being sent, the response time from the clients perspective is 5.2 seconds and the following request will also take 5.2 seconds. However, if the response to the second request can be generated from a cached value rather than from the remote service the response time for that (and any subsequent request with the same cache key) will only be 200ms since the call to the remote service is avoided. In other words, both the synchronous and the asynchronous service may get significantly decreased response times if cache is used. As a side effect, a cache will also decrease the load on the remote service.
      These are just two staged and very simplified examples, but I hope they provide some more understanding to the problem.

  2. Haytham

    I think the link you provided to one of your Colleague’s blogs answers my question:

    Returning a DeferredResult signals to Spring that the request should be treated asynchronously. After invoking setResult the response will be sent back to the client.

  3. Bob

    From a testing perspective, does this work with MockMvc?

    1. Mattias Severson

      @Bob: Yes, MockMvc is supported, however you need to write some extra lines compared to a synchronous flow in order for it work. First, you start the async processing by calling the mockMvc.perform(...) as you would do normally, but make sure to keep a reference to the returned MvcResult. Next, you make an async dispatch by re-using the MvcResult. The following lines are copied from the JavaDoc of the MockMvcRequestBuilders.asyncDispatch(MvcResult mvcResult) method:

       MvcResult mvcResult = this.mockMvc.perform(get("/1"))
  4. Gustavo Orsi

    Also, since 4.1 you can use lambdas to set the callback methods (success and failure).

    repositoryListDto.addCallback(r -> deferredResult.setResult(new ResponseEntity(r, HttpStatus.OK)), t -> deferredResult.setResult(new ResponseEntity(HttpStatus.SERVICE_UNAVAILABLE)));

  5. Carl Moser

    You don’t really need the complex controller code to map the ListenableFuture into the DeferredResult manually. Why do you don’t return the ListenableFuture directly? Spring supports this as return type and is doing the rest for you, see The controller has nothing more to do than pass it through. That’s make the code easier to read and keep it clean. The result is exactly the same.

    1. The Response type of the server and controller method has to be the same. Instead of returning a ResponseEntity of RepoListDto you could simply return a ListenableFuture of RepoListDto.

    2. The exception handling can be implemented in a @ExceptionHandler annotated method.


    ListenableFuture async(@RequestParam(“q”) String query) {

    public ResponseEntity handleException(Exception e) {
    log.error(“Failed to fetch result from remote service”, e);
    return new ResponseEntity(HttpStatus.SERVICE_UNAVAILABLE);

    1. Mattias Severson

      @Carl: Thank you for your comment. You are quite right, the implementation can be simplified in the ways that you suggest. The reason why I chose the implementation that I did was because I used Spring Boot version 1.1.6, i.e. Spring Framework version 4.0.7. Returning ListanableFuture from a controller or other service was not supported until Spring 4.1. I guess that I could have been more clear about which version was used in the blog post, but you can see the versions used in the associated project at GitHub. Moreover, I mentioned the possibility in one of considerations above as a suggestion when migrating to Spring 4.1.

  6. Sting

    Does it work if the restTemplate’s getForEntity timeout?

  7. qeshine

    hi,dose spring support Servlet3.1 Non-blocking IO ,do you have an example?thanks

    1. Mattias Severson

      @qeshine: Spring supports Servlet 3.1 as exemplified in this blog post, provided that you deploy your application in a servlet container that also supports Servlet 3.1.

    1. Mattias Severson

      @Juan: No, I am sorry. The last time I was working with SOAP was long before async came into fashion.

Leave a Reply