Deploying Go applications to Azure Websites

Microsoft recently released HttpPlatformHandler for IIS. It can, for instance, be used to host Ruby on Rails applications on your IIS host. Azure Websites on the other hand has had this feature for quite a while now, e.g. to host customized JVM applications. Through HttpPlatformHandler, Azure Websites can host any HTTP application, including Go.

Hosting Go applications on Azure Websites has been previously documented. However, as Go binaries are statically linked and executed by the IIS through HttpPlatformHandler they are also locked for the duration they are running. Deploying newer versions of an application consequently involve first shutting down the website. This issue is not isolated to Go as discussed in issues 914 and 1122 of the Kudu project. Although, as Go binaries are statically linked it is to some extent easier to handle as we will see below.

Let’s see if we can automate deployment of Go applications to Azure Websites.

In-place deployment of Go applications

Blog posts here and here, describe how to transparently swap two binaries. Both rely on same approach, i.e.:

  1. Spawning a child process and passing the file descriptor for the HTTP-listening TCP socket from parent to child
  2. Closing the listener on parent and let the child listen for traffic using the passed file descriptor
  3. Shutting down parent when its connections are closed

The latter portion of step 2 requires creating a net.FileListener which, as of Go 1.4.1, is unsupported on Windows.

However, as Azure Websites will through HttpPlattformHandler automatically invoke the application on incoming requests we only need to concern ourselves with ensuring a graceful shutdown of the running application and let HttpPlatformHandler invoke the latest application binary on incoming requests. While this approach will cause new connections to be denied for a short duration — between closing of the socket and shutting down the running process — it is less obtrusive than having to manually shutting down and restarting the website.

We can achieve an automated in-place deployment of Go applications by:

  1. Closing the TCP socket on the running process
  2. Shutdown the process once all its connections are closed
  3. Configure HttpPlatformHandler to invoke and proxy new HTTP requests to the new binary once shutdown is completed

Closing the TCP socket

In Go, http.Serve is used to serve HTTP over a net.Listenerhttp.Serve will then block until the Accept method on the net.Listener receiver returns an error, e.g. when the underlying file descriptor is closed by calling Close. It follows that we can shutdown the HTTP server and then the process if we first close the listener.

Through embedding we can by defining a new type stoppableListener to extend the behavior of net.Listener.

Following this, we now have a method on a stoppableListener which will close the underlying net.Listener once the receive operation on the initShutdown-channel unblocks.

As stoppableListener, through embedding net.Listener, still fulfills the net.Listenerinterface we can pass an instance of stoppableListener to http.Serve and determine when to shutdown the HTTP server given send-access on the initShutdown-channel.

Shutting down the process

When control returns from http.Serve, following the closing of net.Listener, we can shut down the process when remaining connections are closed.

As the defined type stoppableListener embeds net.Listener we can also modify the behavior of Accept to return another type that embeds net.Conn and through it determine when all connections returned by Accept have been closed. This detailed in the blogs posts referred to above.

Triggering an deployment

Kudu is the deployment engine behind Azure Websites and supports custom deployment scripts. As Kudu runs in a separate process tree and Azure Websites will isolate processes from one another, Kudu scripts are unable to directly initiate a shutdown of a running application, e.g. by sending SIGTERM to the running process. However, we can use the filesystem as a layer of indirection as it is shared by processes running on the same site and account.

We create a file watcher which will close the initShutdown-channel when a new file is found in the monitored directory referred to by the config.watchDir variable below. Closing the channel will unblock the receiver in the listing above and initiate shutdown by closing the listener.

Configuring HttpPlatformHandler to invoke the latest build

Any new requests to the site following shutdown completion will cause HttpPlatformHandler to execute the binary or script file identified by the value of the processPath attribute.

Provided it is possible to uniquely identify the latest build we can write a batch script to have HttpPlatformHandler execute the latest build on each invocation, e.g.

if file _artifact.txt contains the path to the latest build.

(Unlike Kudu deployment scripts, bash scripts, unfortunately, does not seem to be an option for HttpPlatformHandler.)

Given the name of the script above is go-azure.bat, HttpPlatformHandler can be configured as follows, where the second argument is the directory being monitored for new executables:

Automating deployment with Kudu script

As discussed above, we are able to trigger a shutdown of the application given a new executable is put into the monitored directory. Given the path to the new executable known, e.g. found in the contents of a file, we can let HttpPlatformHandler invoke it through a script.

The remaining steps, then, are to ensure the newly compiled application is given an unique name, stored within the monitored directory, and its path stored in a file.

We can generated an unique name using variables from the Azure Website and Kudu deploymentenvironments, more specifically WEBSITE_SITE_NAME and SCM_COMMIT_ID from respective environment.

Including the listing above in a (bash) Kudu deployment script will install Go when necessary and build a statically linked binary named after the website and the (short) commit hash from which the binary is built. For instance, deploying from commit with hash6417057c21bf311adcb81fdb5ff78bf3b4908e71 to an Azure Website named go-azure will create artifact go-azure-6417057c21.exe.

Conclusion

In this post we show how to use HttpPlatformHandler to automate deployment of Go applications to Azure Websites through Kudu deployment scripts. When the site configured to deploy from a code repository, e.g. from GitHub, the scripts will automatically initiate a build and deploy the resulting binary.

While it is difficult to achieve a (almost) seamless transition between two Go binaries it is possible to minimize the window in which new requests are denied by automatically triggering a shutdown when a new binary is found and let HttpPlatformHandler invoke the new binary once shutdown is completed.

A sample project is available on GitHub.

This Post Has 2 Comments

  1. Great post Hang! Have you managed to get Martini running on Azure Websites? Can’t seem to get static files to load properly on the root.

    1. I haven’t used Martini but serving static content, e.g. through http.FileServer, works fine. If you’re using the deployment scripts from the post, make sure the path to static content is relative to the binary, i.e. relative to the _target directory.

      Specifying an absolute path, e.g. by adding an environment variable to the Azure Website, would work as well, I assume.

Leave a Reply

Close Menu