AWS Cloudformation is a super-powerful utility for defining entire backend infrastructures. Using it we can declaratively define infrastructure components and their inter-dependencies, then instantiate them all in one go. This provides us with repeatability for deployment of information services that is one step beyond what we as software engineers have become accustomed to.

The declaration of resources when using Cloudformation is done as a json document called a template. When working with templates we often encounter a situation where some resource we want to define depends on a VPC, Amazons Virtual Private Cloud component. In many cases this can be implicit, and in these cases the dependency will rest on the default VPC that comes preconfigured with the AWS account.

In the cases where the dependency is not implicit, the template we use will either have to contain the VPC reference “hard coded” or take it via an input parameter. This behaviour breaks a little bit of the generalness of using Cloudformation, because it requires a pre-existing VPC, the state of which is not configured from within the Cloudformation declaration. Pragmatic modus operandum is to never, ever touch your default VPC and all will be well.

There is another way, which is to bite the bullet and define our own VPC to closely resemble the default VPC. Including this custom VPC as part of any Cloudformation deployment would grant us a few nice things; Instantiating the infrastructure would require one less input parameter and thus ease deployment, it could be trivial to replicate across separate AWS accounts, guarantee of repeatability would increase, and customizability would improve. Clearly then, this is what we want, so let’s go ahead and declare a default VPC-like custom VPC:

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Description": "Default VPC replacement",
  "Resources": {
    "InfraVPC": {
      "Type" : "AWS::EC2::VPC",
      "Properties" : {
        "CidrBlock": "10.0.0.0/16",
        "EnableDnsHostnames": "true",
        "Tags": [{"Key": "Name", "Value": {"Fn::Join": ["",[{"Ref": "AWS::StackName"}, " resource"]]}}]
      }
    },
    "InfraInetGW": {
      "Type" : "AWS::EC2::InternetGateway",
      "Properties": {
        "Tags": [{"Key": "Name", "Value": {"Fn::Join": ["",[{"Ref": "AWS::StackName"}, " resource"]]}}]
      }
    },
    "InfraVPCGWAttachment": {
      "Type" : "AWS::EC2::VPCGatewayAttachment",
      "Properties" : {
        "InternetGatewayId": {"Ref": "InfraInetGW"},
        "VpcId": {"Ref": "InfraVPC"}
      }
    },
    "InfraRouteTable": {
      "Type" : "AWS::EC2::RouteTable",
      "Properties" : {
        "VpcId": {"Ref": "InfraVPC"},
        "Tags": [{"Key": "Name", "Value": {"Fn::Join": ["",[{"Ref": "AWS::StackName"}, " resource"]]}}]
      }
    },
    "InfraInternetRoute": {
      "Type" : "AWS::EC2::Route",
      "Properties" : {
        "DestinationCidrBlock": "0.0.0.0/0",
        "GatewayId": {"Ref": "InfraInetGW"},
        "RouteTableId": {"Ref": "InfraRouteTable"}
      }
    },
    "InfraSubnetA": {
      "Type": "AWS::EC2::Subnet",
      "Properties" : {
        "AvailabilityZone": "eu-west-1a",
        "CidrBlock": "10.0.0.0/20",
        "MapPublicIpOnLaunch": "true",
        "VpcId": {"Ref": "InfraVPC"},
        "Tags": [{"Key": "Name", "Value": {"Fn::Join": ["",[{"Ref": "AWS::StackName"}, " resource"]]}}]
      }
    },
    "InfraSubnetB": {
      "Type": "AWS::EC2::Subnet",
      "Properties" : {
        "AvailabilityZone": "eu-west-1b",
        "CidrBlock": "10.0.16.0/20",
        "MapPublicIpOnLaunch": "true",
        "VpcId": {"Ref": "InfraVPC"},
        "Tags": [{"Key": "Name", "Value": {"Fn::Join": ["",[{"Ref": "AWS::StackName"}, " resource"]]}}]
      }
    },
    "InfraSubnetC": {
      "Type": "AWS::EC2::Subnet",
      "Properties" : {
        "AvailabilityZone": "eu-west-1c",
        "CidrBlock": "10.0.32.0/20",
        "MapPublicIpOnLaunch": "true",
        "VpcId": {"Ref": "InfraVPC"},
        "Tags": [{"Key": "Name", "Value": {"Fn::Join": ["",[{"Ref": "AWS::StackName"}, " resource"]]}}]
      }
    },
    "InfraSubnetARouteTableAssociation" : {
      "Type" : "AWS::EC2::SubnetRouteTableAssociation",
      "Properties" : {
        "SubnetId" : { "Ref" : "InfraSubnetA" },
        "RouteTableId" : { "Ref" : "InfraRouteTable" }
      }
    },
    "InfraSubnetBRouteTableAssociation" : {
      "Type" : "AWS::EC2::SubnetRouteTableAssociation",
      "Properties" : {
        "SubnetId" : { "Ref" : "InfraSubnetB" },
        "RouteTableId" : { "Ref" : "InfraRouteTable" }
      }
    },
    "InfraSubnetCRouteTableAssociation" : {
      "Type" : "AWS::EC2::SubnetRouteTableAssociation",
      "Properties" : {
        "SubnetId" : { "Ref" : "InfraSubnetC" },
        "RouteTableId" : { "Ref" : "InfraRouteTable" }
      }
    }
  },
  "Outputs": {
    "VpcID": {
      "Description": "The ID of the created VPC",
      "Value": {"Ref": "InfraVPC"}
    }
  }
}

The above template defines an entire VPC and aims to resemble the default VPC as closely as possible. The anatomy of it is outside the scope of this post, but it has been tested in real-world situations and we’ve found it to be well functioning. A couple of points of interest:

1. Tagging the individual resources of the VPC with dynamically generated names gives us ease of differentiating the origin of AWS components.
2. Outputting the VPC ID is a nice way of reporting the effect of instantiating the template.
3. Quirk alert! Running this will create two route tables and the unused one of these two will be marked as “Main”. As far as I can tell from the AWS documentation this is not a problem but rather by design from Amazon. It sure looks funny though.

We can test this setup by instantiating it through the AWS CLI. Saving the template above to a file we’ll call vpc.tamplate we can do it like this:

[me@laptop]$ aws cloudformation create-stack --stack-name myvpc --template-body file://vpc.template

We can query the stack outputs to get the VPC ID like this:

[me@laptop]$ aws cloudformation describe-stacks --stack-name myvpc
{
    "Stacks": [
        {
            "StackId": "arn:aws:cloudformation:eu-west-1:123456789012:stack/myvpc/12345678-1234-1234-1234-123456789012",
            "Tags": [],
            "CreationTime": "2016-03-15T10:27:46.984Z",
            "Outputs": [
                {
                    "OutputKey": "VpcID",
                    "Description": "The ID of the created VPC",
                    "OutputValue": "vpc-12345678"
                }
            ],
            "NotificationARNs": [],
            "DisableRollback": false,
            "StackName": "myvpc",
            "Description": "Default VPC replacement",
            "StackStatus": "CREATE_COMPLETE"
        }
    ]
}

And once we’re done we can destroy it like this:

[me@laptop]$ aws cloudformation delete-stack --stack-name myvpc

Final tip: For easy usage of the AWS CLI you might want to try out awscli.sh, which is a containerised way of working. See previous blog post and corresponding GitHub repo for details.