Dutch governments need to be available over IPv6 according to this mandate. Therefore I had to make sure my Application Gateway with Web Application Firewall (WAF) is also available over IPv6. In this blog I will explain some of the challenges we faced and how we solved them.

System context

In the existing system we have multiple apps running on Azure App Services behind an Application Gateway with WAF. The Application Gateway is currently only available over IPv4.

We have a listener and rule in place for each app. Each app is configured with its own FQDN (Fully Qualified Domain Name) and a TLS/SSL certificate is used for secure communication.

The system was geo redundant with two Application Gateways in different regions, each with its own public IP address.

A traffic manager profile is used to distribute traffic between the two Application Gateways based on priority. A common active-passive setup.

Architecture v1 - Application gateway behind an Traffic Manager
Architecture v1 - Application gateway behind an Traffic Manager

Dual stack

To make the Application Gateway available over IPv6, we needed to configure dual stack support. This means you have to redeploy the Application Gateway. This has a massive impact because of our current strategy with multiple listeners and rules. We had to make sure that all configurations were preserved during the redeployment.

If you need to do this I recommend:

  1. Add Ipv6 address prefixes to your virtual network.
  2. Use an AVM module to build your new Application Gateway with dual stack support.
  3. Reconsider if you need zones for your Application Gateway. Those also require a full redeployment.
application-gateway-ipv6.bicep
  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129

param region string
param environment string
param applicationGatewayNr int

module vnet 'br/public:avm/res/network/virtual-network:0.7.1' = {
  params: {
    name: 'ag-vnet'
    addressPrefixes: [
      '10.0.0.0/16' // IPv4 address prefix
      'fd00::/48' // IPv6 address prefix
    ]
    subnets: [
      {
        name: 'appgw-subnet'
        addressPrefixes: [
          '10.0.1.0/24' // IPv4 subnet for Application Gateway
          'fd00:0:0:1::/64' // IPv6 subnet for Application Gateway
        ]
        delegation: 'Microsoft.Network/applicationGateways'
      }
    ]
  }
}

module publicIpV4 'br/public:avm/res/network/public-ip-address:0.9.0' = {
  params: {
    // Required parameters
    name: 'pip-v4-mart'
    // Non-required parameters
    availabilityZones: [1, 2, 3]
    publicIPAddressVersion: 'IPv4'
    publicIPAllocationMethod: 'Static'
  }
}

module publicIpV6 'br/public:avm/res/network/public-ip-address:0.9.0' = {
  params: {
    // Required parameters
    name: 'pip-v6-mart'
    // Non-required parameters
    availabilityZones: [1, 2, 3]
    publicIPAddressVersion: 'IPv6'
    publicIPAllocationMethod: 'Static'
  }
}
var agName = 'ag-${applicationGatewayNr}-${region}-${environment}'
module appGateway 'br/public:avm/res/network/application-gateway:0.7.1' = {
  name: '${deployment().name}-agw'
  scope: resourceGroup()
  params: {
    name: agName
    sku: 'WAF_v2'
    availabilityZones: [1, 2, 3] //TODO: Remove this line if you don't need zones
    gatewayIPConfigurations: [
      {
        name: 'agconfig'
        properties: {
          subnet: {
            id: vnet.outputs.subnetResourceIds[0]
          }
        }
      }
    ]
    frontendIPConfigurations: [
      {
        name: 'appGwFrontendIPv4'
        properties: {
          publicIPAddress: {
            id: publicIpV4.outputs.resourceId
          }
          privateAllocationMethod: 'Dynamic'
        }
      }
      {
        name: 'appGwFrontendIPv6'
        properties: {
          publicIPAddress: {
            id: publicIpV6.outputs.resourceId
          }
          privateAllocationMethod: 'Dynamic'
        }
      }
    ]
    listeners: [
      {
        name: 'mart-listener'
        properties: {
          frontendIPConfiguration: {
            id: resourceId(
              'Microsoft.Network/applicationGateways/frontendIPConfigurations',
              agName,
              'appGwFrontendIPv6'
            )
          }
          frontendPort: {
            id: resourceId('Microsoft.Network/applicationGateways/frontendPorts', agName, 'port_443')
          }
          protocol: 'Https'
          hostname: 'ag.martdegraaf.nl'
          sslCertificate: {
            id: resourceId('Microsoft.Network/applicationGateways/sslCertificates', agName, 'sslCert')
          }
        }
      }
    ]
    routingRules: [
      {
        name: 'route-ipv6'
        properties: {
          ruleType: 'Basic'
          priority: 1
          httpListener: {
            id: resourceId('Microsoft.Network/applicationGateways/httpListeners', agName, 'mart-listener')
          }
          backendAddressPool: {
            id: resourceId('Microsoft.Network/applicationGateways/backendAddressPools', agName, 'mart-backendpool')
          }
          backendHttpSettings: {
            id: resourceId(
              'Microsoft.Network/applicationGateways/backendHttpSettingsCollection',
              agName,
              'mart-backendSetting'
            )
          }
        }
      }
    ]
  }
}

🤖 In the example ‘Azure Verified Modules’ are used, see the documentation of these components here:

public ip addresses connected

Let’s take a look what this means for our architecture. For each Application Gateway we now have two public ip addresses. We just need to configure that the DNS can find this new IPV6 address.

Architecture v2 - added IPV6 support
Architecture v2 - Added Public ip addresses for IPV6

Traffic Manager

We had a traffic manager profile in place to distribute traffic between the two Application Gateways. The traffic manager profile also needed to be updated to support IPv6.

This was actually really complex because Traffic Manager does not support dual stack endpoints directly. Therefore we had to create separate endpoints for IPv4 and IPv6 in the traffic manager profile.

Thereby the Traffic manager routing method priority only supports one endpoint per target. So we had to create three traffic manager profiles to achieve the desired failover behavior. One with the routing method priority with two external endpoints pointing to the nested traffic manager profiles. Each nested traffic manager profile with routing method ‘MultiValue’ had one endpoint for IPv4 and one for IPv6.

Serve traffic over IPv4 and IPv6

Because we want to serve traffic over both IPv4 and IPv6, we had to create a traffic manager profile with the routing method ‘MultiValue’. This profile contains two external endpoints, one for the IPv4 address and one for the IPv6 address of the Application Gateway.

Architecture v3 - added trafficmanager with MultiValue support
Architecture v3 - Added trafficmanager with MultiValue support

This can be achieved with the following Bicep code:

traffic-manager-multivalue.bicep
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

param region string
param environment string
param applicationGatewaySequence string

resource publicIpV4 'Microsoft.Network/publicIPAddresses@2023-11-01' existing = {
  name: 'pip-ag-${applicationGatewaySequence}-${region}-${environment}-v4'
}

resource publicIpV6 'Microsoft.Network/publicIPAddresses@2023-11-01' existing = {
  name: 'pip-ag-${applicationGatewaySequence}-${region}-${environment}-v6'
}

module agTrafficManager 'br/public:avm/res/network/trafficmanagerprofile:0.3.0' = {
  name: '${deployment().name}-tm'
  scope: resourceGroup()
  params: {
    name: 'tm-ag-${applicationGatewaySequence}-${region}-${environment}'
    ttl: 15
    trafficRoutingMethod: 'MultiValue'  
    maxReturn: 2
    endpoints: [
      {
        name: 'ag-${applicationGatewaySequence}-${region}-${environment}-v4'
        type: 'Microsoft.Network/trafficManagerProfiles/externalEndpoints'
        properties: {
            target: publicIpV4.properties.ipAddress
            endpointStatus: 'Enabled'
            endpointMonitorStatus: 'Unmonitored'
            alwaysServe: 'Enabled'
        }
      }
      {
        name: 'ag-${applicationGatewaySequence}-${region}-${environment}-v6'
        type: 'Microsoft.Network/trafficManagerProfiles/externalEndpoints'
        properties: {
            target: publicIpV6.properties.ipAddress
            endpointStatus: 'Enabled'
            endpointMonitorStatus: 'Unmonitored'
            alwaysServe: 'Enabled'
        }
      }
    ]
  }
}

Parent profile for failover

To achieve failover between the two Application Gateways in different regions, we created a parent traffic manager profile with the routing method ‘Priority’. This profile contains two external endpoints, each pointing to one of the nested traffic manager profiles.

There are three different endpoint types in Traffic Manager:

  1. Azure endpoints
  2. External endpoints
  3. Nested endpoints

Azure endpoints

Azure endpoints point directly to Azure resources. However, in our case, we are not directly refering to one Web App or one Public IP address. Therefore, Azure endpoints are not suitable for our scenario here.

Nested endpoints

I initially attempted to use nested endpoints, but they were incompatible with the health probes.

A solution for the healthchecks to work was to create a nested traffic manager profile for each application instead of per Application Gateway. This approach forced us to create three Traffic Manager profiles per application, which seemed like a good solution until we encountered the subscription limit of 200 Traffic Manager profiles.

External endpoints

Another solution for the healthchecks to work was to use external endpoints instead of nested endpoints in the parent Traffic Manager profile. When we switched to using external endpoints that pointed to the FQDNs of the nested Traffic Manager profiles, this had some major advantages.

  • It allowed us to stay within the subscription limits while still achieving the desired failover behavior.
  • It simplified the overall architecture by reducing the number of Traffic Manager profiles needed.
  • It ensured that the health probes functioned correctly, providing reliable monitoring of the Application Gateways’ availability.
traffic-manager-priority.bicep
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49

param region string
param environment string
param primaryRegion string
param secondaryRegion string
param applicationGatewaySequence string

module serviceTrafficManagerFailover 'br/public:avm/res/network/trafficmanagerprofile:0.3.0' = {
  name: '${deployment().name}-tm-failover'
  scope: resourceGroup()
  params: {
    name: 'tm-ag-failover-${applicationGatewaySequence}-${environment}'
    ttl: 15
    trafficRoutingMethod: 'Priority'
    monitorConfig: {
        protocol: 'HTTPS'
        port: 443
        path: '/healthcheck' //TODO: Adjust based on your health probe requirements
        intervalInSeconds: 30
        timeoutInSeconds: 10
        toleratedNumberOfFailures: 3
        customHeaders: [
            {
                name: 'Host'
                value: 'someapi.martdegraaf.nl' //TODO: Adjust based on your health probe requirements
            }
        ]
    }
    endpoints: [
      {
        name: 'tm-${primaryRegion}-${environment}'
        type: 'Microsoft.Network/trafficManagerProfiles/externalEndpoints'
        properties: {
            target: 'tm-ag-${applicationGatewaySequence}-${primaryRegion}-${environment}.trafficmanager.net'
            endpointStatus: 'Enabled'
            priority: 1
        }
      }
      {
        name: 'tm-${secondaryRegion}-${environment}'
        type: 'Microsoft.Network/trafficManagerProfiles/externalEndpoints'
        properties: {
            target: 'tm-ag-${applicationGatewaySequence}-${secondaryRegion}-${environment}.trafficmanager.net'
            endpointStatus: 'Enabled'
            priority: 2
        }
      }
    ]
  }
}

Update DNS records

If you are using a Traffic Manager profile to expose your application gateway you can update your CNAME record to point to the Traffic Manager FQDN. This way both IPv4 and IPv6 traffic will be routed correctly.

If you are routing directly to the Application Gateway you will need to update both the A record (IPv4) and the AAAA record (IPv6) in your DNS settings to point to the respective IP addresses of the Application Gateway.

After updating the DNS records, it may take some time for the changes to propagate. You can use tools like nslookup or online DNS checkers to verify that the records have been updated correctly.

1

nslookup yourapp.yourdomain.com

Conclusion and discussion

It has consequences to enable IPv6 on your Application Gateway. You have to redeploy the Application Gateway which can be complex depending on your current setup. Also for every listener you will need two listeners (one for IPv4 and one for IPv6). The limit of active listeners is 100 so keep that in mind for your architecture. with two listeners for each application you can only have 50 applications behind one Application Gateway.

Now you have dual stack support. Your Application Gateway is available over both IPv4 and IPv6. Make sure to test your setup thoroughly to ensure that everything is working as expected.

Further reading