A frequently asked question about rolling out Azure VNet with different subnets in a infrastructure as code environment is where and how to define the subnets in your Azure ARM template. In the most examples online the number of subnets is configured in the template and the name and addressPrefix is configured in the parameter file. The big disadvantage with this scenario is that when you deploy a VNet with for example two subnets and later you decide to add more subnets you have two choices both of which are uncomfortable for an Infrastructure as code release pipeline.
Below the choices you have with the accompanying disadvantages:
- First you can change the template file and include the new subnets in the template and redeploy your template. The disadvantage of this solution is that in an enterprise environment with multiple VNet’s with different subnet configuration you need to create and maintenance multiple template that serve the same purpose.
- Second option is to add the new subnets with a separate template file to your existing VNet, however the disadvantage of this solution is that when you redeploy the first template all the other subnets that are not configured within will be deleted because they are not configured in the template (that’s how arm works right :p)
Azure ARM
So, before we jump in code let’s see how Azure ARM works and why this situation happens. We know that an VNet is defined as a resource whiten Azure ARM and Each resource provider offers a set of resources and operations for working with an Azure service and that Subnet is a child resource of Azure VNet. To get a list of the resource types of Azure VNet you can run the following PowerShell command:
(Get-AzureRmResourceProvider -ProviderNamespace Microsoft.network).ResourceTypes.ResourceTypeName
Which will return all the resource types with in Microsoft.Network resource provider, this list includes some of the resources listed below:
- virtualNetworks
- publicIPAddresses
- networkInterfaces
- loadBalancers
- networkSecurityGroups
- applicationSecurityGroups
However, subnet is not an independent resource like a Network Interface or an publicIPAddresses it is the property of a VNet and it only exists in a VNet. And with that said we can also understand why Incremental and complete deployments mode doesn’t work for subnets. The primary difference between these two modes is how Resource Manager handles existing resources in the resource group that are not in the template:
- In complete mode, Resource Manager deletes resources that exist in the resource group but are not specified in the template.
- In incremental mode, Resource Manager leaves unchanged resources that exist in the resource group but are not specified in the template.
If the resource already exists in the resource group and its settings are unchanged, the operation results in no change. If you change the settings for a resource, the resource is provisioned with those new settings.
Now that we understand the underlying structure of Azure ARM and Resources its time to take a look at some code and the differences within.
Most common scenario
First lets start with the most common scenario for configuring the subnets within a JSON template as below in which you define each subnet specifically. In this scenario the amount of the subnets is defined in the template and the subnet name and address prefix is defined trough the parameter file. This is an example of an VNet with static amount of subnets, if you want to add subnets to the VNet you have the choice of changing the template or adding the subnets trough a separate template file (see above for more information).
{ "name": "[parameters('VNetName')]", "type": "Microsoft.Network/virtualNetworks", "location": "[resourceGroup().location]", "apiVersion": "2016-03-30", "dependsOn": [], "tags": { "displayName": "[parameters('VNetName')]" }, "properties": { "addressSpace": { "addressPrefixes": [ "[parameters('VNetPrefix')]" ] }, "subnets": [ { "name": "[parameters('Subnet1Name')]", "properties": { "addressPrefix": "[parameters('Subnet1Prefix')]" } }, { "name": "[parameters('Subnet2Name')]", "properties": { "addressPrefix": "[parameters('Subnet2Prefix')]" } } ] } }
Desired scenario
In the example below I have replaced the above with Property iteration.
To create multiple values for a property on a resource, I have added a copy array in the properties element. This array contains objects, and each object has the following properties:
- name – the name of the property to create multiple values for
- count – the number of values to create
- input – an object that contains the values to assign to the property
I am also using the length function on the array to specify the count for iterations, and copyIndex to retrieve the current index in the array.
The following example shows how to apply copy to the subnet property on a VNet:
{ "name": "[parameters('vNetName')]", "type": "Microsoft.Network/virtualNetworks", "location": "[resourceGroup().location]", "apiVersion": "2017-10-01", "tags": { "displayName": "[parameters('vNetName')]" }, "properties": { "addressSpace": { "addressPrefixes": [ "[parameters('vNetPrefix')]" ] }, "copy": [ { "name": "subnets", "count": "[length(parameters('subnets'))]", "input": { "name": "[parameters('subnets')[copyIndex('subnets')].name]", "properties": { "addressPrefix": "[parameters('subnets')[copyIndex('subnets')].addressPrefix]" } } } ] } }
As you can see I have configured one subnet in this VNet template, but based on the content from the parameter it can deploy one or 10 subnets. The above solution give’s you the possibility to configure in your parameter file the amount of subnets you want to deploy. This way you are always redeploying one template with different parameter files. And when you want to add or remove a subnets you only need to change your parameter file.
Example of the parameter file:
{ "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#", "contentVersion": "1.0.0.0", "parameters": { "vNetName": { "value": "Vnet01" }, "vNetPrefix": { "value": "10.0.0.0/16" }, "subnets": { "value": [ { "name": "Subnet-1", "addressPrefix": "10.0.1.0/24" }, { "name": "Subnet-2", "addressPrefix": "10.0.2.0/24" }, { "name": "Subnet-3", "addressPrefix": "10.0.3.0/24" }, { "name": "Subnet-4", "addressPrefix": "10.0.4.0/24" }, { "name": "Subnet-5", "addressPrefix": "10.0.5.0/24" } ] } } }
As you can see this way you can easily create a default VNet template and based on the properties in your parameter file you will get another result. For an infrastructure as code environment with CI/CD mindset this is a huge requirement because you want to define you Release pipeline once and reuse it for different deployments without editing the template fie or your release pipeline.
In the next post I will explain how you can achieve the same working method for OMS and other Azure Resource.
Great stuff,
You really should publish this on the GitHubs quick starts, super useful.
Hi Micah . Thank you for this wonderful template. I was wondering why you’re template works without deleting the parent Vnet when adding additional subnet. Most of users are facing this issue : https://feedback.azure.com/forums/217313-networking/suggestions/18758545-support-vnet-re-deployment-without-destroying-subn.
Some other users are using “condition” as a workaround
I am trying this, always get …. what am I missing?
21:43:06 – Validation returned the following errors:
21:43:06 – : Deployment template validation failed: ‘The template parameters ‘vNetPrefix, vNetName, subnets’ in the parameters file are not valid; they are not present in the original template and can therefore not be provided at deployment time. The only supported parameters for this template are ”. Please see https://aka.ms/arm-deploy/#parameter-file for usage details.’.
21:43:06 –
21:43:06 – Template is invalid.
21:43:06 –
Are u deploying only the template file? From the error it looks like you are missing the parameter file/object for the deployment. let me know if you need more help.
Hi do you have the json files the template and the deployment template you could upload or send me a link to these?
You can find the template and parameter file on my GitHub account: https://github.com/pkhabazi/ARM-Templates/tree/master/virtualNetwork
When importing the json templates into visual studio get error with subnet array is undefined. Also have you managed to validate and build through visual studio 2019 as get also following error cannot retrieve dynamic parameters error converting value $schema to type
You can ignore the “error” message in your editor that the subnet array is not defined, this is because the variable will be defined when you run the template. subnetarray is result of the copy loop. how are you deploying the templates, trough visual studio? because where the template is created is not important and schema is not related to where you write your code.
Yes I am deploying through Visual Studio at the moment. I am also looking at also adopting and adding additional parameters to the template such as the gateway, tags, routing and potentially peering to other subnets on the vnets etc.
If resources are deployed to the subnet have you found a way to use the increment property to update the subnet with changes?
Do you have the code which will also create the gateway subnet for the vnet as this wont be a copy of the subnets or do you think it would make sense to use another template to create this?
hi Coote98,
I have updated the template to include gateway creation step: https://github.com/pkhabazi/ARM-Templates/tree/master/virtualNetwork-With-GW
Please let me know if you have further questions.
Hi,
Thanks for the post. I was wondering if there is a way to also include subnet level network security groups to the template dynamically.
Hi Andy,
I have uploaded an example for you on my GitHub Repo, here you can find the template and parameter file: https://github.com/pkhabazi/ARM-Templates/tree/master/virtualNetwork-with-NSG
This way of NSG deployment however has 2 disadvantages that you need to take in count:
1. you cannot validate the properties, testing length or type etc
2. every subnet needs to have a NSG or you need to add an if statement to control this (this one is not a disadvantage but more a way of working). Because you can decide to deploy all subnets with a nsg by default.
let me know if you need more info.
kr,
Pouyan
Is there any Template for getting existing deployed subnets from a vnet prefix and create next /28 for every new vm deployment. In this scenario we want to isolate every deployment with subsequent /28 subnet from same prefix.