Wednesday, August 9, 2023

Start with AKS and Azure CNI Overlay

Microsoft released some weeks ago a new great feature for AKS: Azure CNI Overlay: Announcing Azure CNI Overlay in Azure Kubernetes Service | Azure Blog | (

Before, in AKS, we had the choice of two kinds of network plugins:

  • Kubenet: Nice to save a lot of IP, but some limitations, like complex routing with UDR, max 400 nodes, calico only for policy, etc.
  • Azure: Network is easy to manage, because you use a routable IP, that can access/be access, better performances than kubenet, but, it can consume a lot of private IP

And now, Microsoft released the better of these 2 worlds, Azure CNI Overlay.

With Azure CNI Overlay, you can:

  • Use a private IP range, that can be used on all your AKS clusters ( for example)
  • Use Azure Network policies

To create an Azure CNI Overlay cluster, you can’t do this through the portal. So here, we will do this in CLI. The important parameter here is –network-plugin azure –network-plugin-mode overlay:

sysPoolSubnetId=$(az network vnet subnet show -n aks  --vnet-name vnet -g vnet-rg -o tsv --query id)
az group create --name $resourceGroup --location $location
az aks create -n $clusterName -g $resourceGroup --nodepool-name "system" --generate-ssh-keys --vm-set-type VirtualMachineScaleSets --load-balancer-sku standard --enable-private-cluster --enable-managed-identity --dns-name-prefix $clusterName --enable-aad --vnet-subnet-id $sysPoolSubnetId --location $location --kubernetes-version 1.26.3 --network-plugin azure --network-policy azure --network-plugin-mode overlay --node-count 1 --node-vm-size Standard_D2s_v5 --max-pods 30

The deployment can start:

The deployment can start

Azure CNI Overlay

So when the cluster is created, you can do what you want, as it is a CNI cluster.

If you want to read the full description of the overlay, you can read it on the Microsoft website: Configure Azure CNI Overlay networking in Azure Kubernetes Service (AKS) – Azure Kubernetes Service | Microsoft Learn

Now, what to do if you want to change from your old network plugin to CNI Overlay?

You can do, but with some limitations:

  • AKS Cluster in version 1.22+
  • No network policies
  • No Windows node pools

If you meet all of these requirements, then you can upgrade to Azure CNI Overlay, with the following command:

az aks update --name $clusterName --resource-group $resourceGroup --network-plugin-mode overlay --pod-cidr

After some time, depending on your cluster size, you will be with this new network plugin.

To resume, depending on the configuration that you want, and if it is a new cluster, really consider the Azure CNI Overlay option, at it will help you to have all advantages of Azure CNI, without IP constraints.


Related materials:

from StarWind Blog

No comments:

Post a Comment