Enable Auto Scaling¶
You can create a cluster (or nodegroup in an existing cluster) with IAM role that will allow use of cluster autoscaler:
eksctl create cluster --asg-access
Once cluster is running, you will need to install cluster autoscaler itself. This flag also sets
k8s.io/cluster-autoscaler/<clusterName> tags, so nodegroup discovery should work.
Scaling up from 0¶
If you'd like to be able to scale your node group up from 0 and you have
labels and/or taints defined on your nodegroups you'll need corresponding
tags on your ASGs. You can do this with the
tags key on your node group
definitions. For example, given a node group with the following labels and
nodeGroups: - name: ng1-public ... labels: my-cool-label: pizza taints: feaster: "true:NoSchedule"
You would need to add the following ASG tags:
nodeGroups: - name: ng1-public ... labels: my-cool-label: pizza taints: feaster: "true:NoSchedule" tags: k8s.io/cluster-autoscaler/node-template/label/my-cool-label: pizza k8s.io/cluster-autoscaler/node-template/taint/feaster: "true:NoSchedule"
For unmanaged and managed nodegroups, this is done by
eksctl automatically if
propagateASGTags is set to
true like this:
nodeGroups: - name: ng1-public ... labels: my-cool-label: pizza taints: feaster: "true:NoSchedule" propagateASGTags: true
Zone-aware Auto Scaling¶
If your workloads are zone-specific you'll need to create separate nodegroups for each zone. This is because the
cluster-autoscaler assumes that all nodes in a group are exactly equivalent. So, for example, if a scale-up event is triggered by a pod which needs a zone-specific PVC (e.g. an EBS volume), the new node might get scheduled in the wrong AZ and the pod will fail to start.
You won't need a separate nodegroup for each AZ if your environment meets the following criteria:
- No zone-specific storage requirements.
- No required podAffinity with topology other than host.
- No required nodeAffinity on zone label.
- No nodeSelector on a zone label.
If you meet all of the above requirements (and possibly others) then you should be safe with a single nodegroup which spans multiple AZs. Otherwise you'll want to create separate, single-AZ nodegroups:
nodeGroups: - name: ng1-public instanceType: m5.xlarge # availabilityZones: ["eu-west-2a", "eu-west-2b"]
nodeGroups: - name: ng1-public-2a instanceType: m5.xlarge availabilityZones: ["eu-west-2a"] - name: ng1-public-2b instanceType: m5.xlarge availabilityZones: ["eu-west-2b"]