-
Notifications
You must be signed in to change notification settings - Fork 271
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Use asg warm pools for faster buildkite job starts #822
Comments
Woah! You're quick. Yeah this feature looks great; especially for Windows instances that take a while to boot. According to the announcement it's not yet available on Cloudformation, but when it is we'll take a look into it. |
The only param that would require an additional input to use the warm pool would be the min size of it. We use an asg min size of 0 and max size of 10 so a warm pool min size of 3 seems reasonable. The cloudformation docs have been released. We could add the following Parameters:
WarmPoolMinSize:
Description: Minimum number of instances in warm pool
Type: Number
Default: 0
Conditions:
UseWarmPool:
!Not [ !Equals [ !Ref WarmPoolMinSize, 0 ] ]
Resources:
WarmPool:
Type: AWS::AutoScaling::WarmPool
Condition: UseWarmPool
Properties:
AutoScalingGroupName: !Ref AgentAutoScaleGroup
MinSize: !Ref WarmPoolMinSize
PoolState: Stopped What do you folks think? |
This is fantastic. |
Warm Pool is now available in CloudFormation https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-autoscaling-warmpool.html Right now I can't manually update the created ASG from the current template because the template configuring MixedInstancesPolicy elastic-ci-stack-for-aws/templates/aws-stack.yml Lines 1038 to 1060 in 7b3d02c
We have to remove them if we'd like to use WarmPool |
Any progress/caveats on this issue? Would be really nice to have this option rather than trying to bake git checkouts/other slow workspace parts into the AMI itself. |
This would be helpful on our side as well! Would help out with ensuring we have agents on a warm start up to be able to start jobs faster! |
@alex-shakouri-ai We had a look at this and it would be hard to just plug into the existing model. Currently we aren't looking at re architecting the auto scaling. |
https://aws.amazon.com/about-aws/whats-new/2021/04/amazon-ec2-auto-scaling-introduces-warm-pools-accelerate-scale-out-while-saving-money/
https://aws.amazon.com/blogs/compute/scaling-your-applications-faster-with-ec2-auto-scaling-warm-pools/
If we could keep X instances warmed up, we could start jobs much faster without having to set the min count on the asg to something non-zero.
It should be added soon to cloudformation: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-as-group.html
The text was updated successfully, but these errors were encountered: