Skip to content

Conversation

@moko-poi
Copy link
Contributor

Description

Adds support for filtering security groups by VPC ID in securityGroupSelectorTerms, enabling users to disambiguate security groups with identical names across different VPCs.

Fixes #8175

Motivation

When multiple VPCs exist in a single AWS account with security groups sharing the same name, Karpenter currently selects all matching security groups regardless of VPC. This causes the following error when launching nodes:

InvalidParameter: Security group <SECURITY_GROUP_ID> and subnet <SUBNET_ID> belong to different networks.

This is a common scenario in:

  • Multi-tenant environments with VPC-per-customer architecture
  • Development/staging/production environments using separate VPCs
  • Multi-region deployments with similar naming conventions

Changes

  • API: Add optional vpcID field to SecurityGroupSelectorTerm with pattern validation (vpc-[0-9a-z]+)
  • Provider: Update getFilterSets() to support VPC filtering for ID, Name, and Tag-based selectors
  • Tests: Add comprehensive unit and integration tests for VPC filtering
  • CRD: Auto-generated CRD updates with new vpcID field
  • Example: Add security-group-with-vpc.yaml demonstrating usage

Usage Example

apiVersion: karpenter.k8s.aws/v1
kind: EC2NodeClass
metadata:
  name: default
spec:
  securityGroupSelectorTerms:
    # Select security group by name within specific VPC
    - name: k8s-node
      vpcID: vpc-12345678
    # Also works with tags
    - tags:
        Environment: production
      vpcID: vpc-12345678
  # ...

Backward Compatibility

✅ Fully backward compatible - vpcID is optional and existing configurations continue to work unchanged.

Testing

  • Unit tests for getFilterSets() with VPC filtering
  • Validation tests for vpcID field patterns
  • Integration tests for controller reconciliation with VPC filters
  • All existing tests pass

Checklist

  • Added/updated tests
  • Updated CRDs
  • Added example YAML
  • Backward compatible

@moko-poi moko-poi requested a review from a team as a code owner November 22, 2025 14:56
@netlify
Copy link

netlify bot commented Nov 22, 2025

Deploy Preview for karpenter-docs-prod canceled.

Name Link
🔨 Latest commit 1973b40
🔍 Latest deploy log https://app.netlify.com/projects/karpenter-docs-prod/deploys/6921cf60ce55400008d63e82

@moko-poi
Copy link
Contributor Author

moko-poi commented Nov 28, 2025

@AndrewMitchell25
Hi! Just wanted to gently follow up on this PR. Would appreciate any feedback when you have a chance. Thanks!

@ryan-mist
Copy link
Contributor

I wonder if we could validate that the VPC of the subnet and security group we chose is the same? Looking at the outputs of describe SGs and subnets, the VPC ID is included. It has the benefit of not surfacing a new configuration and doesn't require expanding the recommended IAM policy (https://karpenter.sh/docs/reference/cloudformation/)

@chrisdoherty4
Copy link

I've wondered why a security group that's not part of VPC is even included? Are there use-cases where this is valid? Assuming not then why not filter non-VPC security groups out?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support specifying VPC name with security group name on security group selector terms

3 participants