This is 1/3 response to a short prompt about implementation options for GitHub Runner form broken Server to Github Enterprise Cloud:
# EC2-Based GitHub Actions Self-Hosted Runners - Complete Implementation
## Architecture Overview
This solution deploys auto-scaling GitHub Actions runners on EC2 instances that can trigger your existing AWS CodeBuild pipelines. Runners are managed via Auto Scaling Groups with automatic registration and health monitoring.
## Prerequisites
- AWS CLI configured with appropriate credentials
- GitHub Enterprise Cloud organization admin access
- Existing CodeBuild project(s)
- VPC with public/private subnets
## Solution Components
### 1. CloudFormation Template### 2. GitHub Workflow for CodeBuild Integration## Deployment Steps
### Step 1: Create GitHub Personal Access Token
1. Navigate to GitHub → Settings → Developer settings → Personal access tokens → Fine-grained tokens
2. Create token with these permissions:
- *Repository permissions:*
- Actions: Read and write
- Metadata: Read
- *Organization permissions:*
- Self-hosted runners: Read and write
```bash
# Store token securely
export GITHUB_PAT="ghp_xxxxxxxxxxxxxxxxxxxx"
export GITHUB_ORG="your-org-name"
```
### Step 2: Deploy CloudFormation Stack
```bash
# Set variables
export AWS_REGION=us-east-1
export STACK_NAME=github-runner-ec2
export VPC_ID=vpc-xxxxxxxx
export SUBNET_IDS="subnet-xxxxxxxx,subnet-yyyyyyyy"
# Deploy stack
aws cloudformation create-stack \
--stack-name $STACK_NAME \
--template-body file://github-runner-ec2-asg.yaml \
--parameters \
ParameterKey=VpcId,ParameterValue=$VPC_ID \
ParameterKey=PrivateSubnetIds,ParameterValue=\"$SUBNET_IDS\" \
ParameterKey=GitHubOrganization,ParameterValue=$GITHUB_ORG \
ParameterKey=GitHubPAT,ParameterValue=$GITHUB_PAT \
ParameterKey=InstanceType,ParameterValue=t3.medium \
ParameterKey=MinSize,ParameterValue=2 \
ParameterKey=MaxSize,ParameterValue=10 \
ParameterKey=DesiredCapacity,ParameterValue=2 \
ParameterKey=RunnerLabels,ParameterValue="self-hosted,linux,x64,ec2,aws,codebuild" \
ParameterKey=CodeBuildProjectNames,ParameterValue="" \
--capabilities CAPABILITY_NAMED_IAM \
--region $AWS_REGION
# Wait for completion (5-10 minutes)
aws cloudformation wait stack-create-complete \
--stack-name $STACK_NAME \
--region $AWS_REGION
# Get stack outputs
aws cloudformation describe-stacks \
--stack-name $STACK_NAME \
--query 'Stacks[0].Outputs' \
--region $AWS_REGION
```
### Step 3: Verify Runners
```bash
# Check Auto Scaling Group
ASG_NAME=$(aws cloudformation describe-stacks \
--stack-name $STACK_NAME \
--query 'Stacks[0].Outputs[?OutputKey==`AutoScalingGroupName`].OutputValue' \
--output text)
aws autoscaling describe-auto-scaling-groups \
--auto-scaling-group-names $ASG_NAME \
--region $AWS_REGION
# List running instances
aws ec2 describe-instances \
--filters "Name=tag:aws:autoscaling:groupName,Values=$ASG_NAME" \
--query 'Reservations[].Instances[].[InstanceId,State.Name,PrivateIpAddress]' \
--output table
# Check CloudWatch logs
aws logs tail /github-runner/instances --follow
```
### Step 4: Verify in GitHub
Navigate to: `https://github.com/organizations/YOUR_ORG/settings/actions/r...`
You should see your EC2 runners listed as "Idle" with labels: `self-hosted, linux, x64, ec2, aws, codebuild`
## Using One Runner for Multiple Repos & Pipelines
### Organization-Level Runners (Recommended)
EC2 runners registered at the organization level can serve all repositories automatically.
*Benefits:*
- Centralized management
- Cost-efficient resource sharing
- Simplified scaling
- Single point of monitoring
*Configuration in CloudFormation:*
The template already configures organization-level runners via the UserData script:
```bash
./config.sh --url "https://github.com/${GitHubOrganization}" ...
```
### Multi-Repository Workflow Examples### Advanced: Runner Groups for Access Control### Label-Based Runner Selection Strategy
*Create different runner pools with specific labels:*
```bash
# Production runners
RunnerLabels: "self-hosted,linux,ec2,production,high-performance"
# Development runners
RunnerLabels: "self-hosted,linux,ec2,development,general"
# Team-specific runners
RunnerLabels: "self-hosted,linux,ec2,team-platform,specialized"
```
*Use in workflows:*
```yaml
jobs:
prod-deploy:
runs-on: [self-hosted, linux, ec2, production]
dev-test:
runs-on: [self-hosted, linux, ec2, development]
platform-build:
runs-on: [self-hosted, linux, ec2, team-platform]
```
## Monitoring and Maintenance
### Monitor Runner Health
```bash
# Check Auto Scaling Group health
aws autoscaling describe-auto-scaling-groups \
--auto-scaling-group-names $ASG_NAME \
--query 'AutoScalingGroups[0].[DesiredCapacity,MinSize,MaxSize,Instances[].[InstanceId,HealthStatus,LifecycleState]]'
# View instance system logs
INSTANCE_ID=$(aws autoscaling describe-auto-scaling-groups \
--auto-scaling-group-names $ASG_NAME \
--query 'AutoScalingGroups[0].Instances[0].InstanceId' \
--output text)
aws ec2 get-console-output --instance-id $INSTANCE_ID
# Check CloudWatch logs
aws logs get-log-events \
--log-group-name /github-runner/instances \
--log-stream-name $INSTANCE_ID/runner \
--limit 50
```
### Connect to Runner Instance (via SSM)
```bash
# List instances
aws autoscaling describe-auto-scaling-groups \
--auto-scaling-group-names $ASG_NAME \
--query 'AutoScalingGroups[0].Instances[].[InstanceId,HealthStatus]' \
--output table
# Connect via Session Manager (no SSH key needed)
aws ssm start-session --target $INSTANCE_ID
# Once connected, check runner status
sudo systemctl status actions.runner.
sudo journalctl -u actions.runner.* -f
```
### Troubleshooting Common Issues## Advanced Scaling Configuration
### Lambda-Based Dynamic Scaling
For more sophisticated scaling based on GitHub Actions queue depth:### Deploy Scaling Lambda
```bash
# Create Lambda function
zip function.zip github-queue-scaler.py
aws lambda create-function \
--function-name github-runner-scaler \
--runtime python3.11 \
--role arn:aws:iam::ACCOUNT_ID:role/lambda-execution-role \
--handler github-queue-scaler.lambda_handler \
--zip-file fileb://function.zip \
--timeout 30 \
--environment Variables="{
ASG_NAME=$ASG_NAME,
GITHUB_ORG=$GITHUB_ORG,
GITHUB_TOKEN=$GITHUB_PAT,
MAX_RUNNERS=10,
MIN_RUNNERS=2
}"
# Create CloudWatch Events rule to trigger every 2 minutes
aws events put-rule \
--name github-runner-scaling \
--schedule-expression 'rate(2 minutes)'
aws events put-targets \
--rule github-runner-scaling \
--targets "Id"="1","Arn"="arn:aws:lambda:REGION:ACCOUNT:function:github-runner-scaler"
```
## Cost Optimization
### 1. Use Spot Instances
Add to Launch Template in CloudFormation:
```yaml
LaunchTemplateData:
InstanceMarketOptions:
MarketType: spot
SpotOptions:
MaxPrice: "0.05" # Set max price
SpotInstanceType: one-time
```
### 2. Scheduled Scaling
Scale down during off-hours:
```bash
# Scale down at night (9 PM)
aws autoscaling put-scheduled-action \
--auto-scaling-group-name $ASG_NAME \
--scheduled-action-name scale-down-night \
--recurrence "0 21 * * " \
--desired-capacity 1
# Scale up in morning (7 AM)
aws autoscaling put-scheduled-action \
--auto-scaling-group-name $ASG_NAME \
--scheduled-action-name scale-up-morning \
--recurrence "0 7 * MON-FRI" \
--desired-capacity 3
```
### 3. Instance Type Mix
Use multiple instance types for better availability and cost:
```yaml
MixedInstancesPolicy:
InstancesDistribution:
OnDemandBaseCapacity: 1
OnDemandPercentageAboveBaseCapacity: 25
SpotAllocationStrategy: price-capacity-optimized
LaunchTemplate:
Overrides:
- InstanceType: t3.medium
- InstanceType: t3a.medium
- InstanceType: t2.medium
```
## Security Best Practices
1. *No hardcoded credentials* - Using Secrets Manager for GitHub PAT
2. *IMDSv2 enforced* - Prevents SSRF attacks
3. *Minimal IAM permissions* - Scoped to specific CodeBuild projects
4. *Private subnets* - Runners not directly accessible from internet
5. *SSM for access* - No SSH keys needed
6. *Encrypted secrets* - Secrets Manager encryption at rest
7. *CloudWatch logging* - All runner activity logged
## References
- [GitHub Self-hosted Runners Documentation](https://docs.github.com/en/actions/hosting-your-own-runners/...)
- [GitHub Runner Registration API](https://docs.github.com/en/rest/actions/self-hosted-runners)
- [AWS Auto Scaling Documentation](https://docs.aws.amazon.com/autoscaling/ec2/userguide/what-i...)
- [AWS CodeBuild API Reference](https://docs.aws.amazon.com/codebuild/latest/APIReference/We...)
- [GitHub Actions Runner Releases](https://github.com/actions/runner/releases)
- [AWS Systems Manager Session Manager](https://docs.aws.amazon.com/systems-manager/latest/userguide...)
This solution provides a production-ready, cost-effective EC2-based runner infrastructure with automatic scaling, comprehensive monitoring, and multi-repository support for triggering CodeBuild pipelines.