Cloud Layers Architecture¶

The cloud-specific layers extend the base image with AWS and/or GCP tools, creating specialized variants optimized for single-cloud or multi-cloud workflows.

Layer Strategy¶

graph TD
    A[Base Layer ~2.0 GB] --> B[AWS Layer +800 MB]
    A --> C[GCP Layer +900 MB]
    A --> D[All DevOps +1.2 GB]
    B --> E[aws-devops ~2.8 GB]
    C --> F[gcp-devops ~2.9 GB]
    D --> G[all-devops ~3.2 GB]

    style A fill:#90EE90
    style B fill:#FF9900
    style C fill:#4285F4
    style D fill:#FFA500
    style E fill:#FF9900
    style F fill:#4285F4
    style G fill:#FFA500

Design Rationale¶

AWS Layer¶

Tools Added¶

Total AWS layer: ~800 MB

Detailed Breakdown¶

curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
./aws/install --bin-dir /usr/local/bin --install-dir /usr/local/aws-cli

dnf install -y \
  https://s3.amazonaws.com/session-manager-downloads/plugin/latest/linux_64bit/session-manager-plugin.rpm

# Connect to EC2 instance
aws ssm start-session --target i-1234567890abcdef0

# Port forwarding
aws ssm start-session --target i-1234567890abcdef0 \
  --document-name AWS-StartPortForwardingSession \
  --parameters "portNumber=3306,localPortNumber=3306"

pip3 install --no-cache-dir \
  boto3 \
  cfn-lint \
  s3cmd \
  crcmod

AWS-Specific Configurations¶

Environment variables set:

AWS_DEFAULT_REGION=us-east-1  # Default region
AWS_PAGER=""                  # Disable pager for automation

Configuration files: - AWS CLI config location: ~/.aws/config - Credentials location: ~/.aws/credentials - Session Manager plugin: Auto-configured

GCP Layer¶

Tools Added¶

Total GCP layer: ~900 MB

Detailed Breakdown¶

curl https://sdk.cloud.google.com | bash -s -- --disable-prompts \
  --install-dir=/usr/local

curl -fsSL "https://github.com/GoogleCloudPlatform/docker-credential-gcr/releases/download/v2.1.18/docker-credential-gcr_linux_amd64-2.1.18.tar.gz" | \
  tar xz -C /usr/local/bin

docker-credential-gcr configure-docker --registries=gcr.io,us-docker.pkg.dev

GCP-Specific Configurations¶

Environment variables set:

CLOUDSDK_CORE_DISABLE_PROMPTS=1  # Automation-friendly
CLOUDSDK_PYTHON_SITEPACKAGES=1   # Allow system packages

Configuration files: - gcloud config location: ~/.config/gcloud/ - Credentials: Application Default Credentials (ADC) - Docker config: ~/.docker/config.json

Size Optimization Attempts¶

Why not trim components?

We considered installing only specific components:

gcloud components install gcloud gsutil bq --quiet

Issues: - Minimal size savings (~50 MB, only 7% reduction) - Breaks future component installations - Missing dependencies for some features - Complexity not worth the trade-off

Decision: Include full SDK for maximum compatibility.

All-DevOps Layer (Multi-Cloud)¶

The all-devops image combines both AWS and GCP layers.

Architecture¶

FROM base-layer
# Install AWS layer (~800 MB)
RUN install-aws-tools.sh
# Install GCP layer (~900 MB)
RUN install-gcp-tools.sh

Size Optimization¶

Without optimization: 2.0 GB + 800 MB + 900 MB = 3.7 GB

With optimization: 3.2 GB (save 500 MB)

Optimization techniques:

1. Shared dependencies: Both AWS and GCP use Python

   # Share Python packages
   pip3 install --no-cache-dir boto3 google-cloud-storage
   

2. Single layer cleanup: Combine installs, clean once

   RUN install-aws && install-gcp && \
       dnf clean all && \
       rm -rf /var/cache/* /tmp/*
   

3. Deduplicated binaries: Some tools overlap

4. Both use python3 (don't install twice)

5. Both use curl, unzip (already in base)

6. Trivy database reuse: Single vulnerability DB for all scanning

Comparison Table¶

Layer Caching Strategy¶

Build Order for Maximum Caching¶

graph LR
    A[Base Layer<br/>Changes: Quarterly] --> B[AWS Layer<br/>Changes: Monthly]
    A --> C[GCP Layer<br/>Changes: Monthly]
    A --> D[All-DevOps<br/>Changes: Monthly]

Benefits: - Base layer cached for months (rarely changes) - Cloud layers rebuild independently - Parallel builds possible (AWS + GCP simultaneously) - Faster CI/CD (only changed layers rebuild)

Example: AWS CLI Update¶

When AWS CLI updates:

1. Base layer: ✅ Cached (unchanged)
2. AWS layer: ❌ Rebuilds (~3 minutes)
3. GCP layer: ✅ Cached (unchanged)
4. All-DevOps: ⚠️ Partial rebuild (AWS portion only)

Total rebuild time: ~6 minutes (instead of ~25 minutes full rebuild)

Size vs Features Trade-off¶

graph LR
    A[Base<br/>2.0 GB<br/>Platform Tools] --> B[AWS<br/>+800 MB<br/>AWS Tools]
    A --> C[GCP<br/>+900 MB<br/>GCP Tools]
    A --> D[All-DevOps<br/>+1.2 GB<br/>Both Clouds]

    style A fill:#90EE90
    style B fill:#FFD700
    style C fill:#FFD700
    style D fill:#FFA500

Decision Framework¶

Multi-Architecture Considerations¶

Size by Architecture¶

Build Time by Architecture¶

Apple Silicon: Use arm64 native for best performance

AWS Graviton: Use arm64 native (30% faster than emulated amd64)

Security Considerations¶

Attack Surface by Layer¶

Vulnerability Scanning Results¶

Typical Trivy scan (as of Jan 2025):

Note: Most vulnerabilities are in dependencies, not direct threats.

Mitigation: - Weekly automatic rebuilds - Security patches applied immediately - Trivy scans in CI/CD pipeline

Maintenance Strategy¶

Update Frequency¶

Automated Rebuilds¶

GitHub Actions workflow:

schedule:
  - cron: '0 2 * * 1'  # Weekly, Monday 2 AM UTC

jobs:
  build-base:
    # Build base layer
  build-aws:
    needs: build-base
    # Build AWS variant
  build-gcp:
    needs: build-base
    # Build GCP variant
  build-all:
    needs: build-base
    # Build all-devops

Benefits: - Always up-to-date - Security patches applied automatically - Minimal manual intervention

Next Steps¶

• Image Comparison - Detailed tool matrix
• Base Layer Details - Platform tools breakdown
• Building Custom Images - Extend these images