Packer 不可变镜像构建

概述

Packer 是 HashiCorp 的开源工具，用同一份配置代码为多云平台（AWS AMI、GCP Image、VMware OVA、裸金属 raw image）构建预配置好的机器镜像。它将”操作系统 + 驱动 + 配置”打包成一个不可变制品，部署时直接用镜像启动实例，无需运行时配置。

一句话：Packer 把 Ansible Playbook 的执行结果”冻结”成一个镜像，部署时秒级启动，零配置漂移。

不可变基础设施理念

graph TB
    subgraph "传统模式 (Mutable)"
        A1["启动裸 OS"] --> A2["SSH 安装驱动"] --> A3["SSH 安装中间件"] --> A4["SSH 部署应用"]
        A4 --> A5["运行中修改配置"]
        A5 --> A6["❌ 配置漂移<br/>❌ 不可复现<br/>❌ 雪花服务器"]
    end

    subgraph "不可变模式 (Immutable)"
        B1["Packer 构建镜像"] --> B2["镜像包含: OS + 驱动 + 中间件"]
        B3["部署 = 用镜像启动新实例"] --> B4["应用通过容器/K8s 部署"]
        B5["更新 = 构建新镜像 + 替换旧实例"]
        B5 --> B6["✅ 零漂移<br/>✅ 可复现<br/>✅ 快速回滚"]
    end

    style A6 fill:#f8d7da,stroke:#dc3545
    style B6 fill:#d4edda,stroke:#28a745

特性	传统模式	不可变模式 (Packer)
部署速度	慢（需运行时配置 5-30 分钟）	快（镜像启动 30 秒 - 2 分钟）
配置一致性	可能漂移	保证一致
回滚方式	逆向操作（困难且不安全）	切换到旧镜像（秒级）
扩容速度	慢（新节点需配置）	快（直接用镜像）
构建时间	无（运行时配置）	有（镜像构建 10-30 分钟）

Packer 核心概念

架构与工作流

graph LR
    subgraph "Packer 构建流程"
        Template["Template<br/>(HCL/JSON)"] --> Builder
        subgraph Builder["Builder — 创建基础实例"]
            B1["AWS EC2<br/>→ AMI"]
            B2["GCP Compute<br/>→ Image"]
            B3["QEMU/VirtualBox<br/>→ OVA/raw"]
        end
        Builder --> Provisioner
        subgraph Provisioner["Provisioner — 配置实例"]
            P1["Ansible"]
            P2["Shell Script"]
            P3["File Upload"]
        end
        Provisioner --> PostProcessor
        subgraph PostProcessor["Post-Processor — 后处理"]
            PP1["压缩"]
            PP2["上传到 Registry"]
            PP3["导入 Vagrant"]
        end
        PostProcessor --> Artifact["Artifact<br/>最终镜像制品"]
    end

    style Artifact fill:#d4edda,stroke:#28a745,stroke-width:2px

核心组件

组件	作用	示例
Template	Packer 配置文件（HCL 或 JSON）	gpu-ami.pkr.hcl
Builder	创建临时实例并产出镜像	amazon-ebs / googlecompute / qemu
Provisioner	在临时实例上执行配置	ansible / shell / file
Post-Processor	对产出镜像做后处理	compress / vagrant / manifest
Communicator	与临时实例通信	ssh (Linux) / winrm (Windows)
Artifact	最终产物（AMI ID / Image URL）	ami-12345678

Template 语法 (HCL)

基本结构

# gpu-ami.pkr.hcl — GPU 服务器镜像构建

packer {
  required_plugins {
    amazon = {
      version = ">= 1.3.0"
      source  = "github.com/hashicorp/amazon"
    }
    ansible = {
      version = ">= 1.1.0"
      source  = "github.com/hashicorp/ansible"
    }
  }
}

# 变量定义
variable "aws_region" {
  type    = string
  default = "ap-southeast-1"
}

variable "nvidia_driver_version" {
  type    = string
  default = "550"
}

variable "cuda_version" {
  type    = string
  default = "12.4"
}

variable "ubuntu_version" {
  type    = string
  default = "22.04"
}

# 数据源 — 查找最新 Ubuntu AMI
data "amazon-ami" "ubuntu" {
  filters = {
    name                = "ubuntu/images/hvm-ssd/ubuntu-jammy-${var.ubuntu_version}-amd64-server-*"
    root-device-type    = "ebs"
    virtualization-type = "hvm"
  }
  most_recent = true
  owners      = ["099720109477"]  # Canonical
  region      = var.aws_region
}

# Source — 构建源配置
source "amazon-ebs" "gpu-ami" {
  region        = var.aws_region
  ami_name      = "gpu-ubuntu-${var.ubuntu_version}-nvidia-${var.nvidia_driver_version}-cuda-${var.cuda_version}-{{timestamp}}"
  instance_type = "g4dn.xlarge"    # 需要 GPU 实例来构建（安装驱动需要 GPU）

  source_ami    = data.amazon-ami.ubuntu.id
  ssh_username  = "ubuntu"

  # 根卷配置
  launch_block_device_mappings {
    device_name           = "/dev/sda1"
    volume_size           = 100
    volume_type           = "gp3"
    delete_on_termination = true
  }

  # 临时卷用于 CUDA 安装
  launch_block_device_mappings {
    device_name           = "/dev/sdb"
    volume_size           = 50
    volume_type           = "gp3"
    delete_on_termination = true
  }

  tags = {
    Name         = "GPU-Base-Image"
    OS           = "Ubuntu-${var.ubuntu_version}"
    NvidiaDriver = var.nvidia_driver_version
    CUDA         = var.cuda_version
    BuiltBy      = "Packer"
    BuildDate    = "{{timestamp}}"
  }
}

# Build — 构建步骤
build {
  name    = "gpu-ami-build"
  sources = ["source.amazon-ebs.gpu-ami"]

  # Step 1: 系统更新
  provisioner "shell" {
    inline = [
      "sudo apt-get update",
      "sudo apt-get upgrade -y",
      "sudo apt-get install -y python3 python3-pip",
    ]
  }

  # Step 2: Ansible Provisioner — 安装 GPU 驱动 + CUDA
  provisioner "ansible" {
    playbook_file   = "../ansible/site.yml"
    roles_path      = "../ansible/roles"
    ansible_env_vars = [
      "ANSIBLE_HOST_KEY_CHECKING=False",
      "ANSIBLE_PYTHON_INTERPRETER=/usr/bin/python3",
    ]
    extra_arguments = [
      "--extra-vars", "nvidia_driver_version=${var.nvidia_driver_version} cuda_version=${var.cuda_version}",
      "--tags", "nvidia,cuda,docker,k8s_worker",
    ]
  }

  # Step 3: 清理临时文件
  provisioner "shell" {
    inline = [
      "sudo apt-get autoremove -y",
      "sudo apt-get clean",
      "sudo rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*",
      "sudo rm -rf /var/log/*.log",
      "sudo truncate -s 0 /var/log/syslog",
      # 清理 SSH host keys（实例启动时自动重新生成）
      "sudo rm -f /etc/ssh/ssh_host_*",
      "sudo cloud-init clean",
    ]
  }

  # Post-Processor: 生成构建清单
  post-processor "manifest" {
    output = "manifest.json"
    strip_path = true
  }
}

GPU 镜像构建实战

多云 GPU 镜像并行构建

# multi-cloud-gpu.pkr.hcl — 同时构建 AWS AMI + GCP Image

variable "nvidia_driver_version" { default = "550" }
variable "cuda_version"          { default = "12.4" }

# AWS Source
source "amazon-ebs" "gpu-aws" {
  region        = "ap-southeast-1"
  ami_name      = "gpu-aws-nvidia-${var.nvidia_driver_version}-{{timestamp}}"
  instance_type = "g4dn.xlarge"

  source_ami_filter {
    filters = {
      name                = "ubuntu/images/hvm-ssd/ubuntu-jammy-22.04-amd64-server-*"
      root-device-type    = "ebs"
      virtualization-type = "hvm"
    }
    most_recent = true
    owners      = ["099720109477"]
  }
  ssh_username = "ubuntu"

  tags = {
    Name    = "GPU-Base-AWS"
    Managed = "Packer"
  }
}

# GCP Source
source "googlecompute" "gpu-gcp" {
  project_id   = "my-project"
  zone         = "asia-southeast1-a"
  image_name   = "gpu-gcp-nvidia-${var.nvidia_driver_version}-{{timestamp}}"
  image_family = "gpu-base"
  machine_type = "n1-standard-4"

  source_image_family = "ubuntu-2204-lts"
  ssh_username        = "packer"

  # 临时挂载 GPU
  accelerator_type = "nvidia-tesla-t4"
  accelerator_count = 1

  disk_size = 100
}

# 裸金属 / 本地虚拟化 Source (QEMU)
source "qemu" "gpu-qemu" {
  iso_url      = "https://releases.ubuntu.com/22.04/ubuntu-22.04.4-live-server-amd64.iso"
  iso_checksum = "sha256:45f896de8644590598efe3aa6da3bb4977f7f7e5f199c3d01f0f6d0f0bce1b4e"

  output_directory = "output-gpu-qemu"
  vm_name          = "gpu-base.qcow2"

  disk_size     = 100
  memory        = 8192
  cpus          = 4

  # 注意: QEMU 无法安装 NVIDIA 驱动（无 GPU 硬件）
  # 仅用于预装 CUDA Toolkit + Docker + K8s 基础镜像
  format = "qcow2"
}

# 统一构建
build {
  name = "gpu-multi-cloud"
  sources = [
    "source.amazon-ebs.gpu-aws",
    "source.googlecompute.gpu-gcp",
    "source.qemu.gpu-qemu",
  ]

  # 共享 Provisioner（所有平台执行相同配置）
  provisioner "shell" {
    only   = ["amazon-ebs.gpu-aws", "googlecompute.gpu-gcp"]
    inline = [
      "sudo apt-get update",
      "sudo apt-get install -y python3-pip",
    ]
  }

  provisioner "shell" {
    only   = ["qemu.gpu-qemu"]
    inline = [
      "sudo apt-get update",
      "sudo apt-get install -y python3-pip",
    ]
  }

  # Ansible Provisioner
  provisioner "ansible" {
    only          = ["amazon-ebs.gpu-aws", "googlecompute.gpu-gcp"]
    playbook_file = "../ansible/gpu-server.yml"
    roles_path    = "../ansible/roles"
    extra_arguments = [
      "--extra-vars", "has_gpu=true nvidia_driver_version=${var.nvidia_driver_version}",
    ]
  }

  provisioner "ansible" {
    only          = ["qemu.gpu-qemu"]
    playbook_file = "../ansible/gpu-server.yml"
    roles_path    = "../ansible/roles"
    extra_arguments = [
      "--extra-vars", "has_gpu=false skip_nvidia_driver=true",
    ]
  }

  # 清理
  provisioner "shell" {
    inline = [
      "sudo apt-get autoremove -y && sudo apt-get clean",
      "sudo rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*",
      "sudo rm -f /etc/ssh/ssh_host_*",
      "sudo cloud-init clean",
      "sudo truncate -s 0 /var/log/syslog",
    ]
  }

  post-processor "manifest" {
    output = "manifest-{{build.Slug}}.json"
  }
}

NVIDIA 驱动安装的注意事项

# GPU 镜像构建的特殊处理

build {
  sources = ["source.amazon-ebs.gpu-ami"]

  # 安装 NVIDIA 驱动前，确保 NVIDIA 内核模块加载
  provisioner "shell" {
    inline = [
      # 检查 GPU 是否可用
      "lspci | grep -i nvidia || (echo 'No NVIDIA GPU found' && exit 1)",
      # 安装内核头文件（驱动编译需要）
      "sudo apt-get install -y linux-headers-$(uname -r) build-essential dkms",
    ]
  }

  # 使用 Ansible 安装驱动
  provisioner "ansible" {
    playbook_file = "../ansible/nvidia-driver.yml"
    extra_arguments = [
      "--extra-vars", "nvidia_driver_version=${var.nvidia_driver_version}",
    ]
  }

  # 关键: 安装后验证 + 清理编译缓存
  provisioner "shell" {
    inline = [
      # 验证驱动
      "nvidia-smi || (echo 'NVIDIA driver installation failed' && exit 1)",
      "nvidia-smi --query-gpu=driver_version,name,memory.total --format=csv",
      # 清理编译缓存（减小镜像体积）
      "sudo rm -rf /usr/src/linux-headers-*",
      "sudo apt-get remove -y build-essential dkms",
      "sudo apt-get autoremove -y",
    ]
  }
}

CUDA Toolkit 安装

# CUDA Toolkit 单独安装（可选择不安装驱动）
provisioner "shell" {
  inline = [
    # 下载 CUDA Toolkit（不含驱动）
    "wget https://developer.download.nvidia.com/compute/cuda/12.4.0/local_installers/cuda_12.4.0_550.54.14_linux.run",
    # 静默安装: 只装 toolkit，不装驱动
    "sudo sh cuda_12.4.0_550.54.14_linux.run --toolkit --silent --override",
    # 配置环境变量
    "echo 'export PATH=/usr/local/cuda-12.4/bin:$PATH' | sudo tee /etc/profile.d/cuda.sh",
    "echo 'export LD_LIBRARY_PATH=/usr/local/cuda-12.4/lib64:$LD_LIBRARY_PATH' | sudo tee -a /etc/profile.d/cuda.sh",
    # 验证
    "/usr/local/cuda-12.4/bin/nvcc --version",
    # 清理安装包
    "rm -f cuda_12.4.0_550.54.14_linux.run",
  ]
}

镜像版本管理

语义化版本 + 时间戳

variable "image_version" {
  type    = string
  default = "1.2.0"    # major.minor.patch
}

locals {
  # 镜像名称: gpu-base-1.2.0-202608031030
  image_name = "gpu-base-${var.image_version}-{{timestamp}}"

  # 镜像 Family（用于 GCP，最新镜像引用）
  image_family = "gpu-base-${var.image_version}"
}

source "amazon-ebs" "gpu" {
  ami_name    = local.image_name
  # ...
}

source "googlecompute" "gpu" {
  image_name   = local.image_name
  image_family = local.image_family
  # ...
}

构建清单 (Manifest)

post-processor "manifest" {
  output      = "manifests/manifest-{{timestamp}}.json"
  strip_path  = true
  custom_data = {
    git_sha       = "{{build.SHA}}"
    builder       = "{{build.Slug}}"
    version       = var.image_version
    driver        = var.nvidia_driver_version
    cuda          = var.cuda_version
    ubuntu        = var.ubuntu_version
  }
}

// manifest-202608031030.json 示例
{
  "builds": [
    {
      "name": "amazon-ebs.gpu-aws",
      "builder_type": "amazon-ebs",
      "files": null,
      "artifact_id": "ap-southeast-1:ami-0abc123def456",
      "custom_data": {
        "git_sha": "a1b2c3d",
        "version": "1.2.0",
        "driver": "550",
        "cuda": "12.4"
      }
    },
    {
      "name": "googlecompute.gpu-gcp",
      "builder_type": "googlecompute",
      "artifact_id": "projects/my-project/global/images/gpu-base-1-2-0-202608031030",
      "custom_data": {
        "git_sha": "a1b2c3d",
        "version": "1.2.0",
        "driver": "550",
        "cuda": "12.4"
      }
    }
  ],
  "last_run_uuid": "..."
}

镜像清理策略

#!/usr/bin/env python3
"""定期清理旧的 AMI 镜像，只保留最近 N 个版本"""
import boto3
from datetime import datetime, timedelta
from typing import List

class ImageCleaner:
    def __init__(self, region: str, image_prefix: str, keep_count: int = 5):
        self.ec2 = boto3.client('ec2', region_name=region)
        self.image_prefix = image_prefix
        self.keep_count = keep_count

    def list_images(self) -> List[dict]:
        """列出所有匹配前缀的 AMI"""
        response = self.ec2.describe_images(
            Owners=['self'],
            Filters=[{'Name': 'name', 'Values': [f'{self.image_prefix}*']}]
        )
        # 按创建时间排序（最新的在前）
        images = sorted(
            response['Images'],
            key=lambda x: x['CreationDate'],
            reverse=True
        )
        return images

    def delete_old_images(self) -> dict:
        """删除旧镜像，保留最近 N 个"""
        images = self.list_images()
        keep = images[:self.keep_count]
        delete = images[self.keep_count:]

        result = {
            'kept': [img['Name'] for img in keep],
            'deleted': [],
            'errors': []
        }

        for img in delete:
            try:
                # 先 deregister AMI
                self.ec2.deregister_image(ImageId=img['ImageId'])

                # 再删除关联的 snapshot
                for bdm in img.get('BlockDeviceMappings', []):
                    if 'Ebs' in bdm and 'SnapshotId' in bdm['Ebs']:
                        self.ec2.delete_snapshot(
                            SnapshotId=bdm['Ebs']['SnapshotId']
                        )

                result['deleted'].append({
                    'name': img['Name'],
                    'id': img['ImageId'],
                })
                print(f"  Deleted: {img['Name']} ({img['ImageId']})")
            except Exception as e:
                result['errors'].append({
                    'name': img['Name'],
                    'error': str(e)
                })
                print(f"  Error: {img['Name']} - {e}")

        return result

    def audit_report(self) -> str:
        """生成镜像审计报告"""
        images = self.list_images()
        lines = [
            f"AMI Audit Report - {datetime.now().isoformat()}",
            f"Prefix: {self.image_prefix}",
            f"Total: {len(images)} images, Keeping: {self.keep_count}",
            "=" * 80,
        ]
        for i, img in enumerate(images):
            status = "✅ KEEP" if i < self.keep_count else "🗑️ DELETE"
            lines.append(
                f"[{status}] {img['Name']} | {img['ImageId']} | {img['CreationDate']}"
            )
        return "\n".join(lines)


if __name__ == "__main__":
    cleaner = ImageCleaner(
        region="ap-southeast-1",
        image_prefix="gpu-base-",
        keep_count=5,
    )

    # 审计报告
    print(cleaner.audit_report())
    print()

    # 执行清理
    print("Cleaning up old images...")
    result = cleaner.delete_old_images()
    print(f"\nDeleted: {len(result['deleted'])}, Errors: {len(result['errors'])}")

CI/CD 集成

GitHub Actions 镜像构建流水线

# .github/workflows/build-gpu-image.yml
name: Build GPU Base Image

on:
  push:
    branches: [main]
    paths:
      - 'packer/**'
      - 'ansible/**'
  schedule:
    # 每周一自动构建最新镜像（捕获上游安全补丁）
    - cron: '0 2 * * 1'
  workflow_dispatch:
    inputs:
      image_version:
        description: 'Image version (e.g., 1.3.0)'
        required: true
        default: '1.3.0'

env:
  AWS_REGION: ap-southeast-1
  IMAGE_VERSION: ${{ github.event.inputs.image_version || '1.2.0' }}

jobs:
  validate:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Install Packer
        run: |
          curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
          echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
          sudo apt-get update && sudo apt-get install packer

      - name: Packer fmt
        run: packer fmt -check -recursive packer/

      - name: Packer validate
        run: |
          cd packer
          packer init .
          packer validate \
            -var "nvidia_driver_version=550" \
            -var "cuda_version=12.4" \
            gpu-ami.pkr.hcl

  build:
    needs: validate
    runs-on: ubuntu-latest
    timeout-minutes: 60
    steps:
      - uses: actions/checkout@v4

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ env.AWS_REGION }}

      - name: Install Packer
        run: |
          curl -fsSL https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
          echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
          sudo apt-get update && sudo apt-get install packer ansible

      - name: Packer Init
        run: cd packer && packer init .

      - name: Build GPU AMI
        run: |
          cd packer
          packer build \
            -var "nvidia_driver_version=550" \
            -var "cuda_version=12.4" \
            -var "image_version=${{ env.IMAGE_VERSION }}" \
            gpu-ami.pkr.hcl

      - name: Parse manifest
        id: manifest
        run: |
          AMI_ID=$(jq -r '.builds[0].artifact_id' packer/manifests/manifest-*.json | cut -d: -f2)
          echo "ami_id=$AMI_ID" >> $GITHUB_OUTPUT
          echo "Built AMI: $AMI_ID"

      - name: Update Terraform variable
        run: |
          # 自动更新 Terraform 中的 AMI 引用
          cd terraform/environments/prod
          sed -i "s/ami_id = .*/ami_id = \"${{ steps.manifest.outputs.ami_id }}\"/" variables.tf
          git config user.name "github-actions[bot]"
          git config user.email "github-actions[bot]@users.noreply.github.com"
          git add variables.tf
          git commit -m "chore: update GPU AMI to ${{ env.IMAGE_VERSION }} (${{ steps.manifest.outputs.ami_id }})"
          git push

      - name: Notify Slack
        if: always()
        uses: 8398a7/action-slack@v3
        with:
          status: ${{ job.status }}
          fields: repo,message,commit,author
          text: |
            GPU Image Build: ${{ job.status }}
            Version: ${{ env.IMAGE_VERSION }}
            AMI: ${{ steps.manifest.outputs.ami_id }}

Packer + Terraform + Ansible 协作模式

graph TB
    subgraph "构建阶段 (CI)"
        Dev["开发者提交代码"] --> Git["Git 仓库"]
        Git --> Packer["Packer Build"]
        Packer --> Ans["Ansible Playbook"]
        Ans --> AMI["新 AMI 镜像"]
        AMI --> Registry["镜像 Registry<br/>(AMI/Image Family)"]
    end

    subgraph "部署阶段 (CD)"
        TF["Terraform Apply"]
        TF -->|"引用最新 AMI"| Registry
        TF --> New["启动新实例<br/>(使用新镜像)"]
    end

    subgraph "运行阶段"
        New --> K8s["K8s 节点<br/>(OS+驱动已预装)"]
        K8s --> App["应用 Pod<br/>(容器化部署)"]
    end

    style AMI fill:#d4edda,stroke:#28a745,stroke-width:2px
    style K8s fill:#e8f5e9,stroke:#4caf50

完整工作流

1. 开发者修改 Ansible Playbook（如升级 NVIDIA 驱动版本）
2. Git Push → CI 触发 Packer 构建
3. Packer:
   a. 启动临时 GPU 实例
   b. 执行 Ansible Playbook（安装驱动、CUDA、Docker、K8s）
   c. 清理临时文件
   d. 创建 AMI
   e. 销毁临时实例
4. CI 解析 manifest.json 获取新 AMI ID
5. CI 自动更新 Terraform 变量（ami_id）
6. PR → 人工审批 → Terraform Apply（启动新实例用新镜像）
7. 蓝绿/滚动替换旧实例
8. 旧实例 drain + 销毁

Terraform 引用 Packer 镜像

# 方式 1: 硬编码 AMI ID（由 CI 自动更新）
variable "ami_id" {
  type    = string
  default = "ami-0abc123def456"  # ← CI 自动更新此行
}

# 方式 2: 使用 SSM Parameter Store（推荐）
data "aws_ssm_parameter" "gpu_ami" {
  name = "/ami/gpu-base/latest"
}

# CI 构建后将 AMI ID 写入 SSM
# aws ssm put-parameter --name /ami/gpu-base/latest --value ami-xxx --type String --overwrite

resource "aws_launch_template" "gpu" {
  image_id      = data.aws_ssm_parameter.gpu_ami.value
  instance_type = "g4dn.xlarge"
  # ...
}

# 方式 3: GCP Image Family（自动指向最新）
data "google_compute_image" "gpu" {
  family  = "gpu-base-1.2.0"
  project = "my-project"
}

resource "google_compute_instance" "gpu" {
  boot_disk {
    initialize_params {
      image = data.google_compute_image.gpu.self_link
    }
  }
  # ...
}

最佳实践

镜像构建

实践	说明
最小化镜像	只安装必需的包，减小镜像体积（影响启动速度和存储成本）
安全基线	镜像中预置 CIS Benchmark 合规配置
清理痕迹	删除 SSH host keys、bash history、临时文件
版本化	每次构建生成唯一版本号 + 时间戳
不可变标签	AMI 名称包含版本和 timestamp，不用 “latest”
验证步骤	构建后运行健康检查（nvidia-smi、docker info、kubectl version）

镜像管理

实践	说明
Image Family	GCP 用 Image Family 管理版本（自动指向最新）
SSM Parameter	AWS 用 SSM Parameter Store 存储最新 AMI ID
保留策略	保留最近 5-10 个版本用于回滚
自动清理	定期清理超过 30 天的旧镜像（节省存储成本）
审计日志	记录谁、何时、构建了什么版本

常见问题

问题	根因	解决方案
构建超时	GPU 驱动编译慢	增加 timeout 或用预编译包
AMI 创建失败	临时实例 SSH 不通	检查安全组、子网路由、SSH Key
驱动安装失败	内核版本与驱动不兼容	指定 linux-headers-$(uname -r) 版本
镜像太大	未清理缓存	添加清理 step + dd zero 填充空闲空间
多云构建慢	串行构建	使用 sources 列表并行构建
镜像启动后无法 SSH	清理了 host keys	cloud-init 会自动重新生成，确保 cloud-init 已安装

关联知识

• IaC 方法论与实践总览 — Packer 在 IaC 五层中的 Image 层
• Terraform 基础设施即代码 — Terraform 引用 Packer 构建的镜像
• Terraform 生产级实践 — Terraform + Packer 的生产级协作
• Ansible 配置管理实战 — Packer 调用 Ansible Playbook 配置镜像
• ../gpu-cluster-ops/裸金属/GPU 裸金属服务器运维 — 裸金属镜像构建（QEMU/raw）
• ../gpu-cluster-ops/automation/集群自动化部署方案 — 集群部署中的镜像管理
• ../sre/变更管理全流程 — 镜像更新即变更，需走变更流程
• ../sre/弹性伸缩策略 — 不可变镜像是弹性伸缩的基础（快速扩容）
• ../cicd/CI-CD 知识总览 — Packer 在 CI/CD 流水线中的角色

参考资源

• Packer 官方文档: https://developer.hashicorp.com/packer/docs
• Packer Ansible Provisioner: https://developer.hashicorp.com/packer/plugins/provisioners/ansible
• Packer Amazon Builder: https://developer.hashicorp.com/packer/plugins/builders/amazon
• Packer GoogleCompute Builder: https://developer.hashicorp.com/packer/plugins/builders/googlecompute
• 不可变基础设施: https://martinfowler.com/bliki/ImmutableServer.html
• NVIDIA Container Toolkit: https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/

学习时间

阶段	时间	备注
Packer 基础	2026-08-03	Template、Builder、Provisioner
GPU 镜像构建	2026-08-03	NVIDIA 驱动、CUDA、多云并行
CI/CD 集成	2026-08-03	GitHub Actions、Terraform 联动

---

状态: ✅ 已完成 学习时间: 2026-08-03