CI/CD实践

持续集成和持续部署是现代软件开发的核心实践。本文将介绍如何使用GitHub Actions构建Python项目的CI/CD流水线。

GitHub Actions基础

创建.github/workflows/ci.yml：

name: Python CI

on:
  push:
    branches: [main, develop]
  pull_request:
    branches: [main]

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.9", "3.10", "3.11"]
    
    steps:
    - uses: actions/checkout@v4
    
    - name: Set up Python ${{ matrix.python-version }}
      uses: actions/setup-python@v4
      with:
        python-version: ${{ matrix.python-version }}
    
    - name: Install dependencies
      run: |
        python -m pip install --upgrade pip
        pip install -r requirements-dev.txt
        pip install -e .
    
    - name: Lint with flake8
      run: |
        flake8 . --count --select=E9,F63,F7,F82 --show-source --statistics
        flake8 . --count --max-complexity=10 --max-line-length=127 --statistics
    
    - name: Type check with mypy
      run: mypy .
    
    - name: Test with pytest
      run: |
        pytest --cov=src --cov-report=xml --cov-report=term-missing
    
    - name: Upload coverage to Codecov
      uses: codecov/codecov-action@v3
      with:
        file: ./coverage.xml
        flags: unittests
        name: codecov-umbrella

多阶段流水线

name: Complete CI/CD Pipeline

on:
  push:
    branches: [main]
  release:
    types: [created]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    
    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.11'
    
    - name: Install dependencies
      run: |
        pip install -r requirements.txt
        pip install -r requirements-dev.txt
    
    - name: Run tests
      run: |
        pytest --cov=src --cov-report=xml
    
    - name: Upload coverage
      uses: codecov/codecov-action@v3
      with:
        file: ./coverage.xml

  security:
    runs-on: ubuntu-latest
    needs: test
    steps:
    - uses: actions/checkout@v4
    
    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.11'
    
    - name: Run security checks
      run: |
        pip install safety bandit
        safety check -r requirements.txt
        bandit -r src/

  build:
    runs-on: ubuntu-latest
    needs: [test, security]
    steps:
    - uses: actions/checkout@v4
    
    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.11'
    
    - name: Build package
      run: |
        pip install build
        python -m build
    
    - name: Upload artifacts
      uses: actions/upload-artifact@v3
      with:
        name: dist
        path: dist/

  publish:
    runs-on: ubuntu-latest
    needs: build
    if: github.event_name == 'release'
    steps:
    - name: Download artifacts
      uses: actions/download-artifact@v3
      with:
        name: dist
    
    - name: Publish to PyPI
      uses: pypa/gh-action-pypi-publish@release/v1
      with:
        user: __token__
        password: ${{ secrets.PYPI_TOKEN }}

测试自动化配置

# pytest.ini
[pytest]
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*
addopts = 
    -v
    --tb=short
    --strict-markers
    -W error::DeprecationWarning
markers =
    slow: marks tests as slow
    integration: marks tests as integration tests
    unit: marks tests as unit tests

# conftest.py
import pytest
from unittest.mock import Mock

@pytest.fixture(scope="session")
def event_loop():
    import asyncio
    loop = asyncio.get_event_loop_policy().new_event_loop()
    yield loop
    loop.close()

@pytest.fixture
def mock_db():
    db = Mock()
    db.connect.return_value = True
    db.execute.return_value = []
    return db

@pytest.fixture
def sample_data():
    return {
        "users": [
            {"id": 1, "name": "Alice"},
            {"id": 2, "name": "Bob"}
        ]
    }

# tests/test_api.py
import pytest
from fastapi.testclient import TestClient
from app import app

client = TestClient(app)

@pytest.mark.unit
def test_root():
    response = client.get("/")
    assert response.status_code == 200
    assert response.json() == {"message": "Docker Python应用"}

@pytest.mark.integration
def test_process_endpoint(sample_data):
    response = client.post("/process", json=sample_data)
    assert response.status_code == 200
    assert "task_id" in response.json()

代码质量检查

# .github/workflows/code-quality.yml
name: Code Quality

on: [push, pull_request]

jobs:
  quality:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@v4
    
    - name: Set up Python
      uses: actions/setup-python@v4
      with:
        python-version: '3.11'
    
    - name: Install quality tools
      run: |
        pip install black isort flake8 mypy bandit safety
    
    - name: Check formatting with Black
      run: black --check .
    
    - name: Check import sorting with isort
      run: isort --check-only .
    
    - name: Lint with flake8
      run: flake8 .
    
    - name: Type check with mypy
      run: mypy .
    
    - name: Security check with bandit
      run: bandit -r src/
    
    - name: Check dependencies with safety
      run: safety check -r requirements.txt

部署自动化

# .github/workflows/deploy.yml
name: Deploy

on:
  push:
    branches: [main]
  workflow_dispatch:

env:
  REGISTRY: ghcr.io
  IMAGE_NAME: ${{ github.repository }}

jobs:
  build-and-push:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      packages: write
    
    steps:
    - uses: actions/checkout@v4
    
    - name: Log in to Container Registry
      uses: docker/login-action@v2
      with:
        registry: ${{ env.REGISTRY }}
        username: ${{ github.actor }}
        password: ${{ secrets.GITHUB_TOKEN }}
    
    - name: Extract metadata
      id: meta
      uses: docker/metadata-action@v4
      with:
        images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
        tags: |
          type=ref,event=branch
          type=sha,prefix=
    
    - name: Build and push Docker image
      uses: docker/build-push-action@v4
      with:
        context: .
        push: true
        tags: ${{ steps.meta.outputs.tags }}
        labels: ${{ steps.meta.outputs.labels }}

  deploy-staging:
    runs-on: ubuntu-latest
    needs: build-and-push
    environment: staging
    
    steps:
    - name: Deploy to staging
      run: |
        echo "Deploying to staging environment..."
        # 这里添加实际的部署命令
        # 例如: kubectl set image deployment/myapp myapp=${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}:${{ github.sha }}

环境变量管理

# config/settings.py
import os
from pydantic import BaseSettings
from typing import Optional

class Settings(BaseSettings):
    # 应用配置
    app_name: str = "My App"
    app_version: str = "1.0.0"
    debug: bool = False
    
    # 数据库配置
    database_url: str
    database_pool_size: int = 20
    
    # Redis配置
    redis_url: str = "redis://localhost:6379"
    
    # API配置
    api_key: str
    api_secret: str
    
    class Config:
        env_file = ".env"
        case_sensitive = False

# 使用
settings = Settings()

# .env文件示例
# DATABASE_URL=postgresql://user:pass@localhost/db
# API_KEY=your-api-key
# API_SECRET=your-api-secret

监控和告警

# .github/workflows/monitoring.yml
name: Monitoring

on:
  schedule:
    - cron: '0 * * * *'  # 每小时运行

jobs:
  health-check:
    runs-on: ubuntu-latest
    steps:
    - name: Check application health
      run: |
        curl -f https://api.example.com/health || exit 1
    
    - name: Check response time
      run: |
        response_time=$(curl -o /dev/null -s -w '%{time_total}' https://api.example.com)
        if (( $(echo "$response_time > 2.0" | bc -l) )); then
          echo "Warning: Response time too high: ${response_time}s"
          exit 1
        fi
    
    - name: Notify on failure
      if: failure()
      uses: slackapi/slack-github-action@v1
      with:
        payload: |
          {
            "text": "🚨 Application health check failed!"
          }
      env:
        SLACK_WEBHOOK_URL: ${{ secrets.SLACK_WEBHOOK }}

最佳实践

1. 快速反馈：流水线应在10分钟内完成
2. 并行执行：同时运行独立任务
3. 缓存依赖：使用actions/cache加速构建
4. 环境隔离：为不同环境使用不同的 secrets
5. 手动触发：支持workflow_dispatch进行手动部署

总结

CI/CD流水线确保代码质量和部署可靠性。GitHub Actions提供了灵活的配置方式，结合测试自动化、代码质量检查和部署自动化，可以构建完整的DevOps工作流。