CI/CD集成与持续测试
大约 11 分钟
CI/CD集成与持续测试
前言：让测试"自动驾驶"
还记得我刚开始做自动化测试时，每次都要手动运行测试脚本，然后盯着屏幕等结果。有时候忘记运行测试，有时候测试跑了一半电脑死机了，简直是折磨。
直到我接触了CI/CD，才发现原来测试可以"自动驾驶"：代码一提交，测试自动运行；测试一失败，立马通知相关人员；报告自动生成，结果一目了然。这种感觉就像从手动挡换到了自动挡，解放了双手，提升了效率。
今天，我就来分享一下如何将接口自动化测试完美集成到CI/CD流水线中，实现真正的持续测试。
CI/CD与持续测试概述
什么是CI/CD？
CI（Continuous Integration）持续集成：
- 开发人员频繁地将代码集成到主干分支
- 每次集成都通过自动化构建和测试来验证
- 快速发现和修复集成问题
CD（Continuous Delivery/Deployment）持续交付/部署：
- 持续交付：确保代码随时可以部署到生产环境
- 持续部署：自动将通过测试的代码部署到生产环境
持续测试的价值
传统测试流程：
开发 → 提测 → 手工测试 → 发现问题 → 修复 → 重新测试 → 发布
持续测试流程：
开发 → 提交代码 → 自动测试 → 实时反馈 → 快速修复 → 自动发布持续测试的优势：
- 快速反馈：问题在几分钟内就能发现
- 降低风险：每次变更都经过验证
- 提高质量：自动化测试覆盖更全面
- 节省成本：减少手工测试投入
- 加速交付：缩短发布周期
测试策略与分层
测试金字塔
/\
/ \
/ UI \ 少量UI测试（慢、脆弱、昂贵）
/______\
/ \
/ 集成测试 \ 适量集成测试（中等速度、中等成本）
/___________\
/ \
/ 单元测试 \ 大量单元测试（快、稳定、便宜）
/_______________\CI/CD中的测试分层策略
# pytest.ini - 测试分层配置
[tool:pytest]
markers =
unit: 单元测试
integration: 集成测试
smoke: 冒烟测试
regression: 回归测试
performance: 性能测试
e2e: 端到端测试
testpaths = tests
python_files = test_*.py
python_classes = Test*
python_functions = test_*不同阶段的测试策略：
# CI/CD测试策略
stages:
pre-commit:
tests: [unit, lint, security-scan]
duration: < 2分钟
commit:
tests: [unit, integration, smoke]
duration: < 10分钟
nightly:
tests: [regression, performance, e2e]
duration: < 2小时
release:
tests: [full-regression, security, performance]
duration: < 4小时GitHub Actions集成实战
1. 基础工作流配置
# .github/workflows/api-test.yml
name: API自动化测试
on:
push:
branches: [ main, develop ]
pull_request:
branches: [ main ]
schedule:
# 每天凌晨2点运行回归测试
- cron: '0 2 * * *'
env:
PYTHON_VERSION: '3.9'
TEST_ENV: 'ci'
jobs:
smoke-test:
name: 冒烟测试
runs-on: ubuntu-latest
steps:
- name: 检出代码
uses: actions/checkout@v3
- name: 设置Python环境
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'pip'
- name: 安装依赖
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: 运行冒烟测试
run: |
pytest -m smoke --alluredir=allure-results --junitxml=junit.xml -v
- name: 上传测试结果
uses: actions/upload-artifact@v3
if: always()
with:
name: smoke-test-results
path: |
allure-results/
junit.xml
- name: 发布测试报告
uses: dorny/test-reporter@v1
if: always()
with:
name: 冒烟测试报告
path: junit.xml
reporter: java-junit
integration-test:
name: 集成测试
runs-on: ubuntu-latest
needs: smoke-test
if: success()
services:
mysql:
image: mysql:8.0
env:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: testdb
ports:
- 3306:3306
options: >-
--health-cmd="mysqladmin ping"
--health-interval=10s
--health-timeout=5s
--health-retries=3
steps:
- name: 检出代码
uses: actions/checkout@v3
- name: 设置Python环境
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'pip'
- name: 安装依赖
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: 等待服务启动
run: |
sleep 30
- name: 运行集成测试
env:
DB_HOST: localhost
DB_PORT: 3306
DB_USER: root
DB_PASSWORD: root
DB_NAME: testdb
run: |
pytest -m integration --alluredir=allure-results --junitxml=junit.xml -v
- name: 生成Allure报告
if: always()
run: |
allure generate allure-results -o allure-report --clean
- name: 部署报告到GitHub Pages
if: always()
uses: peaceiris/actions-gh-pages@v3
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: allure-report
destination_dir: reports/${{ github.run_number }}
regression-test:
name: 回归测试
runs-on: ubuntu-latest
if: github.event_name == 'schedule' || github.event_name == 'workflow_dispatch'
strategy:
matrix:
environment: [dev, test, staging]
steps:
- name: 检出代码
uses: actions/checkout@v3
- name: 设置Python环境
uses: actions/setup-python@v4
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: 'pip'
- name: 安装依赖
run: |
python -m pip install --upgrade pip
pip install -r requirements.txt
- name: 运行回归测试
env:
TEST_ENV: ${{ matrix.environment }}
run: |
pytest -m regression --alluredir=allure-results-${{ matrix.environment }} --junitxml=junit-${{ matrix.environment }}.xml -v
- name: 上传测试结果
uses: actions/upload-artifact@v3
if: always()
with:
name: regression-test-results-${{ matrix.environment }}
path: |
allure-results-${{ matrix.environment }}/
junit-${{ matrix.environment }}.xml2. 高级工作流特性
# .github/workflows/advanced-test.yml
name: 高级测试流水线
on:
push:
branches: [ main ]
pull_request:
branches: [ main ]
jobs:
test-matrix:
name: 矩阵测试
runs-on: ${{ matrix.os }}
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
python-version: ['3.8', '3.9', '3.10']
exclude:
- os: windows-latest
python-version: '3.8'
steps:
- uses: actions/checkout@v3
- name: 设置Python ${{ matrix.python-version }}
uses: actions/setup-python@v4
with:
python-version: ${{ matrix.python-version }}
- name: 运行测试
run: |
pip install -r requirements.txt
pytest -m "not slow" --junitxml=junit-${{ matrix.os }}-${{ matrix.python-version }}.xml
parallel-test:
name: 并行测试
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- name: 设置Python环境
uses: actions/setup-python@v4
with:
python-version: '3.9'
- name: 安装依赖
run: |
pip install -r requirements.txt
pip install pytest-xdist
- name: 并行运行测试
run: |
pytest -n auto --dist worksteal --alluredir=allure-results
conditional-test:
name: 条件测试
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
with:
fetch-depth: 0 # 获取完整历史记录
- name: 检查变更文件
id: changes
run: |
if git diff --name-only ${{ github.event.before }} ${{ github.sha }} | grep -E "(api/|tests/)" > /dev/null; then
echo "api_changed=true" >> $GITHUB_OUTPUT
else
echo "api_changed=false" >> $GITHUB_OUTPUT
fi
- name: 运行API测试
if: steps.changes.outputs.api_changed == 'true'
run: |
pip install -r requirements.txt
pytest tests/api/ -v
- name: 跳过测试
if: steps.changes.outputs.api_changed == 'false'
run: |
echo "API相关文件未变更，跳过测试"Jenkins集成实战
1. Jenkinsfile配置
// Jenkinsfile
pipeline {
agent any
environment {
PYTHON_VERSION = '3.9'
TEST_ENV = 'jenkins'
ALLURE_RESULTS = 'allure-results'
ALLURE_REPORT = 'allure-report'
}
parameters {
choice(
name: 'TEST_SUITE',
choices: ['smoke', 'regression', 'full'],
description: '选择测试套件'
)
booleanParam(
name: 'SEND_NOTIFICATION',
defaultValue: true,
description: '是否发送通知'
)
}
stages {
stage('准备环境') {
steps {
script {
echo "开始准备测试环境..."
sh '''
python3 -m venv venv
source venv/bin/activate
pip install --upgrade pip
pip install -r requirements.txt
'''
}
}
}
stage('代码检查') {
parallel {
stage('语法检查') {
steps {
sh '''
source venv/bin/activate
flake8 tests/ --max-line-length=120
'''
}
}
stage('安全扫描') {
steps {
sh '''
source venv/bin/activate
bandit -r tests/ -f json -o bandit-report.json
'''
}
post {
always {
archiveArtifacts artifacts: 'bandit-report.json', allowEmptyArchive: true
}
}
}
}
}
stage('运行测试') {
steps {
script {
def testCommand = ""
switch(params.TEST_SUITE) {
case 'smoke':
testCommand = "pytest -m smoke"
break
case 'regression':
testCommand = "pytest -m regression"
break
case 'full':
testCommand = "pytest"
break
}
sh """
source venv/bin/activate
${testCommand} --alluredir=${ALLURE_RESULTS} --junitxml=junit.xml -v
"""
}
}
post {
always {
// 发布JUnit测试结果
junit 'junit.xml'
// 归档测试结果
archiveArtifacts artifacts: "${ALLURE_RESULTS}/**", allowEmptyArchive: true
}
}
}
stage('生成报告') {
steps {
script {
allure([
includeProperties: false,
jdk: '',
properties: [],
reportBuildPolicy: 'ALWAYS',
results: [[path: "${ALLURE_RESULTS}"]]
])
}
}
}
stage('性能测试') {
when {
anyOf {
branch 'main'
expression { params.TEST_SUITE == 'full' }
}
}
steps {
sh '''
source venv/bin/activate
pytest -m performance --alluredir=performance-results
'''
}
post {
always {
archiveArtifacts artifacts: 'performance-results/**', allowEmptyArchive: true
}
}
}
}
post {
always {
// 清理工作空间
cleanWs()
}
success {
script {
if (params.SEND_NOTIFICATION) {
emailext (
subject: "✅ 测试通过 - ${env.JOB_NAME} #${env.BUILD_NUMBER}",
body: """
测试执行成功！
项目: ${env.JOB_NAME}
构建号: ${env.BUILD_NUMBER}
分支: ${env.BRANCH_NAME}
测试套件: ${params.TEST_SUITE}
查看详细报告: ${env.BUILD_URL}allure/
""",
to: "${env.CHANGE_AUTHOR_EMAIL ?: 'team@company.com'}"
)
}
}
}
failure {
script {
if (params.SEND_NOTIFICATION) {
emailext (
subject: "❌ 测试失败 - ${env.JOB_NAME} #${env.BUILD_NUMBER}",
body: """
测试执行失败！
项目: ${env.JOB_NAME}
构建号: ${env.BUILD_NUMBER}
分支: ${env.BRANCH_NAME}
测试套件: ${params.TEST_SUITE}
查看失败详情: ${env.BUILD_URL}console
查看测试报告: ${env.BUILD_URL}allure/
""",
to: "${env.CHANGE_AUTHOR_EMAIL ?: 'team@company.com'}"
)
}
}
}
}
}2. Jenkins共享库
// vars/runAPITests.groovy
def call(Map config) {
pipeline {
agent any
stages {
stage('运行API测试') {
steps {
script {
// 设置环境变量
env.TEST_ENV = config.environment ?: 'dev'
env.TEST_SUITE = config.testSuite ?: 'smoke'
// 运行测试
sh """
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
pytest -m ${env.TEST_SUITE} --alluredir=allure-results
"""
// 生成报告
allure([
includeProperties: false,
jdk: '',
properties: [],
reportBuildPolicy: 'ALWAYS',
results: [[path: 'allure-results']]
])
}
}
}
}
}
}
// 使用共享库
// Jenkinsfile
@Library('shared-library') _
runAPITests([
environment: 'test',
testSuite: 'regression'
])GitLab CI/CD集成
1. .gitlab-ci.yml配置
# .gitlab-ci.yml
stages:
- lint
- test
- report
- deploy
variables:
PYTHON_VERSION: "3.9"
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache/pip"
cache:
paths:
- .cache/pip/
- venv/
before_script:
- python3 -m venv venv
- source venv/bin/activate
- pip install --upgrade pip
- pip install -r requirements.txt
lint:
stage: lint
script:
- source venv/bin/activate
- flake8 tests/ --max-line-length=120
- black --check tests/
only:
- merge_requests
- main
smoke_test:
stage: test
script:
- source venv/bin/activate
- pytest -m smoke --alluredir=allure-results --junitxml=junit.xml -v
artifacts:
when: always
paths:
- allure-results/
- junit.xml
reports:
junit: junit.xml
expire_in: 1 week
only:
- merge_requests
- main
integration_test:
stage: test
services:
- mysql:8.0
variables:
MYSQL_ROOT_PASSWORD: root
MYSQL_DATABASE: testdb
DB_HOST: mysql
DB_PORT: 3306
DB_USER: root
DB_PASSWORD: root
DB_NAME: testdb
script:
- source venv/bin/activate
- sleep 30 # 等待数据库启动
- pytest -m integration --alluredir=allure-results --junitxml=junit.xml -v
artifacts:
when: always
paths:
- allure-results/
- junit.xml
reports:
junit: junit.xml
expire_in: 1 week
only:
- main
regression_test:
stage: test
script:
- source venv/bin/activate
- pytest -m regression --alluredir=allure-results --junitxml=junit.xml -v
artifacts:
when: always
paths:
- allure-results/
- junit.xml
reports:
junit: junit.xml
expire_in: 1 week
only:
- schedules
- web
generate_report:
stage: report
image: frankescobar/allure-docker-service
script:
- allure generate allure-results -o allure-report --clean
artifacts:
paths:
- allure-report/
expire_in: 1 month
dependencies:
- smoke_test
- integration_test
- regression_test
only:
- main
- schedules
deploy_report:
stage: deploy
script:
- mkdir public
- cp -r allure-report/* public/
artifacts:
paths:
- public
dependencies:
- generate_report
only:
- main2. 多环境部署
# .gitlab-ci.yml - 多环境配置
.test_template: &test_template
stage: test
script:
- source venv/bin/activate
- export TEST_ENV=$ENVIRONMENT
- pytest -m $TEST_SUITE --alluredir=allure-results-$ENVIRONMENT --junitxml=junit-$ENVIRONMENT.xml -v
artifacts:
when: always
paths:
- allure-results-$ENVIRONMENT/
- junit-$ENVIRONMENT.xml
reports:
junit: junit-$ENVIRONMENT.xml
test_dev:
<<: *test_template
variables:
ENVIRONMENT: "dev"
TEST_SUITE: "smoke"
only:
- develop
test_staging:
<<: *test_template
variables:
ENVIRONMENT: "staging"
TEST_SUITE: "regression"
only:
- main
test_production:
<<: *test_template
variables:
ENVIRONMENT: "production"
TEST_SUITE: "smoke"
when: manual
only:
- main通知与报告
1. 钉钉通知集成
# utils/notification.py
import requests
import json
from datetime import datetime
class DingTalkNotifier:
"""钉钉通知器"""
def __init__(self, webhook_url: str):
self.webhook_url = webhook_url
def send_test_result(self, result_data: dict):
"""发送测试结果通知"""
# 构建消息内容
if result_data['status'] == 'success':
color = '#00FF00'
emoji = '✅'
title = '测试通过'
else:
color = '#FF0000'
emoji = '❌'
title = '测试失败'
message = {
"msgtype": "markdown",
"markdown": {
"title": f"{emoji} {title}",
"text": f"""
## {emoji} {title}
**项目**: {result_data['project']}
**分支**: {result_data['branch']}
**环境**: {result_data['environment']}
**执行时间**: {result_data['execution_time']}
### 测试结果
- **总用例数**: {result_data['total_tests']}
- **通过**: {result_data['passed_tests']}
- **失败**: {result_data['failed_tests']}
- **跳过**: {result_data['skipped_tests']}
### 详细信息
- **构建号**: {result_data['build_number']}
- **执行人**: {result_data['executor']}
- **报告链接**: [查看详细报告]({result_data['report_url']})
---
*发送时间: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}*
"""
}
}
# 发送通知
response = requests.post(
self.webhook_url,
headers={'Content-Type': 'application/json'},
data=json.dumps(message, ensure_ascii=False).encode('utf-8')
)
return response.status_code == 200
# 在CI/CD中使用
def send_notification():
"""发送测试结果通知"""
import os
notifier = DingTalkNotifier(os.getenv('DINGTALK_WEBHOOK'))
result_data = {
'status': os.getenv('TEST_STATUS', 'unknown'),
'project': os.getenv('CI_PROJECT_NAME', 'Unknown'),
'branch': os.getenv('CI_COMMIT_REF_NAME', 'Unknown'),
'environment': os.getenv('TEST_ENV', 'Unknown'),
'execution_time': os.getenv('EXECUTION_TIME', 'Unknown'),
'total_tests': os.getenv('TOTAL_TESTS', '0'),
'passed_tests': os.getenv('PASSED_TESTS', '0'),
'failed_tests': os.getenv('FAILED_TESTS', '0'),
'skipped_tests': os.getenv('SKIPPED_TESTS', '0'),
'build_number': os.getenv('CI_PIPELINE_ID', 'Unknown'),
'executor': os.getenv('GITLAB_USER_NAME', 'System'),
'report_url': os.getenv('REPORT_URL', '#')
}
notifier.send_test_result(result_data)
if __name__ == '__main__':
send_notification()2. 企业微信通知
# utils/wechat_notification.py
import requests
import json
class WeChatNotifier:
"""企业微信通知器"""
def __init__(self, webhook_url: str):
self.webhook_url = webhook_url
def send_test_summary(self, summary_data: dict):
"""发送测试摘要"""
# 构建卡片消息
message = {
"msgtype": "template_card",
"template_card": {
"card_type": "text_notice",
"source": {
"icon_url": "https://example.com/test-icon.png",
"desc": "自动化测试平台"
},
"main_title": {
"title": f"测试执行{'成功' if summary_data['success'] else '失败'}",
"desc": f"项目: {summary_data['project']}"
},
"emphasis_content": {
"title": f"{summary_data['passed_tests']}/{summary_data['total_tests']}",
"desc": "通过率"
},
"sub_title_text": f"分支: {summary_data['branch']} | 环境: {summary_data['environment']}",
"horizontal_content_list": [
{
"keyname": "执行时间",
"value": summary_data['execution_time']
},
{
"keyname": "失败用例",
"value": str(summary_data['failed_tests'])
}
],
"jump_list": [
{
"type": 1,
"url": summary_data['report_url'],
"title": "查看详细报告"
}
],
"card_action": {
"type": 1,
"url": summary_data['report_url']
}
}
}
response = requests.post(
self.webhook_url,
headers={'Content-Type': 'application/json'},
data=json.dumps(message, ensure_ascii=False).encode('utf-8')
)
return response.status_code == 200性能监控与优化
1. 测试执行时间监控
# utils/performance_monitor.py
import time
import psutil
import json
from datetime import datetime
class TestPerformanceMonitor:
"""测试性能监控器"""
def __init__(self):
self.start_time = None
self.end_time = None
self.start_memory = None
self.end_memory = None
self.metrics = {}
def start_monitoring(self):
"""开始监控"""
self.start_time = time.time()
self.start_memory = psutil.virtual_memory().used
def stop_monitoring(self):
"""停止监控"""
self.end_time = time.time()
self.end_memory = psutil.virtual_memory().used
self.metrics = {
'execution_time': self.end_time - self.start_time,
'memory_usage': (self.end_memory - self.start_memory) / 1024 / 1024, # MB
'cpu_percent': psutil.cpu_percent(),
'timestamp': datetime.now().isoformat()
}
def save_metrics(self, file_path: str):
"""保存性能指标"""
with open(file_path, 'w') as f:
json.dump(self.metrics, f, indent=2)
def get_metrics(self) -> dict:
"""获取性能指标"""
return self.metrics
# pytest插件集成
# conftest.py
import pytest
from utils.performance_monitor import TestPerformanceMonitor
@pytest.fixture(scope="session", autouse=True)
def performance_monitor():
"""性能监控fixture"""
monitor = TestPerformanceMonitor()
monitor.start_monitoring()
yield monitor
monitor.stop_monitoring()
monitor.save_metrics('performance_metrics.json')
# 输出性能指标到环境变量（供CI/CD使用）
metrics = monitor.get_metrics()
print(f"EXECUTION_TIME={metrics['execution_time']:.2f}")
print(f"MEMORY_USAGE={metrics['memory_usage']:.2f}")2. 测试并行化优化
# GitHub Actions - 并行优化
jobs:
test:
runs-on: ubuntu-latest
strategy:
matrix:
test-group: [1, 2, 3, 4]
steps:
- uses: actions/checkout@v3
- name: 设置Python环境
uses: actions/setup-python@v4
with:
python-version: '3.9'
- name: 安装依赖
run: |
pip install -r requirements.txt
pip install pytest-xdist pytest-split
- name: 运行测试组 ${{ matrix.test-group }}
run: |
pytest --splits 4 --group ${{ matrix.test-group }} --alluredir=allure-results-${{ matrix.test-group }}
- name: 上传测试结果
uses: actions/upload-artifact@v3
with:
name: test-results-${{ matrix.test-group }}
path: allure-results-${{ matrix.test-group }}/
merge-reports:
needs: test
runs-on: ubuntu-latest
steps:
- name: 下载所有测试结果
uses: actions/download-artifact@v3
- name: 合并报告
run: |
mkdir -p allure-results
cp -r test-results-*/allure-results-*/* allure-results/
allure generate allure-results -o allure-report --clean总结
CI/CD集成与持续测试是现代软件开发的重要组成部分，通过合理的配置和优化，我们可以实现：
自动化程度高：代码提交即触发测试 反馈速度快：几分钟内获得测试结果 覆盖范围广：多环境、多场景全面覆盖 质量保障强：每次变更都经过验证 团队协作好：实时通知，信息透明
关键要点回顾：
- 分层测试：根据测试金字塔原理设计测试策略
- 工具选择：选择适合团队的CI/CD工具
- 并行优化：通过并行执行提高效率
- 通知机制：及时反馈测试结果
- 性能监控：持续优化测试执行性能
下一篇文章，我们将总结接口自动化测试的最佳实践，敬请期待！
推荐阅读