CI/CD - Continuous Integration & Continuous Deployment¶

Overview¶

This directory contains comprehensive guides for CI/CD tools, pipelines, and best practices. Learn how to automate building, testing, and deploying applications using industry-standard tools.

What is CI/CD?¶

Continuous Integration (CI)¶

Automatically build, test, and merge code whenever developers push commits to a shared repository.

Benefits: - Detect errors early - Prevent integration issues - Maintain a deployable build - Improve code quality

Continuous Deployment (CD)¶

Automatically deploy applications after CI is successful.

Benefits: - Faster delivery - Reduced manual errors - Consistent deployments - Quick rollbacks

Learning Path¶

Beginner Level¶

CI/CD Fundamentals - Start here
CI/CD concepts and workflow
Pipeline stages
Best practices
Tools overview
GitHub Actions - Cloud-native CI/CD
Workflow automation
YAML configuration
Secrets management
Deployment examples

Intermediate Level¶

Jenkins - Traditional CI/CD server
Installation and setup
Pipeline configuration
Docker integration
Real-world projects
GitLab CI - Integrated CI/CD
.gitlab-ci.yml configuration
Runners and executors
Pipeline optimization
Azure DevOps - Microsoft ecosystem
Azure Pipelines
Release management
Integration with Azure services

Advanced Level¶

ArgoCD - GitOps for Kubernetes
Declarative deployments
Automated sync
Rollback strategies
Multi-cluster management
SonarQube - Code quality & security
Static code analysis
Security scanning
Quality gates
CI/CD integration

CI/CD Pipeline Stages¶

┌─────────────┐
│   Source    │  Developer pushes code
└──────┬──────┘
       │
┌──────▼──────┐
│    Build    │  Compile code, create artifacts
└──────┬──────┘
       │
┌──────▼──────┐
│    Test     │  Unit, integration, security tests
└──────┬──────┘
       │
┌──────▼──────┐
│   Deploy    │  Deploy to environment
└──────┬──────┘
       │
┌──────▼──────┐
│   Monitor   │  Track metrics and errors
└─────────────┘

Tools Comparison¶

CI/CD Platforms¶

Tool	Best For	Complexity	Cost	Cloud/Self-Hosted
GitHub Actions	GitHub repos, cloud-native	Low	Free tier	Cloud
GitLab CI	GitLab repos, integrated	Medium	Free tier	Both
Jenkins	Flexibility, plugins	High	Free	Self-hosted
Azure DevOps	Microsoft ecosystem	Medium	Free tier	Cloud
CircleCI	Fast builds, Docker	Low	Free tier	Cloud

Deployment Tools¶

Tool	Type	Best For	Learning Curve
ArgoCD	GitOps	Kubernetes deployments	Medium
FluxCD	GitOps	Kubernetes, Helm	Medium
Spinnaker	CD Platform	Multi-cloud deployments	High
Harness	CD Platform	Enterprise deployments	Medium

Code Quality Tools¶

Tool	Purpose	Language Support	Integration
SonarQube	Code quality, security	25+ languages	Excellent
Snyk	Dependency scanning	Multiple	Good
Trivy	Container scanning	N/A	Good
CodeClimate	Code quality	Multiple	Good

Quick Start Examples¶

GitHub Actions - Simple CI¶

name: CI Pipeline
on: [push]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Run tests
        run: npm test
      - name: Build
        run: npm run build

Jenkins - Declarative Pipeline¶

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                sh 'mvn clean package'
            }
        }
        stage('Test') {
            steps {
                sh 'mvn test'
            }
        }
        stage('Deploy') {
            steps {
                sh './deploy.sh'
            }
        }
    }
}

GitLab CI - Basic Pipeline¶

stages:
  - build
  - test
  - deploy

build:
  stage: build
  script:
    - npm install
    - npm run build

test:
  stage: test
  script:
    - npm test

deploy:
  stage: deploy
  script:
    - ./deploy.sh
  only:
    - main

Common Pipeline Patterns¶

1. Build → Test → Deploy¶

Code Push → Build Artifact → Run Tests → Deploy to Staging → Deploy to Production

2. Feature Branch Workflow¶

Feature Branch → CI Tests → Code Review → Merge to Main → Deploy

3. GitOps Workflow¶

Code Change → Build Image → Update Manifest → ArgoCD Syncs → Deployed

4. Blue-Green Deployment¶

Deploy to Green → Test Green → Switch Traffic → Keep Blue as Backup

5. Canary Deployment¶

Deploy to 10% → Monitor → Deploy to 50% → Monitor → Deploy to 100%

Best Practices¶

Pipeline Design¶

✅ Keep pipelines fast (< 10 minutes)
✅ Fail fast - run quick tests first
✅ Use caching for dependencies
✅ Parallelize independent jobs
✅ Use pipeline templates for consistency

Security¶

✅ Store secrets in secure vaults
✅ Scan dependencies for vulnerabilities
✅ Scan Docker images before deployment
✅ Use least privilege for service accounts
✅ Audit pipeline access and changes

Testing¶

✅ Unit tests in every pipeline
✅ Integration tests before deployment
✅ Security scans (SAST/DAST)
✅ Performance tests for critical paths
✅ Smoke tests after deployment

Deployment¶

✅ Use infrastructure as code
✅ Implement rollback strategies
✅ Deploy to staging first
✅ Use feature flags for gradual rollout
✅ Monitor deployments actively

Monitoring¶

✅ Track pipeline success rates
✅ Monitor deployment frequency
✅ Measure lead time for changes
✅ Track mean time to recovery (MTTR)
✅ Set up alerts for failures

Real-World Pipeline Example¶

Complete DevOps Flow¶

Developer commits code
    ↓
GitHub Actions triggers
    ↓
Build Docker image
    ↓
Run unit tests
    ↓
SonarQube code analysis
    ↓
Security scan (Trivy)
    ↓
Push image to registry
    ↓
Update Kubernetes manifest
    ↓
ArgoCD auto-syncs
    ↓
Deploy to cluster
    ↓
Prometheus monitors
    ↓
Grafana dashboards
    ↓
Slack notification

Troubleshooting Guide¶

Pipeline Fails at Build¶

Check build logs for errors
Verify dependencies are available
Check for syntax errors
Ensure correct build tool version
Verify environment variables

Tests Failing¶

Run tests locally first
Check test environment setup
Verify test data availability
Check for flaky tests
Review recent code changes

Deployment Fails¶

Check deployment logs
Verify credentials and permissions
Check target environment health
Verify image/artifact availability
Check network connectivity

Slow Pipelines¶

Identify bottleneck stages
Implement caching
Parallelize independent jobs
Optimize test execution
Use faster runners/agents

Integration Examples¶

Jenkins + Docker + Kubernetes¶

pipeline {
    agent any
    stages {
        stage('Build') {
            steps {
                sh 'docker build -t myapp:${BUILD_NUMBER} .'
            }
        }
        stage('Push') {
            steps {
                sh 'docker push myapp:${BUILD_NUMBER}'
            }
        }
        stage('Deploy') {
            steps {
                sh 'kubectl set image deployment/myapp myapp=myapp:${BUILD_NUMBER}'
            }
        }
    }
}

GitHub Actions + ArgoCD¶

name: Deploy
on:
  push:
    branches: [main]
jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - name: Build and Push
        run: |
          docker build -t myapp:${{ github.sha }} .
          docker push myapp:${{ github.sha }}
      - name: Update Manifest
        run: |
          sed -i 's|image:.*|image: myapp:${{ github.sha }}|' k8s/deployment.yaml
          git commit -am "Update image"
          git push

Metrics to Track¶

DORA Metrics (DevOps Research and Assessment)¶

Deployment Frequency
How often you deploy to production
Elite: Multiple times per day
Lead Time for Changes
Time from commit to production
Elite: Less than one hour
Mean Time to Recovery (MTTR)
Time to recover from failure
Elite: Less than one hour
Change Failure Rate
Percentage of deployments causing failure
Elite: 0-15%

Additional Resources¶

Official Documentation¶

Learning Resources¶

Directory Structure¶

CICD/
├── README.md                    # This file
├── 0-CICD-Fundamentals.md       # CI/CD fundamentals
├── GitHub-Actions/              # GitHub Actions guides
│   └── GitHub-Actions.md
├── Jenkins/                     # Jenkins setup and pipelines
│   ├── README.md
│   ├── Jenkins.md
│   └── java-maven-sonar-argocd-helm-k8s/
├── GitLab-CI/                   # GitLab CI configuration
│   ├── GitLab-CI.md
│   └── .gitlab-ci.yml
├── Azure-DevOps/                # Azure DevOps pipelines
│   └── Azure-DevOps.md
├── ArgoCD/                      # GitOps with ArgoCD
│   ├── README.md
│   └── [images]
└── SonarQube/                   # Code quality and security
    ├── README.md
    ├── 1-Server-Setup.md
    ├── 2-Project-Local-Scan.md
    └── 3-Docker-Local-Scan.md