CI/CD

CI/CD stands for Continuous Integration, Continuous Delivery, and Continuous Deployment. It is the backbone of modern DevOps — automating the build, test, and release process so that code changes flow from a developer's machine to production reliably and repeatably. A CI/CD pipeline is a series of automated stages that code passes through: build, test, package, deploy.

GitHub Actions is GitHub's native CI/CD platform. Workflows are defined in YAML files under .github/workflows/. It is tightly integrated with GitHub — pull request checks, deployments, issue automation, and more — and only works with GitHub repositories.

Core concepts

Sample workflow

# .github/workflows/ci.yml
name: CI Pipeline

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

permissions:
  contents: read
  id-token: write          # needed for OIDC

jobs:
  test:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        node-version: [20, 22, 24]
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v4
        with:
          node-version: ${{ matrix.node-version }}
          cache: npm
      - run: npm ci
      - run: npm test

  build:
    needs: test
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v6
      - uses: actions/setup-node@v4
        with:
          node-version: 22
          cache: npm
      - run: npm ci
      - run: npm run build
      - uses: actions/upload-artifact@v4
        with:
          name: dist
          path: dist/

  deploy:
    needs: build
    runs-on: ubuntu-latest
    environment: production
    if: github.ref == 'refs/heads/main'
    steps:
      - uses: actions/download-artifact@v4
        with:
          name: dist
      - uses: aws-actions/configure-aws-credentials@v5
        with:
          role-to-assume: arn:aws:iam::123456789:role/deploy
          aws-region: us-east-1
      - run: aws s3 sync dist/ s3://my-bucket/

GitLab CI/CD is deeply integrated into GitLab and only works with GitLab repositories. Pipelines are defined in .gitlab-ci.yml at the repository root. GitLab CI is one of the most feature-complete CI/CD systems available — it handles everything from linting to production deployments within a single platform.

Core structure

Pipeline keyword reference

GitLab Runners

GitLab CI executes jobs on Runners. Runners can be shared (available to all projects), group (available to a group), or project-specific. Executors include Docker, Shell, Kubernetes, Instance, Docker Autoscaler, and VirtualBox. On GitLab.com, shared runners use Google Cloud VMs. The legacy Docker Machine executor is deprecated (GitLab 17.5) and scheduled for removal in GitLab 20.0 — use the Docker Autoscaler executor with the Fleeting plugin architecture instead.

Advanced pipelines

• Parent-child pipelines — a parent pipeline triggers child pipelines defined in separate YAML files. Each child is an independent pipeline with its own stages. Great for monorepos.
• Multi-project pipelines — trigger pipelines in other GitLab projects using trigger: project: other/repo. Used for cross-repository deployment orchestration.
• Auto DevOps — GitLab auto-detects the language and applies a default pipeline (build, test, code quality, SAST, deploy to K8s). Zero-config CI/CD for simple projects.

Sample .gitlab-ci.yml

# .gitlab-ci.yml
stages:
  - build
  - test
  - deploy

variables:
  DOCKER_IMAGE: $CI_REGISTRY_IMAGE:$CI_COMMIT_SHORT_SHA

build:
  stage: build
  image: docker:27
  services:
    - docker:27-dind
  script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker build -t $DOCKER_IMAGE .
    - docker push $DOCKER_IMAGE
  rules:
    - if: $CI_PIPELINE_SOURCE == "merge_request_event"
    - if: $CI_COMMIT_BRANCH == "main"

test:unit:
  stage: test
  image: node:22
  needs: ["build"]
  script:
    - npm ci
    - npm run test:unit
  artifacts:
    reports:
      junit: junit.xml

test:integration:
  stage: test
  image: node:22
  needs: ["build"]
  services:
    - postgres:16
  variables:
    POSTGRES_DB: testdb
    POSTGRES_PASSWORD: testpass
  script:
    - npm ci
    - npm run test:integration

deploy:staging:
  stage: deploy
  needs: ["test:unit", "test:integration"]
  environment:
    name: staging
    url: https://staging.example.com
  script:
    - kubectl set image deployment/app app=$DOCKER_IMAGE
  rules:
    - if: $CI_COMMIT_BRANCH == "main"

deploy:production:
  stage: deploy
  needs: ["deploy:staging"]
  environment:
    name: production
    url: https://example.com
  script:
    - kubectl set image deployment/app app=$DOCKER_IMAGE
  rules:
    - if: $CI_COMMIT_BRANCH == "main"
      when: manual

Jenkins is the most widely deployed CI/CD server in the world. It is fully self-hosted, platform-agnostic, and extensible via thousands of plugins. Jenkins can be triggered by webhooks from GitHub, GitLab, Bitbucket, Gitea, or virtually any Git provider. This flexibility comes at the cost of significant maintenance overhead — you manage the server, plugins, security patches, and scaling yourself.

Pipeline types

Key features

• Credentials store — built-in encrypted secret storage. Supports username/password, SSH keys, secret text, certificates, and cloud credentials. Scoped to global, folder, or pipeline level.
• Shared libraries — reusable Groovy code stored in a Git repo and loaded into pipelines with @Library('my-lib'). Centralizes common build logic across many Jenkinsfiles.
• Blue Ocean UI — a pipeline visualization interface that shows stage-by-stage progress, parallel branches, and log output. Note: Blue Ocean is in maintenance-only mode and will not receive new features. Consider the Pipeline Graph View plugin as a lighter alternative.
• Jenkins X — a Kubernetes-native flavor of Jenkins designed for cloud-native CI/CD. Uses Tekton pipelines under the hood, with GitOps-based promotion between environments. Jenkins X has a steep learning curve and relatively low adoption compared to ArgoCD/FluxCD for the GitOps use case.

Sample Jenkinsfile (Declarative)

// Jenkinsfile
pipeline {
    agent { docker { image 'node:22' } }

    environment {
        REGISTRY = credentials('docker-registry')
        IMAGE    = "myapp:${env.BUILD_NUMBER}"
    }

    stages {
        stage('Install') {
            steps {
                sh 'npm ci'
            }
        }
        stage('Test') {
            parallel {
                stage('Unit Tests') {
                    steps {
                        sh 'npm run test:unit'
                    }
                    post {
                        always {
                            junit 'test-results/unit/*.xml'
                        }
                    }
                }
                stage('Lint') {
                    steps {
                        sh 'npm run lint'
                    }
                }
            }
        }
        stage('Build') {
            steps {
                sh "docker build -t ${IMAGE} ."
                sh "docker push ${IMAGE}"
            }
        }
        stage('Deploy') {
            when {
                branch 'main'
            }
            input {
                message 'Deploy to production?'
                ok 'Deploy'
            }
            steps {
                sh "kubectl set image deployment/app app=${IMAGE}"
            }
        }
    }

    post {
        failure {
            slackSend channel: '#ci', message: "Build failed: ${env.JOB_NAME} #${env.BUILD_NUMBER}"
        }
        success {
            slackSend channel: '#ci', message: "Build passed: ${env.JOB_NAME} #${env.BUILD_NUMBER}"
        }
    }
}

CircleCI is a cloud-native CI/CD platform that works with GitHub, GitLab, and Bitbucket. Pipelines are defined in .circleci/config.yml. CircleCI is known for fast builds, powerful caching, and orbs — reusable configuration packages that simplify common tasks.

Core concepts

Sample config.yml

# .circleci/config.yml
version: 2.1

orbs:
  node: circleci/node@7.2
  docker: circleci/docker@2.6

executors:
  default:
    docker:
      - image: cimg/node:22.14

jobs:
  test:
    executor: default
    parallelism: 4
    steps:
      - checkout
      - node/install-packages
      - run:
          name: Run tests
          command: |
            TESTS=$(circleci tests glob "tests/**/*.test.js" | circleci tests split)
            npx jest $TESTS

  build-and-push:
    executor: default
    steps:
      - checkout
      - setup_remote_docker:
          version: default
      - docker/build:
          image: myapp
          tag: $CIRCLE_SHA1
      - docker/push:
          image: myapp
          tag: $CIRCLE_SHA1

workflows:
  ci:
    jobs:
      - test
      - build-and-push:
          requires: [test]
          context: docker-credentials
          filters:
            branches:
              only: main

Azure Pipelines is the CI/CD component of Azure DevOps. It supports YAML-based pipelines (azure-pipelines.yml) and works with Azure Repos, GitHub, and Bitbucket. Deeply integrated with the Azure ecosystem — Key Vault, Container Registry, App Services, AKS — but also works well for non-Azure deployments.

Core concepts

Classic vs YAML pipelines

Azure DevOps has two pipeline systems: Classic (GUI-based, release pipelines with drag-and-drop stages) and YAML (code-as-config, version-controlled). Microsoft recommends YAML for new pipelines. Classic pipelines are not formally deprecated but are in maintenance mode — all new pipeline features and security improvements are YAML-only. Classic pipelines also lack features like templates and multi-repo triggers.

Sample azure-pipelines.yml

# azure-pipelines.yml
trigger:
  branches:
    include: [main]

pool:
  vmImage: ubuntu-latest

variables:
  - group: production-secrets
  - name: imageTag
    value: $(Build.BuildId)

stages:
  - stage: Build
    jobs:
      - job: BuildApp
        steps:
          - task: NodeTool@0
            inputs:
              versionSpec: '22.x'
          - script: |
              npm ci
              npm run build
              npm test
            displayName: 'Install, build, test'
          - task: Docker@2
            inputs:
              command: buildAndPush
              repository: myapp
              containerRegistry: acr-connection
              tags: $(imageTag)

  - stage: Deploy
    dependsOn: Build
    condition: succeeded()
    jobs:
      - deployment: DeployProd
        environment: production
        strategy:
          runOnce:
            deploy:
              steps:
                - task: KubernetesManifest@1
                  inputs:
                    action: deploy
                    kubernetesServiceConnection: aks-connection
                    manifests: k8s/*.yml
                    containers: myapp:$(imageTag)

Gitea Actions is Gitea's built-in CI/CD system, intentionally designed to be compatible with GitHub Actions. Workflow files go in .gitea/workflows/ (also supports .github/workflows/ for migration ease). It uses the same YAML syntax, supports many GitHub Actions marketplace actions, and runs on Act Runner (self-hosted).

Key details

Differences from GitHub Actions

• No cloud-hosted runners — you must self-host Act Runner instances
• Limited marketplace — most actions work but some GitHub-specific ones (like github-script) need alternatives
• No environment protection rules — deployment environments with required reviewers are not available in most Gitea versions
• No OIDC provider (yet) — Gitea does not yet issue OIDC tokens for workload identity federation with cloud providers (you must use stored secrets). OIDC token support is under active development.
• Workflow path — prefers .gitea/workflows/ but also reads .github/workflows/

GitOps is a different paradigm from traditional CI/CD. Instead of pipelines pushing changes to infrastructure, GitOps uses Git as the single source of truth and operators running inside the cluster that continuously reconcile the desired state (in Git) with the actual state (in the cluster). This is the pull model vs the traditional push model.

ArgoCD

ArgoCD is the most popular GitOps tool for Kubernetes. It watches a Git repository containing Kubernetes manifests (plain YAML, Helm charts, or Kustomize overlays) and automatically syncs the cluster to match.

FluxCD

FluxCD is the CNCF-graduated GitOps toolkit. It takes a more modular, composable approach than ArgoCD — each concern is handled by a separate controller.

• GitRepository CRD — watches a Git repo and makes it available to other controllers
• Kustomization CRD — reconciles Kustomize overlays or plain manifests from a GitRepository
• HelmRelease CRD — manages Helm chart releases declaratively
• Notification controller — sends alerts to Slack, Teams, webhooks when reconciliation events occur
• Image automation — watches container registries and auto-updates image tags in Git

FluxCD is fully CLI-driven (no web UI out of the box) and designed for teams that prefer a Unix-philosophy of small, composable tools.

Octopus Deploy

Octopus Deploy is a deployment automation platform that sits downstream of your CI tool. It is not strictly GitOps but handles deployment orchestration — the CD in CI/CD. CI tools (Jenkins, GitHub Actions, GitLab CI) build and test; Octopus handles the release and deployment.

• Tenanted deployments — deploy the same application differently per tenant (customer, region)
• Runbooks — operational tasks (database migrations, cert renewals) managed alongside deployments
• Deployment targets — Kubernetes, Azure, AWS, SSH, Windows servers, offline drops
• Variable scoping — variables scoped by environment, tenant, role, channel, and deployment step
• Lifecycles — define promotion paths (Dev → Staging → Production) with approval gates

Pull vs Push model

Every CI/CD platform handles secrets differently, but the principle is the same: never hardcode credentials in your pipeline files. Secrets should be injected at runtime from a secure store, masked in logs, and rotated regularly.

Native secrets by platform

HashiCorp Vault / OpenBao integration

For teams that need dynamic, short-lived secrets, HashiCorp Vault (or its MPL-2.0-licensed fork OpenBao, maintained by the Linux Foundation) is the gold standard. Instead of storing a static database password in CI, Vault generates a temporary credential that expires after the pipeline completes. OpenBao is API-compatible with Vault and includes features formerly exclusive to Vault Enterprise, such as namespaces.

OIDC federation

OIDC federation eliminates stored cloud credentials entirely. The CI provider issues a signed JWT, and the cloud provider validates it and returns a short-lived access token.

CI Provider (GitHub/GitLab)          Cloud Provider (AWS/GCP/Azure)
        |                                        |
        |-- 1. issues JWT (signed) ------------->|
        |                                        |-- 2. validates JWT signature
        |                                        |-- 3. checks claims (repo, branch, etc.)
        |<-- 4. returns short-lived token -------|
        |                                        |
        |-- 5. uses token for API calls -------->|

• GitHub Actions — use permissions: id-token: write + aws-actions/configure-aws-credentials with role-to-assume
• GitLab CI — use id_tokens: keyword to generate OIDC tokens, configure cloud provider trust
• Azure DevOps — use workload identity federation on service connections

Secret scanning in pipelines

• gitleaks — scans git history and staged changes for hardcoded secrets (API keys, passwords, tokens). Run as a pre-commit hook or CI step.
• trufflehog — entropy-based and regex-based secret detection. Scans git repos, S3 buckets, and filesystem paths. Verifies found secrets against live APIs.
• Pre-commit hooks — catch secrets before they enter Git history. Prevention is better than detection.

Not every CI/CD tool works with every Git platform. Some are tightly coupled (native only), while others are agnostic. This matrix shows which combinations work.

Compatibility grid

Lock-in rules

Combos that work (despite seeming odd)

• CircleCI + GitLab — fully supported. CircleCI added GitLab support as a first-class integration.
• Azure Pipelines + GitHub — very common. Many teams use Azure DevOps for CI/CD with GitHub repos.
• Jenkins + literally anything — if it has a webhook or can be polled, Jenkins can build it.
• ArgoCD + Gitea — ArgoCD just needs a Git URL. Works with any Git server including Gitea.

Combos that don't work

• GitLab CI from GitHub — nope. .gitlab-ci.yml is meaningless on GitHub.
• GitHub Actions from GitLab — nope. .github/workflows/ is ignored on GitLab.
• Gitea Actions from GitHub — nope (the reverse works though: GitHub workflow files run on Gitea).
• Bitbucket Pipelines from anywhere except Bitbucket — nope.

A side-by-side view of every CI/CD platform covered in this guide.

Platform comparison table

When to use what

Production CI/CD setup checklist. Work through these items when building or auditing your pipeline infrastructure.

Pipeline fundamentals

• Pipeline config is version-controlled — all CI/CD configuration lives in Git alongside the application code. No GUI-only pipeline definitions.
• Stages are clearly separated — build, test, security scan, deploy are distinct stages. Each stage has a clear purpose and failure mode.
• Tests run before deploy — no deployment can happen without passing unit tests, integration tests, and linting. This is non-negotiable.
• Artifacts are immutable — build once, deploy everywhere. The same artifact (container image, binary) is promoted through environments. Never rebuild per environment.
• Pipeline DAG is optimized — use needs: (GitLab) or job dependencies (GitHub Actions) to skip unnecessary waits. Parallel where possible.

Security

• No hardcoded secrets — all credentials come from a secret store (native CI secrets, Vault, Key Vault). Zero secrets in YAML files, Dockerfiles, or source code.
• OIDC federation for cloud access — eliminate long-lived API keys. Use OIDC tokens for AWS, GCP, and Azure authentication from CI.
• Secret scanning enabled — run gitleaks or trufflehog in CI. Block merges if secrets are detected. Use pre-commit hooks as a first line of defense.
• Minimal permissions — CI tokens and service accounts have the least privilege needed. GitHub Actions permissions: block restricts GITHUB_TOKEN. Vault policies are narrow.
• Dependency scanning — run SAST, SCA (Dependabot, Snyk, Trivy) in the pipeline. Fail builds on critical vulnerabilities.
• Signed artifacts — sign container images (cosign/sigstore) and verify signatures before deployment.

Runners & infrastructure

• Runners are ephemeral — use fresh containers or VMs per build. No shared state between builds. Prevents leaking secrets or artifacts across pipelines.
• Runner autoscaling configured — scale runners based on queue depth. Don't over-provision or leave developers waiting for builds.
• Caching strategy defined — cache dependency directories (node_modules, .m2, pip cache) to speed up builds. Invalidate caches on lockfile changes.
• Build times monitored — track pipeline duration over time. Alert when builds exceed SLO. Investigate and optimize slow steps.

Deployment & environments

• Environment promotion path defined — Dev → Staging → Production (or similar). Each promotion has clear criteria (tests pass, approval obtained).
• Production deploys require approval — use environment protection rules (GitHub), manual gates (GitLab when: manual), or approval workflows (Azure DevOps, Octopus Deploy).
• Rollback mechanism tested — know how to roll back before you need to. GitOps: git revert. Traditional: re-deploy previous artifact. Test rollback regularly.
• Deployment notifications configured — Slack, Teams, or email notifications for deploy start, success, and failure. Include commit info and actor.
• Feature flags for risky changes — decouple deploy from release. Ship disabled features and enable them independently of deployment.

Observability & governance

• Pipeline audit trail — every deployment is logged with who triggered it, what was deployed, and when. Required for compliance (SOC2, ISO 27001).
• Branch protection enforced — main/production branches require passing CI, code review, and no direct pushes. CI is a gate, not a suggestion.
• Pipeline-as-code reviewed — changes to CI/CD config go through code review just like application code. A malicious pipeline change can exfiltrate secrets.
• Disaster recovery plan — if your CI/CD platform goes down, how do you deploy? Have a manual deployment runbook for emergencies.