Understanding the Pipeline structure in Azure DevOps

Understanding the Pipeline structure in Azure DevOps

In Azure DevOps, a pipeline is the top-level structure that automates software delivery. Within a pipeline, you create stages, jobs, steps, and define tasks to execute different stages of your CI/CD workflow. Additionally, deployment strategies and lifecycle hooks enable fine-grained control over deployments and can manage different release scenarios.

1. Pipeline

A pipeline is the highest level in the hierarchy, comprising stages that represent phases in the CI/CD process. Each stage can have multiple jobs.

Example Structure:

xxxxxxxxxx
12
1
pipeline:
2
  name: 'MyPipeline'
3
stages:
4
- stage: Build
5
  jobs:
6
   - job: BuildJob
7
    steps: 
8
   - script: echo "Building the project"
9
- stage: Deploy
10
  jobs:
11
   - deployment: DeployToProd
12
    environment: 'Production'

2. Stage

A stage groups multiple jobs, representing logical units of work (e.g., build, test, deploy). Stages are linked together in a sequential or parallel flow.

Example:

xxxxxxxxxx
11
1
stages:
2
- stage: Build
3
  jobs:
4
   - job: BuildJob
5
    steps:
6
   - script: echo "Building the project"
7
- stage: Test
8
  jobs:
9
   - job: TestJob
10
    steps:
11
   - script: echo "Running tests"

3. Job

Jobs are individual tasks performed on a single agent within a stage. A job contains steps that execute a sequence of commands or tasks.

Example:

xxxxxxxxxx
3
1
- job: BuildJob
2
  steps:
3
   - script: echo "Building the application"

4. Steps

Steps define individual actions within a job. They can be simple commands or complex tasks with dependencies.

Example:

xxxxxxxxxx
5
1
steps:
2
- script: dotnet restore
3
  displayName: 'Restore Dependencies'
4
- script: dotnet build --configuration Release
5
  displayName: 'Build Solution'

5. Tasks

Tasks are predefined actions provided by Azure DevOps or custom scripts used in a pipeline step.

Example:

xxxxxxxxxx
5
1
steps:
2
- task: UseDotNet@2
3
  inputs:
4
    packageType: 'sdk'
5
    version: '6.x'

6. Deployment Strategies

Azure DevOps provides various deployment strategies to handle different deployment patterns:

a. RunOnce

Deploy once to a specific environment without any further rollbacks or retries.

Example:

xxxxxxxxxx
6
1
deployment: DeployToProd
2
  strategy:
3
    runOnce:
4
      deploy:
5
        steps:
6
       - script: echo "Deploying to Production"

b. Rolling Deployment

Gradually rolls out a deployment to a subset of instances before completing the entire deployment.

Example:

xxxxxxxxxx
6
1
deployment: DeployToProd
2
  strategy:
3
    rolling:
4
      batchCount: 5
5
      preDeploy: true
6
      postDeploy: true

c. Canary Deployment

Deploys to a small subset of users or servers first, ensuring minimal risk before scaling up.

Example:

xxxxxxxxxx
5
1
deployment: DeployToProd
2
  strategy:
3
    canary:
4
      count: 2
5
      postRouteTraffic: true

7. Lifecycle Hooks

Lifecycle hooks allow you to define custom actions at specific stages in the deployment process, such as before or after deploying changes.

a. preDeploy

Execute actions before deploying changes.

Example:

xxxxxxxxxx
5
1
deployment: DeployToProd
2
  strategy:
3
    canary:
4
      count: 2
5
      preDeploy: true

b. deploy

Perform actions during the deployment.

Example:

xxxxxxxxxx
5
1
deployment: DeployToProd
2
  strategy:
3
    canary:
4
      count: 2
5
      deploy: true

c. routeTraffic

Divert traffic to the newly deployed version.

Example:

xxxxxxxxxx
5
1
deployment: DeployToProd
2
  strategy:
3
    canary:
4
      count: 2
5
      routeTraffic: true

d. postRouteTraffic

Post-traffic monitoring after deploying changes.

Example:

xxxxxxxxxx
5
1
deployment: DeployToProd
2
  strategy:
3
    canary:
4
      count: 2
5
      postRouteTraffic: true

e. on:failure

Run actions if deployment fails.

Example:

xxxxxxxxxx
7
1
deployment: DeployToProd
2
  strategy:
3
    canary:
4
      count: 2
5
      on:failure:
6
        steps:
7
       - script: echo "Sending failure notification"

f. on:success

Run actions if deployment succeeds.

Example:

xxxxxxxxxx
7
1
deployment: DeployToProd
2
  strategy:
3
    canary:
4
      count: 2
5
      on:success:
6
        steps:
7
       - script: echo "Deployment completed successfully"

Example of Complete Deployment Pipeline

xxxxxxxxxx
22
1
pipeline:
2
  name: 'MyPipeline'
3
stages:
4
- stage: Build
5
  jobs:
6
   - job: BuildJob
7
    steps: 
8
   - script: echo "Building the project"
9
- stage: Deploy
10
  jobs:
11
   - deployment: DeployToProd
12
    environment: 'Production'
13
    strategy:
14
      canary:
15
        count: 2
16
        preDeploy: true
17
        deploy: true
18
        routeTraffic: true
19
        postRouteTraffic: true
20
        on:failure:
21
          steps:
22
         - script: echo "Failure handling"

Summary Table of Pipeline Structure in Azure DevOps