Implementing multi-job builds in Azure DevOps

Implementing multi-job builds in Azure DevOps

To implement multi-job builds in Azure DevOps, you can define multiple jobs within a single pipeline YAML file. Multi-job builds allow you to run different tasks in parallel or sequentially, providing flexibility for complex build and release scenarios.

Here’s a step-by-step guide.

1. Define a Basic Pipeline

Start by creating a pipeline YAML file (azure-pipelines.yml).

2. Structure for Multi-Job Builds

Azure DevOps organizes jobs into stages and jobs. Each job can run independently or depend on other jobs. Jobs contain steps, which define tasks to execute.

3. Example: Multi-Job Build Configuration

Here’s an example YAML file demonstrating a multi-job pipeline:

xxxxxxxxxx
40
1
trigger:
2
- main
3
pool:
4
  vmImage: 'ubuntu-latest'
5
stages:
6
- stage: Build
7
  displayName: "Build Stage"
8
  jobs:
9
   - job: Job1
10
    displayName: "Compile Code"
11
    steps:
12
   - task: UsePythonVersion@0
13
      inputs:
14
        versionSpec: '3.x'
15
   - script: |
16
        echo "Compiling code..."
17
        python --version
18
   - job: Job2
19
    displayName: "Run Unit Tests"
20
    dependsOn: Job1 # Runs after Job1
21
    steps:
22
   - script: |
23
        echo "Running unit tests..."
24
        pytest tests/
25
   - job: Job3
26
    displayName: "Linting Code"
27
    steps:
28
   - script: |
29
        echo "Linting code..."
30
        flake8 src/
31
- stage: Deploy
32
  displayName: "Deployment Stage"
33
  dependsOn: Build
34
  condition: succeeded()
35
  jobs:
36
   - job: DeployJob
37
    displayName: "Deploy to Environment"
38
    steps:
39
   - script: |
40
        echo "Deploying application..."

Key Features of the Example

1. Multiple Jobs: The Build stage contains three jobs: Job1, Job2, and Job3.

2. Dependencies:

   • Job2 depends on Job1 (sequential execution).

   • Job3 runs independently (parallel execution).

3. Conditions: The Deploy stage executes only if the Build stage succeeds.

4. Stages: Logical grouping of jobs into stages for better visualization.

4. Run Jobs in Parallel

Azure DevOps runs jobs in parallel by default unless a dependency is specified using dependsOn.

5. Configure Pool and Agent

Ensure the appropriate agent pool is specified for all jobs.

xxxxxxxxxx
2
1
pool:
2
  vmImage: 'ubuntu-latest'

6. Manage Output Between Jobs

To share data between jobs, use pipeline artifacts.

Example: Publish and Download Artifacts

xxxxxxxxxx
18
1
# Job1: Publish artifact
2
- job: Job1
3
  steps:
4
   - script: |
5
      echo "Building artifact..."
6
   - task: PublishBuildArtifacts@1
7
    inputs:
8
      pathToPublish: '$(Build.ArtifactStagingDirectory)'
9
      artifactName: 'drop'
10

11
# Job2: Download artifact
12
- job: Job2
13
  dependsOn: Job1
14
  steps:
15
   - task: DownloadBuildArtifacts@0
16
    inputs:
17
      artifactName: 'drop'
18
      downloadPath: '$(Pipeline.Workspace)'

7. Testing Locally

Use tools like the or validate syntax in Visual Studio Code with the Azure Pipelines extension.