Shrink your Azure Devops YAML pipelines

When you create your pipelines for your projects you probably have to setup some jobs with multiple configurations, depending on the different targeted environments (e.g., dev, staging, production and so on). These configurations can be repetitive and take time. In this tutorial I will show you how to optimize this process.

Context overview

Let’s take the example of a nightly build, where you want to check that all the configurations for your project still build after a day of work.

Here is a basic job with all the task you need to build for your project:

# job_build_app.yml
parameters:
  name: ''
  certificate: ''
  frameworkVersion: ''
  toolVersion: ''

jobs:
- job: Build_app_${{ parameters.name }}

  steps:
  - task: Bash@3
    inputs:
      targetType: 'inline'
      script: |
        echo This job will running on env ${{ parameters.name }} with framework version ${{ parameters.frameworkVersion }} and tool ${{ parameters.toolVersion }} with the certificat ${{ parameters.certificate }}
    
    # All the tasks you need to build the project...

And now the stage that define the nightly build with multiple environments:

# Basic nightly_builds.yml
stages:
- stage: Build
  jobs:
  - template: job_build_app.yml
    parameters:
      name: 'dev'
      certificate: 'certificat-dev'
      frameworkVersion: '5.0'
      toolVersion: '1.0'
      
  - template: job_build_app.yml
    parameters:
      name: 'staging'
      certificate: 'certificat-staging'
      frameworkVersion: '5.0'
      toolVersion: '1.0'

  - template: job_build_app.yml
    parameters:
      name: 'production'
      certificate: 'certificat-production'
      frameworkVersion: '5.0'
      toolVersion: '1.0'

As you can see in this sample, multiple parameters are repetitive:

• The name of the template used for each job (job_build_app.yml).
• The parameters frameworkVersion and toolVersion.

Writing your stage like this has a lot of drawbacks:

• It is useless: copy/pasting does not add any value.
• It does not help for the readability: a lot of lines are filled with uninteresting data.
• The maintenance is tedious: you will have to change the same value at multiple places.
• It is error-prone: you will have to make sure to replace every parameters with the same value. The use of variables in the template would however limit the sources of errors.

Optimization

With that in mind let’s see how we can optimize this. The idea is to iterate over the configurations for each environment with the same YAML template and mutualize the parameters that are duplicated.

Let’s create an intermediate template called: jobs_build_app.yml which will loop over the configurations of each job.

# jobs_build_app.yml
parameters:
  frameworkVersion: ''
  toolVersion: ''
  envs: {}

jobs:
- ${{ each env in parameters.envs }}:
  - template: job_build_app.yml
    parameters:
      name: ${{ env.name }}
      frameworkVersion: ${{ parameters.frameworkVersion }}
      toolVersion: ${{ parameters.toolVersion }}
      certificate: ${{ env.certificate }}

As you can see above the common parameters frameworkVersion and toolVersion are defined once and the template job_build_app.yml is setup just one time too.

The key point here is the line below, which is basically a for loop other the different environments: envs: {}:

${{ each env in parameters.envs }}

Let’s update our nightly build stage now:

# nightly_builds.yml
stages:
- stage: Build
  jobs:
  - template: jobs_build_app.yml
    parameters:
      frameworkVersion: '5.0'
      toolVersion: '1.0'
      envs:
        - env:
          name: 'dev'
          certificate: 'certificat-dev'
        - env:
          name: 'staging'
          certificate: 'certificat-staging'
        - env:
          name: 'production'
          certificate: 'certificat-production'

As you can see, this syntax does exactly the same than the first one but it’s more readable and straightforward.

Final touch

You have now the ability to simplify your YAML pipelines as you need. As usual you will find the sample source code in the associated Github repository.

Happy coding!

You liked this tutorial? Leave a star in the associated Github repository!

Do not hesitate to follow me on Twitter to not miss my next tutorial!