Azure Batch (Service) Automation using AzureDevOps CI/CD pipeline

5 min readJun 26, 2020

One of my current projects is working with Azure Batch, which we use to run an to processing large calculations. I saw most of the developers were deploying this application via manual process. Every-time they have to run Microsoft .net solution on their machines to generate zip file, manually set the version, and do this every time we needed a push to our Dev or prod environments.

I took the opportunity to automate this entire process end to end. Currently not much documentation available pertaining to straight forward approach, Hope this will help others as well.

Agenda behind writing this article is to achieve Azure batch service deployment along with required packages. Not many articles available over internet in order to achieve CI/CD automation using AzureDevOps pipeline.

My goal was to automate deployment, and to do this, we need to look at Batch clients. Microsoft splits the API structure into two different clients:

Management: Manage Batch accounts, storage, and account keys
Service: Manage compute nodes, pools, tasks and jobs.

Requirements

We need a couple of Azure Resources to get this to work.

Azure Batch Account
Azure Storage Account
Azure Batch Account : while creating Azure batch account it needs to be followed in below work flow.

New Azure batch account creation
New Application creation
Application package assignment
New Job creation
New job schedule creation
Nodes restart

In order to achieve this i have created few ARM templates & 1 zip file (with .exe — dt.zip) which will does our work seamlessly.

all these templates have been uploaded at:

As a first step i have created a Build pipeline

Click on 1published artifact and make sure it has all required files

Now lets start creating a Release Pipeline, as a part of this step we are going to achieve in following order

a. Need to update variables

b. Create new Azure storage

c. Create new Azure batch

d. Create new Application

e. Upload New application package

f. Create new job

g. Create new job schedule

h. Restart nodes

a. As a part of updating variables this create a new release job and click on Variables and update keyvalue pair based on your Org requirements.

Make sure you update : Resource group name which is related with your Azure.

resourcegroupname
batchapplicationversion: One of the reason for using this variable is because everytime i generate a new .exe file i want to CI/CD pipeline to pick a new version automatically as counter.
Location: this is the location where you want to deploy Azure Batch.

As once you trigger a new release it will start creating below components

Now lets look into step by step to show how it really creates in Azure

Storage account creation

2. AzureBlob file copy : this step will upload all required deployment and exe(zip) file to Blob storage.

3. New Azure Batch Service Deployment: this service will get deploy using “Deployment-application1.json” file.

4. Azure Batch Application creation: this will create a new application.

templateContainerUri $(templateContainerUri) -templateContainerSasToken $(templateContainerSasToken) -batchAccountName $(batchAccountName) -batchAccountPoolName $(batchAccountPoolName) -applicationStorageAccountName $(applicationStorageAccountName)

In Azure portal.

It will show recently uploaded zip file.

5. Azure batch application package creation: this will create and associate a new zip file to application, click on Pools → click on respective PoolID → go to application package & then you can see associated application and versioned zip file.