Dynamic Provisioning with Jenkins

In this article we will learn how to dynamically provision our slave nodes

What is Jenkins

Jenkins comes under Continuous Integration and Continuous Delivery field. Manual ways of doing tasks is not good in today’s world. Manual operations always has a delay. It improves Software Development Cycle. How agile the releases are will determine the quality of the company. Our main motto is to remove manual work. For every integration in Jenkins we have plugins.

Jobs in Jenkins: For every step we have job and if you want to execute any job then that it is known as Build.

Jenkins provides every job a personal workspace. Whatever the work done by that job is saved inside that workspace.

Jenkins server provides three ways to interact: WebUI, CLI and Pipeline as a Code.

Jenkins is purely build on java . It works on the top of Java so before installing Jenkins we must have jdk installed in our system.

For this demo I am using RHEL 8. By default Jenkins works on port 8080.

Also it works on top of java so you can confirm it by searching java process in your VM.

netstat -tnlp | grep java

Jenkins server is running

Now we will use WebUI to interact with the server.

http://your_ip_address:8080

Dashboard

For installing plugins go to Manage Jenkins -> Mange Plugins, there you can download various plugins according to the requirement.

Distributed Jenkins Cluster

Based on jobs we are distributing the cluster. This is known as Jenkins Distributed Cluster.

Topology of Jenkins Cluster

No need to install Jenkins in slave nodes. Where you install Jenkins that node works as master as well as slave. Therefore it is known as Single Node Cluster.

But in this task we want to dynamically provision the slave node for some job, after job is executed the OS will be dynamically eliminated. Here we will see how we can integrate Jenkins , Maven and docker to solve this challenge.

We can restrict the particular job in particular node.

Maven: In java we include the package and use them. While compiling the code if there are some packages that have dependencies then we have to download these dependencies too. Rather doing it manually we can do automation, that will go to Maven Central (a centralized repository) and will download the packages with right version and right dependencies. We only need to tell maven which software and version you want to download and maven will download all the dependencies of the required software.

To use Maven with Jenkins download Maven Integration Plugin by going to Manage Jenkins

Benefits of Dynamic Provisioning

Consider if a job is running in slave when data is available through git. But if the developer doesn’t push the code in the GitHub then these jobs will be running and will consume RAM and CPU, and will effect the company in a long run.

So to solve this challenge what can we do?

We can ask the master node only to launch the slave when the code is pushed by the developer in the GitHub. By this the slave will run the code and test it. After testing it will be removed. In this way we reduce the consumption of RAM and CPU.

Which technology we can use to dynamically provision the OS on demand basis?

One can have this setup on Cloud or Virtual machine but there it will take time for setting the environment. This problem is easily solved by container technology where we can launch any OS within seconds. Here we will use docker tool.

For integrating docker with Jenkins we need to install Docker Plugin. Only way to manage docker is locally. It cannot be managed remotely.

Here we want that Jenkins Master must be able to start docker containers. So for that we need to add the ip address and random port in /usr/lib/systemd/system/docker.service file.

Execute the commands shown below

1. systemctl daemon-reload
2. systemctl restart docker
3. Stop the firewall services in docker host

Now go to Manage Jenkins -> Manage Nodes and Clouds -> Go to Configure Clouds -> Add a new cloud -> Docker

Note: Docker will be shown only if you have downloaded the docker plugin

After setting up Docker Host URI click on Test Connection to check the connectivity.

Docker Agent Template

Here we have specified he Labels. You can also setup other services as per your expertise in the docker field but this is a required setup you must have.

Here I have used the docker image vimal13/jenkins-slave-maven created by Vimal Daga, he is also my mentor for Jenkins training. In this image there is pre maven setup.

Now create a job in Jenkins.

Now if we want that Jenkins must pull the code from the GitHub then we have to install the GitHub Plugin.

Specify the branch

Few things before we proceed further.

What is our plan ?

1. Developer push the code in github
2. Master launch the slave dynamically
3. Jenkins will go to the repository and will pull the code
4. Deploy the code in container
5. Maven will download packages with right dependencies
6. Maven will compile the code
7. Maven will test the code using JUnit
8. Generate the report using Surefire
9. Create the package

Build Steps

Maven option will prompt in build action only if you have downloaded the Maven Integration Tool Plugin.

In Post Build Action

We are archiving the artifact.

Now let’s run the Job

Can Check that last Build was successful

I have done testing on this many times that’s why it is showing Build History.

I hope you have liked this article.

Thanks for reading.