Deploying Jenkins with Docker and Terraform: A Comprehensive Guide

In modern software development, efficient CI/CD processes are crucial. Jenkins, Docker, and Terraform offer a robust solution for seamless CI environments. In this guide, we’ll deploy Jenkins using Docker containers managed by Terraform. Whether you’re a seasoned DevOps pro or an enthusiast, this tutorial provides a precise roadmap for scalable CI infrastructure. Let’s dive in.

We’ll start by establishing the groundwork. Creating the project structure and configuration files sets the foundation for deploying Jenkins with Docker and Terraform.

Setting Up the Project Directory

Create Project Directory: Begin by creating a dedicated directory for your Jenkins deployment project. Open your terminal or command prompt and execute the following command:

mkdir jenkins-docker-terraform

Navigate to the Project Directory:
Move into the newly created directory:

cd jenkins-docker-terraform

Creating Configuration Files

Now, let’s create the essential configuration files for our Jenkins deployment using Docker and Terraform. We hope Docker and Terraform is installed in your machine.

Dockerfile

The Dockerfile defines the specifications for building a Docker image containing the Jenkins server.

Create a file named Dockerfile in your project directory and add the following contents:

FROM jenkins/jenkins:lts-jdk17

USER root  # We need to install some required packages

# Install required packages
RUN apt-get update && \
    apt-get install -y \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg \
    lsb-release \
    software-properties-common

# Install Docker(optional)
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add - && \
    add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/debian $(lsb_release -cs) stable" && \
    apt-get update && \
    apt-get install -y docker-ce docker-ce-cli containerd.io

# Install Docker Compose(optional)
RUN curl -L "https://github.com/docker/compose/releases/download/1.29.2/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose && \
    chmod +x /usr/local/bin/docker-compose


COPY --chown=jenkins:jenkins plugins.txt /usr/share/jenkins/ref/plugins.txt
RUN ls -lsta /usr/share/jenkins/ref/plugins.txt

RUN jenkins-plugin-cli -f /usr/share/jenkins/ref/plugins.txt

List of Jenkins common plugins

Create a file named plugins.txt in your project directory and add the following plugins. You can customize this list based on your specific requirements:

# Core Functionality
workflow-aggregator
git
pipeline
docker-workflow

# Code Quality and Analysis
checkstyle
findbugs
pmd
jacoco
cobertura
sonar

# Notifications and Reporting
email-ext
slack
htmlpublisher
junit

# Version Control
git
github
subversion

# Deployment and Provisioning
ansible
docker-build-publish
kubernetes
terraform

# Monitoring and Metrics
metrics
performance

# User Authentication and Authorization
matrix-auth
ldap
role-strategy

# Miscellaneous
credentials-binding
envinject
build-timeout
pipeline-github-lib

Terraform Configuration

The Terraform configuration file defines the infrastructure resources required for deploying Jenkins.

Create a file named main.tf in your project directory and add the following contents:

terraform {
  required_providers {
    docker = {
      source  = "kreuzwerker/docker"
      version = "~> 3.0.1"
    }
  }
}

provider "docker" {}

resource "docker_image" "jenkins" {
  name         = "jenkins-gundard:latest"
  build {
    context    = "."
    dockerfile = "Dockerfile"
  }
}

resource "docker_volume" "jenkins_data" {
  name = "jenkins_data"
}

resource "docker_container" "jenkins" {
  name   = "jenkins"
  image  = docker_image.jenkins.name
  ports {
    internal = 8080
    external = 8080
  }
  ports {
    internal = 50000
    external = 50000
  }
  restart = "unless-stopped"

  volumes {
    container_path  = "/var/jenkins_home"
    host_path       = "/home/arif/jenkins_data"
    read_only       = false
  }
  
}

Let’s break down the provided Terraform configuration code into chunks and describe each part:

Terraform Configuration

This block specifies the required provider for Docker and its version. It ensures that Terraform uses the correct Docker provider when managing Docker-related resources.

terraform {
  required_providers {
    docker = {
      source  = "kreuzwerker/docker"
      version = "~> 3.0.1"
    }
  }
}

Docker Provider

This block configures the Docker provider, indicating that Terraform will interact with Docker to manage container-related resources.

provider "docker" {}

Docker Image Resource

This block defines a Docker image resource named “Jenkins”. It specifies the name of the Docker image to build (jenkins-gundard:latest) using the Dockerfile present in the current directory (.).

resource "docker_image" "jenkins" {
  name         = "jenkins-gundard:latest"
  build {
    context    = "."
    dockerfile = "Dockerfile"
  }
}

Docker Volume Resource

This block defines a Docker volume resource named “jenkins_data”. It creates a Docker volume with the name “jenkins_data”.

resource "docker_volume" "jenkins_data" {
  name = "jenkins_data"
}

Docker Container Resource

This block defines a Docker container resource named “jenkins”. It specifies the Docker image to use (docker_image.jenkins.name) and exposes ports 8080 and, 50000 for Jenkins web UI and agent communication, respectively. The restart attribute is set to “unless-stopped”, indicating that the container should automatically restart unless explicitly stopped by the user.

resource "docker_container" "jenkins" {
  name   = "jenkins"
  image  = docker_image.jenkins.name
  ports {
    internal = 8080
    external = 8080
  }
  ports {
    internal = 50000
    external = 50000
  }
  restart = "unless-stopped"

  volumes {
    container_path  = "/var/jenkins_home"
    host_path       = "/home/arif/jenkins_data" # It should be an absolute path.
    read_only       = false
  }
  
}

The volumes section specifies the volume mappings between the container and the host. In this case:

• container_path: Specifies the path inside the container where the volume will be mounted (/var/jenkins_home).
• host_path: Specifies the path on the host system where the volume data will be stored (/home/arif/jenkins_data). It should be an absolute path.
• read_only: Specifies whether the volume should be mounted as read-only or read-write (false in this case).

Terraform Workflow: Deploying Infrastructure as Code

Initializing Terraform

Before using Terraform to manage your infrastructure, you need to initialize the working directory. This step downloads the necessary provider plugins and initializes the backend.

terraform init

Validating Configuration Syntax

Once initialized, you can validate the syntax and configuration of your Terraform files to ensure they’re correctly formatted and without errors.

terraform validate

Planning Infrastructure Changes

Terraform generates an execution plan based on your configuration files. This plan describes what actions Terraform will take to change the infrastructure to match the desired state defined in the configuration.

terraform plan -out=tfplan

This command generates a plan and saves it to a file named tfplan for review before applying.

Applying Infrastructure Changes

After reviewing the execution plan, you can apply the changes to your infrastructure. Terraform will prompt for confirmation before proceeding.

terraform apply tfplan

Alternatively, you can apply changes directly without saving a plan:

terraform apply

You can apply the planned changes with the --auto-approve flag:

terraform apply --auto-approve tfplan

Terraform will start provisioning the infrastructure according to the defined specifications in our configuration files. This includes:

• Building the Docker image for Jenkins using the specified Dockerfile.
• Creating a Docker volume to persist Jenkins data.
• Spinning up a Docker container named “jenkins” based on the Jenkins image.
• Exposing ports 8080 and, 50000 for Jenkins web UI and agent communication, respectively.
• Mounting the Docker volume to store Jenkins data persistently.

Sometimes the plugins installations creates some issues. You can remove/comment the line and manually install the plugins on the built docker container:

RUN jenkins-plugin-cli -f /usr/share/jenkins/ref/plugins.txt

Monitoring Deployment Progress

During the deployment process, Terraform will provide real-time feedback on the progress of each step. You’ll see logs indicating the creation of resources and any associated changes.

Once the deployment is complete, Terraform will display a summary of the changes applied and the overall state of the infrastructure like the following:

Apply complete! Resources: 3 added, 0 changed, 0 destroyed.

If your Jenkins plugins installation failed earlier, follow the bellow command on docker container.

docker ps
docker exec jenkins sh -c "jenkins-plugin-cli -f /usr/share/jenkins/ref/plugins.txt"

Verifying Jenkins Deployment

To verify that Jenkins has been successfully deployed, you can access the Jenkins web interface using your web browser. Navigate to http://<your-server-ip>:8080 and follow the instructions to complete the initial setup wizard.

Run the following command to get the initial password:

docker exec jenkins sh -c "cat /var/jenkins_home/secrets/initialAdminPassword"

Destroying Infrastructure

If you need to tear down the infrastructure managed by Terraform, you can use the destroy command. Be cautious, as this will permanently delete all resources defined in your configuration.

terraform destroy --auto-approve

However, if you reapply the same Terraform code after destroying the infrastructure, it will not restore the data stored in the volume folder automatically. Terraform treats each execution as a fresh deployment and does not retain state between executions.

To ensure that data stored in the volume folder persists across infrastructure deployments and destruction, you may need to implement additional mechanisms, such as backing up and restoring data manually or integrating with external storage solutions.

When working with persistent data in Docker containers managed by Terraform, it’s essential to consider data management strategies to maintain data integrity and continuity across infrastructure lifecycle events.

Recap and Conclusion

In this article, we’ve covered the essential Terraform commands for managing infrastructure as code. From initializing the working directory to applying changes and destroying resources, Terraform provides a streamlined workflow for provisioning and managing infrastructure across various cloud providers and environments.

Stay tuned for the next part, where we’ll delve into advanced Terraform concepts and techniques for infrastructure orchestration.

Check my other development and deployment articles.