Continuous Integration: Part 1 - Setting Up VMs, Docker, and Jenkins

July 6, 2017 by Phu Ha <pha@atlassoftwaregroup.com>

The goal is to setup a CD/CI (continuous delivery/continuous integration) workflow using opensource products.

What’s in our toolbox?

Before We Begin

Can I use this for production?

Yes and No. Yes, some steps can be apply to production. And while some steps such as disabling the firewall without a viable replacement (e.g aws security group) is definitely a “no-no” in production.

What does “local”, “rancher-ui”, and “rancher-host” in the code block mean?

I have included “local”, “rancher-ui”, “rancher-host” in the code block to help readers understand which terminal to enter each command.

Why do you use semicolons instead of logical AND operators (&&)?

My terminal is configured to use fish and fish does not support logical AND operator (&&).

Install Vagrant & VirtualBox

Vagrant can be download at https://www.vagrantup.com/downloads.html.

VirtualBox can be download at https://www.virtualbox.org/wiki/Downloads.

Create a Virtual Machine (Rancher UI)

Create a workspace and initialize the Vagrantfile

(local): mkdir rancher-ui; cd rancher-ui; vagrant init ubuntu/xenial64

Add this code on line 16 in the Vagrantfile. This will configure our virtual machine with an IP address of 192.168.2.123.

config.vm.network "private_network", ip: "192.168.2.123"
config.vm.network "public_network"

Log into our virtual machine

(local): vagrant up --provider=virtualbox; vagrant ssh

Install Docker

Update our apt repository and upgrade our system.

(rancher-ui): sudo apt-get update; sudo apt-get upgrade -y

Install dependencies (add-apt-repository)

(rancher-ui): sudo apt-get install -y \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common \
    python-software-properties

Add Docker apt key

(rancher-ui): curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

Add Docker repository

(rancher-ui): sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

Update apt cache

(rancher-ui): sudo apt-get update

Install Docker

(rancher-ui): sudo apt-get install -y docker-ce

Test if docker is running.

(rancher-ui): sudo service docker status

Example output

● docker.service - Docker Application Container Engine
   Loaded: loaded (/lib/systemd/system/docker.service; enabled; vendor preset: enabled)
   Active: active (running) since Fri 2017-07-07 13:48:10 UTC; 2s ago
     Docs: https://docs.docker.com
 Main PID: 5388 (dockerd)
   CGroup: /system.slice/docker.service
           ├─5388 /usr/bin/dockerd -H fd://
           └─5396 docker-containerd -l unix:///var/run/docker/libcontainerd/docker-containerd.sock --metrics-interval=0 --start-timeout 2m --state-dir
...

If there aren’t any errors, then docker is running.

For more details on the above commands, see https://docs.docker.com/engine/installation/linux/docker-ce/ubuntu/

Install Rancher UI

This command will create a container containing the rancher ui.

(rancher-ui): sudo docker run -d --restart=unless-stopped -p 8080:8080 rancher/server

Note: This configuration is the “Single Container (non-HA)” deployment and is not recommended for production.

See http://rancher.com/docs/rancher/v1.2/en/installing-rancher/installing-server/

Test if Rancher UI container is running.

(rancher-ui): sudo docker ps -a

Example output:

CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                              NAMES
9c25747d6fb1        rancher/server      "/usr/bin/entry /u..."   11 seconds ago      Up 8 seconds        3306/tcp, 0.0.0.0:8080->8080/tcp   jolly_mahavira

Disable iptables for now. This is highly discourage in production environment. Because we are testing, we want to disable iptables to prevent any frustration while we evaluate this product.

(rancher-ui): sudo ufw disable

Open Rancher UI in your browser by typing http://192.168.2.123:8080.

Setup a Rancher Host

Authentication

I’m not going to cover authentication. There are multiple options available, fairly easy to configure, and not neccessary at this moment.

See http://rancher.com/docs/rancher/v1.6/en/configuration/access-control/

Initialize VM and Workspace

Create a rancher-host workspace and initialize the virtual machine.

(local): mkdir rancher-host; cd rancher-host; vagrant init ubuntu/xenial64

Add this code on line 16 in the Vagrantfile. This will configure our virtual machine with an IP address of 192.168.2.124.

config.vm.network "private_network", ip: "192.168.2.124"
config.vm.network "public_network"

Log into our virtual machine

(local): vagrant up --provider=virtualbox; vagrant ssh

Install Docker into the rancher-host machine. This command combines the Rancher UI commands to install Docker into a single command.

(rancher-host): sudo apt-get update; sudo apt-get upgrade -y; \
sudo apt-get install -y \
    apt-transport-https \
    ca-certificates \
    curl \
    software-properties-common \
    python-software-properties; \
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -; \
sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"; \
sudo apt-get update; \
sudo apt-get install -y docker-ce;

Disable iptables for now. This is highly discourage in production environment. Because we are testing, we want to disable iptables to prevent any frustration while we evaluate this product.

(rancher-host): sudo ufw disable

This fixes the slow outgoing network traffic between the host and Docker.

(rancher-host): sudo vi /lib/systemd/system/docker.service

Replace (on line 13)

ExecStart=/usr/bin/dockerd -H fd://

With

ExecStart=/usr/bin/dockerd -H fd:// --mtu=1000

Reload systemd and restart Docker

(rancher-host): sudo systemctl daemon-reload; sudo service docker restart

Add Rancher Host (Rancher Machine)

In the Rancher UI, we will add our host.

Note: The default environment is using Cattle. I will explain the pro/cons of each scheduler (cattle, kubernetes, etc.) in the next post. For now, see http://rancher.com/cattle-swarm-kubernetes-side-side/

  1. Go to Infrastructure > Hosts.

  2. Click “Add Host”.

  3. Click “Save”. Note: The IP address is incorrect in this image. I took a screen shot on my first attempt and forgot to take another screen shot for 192.168.2.123.

  4. Copy and paste the command into the rancher-host vagrant machine.

Install Jenkins on the Rancher Host from Rancher UI

In the Rancher UI:

  1. Navigate to Catalog > All

  2. Search for Jenkins

  3. Select Jenkins (Do not select Jenkins Swarm Plugin)

  4. Click “Launch”

Wait for Jenkins to be active

  1. Go to Stacks > All

  2. Click on “jenkins-ci”

  3. Click on “jenkins-primary”

  4. On “jenkins-ci-jenkins-primary”, view logs. See screen shot.

  5. Look for Note: This might take awhile.

  6. Copy the password. The password will be use to complete Jenkins’ initial setup.

  7. We can access Jenkins by typing http://192.168.2.124:8080 in our browser.

  8. Enter password and select standard plugins.

  9. After you log into Jenkins, click on Manage Jenkins. If you see dependency errors, we need to restart Jenkins.

  10. In Rancher UI, we can restart Jenkins by clicking “Restart” on “jenkins-ci-jenkins-primary-1” container.

TODOs

Sources

© 2017 | Atlas Software Group | Hucore theme & Hugo