Foray into Jenkins, Puppet, Docker, and Photon: Part 2

In my Foray into Jenkins, Puppet, Docker, and Photon article, I discussed how to set up Photon, Jenkins, and Docker and how I was able to use Jenkins to deploy a Photon template. However, this template was a simple template with just Docker set up within it: a template that could become anything required, run any Docker container. The next step was to deploy a container within Photon. For this, I chose NGINX.

Deploying a Docker container remotely required the IP address of the container. “Simple,” you say, “you can just expose the guest info from within the vSphere plugin within Jenkins.” Well, not really, as that exposes the Docker0 Ethernet address used internally by Docker to communicate with the container, and not the external address of the VM. “Then, could I use PowerShell?” Definitely possible, but then how do you access vCenter using a stored password? “Could I use Linux with Perl?” Once more, how do you access vCenter using a stored password?

Not only do I need to gain access to the IP of the newly cloned VM, but I have to do it securely. Before I could run any script, I had to store a credential for use by PowerCLI. That credential I stored within the C:\Jenkins directory using the following:

PS > Add-PSSnapin VMware.VimAutomation.Core
PS > . "C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\Scripts\Initialize-PowerCLIEnvironment.ps1"
PowerCLI > New-VICredentialStoreItem -Host VIServer -User JenkinsUser -Password Password -File C:\Jenkins\credstore.xml

My first attempt was a simple PowerShell script that produced way too much output, as I really only want the 10.0.0.xxx address, not the 172. address that was reported with the guest info integration.

PS > Add-PSSnapin VMware.VimAutomation.Core
PS > . "C:\Program Files (x86)\VMware\Infrastructure\vSphere PowerCLI\Scripts\Initialize-PowerCLIEnvironment.ps1"
PowerCLI > Get-VICredentialStoreItem -User JenkinsUser -Host VIServer -File C:\Jenkins\credstore.xml
PowerCLI > Connect-VIServer VIServer
PowerCLI > $vm=Get-VM VMName; $addr=foreach ($ip in $vm.guest.IPAddress) { if($ip.Contains("10")) { $ip } }; echo $addr
10.0.0.xxx

Without the .Contains check, the results contained five MAC addresses and two IP addresses, not what I needed to get the external IP address of the VM. However, when I attempted to run this via Jenkins on my Windows Jenkins slave, I ran into permission problems accessing the credential store. It worked fine from the PowerShell command line, but not from Jenkins.

So instead, I went ahead and installed the Perl version of vCLI onto my Jenkins server using the following script. Note that these steps are required for CentOS/RHEL 7 versions of Linux. If you used the prebuilt versions provided with vCLI, you would end up with a broken Perl.

sudo yum -y install openssl-devel perl-CPAN perl-HTML-Parser perl-Compress-Raw-Zlib perl-SOAP-Lite perl-Data-Dump perl-Archive-Zip perl-Class-Data-Inheritable perl-Class-MethodMaker perl-Convert-ASN1 perl-Crypt-OpenSSL-RSA perl-Crypt-SSLeay perl-Crypt-X509 perl-Data-Dump perl-Devel-StackTrace perl-IO-Socket-INET6 perl-JSON-PP perl-Socket6 perl-URI perl-UUID perl-UUID-Random perl-LibXML perl-LibXML-Common perl-XML-NamespaceSupport perl-XML-SAX gcc make perl-devel perl-XML-LibXML uuid-perl libuuid uuid libuuid-devel
sudo perl -MCPAN -e shell << EOF
install CFABER/UUID-0.03.tar.gz
install MIME::Base64
install Socket6
install IO::Socket::INET6
EOF
tar -xzf VMware-vSphere-CLI-6.0.0-2503617.x86_64.tar.gz
cd vmware-vsphere-cli-distrib
sudo ./vmware-install.pl << EOF

q
yes
no
yes

EOF

Once I did this, I was able to write a very simple script to extract the information I needed so I could then use remote Docker calls to load up NGINX. However, since I now wrote a script to get the IP via Perl using the Perl version of the credential store, I needed a way to store that script within Git so that I can extract it using Jenkins, execute it, and deploy my Docker container. That was achieved by adding SCM-Manager to my private Git server. I added SCM-Manager to give me the ability to use a web interface to control Git as well as add in Jenkins and other plugins and integration. I installed the following SCM-Manager plugins:

• scm-groupmanager-plugin
• scm-jenkins-plugin
• scm-graph-plugin
• scm-script-plugin
• scm-mail-plugin
• scm-pushlog-plugin

I found that adding in the scm-activity-plugin and scm-userrepo-plugin caused SCM-Manager to behave improperly. Furthermore, I disabled any non-Git repository.

After creating a new repository for my library of tools for Jenkins, I added in my script as well as the credential store used by the script. Yes, I ensured the credentials were properly encrypted before doing so. Furthermore, I would never do this for a public Git repository. However, since I own the server my Git is running and can control access and security, I have no real qualms. My get the IP script follows:

#!/usr/bin/perl -w
#
# Copyright (c) 2015 AstroArch Consulting, Inc. All rights reserved
#
# requires credstore

use strict;
use VMware::VIRuntime;
use VMware::VILib;
use VMware::VICredStore;
use File::Basename;

my $vmname=$ARGV[0];

VMware::VICredStore::init(filename => "./vicredentials.xml");
my @server_list = VMware::VICredStore::get_hosts();
my @user_list = VMware::VICredStore::get_usernames(server => $server_list[0]);
my $password = VMware::VICredStore::get_password(server => $server_list[0], username => $user_list[0]);
my $url = "https://".$server_list[0]."/sdk/vimService";
VMware::VICredStore::close();

eval {
  Vim::login(service_url => $url, user_name => $user_list[0],
    password => $password);
};
if ($@) {
  print "$@"; exit 3;
}

my $vdata = Vim::find_entity_views(view_type => 'VirtualMachine', filter => {"config.name" => $vmname});
foreach (@$vdata) {
  my $vm_view = $_;
  if (defined $vm_view->guest->net) {
    if (defined $vm_view->guest->net) {
      my $net_len = @{$vm_view->guest->net};
      my $cnt = 0;
      while ($cnt < $net_len) {
        if (defined $vm_view->guest->net->[$cnt]->ipAddress) {
          my $ip_len = @{$vm_view->guest->net->[$cnt]->ipAddress};
          my $cnt_ip = 0;
          while ($cnt_ip < $ip_len) {
            print $vm_view->guest->net->[$cnt]->ipAddress->[$cnt_ip]."\n";
            $cnt_ip++;
          }
        }
        $cnt++;
      }
    }
  }
}
Vim::logout();

The last line of the script is very important: without it, the SDK gets confused and does not always run appropriately.

Now, for the Jenkins integration, I created a subproject to my previous Jenkins build project. This way I have one project to clone the VM and another project to deploy the Docker container. In this new project, I used the following configurations:

• Source Code Management, Enable Git to pull from my library of Jenkins Build Commands with the proper username, etc.
• Build Triggers, Build after other projects are built and specify the original Photon Clone build.

Then all I had to do was add in the proper Build steps, of which there is really only one, an Execute Shell step with the following code:

ip=`nohup ./get_ip.pl VMName | grep 10.0`
export DOCKER_HOST=${ip}:2375
docker ps |grep vmwarecna/nginx
if [ $? -eq 1 ]
then
   docker run -d -p 80:80 vmwarecna/nginx
fi
curl http://${ip}/

The above Build step does several things:

1. Gets the IP for the network we desire from the VM we are working on.
2. Sets the DOCKER_HOST environment variable so that the Docker commands know to contact my VM and run there instead of locally.
3. If the Docker container is not already running, starts it, and if the bits are not there, pulls it down from a registry.
4. Tests to see if NGINX is running.

Now, one thing I did not mention is that I had to install the Docker bits onto my Jenkins server in order to use the Docker command in my build. So now, my Jenkins server contains the following:

• Jenkins
• Java (needed to run Jenkins)
• Docker (needed to run remote Docker commands)
• Puppet
• Git
• vCLI (needed to get the IP of the VM)
• TCL (needed for the next build)
• GCC (needed to build CPAN bits for vCLI)
• Make (needed to build CPAN bits for vCLI)

My next build within Jenkins will be to do some load testing of my newly deployed Photon image with an NGINX container. The very cool aspect of this is that to prepare the Photon image, all I had to do was enable Docker to take remote commands. That was the sole change to my Photon installation and the basis for my Photon template and this foray into Jenkins, Puppet, Docker, and Photon.

I have now deployed Docker containers from Jenkins onto Photon without having first created an NGINX container template for Photon. All I have is a Docker-enabled Photon template.

My next build for Jenkins will be to load test my NGINX container.