Ansible in a (docker) container

Ansible Playbooks vs Roles: Part I - The Playbook ...

A year ago, after following a few sessions at the dutch VMATechcon I decided to dive into the world of Ansible. With a background in industrial automation (programming PLCs) I have a soft spot for writing code. I saw the benefits of declarative languages, and decided to dive into Ansible.
But where to start. And what should I use it for.

Image

I’m working on a small VM which provides DHCP,DNS,TFTP,FTP,HTTP and NTP services so I can easily (PXE) boot nested ESXi hosts on VMware workstation and my homelab. All running in docker containers on Photon OS.
For maintenance (and learning) I decided to use Ansible for building, running these containers.

One of the issues I had, was that I had to maintain the VM and the software installed on it… I’m quite proficient with Linux, but it can still be a challenge.
And keeping installations logs update about how to install software… yeah…. that is an issue. A embraced the solutions docker, git and ansible are giving me. (And vagrant, when using VMware workstation).

Using docker to run the services, using ansible to automate workflows and using git to keep track of changes and store my code in my github repositories.
With this approach I can easily rebuild the VM when it breaks down. And I see the VM more as a processing entity, then a virtual computer that needs to be maintained.

The Ansible container

browsing the internet I found this blog post about running ansible inside a docker container and using it interactively on the docker host. This gave me the possibility to have ‘portable’ ansible installation, without any ties to the VM OS.
In my opinion it is great. With adding some small bash scripts so I don’t need to write something like ‘docker run -rm -it …..’ everytime, I have an ideal solution.
The original code for the container is based on alpine:3.7 and python 2. Everytime I build the docker image I received a warning that python 2 is outdated…. so I (finaly) updated the dockerfile using alpine:3.11 as the base image and using python 3.
You can find my code here on github, and the docker image here on dockerhub.

How to use

simple. There are 2 ways to use it.

  1. clone git repository
  2. pull image from dockerhub

Option 1, clone git repository

I’ve created a small installation script to make things easier. It will run docker build to build the ansible image and it will copy the scripts under ./scripts/ to /usr/local/bin/
And your all set… well almost

Option 2, pull image from dockerhub

Instead of building the image, you can pull it from dockerhub into your local docker environment. The scripts assume that the docker image is tagged like ansible:2.9.0, but when you pull it from dockerhub, the name is brtlvrs/ansible-helper:v.0.3. So you need to retag them (or edit the scripts).

docker image tag brtlvrs/ansible-helper:v.0.3 ansible:2.9.0
docker image tag ansible:2.9.0 ansible:latest

The wrapper scripts are stored in the image, in the folder /2installOnHost
To copy them out of the image do the following

docker create -n test ansible:2.9.0
docker cp test:/2InstallOnHost/ah* /usr/local/bin/
chmod +x /usr/local/bin/ah*
docker rm test

This will create a docker container named test, using the imported image ansible:2.9.0.
Then copies the files from inside the container to the /usr/local/bin/ and sets the executable bit on these files.
Finally it removes the created docker container.
Now you should be able to run ah or ah-playbook.

Both scripts will mount the folowing volumes into the container

localmountpointremark
~/.ssh/id_rsa/root/.ssh/id_rsaneeded to access localhost via ssh (ansible)
~/.ssh/id_rsa.pub/root/.ssh/id_rsa.pubneeded to access localhost via ssh (ansible)
$(pwd)/ansible/playbooksMountpoint is the default location inside the container.
/var/log/ansible/ansible.logMaking the ansible logs persistent.

Finishing touch

To make it work you need to do the following:

  • install docker-py on the docker host ( if you want ansible to be able to control docker)
  • add the ~/.ssh/id_rsa.pub content to ~/.ssh/authorized_keys file, so you don’t need to store the user password into the ansible inventory file.

Final

An now it should work. Type ‘ah’ and you’ll enter a temporary docker container running ansible. Type ah-playbook and you’ll execute the command ansible-playbook inside a temporary docker container.
Have fun.

Custom Welcome DCUI screen

Recently I discovered you can change the default ESXi DCUI screen.
This feature is already present for a long time…. just check one of my favorite blog sites (virtualGhetto) for this item here. Yes he blogged about it in 2010….. I know

Why would you want to change it

  1. Your Company wants a warning shown on the DCUI
  2. Security, don’t show hostname, ip etc…
  3. showing off ASCI-ART – just for fun
  4. ….

For me, it was point 3….
I run on my laptop VMware Workstation and am sometimes in need of spinning up nested ESXi with different kind of versions. For this reason I build a provisioning VM, so I can network boot empty VMs and choose from an iPXE menu the ESXi version I want to work with. (maybe some interesting stuff for a blog serie) And this setup wasn’t finished with a custom DCUI screen.

How does it work

Until version ESXi 7.0 there is a text file /etc/vmware/welcome, which you can edit. When it is succesfully edited, the DCUI wil show the result.
My screen shows: VMware product and version, license, Memory, IP, SSL Thumbprint and ASCI-ART 🙂

You can also edit the logonbanner, shown when you access the host via SSH. And the support info

Until ESXi 7 ?

Yes, until… well almost. Until version 7.x you just could simply edit the file /etc/vmware/welcome. But it isn’t there anymore.
But there is a different way. Called CLI. You can edit the welcome message with esxcli and powercli. Information is found in the VMware knowledge base kb2046347. And this works for a lot off ESXi versions. I’ve tested it succesfully from 5.5 to 7.0.

ASCII-ART

Well, you can show your creative side. Some guidelines I found are:

  1. screen with is 125 characters
  2. max. rows about 25
  3. check for ASCI TAB character, ASCII nr 9
  4. use a site for creating art like this one

Especially number 3 is a killer. A lot of text editors will replace a certain amount of spaces with a TAB character… and this will mess up your screen.

ESXCLI method

The syntax is easy, just

esxcli system welcomemsg set -m 'your welcome'

And that is what is shown in the kb article. But how do you create a complete screen. Well, there are some undocumented parameters you can use. The article by William Lam (see above) shows some of them. But I haven’t find a site with all of them.
With those parameters you create in a text editor your screen…. like this one

Every line that uses a different background color then black is 125 characters long, not including the length of any fields that are used, like {color:white}.

After you created your file, you can place it in /etc/vmware or use esxcli to edit the welcome message.
If the file is placed in /tmp/welcome.txt then your instruction is like

esxcli system welcomemsg set -m "$(cat /tmp/welcome.txt)"

And voila, you have a custom welcome screen.

DCUI with SSH

The DCUI is not only shown on the console screen of ESXi, but also accessed via SSH by >dcui
The result is shown below.

DCUI screen when accessed in SSH

Enjoy.

Set Email notificions on vCenter Alarm definitions

One of the task when configuring a vCenter is to setup the alarm notifications. But to configure 268 alarms by hand…. well it is just a scary task when you do it via the gui / webclient.
In short I created this script that you can find on my github.

The full story

Continue reading “Set Email notificions on vCenter Alarm definitions”

Issue 2 – bypassing the fingerprint cache message when using PLINK

This article is part of a series of articles about issues I encountered during implementation of a vSphere stretched cluster based on vSphere 6.7 U1.
You can find the introduction article here

Issue 2

For some configuration settings I need SSH access to the host.  I use plink.exe to execute instructions through the SSH session. One issue, the first time when you connect with plink you get a message about storing the fingerprint ID in the cache. Normally you would accept this when using putty. But now this is going to be a challenge.
On some other blogs I found the solution. You echo the ‘Y’ which results in storing the ID in the cache.
In my code I now  call plink two times. The first time to accept the fingerprint, the second time to execute the command.
Why two times ? Well, I can’t assume that the fingerprint ID is already known.
The first plink instruction is a simple exit, we only want to check if we can logon.

$credential=get-credential
$plink="d:\plink.exe $hostname -l "+ $credential.username + " -pw " + $credential.getnetworkcredential().password
$command="ls"
invoke-expression ("echo Y | " + $plink +  " -ssh exit")
invoke-expression ($plink + " "+ $command)

Issue 1 – changing root password

This article is part of a series of articles about issues I encountered during implementation of a vSphere stretched cluster based on vSphere 6.7 U1.
You can find the introduction article here

Issue

All the hosts are delivered with 6.5 U2 pre-installed, and they have their own root password. For the implementation we want to have just one general root account password. So after adding all the hosts to the cluster I want to change the root password with powercli. But I tripped over a bug in get-esxcli (thanks to this thread ). The ‘&’ character is not correctly being interpreted when using get-esxli.
The script I wrote checks if the new password contains that character and will kindly ask to change it. After succesfull validation of the password it will apply it to all selected esxi hosts.
I

#-- select one or more hosts
[array]$esxiHosts=get-vmhost | select name | sort | out-gridview -Title "Select one or more ESXi Hosts"-OutputMode Multiple
if ($esxiHosts.count -eq 0) {
write-host "No host(s) selected, will exit." -foregroundcolor yellow
exit
}
#-- ask for root password and validate it agains known bug
Do {
$newCredential = Get-Credential -Username root -Message "Enter the password for the ESXi root account."
$isValid=$true
if ($newCredential.getNetworkCredential().Password -imatch "[\&]") {
$isValid=$false
write-host"Password contains character & which get-esxcli can't handle (bug)..... please consider a different password." -foregroundcolor yellow
}
}
until ($isValid)

#-- change root password for all selected esxi hosts
foreach ($esxiHost in $esxiHosts) {
$esxiHost=get-vmhost -Name -$esxiHost.name
$esxiCli=get-esxcli -v2 -vmhost $esxiHost
$arguments=$esxcli.system.account.set.createArgs()
$arguments.id=$newCredential.UserName
$arguments.password=$newCredential.GetNetworkCredential().password
$arguments.passwordconfirmation=$arguments.password
try {$esxcli.system.account.set.Invoke($arguments)}
catch{write-host "Setting password failed for " $esxiHost.name -ForegroundColor Yellow}
}

Issues I encountered with a stretched cluster implementation on 6.7 U1

At the moment I’m busy with a stretched cluster implementation based on vSphere 6.7 U1. Most of the configuration is straight forward. But I encounter some snags.
So this post is about these snags, and how I solved them.

For configuring 16 hosts I use a lot of powerCLI. Why ? Well I have some issues with host profiles, and not the time (yet) to figure out what is going on.
Edit: I found out what the issue is, I’ll explain it in Issue 3.

I encountered the following issues

Powershell script template

To make live a bit easier when writing powershell scripts, it is a good idea to have some kind of a script template.

Why ?

By using a template you don’t have to worry about the standard stuff you always put in a script. And you can also have some customs functions already available. It is a good starting point for writing the script you want.
And it helps you to standardize you scripting. Which makes it in the end easier to maintain etc… Continue reading “Powershell script template”

import-PowerCLI

For my work I use PowerCLI a lot, but I don’t like the shell that is available when installing PowerCLI. I just want to start PowerShell (mostly through PowerShell ISE) and load with one command the PowerCLI modules and/or snapins and not be limited.

Before PowerCLI 6.5, PowerCLI was a mix of powershell snappins and modules. To support older PowerCLI versions, the script will check if there are vmware powershell snappins registered. And will load them.
Powershell modules will have precedence over the snappins. Continue reading “import-PowerCLI”