Creating a Image for MaaS with Packer

In this post we are going to build an image with Packer which will be used to deploy via MaaS. After image built and uploaded to MaaS, it can be used to provision virtual machine or deploy OS on Bare-Metal machines. In order to build an image that is deployable with MaaS, we need couple of files which you can clone here.

For this post, It will be created minimal CentOS7 image including httpd package to test. One of the cool thing is with Packer that you can also run execute your Ansible playbook inside the machine being provisioned by Packer. Ansible(remote) provisioner is used to configure ntp server and install httpd.

As Qemu used as a builder, qemu-system-x64 has to be installed on the host where Packer runs. Packer will create an qcow2 image after successful image creation. But we are going to use tar.gz image file as we deploy image via MaaS.

centos7.json (do not forget to change iso_url in accordance with your environment.)

{
    "builders": [
        {
            "type": "qemu",
	    "iso_url": "/home/tesla/packer/centos7/isos/CentOS-7-x86_64-NetInstall-2003.iso",
            "iso_checksum_type": "sha256",
	    "iso_checksum": "101bc813d2af9ccf534d112cbe8670e6d900425b297d1a4d2529c5ad5f226372",
            "boot_command": [
                "<tab> ",
                "inst.ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/centos7.ks ",
                "<enter>"
            ],
	    "ssh_username": "tesla",
	    "ssh_password": "tesla",
	    "ssh_wait_timeout": "12000s",
            "boot_wait": "3s",
            "disk_size": "4G",
	    "display": "none",
            "headless": false,
            "memory": 4096,
	    "accelerator": "kvm",
	    "cpus": 4,
            "http_directory": "http",
            "shutdown_timeout": "20m",
	    "disk_interface": "virtio",
            "format": "qcow2",
            "net_device": "virtio-net"

        }
    ],

"post-processors": [
        {
            "type": "shell-local",
            "inline_shebang": "/bin/bash -e",
            "inline": [
                "TMP_DIR=$(mktemp -d /tmp/packer-maas-XXXX)",
                "echo 'Mounting image...'",
                "modprobe nbd",
                "qemu-nbd -d /dev/nbd4",
                "qemu-nbd -c /dev/nbd4 -n output-qemu/packer-qemu",
                "echo 'Waiting for partitions to be created...'",
                "tries=0",
                "while [ ! -e /dev/nbd4p1 -a $tries -lt 60 ]; do",
                "    sleep 1",
                "    tries=$((tries+1))",
                "done",
                "echo 'Tarring up image...'",
                "mount /dev/nbd4p1 $TMP_DIR",
                "tar -Sczpf centos7.tar.gz --selinux -C $TMP_DIR .",
                "echo 'Unmounting image...'",
                "umount $TMP_DIR",
                "qemu-nbd -d /dev/nbd4",
                "rmdir $TMP_DIR"
            ]
        }
    ],
    "provisioners": [
    {
  "type": "shell",
  "pause_before": "5s",
  "inline": [
	"sudo yum -y install epel-release",
	"sudo yum -y update",
	"sudo yum -y remove cloud-init",
	"sudo yum -y install python-jsonschema python-devel",
	"sudo yum -y install cloud-init --disablerepo=* --enablerepo=group_cloud-init-el-stable",
	"sudo yum -y install qemu-guest-agent wget"
  ]
},
    {
      "user": "tesla",
      "type": "ansible",
      "playbook_file": "./ansible/main.yml"
    },
    {
     "type": "shell",
     "inline": [
     
	"sudo systemctl enable cloud-init",
	"sudo rm -rf /var/lib/cloud/",
	"sudo rm -rf /etc/cloud/cloud-init.disabled",
	"sudo /usr/bin/truncate -s 0 /etc/fstab",
	"sudo /usr/bin/truncate -s 0 /etc/resolv.conf",
        "sudo rm -f /etc/sysconfig/network-scripts/ifcfg-[^lo]*",
        "sudo sync"
  ]
}
    ]
}
#main.yml
---
- hosts: default
  become: yes
  roles:
    - configure_httpd
    - configure_chrony

centos7.ks

url --mirrorlist="http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os"
text
reboot
firewall --enabled --service=ssh,http
firstboot --disable
ignoredisk --only-use=vda
lang en_US.UTF-8
keyboard us
network --bootproto=dhcp --hostname=packer-maas.manintheit.org
selinux --enforcing
timezone UTC --isUtc
bootloader --location=mbr --driveorder="vda" --timeout=1
rootpw --plaintext root1234
user --name=tesla --groups=wheel --plaintext --password=tesla


repo --name="Base" --mirrorlist="http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=os"
repo --name="Updates" --mirrorlist="http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=updates"
repo --name="Extras" --mirrorlist="http://mirrorlist.centos.org/?release=7&arch=x86_64&repo=extras"
repo --name="cloud-init" --baseurl="http://copr-be.cloud.fedoraproject.org/results/@cloud-init/el-stable/epel-7-x86_64"

zerombr
clearpart --all --initlabel
part / --size=1 --grow --asprimary --fstype=ext4


%packages
@core
#httpd
bash-completion
cloud-init-el-release
cloud-init
# cloud-init only requires python-oauthlib with MAAS. As such upstream
# has removed python-oauthlib from cloud-init's deps.
python2-oauthlib
cloud-utils-growpart
rsync
tar
yum-utils
# bridge-utils is required by cloud-init to configure networking. Without it
# installed cloud-init will try to install it itself which will not work in
# isolated environments.
bridge-utils
# Tools needed to allow custom storage to be deployed without acessing the
# Internet.
grub2-efi-x64
shim-x64
# Older versions of Curtin do not support secure boot and setup grub by
# generating grubx64.efi with grub2-efi-x64-modules.
grub2-efi-x64-modules
efibootmgr
dosfstools
lvm2
mdadm
device-mapper-multipath
iscsi-initiator-utils
-plymouth
# Remove ALSA firmware
-a*-firmware
# Remove Intel wireless firmware
-i*-firmware
%end


%post --erroronfail
systemctl disable cloud-init
touch /etc/cloud/cloud-init.disabled
yum install -y sudo
echo "tesla        ALL=(ALL)       NOPASSWD: ALL" >> /etc/sudoers.d/tesla
sed -i "s/^.*requiretty/#Defaults requiretty/" /etc/sudoers
yum clean all
%end

Validating Packer config.

sudo PACKER_LOG=1 packer validate centos7.json

After successful validation, we can start building machine image.

sudo PACKER_LOG=1 packer build centos7.json

Once packer finished successfully, centos7.tar.gz file should be created, which will be uploaded to MaaS to provision VMs or Install OS in Bare-Metal servers.

Uploading image to MaaS.

maas $PROFILE boot-resources create name="centos/centos7Packer1" architecture=amd64/generic content@=centos7.tar.gz

Important Takeaways on Packer.

Do not forget to issue sync command inside machine provisioned by Packer. Otherwise, you will experience with the empty systemd unit files. So, sync command will flush data from cache to disk.

As MaaS uses cloud-init to setup various host settings such as partitioning disk, dns server, network configuration etc. Some of the files are removed or truncated prior to image creation. Because MaaS will populate these configurations during the first boot of the system. If you deploy the OS via MaaS, configure dns setting on the MaaS, it will be anyway overwritten by cloud-init even, you configure resolv.conf inside the image.

Sample Expect Script

Expect is a programming language for automating systems which expose interactive text terminal such as telnet, ssh, ftp, scp etc,. It is extension to Tcl language. It is created by Don Libes.

In this post I will share you sample expect script, which connects list of servers over ssh, executes command and prints it out on the screen.

Install Package

To use Expect language (It is actually a programming language–Extension of Tcl.), we need to install expect package.

#For CentOS and RedHat
[root@rhce ~]# yum install expect
#For Ubuntu, Debian and Mint
gns3@gns3:~/Programs$ sudo apt-get install expect

Create a server list.

Create a serverlist.txt file and put your server’ IP address or Domain names in this file line by line.  You can see below sample file contents.

gns3@gns3:~/Programs$ cat serverlist.txt 
192.168.59.133
192.168.59.134

Create Expect Script.

You can use below sample script and tweak it for your own purpose. For this script I have servers, username demo and password is demo

#!/usr/bin/expect -f
set fd "serverlist.txt"
set fp [open "$fd" r]
set data [read $fp]
 
# Read line by line
set timeout 15
 
foreach server $data {
puts "================ssh for $server============================"
 
 spawn ssh -l demo "$server"
 expect "password: "
 send "demo\r"
 expect "$ "
 send "uptime\r"
 expect "$ "
 send "ifconfig\r"
 expect "$ "
 send "exit\r"
}

Experiment

I actually have only one virtual machine (192.168.59.133) second machine (192.168.59.134) is an artificial. It is added for the sake of  for-loop demonstration.

gns3@gns3:~/Programs$ expect exp.exp 
================ssh for 192.168.59.133============================
spawn ssh -l demo 192.168.59.133
demo@192.168.59.133's password: 
Last login: Sat Jul 22 10:24:05 2017 from 192.168.59.10
[demo@rhce ~]$ uptime
10:26:20 up 19:46, 3 users, load average: 0.51, 0.14, 0.08
[demo@rhce ~]$ ifconfig
ens33: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.59.133 netmask 255.255.255.0 broadcast 192.168.59.255
inet6 fe80::501e:8a2:2c44:a64 prefixlen 64 scopeid 0x20<link>
ether 00:0c:29:81:06:9a txqueuelen 1000 (Ethernet)
RX packets 162104 bytes 201013659 (191.7 MiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 70217 bytes 4834892 (4.6 MiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 1 (Local Loopback)
RX packets 8 bytes 440 (440.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 8 bytes 440 (440.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

virbr0: flags=4099<UP,BROADCAST,MULTICAST> mtu 1500
inet 192.168.122.1 netmask 255.255.255.0 broadcast 192.168.122.255
ether 52:54:00:08:7f:61 txqueuelen 1000 (Ethernet)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

[demo@rhce ~]$ ================ssh for 192.168.59.134============================
spawn ssh -l demo 192.168.59.134
ssh: connect to host 192.168.59.134 port 22: No route to host
send: spawn id exp8 not open
while executing
"send "demo\r""
("foreach" body line 6)
invoked from within
"foreach server $data {
puts "================ssh for $server============================"

spawn ssh -l demo "$server"
expect "password: "
send "d..."
(file "exp.exp" line 9)

 

That is all for now. Happy expecting. 🙂

Sending an Email Using Shell

In this post I will introduce you about sending an email by using terminal, as sending an email via graphical MTA can be cumbersome.:) To  send an email on shell terminal, I will use ssmtp which sends mails local computer to configured mail hub. It is not  mail server such as postfix. Ssmtp package exists in epel-repo, so you need to add epel-repo, if you do not already add it.

Install Epel Repo:

#yum install epel-release

Install SSMTP:

#yum install ssmtp

After Installing ssmtp you need to configure files on /etc/ssmtp/ folder. There are two configuration files  in  /etc/ssmtp folder. ssmtp.conf file is actual configuration file that you add your mail account information. revealiases file is used for reverse mail aliases.

Configuration of SSMTP

Open ssmtp.conf  file with your favourite text editor and write down below configuration on end of the file and change it with your own account information. You can see my configuration.(Figure-1)

 

 

 

Figure-1(ssmtp.conf)

Testing Time:

Actually when I test it without configuring revealiases(reverse mail alias file) I got an error because of my local domain which is tbag.local. You can see below it MAIL FROM section.(Figure-2). I tried to send  an email on verbose mode. To solve the problem you also need to configure revealiases file (Figure3). Add your user and mail domain like below. After configuration of revealiases, I am able to send an email successfully.(Figure-4)

ssmtp -v manintheit@gmail.com
Hello world !
[<-] 220 smtp.mail.yahoo.com ESMTP ready
[->] EHLO dns.tbag.local
[<-] 250 STARTTLS
[->] STARTTLS
[<-] 220 2.0.0 Start TLS
[->] EHLO dns.tbag.local
[<-] 250 AUTH PLAIN LOGIN XOAUTH2 XYMCOOKIE
[->] AUTH LOGIN
[<-] 334 VXNlcm5hbWU8
[->] c2VuZG1lbWVsb25AeWFob28uTv42
[<-] 334 UGFzc3cvctQ6
[<-] 235 2.0.0 OK
[->] MAIL FROM:<root@dns.tbag.local>
[<-] 553 From address not verified - see http://help.yahoo.com/l/us/yahoo/mail/original/manage/sendfrom-07.html
ssmtp: 553 From address not verified - see http://help.yahoo.com/l/us/yahoo/mail/original/manage/sendfrom-07.html

Figure-2 (Error while sending an email)

root:sendmemelon@yahoo.com:smtp.mail.yahoo.com:587
demo:sendmemelon@yahoo.com:smtp.mail.yahoo.com:587

Figure-3 (revealiases)

[root@dns ssmtp]# ssmtp -v manintheit@gmail.com
Hello World !
[<-] 220 smtp.mail.yahoo.com ESMTP ready
[->] EHLO dns.tbag.local
[<-] 250 STARTTLS
[->] STARTTLS
[<-] 220 2.0.0 Start TLS
[->] EHLO dns.tbag.local
[<-] 250 AUTH PLAIN LOGIN XOAUTH2 XYMCOOKIE
[->] AUTH LOGIN
[<-] 334 VXNpnx5hlWU9
[->] c2VuZG1lbWVsb25AetGob32uY29m
[<-] 334 UGFzc3uvcmQ6
[<-] 235 2.0.0 OK
[->] MAIL FROM:<sendmemelon@yahoo.com>
[<-] 250 OK , completed
[->] RCPT TO:<manintheit@gmail.com>
[<-] 250 OK , completed
[->] DATA
[<-] 354 Start Mail. End with CRLF.CRLF
[->] Received: by dns.tbag.local (sSMTP sendmail emulation); Fri, 10 Mar 2017 01:40:30 +0300
[->] From: "root" <sendmemelon@yahoo.com>
[->] Date: Fri, 10 Mar 2017 01:40:30 +0300
[->] Hello World !
[->]
[->] .
[<-] 250 OK , completed
[->] QUIT
[<-] 221 Service Closing transmission

Figure-4(Sending an email)

 

 

 

 

 

 

One more thing!

Mail providers such as gmail, yahoo prohibit sending an email by some third party applications such as ssmtp, sendmail etc. To allow sending an email such applications, you should turn on less secure sign-in property. For yahoo you can do that on the Account Security tab.(Figure-5)

Figure-5(Turning on less secure sign-in for yahoo)

 

 

 

What is My IP Address ?(in a shell)

There are couple of web sites that outputs your public IP as a plain. I wrote it down  you can pick one of them.

  • curl icanhazip.com
  • curl ipinfo.io/ip
  • wget -qO- http://ipecho.net/plain ; echo
  • curl ident.me ;echo