Packer

Hi Folks!

Introduction:

In this post, I will introduce you about packer. Packer is actually open source tool for creating machine and container images for multiple platforms from single source of configuration. As a configuration It uses a json syntax for creating machines for different platforms such as  Amazon EC2, Docker, VirtualBox, VMware etc. Actually I will not get into more detail. For more information please visit website. As an example, I created very simple configuration for VMware Workstation platform by installing CentOS 7 without keyboard interaction. For non-interactive installation(kick-start installation). I have used kickstart file. Next tutorial will be about kick-start installation of RedHat system.

Installing Packer:

I have a CentOS 7 system, So you can download packer.zip file by using wget command.(Figure-1)

For 64 bit system.

[root@hulk ~]# wget -O packer.zip https://releases.hashicorp.com/packer/0.12.3/packer_0.12.3_linux_amd64.zip?_ga=1.60002541.1517413558.1489953845

Figure-1 Download packer.zip file

Unzip the file.

[root@hulk ~]# unzip packer.zip

Usage of Packer:

You can run packer the same as ordinary executable Linux file like below.(Figure-2)

./packer -h
usage: packer [--version] [--help] <command> [<args>]

Available commands are:
    build       build image(s) from template
    fix         fixes templates from old versions of packer
    inspect     see components of a template
    push        push a template and supporting files to a Packer build service
    validate    check that a template is valid
    version     Prints the Packer version

Figure-2 Usage of Packer

We have packer, but we do not have any configuration file to tell the packer what to do. Below, I shared very simple packer configuration for VMware Workstation platform to give you an overview. For other platforms please see packer documents. Plus, For auto-installation, you will need a machine to serve kick-start.(dhcp, tftp, http). Do not worry! Next tutorial will be about kick-start installation. So For now, stick with packer configuration.

{
    "builders": [{
        "name": "centos-latest",
        "type": "vmware-iso",
        "guest_os_type": "centos-64",
        "tools_upload_flavor": "linux",
        "iso_url": "http://mirror.fibersunucu.com.tr/centos/7/isos/x86_64/CentOS-7-x86_64-Minimal-1611.iso",
        "iso_checksum": "27bd866242ee058b7a5754e83d8ee8403e216b93d130d800852a96f41c34d86a",
        "iso_checksum_type": "sha256",
        "ssh_username": "root",
        "ssh_password": "vagrant",
        "ssh_port": "22",
        "ssh_host": "192.168.17.38",
        "ssh_wait_timeout": "10000s",
        "shutdown_timeout": "20s",
        "shutdown_command": "shutdown -P now",
        "vm_name": "mypacker",
        "format": "vmx",
        "boot_wait": "10s",
        "http_directory": "/var/www/html/http",
        "http_port_min": "8008",
        "http_port_max": "8008",
        "disk_size": "12000",
        "vmx_data": {
        "cpuid.coresPerSocket": "1",
        "memsize": "1024",
        "numvcpus": "1",
        "ethernet0.present": "TRUE",
        "ethernet0.connectionType": "nat"

      },
        
        "boot_command": [
        "<tab> ks=http://{{ .HTTPIP }}:{{ .HTTPPort }}/ks.cfg<enter><wait>"
      ]

    }],


    "provisioners": [
  {
    "type": "shell",
    "inline": [
      "touch /home/vagrant/test1234.packer",
      "echo 'Hello Packer !' > /home/vagrant/test1234.packer"
    ]
  }]

}

Figure-3 Simple packer directive first.json file

Verify first.json with Packer

Before building a machine, configuration file should be verified for error-free.

[root@hulk packer]# ./packer validate first.json
Template validated successfully.

Build first.json with Packer

After successful validation, we can build our guest on VMware Workstation. Just run ./packer build first.json. While packer is building,  take some coffee 🙂

[root@hulk packer]# ./packer build first.json

Figure-4 Packer building

Final!

Kickstart File

Kickstart(ks.cfg) is  a actually configuration file for RedHat based systems to install system without human interaction. In doing so, hundreds of system can be installed just a snippet of directives.(Figure-4)

#version=DEVEL
# System authorization information
auth --enableshadow --passalgo=sha512
# Use CDROM installation media
url --url http://192.168.17.25/rhel7/
# Use graphical install
text
# Run the Setup Agent on first boot
firstboot --enable
ignoredisk --only-use=sda
# Keyboard layouts
keyboard --vckeymap=tr --xlayouts='tr'
# System language
lang en_US.UTF-8
reboot
# Network information
network  --bootproto static --device=ens33 --gateway 192.168.17.2 --ip 192.168.17.38 --nameserver 192.168.17.2,8.8.8.8 --netmask=255.255.255.0  --activate
network  --hostname vagrant.tbag.local

# Root password
rootpw vagrant --plaintext
# System services
services --disabled="chronyd"
# System timezone
timezone Europe/Istanbul --isUtc --nontp
user --name=vagrant --password=vagrant --plaintext
# System bootloader configuration
bootloader --append=" crashkernel=auto" --location=mbr --boot-drive=sda
# Partition clearing information
clearpart --none --initlabel
# Disk partitioning information
autopart --type=lvm

%packages
@base
kexec-tools

%end



Figure-4 Sample Kickstart File

 

 

 

 

 

SSH Two-Factor Authentication

Hi Folks!

Proliferation of the Internet changes our life that almost anything demands for the Internet. Our personal information, bank accounts, Company’ secrets etc.. So, how we protect our computer systems from intruders ? As you know most common method is password authentication. But People are so inclined to use short passwords. Brute-Force methods prevails it. So, We need additional authentication information beside password. One of them is Two-Factor authentication. And it has different kind of versions. In this post we will use OTP which generates code from  time-synchronization. So Time accuracy is important. To do so, my host pulls time from reliable NTP servers.

Install necessary Packages.

You  have to be root user to  install necessary packages.

#yum install google-authenticator

Run google-authenticator

Now, You can switch your normal account. And run google-authenticator command on a shell. It will ask you a couple of questions. You can see the picture. (Figure-1)

                                                            Figure-1 It creates a QR code

                                      Figure-2 It asks a couple of options to configure

Install Google Authenticatior on Your Android Phone

You also need to install Google Authenticator Application on your Android phone. Run the application and scan the QR code that we have already created it by running google-authenticator command on the shell.(Figure-1)

Figure-3 Google Authenticatior Android Application

Configuration of google-authenticator for SSH.

We will use two-factor authentication for SSH protocol. So we need to add google-authenticator pam module. Add the line below top of the sshd file on the path /etc/pam.d

auth required pam_google_authenticator.so

Configure sshd_config File

Edit your sshd_config file on the path /etc/ssh and change one option from “ChallengeResponseAuthentication no” to “ChallengeResponseAuthentication yes”

ChallengeResponseAuthentication yes

Restart SSH service

systemctl restart sshd.service

Test Time!

Now try to login to the host that you configured for Two-Factor authentication. You can see that I also need a verification code.

      Figure-3 It asks me verification code beside password.

 

 

Kick Start Questions and Answers

Q1: I configured my kickstart(ks.cfg) network configuration as static IP. But  it reverts to dhcp after installiation. Why?

network  --bootproto static --device=eth0 --gateway 192.168.17.2 --ip 192.168.17.38 --nameserver 192.168.17.2,8.8.8.8 --netmask=255.255.255.0  --activate
network  --hostname vagrant.tbag.local

And I said : do not forget to add device parameter in your kick start file and it has to be proper device name. If you configure wrong device name it reverts to dhcp after installiation. eth0 is wrong device name according to my device  configuration. After proper device configuration(here is ens33) it works.

network  --bootproto static --device=ens33 --gateway 192.168.17.2 --ip 192.168.17.38 --nameserver 192.168.17.2,8.8.8.8 --netmask=255.255.255.0  --activate
network  --hostname vagrant.tbag.local

Q2: When Kickstart starts dracut  gives me an error like “no space has left on disk….”

And I said: Do not use minimal installation ISO. Instead,  Use DVD ISO image !

 

Java Time Zone Update

Hi Folks!

We may face consequences that we need to change time zone due to government regulations such as daylight saving time and other purposes. And it is inevitable. For OS side it is very easy to update system time zone. For Linux tzdata package comes for this purpose. But If we have Java applications, updating an OS tzdata package is not enough because Java has its own time zone database. So we need to update Java time zone as well.

I confronted last week that, one of Java application(Manage Engine) of the company logged its log file one hour back, even OS time is as expected. Short time later, I noticed that Java time zone is not updated. After some working I have found — In java 1.7 uses zi folder for time zone information whereas, Java 1.8 uses  tzdb.dat file. It sounds complicated ? No, Actually does not, Oracle has a tiny tool to update Java time zone. you can download it here.

After download, unzip the file. you will have tzupdate.jar file and it has some options you can run it .(Figure-1)

 #./jre/bin/java -jar  tzupdater.jar
Usage: java -jar tzupdater.jar options
options:

        -h, --help     Print this message and exit.
        -V, --version  Print tool version, tzdata version in JRE, tzdata version
                       that tool would update to and then exit.

        -l, --location <URL link to tzdata.tar.gz archive file>

                       Compile, test and update JRE timezone data from provided tzdata.tar.gz bundle

                       e.g -l http://www.iana.org/time-zones/repository/tzdata-latest.tar.gz

                       Supported URL protocols : http://, https://, file://

                       If no URL link is provided, tool will use latest IANA tzdata bundle at:

                           http://www.iana.org/time-zones/repository/tzdata-latest.tar.gz

        -f, --force    Force a tzdata update. Use this option if updating to an older tzdata version.

        -v, --verbose  Display detailed messages to stdout.

Figure-1

In order to update time zone for Java we need to have something that holds newest time zone information. Actually you can find it. The Internet Assigned Numbers Authority (IANA). Actually, I will not get in more detail what IANA does. You can find more information its official website. For time zone file you can download it here. it is periodically updated by IANA. For package, We chose Data Only Distribution package, in which case we will download tzdata2017a.tar.gz  So far so good.

TIP.

To download a file on shell you can use wget command.

wget https://www.iana.org/time-zones/repository/releases/tzdata2017a.tar.gz
--2017-03-16 07:57:50--  https://www.iana.org/time-zones/repository/releases/tzdata2017a.tar.gz
Resolving www.iana.org (www.iana.org)... 192.0.32.8, 2620:0:2d0:200::8
Connecting to www.iana.org (www.iana.org)|192.0.32.8|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 323983 (316K) [application/x-gzip]
Saving to: ‘tzdata2017a.tar.gz’
100%[================================================================================================================================================
2017-03-16 07:57:52 (306 KB/s) - ‘tzdata2017a.tar.gz’ saved [323983/323983]

Update Java Timezone with tzupdater.

Do not forget to stop the Java applications before update the time zone.

Actually when I first run the command It gave me an error, lucky it is clear.

./jre/bin/java -jar  tzupdater.jar  -l file:///home/ansible/tzdata2017a.tar.gz
Using file:///home/ansible/tzdata2017a.tar.gz as source for tzdata bundle.
No SHA-512 hash file found: file:///home/ansible/tzdata2017a.tar.gz.sha512. If using http(s) resource location, the file may not yet be available

To solve the problem follow steps below.

1- Take the sha512sum of the tzdate2017a.tar.gz file

#sha512sum tzdata2017a.tar.gz
fe57930711998a0755749a2bf1b32fdbc9e6ee249762017beea52eae7e521236396300fe2d79298fd81eb6d73fb7567884c35995ed248be06188329d937e3873  tzdata2017a.tar.gz

2- Create a file with the name tzdata2017a.tar.gz.sha512

#touch tzdata2017a.tar.gz.sha512

3- Add sha512 hash value of the tzdate2017a.tar.gz to the file tzdata2017a.tar.gz.sha512

#echo fe57930711998a0755749a2bf1b32fdbc9e6ee249762017beea52eae7e521236396300fe2d79298fd81eb6d73fb7567884c35995ed248be06188329d937e3873 > tzdata2017a.tar.gz.sha512

4- Try again.

./jre/bin/java -jar  tzupdater.jar  -l file:///home/ansible/tzdata2017a.tar.gz

Using file:///home/ansible/tzdata2017a.tar.gz as source for tzdata bundle.

We got no error. When you look into the folder(<java folder>/jre/lib/) where Java time zone exists, you can see old time zone was replaced with the newest one.And older one still preserved with different name. zi2014c in this case.

Actually, my application has its own bundle(application’s Java folder). So that, I copied it to the other machine and applied time zone update it in that machine due to the production environment issues :). Then copy the zi(newest time zone data) folder the machine, where applications exist.

ME/working/jre/lib

drwxr-xr-x. 13 ansible ansible     4096 Mar 16 07:56 zi2014c
drwxrwxr-x. 13 ansible ansible     4096 Mar 16 08:00 zi

 

 

 

Sending an Email Using Shell

In this post I will introduce you about sending an email by using terminal, as sending an email via graphical MTA can be cumbersome.:) To  send an email on shell terminal, I will use ssmtp which sends mails local computer to configured mail hub. It is not  mail server such as postfix. Ssmtp package exists in epel-repo, so you need to add epel-repo, if you do not already add it.

Install Epel Repo:

#yum install epel-release

Install SSMTP:

#yum install ssmtp

After Installing ssmtp you need to configure files on /etc/ssmtp/ folder. There are two configuration files  in  /etc/ssmtp folder. ssmtp.conf file is actual configuration file that you add your mail account information. revealiases file is used for reverse mail aliases.

Configuration of SSMTP

Open ssmtp.conf  file with your favourite text editor and write down below configuration on end of the file and change it with your own account information. You can see my configuration.(Figure-1)

 

 

 

Figure-1(ssmtp.conf)

Testing Time:

Actually when I test it without configuring revealiases(reverse mail alias file) I got an error because of my local domain which is tbag.local. You can see below it MAIL FROM section.(Figure-2). I tried to send  an email on verbose mode. To solve the problem you also need to configure revealiases file (Figure3). Add your user and mail domain like below. After configuration of revealiases, I am able to send an email successfully.(Figure-4)

ssmtp -v manintheit@gmail.com
Hello world !
[<-] 220 smtp.mail.yahoo.com ESMTP ready
[->] EHLO dns.tbag.local
[<-] 250 STARTTLS
[->] STARTTLS
[<-] 220 2.0.0 Start TLS
[->] EHLO dns.tbag.local
[<-] 250 AUTH PLAIN LOGIN XOAUTH2 XYMCOOKIE
[->] AUTH LOGIN
[<-] 334 VXNlcm5hbWU8
[->] c2VuZG1lbWVsb25AeWFob28uTv42
[<-] 334 UGFzc3cvctQ6
[<-] 235 2.0.0 OK
[->] MAIL FROM:<root@dns.tbag.local>
[<-] 553 From address not verified - see http://help.yahoo.com/l/us/yahoo/mail/original/manage/sendfrom-07.html
ssmtp: 553 From address not verified - see http://help.yahoo.com/l/us/yahoo/mail/original/manage/sendfrom-07.html

Figure-2 (Error while sending an email)

root:sendmemelon@yahoo.com:smtp.mail.yahoo.com:587
demo:sendmemelon@yahoo.com:smtp.mail.yahoo.com:587

Figure-3 (revealiases)

[root@dns ssmtp]# ssmtp -v manintheit@gmail.com
Hello World !
[<-] 220 smtp.mail.yahoo.com ESMTP ready
[->] EHLO dns.tbag.local
[<-] 250 STARTTLS
[->] STARTTLS
[<-] 220 2.0.0 Start TLS
[->] EHLO dns.tbag.local
[<-] 250 AUTH PLAIN LOGIN XOAUTH2 XYMCOOKIE
[->] AUTH LOGIN
[<-] 334 VXNpnx5hlWU9
[->] c2VuZG1lbWVsb25AetGob32uY29m
[<-] 334 UGFzc3uvcmQ6
[<-] 235 2.0.0 OK
[->] MAIL FROM:<sendmemelon@yahoo.com>
[<-] 250 OK , completed
[->] RCPT TO:<manintheit@gmail.com>
[<-] 250 OK , completed
[->] DATA
[<-] 354 Start Mail. End with CRLF.CRLF
[->] Received: by dns.tbag.local (sSMTP sendmail emulation); Fri, 10 Mar 2017 01:40:30 +0300
[->] From: "root" <sendmemelon@yahoo.com>
[->] Date: Fri, 10 Mar 2017 01:40:30 +0300
[->] Hello World !
[->]
[->] .
[<-] 250 OK , completed
[->] QUIT
[<-] 221 Service Closing transmission

Figure-4(Sending an email)

 

 

 

 

 

 

One more thing!

Mail providers such as gmail, yahoo prohibit sending an email by some third party applications such as ssmtp, sendmail etc. To allow sending an email such applications, you should turn on less secure sign-in property. For yahoo you can do that on the Account Security tab.(Figure-5)

Figure-5(Turning on less secure sign-in for yahoo)

 

 

 

What is My IP Address ?(in a shell)

There are couple of web sites that outputs your public IP as a plain. I wrote it down  you can pick one of them.

  • curl icanhazip.com
  • curl ipinfo.io/ip
  • wget -qO- http://ipecho.net/plain ; echo
  • curl ident.me ;echo

 

Dynamic DNS

Hi Folks!

It has been long time, I have not posted anything since December.  Actually I had to do many things. I was battling upkeep of Company Servers and upgrading them. But during that time  I learned many things and I would like to share  with you all. In this post I will introduce you about dynamic DNS, which saves you hassle of persistently changing of your  Router IP  by ISP.

What is Dynamic DNS ?

Dynamic DNS (DDNS) is a service that maps Internet domains name  to IP addresses. It is similar Internet Domain Name service(DNS) but some differences.

Unlike DNS that allows mapping static IP to domain name and domain name to static IP, Dynamic DNS maps your domain name to your dynamic IP. By doing that even though, your IP changes you will access your home router with the your domain name that you choose. And you will be able to access your IP camera or IoTs. But Unlike DNS service that you configure it only once for one domain name, DDNS needs to be informed each time IP has changed. But do be afraid.:)

There are many Dynamic DNS services on the internet enterprise or free. In this post I will introduce you about free dynamic dns, which I am currently using it.–duckns  Duck DNS is free dynamic domain name services. You can signup with your google, twitter, facebook or reddit account. After successfully login, duckdns create a token for you. You will update your new IP with this token so keep it secret.(Figure-1)

                                                  Figure-1

Also write your domain name you choose in to the box with the name domain.(Figure-2)

                                                  Figure-2

Almost done.We have just couple of things to do. As I mention before, we have to feed dynamic DNS service with the new IP, each time IP changes.

To do so, I wrote a shell script which pools every 5 minutes to check if  IP changes. For more information you can visit the link. https://www.duckdns.org/install.jsp

You can tweak the shell script for your own purpose. (If you use this script do not forget to replace  XYXY, xxxxxxxx-yyyy-xxxx-yyyy-zzzzzzzzzzzz  and mail addresses with yours!)

Edited: To execute script below every 5 minutes, we need to add the script on  a crontab.

 

 */5 * * * *  ipchecker.sh

 

ipchecker.sh script

#!/bin/bash
newip=$(curl -s ifconfig.co)
oldip=$(head ip.txt)

echo "old:$oldip"
echo "new:$newip"

if [ "$oldip" != "$newip" ] ; then
        echo "$newip" > ip.txt
        /usr/bin/mail -s "oldIP:$oldip/NewIP:$newip" admin@manintheit.org < ip.txt
#do not forget to create a folder with the name "duckdns"
#$mkdir ~/duckdns
        echo url="https://www.duckdns.org/update?domains=XYXY&token=xxxxxxxx-yyyy-xxxx-yyyy-zzzzzzzzzzzz&ip=" | curl -k -o ~/duckdns/duck.log -K -
        res=$?
        if [ "$res" -eq 0 ] ; then
                /usr/bin/dig XYXY.duckdns.org +short | /usr/bin/mail -s "DuckDNS IP changed" admin@manintheit.org
        else
                /usr/bin/mail -s "DuckDNS Error!" admin@manintheit.org<.
        fi
fi

Figure-3 ipchecker.sh