Write Your own Custom Plugin on check_mk

The check_mk is a open source monitoring solution with hundreds of checks which enables you to monitor your IT infrastructure. Besides it allows you to configure most of your monitor related activities from the graphical interface called “WATO”. Most of the time it automatically discover the system, once it is added to the check_mk inventory. But Not always! Nevertheless, we are still able to monitor the system by writing custom plugin. In this post, I will share you how to write your own custom plugin on check_mk. As an experiment, I used my home router. which is Ubiquiti Edge Router X. I enabled the SNMP version 2 on my router.

Before writing a plugin we need to decide on what we are going to monitor and second thing figuring out the related SNMP oid. In this post, I am going to monitor the bandwidth usage of the interface eth0 on my router. To find the related oid’s you can use the snmpwalk tool. When you run the code with the correct parameters system give the all the information. If you do not download the necessary MIB file on your system you will see all the elements with its oid, which is hard to know.

#snmpwalk -v2c -c public 192.168.1.1 . | less
iso.3.6.1.2.1.1.1.0 = STRING: "EdgeOS v1.10.8.5142457.181120.1809"
iso.3.6.1.2.1.1.2.0 = OID: iso.3.6.1.4.1.41112.1.5
iso.3.6.1.2.1.1.3.0 = Timeticks: (11759) 0:01:57.59
iso.3.6.1.2.1.1.4.0 = STRING: "root"
iso.3.6.1.2.1.1.5.0 = STRING: "ubnt"
iso.3.6.1.2.1.1.6.0 = STRING: "Unknown"
iso.3.6.1.2.1.1.7.0 = INTEGER: 14
iso.3.6.1.2.1.1.8.0 = Timeticks: (18) 0:00:00.18
iso.3.6.1.2.1.1.9.1.2.1 = OID: iso.3.6.1.2.1.10.131
iso.3.6.1.2.1.1.9.1.2.2 = OID: iso.3.6.1.6.3.11.3.1.1
iso.3.6.1.2.1.1.9.1.2.3 = OID: iso.3.6.1.6.3.15.2.1.1
...(omitted)

If your output like the above, you may need to download MIB files. You can download the snmp-mibs-downloader on GNU/Linux. If you are lucky the snmpwalk give you all the information by translating to text notation which is more meaningful for us.

# apt-get install snmp-mibs-downloader
# download-mibs
# snmpwalk -v2c -c public 192.168.1.1 . | less

SNMPv2-MIB::sysDescr.0 = STRING: EdgeOS v1.10.8.5142457.181120.1809
SNMPv2-MIB::sysObjectID.0 = OID: SNMPv2-SMI::enterprises.41112.1.5
SNMPv2-MIB::sysUpTime.0 = Timeticks: (44730) 0:07:27.30
SNMPv2-MIB::sysContact.0 = STRING: root
SNMPv2-MIB::sysName.0 = STRING: ubnt
SNMPv2-MIB::sysLocation.0 = STRING: Unknown
SNMPv2-MIB::sysServices.0 = INTEGER: 14
SNMPv2-MIB::sysORLastChange.0 = Timeticks: (18) 0:00:00.18
SNMPv2-MIB::sysORID.1 = OID: SNMPv2-SMI::transmission.131
SNMPv2-MIB::sysORID.2 = OID: SNMPv2-SMI::snmpModules.11.3.1.1
SNMPv2-MIB::sysORID.3 = OID: SNMPv2-SMI::snmpModules.15.2.1.1
SNMPv2-MIB::sysORID.4 = OID: SNMPv2-SMI::snmpModules.10.3.1.1



As I am going to monitor the bandwidth usage of the the interface for eth0. I need to find the related oid number, which are below.

#Text Notation
IF-MIB::ifDescr.1 = STRING: lo
IF-MIB::ifDescr.2 = STRING: switch0
IF-MIB::ifDescr.3 = STRING: imq0
IF-MIB::ifDescr.4 = STRING: eth0
IF-MIB::ifDescr.5 = STRING: eth1
IF-MIB::ifDescr.6 = STRING: eth2
IF-MIB::ifDescr.7 = STRING: eth3
IF-MIB::ifDescr.8 = STRING: eth4
IF-MIB::ifDescr.9 = STRING: eth1.20
IF-MIB::ifDescr.10 = STRING: eth1.10


IF-MIB::ifInOctets.1 = Counter32: 38770
IF-MIB::ifInOctets.2 = Counter32: 0
IF-MIB::ifInOctets.3 = Counter32: 0
IF-MIB::ifInOctets.4 = Counter32: 307201
IF-MIB::ifInOctets.5 = Counter32: 0
IF-MIB::ifInOctets.6 = Counter32: 0
IF-MIB::ifInOctets.7 = Counter32: 0
IF-MIB::ifInOctets.8 = Counter32: 0
IF-MIB::ifInOctets.9 = Counter32: 0
IF-MIB::ifInOctets.10 = Counter32: 0

IF-MIB::ifOutOctets.1 = Counter32: 38770
IF-MIB::ifOutOctets.2 = Counter32: 424
IF-MIB::ifOutOctets.3 = Counter32: 0
IF-MIB::ifOutOctets.4 = Counter32: 295279
IF-MIB::ifOutOctets.5 = Counter32: 0
IF-MIB::ifOutOctets.6 = Counter32: 0
IF-MIB::ifOutOctets.7 = Counter32: 0
IF-MIB::ifOutOctets.8 = Counter32: 0
IF-MIB::ifOutOctets.9 = Counter32: 0
IF-MIB::ifOutOctets.10 = Counter32: 0

#OID Notation

.1.3.6.1.2.1.2.2.1.2.1 = STRING: lo
.1.3.6.1.2.1.2.2.1.2.2 = STRING: switch0
.1.3.6.1.2.1.2.2.1.2.3 = STRING: imq0
.1.3.6.1.2.1.2.2.1.2.4 = STRING: eth0
.1.3.6.1.2.1.2.2.1.2.5 = STRING: eth1
.1.3.6.1.2.1.2.2.1.2.6 = STRING: eth2
.1.3.6.1.2.1.2.2.1.2.7 = STRING: eth3
.1.3.6.1.2.1.2.2.1.2.8 = STRING: eth4
.1.3.6.1.2.1.2.2.1.2.9 = STRING: eth1.20
.1.3.6.1.2.1.2.2.1.2.10 = STRING: eth1.10

.1.3.6.1.2.1.2.2.1.10.1 = Counter32: 55901
.1.3.6.1.2.1.2.2.1.10.2 = Counter32: 0
.1.3.6.1.2.1.2.2.1.10.3 = Counter32: 0
.1.3.6.1.2.1.2.2.1.10.4 = Counter32: 967790
.1.3.6.1.2.1.2.2.1.10.5 = Counter32: 0
.1.3.6.1.2.1.2.2.1.10.6 = Counter32: 0
.1.3.6.1.2.1.2.2.1.10.7 = Counter32: 0
.1.3.6.1.2.1.2.2.1.10.8 = Counter32: 0
.1.3.6.1.2.1.2.2.1.10.9 = Counter32: 0
.1.3.6.1.2.1.2.2.1.10.10 = Counter32: 0

.1.3.6.1.2.1.2.2.1.16.1 = Counter32: 55901
.1.3.6.1.2.1.2.2.1.16.2 = Counter32: 424
.1.3.6.1.2.1.2.2.1.16.3 = Counter32: 0
.1.3.6.1.2.1.2.2.1.16.4 = Counter32: 1031485
.1.3.6.1.2.1.2.2.1.16.5 = Counter32: 0
.1.3.6.1.2.1.2.2.1.16.6 = Counter32: 0
.1.3.6.1.2.1.2.2.1.16.7 = Counter32: 0
.1.3.6.1.2.1.2.2.1.16.8 = Counter32: 0
.1.3.6.1.2.1.2.2.1.16.9 = Counter32: 0
.1.3.6.1.2.1.2.2.1.16.10 = Counter32: 0


As it is depicted above, we need a interface name, IfInOctets and IfOutOctets values of the Interface to monitor the bandwidth usage.

Math!

We also need some basic math to abstract the the data. Because SNMP gives us the bandwidth usage values counter based octets. So, we need to know how to abstract it.

Formula:

   (Counter2 - Counter1)
============================ * 8  Gives the bps.(bit-per-second)
       (Time2 - Time1)

If you still do not understand you can chek the page. It is a piece of cake! 🙂

Actually check_mk has a very nice function that keeps track of the time and counter values and takes the delta automatically . So we do not need to store anything. We only need to pass the correct values to the get_rate() function.

For custom plugin we need to put the custom plugin into the correct folder.

/omd/sites/<your site>/local/share/check_mk/checks

I created my plugin called “edge_router_bw”

#!/usr/bin/env python
edge_router_default_bw_values = (30.0, 35.0, 30.0, 35.0)
def inventory_edge_router_bw(info):
    for interface, inoctets, outoctets  in info:
        yield interface, "edge_router_default_bw_values"

def check_edge_router_bw(item, params, info):
    warntx, crittx, warnrx, critrx = params
    for interface, inoctets, outoctets  in info:
        if interface == item:
            this_time = time.time()
            if interface == "eth0":
                rx = 8.0 * get_rate("RX.%s" % interface, this_time, float(inoctets))
                tx = 8.0 * get_rate("TX.%s" % interface, this_time, float(outoctets))
                perfdata = [("RX", float(rx)/1024.0), ("TX", float(tx)/1024.0)]
                tx = float(tx)/1024.0
                rx = float(rx)/1024.0
                if rx >= critrx or tx >= crittx:
                    return 2, ("RX: %.2f Kbps, TX: %.2f Kbps" % (rx, tx)), perfdata
                elif rx >= warnrx or tx  >= warntx:
                    return 1, ("RX: %.2f Kbps, TX: %.2f Kbps" % (rx, tx)), perfdata
                else:
                    return 0, ("RX: %.2f Kbps, TX: %.2f Kbps" % (rx, tx)), perfdata
            
check_info["edge_router_bw"] = {
    "check_function"        : check_edge_router_bw,
    "inventory_function"    : inventory_edge_router_bw,
    "service_description"   : "Edge Router NICs bandwith %s",
    "snmp_info"             : ( ".1.3.6.1.2.1.2.2.1", [ "2", "10", "16"] ),
    "has_perfdata"          : True,
    "group"                 : "edge_router_bw",
}

One of of the nice feature of check_mk is that, you can set your threshold values for warning and critical levels. Then you can set/change these levels from the WATO. No more editing of the files. For that, you need to create below configuration file, which allows you to make changes from the WATO.

Create a file on the /omd/sites/local/share/check_mk/web/plugins/wato/.

I created a file called check_param_router_edge_bw.py

register_check_parameters(
        subgroup_networking,
        "edge_router_bw",
        _("Edge router Bandwith Kbps"),
        Tuple(
            title = _("Edge Router Interface Bandwith"),
            elements = [
                Float(title = _("Set WARNING if TX above Kbps"), minvalue = 0.0, maxvalue = 10000.0, default_value = 30.0),
                Float(title = _("Set CRITICAL if TX  above Kbps"), minvalue = 0.0, maxvalue = 10000.0, default_value = 35.0),
                Float(title = _("Set WARNING if RX above Kbps"), minvalue = 0.0, maxvalue = 10000.0, default_value = 30.0),
                Float(title = _("Set CRITICAL if RX above Kbps"), minvalue = 0.0, maxvalue = 10000.0, default_value = 35.0),
            ]),
            TextAscii(
                title = _("Inteface Bandwith Kbps"),
                allow_empty = False),
            "first"
)

Once everything has finished, we can check our plugin. Check the script with the –debug option if there is an error in the script.

OMD[monitoring]:~/local/share/check_mk/checks$ cmk --debug --checks=edge_router_bw -I edgerouter

If there is no any error we can inventory the host in the check _mk

OMD[monitoring]:~/local/share/check_mk/checks$ cmk -IIv edgerouter
Discovering services on edgerouter:
edgerouter:
   10 edge_router_bw
   10 edge_router_params
    1 hr_cpu
    4 hr_fs
    1 hr_mem
    2 if64
    1 snmp_info
    1 snmp_uptime

Finally, testing the plugin on the command line.

OMD[monitoring]:~/local/share/check_mk/checks$ cmk -nvp edgerouter Check_MK version 1.4.0p38<br> CPU utilization      OK - 1.5% used                                           (util=1.5;80;90;0;100)<br> Edge Router NICs bandwith eth0 OK - RX: 2.45 Kbps, TX: 6.59 Kbps                        (RX=2.453042;;;; TX=6.589328;;;;)<br>

You can then login to the check_mk graphical interface.

Experiment

As you see below, you can change the WARNING and CRITICAL levels from the WATO.

That’s all for now. Happy monitoring 🙂