CentOS ISCSI Target Server Setup

Requirements & Best Practices

ISCSI should have a dedicated disk or a dedicated LV to use, ensure you have sufficient space to create a new LV, otherwise storage should be added. If you are using SELInux please make appropriate changes or do the following to disable SELinux:

setenforce 0
sed -i 's/=enforcing/=disabled/g' /etc/sysconfig/selinux

 

If you are using iptables firewall be sure to add a rule to iptables to allow communication on port 3260 for iSCSI.

vim /etc/sysconfig/iptables

Above the explicit deny line in the iptables config enter the following line item:

-A INPUT -m state --state NEW -m tcp -p tcp --dport 3260 -j ACCEPT

Restart IP tables:

service iptables restart

 

Install and Enable Target Server

install the required packages

yum install scsi-target-utils -y

Enable for auto-start

chkconfig tgtd on

Start the Daemon

/etc/init.d/tgtd start

 

Configure Targets

Edit the contents of of the targets configuration file

vim /etc/tgt/targets.conf

Edit the following block:

# Set the driver. If not specified, defaults to "iscsi".
#
# This can be iscsi or iser. To override a specific target set the
# "driver" setting in the target's config.
default-driver iscsi
#<target iqn.2008-09.com.example:iser>
#       Example: the next line would override default driver type.
#       driver iser
#<target>

You’ll need to include the target iqn info as well as the backing store information. When you are finished it should look something like:

# Set the driver. If not specified, defaults to "iscsi".
#
# This can be iscsi or iser. To override a specific target set the
# "driver" setting in the target's config.
default-driver iscsi
<target iqn.2015-08.com.tgsrv1:tgt1>
        backing-store /dev/vg_centos6/iscsi
</target>
#       Example: the next line would override default driver type.
#       driver iser
#<target>

Once you have completed this step you will need to restart the tgtd servie

service tgtd restart

At this point we can verify that the ISCSI target LUN we configured is now visible:

tgtadm --mode target --op show

If you completed the configuration correctly you should see the LUN0 the control LUN and LUN1 which will the LUN we just created. It will look something like the example below:

[root@localhost ~]# tgtadm --mode target --op show
Target 1: iqn.2015-08.com.tgsrv1:tgt1
    System information:
        Driver: iscsi
        State: ready
    I_T nexus information:
    LUN information:
        LUN: 0
            Type: controller
            SCSI ID: IET     00010000
            SCSI SN: beaf10
            Size: 0 MB, Block size: 1
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            Backing store type: null
            Backing store path: None
            Backing store flags:
        LUN: 1
            Type: disk
            SCSI ID: IET     00010001
            SCSI SN: beaf11
            Size: 20401 MB, Block size: 512
            Online: Yes
            Removable media: No
            Prevent removal: No
            Readonly: No
            Backing store type: rdwr
            Backing store path: /dev/vg_centos6/iscsi
            Backing store flags:
    Account information:
    ACL information:
        ALL

Monitoring Your Servers for Free (Part 4)

It has certainly been a month of monitoring blog posts over here! I have been running a little behind schedule thanks to this chest cold I caught, however I’m on my way to recovery and with a little coffee and determination we’ll get back to it! This post will be the epic conclusion of our monitoring your servers for free series. For this demonstration we will use the Turn Key Linux virtual appliance for Observium built on top of Debian Wheezy. If you’d like to install Observium on your RHEL or CentOS server, instructions can be found at http://www.observium.org/wiki/RHEL_Installation

 

 

Download & Deploy TurnKey Linux Observium Virtual Appliance

To download the appliance visit http://www.turnkeylinux.org/observium. You will have several choices as to what format you want to download the appliance in. If you are using Hyper-V I would recommend using the ISO, however since I will be using VMware ESXi 5.5 we will download the OVA template. Once it’s downloaded to your local machine follow the steps below:

  1. Unzip the OVA template
  2. Open vSphere client and click File>Deploy OVF Template
  3. In the Deploy OVF Template wizard click Browse and point to the OVF template you unzipped
  4. Proceed through the remaining steps in the wizard customizing as appropriate for your environment
  5. Power on the Turnkey VM and open the console
  6. Proceed through the setup wizard for the virtual appliance

 

Prepare Windows Servers for Observium

Before we can begin reporting data to Observium, we must first ensure that we have enabled SNMP. This can either be done through server manager using the add roles/features wizard. In addition you will need to ensure that the hostname the servers you’re monitoring can be resolved Observium, this may require entries into your local DNS server if DNS entries do not already exist, or entries into the /etc/hosts on the Observium server.

  1. Install SNMP (see above)
  2. Restart the Server (the SNMP options needed aren’t active until after reboot)
  3. Open Services (run command: services.msc)
  4. Double click the SNMP service
  5. Click the Security Tab
  6. Add a community name (ie: Observe)
  7. Add the IP of the Observium server to the “Accept SNMP packets from these hosts” list
  8. Open a browser and navigate to your Observium server’s web page and login
  9. Under the devices menu choose add device
  10. Provide the hostname
  11. Choose SNMP v1
  12. Enter SNMP community name set in step 6 and click add device
  13. Wait a few minutes for Observium to begin collecting data on your new server

 

Monitoring Linux Servers and Cisco Equipment

While I typically use New Relic for Linux monitoring, Observium has published instructions for adding Linux servers. Additionally there are also instructions available for adding Cisco switches and firewalls. Again the gotcha here is that A records must exist in local DNS or entries must be created in /etc/hosts on the Observium server, as all devices are managed by hostname not by IP.

Linux: http://www.observium.org/wiki/NetSNMPd_Client_Configuration

Cisco: http://www.observium.org/wiki/Cisco_IOS_SNMP_Configuration

 

Video

Coming Soon!

Monitoring Your Servers for Free (Part 3)

As we have reached part 3 or 4 in our series on monitoring your servers for free, I would like to take a moment to highlight a few gotcha with Nagios before moving into our final section where we will setup graphical monitoring with Observium. At this point we’ve gone through building our a Nagios server, setting up contacts, monitoring printers, as well as monitoring both Windows and Linux servers. There are certainly a few issues you are bound to encounter on your journey with Nagios monitoring. These issues cost me hours of struggle, digging, testing, and ultimately coming upon a resolution. I would like to pay forward some of this effort in the hope that I can help save some poor soul out there the time, effort, untold amounts of silent cursing and coffee drinking.

 

Monitoring SQL Express

Chances are if you have been a Windows administrator for very long and have deployed and application server or two, you have undoubtedly had to setup at least one SQL Express database. The challenge of monitoring SQL Express with Nagios is that our friends at Microsoft have created a service name that contains a dollar sign (MSSQL$SQLEXPRESS). The challenge is that the $ character must be escaped using $ in order for Nagios to correctly read the service name. Use the following service definition below as reference for making SQL Express play nice with Nagios.

define service{

use generic-service

host_name host.example.com

service_description Service: SQL Server – SQLEXPRESS

check_command check_nt!SERVICESTATE!-d SHOWALL -l MSSQL$$SQLEXPRESS

}

Nagios localhost Warnings for HTTP

On your Nagios server you may encounter a yellow warning message regarding the HTTP service. The reason for this is that the NRPE client running on Linux servers is checking to ensure that apache is running and that it can locate and index file within the default docroot for apache. Resolving this issue on your Nagios server can be as simple as creating a blank file called index.html in the /var/www/html directory. A simple way to accomplish this is to use the touch command (ex touch /var/www/html/index.html

Once you create this dummy file you will need to restart apache (service httpd restart on RHEL or service apache2 restart on Debian variants) as well as the nagios service (service nagios restart).

 

Nagios Error When Rescheduling a Service Check (Error: Could not open command file ‘usr/local/nagios/var/rw/nagios.cmd’ for update!)

This and the SQL Express issue may very well be the most aggravating Nagios issues I encountered in my first deployment. However through being highly caffeinated and stubborn I did eventually find a solution. The root of the problem is that there is a permission setting that is getting flipped each time the Nagios service restarts. I have seen a few different fixes for this issue that both work. The first method I have seen corrects the problem at its root by correcting the broken permissions. However if this doesn’t resolve the issue for you, another alternative is to use a script that resets the permission each time the Nagios service runs. I recommend attempting the first method first as it is the more preferable fix, however if this doesn’t resolve your issue, try method two. Please note, terminal commands are in italics

Method 1:

#usermod -G nagios apache

#grep nagios /etc/group (ensure that the result shows that nagios is part of the apache group)

#service httpd restart (substitute apache2 instead of httpd on Debian based variants)

 

Method 2:

Alex Nogard’s blog lays out the methodology for creating a script that fixes the permission each time Nagios starts by adding the script into init.d/nagios. Please follow the link below for his instructions:

http://en.alexnogard.com/error-could-not-open-command-file-usrlocalnagiosvarrwnagios-cmd-for-update/

 

Automating NRPE Agent Deployment Through Puppet

Although outside the scope of this particular discussion, one way to automate deployment of NRPE throughout multiple web servers is to use puppet to facilitate this. I will perhaps visit this topic in later posts regarding Puppet Labs, however I will sum this point up quickly in a nutshell before we wrap up. If you have an existing Puppet infrastructure, you can simply have Puppet add the EPEL repo to each LAMP server, create an ensure installed statement to ensure that NRPE is installed, and finally push out a preconfigured nrpe.cfg that contains the correct server information for your Nagios server (located in /etc/nagios).

 

That brings me to the end of part 3 in our series on monitoring your servers for free. In the next installment, we’ll take a look at deploying Observium to give us graphical output for our servers. Til next time, may the coffee be endless and the uptime in your favor!