Infrastructure Monitoring : XorMon NG Installation

Network

SAN
SANnav
SAN BNA
LAN

SAN network monitoring through Brocade Network Advisor is not implemented yet in XorMon NG, we expect its support later in 2024.
So far you might use STOR2RRD for its monitoring.

Do you want to let us know your priorities? Vote for your missing features and device support. More votes, bigger priority for us.

Prerequisites
SANnav Northbound Streaming
Install Python3 and modules
SANnav configuration in XorMon NG
(Optional) Kafka deployment in docker container

Prerequisites

SANnav 2.2.0.2+ configured to stream performance data to a Kafka cluster
Ability to install additional software packages to XorMon NG host
Network connectivity:
- SANnav ➡ Kafka Registry:8081 TCP
- SANnav ➡ Kafka Broker:9093 TCP
- XorMon NG ➡ Kafka Registry:8081 TCP
- XorMon NG ➡ Kafka Broker:9092 TCP
- XorMon NG ➡ SANnav:443 TCP

SANnav Northbound Streaming

XorMon NG collects data from SANnav using the Northbound Streaming feature.
Northbound Streaming provides support to securely stream performance and flow metrics from the switch to an external Kafka cluster (the northbound server).

Kafka cluster with schema registry must be configured in the customer's environment.

SANnav must be configured to stream data to the Kafka cluster.

SANnav NorthBound streaming is configured using SANnav REST API.
SANnav REST API documentation is available on https://support.broadcom.com/
Search for: "sannav rest api"

Before you configure northbound streaming, note the following prerequisites:

You must have the Northbound Streaming privilege with read/write permission and the All Fabrics area of responsibility (AOR). Refer to the Brocade SANnav Management Portal User Guide for details.
Historic data collection must be enabled on the SANnav server (it is enabled by default).
The switch must be discovered with proper SNMPv3 credentials.
For a northbound server environment with HTTPS access for the schema registry, the northbound Kafka cluster must be configured with the same public certificate as the northbound server.
Only one northbound server is supported.
Streaming is only through a secure channel (TLS).
IPv6 is not supported. SANnav-to-northbound-server communication uses IPv4.

Install Python3 and modules

Ubuntu / Debian

install additional packages (as root)

apt update && apt install python3 python3-pip python3-dev python3-requests gcc

add Confluent repository (as root)
https://docs.confluent.io/platform/current/installation/installing_cp/deb-ubuntu.html

wget -qO - https://packages.confluent.io/deb/7.7/archive.key | apt-key add -

cat > /etc/apt/sources.list.d/confluent.list << EOF
deb [arch=amd64] https://packages.confluent.io/deb/7.7 stable main
deb https://packages.confluent.io/clients/deb $(lsb_release -cs) main
EOF

install 'librdkafka-dev' (as root)

apt update && apt install librdkafka-dev

install Python modules in a virtual environment (as xormon)

su - xormon
cd ~/xormon-ng/server-nest/
python3 -m venv venv-sannav
venv-sannav/bin/python3 -m ensurepip --upgrade
venv-sannav/bin/python3 -m pip install --upgrade pip

venv-sannav/bin/python3 -m pip install kafkian==0.13.0 confluent-kafka==2.4.0 requests

Red Hat, CentOS, Rocky Linux, OracleVM

In case you are on the Virtual Appliance and have no access to the Internet repos then use this procedure

Install Python3 packages as root

[root@xorux]# yum install python3 python3-pip python3-devel python3-requests gcc

Import Confluent repository as root

[root@xorux]# rpm --import https://packages.confluent.io/rpm/7.7/archive.key

[root@xorux]# cat > /etc/yum.repos.d/confluent.repo << EOF
cat > /etc/yum.repos.d/confluent.repo << EOF
[Confluent]
name=Confluent repository
baseurl=https://packages.confluent.io/rpm/7.7
gpgcheck=1
gpgkey=https://packages.confluent.io/rpm/7.7/archive.key
enabled=1

[Confluent-Clients]
name=Confluent Clients repository
baseurl=https://packages.confluent.io/clients/rpm/centos/\$releasever/\$basearch
gpgcheck=1
gpgkey=https://packages.confluent.io/clients/rpm/archive.key
enabled=1
EOF

Install librdkafka as root

[root@xorux]# yum clean all && yum install librdkafka-devel

install kafkian module as stor2rrd
(lpar2rrd in case of Virtual Appliance)

[stor2rrd@xorux]$ pip3 install --user kafkian==0.13.0 confluent-kafka==2.4.0

When it ends with this error:

/tmp/pip-build-ka_v989g/confluent-kafka/src/confluent_kafka/src/confluent_kafka.c:1633:17: note: use option -std=c99 or -std=gnu99 to compile your code
    error: command 'gcc' failed with exit status 1

Then use this cmd instead of above one:

[stor2rrd@xorux]$ CFLAGS=-std=c99 pip3 install --user kafkian==0.13.0 confluent-kafka==2.4.0

SANnav configuration in XorMon NG

Note: SANnav can only be monitored from one XorMon NG instance at a time

Create SANnav user for XorMon NG
- Login to SANnav GUI
- Navigate to User Management
  SANnav ➡ Security ➡ SANnav User Management
- Create new user with Operator role
Add SANnav in the XorMon NG UI ➡ Settings icon ➡ Device ➡ Network ➡ SAN ➡ New ➡ SANnav
- REST API: SANnav IP address, port and XorMon NG user credentials
- KAFKA: Kafka NB server IP address and ports - this should be the IP address of your XorMon NG host

Product uses SNMP v1,2c,3 read-only access (port 161 UDP) to get data from LAN switches.
Generally all network devices having SNMP MIBs compatible with Cisco should work.

Enable SNMP communication from XorMon NG server to all LAN switches on port 161, UDP
Connections are always initiated from XorMon NG server side
To monitor VLAN must be enabled on the switch this:
```
$ interface vlan { vlan-id | vlan-range}
```
Add switches into configuration:
Settings icon ➡ Devices ➡ Network ➡ LAN ➡ New ➡ Vendor:device ...
Troubleshooting:
This must work from cmd line, it must provide name of the switch (place your IP and community string)
```
snmpwalk -v 2c -c public 192.168.1.1 1.3.6.1.2.1.1.5
```

SAN switch access summary

Vendor	Storage type	User role	Interface	Used ports
Brocade	SAN switch	read-only	SNMP v1,2,3	161 UDP
QLogic	SAN switch	read-only	SNMP v1,2,3	161 UDP
Cisco	MDS and Nexus	read-only	SNMP v1,2	161 UDP

Connections are always initiated from XorMon NG server side

Installation procedure is same for Cisco, Brocade and QLogic switches.
Only in case of usage Brocade/QLogic Virtual Fabric you have to do special access configuration.
Cisco VSAN support works automatically.

Brocade note: SNMP v2 is not supported in FOS v9.0.1a but is not blocked. SNMP v2 is blocked beginning with FOS v9.1.0.

Configure access to switches

Allow SNMP communication from XorMon NG host to all SAN switches on port 161, UDP
Brocade SNMPv3 setup

Test SNMP communication from cmd line

Follow that in case connection test does not work only.

# export PATH=$PATH:/opt/freeware/bin
# snmpwalk -v 2c -c public  <Switch_IP> 1.3.6.1.2.1.1.5
  SNMPv2-MIB::sysName.0 = STRING: SAN_switch_name

This must work exactly as above!

  Timeout: No Response from <Switch_IP>

Make sure there is SNMP allowed on the SAN switch with public read only role (Community 4)
- Brocade/QLogic:
```
# ssh <Switch_IP> -l admin
SAN:admin> snmpconfig --show snmpv1 
  ...
  Community 4: public (ro)
  ...
```
  Change community string to whatever you need via "snmpConfig --set snmpv1"
- Cisco:
```
# ssh <Switch_IP> -l admin
switch# show snmp community
  Community Access
  --------- ------
  private rw
  public ro
```
  When there is other community string than public then use it in test snmpwalk cmds above
  and use that community string in switch cfg.
Make sure that Access Host list contains XorMon NG host:
```
SAN:admin> snmpconfig --show accessControl
```

Make sure you have enabled all listed MIBs below:

SAN:admin> snmpconfig --show mibCapability
  FE-MIB: YES
  SW-MIB: YES
  FA-MIB: YES
  FICON-MIB: YES
  HA-MIB: YES
  FCIP-MIB: YES
  ISCSI-MIB: YES
  IF-MIB: YES
  BD-MIB: YES
  BROCADE-MAPS-MIB: YES

Enable all missing MIBs each by each::

SAN:admin> snmpconfig --enable mibCapability -mib_name FICON-MIB

set SNMP GET security level to 0

SAN:admin> snmpconfig --show seclevel
SAN:admin> snmpconfig --set seclevel
  Select SNMP GET Security Level
  (0 = No security, 1 = Authentication only, 2 = Authentication and Privacy, 3 = No Access): (0..3) [3] 0

Do you have some other switch which works fine, is there any configuration difference?

Brocade/QLogic Virtual Fabric support

you have to use SNMP v3 with proper security user
Configure security user on the switch (under admin account):
```
userconfig --add snmpuser1 -r user -l 1,128
userconfig --change snmpuser1 -r user -l 1,128 -h 128 -e no
```
Parameter -l 1,128 defines virtual fabrics visible for this user.
Parameter -h 128 defines home virtual fabrics.
Parameter -e no disables user expiration
Example above allows virtual fabrics 1 and 128 for user snmpuser1.

Test access, you should see ports configured for your specific VF (Virtual Fabric).

$ snmpwalk -v 3 -u snmpuser1 -n VF:<your_virtual_fabric_ID> <Switch_IP> 1.3.6.1.2.1.1.5
  SNMPv2-MIB::sysName.0 = STRING: SAN_switch_name

If you get one of below errors then user is not configured properly:

Error in packet.
Reason: noAccess
snmpwalk: Unknown user name

Switch configuration

Add switches into configuration from the UI:
XorMon NG UI ➡ Device ➡ Network ➡ SAN switches ➡ New ➡ Vendor:device ...
Run "Test" for configured switch, it must show "OK"
Wait about 1 hour, then refresh the web browser cache by Ctrl-F5

Troubleshoting

If you use SNMP v3 and have several virtual fabrics then monitoring might cause unwanted CPU peaks every 5 minutes on monitored switches
To decrease such CPU peaks into acceptable level configure SAN microservices like that
- Increase data fetch from default 5 minutes into 10 or 15 minutes
- Increase default timeout between SNMP queries from default 1 sec to 5 secs
Open XorMon NG UI: Settings ➡ Application ➡ select SAN Brocade / Cisco ➡ Advanced icon
SAN troubleshooting

Table of Contents