Infrastructure Monitoring : XorMon NG Installation

Clean Linux OS

RedHat Linux 8/9 (x86)
Debian: 11/12
RedHat Linux 9 (ppc64le)
XorMon NG application install

RedHat Linux / Rocky Linux (x86)

As soon as you prepare the hosting platform, follow tab XorMon NG app install to get installed XorMon NG app.
HW sizing

Disable SELinux (as root)
- Edit /etc/selinux/config
- Set SELINUX=disabled
- reboot
In case you do not want to disable SELinux then follow this manual to configure SELinux in enforcing mode

Create xormon user (as root)

adduser xormon
passwd xormon

Increase user system limits
Append following lines to /etc/security/limits.conf

xormon          hard    stack           -1
xormon          soft    stack           -1
xormon          hard    data            -1
xormon          soft    data            -1
xormon          hard    nofile          32768
xormon          soft    nofile          32768
xormon          hard    nproc           64000
xormon          soft    nproc           64000

Enable PowerTools / CodeReady Builder repository (as root)

# RHEL 8:
subscription-manager repos --enable codeready-builder-for-rhel-8-x86_64-rpms

# RHEL 9:
subscription-manager repos --enable codeready-builder-for-rhel-9-x86_64-rpms

# Rocky Linux 8.x
dnf config-manager --set-enabled powertools

# Rocky Linux 9.x
dnf config-manager --enable crb

Install additional packages (as root)

dnf install -y perl ed libxml2 epel-release wget curl gnupg net-tools sblim-wbemcli libaio wget curl strace 
dnf install -y perl-TimeDate perl-XML-Simple perl-XML-SAX perl-XML-LibXML perl-Env perl-CGI perl-Data-Dumper
dnf install -y perl-LWP-Protocol-https perl-libwww-perl perl-Time-HiRes perl-IO-Tty openldap-clients
dnf install -y perl-JSON perl-JSON-PP perl-JSON-XS net-snmp-perl net-snmp-utils perl-DBI perl-DBD-ODBC unixODBC unixODBC-devel freetds
dnf install -y perl-DBD-Pg # for oVirt/RHV/OLVM support only

Python: required only for prediction graphs
You can run XorMon NG without it if you have trouble downloading necessary Python modules

dnf install -y python3-virtualenv python3-pip libpq-devel python3-devel gcc-c++ gcc

Install Java 11
(required by Hitachi Export Tool for Hitachi VSP monitoring)
```
dnf install -y java-11-openjdk-headless
```
Install Node.js (as root)
XorMon NG should work with Node.js 16+.
We recommend Node.js 20.x or newer.
- Repository installation
```
dnf module enable nodejs:20
dnf install nodejs
npm update -g npm
node -v
```
- Manual download install:
```
curl -fsSL https://rpm.nodesource.com/setup_20.x | bash -
dnf install -y nodejs
npm update -g npm
node -v
```
  You can download and install Node.js from RPM if you cannot access the repository directly.
  EL8: rpm.nodesource.com/pub_20.x/el/8/x86_64/nodejs-20.5.0-1nodesource.x86_64.rpm
  EL9: rpm.nodesource.com/pub_20.x/el/9/x86_64/nodejs-20.5.0-1nodesource.x86_64.rpm
```
dnf install ./nodejs-20.5.0-1nodesource.x86_64.rpm -y
npm update -g npm
node -v
```
Install Timescale DB (as root)
XorMon NG requires TimescaleDB 2.13 and newer (community edition).
- Follow these installation instructions: TimescaleDB/PostgreSQL installation docu
- PostgreSQL vrs TimescaleDB support matrix
- Optional: Change database location if necessary
  Default database location is '/var/lib/pgsql/16/data/'
  Note: database requires at least 100GB (500GB in large environment)
  - Create new folder for the database
```
mkdir /opt/pgsql
chown postgres:postgres /opt/pgsql/
chmod 750 /opt/pgsql/
```
  - Edit postgres-16 systemd service
```
vi /usr/lib/systemd/system/postgresql-16.service
Environment=PGDATA=/opt/pgsql
```
  - Reload services
```
systemctl daemon-reload
```
  - Edit 'postgres' user bash profile
```
vi /var/lib/pgsql/.bash_profile
PGDATA=/opt/pgsql
```
- Initialize DB
```
/usr/pgsql-16/bin/postgresql-16-setup initdb
```
- Tune DB
  Make sure you are using the correct configuration file if your database is not in the default location
  
  Default location:
```
timescaledb-tune -yes --pg-config=/usr/pgsql-16/bin/pg_config
sed -i 's/#synchronous_commit = on/synchronous_commit = off/' /var/lib/pgsql/16/data/postgresql.conf
```
  Alternative database location:
```
timescaledb-tune -yes --pg-config=/usr/pgsql-16/bin/pg_config -conf-path=/opt/pgsql/postgresql.conf
sed -i 's/#synchronous_commit = on/synchronous_commit = off/' /opt/pgsql/postgresql.conf
```
- Enable automatic start
```
systemctl enable --now postgresql-16
```
- Set password for user postgres
```
sudo -u postgres psql

postgres=# \password postgres
postgres=# \q
```
  Make note of the password. It will be needed later during XorMon NG installation.

Configure firewall (optional, as root)
Open XorMon NG port in the firewall 8443 TCP (XorMon UI) and 8162 TCP (OS agent communication).

firewall-cmd --zone=public --add-port=8443/tcp --permanent
firewall-cmd --zone=public --add-port=8162/tcp --permanent
firewall-cmd --reload

Debian: 11/12, Ubuntu: 22.04

As soon as you prepare the hosting platform, follow tab XorMon NG app install to get installed XorMon NG app.
HW sizing

Create xormon user (as root)

adduser xormon

Increase user system limits, append this into /etc/security/limits.conf file

xormon          hard    stack           -1
xormon          soft    stack           -1
xormon          hard    data            -1
xormon          soft    data            -1
xormon          hard    nofile          32768
xormon          soft    nofile          32768
xormon          hard    nproc           64000
xormon          soft    nproc           64000

Add 'non-free' repository to apt sources

vi /etc/apt/sources.list

Your '/etc/apt/sources.list' should look like:

# Debian 11
deb http://deb.debian.org/debian/ bullseye main contrib non-free
deb-src http://deb.debian.org/debian/ bullseye main contrib non-free

deb http://security.debian.org/debian-security bullseye-security main contrib non-free
deb-src http://security.debian.org/debian-security bullseye-security main contrib non-free

deb http://deb.debian.org/debian/ bullseye-updates main contrib non-free
deb-src http://deb.debian.org/debian/ bullseye-updates main contrib non-free


# Debian 12
deb http://deb.debian.org/debian/ bookworm main non-free-firmware non-free
deb-src http://deb.debian.org/debian/ bookworm main non-free-firmware non-free

deb http://security.debian.org/debian-security bookworm-security main non-free-firmware non-free
deb-src http://security.debian.org/debian-security bookworm-security main non-free-firmware non-free

deb http://deb.debian.org/debian/ bookworm-updates main non-free-firmware non-free
deb-src http://deb.debian.org/debian/ bookworm-updates main non-free-firmware non-free

Install additional packages (as root)

apt update
apt install -y perl ed wget ca-certificates curl gnupg sudo snmp snmp-mibs-downloader nmap alien libaio1 ksh tcpdump sblim-wbemcli net-tools strace 
apt install -y librrdp-perl libxml-sax-perl libxml-simple-perl libtimedate-perl libenv-sanctify-perl libcgi-pm-perl 
apt install -y libsnmp-perl libdata-dumper-simple-perl libxml2-utils liblwp-protocol-https-perl libcrypt-ssleay-perl ldap-utils
apt install -y libio-socket-ssl-perl libmozilla-ldap-perl libxml-parser-perl libjson-xs-perl libio-pty-perl libxml-libxml-perl
apt install -y postgresql-common apt-transport-https lsb-release libdbd-odbc-perl default-jre libdbd-pg-perl
apt install -y libdbd-pg-perl # for oVirt/RHV/OLVM support only

Python: required only for prediction graphs
You can run XorMon NG without it if you have trouble downloading necessary Python modules

apt install -y python3-venv

Java 11 Installation
(required by Hitachi Export Tool for Hitachi VSP monitoring)

Install Node.js (as root)
XorMon NG should work with Node.js 16+.
We recommend Node.js 20.x or newer.

curl -fsSL https://deb.nodesource.com/setup_20.x | bash -
apt-get install -y nodejs
npm update -g npm
node -v

Install Timescale DB (as root)
XorMon NG requires TimescaleDB 2.13 and newer (community edition).
- Follow these installation instructions: TimescaleDB / PostgreSQL installation docu
- PostgreSQL vrs TimescaleDB support matrix
- Optional: Change database location if necessary
  Default database location is '/var/lib/postgresql/16/main'
  Note: database requires at least 100GB (500GB in large environment)
  - Stop and drop the default postgres cluster
```
systemct stop postgresql.service
pg_dropcluster --stop 16 main
```
  - Create postgres cluster with new data location
```
pg_createcluster -d /opt/pgsql 16 main
systemctl daemon-reload
```
- Start postgres service
```
systemct enable --now postgresql.service
```
- Tune DB
```
timescaledb-tune -yes
sed -i 's/#synchronous_commit = on/synchronous_commit = off/' /etc/postgresql/16/main/postgresql.conf
systemctl restart postgresql.service
```
- Set password for user postgres
```
sudo -u postgres psql

postgres=# \password postgres
postgres=# \q
```
  Make note of the password. It will be needed later during XorMon NG installation.

Configure firewall (optional, as root)
Open XorMon NG port in the firewall 8443 TCP (XorMon UI) and 8162 TCP (OS agent communication).
Install UFW firewall if needed
```
apt install -y ufw
ufw enable
```
Configure firewall
```
ufw allow ssh
ufw allow http
ufw allow https
ufw allow 8443/tcp
ufw allow 8162/tcp
ufw reload
```

RedHat Linux / Rocky Linux (ppc: PowerPC Little Endian)

As soon as you prepare the hosting platform, follow tab XorMon NG app install to get installed XorMon NG app.
HW sizing

Disable SELinux (as root)
- Edit /etc/selinux/config
- Set SELINUX=disabled
- reboot
In case you do not want to disable SELinux then follow this manual to configure SELinux in enforcing mode

Create xormon user (as root)

adduser xormon
passwd xormon

Increase user system limits
Append following lines to /etc/security/limits.conf

xormon          hard    stack           -1
xormon          soft    stack           -1
xormon          hard    data            -1
xormon          soft    data            -1
xormon          hard    nofile          32768
xormon          soft    nofile          32768
xormon          hard    nproc           64000
xormon          soft    nproc           64000

Enable PowerTools / CodeReady Builder repository (as root)

# RHEL 9:
subscription-manager repos --enable codeready-builder-for-rhel-9-x86_64-rpms

# Rocky / Alma / Centos Linux 9.x
crb enable

Install additional packages (as root)

dnf install -y perl ed libxml2 epel-release wget curl gnupg net-tools sblim-wbemcli libaio wget curl strace 
dnf install -y perl-TimeDate perl-XML-Simple perl-XML-SAX perl-XML-LibXML perl-Env perl-CGI perl-Data-Dumper
dnf install -y perl-LWP-Protocol-https perl-libwww-perl perl-Time-HiRes perl-IO-Tty openldap-clients
dnf install -y perl-JSON perl-JSON-PP perl-JSON-XS net-snmp-perl net-snmp-utils perl-DBI perl-DBD-ODBC unixODBC unixODBC-devel freetds
dnf install -y perl-DBD-Pg # for oVirt/RHV/OLVM support only

Python: required only for prediction graphs
You can run XorMon NG without it if you have trouble downloading necessary Python modules

dnf install -y python3-virtualenv python3-pip libpq-devel python3-devel

Install Java 11 (necessary for Hitachi VSP monitoring, required by Hitachi Export Tool)
```
dnf install -y java-11-openjdk-headless
```

Install Node.js (as root)
XorMon NG should work with Node.js 16+.
We recommend Node.js 20.x or newer.

Repository installation

dnf module enable nodejs:20
dnf install nodejs
npm update -g npm
node -v

Manual download install: There are no RPMs available for ppc64le, get Power LE binaries from here: nodejs.org/en/download/prebuilt-binaries, i.e. LTS 20.12.0:

cd /tmp
wget https://nodejs.org/dist/v20.12.0/node-v20.12.0-linux-ppc64le.tar.xz
# extract package to /usr/local
cd /usr/local
tar xf /tmp/node-v20.12.0-linux-ppc64le.tar.xz --strip-components=1
# check node version
node --version
# remove source file
rm /tmp/node-v20.12.0-linux-ppc64le.tar.xz

Allow node binary to use privilleged (< 1024) ports, so we can run XorMon on a standard HTTPS port (443)

setcap 'cap_net_bind_service=+ep' /usr/bin/node

Install Timescale DB (as root)
XorMon NG requires TimescaleDB 2.13 and newer (community edition).

Follow these installation instructions: TimescaleDB/PostgreSQL installation docu
PostgreSQL vrs TimescaleDB support matrix
Optionaly you might change database location from default /var/lib/pgsql/data
Note that database must be on a filesystem having 100GB free at least (in big environments rather 500GB)
```
postgresql-setup --help
```
Initialize DB
```
postgresql-setup initdb
```

Change PostgreSQL Client Authentication Configuration

# replace all ident methods with scram-sha-256 to anable login via TCP
sed -i 's/ident$/scram-sha-256/g' /var/lib/pgsql/data/pg_hba.conf

Timescale DB
As there are no RPMs for PowerPC architecture, we must compile from sources

# essentials for building
dnf install cmake gcc postgresql-server-devel

If above fails with

Problem: problem with installed package libpq-devel-13.11-1.el9.ppc64le
  - package postgresql-private-devel-15.6-1.module+el9.3.0+16128+52b853ae.ppc64le from appstream conflicts with libpq-devel provided by libpq-devel-13.11-1.el9.ppc64le from @System
  ...

Then remove libpq-devel and try installation again

dnf remove libpq-devel

Get the latest release (tar.gz source codes) from github.com/timescale/timescaledb/releases and extract to /tmp/timescaledb-2.xx.x

cd /tmp
wget https://github.com/timescale/timescaledb/archive/refs/tags/2.14.2.tar.gz -O - | tar xz
# bootstrap the build system
cd timescaledb-2.14.2/
./bootstrap
# to build the extension
cd build && make
# to install
make install

We need timescaledb-tune to configure PGSQL, install with Go lang if you are online

dnf install golang
go install github.com/timescale/timescaledb-tune/cmd/timescaledb-tune@latest
# tune PGSQL for TSDB
/root/go/bin/timescaledb-tune -yes -pg-config /usr/bin/pg_config -conf-path /var/lib/pgsql/data/
sed -i 's/#synchronous_commit = on/synchronous_commit = off/' /etc/postgresql-setup/upgrade/postgresql.conf

or get ready to use RPM to install timescaledb-tune:

dnf install timescaledb-tune.el9.ppc64le.rpm
# tune PGSQL for TSDB
timescaledb-tune -yes -pg-config /usr/bin/pg_config -conf-path /var/lib/pgsql/data/
sed -i 's/#synchronous_commit = on/synchronous_commit = off/' /etc/postgresql/16/main/postgresql.conf

Enable and run PostgreSQL service
```
systemctl enable --now postgresql
```
Set password for user postgres
```
sudo -u postgres psql

postgres=# \password postgres
postgres=# \q
```
Make note of the password. It will be needed later during XorMon NG installation.

Configure firewall (optional, as root)
Open XorMon NG ports in the firewall: HTTPS (443) (XorMon UI) and 8162 TCP (OS agent communication)

# OS agents connect to the port 8162
firewall-cmd --zone=public --add-port=8162/tcp --permanent
# UI runs on port 443
firewall-cmd --zone=public --add-service=https --permanent
# apply rules
firewall-cmd --reload

XorMon NG Installation

Once you have prepared Linux then install XorMon NG itself.

Log in as 'xormon' and create installation directory
```
su - xormon
mkdir xormon-ng
cd xormon-ng
```
Download latest XorMon NG package to /home/xormon/xormon-ng
```
tar zxf xormon-ng-<ver>.tar.gz
./xormon.sh start
```

Provide database password created during OS configuration when you are prompted

APP_PORT[8443]=

DB_HOST[127.0.0.1]=
DB_PORT[5432]=
DB_USERNAME[postgres]=
DB_DATABASE[xormon]=
DB_PASSWORD[]= <password>

Optional: change default port

XorMon NG uses port 8443 by default. It can be configured to use any other port, even 443 (https).
- Make sure the selected port is free
  
  Following command should not provide any output, if it does, then the port is being used by some other application.
```
netstat -an| grep LISTEN |grep ":443 "
```
- Allow the application to use low port number (as root)
  
  This is necessary if you want to use port 443 or any other port lower than 1024.
  Note: you have to re-run this command after each Node.js upgrade
```
su -
setcap 'cap_net_bind_service=+ep' `which node`
```
- Configure application port (as xormon)
```
vi ~/xormon-ng/server-nest/.env
APP_PORT=443
```
- Restart XorMon NG (as xormon)
```
cd ~/xormon-ng
./xormon.sh stop
cd server-nest/
npx pm2 kill
cd ..
./xormon.sh start 
```

Configure Python3 virtual environment (as xormon)

prediction graphs

cd ~/xormon-ng/server-nest/
python3 -m venv venv
venv/bin/python3 -m ensurepip --upgrade
venv/bin/python3 -m pip install --upgrade pip
venv/bin/python3 -m pip install -r microservices/analyzer/requirements/requirements.txt

cd ~/xormon-ng
./xormon.sh stop
./xormon.sh start

Setup XorMon NG startup script (as root)

systemd based (RedHat 8+, Debian 10+ )

env PATH=$PATH:/usr/bin /home/xormon/xormon-ng/server-nest/node_modules/pm2/bin/pm2 startup systemd -u xormon --hp /home/xormon

Navigate to XorMon NG GUI

https://<IP>:8443/