XorMon Next Generation (NG)

What's new

We are building a new generation of our infrastructure monitoring tool.
It will bring a new level of infrastructure monitoring by relying on a modern technology stack.
In particular reporting, exporting, alerting and presentation capabilities are far ahead of our current tools.

Beta version testing is already available, get the package from the download page.

Our goals

  • Better user experience with interactive graphs and more customizable interface
  • Enhanced alerting, reporting and exporting
  • Better integration of all monitored platforms with end-to-end visibility
  • Modern, scalable and future-proof backend
  • Integration with customer's IT processes using REST APIs

Features

  • Interactive graphs - filter, sort, zoom, select individual items
  • APIs for better integration with other tools - configuration, data export
  • Alerting and exporting available for all devices, technologies and all their metrics
  • Structured Custom Groups and Dashboards
  • Improved reporting and data exporting
  • Highly intuitive forms using previews, filters and regular expressions
  • Mapping between virtualization platforms and storage systems - VM to Volume
  • Microservice architecture
  • Ready for containerized deployment

Technologies

  • Frontend: JavaScript, React, Plotly.js
  • Backend: Typescript, Node.JS, Nest.JS, Python, Perl, Unix shell
  • Database: TimescaleDB
  • Microservice architecture

Current state

You can check the current state of XorMon NG on our demo website.
(It is an early preview version and some functionality is still missing.)

demo-ng.xormon.com

Do you want to help us shape the future of XorMon NG?

We would be happy to involve you in the process and hear your feedback.
Let us know if you are interested: support@xorux.com

Some examples what to expect from XorMon NG

The overall look and feel remains the same
Overall look and feel

Interactive graphs
You can select and view only the items you need, zoom and pan the timeline, search and filter.
Filter items by value

View individual item

Better organization of items
Custom groups and dashboards support all technologies and allow you to create multi-level folders to easily organize and navigate in complex environments.
Organization of items

Pinning a graph to a dashboard is more intuitive. It will also allow you to pin a single graph to multiple dashboards.
Pin graph to multiple dashboards

Improved alerting
Alerting is supported for all technologies. It is possible to create an alert definition for multiple items.
Alerting overview

Storage group totals
Storage groups now show aggregated metrics for all devices in the group.
Storage group totals

Exporter API
The Exporter API allows exporting data in JSON, CSV and XLSX formats.
Exporter form

SAN Topology
Improved SAN topology visualisation.
SAN topology

Navigation

The user interface contains 5 navigation areas:
  1. (top) global menu bar provides access to features and configured technologies
  2. (top-right) configuration menu (hamburger-like menu)
  3. (left) local menu tree
  4. (top) local tabs menu
  5. (bottom right) notification area

Dashboard

Dashboard shows selected graphs in one place.
How to add a graph to dashboard
  1. In any graph's top-right corner click the [star] icon Manage in dashboards.
  2. In the new modal window, click + to add a dashboard. Any graph can be in multiple dashboards.
  3. Select a dashboard from the drop-down list. Dashboards can be organized in folders. Use context menu (secondary click) to create, rename and delete folders.
  4. Changes are saved instantly. If you want to remove the graph from a dashboard, use the [X] Remove from this dashboard icon and confirm you choice.

Custom groups

Custom groups show aggregated data for selected items across a subsystem. The feature can be used to keep a filtered selection of items or aggregate data from one subsystem across multiple devices.
  • How to create a new custom group
    1. Go to page [cog] ➡ Custom groups.
    2. Select tab for the desired technology class: Server, Storage, SAN or LAN
    3. Click + New in the table heading.
    4. Define the custom group. Mandatory fields are highlighted red when empty. The new Custom group is saved, only when you click the [save custom group] icon. If you leave the Custom groups page, unsaved changes will be lost. Creating a new custom group can be cancelled by clicking the [delete custom group] icon.
      1. Enter the new custom group's Name.
      2. If creating a new Server [Virtualization] custom group, select Type, the technology for this Custom group.
      3. Select Subsystem of this custom group.
      4. optional: Select menu a custom Folder.
      5. optional: If you want to filter items by their label, entry the label in the Select items field. This option supports regular expressions; enable them by checking Regex. Then, you can use multiple rules, click Add rule to add each new rule. Preview a selection of items using the [Preview] icon for all rules combined or on the right of each rule for that rule. To remove a rule click the Remove rule on the right.
  • How to delete a custom group
    1. Go to page [cog] ➡ Custom groups.
    2. Select tab for the desired technology class: Server, Storage, SAN or LAN
    3. Find the custom group in the table. The table can be sorted or filtered using the icons on the right of each main column in the heading.
    4. Click the [delete custom group] icon in the custom group's row on the right.
    5. regexp syntax: JavaScript uses Perl syntax, but regexps are passed to SQL queries, hence probably POSIX Regular Expressions

Alerting

How to add a new alert
  • Alert group
    1. Go to page [cog] ➡ Alerting.
    2. Click Add new alert.
    3. In the modal window:
      1. Type Personal / Global: If you select Personal, you will get alerts from this group only to your e-mail address from your XorMon user account. Global allows you to select custom mail groups for each alert.
      2. Select whether the alert group should be for server platforms, storages, or custom groups (TODO reconsider label HW Group)
      3. Select the specific technology and Subsystem, or Custom group.
      4. Enter the new alert group's name. (TODO move this field to the top/bottom)
      5. If the alert is for servers or storages, select items either by name from the list, or using a regular expression.
  • E-mail group
    1. Go to page [cog] ➡ Alerting, select tab Email.
    2. Create a new e-mail group:
      1. Click New Email Group button.
      2. Enter the e-mail group's name.
      3. Enter an e-mail address in the Email field. Add more e-mail addresses by clicking the + Email button. Remove an address or an empty field by clicking the [bin] icon on the right.
      4. Click Save.
  • Alert
    1. Click an alert group in the table to unfold its list of alerts.
    2. Click + icon (Add alert to this group) on the right.
    3. In the new table row, specify the new alert:
      1. Select the metric.
      2. Enter the limit (threshold value) that will trigger the alert.
      3. optional: Peak
      4. optional: Repeat
      5. optional: Exclude hours
      6. Select Level: when you get notified, how the alert will be labeled.
      7. If the Alert group is Type: Global, you can select e-mail groups that will be notified.

Storage groups

...
There are several ways to install and try our new XorMon NG

  • 1) Pre-installed instance included in version 7.60+ of Xorux Virtual Appliance
  • 2) Running XorMon NG in a Docker container
  • 3) Manual installation on a clean Linux OS

1) Pre-installed version of XorMon NG

This is the recommended option.
Beta version of our new monitoring system, XorMon NG, comes pre-installed in version 7.60 of the Virtual Appliance.
If you already use an older Appliance, then use method 3) for RedHat.

  • Deploy Virtual Appliance v7.60+

    Download and follow Virtual Appliance installation

    You can leave STOR2RRD and LPAR2RRD unconfigured.
    XorMon NG is independent of legacy applications.
    We recommend deleting or commenting out all lines in the lpar2rrd user crontab unless you plan to use LPAR2RRD and STOR2RRD.

  • Upgrade Node.js to version 20.x(skip it for Virtual Appliance v7.70+)
    # switch to root
    su -
    npm install -g npm@latest
    npm install -g n
    n 20
    
    # check node version, it should display v20.X 
    node --version
  • Enable and start XorMon NG

    Start the database (as root)
    systemctl enable --now postgresql-14
    Start XorMon NG (as lpar2rrd)
    su - lpar2rrd
    cd /opt/xorux/xormon-ng/
    ./xormon.sh start
    Navigate to XorMon NG GUI
    https://<IP>/xormon-ng
    Default login: xormon / xormon

  • Upgrade to the newest available version

    Please upgrade XorMon NG before configuring any devices.

    • Download xormon-ng-<ver>.tar.gz to your workstation
    • Navigate to https://<IP>/xormon-ng
    • Go to Settings icon on the top-right
    • Select 'Application upgrade' under 'Configuration'
    • Click 'Select File' and provie path to xormon-ng-<ver>.tar.gz
    • Click 'Upload & Apply' and wait for 'Upgrade Successful' message
    • Wait two or three minutes, refresh your browser and start configuring devices

    XoprMon NG upgrade

  • Restart XorMon NG if necessary

    It may be necessary to restart XorMon NG after the first upgrade.
    Follow these steps if Xormon GUI doesn't become available within 5 minutes after the upgrade.
    • SSH to the virtual appliance as lpar2rrd
    • Restart XoprMon NG
      cd /opt/xorux/xormon-ng/
      ./xormon.sh start


2) Running XorMon NG in a Docker container

Running XorMon NG is a quick and simple way to test the new version.
However, we do not currently recommend using Docker version of XorMon NG for long-term production use.

Important notice:
Some devices require installation of third-party tools, which is currently not supported in the Docker version.
This restriction applies to: All Databases, AWS, Google Cloud and some Storage systems.
You can see which storage systems require additional software: https://stor2rrd.com/install.php#storages

VMware, on the other hand, is supported and VMware Perl SDK can be installed using XorMon NG GUI.

There are two options:
  • Docker-compose method
  • Docker run method
Docker-compose method

  • Prerequisites

    • Linux VM with the latest version of Docker Engine
    • access to Docker Hub
    • available ports: 5432 and 3000
    • Docker-compose

  • Create .env file
    cd ~/xormon-ng
    vi .env
    
    APP_PORT=3000
    APP_TIMEZONE=<timezone>
    DB_HOST=timescaledb
    DB_PORT=5432
    DB_USERNAME=postgres
    DB_PASSWORD=<db_password>
    DB_DATABASE=xormon
    APP_DEBUG=false
    JWT_SECRET=<xormon-secret>
    • <timezone> : Linux timezone, e.g. 'Europe/Prague' (see TZ in /usr/share/zoneinfo)
    • <db_password> : password used for running Timescale DB container
    • <xormon-secret> : any unique string, e.g. 'my-xormon-secret'

  • Create docker-compose YAML file
    cd ~/xormon-ng
    
    vi docker-compose.yml
    
    version: "3.9" 
    services:
      timescaledb:
        image: timescale/timescaledb:latest-pg14
        restart: unless-stopped
        environment:
          - POSTGRES_USER=${DB_USERNAME}
          - POSTGRES_PASSWORD=${DB_PASSWORD}
        volumes:
          - ./data:/var/lib/postgresql/data
        ## uncomment if you want to have Postgres (TSDB) port opened on host
        # ports:
        #  - '5432:5432'
    
      xormon:
        image: xorux/xormon-ng:latest
        restart: unless-stopped
        depends_on:
          - timescaledb
        env_file: .env
        volumes:
          - ./files:/usr/src/app/server-nest/files
        ports:
          - ${APP_PORT}:${APP_PORT}
          - 8162:8162
  • Run containers
    docker-compose up -d
  • Navigate to XorMon NG GUI
    https://<docekr_vm_IP>:3000/
    Default login: xormon / xormon

  • XorMon NG upgrade

    Do not use 'Application Upgrade' form in GUI!
    docker-compose pull
    docker-compose up -d
Docker run method

  • Prerequisites

    • Linux VM with the latest version of Docker Engine
    • access to Docker Hub
    • available ports: 5432 and 3000

  • Create a directories for permanent data storage
    mkdir -p /var/xormon-ng/data
    mkdir -p /var/xormon-ng/files
  • Timescale DB

    • Prepare a password for the databse and note it down

    • Run thedatabase container
      docker run -v /var/xormon-ng/data:/var/lib/postgresql/data \
      -d --name timescaledb -p 5432:5432 \
      -e POSTGRES_PASSWORD=<db_password> timescale/timescaledb:2.10.0-pg14
  • XorMon NG

    • Prepare a configuration file
      vi ./xormon-ng.env
      
      APP_PORT=3000
      APP_TIMEZONE=<timezone>
      DB_HOST=timescaledb
      DB_PORT=5432
      DB_USERNAME=postgres
      DB_PASSWORD=<db_password>
      DB_DATABASE=xormon
      JWT_SECRET=<xormon-secret>
      • <timezone> : Linux timezone, e.g. 'Europe/Prague' (see TZ in /usr/share/zoneinfo)
      • <db_password> : password used for running Timescale DB container
      • <xormon-secret> : any unique string, e.g. 'my-xormon-secret'

    • Run XorMon NG container
      docker run -d --name xormon -p 3000:3000 \
      -v /var/xormon-ng/files:/usr/src/app/server-nest/files \
      --link timescaledb:timescaledb --env-file ./xormon-ng.env xorux/xormon-ng:<latest>
  • Navigate to XorMon NG GUI
    https://<docekr_vm_IP>:3000/
    Default login: xormon / xormon

  • XorMon NG upgrade

    Do not use 'Application Upgrade' form in GUI!

    • Stop, remove and re-run XorMon NG container
      docker rm -f {UUID|name}
      docker run -d --name xormon -p 3000:3000 \
      -v /var/xormon-ng/files:/usr/src/app/server-nest/files \
      --link timescaledb:timescaledb --env-file ./xormon-ng.env xorux/xormon-ng:latest


3) Manual installation on a clean Linux OS

XorMon NG can be installed on an existing Xorux Virtual Appliance or on a clean Linux OS.

  • Prerequisites
    • Supported operating systems: RHEL / CentOS / Rocky Linux 8.x, Debian 10 / 11, Ubuntu 22.04
    • Root access
    • Internet aceess
    • May require reboot

  • Disable SELINUX (as root)

    Skip if you are installing on an existing Xorux Virtual Appliance
    • Edit /etc/selinux/config
    • Set SELINUX=disabled
    • reboot

  • Install required software packages (as root)

    Skip if you are installing on an existing Xorux Virtual Appliance

    RHEL / Rocky Linux
    dnf config-manager --set-enabled powertools
    yum install -y ed bc libxml2 sharutils tar graphviz epel-release wget curl
    yum install -y perl-TimeDate perl-XML-Simple perl-XML-SAX perl-XML-LibXML perl-Env perl-CGI perl-Data-Dumper
    yum install -y perl-LWP-Protocol-https perl-libwww-perl perl-Time-HiRes perl-IO-Tty perl-Want
    yum install -y perl-JSON perl-JSON-PP perl-JSON-XS net-snmp-perl
    Debian / Ubuntu
    apt install -y apache2 perl rrdtool ed bc curl sudo
    apt install -y librrdp-perl libxml-sax-perl libxml-simple-perl libtimedate-perl libenv-sanctify-perl
    apt install -y libcgi-pm-perl libdata-dumper-simple-perl libpdf-api2-perl libxml2-utils
    apt install -y libcrypt-ssleay-perl libio-socket-ssl-perl libmozilla-ldap-perl libxml-parser-perl
    
    apt install -y wget
    apt remove liblwp-protocol-https-perl
    cd /tmp/
    wget http://archive.ubuntu.com/ubuntu/pool/main/libl/liblwp-protocol-https-perl/liblwp-protocol-https-perl_6.04-2_all.deb
    apt install -y libwww-perl
    dpkg -i liblwp-protocol-https-perl_6.04-2_all.deb 
    dpkg -l | grep liblwp-protocol-https-perl
    apt-mark hold liblwp-protocol-https-perl
    dpkg -l | grep  liblwp-protocol-https-perl
    apt install -y libxml-parser-perl libxml-sax-expat-perl
  • Install NODE.JS version 20.x+ (as root)

    RHEL / Rocky Linux
    curl -sL https://rpm.nodesource.com/setup_20.x | sudo bash -
    dnf install -y nodejs
    npm cache clean -f
    npm install -g npm@latest
    Debian / Ubuntu
    curl -sL https://deb.nodesource.com/setup_20.x | sudo bash -
    sudo apt-get install -y nodejs
    npm cache clean -f
    npm install -g npm@latest
  • Install Timescale DB (as root) - TSDB documentation

    RHEL / Rocky Linux

    • Disable built-in PostgreSQL module
      dnf -qy module disable postgresql
    • Create TSDB repository
      yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-8-x86_64/pgdg-redhat-repo-latest.noarch.rpm
      Execute this as single cmd, copy&paste all lines below on the cmd line
      tee /etc/yum.repos.d/timescale_timescaledb.repo <<EOL
      [timescale_timescaledb]
      name=timescale_timescaledb
      baseurl=https://packagecloud.io/timescale/timescaledb/el/$(rpm -E %{rhel})/\$basearch
      repo_gpgcheck=1
      gpgcheck=0
      enabled=1
      gpgkey=https://packagecloud.io/timescale/timescaledb/gpgkey
      sslverify=1
      sslcacert=/etc/pki/tls/certs/ca-bundle.crt
      metadata_expire=300
      EOL
    • Install TSDB and PostgeSQL packages
      yum install -y timescaledb-2-postgresql-14
      yum install -y postgresql14-contrib
    • Initialize DB
      /usr/pgsql-14/bin/postgresql-14-setup initdb
    • Tune DB
      timescaledb-tune -yes --pg-config=/usr/pgsql-14/bin/pg_config
    • Enable automatic start
      systemctl enable postgresql-14
      systemctl start postgresql-14
    • Set password for user postgres
      sudo -u postgres psql
      
      postgres=# \password postgres
      postgres=# \q
    • Make note of the password. It will be needed later during XorMon NG installation.

    Debian / Ubuntu

    • Install self-hosted TimescaleDB:
      apt install -y gnupg postgresql-common apt-transport-https lsb-release wget
      /usr/share/postgresql-common/pgdg/apt.postgresql.org.sh
      
      ## DEBIAN
      echo "deb https://packagecloud.io/timescale/timescaledb/debian/ $(lsb_release -c -s) main" \
        | sudo tee /etc/apt/sources.list.d/timescaledb.list
      
      ## UBUNTU
      echo "deb https://packagecloud.io/timescale/timescaledb/ubuntu/ $(lsb_release -c -s) main" \
        | sudo tee /etc/apt/sources.list.d/timescaledb.list
      
      wget --quiet -O - https://packagecloud.io/timescale/timescaledb/gpgkey | sudo apt-key add -
      apt update
      apt install -y timescaledb-2-postgresql-14
    • Tune DB
      timescaledb-tune -yes
      systemctl restart postgresql
    • Install psql:
      apt-get update
      apt-get install -y postgresql-client
    • Set password for user postgres
      sudo -u postgres psql
      
      postgres=# \password postgres
      postgres=# \q
    • Make note of the password. It will be needed later during XorMon NG installation.

  • Configure firewall (as root)

    Open XorMon NG port in the firewall (3000 TCP by default).

    RHEL / Rocky Linux
    firewall-cmd --zone=public --add-port=3000/tcp --permanent
    firewall-cmd --reload
    Debian / Ubuntu
    dpkg -l | grep ufw 
    ufw status verbose
    uwf enable
    
    ufw allow ssh
    ufw allow http
    ufw allow https
    ufw allow 3000/tcp
    ufw reload
  • Create xormon user (as root)

    RHEL / Rocky Linux
    useradd -c "XorMon NG user" -m xormon
    passwd xormon
    Debian
    adduser xormon
    passwd xormon
  • Install XorMon NG (as xormon user)
    su - xormon
    mkdir xormon-ng
    cd xormon-ng
    Download latest XorMon NG package to /home/xormon/xormon-ng
    Provide database password created earlier when you are prompted
    tar zxf xormon-ng-<ver>.tar.gz
    rm xormon-ng-<ver>.tar.gz
    ./xormon.sh start
    
    DB_HOST[127.0.0.1]=
    DB_PORT[5432]=
    DB_USERNAME[postgres]=
    DB_DATABASE[xormon]=
    DB_PASSWORD[]= <password>
    Setup Xormon NG startup script (as root)
    su -
    env PATH=$PATH:/usr/bin /home/xormon/xormon-ng/server-nest/node_modules/pm2/bin/pm2 startup systemd -u xormon --hp /home/xormon
  • Navigate to XorMon NG GUI
    https://<IP>:3000/
    Default login: xormon / xormon

Supported technologies

The latest beta release of XorMon NG support monitoring of following technologies and features

Server (no OS agent support)
  • IBM Power Systems
  • VMware
  • Proxmox
  • Nutanix

Database
  • Oracle DB
  • PostgreSQL
  • MS SQL
  • IBM Db2

Container
  • Kubernetes
  • Docker

Network
  • SAN Brocade
  • SAN Cisco

Storage
  • Dell EMC² PowerFlex (ScaleIO)
  • Dell EMC² PowerScale (Isilon)
  • Dell EMC² DataDomain 6.x+
  • Dell EMC² Compellent, SC series
  • Dell EMC² MD3000
  • Dell EMC² PowerMAX , VMAX
  • HPE 3PAR, Primera
  • HPE Storeonce V4
  • HPE MSA
  • HPE Nimble
  • IBM FlashSystem, Storwize, SVC, Spectrum Virtualize
  • IBM DS3xxx, 4xxx, 5xxx
  • IBM Cloud Object Storage
  • IBM XIV
  • IBM A9000
  • NetApp E/EF-series
  • Lenovo DE Series
  • Lenovo V Series
  • Oracle ZFS
  • Inspur AS5600/5500/2600/2200
  • QNAP NAS
  • Rubrik
  • DotHill

Not supported features yet

  • IBM Power Systems: CPU Workload Estimator
  • IBM Power Systems: Resource configuration Advisor
  • IBM Power Systems: NMON support
  • VMware: Resource configuration Advisor
  • OS agent: Linux / Solaris / IBM i / Windows / Hyper-V