Analyzer Service Troubleshooting

Python version

Prediction graphs and Anomaly detection require Python 3.10 or newer.
You can run XorMon without it if you have trouble downloading necessary Python modules.

Check the the default Python 3 version installed in your system:
This version needs to be 3.10 or newer
```
python3 -V
```

Check other installed Python versions

ls -ls /usr/bin/python*

lrwxrwxrwx 1 root root     9 Aug  4 13:28 /usr/bin/python3 -> python3.6
-rwxr-xr-x 1 root root  6288 Nov 24 15:30 /usr/bin/python3.12
-rwxr-xr-x 1 root root  3378 Nov 24 15:30 /usr/bin/python3.12-config
-rwxr-xr-x 2 root root 10560 Aug  4 13:28 /usr/bin/python3.6
lrwxrwxrwx 1 root root    17 Aug  4 13:28 /usr/bin/python3.6-config -> python3.6m-config
-rwxr-xr-x 2 root root 10560 Aug  4 13:28 /usr/bin/python3.6m
-rwxr-xr-x 1 root root  3440 Aug  4 13:28 /usr/bin/python3.6m-config
lrwxrwxrwx 1 root root    16 Aug  4 13:28 /usr/bin/python3-config -> python3.6-config

python3.12 is available in the above example.

Install Python 3.10 or newer if necessary
Required python packages: venv/virtualenv, dev/develop, pip
(Detailed procedure is beyond the scope of this guide.)

Check analyzer status:

su - xormon
cd ~/xormon-ng/server-nest/
npx pm2 list --sort name:asc

Look for 'analyzer' in the process list.

You may try to restart the process

su - xormon
cd ~/xormon-ng/server-nest/
npx pm2 start analyzer

Check again.

Issue: Alerting -> Anomaly: Analyzer is not capable of anomaly detection

Solution:
Login as 'xormon' and install required Python modules.

su - xormon
cd ~/xormon-ng/server-nest
venv/bin/python3 -m pip install -r microservices/analyzer/requirements/requirements.txt

Restart XorMon

su - xormon
cd ~/xormon-ng
./xormon.sh restart

Issue: Alerting -> Anomaly: Analyzer does not respond properly! Processing Error.

Solution:
You have probably used an usupported Python version for creating the virtual environment.
Check the first note dealing with Python versions
Look for available versions and install a new one if necessary.
Login as 'xormon', remove the virtual environment, clear pip cache and reinstall requirements.
Note: using 'python3.12' in this example. You may have a different version installed in your OS.

su - xormon
cd ~/xormon-ng/server-nest
rm -rf venv/
rm -rf ~/.cache/pip
python3.12 -m venv venv
venv/bin/python3 -m ensurepip --upgrade
venv/bin/python3 -m pip install --upgrade pip
venv/bin/python3 -m pip install -r microservices/analyzer/requirements/requirements.txt

Restart XorMon

su - xormon
cd ~/xormon-ng/
./xormon.sh restart

Issue: Prediction graphs stopped working after upgrade to v2.0.0

Solution:
Download microservice patch: xormon.com/download-static/xormon_ms_patch-analyzer.tar
And apply it following these steps:

su - xormon 
cd ~/xormon-ng
./xormon.sh mspatch xormon_ms_patch-analyzer.tar

If the problem persists, then please contact our support and send us logs via upload.xormon.com
UI -> Settings -> Logs -> Support logs -> follow the form

Issue: RHEL / Rocky Linux: error during requirements installation

venv/bin/python3 -m pip install -r microservices/analyzer/requirements/requirements.txt

Command fails to build with a JPEG error

Solution:
Following packages need to be installed: libjpeg-turbo-devel, libjpeg-turbo-utils

dnf install libjpeg-turbo-devel libjpeg-turbo-utils

Issue: Failed to establish a new connection: [Errno 101] Network is unreachable

venv/bin/python3 -m pip install -r microservices/analyzer/requirements/requirements.txt

Fails with an error:

Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')': /simple/pip/

Solution:
Try to avoid it by using proxy for Python packages installation:

venv/bin/python3 -m pip install --proxy http://<username>:<password>@<hostname>:<port> --upgrade pip
venv/bin/python3 -m pip install --proxy http://<username>:<password>@<hostname>:<port> -r microservices/analyzer/requirements/requirements.txt

Restart XorMon:

su - xormon 
cd ~/xormon-ng
./xormon.sh restart

Xorux Virtual Appliance

This is about our Xorux Virtual Appliance used by the first generation of our tools: XorMon Original, LPAR2RRD and STOR2RRD
It is not about XorMon Virtual Appliance you can download from this site which already contains XorMon.

Rocky Linux (7.60+)
Use this procedure to upgrade and configure XorMon distributed with Xorux Virtual Appliance 7.60+ which is based on Rocky Linux 8.x, you can check this by

grep "Virtual Appliance version" /var/www/html/index.html| sed -e 's/^.*Virtual Appliance version/Virtual Appliance version/' -e 's/is brought to you by.*//'
Virtual Appliance version 7.60

First as root:

su -
dnf install -y python3-virtualenv python3-devel libpq-devel

Then as lpar2rrd user:

su - lpar2rrd
export PATH="$PATH:/usr/pgsql-12/bin" 
cd /opt/xorux/xormon-ng/server-nest
python3 -m venv venv
venv/bin/python3 -m ensurepip --upgrade
venv/bin/python3 -m pip install --upgrade pip
venv/bin/python3 -m pip install -r microservices/analyzer/requirements/requirements.txt

If you get this error:

Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')': /simple/pip/

Try to avoid it by using internal proxy for Python packages installation

venv/bin/python3 -m pip install --proxy http://<username>:<password>@<hostname>:<port> --upgrade pip
venv/bin/python3 -m pip install --proxy http://<username>:<password>@<hostname>:<port> -r microservices/analyzer/requirements/requirements.txt

Restart XorMon:

su - lpar2rrd
cd /opt/xorux/xormon-ng
./xormon.sh stop
./xormon.sh start

CentOS (older than 7.60)
As root:

su -
yum -y install python3-virtualenv python3-pip python3-devel
yum -y install postgresql-devel gcc

As lpar2rrd user install:

su - lpar2rrd
cd ~/xormon-ng/server-nest/
python3 -m venv venv
venv/bin/python3 -m ensurepip --upgrade
venv/bin/python3 -m pip install --upgrade pip
venv/bin/python3 -m pip install -r microservices/analyzer/requirements/requirements.txt

If you get this error:

Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')': /simple/pip/

Try to avoid it by using internal proxy for Python packages installation

venv/bin/python3 -m pip install --proxy http://<username>:<password>@<hostname>:<port> --upgrade pip
venv/bin/python3 -m pip install --proxy http://<username>:<password>@<hostname>:<port> -r microservices/analyzer/requirements/requirements.txt

Restart XorMon:

su - lpar2rrd
cd ~/xormon-ng
./xormon.sh stop
./xormon.sh start

Other issues

In case you did not find solution, please contact our support and send us logs via upload.xormon.com
UI -> settings -> Logs -> Support logs -> follow the form