Analyzer Service Troubleshooting

  • XorMon version v2.1.20 introduces a script to check analyzer service prerequisites and see installation hints: 
    su - xormon
python3 ~/xormon-ng/server-nest/microservices/analyzer/check_requirements.py
    It checks all necessary python environment and suggests installation changes and additions, just follow its output.

  • Python version

    Prediction graphs and Anomaly detection require Python 3.10 or newer.
    You can run XorMon without it if you have trouble downloading necessary Python modules.

    • Check the the default Python 3 version installed in your system:
      This version needs to be 3.10 or newer
      python3 -V
    • Check other installed Python versions
      ls -ls /usr/bin/python*
      lrwxrwxrwx 1 root root     9 Aug  4 13:28 /usr/bin/python3 -> python3.6
-rwxr-xr-x 1 root root  6288 Nov 24 15:30 /usr/bin/python3.12
-rwxr-xr-x 1 root root  3378 Nov 24 15:30 /usr/bin/python3.12-config
-rwxr-xr-x 2 root root 10560 Aug  4 13:28 /usr/bin/python3.6
lrwxrwxrwx 1 root root    17 Aug  4 13:28 /usr/bin/python3.6-config -> python3.6m-config
-rwxr-xr-x 2 root root 10560 Aug  4 13:28 /usr/bin/python3.6m
-rwxr-xr-x 1 root root  3440 Aug  4 13:28 /usr/bin/python3.6m-config
lrwxrwxrwx 1 root root    16 Aug  4 13:28 /usr/bin/python3-config -> python3.6-config
      python3.12 is available in the above example.

    • Install Python 3.10 or newer if necessary
      Required python packages: venv/virtualenv, dev/develop, pip
      (Detailed procedure is beyond the scope of this guide.)

    Check analyzer status:
    su - xormon
cd ~/xormon-ng/server-nest/
npx pm2 list --sort name:asc
    Look for 'analyzer' in the process list.

    You may try to restart the process
    su - xormon
cd ~/xormon-ng/server-nest/
npx pm2 start analyzer
    Check again.

  • Issue: Alerting -> Anomaly: Analyzer is not capable of anomaly detection

    Solution:
    Login as 'xormon' and install required Python modules.
    su - xormon
cd ~/xormon-ng/server-nest
venv/bin/python3 -m pip install -r microservices/analyzer/requirements/requirements.txt
    Restart XorMon
    su - xormon
cd ~/xormon-ng
./xormon.sh restart

  • Issue: Alerting -> Anomaly: Analyzer does not respond properly! Processing Error.

    Solution:
    You have probably used an usupported Python version for creating the virtual environment.
    Check the first note dealing with Python versions
    Look for available versions and install a new one if necessary.
    Login as 'xormon', remove the virtual environment, clear pip cache and reinstall requirements.
    Note: using 'python3.12' in this example. You may have a different version installed in your OS.
    su - xormon
cd ~/xormon-ng/server-nest
rm -rf venv/
rm -rf ~/.cache/pip
python3.12 -m venv venv
venv/bin/python3 -m ensurepip --upgrade
venv/bin/python3 -m pip install --upgrade pip
venv/bin/python3 -m pip install -r microservices/analyzer/requirements/requirements.txt
    Restart XorMon
    su - xormon
cd ~/xormon-ng/
./xormon.sh restart

  • Issue: Prediction graphs stopped working after upgrade to v2.0.0

    Solution:
    Upgrade to XorMon 2.1.0+

  • Issue: RHEL / Rocky Linux: error during requirements installation

    venv/bin/python3 -m pip install -r microservices/analyzer/requirements/requirements.txt
    Command fails to build with a JPEG error

    Solution:
    Following packages need to be installed: libjpeg-turbo-devel, libjpeg-turbo-utils 
    dnf install libjpeg-turbo-devel libjpeg-turbo-utils

  • Issue: Failed to establish a new connection: [Errno 101] Network is unreachable

    venv/bin/python3 -m pip install -r microservices/analyzer/requirements/requirements.txt
    Fails with an error:
    Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')': /simple/pip/

    Solution:
    Try to avoid it by using proxy for Python packages installation:
    venv/bin/python3 -m pip install --proxy http://<username>:<password>@<hostname>:<port> --upgrade pip
venv/bin/python3 -m pip install --proxy http://<username>:<password>@<hostname>:<port> -r microservices/analyzer/requirements/requirements.txt
    If you cannot use proxy, you can run following script (since XorMon 2.1.20) to generate instructions for manual download and installation of required Python packages: 
    su - xormon
python3 ~/xormon-ng/server-nest/microservices/analyzer/check_requirements.py --package-download-steps
    Restart XorMon: 
    su - xormon 
cd ~/xormon-ng
./xormon.sh restart

  • Xorux Virtual Appliance

    This is about our Xorux Virtual Appliance used by the first generation of our tools: XorMon Original, LPAR2RRD and STOR2RRD
    It is not about XorMon Virtual Appliance you can download from this site which already contains XorMon.
    • Rocky Linux (7.60+)
      Use this procedure to upgrade and configure XorMon distributed with Xorux Virtual Appliance 7.60+ which is based on Rocky Linux 8.x, you can check this by
      grep "Virtual Appliance version" /var/www/html/index.html| sed -e 's/^.*Virtual Appliance version/Virtual Appliance version/' -e 's/is brought to you by.*//'
Virtual Appliance version 7.60
      First as root: 
      su -
dnf install -y python3-virtualenv python3-devel libpq-devel
      Then as xormon user: 
      su - xormon
export PATH="$PATH:/usr/pgsql-12/bin" 
cd xormon-ng/server-nest
python3 -m venv venv
venv/bin/python3 -m ensurepip --upgrade
venv/bin/python3 -m pip install --upgrade pip
venv/bin/python3 -m pip install -r microservices/analyzer/requirements/requirements.txt
      If you get this error: 
      Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')': /simple/pip/
      Try to avoid it by using internal proxy for Python packages installation
      venv/bin/python3 -m pip install --proxy http://<username>:<password>@<hostname>:<port> --upgrade pip
venv/bin/python3 -m pip install --proxy http://<username>:<password>@<hostname>:<port> -r microservices/analyzer/requirements/requirements.txt
      If you cannot use proxy, you can run following script (since XorMon 2.1.20) to generate instructions for manual download and installation of required Python packages: 
      su - xormon
python3 ~/xormon-ng/server-nest/microservices/analyzer/check_requirements.py --package-download-steps
      Restart XorMon: 
      su - xormon   
cd xormon-ng
./xormon.sh stop
./xormon.sh start

    • CentOS (older than 7.60)
      As root: 
      su -
yum -y install python3-virtualenv python3-pip python3-devel
yum -y install postgresql-devel gcc
      As xormon user install: 
      su - xormon   
cd ~/xormon-ng/server-nest/
python3 -m venv venv
venv/bin/python3 -m ensurepip --upgrade
venv/bin/python3 -m pip install --upgrade pip
venv/bin/python3 -m pip install -r microservices/analyzer/requirements/requirements.txt
      If you get this error: 
      Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable')': /simple/pip/
      Try to avoid it by using internal proxy for Python packages installation 
      venv/bin/python3 -m pip install --proxy http://<username>:<password>@<hostname>:<port> --upgrade pip
venv/bin/python3 -m pip install --proxy http://<username>:<password>@<hostname>:<port> -r microservices/analyzer/requirements/requirements.txt
      If you cannot use proxy, you can run following script (since XorMon 2.1.20) to generate instructions for manual download and installation of required Python packages: 
      su - xormon
python3 ~/xormon-ng/server-nest/microservices/analyzer/check_requirements.py --package-download-steps
      Restart XorMon: 
      su - xormon   
cd ~/xormon-ng
./xormon.sh stop
./xormon.sh start

  • Other issues

    In case you did not find solution, please contact our support and send us logs via upload.xormon.com
    UI -> settings -> Logs -> Support logs -> follow the form