Installation

Kafka deployment

Deploying a single-node Kafka cluster with schema registry in Docker
(this procedure should work generaly on any Linux with some modifications)

Kafka can be deployed on a standalone VM - preferred solution

Kafka can be deployed on the XorMon appliance - not recommended.

Read before proceeding! This procedure is intended for testing SANnav monitoring in XorMon.
The solution described in this guide comes without any guarantee or support.

Prerequisites
Kafka deployment
Management Scripts
Basic Troubleshooting

Prerequisites

Disable SELinux and reboot

RedHat/Rocky Linux

sudo sed -i 's/SELINUX=enforcing/SELINUX=disabled/g' /etc/selinux/config
sudo reboot

Update OS and Install additional packages

Debian

sudo apt clean; sudo apt update
sudo apt upgrade
sudo apt install bash-completion wget bind9-utils tar jq curl ca-certificates

RedHat/Rocky Linux

sudo dnf clean all; sudo dnf update
sudo dnf install bash-completion wget bind-utils tar jq curl nmap-ncat

Disable or configure firewall

Required ports

XorMon - SANnav

API TCP/443

SANnav - Kafka

Schema registry TCP/8081
Kafka broker TCP/9093

XorMon - Kafka

Schema registry TCP/8081
Kafka broker TCP/9092 and TCP/9093

Debian
Disable UFW

sudo ufw disable

Configure UFW

# Kafka NB
sudo ufw allow 8081/tcp
sudo ufw allow 9092/tcp
sudo ufw allow 9093/tcp

# XorMon
sudo ufw allow 8081/tcp
sudo ufw allow 9092/tcp
sudo ufw status verbose

RedHat/Rocky Linux
Disable firewalld

sudo systemctl stop firewalld; sudo systemctl disable firewalld

Configure firewalld

# Kafka NB
sudo firewall-cmd --zone= --add-port=8081/tcp --permanent
sudo firewall-cmd --zone= --add-port=9092/tcp --permanent
sudo firewall-cmd --zone= --add-port=9093/tcp --permanent
sudo firewall-cmd --reload

# XorMon
sudo firewall-cmd --zone= --add-port=8081/tcp --permanent
sudo firewall-cmd --zone= --add-port=9092/tcp --permanent
sudo firewall-cmd --reload
sudo firewall-cmd --list-port

Install Docker Engine

Debian: https://docs.docker.com/engine/install/debian/
RedHat/Rocky Linux: https://docs.docker.com/engine/install/rhel/

Create user kafkanb

This procedure requires that the user "kafkanb" have sudo privileges and be able to use Docker

Debian

sudo adduser kafkanb
sudo usermod -aG docker kafkanb
sudo usermod -aG sudo fakfkanb

RedHat/Rocky Linux

sudo useradd -m kafkanb
sudo passwd kafkanb
sudo usermod -aG docker kafkanb
sudo usermod -aG wheel kafkanb

Kafka deployment

Perform following steps a user kafkanb

Login as kafkanb

su - kafkanb

Download and unpack depolyment scripts

cd ~
wget https://download.xormon.com/kafkanb.tgz
tar zxvf kafkanb.tgz
chown -R kafkanb:kafkanb kafkanb
rm -f kafkanb.tgz

Deploy and register Kafka NB (as kafkanb)

Create configuration
Following script will create ~/kafkanb/etc/kafkanb direcotry and populate it with necessary configuration files.
You will have to confirm or enter the local host's IP address.
```
cd ~/kafkanb/bin
./kafka_configure.sh
```
Deploy containers
This script will create ~/kafkanb/kafkanb-run direcotry, pull required images from Docker Hub and start Kafka NB containers.
```
cd ~/kafkanb/bin
./kafka_deploy.sh
```

Check containers' status after a minute
All three containers have to be up and running.

docker ps

CONTAINER ID   IMAGE                                    COMMAND                  CREATED          STATUS          PORTS                                        NAMES
92da81c54966   confluentinc/cp-schema-registry:6.0.1    "/etc/confluent/dock…"   51 minutes ago   Up 50 minutes   0.0.0.0:8081->8081/tcp                       kafkanb_schema_registry
ce727dac41c8   confluentinc/cp-kafka:6.0.1              "/etc/confluent/dock…"   51 minutes ago   Up 50 minutes   0.0.0.0:9092-9093->9092-9093/tcp             kafkanb_broker
7cd3f1437dd6   confluentinc/cp-zookeeper:6.0.1          "/etc/confluent/dock…"   51 minutes ago   Up 51 minutes   2888/tcp, 0.0.0.0:2181->2181/tcp, 3888/tcp   kafkanb_zookeeper

Register Kafka NB in SANnav
Following script will register Kafks NB server in SANnav.
It requires SANnav IP addres and administrator credentials.
It may also require confirmation of the local host's IP address.
```
cd ~/kafkanb/bin
./kafka_register.sh
```

Management Scripts

The ~/kafkanb/bin directory contains several useful scripts for administering the Kafka NB server.

kafka_deploy.sh
Creates and starts docker containers with Kafka northbound server during initial installation.
Creates ~/kafkanb/kafkanb-run directory.
You can use this script to re-deploy Kafka NB server if necessary.
kafka_register.sh
Registers Kafka NB server to SANnav.
Requires SANnav IP addres and administrator credentials.
It may also require confirmation of the local host's IP address.
kafka_listnb.sh
Lists northbound servers registered to SANnav.
kafka_stop.sh
Stops Kafka NB containers without removing them.
kafka_start.sh
Starts previously stopped Kafka NB containers.
kafka_unregister.sh
Removes previously registered Kafka NB server from SANnav.
kafka_delete.sh
Force stops and deletes Kafka NB containers.
Removes ~/kafkanb/kafkanb-run directory.

Basic Troubleshooting

Check log the file: ~/kafkanb/logs/kafkanb.log

Check Kafka NB containers' status

All three containers must be Up

docker ps

CONTAINER ID   IMAGE                                    COMMAND                  CREATED          STATUS          PORTS                                        NAMES
92da81c54966   confluentinc/cp-schema-registry:6.0.1    "/etc/confluent/dock…"   51 minutes ago   Up 50 minutes   0.0.0.0:8081->8081/tcp                       kafkanb_schema_registry
ce727dac41c8   confluentinc/cp-kafka:6.0.1              "/etc/confluent/dock…"   51 minutes ago   Up 50 minutes   0.0.0.0:9092-9093->9092-9093/tcp             kafkanb_broker
7cd3f1437dd6   confluentinc/cp-zookeeper:6.0.1          "/etc/confluent/dock…"   51 minutes ago   Up 51 minutes   2888/tcp, 0.0.0.0:2181->2181/tcp, 3888/tcp   kafkanb_zookeeper

List northbound servers registered in SANnav

kafkaClusterUrl and schemaRegistryUrl must match XorMon host's IP address
FC Port and Switch streams must be enabled: streamState: 1

~/kafkanb/bin/kafka_listnb.sh

# SANnav IP address, user and password
Make sure user SANnav user has a priviledge to register and manage Northbound servers
SANnav IP address [192.168.0.10]:
SANnav username [Administrator]:
SANnav password:
[
  {
    "name": "kafka4stor",
    "kafkaClusterUrl": "192.168.0.24:9093",
    "schemaRegistryUrl": "http://192.168.0.24:8081",
    "caPublicCertificate": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURMVENDQWhXZ0F3SUJBZ0lVYmxJa0Z5R3pnWTRUelNJZTVvU2tTZTA3b1Rzd0RRWUpLb1pJaHZjTkFRRUwKQlFBd0pqRU9NQXdHQTFVRUNnd0ZXRzl5ZFhneEZEQVNCZ05WQkFNTUMxaHZjblY0SUV0aFptdGhNQjRYRFRJeQpNRFl5TWpFME1EWTFORm9YRFRNeU1EWXhPVEUwTURZMU5Gb3dKakVPTUF3R0ExVUVDZ3dGV0c5eWRYZ3hGREFTCkJnTlZCQU1NQzFodmNuVjRJRXRoWm10aE1JSUJJakFOQmdrcWhraUc5dzBCQVFFRkFBT0NBUThBTUlJQkNnS0MKQVFFQW96cHl1WmlCcjdFK3NMeDgxWkcvdXZ1V2M2eHlsbXY4WE1ZVlgvSnlkU2Y4TE82czlkSkd0WTBBVXZOYwpuK3V6VFZITnNZbDhwczN1eW1sVDF6MXJzY3JYQXlhRDBMTmtKZFRTVG02SEpCT2czSzE5Yk1hMXBFUVd1b0k1CkFBTHFtK1ZDWnRodXZGVkdLZWI0MncxR2NlNlhsVTdWWEpkcGx2eFdRUDFmdjN6QVF6ckRQS0RTcStQMUhablYKZ2l4T2V1S1FkMFFIS3NQb1lSeStoS1Iyc2NjUis3a0xYTXJtRFIvMnA3OWlZR2U4SzN1US9INmc3MTNGakdlSgpNWi9ocHZoWHFxL2J2cjhlQm92VUdRSjRJOXRGSHQyWG9SWVpHaWVjU25GdE1rOFNCZUV6am52SzRFTTlQcm1zClBHM0RRMTdtOUZQclpKMkYvYk1tcERpQ2h3SURBUUFCbzFNd1VUQWRCZ05WSFE0RUZnUVU2QmhYUXdqbGdjZ28KWmhGeHlSMnpoM3pLczlBd0h3WURWUjBqQkJnd0ZvQVU2QmhYUXdqbGdjZ29aaEZ4eVIyemgzektzOUF3RHdZRApWUjBUQVFIL0JBVXdBd0VCL3pBTkJna3Foa2lHOXcwQkFRc0ZBQU9DQVFFQW5UbkVuVk81RUljTWNlaitiMkhNCkFLVGxmd3c3bXBLc2N0QTN0SXNnOEFuZnVFSTZaaFBueTFKdGkwVC9abHY2clJnanBnaUF3TmV4bFdwK25vUVcKUjQybm92aDRNanZqTFBrU1d1OHdlR0w3SUM3aFhiTlYrYVhpWjZUNFAydGFDVk1UNyQ0R6djY4SUo4NkRGZApkMWJW42NUNRVGtEWFdiZW9PZTempXcUhVbnM0TDIrY2VJMDZDFndkdlMTRoWlMkR3OFdZemJpWHpxCm9TlYTmRMTTlFYkgwV2aXNCVllTUnc1M1RJMmZ0dOVlZBYmZxYnNFdnNE55b0daNldsaEx0ZraVgKQzRIMEhiVaZHFyNW13MzV6YUwxdFQld6TFlONWNQbkFQNEUDJ4ZUpyNmV2TmFaa3ZmVnVyRjJ5F2MworUT09Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0=",
    "id": 18,
    "connectionState": 0,
    "connectionStateReason": "SANnav successfully connected to Northbound Server.",
    "streamDetails": [
      {
        "streamType": 1,
        "streamName": "FC Port",
        "streamState": 1
      },
      {
        "streamType": 2,
        "streamName": "Eth/GigE Port",
        "streamState": 0
      },
      {
        "streamType": 3,
        "streamName": "Extension Tunnel/Circuit",
        "streamState": 0
      },
      {
        "streamType": 4,
        "streamName": "Switch",
        "streamState": 1
      },
      {
        "streamType": 5,
        "streamName": "Flow",
        "streamState": 0
      }
    ]
  }
]

Table of Contents

Prerequisites

Disable SELinux and reboot

Update OS and Install additional packages

Disable or configure firewall

Install Docker Engine

Create user kafkanb

Kafka deployment

Login as kafkanb

Download and unpack depolyment scripts

Deploy and register Kafka NB (as kafkanb)

Management Scripts

Basic Troubleshooting