Setting Up Kube in a Box: Running Kubernetes Visibility Agent in My In-House Environment

Sakshi_Tiwari · ‎09-04-2024

Kubernetes has rapidly become the leading container orchestration platform, with over 90% of organizations using containers opting for it due to its scalability and flexibility. As enterprises increasingly adopt multi-cloud strategies—85% now have one—managing multiple Kubernetes clusters across diverse environments has become a complex challenge. By consolidating cluster management, organizations can optimize resource utilization, enforce consistent governance, and simplify compliance, ultimately improving operational efficiency and reducing risk.

ServiceNow’s Kubernetes Visibility Agent can help you discover all your K8s resources. In this blog, I’ll share my experience of setting up a Kubernetes cluster(minikube) locally and discovering the resources using the Kubernetes Visibility Agent feature.

This comprises of three major steps:-

Installing minikube on your machine
Installing relevant plugins on your ServiceNow instance
Deploying Informer onto the minikube cluster

What is Kubernetes Visibility Agent (KVA)?

Kubernetes Visibility Agent detects resource changes in a Kubernetes cluster. It performs continuous discovery, reports changes to your instance, and keeps your Configuration Management Database (CMDB) up to date with the latest data for optimal accuracy. Upon deployment,Kubernetes Visibility Agent sets up a Deployment resource in your cluster, featuring an Informer pod. This pod connects to the Kubernetes API server, efficiently receiving and processing resource events. KVA enables you to gain visibility into on-premises Kubernetes clusters as well as the following Cloud deployments: Amazon Elastic Kubernetes Service (EKS), Google Kubernetes Engine (GKE), Azure Kubernetes Engine (AKS), RedHat OpenShift, and Rancher.

[I]. Setting up ‘Kube in a box’

"Kubernetes in a box" typically refers to setting up a local Kubernetes cluster on a single machine for development, testing, or learning purposes. This can be done using tools like Minikube, Kind (Kubernetes IN Docker), or MicroK8s. Below are the steps to install Kubernetes using Minikube, which is one of the most popular tools for this purpose.

Pre-requisites:

Operating System: Minikube supports Linux, macOS, and Windows.
Virtualization: Minikube can use a hypervisor such as VirtualBox, HyperKit, Hyper-V, KVM, or Docker. Ensure you have one installed.

Steps:

1. Install ‘Minikube’ (On a linux VM)

> curl -LO https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
> sudo install minikube-linux-amd64 /usr/local/bin/minikube

2. Install ‘kubectl’ (command-line tool to interact with Kubernetes cluster

> curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/ bin/linux/amd64/kubectl"
> sudo install kubectl /usr/local/bin/

3. Start Minikube

> minikube start --driver=<your_chosen_driver>  [docker in my case]

4. Verify installation (you should see output indicating that Minikube and its components are running)

> minikube status

5. Use kubectl to interact with Minikube (for e.g., to see the nodes)

> kubectl get nodes -n <namespace>

[II]. ServiceNow Instance: Installing the KVA plugin

On the ServiceNow instance:

Ensure that the following plugins are installed and activated -
- Discovery and Service Mapping Patterns
- The latest version of Kubernetes Visibility Agent[KVA]
Navigate to All > User Administration > Users
Choose or create a user with at least a discovery_admin role.

[III]. Configuring the Informer

When you deploy KVA, Kubernetes creates a Deployment resource in the cluster. This resource uses a secret stored in Kubernetes to connect to your ServiceNow instance. The KVA Deployment resource contains a pod called Informer, which connects to the Kubernetes API server and receives events on the resources in the cluster from it.

Prepare for deployment:

1. On the Kubernetes cluster Run the following command, after replacing <NAMESPACE> with the relevant value:

> kubectl create namespace <NAMESPACE>

2. Create a Kubernetes secret that contains the ServiceNow credentials for accessing your ServiceNow instance

> kubectl create secret generic k8s-informer-cred-INSTANCE_NAME --from-literal=.user=USERNAME --from-literal=.password=PASSWORD -n NAMESPACE

Note: Ensure that the credentials correspond to the user you created or identified in the earlier step.

Steps to deploy informer using Kubernetes YAML file:

1. Download the Kubernetes YAML file from here: CNO for Visibility Helm Chart and Kubernetes YAML file releases - Support and Troubleshooting (servi...

2. Extract the k8s_informer.yaml and EULA.pdf files from the zip file

> Edit the k8s_informer.yaml file > Change the value of ACCEPT_EULA to "Y"

3. Run the command

> kubectl apply -f k8s_informer.yaml

Following the completion of these steps, the informer has been effectively set up inside the Kubernetes cluster, as depicted:

After the Informer has been deployed in the Kubernetes Cluster, it begins to transmit details regarding K8s resources back to the instance, allowing you to observe the specifics on the dashboards as illustrated below.

ServiceNow Instance: Kubernetes Dashboard

Kubernetes dashboard offers a unified view for monitoring and managing Kubernetes clusters. It includes features like cluster and node overviews, pod management, deployment tracking, and resource utilization insights. This centralized view helps teams efficiently manage their Kubernetes environments and ensures optimal performance and timely issue resolution

Cloud Discovery Workspace [Moving to DAW soon]

The Cloud Discovery Workspace [CDW] – Kubernetes Explorer view provides comprehensive insights, including the total number of clusters, a detailed breakdown of pods, and the namespaces within your Kubernetes resources for efficient management and monitoring.

Wow Moment!

Deploying the informer was like flipping a switch—suddenly, everything clicked into place. Instantly, data started pouring into the ServiceNow instance, and I was met with vibrant visualizations of pod names, clusters, and memory usage in a beautifully crafted pie chart. The ease of the deployment, paired with such clear and engaging visuals, were the defining highlights of my experience.

Observe how swiftly the pod updates are mirrored in the ServiceNow instance: