Thursday, December 17, 2020

Automate Telegraph Agent Installation using SaltStack for Application Monitoring in vROPs Cloud

In this blog, I would demonstrate how to automate the installation of Telegraph Agent using SaltStack. It is required for vROPs Cloud Application Monitoring. 

Once you have it successfully installed you can discover the supported services in vROPs Cloud. 

In my example, I have Apache HTTPd service installed on the CentOS machine running on VMware Cloud on AWS SDDC. 

You can also refer to my previous blog on how to automate log insight agent using salt stack. 










Pre-requisites

Following are the pre-requisites 
  • Working Setup of Saltstack with Master and Minion
    • I have 1 Master and 1 Minion on CentOS, and I will be installing a telegraph agent on the minion 
  • vROPs Cloud Proxy deployed and the vCenter where the VM has been deployed is added in vROPs Cloud 
  • vROPs Cloud Proxy is accessible from the Salt Stack minion. 
  • As we will be using the install script to deploy the agent all the required pre-requisites should be completed as per the documentation.

Procedure


The following section includes steps for installing Telegraph Agent using salt stack.

Step 1


Create the state file vrops-agent.sls. You can refer to the documentation on How to use/create the Salt States




copy-script:
   cmd.run:
      - name: sudo curl -k "https://CloudProxyIPorFQDN/downloads/salt/download.sh" --output /tmp/download.sh
make-script-executable:
   cmd.run:
      - name: sudo chmod +x /tmp/download.sh
install-vrops-agent:
   cmd.run:
      - name: sudo /tmp/download.sh -o install -t <refreshToken> -d /tmp

Update the Cloud Proxy IP/FQDN which will be the source of the download.sh script







Step 2 

Test execute of the state file and ensure it succeeds 

salt <minionhostname> state.sls vrops-agent test








Step 3

Apply the state file to the minion

salt <minionhostname> state.sls vrops-agent

If you want to push it to all the minions that you can just use the following

salt '*' state.sls vrops-agent

When  you use an automation script, concurrent agent installation with a batch size of 20 is supported 















Step 4

Verify agent on the vROPs 







Once you configure the Service Discovery you should be able to see discover services 




 

 

Monday, December 14, 2020

Automate Log Insight Agent Installation using SaltStack


In this blog, I would demonstrate how to automate install VMware Log Insight Agent using SaltStack. 

What is VMware Log Insight Agent? 

A Log Insight agent collects events from log files and forwards them to a vRealize Log Insight and Log Insight Cloud 





















What is Saltstack?

SaltStack is built on a powerful event-driven automation engine that provides IT, devops and SRE professionals with a simple, declarative management framework for their entire global infrastructure.


















Pre-requisites

Following are the pre-requisites 
  • Working Setup of Saltstack with Master and Minion
    • I have 1 Master and 1 Minion on CentOS, and I will be installing liagent on the minion
  • Log Insight Agent saved on Webserver. 
    • I will be installing LI Agent 8.2 
  • Log Insight Agent ini file (Optional, If you are going to use the default from the log insight server) 

Procedure


The following section includes steps for installing LI Agent using salt stack

Step 1


Create the state file liagent.sls . You can refer to the documentation on How to use/create Salt States


 
copy-liagent-rpm:
   cmd.run:
      - name: sudo curl -k http://webserver/LIAgent/VMware-Log-Insight-Agent-8.2.0-16776561.rpm --output /tmp/VMware-Log-Insight-Agent-8.2.0-16776561.rpm
install-liagent:
   cmd.run:
      - name: sudo rpm -i /tmp/VMware-Log-Insight-Agent-8.2.0-16776561.rpm
copy-liagentconfig:
   cmd.run:
      - name: sudo curl -k http://webserver/LIAgent/liagent.ini --output /var/lib/loginsight-agent/liagent.ini
restart-liagentd:
   cmd.run:
      - name: sudo systemctl restart liagentd     








Step 2 

Test execute of the state file and ensure it succeeds 

salt <minionhostname> state.sls liagent test




















Step 3

Apply the state file to the minion

salt <minionhostname> state.sls liagent
If you want to push it to all the minions that you can just use the following

salt '*' state.sls liagent





































Step 4

Verify liagent on the minion

systemctl status liagentd




Wednesday, November 18, 2020

Introducing vRealize LogInsight Cloud Helm Chart for Kubernetes Logs

 In this blog, I would like to Introduce Helm Chart which I built for vRealize LogInsight Cloud. 

It deploys fluentd as deamonset to collect logs from Kubernetes Cluster. I have verified with the following flavors 

  • Tanzu Kubernetes Grid (TKG) K8 Cluster
  • Vanilla K8 Clusters deployed on vSphere/AWS 
Once you have logs flowing you can create your own dashboard to visualize the environment. I have created the following dashboard for my TKG environment

















What is Helm ? 

Helm is a package manager for Kubernetes. Helm is the best way to find, share, and use software built for Kubernetes.

What are Helm charts?

Helm Charts are Kubernetes YAML manifests combined into a single package that can be deployed to on K8 environments. Once packaged, installing a Helm Chart into your cluster is as easy as running a single helm command, which simplifies the deployment & upgrade process

Architecture 




















    Pre-requisites 

    You need to have the following pre-requisites 

    1. vRealize LogInsight Cloud API Token 
    2. Helm Version = '3.x'
    3. Admin access to the Kubernetes Cluster

    Installing the Chart - Procedure 1 


    Step 1 - Add Chart Repo 

    helm repo add loginsight-cloud https://munishpalmakhija.github.io/loginsight-cloud-helm/

    Step 2 - Get Values file in your working directory 

    helm show values loginsight-cloud/loginsight-cloud-helm  > values.yaml

    Step 3 - Update Values file with API Token and other relevant details.  

    cat values.yaml

    Step 4 - Install Chart.  

    helm install test-vrlic loginsight-cloud/loginsight-cloud-helm -f values.yaml











    Step 5 - Verify Kubernetes Pods  

    kubectl get pods -A | grep test-vrlic



    Step 6 - Verify Helm Release 

    helm list




    Installing the Chart - Procedure 2


    Step 1 - Add Chart Repo 

    helm repo add loginsight-cloud https://munishpalmakhija.github.io/loginsight-cloud-helm/

    Step 2 - Install Chart by setting values during run time.  

    helm install test-vrlic loginsight-cloud/loginsight-cloud-helm --set vrlic.apiKey=SETME --set tag.environment=DEMO

    Step 3 - Verify Kubernetes Pods  

    kubectl get pods -A | grep test-vrlic

    Step 4 - Verify Helm Release 

    helm list








    Verify Logs



    Wednesday, October 21, 2020

    Monitor VMware Cloud Proxy with vROPs Docker Management Pack

     This blog includes the details on how to monitor VMware Cloud Proxy with vROps Docker Management Pack 

    https://marketplace.cloud.vmware.com/services/details/b239846b-2aab-4953-901a-959f0009cc7e 



    Value Add 

    It is a Management Pack for collecting performance data from your Docker container platform and providing predictive analysis and real-time information about problems in your infrastructure—all within the vRealize Operations user interface.

    System Requirements 

    Your system meets the following requirements:


    vRealize Operations (vRops) Requirements
    Docker Requirements
    Version(s)

    vRealize Operations 7.x or 8.x

     (Advanced and Enterprise editions)

    Docker, versions 17.x+ (Community or Enterprise editions)

    ConnectionHostname (or IP address) of vROps system

    Hostname (or IP address) of Docker container host

    cAdvisor(Container Advisor), versions 0.25.0+ (See: Configuring cAdvisor (Docker))
    Note:cAdvisor supports HTTP only.

    Default Port: 8080


    Install Docker Management Pack

    You can refer procedure here to install Management Pack 

    https://docs.vmware.com/en/VMware-vRealize-True-Visibility-Suite/1.0/docker/GUID-BA0B33BC-68C6-49AA-9583-F4C33469AB39.html

    Configuring cAdvisor (Docker)

    Execute following command on the Cloud Proxy 

    docker run \
    --restart=always \
    --privileged=true \
    --volume=/cgroup:/cgroup:ro \
    --volume=/:/rootfs:ro \
    --volume=/var/run:/var/run:rw \
    --volume=/sys:/sys:ro \
    --volume=/var/lib/docker/:/var/lib/docker:ro \
    --publish=8080:8080 \
    --detach=true \
    --name=cadvisor \
    --cpus=0.25 \
    --memory=250M \
    google/cadvisor:latest







    Note - Official Documentation doesn't specify any CPU, Memory Configuration but I added the same to ensure it doesn't impact other containers in the RDC. 

    Configuring the Management Pack (Docker)

    You will need to add Adapter Instance in vROPs 

    Procedure

    Log in to vRealize Operations as an admin user and navigate to Administration then

    Select Other Cloud Accounts and Click Add Account













    Select Docker 

    Enter Hostname or IP of the Cloud Proxy and specify a port (Default is 8080)



















    You can review other details in the documentation. For my setup, I have kept it as Default 

    https://docs.vmware.com/en/VMware-vRealize-True-Visibility-Suite/1.0/docker/GUID-A34F22B8-037E-4993-912E-69C179FC14E3.html

    Validate

    Once it finishes data collection then you will see 3 Dashboards populate with Data 































    You can also set Notifications. Like here I have configured the notification to notify me on slack when a new container is created 




























    Saturday, October 10, 2020

    Configure Log forwarding from VMware Tanzu Kubernetes Cluster to vRealize Log-Insight

    As a personal project, I tested forwarding logs from TKG Cluster to vRealize Log Insight. I would love if anyone is willing to try it and provide feedback

    Once the logs are flowing you can create a Dashboard to visualize your TKG environment like the below sample dashboard



    Procedure

    The following section includes steps for running vRealize Log Insight Fluentd plugin as a Daemon set

    Step 1

     Copy the following files from the shared repo https://github.com/munishpalmakhija/log-insight-fluentd 

    • fluent.conf
    • vrli.yaml

    Step 2

    Update the fluent.conf file with the given configuration. You only need to update details about the log insight endpoint under the match section. I have tested with 2 configs

    Config 1

    scheme http

    ssl_verify false

    port 9000

    Config 2

    scheme https

    ssl_verify false

    port 9543


    Step 3

    Create a ConfigMap in Kubernetes for fluent.conf

    kubectl -n kube-system create configmap li-fluentd-config --from-file=fluent.conf

    Step 4

    Apply the creation/changes of new daemon set configuration to the cluster

    kubectl apply -f vrli.yaml

    You can verify that pod is successfully running

    kubectl get pods --all-namespaces | grep log-collector



    Verify Logs 

    If everything is successful you can search for logs using “environment contains tanzu_k8s_grid”






    Saturday, August 8, 2020

    Deploy VMware vROPs Cloud Proxy on VMC/ vSphere using Terraform vSphere Provider

     In my previous posts, I had shared the procedure for following 


    In this blog, I will walk you through how to deploy VMware vROPs Cloud Proxy using Terraform vSphere Provider 


    What is Terraform

     

    Terraform provides Infrastructure as a Code to provision and manage any cloud, infrastructure, or service

     

    Pre-requisites

     

    Following are the pre-requisites

    1. Access to VMware vROPs Cloud Services OTK
    2. VMC /vSphere Endpoint Details are updated in terraform.tfvars
    3. This script assumes has DHCP enabled and has outbound access as required by VMware vROPs Cloud Proxy
    4. This script assumes that you have network connectivity to vCenter/ESXi host where it needs to be deployed
    5. This script assumes that you have outbound connectivity from where you are running the terraform code 

    Usage

     

    The script is published on my GitHub. It uses Terraform vSphere Provider 

     

    https://github.com/munishpalmakhija/Terraform.git

     

     Clone my git repository locally and navigate to the folder Deploy-vROPs-CloudProxy

     




     

    Update the terraform.tfvars with your environment details

     


     

     

    Execute following commands

     

    terraform init





    terraform plan

     



     

    terraform apply -auto-approve

     



     

    In my environment, it took 20-25 mins to deploy Cloud Proxy on VMC. It will depend on how quick your connection is from where you are executing the script

     

    You can reach out to me via Twitter  if you need further details