Sunday, August 26, 2018

How to create your own Fluentd container to send logs to VMware Log Intelligence


I will walk through how to build your own fluentd container which can be used to send logs to VMware Log Intelligence

Earlier blogs had procedure listed how to install fluentd on linux and windows using rpms.

Linux – Click Here
Windows – Click Here

Use Cases


This container helps with  following use cases

  • Centralized Syslog Receiver on standard syslog port (514) and forwards to VMware Log Intelligence
  • Centralized App Logs Receiver on HTTP port 9880 and forwards to VMware Log Intelligence
  • Centralized Server to receive events from other fluentd agents over tcp port 24224 and forwards to VMware Log Intelligence











VMware Log Intelligence is a Cloud based service which means you will need to send logs over WAN and users may not want to send it from all the Machines hence having centralized server which sends out to Cloud would be a good idea 

Steps


We will be using https://github.com/fluent/fluentd-docker-image and modify to install http-out-ext plugin which is needed to send logs to VMware Log Intelligence

Execute the following command to clone the fluentd-docker-image repo

git clone https://github.com/fluent/fluentd-docker-image.git







You will notice it will create a folder "fluentd-docker-image"







I will be using fluentd version 1.2 which latest with Debian image as it is recommended one

Navigate to cd fluentd-docker-image/v1.2/Debian 

You will see Dockerfile 




















The only part which we will be adding is to install the http-out-ext plugin in the docker image

Add following line in the Docker file

&&gem install fluent-plugin-out-http-ext -v 0.1.10 \





















Next, we will run docker build to build the container image

docker build -t docker.io/mmakhija/fluentdcontainer:v0 ./





















A couple of points to note

  • The reason I am choosing this name is to that I can push the container name to public docker hub. You don't need to do it unless you want to share it with others. If you have your own private repo you can use that
    • mmakhija – docker hub username
    • fluentdcontainer - repo name on docker hub
  • It will take a couple of mins to build the docker. It will show some red color. Don't be scared it like I was when I did it for the first time


You can view the image by executing the following command

docker images




Next, you will need to copy fluentd.conf listed below on the docker host. Please remember the path where you are saving it because you will need to specify that when running docker run command.

I have saved the file at which I have specified below "/fluent/fluentd.conf"





















https://github.com/munishpalmakhija/fluentd/blob/master/fluent.conf


Execute the following command to run the fluentd docker container.

docker run -d -p 24224:24224 -p 24224:24224/udp -p 514:5140 -p 514:5140/udp -p 9880:9880 -v /fluent:/fluentd/etc/  mmakhija/fluentdcontainer:v0

Please ensure to match the name of the image which you had it when you ran docker build command and the path of the fluent.conf file





If everything goes you will see a fluentd container running as below and you can configure ESXi host to forward logs to ip of the docker host where this container is running on port 514 and you should see logs flowing









Please feel free to leave comments or suggestions of if something not working 


Saturday, August 25, 2018

Windows vCenter Log forwarding to VMware Log Intelligence using Fluentd

I co-authored the blog where we displayed how to install fluentd and send logs to VMware Log Intelligence (Click Here) however we did that for Linux which covers most of the scenarios however in this blog I will walk through fluentd installation on Windows where I have vCenter installed as an Application

Steps


In following section I will walk through how to install fluentd on windows 2012 R2 which has vCenter 6.5 installed

Install td-agent


You can download the ".msi" file from here, and install the software












Configure and Run td-agent


After you've installed .msi package, you'll see the program called Td-agent Command Prompt installed. Please double click this icon in the Windows menu and execute following command

fluentd -c etc\td-agent\td-agent.conf






























Please launch another Td-agent Command Prompt and type the command below

echo {"message":"hello"} | fluent-cat debug.event

It's working properly if td-agent process outputs the message

















Register Run td-agent as Windows Service


Please execute Td-agent Command Prompt again but with administrative privilege, and type the two commands below.

fluentd --reg-winsvc i
fluentd --reg-winsvc-fluentdopt '-c C:/opt/td-agent/etc/td-agent/td-agent.conf -o C:/opt/td-agent/td-agent.log'









You will now be able to see "Fluentd Windows Service" as one of the services installed

Right Click and start the service. By default, the Startup Type is Manual you can change it to Automatic if needed

You should see log file being created at "C:/opt/td-agent/td-agent.log"





Install plugins


We will need to install HTTP Output plugin additionally to send logs directly to VMware Log Intelligence.

Please execute following command in Td-agent Command Prompt

fluent-gem install fluent-plugin-out-http-ext









Configure td-agent.conf with Windows vCenter Details


Default location for vCenter vpxd logs is "C:\ProgramData\VMware\vCenterServer\logs\vmware-vpx"

We will need a pos file. I have created the file in the same location as vpxd to keep it simple. Please ensure it has appropriate permissions

Modify the default td-agent.conf which is located at "C:\opt\td-agent\etc\td-agent.conf" by replacing it will following configuration.




https://github.com/munishpalmakhija/fluentd/blob/master/td-agent-windows.conf

Please note you will need to generate your own API key for your org as mentioned in my blog here

Restart the "Fluentd Windows Service" for the new config to be used

If everything goes well you should be able to view logs by filtering out hostname contains windowsvcenter