Collecting vSphere Syslog Data with Graylog2

Are you looking for a good syslog collector for your environment but not wanting to shell out the money for Splunk or VMware vCenter Log Insight? Not saying there is anything wrong with either one of these products except they can be very expensive and they are overall excellent products. Especially how tightly integrated VMware vCenter Log Insight is with other VMware products but if all you really care about is getting good syslog data which is presented very well and extremely useful well Graylog2 may be just want you need for your environment. I have been using Graylog2 for a few months now and I absolutely love it. Well of course one reason is the cost (Open Source) and over the years of using other open source syslog tools it is an extremely well groomed product and worth looking into. There is also a great community around this product as well which are very helpful and responsive.

So my intention of this post is to make those aware of the product that may not be and to help those who know about the product but may have issues getting it up and running successfully. I would like to first direct you to a previous post I did which is all about a shell script available on github which will automate the complete installation of Graylog2 on Ubuntu and Debian. I have been working on this script for a few months now and it is has been very successful for many including myself. I keep this script updated with the latest versions as well as tweaking it along the way as the need arises. Several have requested certain features and checks to be included so I like to incorporate some of those ideas into the script when they make sense. So to follow the install instructions and get an idea of how to get Graylog2 up and running head over to this post and then return to this post once you have Graylog2 up and running using the script from Method #1.

Now assuming that you have Graylog2 up and running let’s configure our ESXi hosts to send their syslogs to our new Graylog2 server.

Open your vSphere Client and select your host from vCenter or connect directly to your host. Select configuration and then advanced settings under the software section. Scroll down and expand Syslog and select global. Now under Syslog.global.loghost fill in udp://ipaddressofgraylog2:514 then click ok.

11-59-45

 

Now you have to allow syslog data to be sent from your host. In order to do this you must configure the firewall on the host to allow this. So on the configuration page select security profile under the Software section. Scroll down to syslog and enable the checkbox and click ok.

13-16-34

Once that is done you should start to see syslog data showing up in Graylog2. The only other thing you might want to do is make sure that each of your hosts are showing up as unique hosts within Graylog2. Other than that you are good to go. You can also configure other devices in your environment to send their syslog data back to your Graylog2 server.

Enjoy!

9 thoughts on “Collecting vSphere Syslog Data with Graylog2

  1. Hello, Larry! Tell me please, how do you manage elasticsearch queries? For example, how can i delete some two months ago logs? Could you recommend me some utility to managing elasticsearch's requests?

    Спасибо!

  2. I'm very grateful for your answer, Larry! I did elasticsearch curator's installation and executed sudo curator -d 1 (that means delete all Indices older than 1 day) output was: 2014-06-25T16:15:39.570 INFO main:332 Job starting… 2014-06-25T16:15:39.571 INFO _new_conn:187 Starting new HTTP connection (1): localhost 2014-06-25T16:15:39.573 INFO log_request_success:49 GET http://localhost:9200/ [status:200 request:0.002s] 2014-06-25T16:15:39.573 INFO main:358 Deleting indices older than 1 days… 2014-06-25T16:15:39.575 INFO log_request_success:49 GET http://localhost:9200/_settings [status:200 request:0.001s] 2014-06-25T16:15:39.575 INFO index_loop:308 DELETE index operations completed. 2014-06-25T16:15:39.575 INFO main:378 Done in 0:00:00.008550.

    But nothing happen! Tell me please, Larry – index is a message (example:"2014-06-25 15:46:29.581 syslog host name not found: 0.172.31.0.1 " or something also? I installed elasticsearch just 2 days ago (thanks a lot to your magic script) and can't understand structure of elasticsearch's database. Whether it has anything common with mysql database structure or totally different from it? If i'll "delete indices" older-than-1-day – it must delete all messages-older-one-day or "hyperlink" to that messages?

  3. Do not panic! I have found answer in Curator FAQ:

    Q: Can I delete only certain data from within indices?

    A: It's complicated

    TL;DR: No. Curator can only delete entire indices.

    So, how to make every day a new index was created? If I understand correctly, now all my logs for 3 days are in a single index, right?

    • I'll answer to myself again 🙂 graylog2.conf contain 2 parameters that we need ( elasticsearch_max_number_of_indices (When that number is reached the oldest indices will automatically be deleted) and elasticsearch_max_docs_per_index ). elasticsearch_max_docs_per_index * elasticsearch_max_number_of_indices = maximum number of messages stored. For example, if i have 1000 messages around per day =30000 messages per month – i take this value 30000 and dividing on 5000 messages per index. I need 6 indecies as a result. So, i set elasticsearch_max_docs_per_index=5000 and elasticsearch_max_number_of_indices=6. I suppose now then number of messages will attain 30000, graylog will delete oldest index with 5000 writes. I think that it is too crude method, but the other has not yet found. I would like to be able to remove only certain messages, such as messages from a particular source, or some "noice" verbose messages from esxi for example

      • @Gekm – I have to do some digging to try and sort all of this out. I would hate to see you lose data. I just use elasticsearch curator as a cron job to delete everything over 90 days old in my setup.

Leave a Reply

Your email address will not be published. Required fields are marked *

*