Sending Windows Event Logs to Logstash / Elasticsearch / Kibana with nxlog

Edit: This post is pretty old and Elasticsearch/Logstash/Kibana have evolved a lot since it was written.

Part 3 of 4 – Part 1Part 2Part 4
This is a continuation of http://www.ragingcomputer.com/2014/02/logstash-elasticsearch-kibana-for-windows-event-logs

Again, I took a lot of inspiration from http://sysxfit.com/blog/2013/07/18/logging-with-logstash-part-3/

The nxlog reference manual is surprisingly well written with excellent examples.
http://nxlog.org/nxlog-docs/en/nxlog-reference-manual.pdf

Loggly has some examples I found useful, even if I’m not using their service.
http://community.loggly.com/customer/portal/articles/1266344-nxlog-windows-configuration
https://www.loggly.com/docs/logging-from-windows/

There are other options.
http://www.canopsis.org/2013/05/windows-eventlog-snare-logstash/
http://docs.fluentd.org/articles/windows
http://cookbook.logstash.net/recipes/log-shippers/
http://cookbook.logstash.net/recipes/windows-service/

INSTALL NXLOG
Download and run the windows installer. This is a very fast install.
http://sourceforge.net/projects/nxlog-ce/files/

Edit your nxlog.conf. Its location will depend on your OS.

32bit OS
[text]
C:Program Filesnxlogconfnxlog.conf
[/text]
64bit OS
[text]
C:Program Files (x86)nxlogconfnxlog.conf
[/text]

Note: You will need to modify the Define ROOT depending on 32bit or 64bit install.
Note: You will need to modify the input eventlog section depending on windows version.
Note: You will need to modify Host in the Output section to the IP address or hostname of you logstash computer.
[text]
## Please set the ROOT to the folder your nxlog was installed into,
## otherwise it will not start.

#define ROOT C:Program Filesnxlog
define ROOT C:Program Files (x86)nxlog

Moduledir %ROOT%modules
CacheDir %ROOT%data
Pidfile %ROOT%datanxlog.pid
SpoolDir %ROOT%data
LogFile %ROOT%datanxlog.log

<Extension json>
Module xm_json
</Extension>

# Nxlog internal logs
<Input internal>
Module im_internal
Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json();
</Input>

# Windows Event Log
<Input eventlog>
# Uncomment im_msvistalog for Windows Vista/2008 and later
Module im_msvistalog

# Uncomment im_mseventlog for Windows XP/2000/2003
# Module im_mseventlog

Exec $EventReceivedTime = integer($EventReceivedTime) / 1000000; to_json();
</Input>

<Output out>
Module om_tcp
Host 192.168.1.126
Port 3515
</Output>

<Route 1>
Path internal, eventlog => out
</Route>
[/text]

START NXLOG SERVICE

Finally, start the service. Either open computer management, open services, find nxlog in the list and start or from an administrator command prompt
[text]net start nxlog[/text]

I’m kinda lazy and doing repetitive tasks by hand isn’t my cup of tea, so since I had 20 identical machines to install this on, I whipped up this bat file for installing it while I remotely connected for other maintenance.

install-nxlog.bat
[text]
@echo off
echo installing nxlog
msiexec /passive /i "\shareserversharenamepathtonxlognxlog-ce-2.6.1131.msi"
echo copying configuration
move "C:Program Filesnxlogconfnxlog.conf" "C:Program Filesnxlogconfnxlog.conf.default"
copy "\shareserversharenamepathtonxlognxlog.conf" "C:Program Filesnxlogconfnxlog.conf"
echo starting service
net start nxlog
echo done
[/text]

7 thoughts on “Sending Windows Event Logs to Logstash / Elasticsearch / Kibana with nxlog

  1. Hello, I deploy the same enviroment base on your instruction, but whe I started nxlog I see the following in the log file:
    2014-07-02 18:03:39 INFO connecting to 10.240.48.53:3515
    2014-07-02 18:03:39 INFO nxlog-ce-2.7.1191 started
    But any records haven’t copied to logstash.
    How to debug nxlog or maybe receive more detail logs.

    Like

  2. i am seeing this

    ERROR couldn’t connect to tcp socket on 192.168.1.22:3515; No connection could be made because the target machine actively refused it.

    any thoughts ?

    Like

  3. Das, Mike

    If the machine is actively refusing it… I’d check that your logstash service is listening on that port on that host and there is not a firewall blocking communication.

    Things to check:
    Logstash service is running
    Logstash listening port is correct
    Logstash listening interface is correct
    Logstash is allowed through machine firewall
    nxlog config is pointed to the correct host
    nxlog config is pointed to the correct port

    Like

  4. Nice document any chance you can do a revision to to get more current with some of the products

    it seems some of the links in your post have expired or are not available

    Thanks. Alex

    Like

  5. Wonderful document. A unique and indispensable guide. I followed it to the “t”.
    But I cannot get the hostnames of the windows machines in the logs, I am testing this at home on windows 7/8, at work I need to implement this for our PDC’s.

    These are the messages I am getting(not very explanatory):

    September 30th 2015, 06:03:49.685 message: @version:1 @timestamp:September 30th 2015, 06:03:49.685 host:%{host2} type:WindowsEventLog tags:[“_grokparsefailure”,”_jsonparsefailure”] FileName: source_host: eventlog_severity: AccountName: eventlog_channel: EventType: Hostname: Severity: _source:{“message”:”\u0000\b\u0000\u0006\u0000\u0017\u0000\u0018\u0000\u0019\u0000\u000b\u0000\u0002\u0001\u0000\u0000\r\u0000″,”@version”:”1″,”@timestamp”:”2015-09-30T11:03:49.685Z”,”host”:”%{host2}”,”type”:”WindowsEventLog”,”tags”:[“_grokparsefailure”,”_jsonparsefailure”],”FileName”:null,”source_host”:null,”eventlog_severity”:null,”AccountName”:null,”eventlog_channel”:null,”EventType”:null,”Hostname”:null,”Severity”:null} _id:AVAd7DtTaXmKBthRAE0K _type:WindowsEventLog _index:logstash-2015.09.30

     September 30th 2015, 06:03:49.685 message: @version:1 @timestamp:September 30th 2015, 06:03:49.685 host:%{host2} type:WindowsEventLog tags:[“_grokparsefailure”,”_jsonparsefailure”] FileName: source_host: eventlog_severity: AccountName: eventlog_channel: EventType: Hostname: Severity: _source:{“message”:”\u0000\b\u0000\u0006\u0000\u0017\u0000\u0018\u0000\u0019\u0000\u000b\u0000\u0002\u0001\u0000\u0000\r\u0000″,”@version”:”1″,”@timestamp”:”2015-09-30T11:03:49.685Z”,”host”:”%{host2}”,”type”:”WindowsEventLog”,”tags”:[“_grokparsefailure”,”_jsonparsefailure”],”FileName”:null,”source_host”:null,”eventlog_severity”:null,”AccountName”:null,”eventlog_channel”:null,”EventType”:null,”Hostname”:null,”Severity”:null} _id:AVAd7EdbaXmKBthRAF6e _type:WindowsEventLog _index:logstash-2015.09.30

     September 30th 2015, 06:03:49.685 message: @version:1 @timestamp:September 30th 2015, 06:03:49.685 host:%{host2} type:WindowsEventLog tags:[“_grokparsefailure”,”_jsonparsefailure”] FileName: source_host: eventlog_severity: AccountName: eventlog_channel: EventType: Hostname: Severity: _source:{“message”:”\u0000\b\u0000\u0006\u0000\u0017\u0000\u0018\u0000\u0019\u0000\u000b\u0000\u0002\u0001\u0000\u0000\r\u0000″,”@version”:”1″,”@timestamp”:”2015-09-30T11:03:49.685Z”,”host”:”%{host2}”,”type”:”WindowsEventLog”,”tags”:[“_grokparsefailure”,”_jsonparsefailure”],”FileName”:null,”source_host”:null,”eventlog_severity”:null,”AccountName”:null,”eventlog_channel”:null,”EventType”:null,”Hostname”:null,”Severity”:null} _id:AVAd7FePaXmKBthRAHj8 _type:WindowsEventLog _index:logstash-2015.09.30

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: