Osquery is a cross-platform system monitoring tool that exposes information about a system through a SQL interface. Elasticsearch is a distributed schema-free search server based on Lucene. In this tutorial, you will learn first how to send osquery logs to syslog, and then send those logs to Elasticsearch.

Osquery

Osquery reporting

Osquery runs in two modes - osqueryi, an interctive mode, and osqueryd, a daemon mode. Osquery’s daemon mode allows for scheduled configurable queries to be run without user interaction. Conveniently, osqueryd natively supports sending the result of its scheduled queries to syslog in a JSON format.

To quickly setup osqueryd, once installed, copy its example configuration file osquery.example.conf (in my system it islocated in /usr/share/osquery/) to /etc/osquery/osquery.conf. Once copied, change the logger plugin in the osquery configuration file from file to syslog. To do this, change the line

"logger_plugin": "filesystem",

to

"logger_plugin": "syslog",

For reference, these logs are sent with a facility of 19, or “local3”. On my Ubuntu 14.04 installation, Rsyslog was not logging these messages. To fix that, I have added the following line to the end of my rsyslog.conf file:

local3.* -/var/log/syslog

The syslog-ng OSE config file

To gather the logs from osquery, you need a source that collects system messages, and a filter that can identify logs from osqueryd.

An example of such a source is:

source s_local {
        system();
};

An example of such a filter is:

filter osqueryd {

program("^osqueryd.*");
};

Elasticsearch

At the time of writing, there is not an official Elasticsearch destination in the syslog-ng OSE repository, but it will be added soon. Until then, the source for a suitable destination can be found at https://github.com/juhaszviktor/ESDestination. Full documentation on the destination can be found there.

Requirements

  • Elasticsearch libraries
  • SyslogNg.jar
  • JDK min 1.7 for building
  • JRE min 1.7 for usage

Building

You may need to change the directory of the relevant .jar files. If you do, you can specify their locations by:

javac -classpath "/usr/lib/syslog-ng/3.6/SyslogNg.jar:/usr/share/elasticsearch/lib/elasticsearch-1.4.4.jar:/usr/share/elasticsearch/lib/log4j-1.2.17.jar" -d bin src/org/syslog_ng/elasticsearch/ElasticSearchDestination.java

Packaging

jar -cvf ESDestination.jar -C bin .

Example Configuration:

Environment

  • In this example the SyslogNg.jar file is under the directory /usr/lib/syslog-ng/3.6
  • The elasticsearch libraries are under the /usr/share/elasticsearch/lib/ directory
  • The built .jar file is under the /usr/local/ directory

Configuration example

The following configuration file example takes messages from osquery and sends them to Elasticsearch:

@version: 3.6

options {
  threaded(yes);
  use_rcptid(yes);
};

source s_local {
        system();
};

filter osqueryd {

program("^osqueryd.*");
};


destination d_es {
  java(
    class_path("/usr/local/ESDestination.jar:/usr/share/elasticsearch/lib/*.jar")
    class_name("org.syslog_ng.elasticsearch.ElasticSearchDestination")
    option("index", "syslog-ng_${YEAR}.${MONTH}.${DAY}")
    option("type", "test")
    option("cluster", "syslog-ng")
    option("flush_limit", "100")
  );
};

log {
  source(s_local);
  filter(osqueryd);
  destination(d_es);
  flags(flow-control);
};

Updated: