This section describes the options of the regexp-parser() in syslog-ng OSE.

The Regular expression parser has the following options.

Literal string searches

Literal string searches have the following flags() options:

global

Usable only in rewrite rules, flags("global") matches for every occurrence of the expression, not only the first one.

ignore-case

Disables case-sensitivity.

prefix

During the matching process, patterns (also called search expressions) are matched against the input string starting from the beginning of the input string, and the input string is matched only for the maximum character length of the pattern. The initial characters of the pattern and the input string must be identical in the exact same order, and the pattern's length is definitive for the matching process (that is, if the pattern is longer than the input string, the match will fail).

Example: matching / non-matching patterns for the input string 'exam'

For the input string 'exam',

  • the following patterns will match:

    • 'ex' (the pattern contains the initial characters of the input string in the exact same order)

    • 'exam' (the pattern is an exact match for the input string)

  • the following patterns will not match:

    • 'example' (the pattern is longer than the input string)

    • 'hexameter' (the pattern's initial characters do not match the input string's characters in the exact same order, and the pattern is longer than the input string)

store-matches

Stores the matches of the regular expression into the $0, ... $255 variables. The $0 stores the entire match, $1 is the first group of the match (parentheses), and so on. Named matches (also called named subpatterns), for example, (?<name>...), are stored as well. Matches from the last filter expression can be referenced in regular expressions.

NOTE: To convert match variables into a syslog-ng OSE list, use the $* macro, which can be further manipulated using List manipulation, or turned into a list in type-aware destinations.

substring

The given literal string will match when the pattern is found within the input. Unlike flags("prefix"), the pattern does not have to be identical with the given literal string.

Perl Compatible Regular Expressions (PCRE)

Starting with syslog-ng OSE version 3.1, PCRE expressions are supported on every platform. If the type() parameter is not specified, syslog-ng OSE uses PCRE regular expressions by default.

The following example shows the structure of PCRE-style regular expressions in use.

Example: Using PCRE regular expressions

rewrite r_rewrite_subst {
    subst("a*", "?", value("MESSAGE") flags("utf8" "global"));
};

PCRE-style regular expressions have the following flags() options:

disable-jit

Switches off the just-in-time compilation function for PCRE regular expressions.

dupnames

Allows using duplicate names for named subpatterns.

Configuration example:

filter { match("(?<DN>foo)|(?<DN>bar)" value(MSG) flags(store-matches, dupnames)); };
...
destination { file(/dev/stdout template("$DN\n")); };

global

Usable only in rewrite rules, flags("global") matches for every occurrence of the expression, not only the first one.

ignore-case

Disables case-sensitivity.

newline

When configured, it changes the newline definition used in PCRE regular expressions to accept either of the following:

  • a single carriage-return

  • linefeed

  • the sequence carriage-return and linefeed (\r, \n and \r\n, respectively)

This newline definition is used when the circumflex and dollar patterns (\^ and $) are matched against an input. By default, PCRE interprets the linefeed character as indicating the end of a line. It does not affect the \r, \n or \R characters used in patterns.

store-matches

Stores the matches of the regular expression into the $0, ... $255 variables. The $0 stores the entire match, $1 is the first group of the match (parentheses), and so on. Named matches (also called named subpatterns), for example, (?<name>...), are stored as well. Matches from the last filter expression can be referenced in regular expressions.

NOTE: To convert match variables into a syslog-ng OSE list, use the $* macro, which can be further manipulated using List manipulation, or turned into a list in type-aware destinations.

unicode

Uses Unicode support for UTF-8 matches: UTF-8 character sequences are handled as single characters.

utf8

An alias for the unicode flag.

Glob patterns without regular expression support

There are no supported flags() options for glob patterns without regular expression support.

patterns()

Synopsis: patterns("pattern1" "pattern2")
Mandatory: yes

Description: The regular expression patterns that you want to find a match. regexp-parser() supports multiple patterns, and stops the processing at the first successful match.

prefix()

Synopsis: prefix()

Description: Insert a prefix before the name part of the parsed name-value pairs to help further processing. For example:

  • To insert the my-parsed-data. prefix, use the prefix(my-parsed-data.) option.

  • To refer to a particular data that has a prefix, use the prefix in the name of the macro, for example, ${my-parsed-data.name}.

  • If you forward the parsed messages using the IETF-syslog protocol, you can insert all the parsed data into the SDATA part of the message using the prefix(.SDATA.my-parsed-data.) option.

Names starting with a dot (for example, .example) are reserved for use by syslog-ng OSE. If you use such a macro name as the name of a parsed value, it will attempt to replace the original value of the macro (note that only soft macros can be overwritten, see Hard versus soft macros. To avoid such problems, use a prefix when naming the parsed values, for example, prefix(my-parsed-data.)

This parser does not have a default prefix. To configure a custom prefix, use the following format:

parser p_regexp{
    regexp-parser(
        patterns( ... )
        prefix("myprefix.")
    );
};

template()

Synopsis: template("${<macroname>}")

Description: The macro that contains the part of the message that the parser will process. It can also be a macro created by a previous parser of the log path. By default, the parser processes the entire message (${MESSAGE}).

Updated: