In this series, we’ve introduced you to the basics of syslog, how it works and its key components. We looked at the configuration of syslog using the rsyslog.conf file, how to set up log rotation and the best practices to follow for collecting and analyzing logs.

Now, we’re introducing you to syslog-ng, one of the most feature-rich, well-documented versions of syslog. It can make your logging environment faster, more secure and more reliable. It also integrates with security information and event management (SIEM) and log management solutions.

In this third part, we cover the background of syslog-ng, consider its benefits over traditional versions of syslog and show you how to install and configure it.

What Is Syslog-ng?

Syslog-ng (“syslog new-generation”) facilitates the transmission of source logs to a remote destination using predefined filters. It goes beyond basic syslog functionality by supporting TCP, TLS encryption, advanced filtering and logging to a database. Syslog-ng also allows customization and can facilitate almost any logging need.

Syslog-ng was developed in 1998 by Balázs Scheidler and became the default logging daemon for operating systems like Debian, Gentoo and SUSE. It is now maintained by One identity (after their acquisition of Balabit), which develops both open-source and commercial products under the syslog-ng brand.

Syslog-ng and rsyslog (which we discussed in part two of this series) are both syslog daemons that enhanced the original syslog to provide faster data transmission with wider support across operating systems and programs. While rsyslog was forked from the standard syslogd package, syslog-ng is a totally separate daemon with a new configuration and syntax.

Benefits of Using Syslog-ng

Syslog-ng built upon the traditional versions of syslogd and rsyslog to enhance its features and improve the security, ease of use and documentation for its users. Let’s consider some of the specific enhancements.

Support for TCP and TLS encryption

Syslog-ng overcame one of the biggest drawbacks of traditional syslog by introducing support for TCP and TLS encryption. TCP gives you a more reliable log transfer than UDP and ensures log messages reach their destination. Some logs contain highly sensitive information, so TLS encryption ensures your logs are safe during transmission.

Simpler Configuration

Syslog-ng developed a new configuration format that most people find easier to understand than syslogd or rsyslog. The configuration file is clean, well structured and gives you the ability to reuse sources, destinations or filters throughout the configuration file, making it simpler to maintain.

Ability to Classify, Tag and Correlate Log Messages in Real Time

Syslog-ng can scan the content and parameters of incoming logs in real time. This makes it easy to apply complex filters that direct logs to different destinations or drop logs that are unimportant to reduce noise in the logging system.

Syslog-ng can also enrich logs by adding data from an external lookup file or by correlating incoming logs with a common field such as hostname or program that generated the log. This correlation can make analysis easier and enhance business insights.

Wide Support

Syslog-ng supports a wide range of message formats, including RFC3164, RFC5424, JSON and Journald. It can also run on multiple operating systems and architecture, including Linux, Unix, BSD and Solaris. The high level of adoption across the Linux community gives users access to more forums, articles and documentation with use cases to improve their usage. This also improves the likelihood that syslog-ng will continue to be used well into the future.

Message Flow Control Across the Network

From a networking level, syslog-ng can send and receive messages in IPv4 and IPv6 environments.

Network connectivity issues or performance degradation can prevent logs from being accepted by the destination server. With syslogd, these messages would likely have been lost. Syslog-ng flow control overcomes this, as it will pause the reading of messages until it has determined that it can send them to the destination successfully, acting as a buffer to maintain high log delivery rates.

Configuring Syslog-ng

In this section, we show you how to install syslog-ng and configure the syslog-ng conf file.

Installing Syslog-ng

Depending on your Linux distribution, you can use a package manager to install syslog-ng. For Ubuntu, Mint or other Debian operating systems, you can use apt.

Configuring Syslog-ng.conf

The syslog-ng.conf file contains configuration for the source, destination, filter and log statements to determine how to deal with incoming and outgoing log messages. You’ll find the configuration file at /etc/syslog-ng/syslog-ng.conf. Open a text editor to edit this file.

Source

The source configures how syslog-ng receives log messages. Each source contains one or more source drivers which define how and where we receive messages.

source <identifier> {
source-driver(parameter); source-driver(parameter); ...
};

We define the source once and give it an <identifier>, and then we can reference this <identifier> throughout the rest of the configuration file.

The source-driver defines how and where syslog-ng receives messages. The table below is from the syslog-ng documentation and shows the available source drivers.

Example

This example source statement (s_tcp) tells the syslog server to listen on TCP port 9999.

source s_tcp {
network (
transport("tcp")
port(9999)
);
};

Destination

The destination is where syslog-ng directs log messages. It follows the same syntax as the source block.

destination <identifier> {
destination-driver(params); destination-driver(params); ...
};

Example 1

destination d_router_logs {
file("/var/log/network/router/routerlogs.log");
};

You can also use variables to create dynamic file names, splitting them out logically by date, time, host or other factors. In the example above, all logs, regardless of their source host or log time, are sent to the file routerlogs.log. In the following example, we use variables to send logs to a dynamic file location depending on the source host and date.

Example 2

destination d_router_logs {
file("/var/log/network/router/$HOST/$YEAR-$MONTH-$DAY.log");
};

The table below shows the available syslog-ng destination drivers. You can find out more information about each destination driver in the syslog-ng documentation.

Filter

Filters are optional and define criteria for how syslog-ng will route log messages to different destinations. You can create simple filters for things like host or severity, or you could add more complex logic with Boolean expressions.

filter <identifier> {
<filter_type>("<filter_expression>");
};

Example 1

The example below declares a filter, f_router, and matches any logs with the text “allow” from host 10.1.2.3.

filter f_router {
host("10.1.2.3") and match("allow" value("MESSAGE"));
};

The table below shows the available syslog-ng filters.

Log Path

Log paths are found at the end of the configuration file, defining the connections between the source, destination, and any filters. To define a log path, we can add a log statement based on the syntax shown below. Log statements are processed in order, so you must position them correctly to ensure your logs end up at the right destination.

log {
source(s1); source(s2); ...
optional_element(filter1|parser1|rewrite1);
optional_element(filter2|parser2|rewrite2);
...
destination(d1); destination(d2); ...
flags(flag1[, flag2...]);
};

Example 1

The following log statement sends messages arriving to localhost on port 514 to a remote server on port 9997. Only messages containing error are included.

source s_localhost {
network(
ip(127.0.0.1)
port(514)
);
};
destination d_tcp {
network("10.1.2.3"
port(9997)
);
};
filter f_error {
host("example") and match("error" value("MESSAGE"));
};
log {
source(s_localhost);
filter(f_error);
destination(d_tcp);
};

When declaring the source, destination and filters, the common convention is to prefix the name with s_, d_, or f_. This makes it easy to distinguish references later in the log statement.

Log your data with CrowdStrike Falcon Next-Gen SIEM

Elevate your cybersecurity with the CrowdStrike Falcon^® platform, the premier AI-native platform for SIEM and log management. Experience security logging at a petabyte scale, choosing between cloud-native or self-hosted deployment options. Log your data with a powerful, index-free architecture, without bottlenecks, allowing threat hunting with over 1 PB of data ingestion per day. Ensure real-time search capabilities to outpace adversaries, achieving sub-second latency for complex queries. Benefit from 360-degree visibility, consolidating data to break down silos and enabling security, IT, and DevOps teams to hunt threats, monitor performance, and ensure compliance seamlessly across 3 billion events in less than 1 second.

Schedule Falcon Next-Gen SIEM Demo