In Part Two of our series, we explored the customization of Apache web server logging by using logging modules and conditional logging. We also covered the use of centralized logging to increase the ease of management and features available, using the collected log data for enhanced analysis and visualization.

In this post, we’ll explore advanced logging options for the Apache web server. We’ll look at how you can redirect and control the formatting of your web server logs to meet logging specifications. Then, we’ll look at how to set up log rotation to ensure your critical applications continue running. Finally, we’ll cover some common logging transports.

Redirecting Apache Logs

A typical installation of an Apache web server will write log files to the disk by default, capturing both the error log and the access log for the server.

Error log

On a Debian-based operating system, the error log is located at /var/log/apache2/error.log. In this file, you’ll find all the error messages that are generated by the Apache web server process. These messages are generated at runtime when triggered by a request made to a web server endpoint.

Access log

On a Debian-based operating system, the access log is located at /var/log/apache2/access.log. This log file contains an audit record of all the requests made to the server. This can be useful for generating metrics or auditing your web server traffic.

For Rhel-based operating systems, these log files can be found within the /var/log/httpdfolder.

Updating log file locations

To update the location of these two default log files mentioned above, you would edit the log file output directives in your web server config file, found at /etc/httpd/conf/httpd.conf. For example:

ErrorLog /var/log/custom/dir/error.log
CustomLog /var/log/custom/dir/access.log common

Don’t forget to restart Apache in order to apply these changes. After restarting the server, you’ll begin to see log messages populate these new files.

Separating log files by a virtual host

A much more maintainable Apache web server configuration uses virtual hosts to group traffic. Separating log messages by traffic grouping—with each group sending its logs into its own log files—is extremely helpful. Separate log data files can be formatted or processed with individual requirements per virtual host. Below is the configuration of two virtual hosts, along with  separate log files for the messages generated from each host.

<VirtualHost *:80>
ServerName www.site-foo.com
CustomLog /var/log/custom/dir/site-foo/access.log common
ErrorLog /var/log/custom/dir/site-foo/error.log
</VirtualHost>
<VirtualHost *:80>
ServerName www.site-bar.com
CustomLog /var/log/custom/dir/site-bar/access.log common
ErrorLog /var/log/custom/dir/site-bar/error.log
</VirtualHost>

A common pitfall with log files

When making configuration updates to use a custom log file location, you might notice the following error as you restart the Apache web server process:

[Fri Sep 28 03:58:13 2007] [notice] caught SIGTERM, shutting down
(2)No such file or directory: custom: could not open error log file
/var/log/custom/dir/error_log.
Unable to open logs

This issue can occur if the new path provided in your configuration contains folders that do not exist or that the web server process cannot access. So, when you want to write to log files in a custom folder, ensure that the full path to the folder exists and that a log file with sufficient read and write permissions for the Apache web server user is present.

Customizing Apache Log Formats

An Apache web server can serve traffic to multiple internal or external services. A different team or organization may be responsible for its operations or log data generated by the traffic to the services endpoint with each of these services. These teams may have formatting requirements necessary to achieving their log-data goals, whether alerting, analyzing, or debugging for developers.

To customize the log format, edit the httpd.conf configuration file mentioned above, located at /etc/httpd or /usr/local/apache2/conf folder, depending on your operating system. Then, using the CustomLog directive, specify a file path and a format for the log messages.

CustomLog path/to/log/file common

The commonformat in the above configuration is predefined in the Apache web server and is the default for the web server access log. The log format contains the following properties:

• Client's IP address
• User making the request (if not an anonymous request)
• Timestamp for when the request commenced
• Request method and URL
• HTTP status code
• Total bytes sent in the response

The following is an example of an access log message in this format:

127.0.0.1 - Joe [10/Jan/2022:12:00:00 -0000] "GET /hello HTTP/1.1" 200 1024

Suppose you have custom log format requirements, or a predefined log format does not exist. You can define your custom log format by specifying a string of log formatting directives separated by spaces or tabs, like the following:

CustomLog /var/log/httpd/custom.log "%h %m %U %>s"

For any custom logging formats that you’d like to use across your Apache web server configuration, view the documentation for the LogFormat directive, which allows you to declare a template string (as in the above example) and save it to a variable name that you can reuse.

Setting Up Log Rotation

To maintain the stable operation of your Apache web server, it is essential to set up simple utilities on your host to keep it clean and operating normally. Monitor the quantity of data generated by your log files as web server requests are made. Since the log files grow over time, ensure that the size of each log file does not get out of hand. Left unchecked, a disk can fill, leaving no space to read and write new files required for stable operation.

To avoid errors, use the logrotate utility to easily configure and schedule jobs to clean up log files on your system, including those created by the Apache web server. Logrotate monitors the size of log files and backs up their contents into new files when they exceed a predefined size. This prevents a runaway of stored log data as your web server traffic increases.

For a standard installation of logrotate, you can find the configuration file at /etc/logrotate.d/logrotate. Below is a simple configuration file that will rotate Apache web server log files.

/var/log/httpd/*.log {
rotate 5
size 1M
missingok
compress
}

This configuration specifies that the utility will perform the following tasks:

• Monitor the Apache log files in the /var/log/httpd folder.
• Keep only the latest five log files.
• Rotate the log file when its size reaches 1MB.
• Continue without error if the file is missing.
• Compress the backup log files.

With this simple configuration syntax, you can achieve configurable control over what happens to log files as data begins to fill.

Logging Transport Protocols and Methods

Depending on whether you run a more extensive logging infrastructure, your Apache web server may also be required to integrate with other logging transports to push log data where needed. However, some logging transport options will cater to your specific needs better than others.

For these logging transports, weigh the features of your logging infrastructure to select the logging transport that fits best.

Maintaining log data integrity

When log data needs audit and verification, transports such as TCP, HTTP, and syslog employ effective checksum capabilities to ensure data is not lost. However, this process requires computing time and resources to complete, adding time to transporting log data. This could be an issue for high throughput logging, like analytics or high-traffic web services.

Securing log data transporting

Log data can potentially contain private or sensitive information, or you may need to transport log data over an insecure network. In these scenarios, securely transmitting log data using TLS with the TCP or HTTP transports is possible. For UDP connections, encrypt the traffic using DTLS instead. However, keep in mind the cryptographic process of encrypting and decrypting log data will require computing resources from the sender and receiver.

High throughput log data

A UDP transport is ideal in applications where the timely delivery or collection of log data is needed for analysis, alerting, or notification. With UDP’s send-and-forget design, this protocol allows many messages to be sent without a high resource impact on the sender or receiver. However, using a UDP transport can lead to data loss, as there is no validation of sent data at the destination.

Log your data with CrowdStrike Falcon Next-Gen SIEM

Elevate your cybersecurity with the CrowdStrike Falcon^® platform, the premier AI-native platform for SIEM and log management. Experience security logging at a petabyte scale, choosing between cloud-native or self-hosted deployment options. Log your data with a powerful, index-free architecture, without bottlenecks, allowing threat hunting with over 1 PB of data ingestion per day. Ensure real-time search capabilities to outpace adversaries, achieving sub-second latency for complex queries. Benefit from 360-degree visibility, consolidating data to break down silos and enabling security, IT, and DevOps teams to hunt threats, monitor performance, and ensure compliance seamlessly across 3 billion events in less than 1 second.

Schedule Falcon Next-Gen SIEM Demo