Log Types

There are mainly two types of Apache web server logs: the access logs and the error logs.

Access Logs

Access logs contain information about client requests processed by the web server. Information in each log entry includes:

• Client IP address
• Time of the request
• The resource requested
• The user agent string identifying the requester’s OS and browser
• HTTP response code returned
• The number of bytes returned to the client

The following snippet shows a sample access log:

116.35.41.41 - - [21/May/2022:11:22:39 +0000] "GET / HTTP/1.1" 403 199691 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.4 Safari/605.1.15"
116.35.41.41 - - [21/May/2022:11:22:41 +0000] "GET /icons/poweredby.png HTTP/1.1" 200 643 "http://34.227.9.153/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.4 Safari/605.1.15"

Error Logs

Error logs contain information about any internal errors encountered when the Apache web server starts or runs, as well as errors raised when processing a client request. The following is a typical example of an error log entry:

[Sat May 21 11:26:28.733503 2022] [autoindex:error] [pid 13075:tid 139684223059712] [client 116.15.141.41:57782] AH01276: Cannot serve directory /var/www/html/: No matching DirectoryIndex (index.html) found, and server-generated directory index forbidden by Options directive

Log Locations

The location of log files for the Apache web server can vary based on the operating system. For example, the default location on a CentOS/RedHat system is the

/var/log/httpd directory, whereas in an Ubuntu system, the location is typically

/var/log/apache2.

You can change the default log location by setting certain parameters in the Apache configuration file. For example, to change the error log location on a CentOS system, you can edit the

/etc/httpd/conf/httpd.conffile and set the

ErrorLog directive like this:

ErrorLog "/var/log/httpd2/error_log"

When you change the log location, you need to make sure the new directory exists on the server and the Apache process user has the permissions to read from and write to that directory. The changes take effect once you restart the Apache web service. 

Similarly, you can change the access log location by setting the

CustomLog directive:

CustomLog "/var/log/httpd2/access_log" common

Log Levels

The log level configuration dictates the type of messages logged in the error log. You can set this value to any of the following values:

• debug
  debug (lowest level)
• info
  info
• notice
  notice
• warn
  warn
• error
  error
• crit
  crit
• alert
  alert
• emerg
  emerg

A value between

trace1 to

trace8 (highest level)

The lower the log level, the more verbose log entries are. The

warn level is the default log level setting, but you can change it by setting the

LogLevel directive in the Apache configuration file to a different value.

Let’s see how changing the log level affects the verbosity. The following entries are recorded when the

LogLevel is set to

warn:

[Sat May 21 12:40:54.624596 2022] [core:info] [pid 15124:tid 139735416445248] AH00096: removed PID file /etc/httpd/run/httpd.pid (pid=15124)
[Sat May 21 12:40:54.624633 2022] [mpm_event:notice] [pid 15124:tid 139735416445248] AH00492: caught SIGWINCH, shutting down gracefully
[Sat May 21 12:40:55.679086 2022] [core:notice] [pid 15362:tid 140527999998272] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Sat May 21 12:40:55.679815 2022] [suexec:notice] [pid 15362:tid 140527999998272] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
…

After changing the

LogLevel to

info, we restart Apache and clear the old log entries. The resulting error log looks like this:

…
[Sat May 21 12:45:09.179408 2022] [suexec:notice] [pid 15612:tid 140629236357440] AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
[Sat May 21 12:45:09.196807 2022] [lbmethod_heartbeat:notice] [pid 15612:tid 140629236357440] AH02282: No slotmem from mod_heartmonitor
[Sat May 21 12:45:09.197368 2022] [http2:info] [pid 15612:tid 140629236357440] AH03090: mod_http2 (v1.15.7-git, feats=CHPRIO+SHA256+INVHD+DWINS, nghttp2 1.33.0), initializing...
[Sat May 21 12:45:09.197397 2022] [http2:warn] [pid 15612:tid 140629236357440] AH02951: mod_ssl does not seem to be enabled
[Sat May 21 12:45:09.197448 2022] [proxy_http2:info] [pid 15612:tid 140629236357440] AH03349: mod_proxy_http2 (v1.15.7-git, nghttp2 1.33.0), initializing...
…

As you can see, the error log now has some entries of type

http2:infowhich were not there before. That’s because the server is now also logging the

info type messages.

Log Format

By default, the Apache web server access log uses a

combined log format. You can see the field definition for the log format from the Apache configuration file (

httpd.conf or

apache2.conf). The following snippet, for example, shows the

LogFormat directive in a CentOS server:

LogFormat "%h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined

The

common log format is defined as:

LogFormat "%h %l %u %t \"%r\" %>s %O" common

As you can see, the

combined log format is similar to the

common log format, except it has two extra fields:

Referrer and

User-Agent. A

% symbol precedes the field names. Some of the important fields in the access log are:

Using these basic field definitions, you can easily identify the key information from the access log. Let’s consider the access log snippet below.

116.35.41.41 - - [21/May/2022:11:22:39 +0000] "GET / HTTP/1.1" 403 199691 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.4 Safari/605.1.15"
116.35.41.41 - - [21/May/2022:11:22:41 +0000] "GET /icons/poweredby.png HTTP/1.1" 200 643 "http://34.227.9.153/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/15.4 Safari/605.1.15"

You can see the first entry shows the client IP address is

116.35.41.41.. The client was using the Safari browser running on Mac OS X. The HTTP request method was

GET, and the web server responded with a status code of

403 (forbidden). In other words, it refused the client’s request. The total number of bytes transferred to the client was 199691.

You can also change the log format by specifying the value of the

CustomLog directive in the Apache configuration file. In the snippet below, we change the log format to

common:

CustomLog "/var/log/httpd2/access_log" common

After setting the

CustomLog directive and restarting the Apache web server, you can see the access log now contains some important fields like timestamp, level of the message, process id, and a message:

[Sat May 21 12:45:08.082210 2022] [mpm_event:notice] [pid 15362:tid 140527999998272] AH00492: caught SIGWINCH, shutting down gracefully
[Sat May 21 12:45:09.178590 2022] [core:notice] [pid 15612:tid 140629236357440] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0

If you wish to change the error log format, you can add an

ErrorLogFormat directive in the Apache configuration file. The following snippet shows an example:

ErrorLogFormat "[%t] [%l] [pid %P] %F: %M"

The following table shows some of the important fields in the error log:

Restarting the Apache web server service will show error messages like the following:

[Sun May 22 07:06:15 2022] [notice] [pid 19297] event.c(3127): [client AH00492: caught SIGWINCH, shutting down gracefully
[Sun May 22 07:06:16.191416 2022] [core:notice] [pid 19535:tid 140251660302656] SELinux policy enabled; httpd running as context system_u:system_r:httpd_t:s0
[Sun May 22 07:06:16 2022] [notice] [pid 19535] mod_suexec.c(101): AH01232: suEXEC mechanism enabled (wrapper: /usr/sbin/suexec)
…

Configuring Apache Logging For VirtualHosts

When you have multiple websites running on the same Apache web server, it’s best to set a separate log file location for each VirtualHost. This makes log management and error triage a lot easier. You can troubleshoot issues with a particular VirtualHost by searching through its log file only.

The snippet below shows how to set up a log file location for a VirtualHost in the Apache configuration file:

<VirtualHost *:80>
    ServerName www.test.com
    ServerAlias test.com
    DocumentRoot /var/www/html/test.com
    ErrorLog /var/log/httpd/test.com/error_log
    CustomLog /var/log/httpd/test.com/access_log combined
</VirtualHost>

Here, we configure the VirtualHost

www.test.com and specify a custom location for its access and error logs. 

You can also override server-level log directives at the VirtualHost level. For example, if you set up a

LogLevel of

warn at the overall Apache web server level, you can set a different

LogLevel for a particular VirtualHost, as shown below:

<VirtualHost *:80>
    ServerName www.test.com
    ServerAlias test.com
    DocumentRoot /var/www/html/test.com
    ErrorLog /var/log/httpd/test.com/error_log
    CustomLog /var/log/httpd/test.com/access_log combined
    LogLevel debug
</VirtualHost>

With this configuration, the log level for

www.test.com will be

debug instead of the default

warn.

Log Rotation

Log rotation is a log management technique in which log files older than a specified time or larger than a specific size are deleted, moved, renamed, or compressed. Without log rotation, the same log file continues to be used. Over time, your web server may run out of disk space. Also, since log files grow very large over time, this creates performance bottlenecks when reading from or writing to those log files.

You can use utilities like the Linux

logrotate to configure log rotation for Apache web server. It’s also advisable to set proper log retention policies so older log files can be deleted or moved to a different location. In a production system with multiple Apache web servers running, it’s best to send all the logs to a central log management system.

Setting up log rotation With logrotate

Both CentOS and Ubuntu ship with the

logrotate package. In order to set up Apache web server log rotation in RHEL-based systems, you can create a file called

/etc/logrotate.d/httpd.conf with the following settings:

/var/log/httpd/* {
    daily
    rotate 7
    size 100M
    compress
    delaycompress
}

This configuration will rotate all files under the

/var/log/httpd directory (in Debian-based systems like Ubuntu, change this to

/var/log/apache2) every day. Apache will rotate the log files in this directory when they reach 100MB in size and keep only the last seven rotated logs. All rotated log files will also be automatically compressed.

Log your data with CrowdStrike Falcon Next-Gen SIEM

Elevate your cybersecurity with the CrowdStrike Falcon^® platform, the premier AI-native platform for SIEM and log management. Experience security logging at a petabyte scale, choosing between cloud-native or self-hosted deployment options. Log your data with a powerful, index-free architecture, without bottlenecks, allowing threat hunting with over 1 PB of data ingestion per day. Ensure real-time search capabilities to outpace adversaries, achieving sub-second latency for complex queries. Benefit from 360-degree visibility, consolidating data to break down silos and enabling security, IT, and DevOps teams to hunt threats, monitor performance, and ensure compliance seamlessly across 3 billion events in less than 1 second.

Schedule Falcon Next-Gen SIEM Demo