Generally, computer software records information to application logs when an event occurs. Examples can be a web server logging a data transfer, a database instance logging successful backup completion, or a firewall logging unsuccessful connection requests.

IT teams typically use application log data to investigate outages, troubleshoot bugs, or analyze security incidents. Analyzing application logs can help IT teams determine the root cause of incidents.

As more log management systems enter the market, businesses are using application logs for more than troubleshooting. They are using logged events to investigate security incidents, track customer behavior, plan system capacity, and audit regulation compliance.

In this article, we’ll look at what an application log is, the different types of application logs, and how a centralized log management tool can help you extract actionable business insights from those logs. We will also introduce you to Humio, a modern log management solution.

What is an Application Log?

Software applications generate logs when something occurs within (or affects) the application. Usually, applications write the relevant log information to a file on the application's server. To save disk space, some organizations configure applications to send their logs to a network share, a dedicated Syslog server, or a logging application through a webhook. Cloud-based applications can write their logs to the cloud’s logging service.

Some log events relate to internal application states. Examples include:

• Memory leaks
• Non-existent path errors
• Unhandled exceptions in the code
• A system crash

Other types of events relate to external factors the application recognizes. Examples include:

• Disk space warnings
• Server reboots
• Lost network access

Each application is different, so the events logged will depend on how the developers have configured it to record events. Usually, critical events like errors or warnings are logged by default. However, you could also see events relating to successful operations such as data transfer complete, backup successful, or shutting down gracefully.

Some common fields are present in most application logs:

• Timestamp: the date and time of the event, which can be in the server's local time or another time zone like UTC.
• Log Level: the severity of the event. Some example log levels include DEBUG, INFO, WARNING, and CRITICAL.
• Username: the network (or logged-in) user responsible for the event. For some system-related events, this field will be blank.
• Source IP: the client (application process or user) IP address. For local events, this field may be empty.
• Destination IP: the local server IP address.
• Event Message: the event description, usually in short text form.

Application Log Types

Depending on the events logged, there can be different types of application logs.

Access, Authentication, and Authorization Logs

These logs show:

• Who logged in (and logged out)
• Whether the operation was successful
• Who accessed application resources like files or database tables
• Who requested data
• How they moved across the network

These logs are helpful for re-tracing a user’s actions across the network. You can use these logs to troubleshoot access problems, investigate security incidents, or even analyze user behavior.

The snippet below shows an Apache access log:

127.0.0.1 - jbloggs [12/Oct/2021:13:55:36 -0700] "GET /server-status HTTP/1.1" 404 2457

Here, the logged event shows user jbloggs accessed the local web server on October 12, 2021, at 1:55 PM local server time. The user accessed the server-status page under the web server root, but the web server returned a 404 (page not found error). The data transferred was 2457 bytes in size.

Change Logs

Change logs show you the changes made within the application. Example changes include:

• Adding new users
• Changing permissions
• Whitelisting IP addresses
• Changing configuration parameters
• Upgrading software versions

When something in an application stops working, one of the possible root causes can be a configuration change that was recently made. Change logs help identify what was changed, when it was changed, and who made the change. This information can help you address the issue quickly.

The SecOps team can also use change logs to detect security incidents. For example, the team can set up alerts when creating new privileged users or registering a new software library.

The event below shows that Active Directory administrator has created a new user.

A user account was created.
Subject:
Security ID:  DOMAIN-FRadministrator
Account Name:  administrator
Account Domain:  DOMAIN-FR
Logon ID:  0x20f9d
New Account:
Security ID:  DOMAIN-FRJohn.Bloggs
Account Name:  John.Bloggs
Account Domain:  DOMAIN-FR

Error Logs

Error logs provide more details about operations that have gone wrong in the application. Some errors won’t cause performance degradation; however, if left unresolved, they can cause an outage. Monitoring error logs can help identify issues before they become incidents, thereby reducing mean time to recovery (MTTR). Historical analysis can highlight performance or user behavior patterns leading to error conditions.

For example, if users constantly receive timeout errors, this can indicate network latencies, capacity issues, or poorly configured timeout thresholds. Errors from application libraries can mean an incompatibility issue after an upgrade or an unsupported library version.

When error logs reveal unhandled exceptions within the application, this information can help developers improve the system with a patch.

Application errors can happen for many reasons, and that’s why the events in the error log can vary widely. Some commonly logged errors include:

• Disk space warnings
• Impending license expiration
• Application hangs
• Memory exhaustion
• Memory dumps
• … and more.

The code snippet below shows one such error event that was generated due to insufficient disk space.

"EventTime": "2022-06-22T13:48:55.445645-08:00",
"Hostname": "WIN-26101992",
"Message": "There is not enough disk space available to complete this operation",
"Severity": "CRITICAL"

Availability Logs

Availability logs show information about your application’s availability. Examples of availability logs include:

• Cluster logs showing failovers or the addition/removal of nodes
• System logs showing successful data backups
• Operating system logs showing shutdowns and reboots
• Replication logs showing latencies

Availability logs are useful for creating SLA reporting. You can calculate the application's uptime, investigate any unplanned shutdowns or failovers, or confirm expected behavior during maintenance windows. The following snippet shows an example of an availability log entry:

"EventTime": "2022-06-22T13:48:55.445645-08:00",
"Hostname": "WIN-12101992",
"Uptime": "72000",
"LastReboot": "2022-06-21T17:48:55.487655-08:00"
"Severity": "INFO"

Benefits of Using Log Management Solutions

Analyzing application logs directly on the host system can be challenging. Application logs are often verbose, complex, and difficult to correlate. Also, some logs are continuously updated, so keeping track of all the messages can be cumbersome. Finally, distributed applications have many moving parts, and each part has its own log. Correlating all the logs for a specific event can be like searching for a needle in the haystack.

This is where log management solutions can help. A log management solution ingests application logs from all your systems, parses and indexes those logs, and then allows you to run searches across the data. This leads to several benefits.

Faster troubleshooting

Log management solutions make it easy and faster to troubleshoot incidents because all logs are accessible from one easy-to-navigate interface. Operators don’t need to log onto each server individually or use commands like tail and grep to filter results.

Relevant information access

You can use custom log parsing configurations to extract only relevant data from application logs. This can help narrow down data returned in queries and simplify dashboards and charts.

Historical analysis

Log management solutions allow you to retain more data for longer than it’s possible to store in a single server or a network share. As a result, you can perform a better historical analysis of the application data. Such analyses are valuable for trend reporting.

Log correlation

Log management solutions have built-in parsers that recognize common log types. This makes it easy to ingest logs from different applications and correlate them using common fields like username or IP address. Correlating seemingly unrelated events can create context around events that may not be obvious otherwise.

Visualization

Dashboards with graphs, charts, or geolocation maps make it easy to spot trends, anomalies, or spikes in applications.

Alerts

Log management solutions allow you to set up alerts based on predefined criteria. Such criteria include thresholds set on a particular field’s value. The alerts are triggered when logged events show the field’s value has crossed the threshold, notifying relevant teams.

For example, you can set up alerts for fatal or critical errors, suspicious login activities, or high CPU usage. Some logging solutions can trigger an action automatically, such as creating a service desk ticket or running automated remediation steps from a playbook.

Discover the world’s leading AI-native platform for next-gen SIEM and log management

Elevate your cybersecurity with the CrowdStrike Falcon^® platform, the premier AI-native platform for SIEM and log management. Experience security logging at a petabyte scale, choosing between cloud-native or self-hosted deployment options. Log your data with a powerful, index-free architecture, without bottlenecks, allowing threat hunting with over 1 PB of data ingestion per day. Ensure real-time search capabilities to outpace adversaries, achieving sub-second latency for complex queries. Benefit from 360-degree visibility, consolidating data to break down silos and enabling security, IT, and DevOps teams to hunt threats, monitor performance, and ensure compliance seamlessly across 3 billion events in less than 1 second.