Most modern software development projects involve multiple developers, operations engineers, QA testers, and managers — and they all require access to the project infrastructure. However, enabling access to modify infrastructure leads to a higher risk of security breaches and legal disputes. Because of this, organizations rely on audit logs to address possible security concerns by recording precise actions in, and changes to, their infrastructure.

In this article, we will define audit logs and explore their use cases in infrastructure. We’ll look specifically at cloud and database activity. We’ll also consider legal compliance, and common challenges with data volumes, log retention periods, and correlations across different systems that require careful evaluation.

What are audit logs?

Audit logs are a collection of records of internal activity relating to an information system. To be useful, these records must contain the following information:

• Which component was affected by an action?
• Who performed this action?
• When was this action executed?
• What kind of action was executed?

Audit logs differ from application logs and system logs. Application logs record activity performed by external users, and system logs record activity performed by software, such as operating systems. However, audit logs are exclusively concerned with activities performed by internal users and services on system infrastructure.

Immutability is an important aspect of audit logs. No one should be allowed to modify audit log records, as that would diminish the integrity of the logs and render them useless.

Use cases

Besides being convenient and useful, audit logs serve an important purpose for security and legal compliance, particularly when it comes to cloud infrastructure and databases.

Tracking changes to cloud infrastructure

Cloud-native projects often comprise numerous cloud services. The attractiveness of the public cloud hinges on its infrastructure elasticity, meaning the scale and configurations of these cloud services are very dynamic. Because of this, organizations that use cloud providers to deploy and manage their software need the capability to track each change made within their infrastructure, no matter how insignificant the change may appear. In addition, malicious activity on cloud infrastructure must be detectable to establish the exact actor, time, and nature of malicious actions.

Many cloud providers provide built-in audit logging services. For example:

• CloudTrail for AWS
• Cloud Audit Logs for GCP
• Azure Monitor for Microsoft Azure

Tracking database administration and system activity

Backend infrastructure depends on databases. Although developers may consider databases deployed in the cloud as a part of cloud infrastructure, database audit logs focus more deeply on actions performed on the database system. For example, database audit logs report on when clients connect and disconnect and the reasons for those actions. This information is important for establishing potential misuse and distortion of data.

Audit logs are also essential for tracking who makes alterations to a database schema, along with changes to schema components that affect the format, data structure, and record updates. Some databases are schema-less, so audit logs may be less significant in those cases. However, for relational databases, audit logs are crucial.

The most commonly used database systems provide audit logging plugins and extensions. For example:

• PgAudit for PostgreSQL
• Audit Logging for MySQL
• Auditing for MongoDB

Compliance and legal defense

Certain industries rigorously regulate audit logging levels. For example, a company in the Fintech or Medtech space must be prepared to implement audit logging at a comprehensive level, often requiring regular log analysis to confirm no malicious activity over a specified period. In addition to industry regulations, specific certifications impose high-level audit logging mandates. For example:

• HIPAA compliance certification for organizations in the healthcare industry.
• ISO 27001 certification for organizations in the IT industry.

Some companies must present information security compliance certifications to their partners and clients. As a result, audit logs provide proof of meeting regulation and certification requirements.

Audit logs are also beneficial when resolving legal disputes. Disputes arise more often than expected, especially in sensitive industries like finance and medicine, so audit logs help companies by serving as crucial evidence.

The challenges of audit logging

Although audit logs play a critical role in security and compliance, implementing them can be challenging. Let’s consider some potential issues related to storing audit logs.

Data volume

Software projects are often sophisticated systems consisting of dynamic components. The sheer amount of data in audit logs can be enormous and very costly to store. For this reason, organizations must carefully consider which logs are most important to store, based on compliance requirements, security posture, and overall system functionality.

Log retention period

While some regulations and certifications specifically require audit logging, companies must retain these logs for prolonged periods in order to satisfy compliance demands. For example, one requirement for HIPAA compliance certification is that a company retain its audit logs for six years. For ISO 27001 certification, companies must store their audit logs for at least three years.

Retaining logs for long periods of time incurs financial costs and also requires resources for maintenance and management.

Correlating audit logs across different systems

Correlating, comparing, and analyzing audit logs across cloud and database vendors for different log formats and protocols can be strenuous. Dealing with large volumes of logs requires smoothing out differences, which requires significant time and effort. Additionally, discrepancies are common at the level of the details, with the possibility of causing compliance complications.

Discover the world’s leading AI-native platform for next-gen SIEM and log management

Elevate your cybersecurity with the CrowdStrike Falcon^® platform, the premier AI-native platform for SIEM and log management. Experience security logging at a petabyte scale, choosing between cloud-native or self-hosted deployment options. Log your data with a powerful, index-free architecture, without bottlenecks, allowing threat hunting with over 1 PB of data ingestion per day. Ensure real-time search capabilities to outpace adversaries, achieving sub-second latency for complex queries. Benefit from 360-degree visibility, consolidating data to break down silos and enabling security, IT, and DevOps teams to hunt threats, monitor performance, and ensure compliance seamlessly across 3 billion events in less than 1 second.