This blog was originally published Sept. 17, 2020 on humio.com. Humio is a CrowdStrike Company.

Collecting and monitoring Microsoft Office 365 logs is an important means of detecting indicators of compromise, such as the mass deletion or download of files.

However, exporting logs to a log management platform involves running an Elastic Stack with Logstash, which requires significant configuration and infrastructure. In response to customer requests, we’ve discovered an easier way to connect Office 365 logs to Humio with minimal infrastructure and configuration: the O365beat log shipper.

O365beat is an open source log shipper used to collect Office 365 audit logs from the Office 365 Management Activity API and forward them to Humio. To set it up, follow these instructions from the official O365beat documentation.

Before you begin

To perform the setup required to grant Humio permission to collect Office 365 logs, you need access to the following:

• A Microsoft Office 365 account with administrative privileges
• A Microsoft Azure account with administrative privileges

You cannot complete this procedure without these administrative privileges.

You must also have a system available to install O365beat on. Installers are available for both Windows and Linux.

Register a new Office 365 web application

To get started collecting Office 365 logs, register an Office 365 web application:

1. Log into the Office 365 portal as an Active Directory tenant administrator.
2. Click Show all to expand the left navigation area, and then click Azure Active Directory.
3. Select App Registrations, and then click + New application registration.
4. Provide the following information in the fields:
   1. Name – for example, o365humio.
   2. Select Single tenant for supported account types.
   3. Leave the Redirect URI blank.
5. Click Register and note the Application (client) ID.

Set up Active Directory security permissions

The Active Directory security permissions allow the application you created to read threat intelligence data and activity reports for your organization.

To set up Active Directory permissions:

1. On the main panel under the new application, click API Permissions, and then click + Add a permission.
2. Locate and click on Office 365 Management APIs.
3. In Application permissions, expand and select ActivityFeed.Read, ActivityFeed.ReadDlp, ActivityReports.Read, and ServiceHealth.Read
4. Ensure all necessary permissions are selected, and then click Add permissions.
5. Click Grant admin consent, and then click Accept to confirm.
6. On the left navigation area, select Certificates & secrets, and then click + New client secret.
7. Type a key Description and set the duration to Never.
8. Click Add.
9. Click Overview to return to the application summary, and then click the link under Managed application in local directory.
10. Click Properties, and then note the Object ID associated with the application.

Install O365beat on a Windows host

1. Download the latest version of O365beat.
2. Create a C:\Program Files\O365beat directory.
3. Extract the contents of the .zip file into the directory you created.
4. Open a PowerShell prompt as an Administrator.
5. Navigate to the O365beat directory PS C:\Users\Administrator>cd 'c:\Program Files\O365beat'
6. Run the O365beat installation script PS C:\Program Files\O365beat> .\install-service-o365beat.ps1. If script execution is disabled on the system you will need to enable it for the current session using the following command: powershell.exe -executionpolicy unrestricted -file .\install-service-winlogbeat.ps1.

Configure O365beat log collection

The O365beat package includes a configuration file that contains all the necessary credential information to connect to the audit logs for your tenancy. The default configuration file o365beat.yml pulls this information from your environment.

The following example shows a typical O365beat log collection configuration:

############################# O365beat ######################################
## pull secrets from environment (e.g, > set -a; . ./ENV_FILE; set +a;)
## or a key store (https://www.elastic.co/guide/en/beats/filebeat/current/keystore.html)
## or hard-code here:
tenant_domain: "$O365_TENANT_DOMAIN.onmicrosoft.com"
client_secret: "$O365_CLIENT_SECRET"
client_id: "$O365_CLIENT_ID" # aka application id (GUID)
directory_id: "$O365_DIRECTORY_ID" # aka tenant id (GUID)
registry_file_path: ${O365BEAT_REGISTRY_PATH:./o365beat.state}
## the following content types will be pulled from the API
## for available types, see https://docs.microsoft.com/en-us/office/office-365-management-api/office-365-management-activity-api-reference#working-with-the-office-365-management-activity-api
content_types:
- Audit.AzureActiveDirectory
- Audit.Exchange
- Audit.SharePoint
- Audit.General

In the o365beat.yml configuration file, provide the following information from the Office 365 web application you created:

• tenant_domain
• client_secret
• client_id
• directory_id

Note: The registry_file_path is where O365beat stores its state. The default path does not need to be changed.

Configuring O365beat output to Humio

Data can be sent to Humio by configuring O365beat to use the built-in ElasticSearch output.

The following example shows a O365beat output configuration that sends data to Humio’s cloud:

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["https://cloud-es.us.humio.com:443"]
# Optional protocol and basic auth credentials.
#protocol: "https"
#username: "elastic"
password: "$HUMIO_INGEST_TOKEN"
compression_level: 5
bulk_max_size: 200
worker: 5

You must make the following changes to the sample configuration:

• Insert the URL containing the Humio host in the hosts: field in the ElasticSearch output. For example, https://$YOUR_HUMIO_URL:443 where $YOUR_HUMIO_URL is the URL for your Humio Cloud installation. Note that the URL specifies the repository that Humio sends events to. It is important to specify the port number in the URL, otherwise O365beat defaults to using 9200. If you’re using Humio’s US cloud, the ElasticSearch interface is available at https://cloud-es.us.humio.com:443. The usage of port 9200 is not supported for Humio’s US cloud.
• Insert an ingest token from the repository as the password. Set the username to anything — it will get logged in the access log of any proxy on the path so using the hostname of the sender is a good option.
• You may want to increase the number of worker instances (worker) from the default of 1 to 5 or 10 to achieve more throughput if O365beat is not able to keep up with the inputs. To get higher throughput, also increase queue.mem.events to 32000 to allow buffering for more workers.

Run

1. Verify that your o365beat.yml file is valid using the following command in PowerShell: PS C:\Program Files\o365beat> .\o365beat.exe test config -c .\o365beat.yml -e
2. Start O365beat using the following command: PS C:\Program Files\o365beat> Start-Service o365beat. In the future you can start and stop the O365beat service using the Windows Services Control Panel.

State is maintained in the registry_file_path location. The default working directory is o365beat.state. To prevent repeat downloads, this file contains a timestamp representing the creation date of the last content blob retrieved.

Final Note

Log collection may not immediately start when your configuration is complete. Per Microsoft’s documentation, it can take up to 12 hours for logs to appear in Humio.

Once you’ve started to collect Office 365 logs into Humio, get value from them immediately by building dashboards or setting up security alerts.

Learn more about how to set up over 2000 threat detection rules in Humio, including rules for Office 365, by watching our SOC Prime Workshop.