How to collect Microsoft Office 365 logs with O365beat

This blog was originally published Sept. 17, 2020 on humio.com. Humio is a CrowdStrike Company.

Before you begin

• A Microsoft Office 365 account with administrative privileges
• A Microsoft Azure account with administrative privileges

Register a new Office 365 web application

1. Log into the Office 365 portal as an Active Directory tenant administrator.
2. Click

    

   Show all

    

   to expand the left navigation area, and then click

    

   Azure Active Directory.
3. Select

    

   App Registrations, and then click

    

   + New application registration.
4. Provide the following information in the fields:
   1. Name - for example, o365humio.
   2. Select

       

      Single tenant

       

      for supported account types.
   3. Leave the Redirect URI blank.
5. Click

    

   Register

    

   and note the Application (client) ID.

Set up Active Directory security permissions

1. On the main panel under the new application, click

    

   API Permissions, and then click

    

   + Add a permission.
2. Locate and click on

    

   Office 365 Management APIs.
3. In

    

   Application permissions, expand and select

    

   ActivityFeed.Read,

    

   ActivityFeed.ReadDlp,

    

   ActivityReports.Read, and

    

   ServiceHealth.Read
4. Ensure all necessary permissions are selected, and then click

    

   Add permissions.
5. Click

    

   Grant admin consent, and then click

    

   Accept

    

   to confirm.
6. On the left navigation area, select

    

   Certificates & secrets, and then click

    

   + New client secret.
7. Type a key

    

   Description

    

   and set the duration to

    

   Never.
8. Click

    

   Add.
9. Click

    

   Overview

    

   to return to the application summary, and then click the link under

    

   Managed application in local directory.
10. Click

     

    Properties, and then note the Object ID associated with the application.

Install O365beat on a Windows host

1. Download the

    

   latest version of O365beat.
2. Create a

    

   C:\Program Files\O365beat

    

   directory.
3. Extract the contents of the

    

   .zip

    

   file into the directory you created.
4. Open a PowerShell prompt as an Administrator.
5. Navigate to the O365beat directory

    

   PS C:\Users\Administrator>cd 'c:\Program Files\O365beat'
6. Run the O365beat installation script

    

   PS C:\Program Files\O365beat> .\install-service-o365beat.ps1.

    

   If script execution is disabled on the system you will need to enable it for the current session using the following command:

    

   powershell.exe -executionpolicy unrestricted -file .\install-service-winlogbeat.ps1.

Configure O365beat log collection

############################# O365beat ######################################
## pull secrets from environment (e.g, > set -a; . ./ENV_FILE; set +a;)
## or a key store (https://www.elastic.co/guide/en/beats/filebeat/current/keystore.html)
## or hard-code here:
tenant_domain: "$O365_TENANT_DOMAIN.onmicrosoft.com"
client_secret: "$O365_CLIENT_SECRET"
client_id: "$O365_CLIENT_ID" # aka application id (GUID)
directory_id: "$O365_DIRECTORY_ID" # aka tenant id (GUID)
registry_file_path: ${O365BEAT_REGISTRY_PATH:./o365beat.state}
## the following content types will be pulled from the API
## for available types, see https://docs.microsoft.com/en-us/office/office-365-management-api/office-365-management-activity-api-reference#working-with-the-office-365-management-activity-api
content_types:
- Audit.AzureActiveDirectory
- Audit.Exchange
- Audit.SharePoint
- Audit.General

• tenant_domain
• client_secret
• client_id
• directory_id

Configuring O365beat output to Humio

#-------------------------- Elasticsearch output ------------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: <"https://cloud-es.us.humio.com:443">
# Optional protocol and basic auth credentials.
#protocol: "https"
#username: "elastic"
password: "$HUMIO_INGEST_TOKEN"
compression_level: 5
bulk_max_size: 200
worker: 5

• Insert the URL containing the Humio host in the

   

  hosts:

   

  field in the ElasticSearch output. For example,

   

  https://$YOUR_HUMIO_URL:443

   

  where

   

  $YOUR_HUMIO_URL

   

  is the URL for your Humio Cloud installation. Note that the URL specifies the repository that Humio sends events to. It is important to specify the port number in the URL, otherwise O365beat defaults to using 9200. If you’re using Humio’s US cloud, the ElasticSearch interface is available at

   

  https://cloud-es.us.humio.com:443. The usage of port 9200 is not supported for Humio’s US cloud.
• Insert an

   

  ingest token

   

  from the repository as the password. Set the username to anything — it will get logged in the access log of any proxy on the path so using the hostname of the sender is a good option.
• You may want to increase the number of worker instances (worker) from the default of 1 to 5 or 10 to achieve more throughput if O365beat is not able to keep up with the inputs. To get higher throughput, also increase

   

  queue.mem.events to 32000 to allow buffering for more workers.

Run

1. Verify that your

    

   o365beat.yml

    

   file is valid using the following command in PowerShell:

    

   PS C:\Program Files\o365beat> .\o365beat.exe test config -c .\o365beat.yml -e
2. Start O365beat using the following command:

    

   PS C:\Program Files\o365beat> Start-Service o365beat. In the future you can start and stop the O365beat service using the Windows Services Control Panel.

Final Note