![Helping Non-Security Stakeholders Understand ATT&CK in 10 Minutes or Less [VIDEO]](https://assets.crowdstrike.com/is/image/crowdstrikeinc/video-ATTCK2-1)
![Qatar’s Commercial Bank Chooses CrowdStrike Falcon®: A Partnership Based on Trust [VIDEO]](https://assets.crowdstrike.com/is/image/crowdstrikeinc/Edward-Gonam-Qatar-Blog2-1)
![Endpoint Protection and Threat Intelligence: The Way Forward [VIDEO]](https://assets.crowdstrike.com/is/image/crowdstrikeinc/GK-Blog_Images-1)
This blog was originally published January 24, 2022 on humio.com. Humio is a CrowdStrike Company.
In this blog, we’ll show you, step by step, how to download stock data and then upload it to Humio. You can then search that data and build a dashboard for fast insights. Subsequent blog posts will expand on this dashboard and show you how to move from analyzing historical data to live data. To get started, you’ll need to accessHumio Community Edition, which is available at no cost.
Step 1: Download stock price data
After you’ve created an account for Humio Community Edition, you’ll need a dataset to analyze. While the possibilities are endless, for this blog post, we’ll use the Python script below to download stock prices for the past week. Copy the script below and save it where you can execute a Python script.#!/usr/bin/Python3
from datetime import datetime
import yfinance as yf
import json
import sys
# Take ticker symbols from the command line arguments
tickers = sys.argv<1:>
stockData = <>
for ticker in tickers:
data = yf.download(tickers=ticker,period="7d", interval = "1m", progress=False)
data.index = data.index.astype(str) #convert from DateTime to string
jdata = data.to_dict(orient='index')
for item in jdata:
#add ticker and timestamp to flatten data
jdata.update( {"ticker":ticker} )
jdata.update( {"MyTimestamp":item} )
stockData.append(jdata)
for item in stockData:
print(json.dumps(item))$ pip3 install yfinance
You can execute this script like this:
$ stocks.py CRWD > crwd.json
Essentially, this command takes stock ticker symbols as command line arguments and prints the price data for the past week JSON format. The command above gets the price data for CrowdStrike, and then saves it to a file called ‘crwd.json.’
Check the data from the command line to verify it worked:
$ tail crwd.json
{"Open": 268.989990234375, "High": 269.17999267578125, "Low": 268.9599914550781, "Close": 269.1700134277344, "Adj Close": 269.1700134277344, "Volume": 81464, "ticker": "CRWD", "timestamp": "2021-11-01 15:59:00-04:00"}
{"Open": 269.1700134277344, "High": 269.1700134277344, "Low": 269.1700134277344, "Close": 269.1700134277344, "Adj Close": 269.1700134277344, "Volume": 0, "ticker": "CRWD", "timestamp": "2021-11-01 16:00:00-04:00"}Step 2: Parse the data
Humio does not require data to be parsed, and it can easily allow you to store and search unstructured data. But for the sake of the exercise, we want to be able to graph this data, so it’s best to start by parsing the various fields. Because the data is in JSON format, this should be fairly straightforward. While in Humio Community Edition, click into your repository (you named this when you signed up) and then clickParsers
at the very top menu and then the
+ New Parser
button. Name this new one “stocks” and then click
Create Parser. For this parser, we’re copying the default JSON parser with minor modifications:
parseJson() | findTimestamp(field=MyTimestamp, timezone="America/New_York")
This tells the parser to read the field labels from JSON and where to find the timestamp.
Step 3: Create an ingest token
The next step is to prepare Humio to accept this data. We need to create an ingest token and assign it to the Parser we just created. From the top menu, selectSettings. Then, from the left-side menu, click
Ingest Tokens. Click the
+ Add Token
button. Set the
Token Name
to “StocksJSON” and then from the
Assigned Parser
drop-down, select the “stocks” parser you just created. From the Ingest Tokens page, click the eye icon next to your newly created token to reveal the token key. Click the
copy
button.
Step 4: Send the data to Humio
Now that we have our ingest token, we can ship our stock data to Humio. From the command line, run the following curl command, but paste in your ingest token from above.curl https://cloud.community.humio.com/api/v1/ingest/hec/raw -X POST -H "Authorization: Bearer TOKEN" -T "crwd.json"
Step 5: Verify the data
You can quickly see if any data has arrived from your Settings page by selectingData Sources
from the left-side menu. It should look just like this.
We can see the data has been collected here. But let’s explore it directly and start working with it.
Step 6: Explore the data
From the top menu, selectSearch. You should see data from today, but click the time picker on the top and select “Last 7 days.”
(Click to enlarge)ticker. From the field pop-up box, you’ll see the CRWD row. From there, click the `=`
button.
The search bar at the top will now read:
ticker = CRWD
Since we only have one stock, this doesn’t change our returned dataset in any way. But assuming you may add additional ticker datasets, we’ll include it now. Similarly, if you are working with separate datasets, you may want to include Type, which will have the name of the parser used. You can select it from the menu just like we did with ticker or manually edit the Search to read as shown below. You can type the following query as one line, but for legibility, I’ve used two. You can use shift-enter while typing in the Search bar to move to the next line (and then press enter to execute it).
ticker = CRWD
| #type = stocks
If this is your first dataset imported, nothing will have changed with these search filters, but if you add additional data sources later, it will ensure you’re still focused on this one.
Step 7: High/low queries
Now, let’s start calculating some interesting stats from our data. To find the week’s high, run the following search:ticker = CRWD
| #type = stocks
| max(High)
To stylize this result, use the drop-down in the upper left corner to selectGauge. Now that we have a data point of interest, let’s save it to a dashboard. On the right side of your screen, find and click the
Save as…
button. Select
Dashboard Widget. Fill out the dialogue box to add this to the “Stock” dashboard you created previously, then add a
Widget Title and click
Save. This will bring you to your new dashboard. For the sake of symmetry, let’s round out this dashboard with the week’s low as well. Return to the
Search
page and enter this query.
ticker = CRWD
| #type = stocks
| min(Low)
Once again clickSave as…
and then
Dashboard Widget
and name this appropriately. It should default to selecting the Dashboard you already created. Your dashboard should now look like this:
Step 8: Graph the stock price
Now let’s graph this stock’s price over the week. To do this, we will use the timeChart function, which allows us to bucket up the data we’re analyzing and apply a function to it. For simplicity’s sake, we will use one day as the bucket span. Since we’re interested in the close price at the end of the data, the function we’ll use is SelectLast to choose the latest value in that bucket. Enter the following query and ensure that you change the time picker to the Last (7d) seven days.#type=stocks |ticker="CRWD"
| timeChart(series=ticker, function=SelectLast(Close), span=15m)
Our graph looks like this:
If you mouse over any data point you can view the data for that day.
Save this widget to your new dashboard!
We named this widget “Price Chart.” In this case, we uncheckedOpen dashboard after save
and then clicked
Save. This leaves me at the same search screen. But given the setup here, quickly edit to extend our analysis here by simply changing “Close” to “Volume” in the query window. Save this as a widget named “Volume Chart.” Your dashboard should now look just like this:
Additionally, we can do some extra work to enhance the visibility and aesthetics. In the upper right corner, click the pencil icon. Now you can rearrange your widgets. Start by dragging the Price Chart widget to the top. Then drag the edge to the right to resize it across the page. Drag the Volume widget up to below the price one, and resize it to the right edge as well. The price is far more interesting, so drag the Price widget down to enlarge it, and resize the Volume widget to be shorter.
Finally, move the High and Low widgets to the bottom and resize them to be a bit smaller.
Looking at the dashboard now, it’s quite functional, but let’s bring more contrast between the price and volume data, aside from their relative size. In the upper right of the Price widget, click the three vertical dots and thenEdit Style. From the right side panel, scroll to the bottom and expand the
Series
menu. In the
Field
text box, enter “CRWD.” Click the next box (which by default says AUTO) to open the color selector. I chose red. Then, to highlight the value differences better, find the
Min Value
field and put something under the Low value we discovered earlier. Finally, click the
Save
button. Here’s the final dashboard.
One final thing to check: When you save a query to a dashboard widget, it will keep the time frame you used in the original query. You can use this to have different widgets looking at different time frames. If you wish to have an overriding time frame, click the slider button next toShared Time
and choose the time window you want. It will then be applied to all widgets on the page. Thank you for checking out Humio Community Edition! In our next post we’ll extend the dashboard and make it flexible enough to handle multiple stocks. See you then.
