How to Setup Shape Log Analysis in Fastly

Update 8/3: Shape Log Analysis is now a supported log streaming endpoint on Fastly. Read the full details here.

Shape Log Analysis is a non-invasive technique used to analyze HTTP and application logs for a clearer view into attackers that are bypassing current security measures. Oftentimes bad actors, botnets, and drive by attacks will consume system resources and commit fraud against APIs in the form of Credential Stuffing, Scraping, Account Takeover and more. Without the proper defenses in place, these attacks are a pain to stop for most security teams who are forced to play “whack-a-mole" with solutions that are not built to permanently defeat fraudulent and automated attacks.

Shape Security has a unique corpus of data from attacks that have been identified and blocked over the years for the world's largest banks, airlines, hotels and many other types of infrastructure exposed to the public internet. This anonymized attack data is used to examine application logs revealing automation and fraud that is bypassing perimeter security mechanisms and making its way to your origin servers.

Through analyzing data points in Layer 7 traffic, Shape will create a threat assessment on old and new campaigns that are currently attacking specific parts of your applications.

Log Analysis Example - Figure 1

The visualization shown in Figure 1 represents all malicious and fraudulent traffic against a specific application. The green pattern hidden in the back is the normal diurnal flow of legitimate user traffic. All other colors are automated attacks driving abuse of APIs and important parts of the application.

This type of reporting can be used to not only understand types of attacks and abuse but can also be used to create a plan for integrating a mitigation solution.

Types of attacks that will be uncovered:

Credential Stuffing
Account Takeover
Scraping
API Abuse
System Resource Consumption

Getting Started

Shape Log Analysis is a free service that is now integrated with Fastly CDN. To avoid complications of compressing, securing and manually sending log data to Shape, we now have the ability to securely send logs to Shape through Fastly's real time log streaming configuration. This is a simple “flip of the switch” configuration, doesn't involve sending any PII data to Shape, and gives organizations the visibility required to take action and prevent these types of attacks.

To configure Fastly CDN for Shape Log Analysis, follow these steps:

1) Request a secure S3 Bucket from Shape (send an email to fastly@f5.com with title "Fastly Log Streaming Setup")

Once Shape has setup your designated S3 bucket, you will receive an email with a private access key that will be required to complete the configuration in the next step. Keep in mind that Shape uses network and security access controls between Fastly and AWS to ensure data is kept private and confidential. If there are any concerns around how log data is kept safe and secure, please ask in the setup request email.

2) Follow Fastly’s well written instructions on creating a new log endpoint and copy in the Shape specific configuration from below.

Log format for Shape Log Analysis (Non-PII data)

{
  "timestamp": "%{begin:%Y-%m-%dT%H:%M:%S%z}t",
  "ts": "%{time.start.sec}V",
  "id.orig_h": "%h", "status_code": "%>s",
  "method": "%m",
  "host": "%{Host}i",
  "uri": "%U%q",
  "accept_encoding": "%{Accept-Encoding}i",
  "request_body_len": "%{req.body_bytes_read}V",
  "response_body_len": "%{resp.body_bytes_written}V",
  "location": "%{Location}i",
  "x_forwarded_for": "%{X-Forwarded-For}i",
  "user_agent": "%{User-Agent}i",
  "referer": "%{Referer}i",
  "accept": "%{Accept}i",
  "accept_language": "%{Accept-Language}i",
  "content_type": "%{Content-Type}o",
  "geo_city": "%{client.geo.city}V",
  "geo_country_code": "%{client.geo.country_code}V",
  "is_tls": %{if(req.is_ssl, "true", "false")}V,
  "tls_version": "%{tls.client.protocol}V",
  "tls_cipher_request": "%{tls.client.cipher}V",
  "tls_cipher_req_hash": "%{tls.client.ciphers_sha}V",
  "tls_extension_identifiers_hash": "%{tls.client.tlsexts_sha}V"
}

S3 Bucket Details

When you receive the S3 Bucket confirmation from the fastly@f5.com email address, it will contain the following 5 items that you'll need to insert into your Fastly configuration.

1.) Bucket Name

2.) Access Key

3.) Secret Key

4.) Path

5.) Domain

Click on "Advanced options" and add the following:

After completing the setup, your configuration summary for Shape Log Analysis will look like the following:

Once the Fastly logging configuration is complete, logs will be sent to Shape's secure S3 bucket for analysis. Typically we collect around two weeks worth of log data to provide a comprehensive analysis of attack traffic.

Additionally, an F5 or Shape representative will be available to provide support during the logging setup and a Threat Assessment Report will be provided as part of the service.