Collect Cisco Umbrella IP logs
This document explains how to ingest Cisco Umbrella IP logs into Google Security Operations using Amazon S3.
Cisco Umbrella is a cloud-delivered security service that provides the first line of defense against threats on the internet. The IP (Cloud Firewall) logs capture network traffic handled by Umbrella network tunnels, including source and destination IP addresses, ports, protocols, actions taken, and associated identities. These logs are exported as gzipped CSV files to Amazon S3 every 10 minutes.
Before you begin
Make sure you have the following prerequisites:
- A Google SecOps instance
- Privileged access to Cisco Umbrella console (Admin role)
- Privileged access to AWS (S3, IAM)
Configure Cisco Umbrella log export to S3
To configure Cisco Umbrella to export IP logs to S3, do the following:
- Sign in to the Umbrella dashboard at https://dashboard.umbrella.com
- Go to Admin > Log Management.
- In the Log Management page, locate the Amazon S3 section.
- Click Add to configure S3 export.
In the Amazon S3 Configuration dialog, select one of the following options:
- Cisco-managed S3 Bucket: Cisco creates and manages the S3 bucket for you. Cisco provides AWS credentials to access the bucket.
- Customer-managed S3 Bucket: You provide your own S3 bucket details and AWS credentials.
For this guide, select Cisco-managed S3 Bucket (recommended for easier setup).
In the Log Types section, select the checkbox for IP Logs (Cloud Firewall logs).
You can also select additional log types if needed:
- DNS Logs
- Proxy Logs
- Intrusion Prevention Logs
- Cloud Data Loss Prevention Logs
In the Data Center dropdown, select the AWS region closest to your location, or select All Data Centers to export logs from all regions.
Click Save.
After saving, Umbrella displays the AWS Credentials section with the following information:
- S3 Bucket Name (for example,
umbrella-logs-1234567890) - S3 Path (for example,
2024-01-15/) - AWS Access Key ID
- AWS Secret Access Key
- AWS Region
- S3 Bucket Name (for example,
Click Download Credentials to save these credentials as a CSV file for future reference.
Configure AWS S3 bucket and IAM for Google SecOps
If you selected Cisco-managed S3 Bucket in the previous section, Cisco has already created the S3 bucket and provided AWS credentials. You can skip to the "Configure a feed in Google SecOps" section below.
If you selected Customer-managed S3 Bucket, follow these steps:
- Create an Amazon S3 bucket by following this user guide: Creating a bucket.
- Save bucket Name and Region for future reference (for example,
umbrella-logs-custom). - Create a User by following this user guide: Creating an IAM user.
- Select the created User.
- Select Security credentials tab.
- Click Create Access Key in the Access Keys section.
- Select Third-party service as Use case.
- Click Next.
- Optional: Add a description tag.
- Click Create access key.
- Click Download .csv file to save the Access Key and Secret Access Key for future reference.
- Click Done.
- Select Permissions tab.
- Click Add permissions in Permissions policies section.
- Select Add permissions.
- Select Attach policies directly.
- Search for the AmazonS3FullAccess policy.
- Select the policy.
- Click Next.
- Click Add permissions.
Configure Cisco Umbrella customer-managed S3 bucket connection
If you selected Cisco-managed S3 Bucket, skip this section.
If you selected Customer-managed S3 Bucket, follow these steps:
- From the Log Management page (continuing from the "Configure Cisco Umbrella log export to S3" section), you should be on the Amazon S3 Configuration dialog.
In the Customer-managed S3 Bucket section, enter the following:
- S3 Bucket Name: Enter the bucket name you created (for example,
umbrella-logs-custom). - S3 Path (optional): Enter a prefix for organizing logs (for example,
umbrella-ip-logs/). - AWS Access Key ID: Enter the access key from step 11 of the AWS configuration.
- AWS Secret Access Key: Enter the secret key from step 11 of the AWS configuration.
- AWS Region: Select the region matching your S3 bucket from the dropdown.
- S3 Bucket Name: Enter the bucket name you created (for example,
In the Log Types section, select the checkbox for IP Logs (Cloud Firewall logs).
Click the Test Connection button.
Wait for the test to complete. A green checkmark with the "Connection successful" message should appear.
Click Save.
On the Log Management page, verify that the Amazon S3 section shows Status: Active and Last Export shows a recent timestamp.
Configure a feed in Google SecOps to ingest Cisco Umbrella IP logs
- Go to SIEM Settings > Feeds.
- Click Add New Feed.
- On the next page, click Configure a single feed.
- Enter a unique name for the Feed name (for example,
Umbrella IP Logs). - Select Amazon S3 V2 as the Source type.
- Select Umbrella IP as the Log type.
- Click Next and then click Submit.
Specify values for the following fields:
S3 URI: Enter the S3 URI in the format:
s3://<bucket-name>/<path>/- For Cisco-managed bucket: Use the S3 Bucket Name and S3 Path from step 11 of "Configure Cisco Umbrella log export to S3" (for example,
s3://umbrella-logs-1234567890/2024-01-15/). - For customer-managed bucket: Use your bucket name and path (for example,
s3://umbrella-logs-custom/umbrella-ip-logs/).
- For Cisco-managed bucket: Use the S3 Bucket Name and S3 Path from step 11 of "Configure Cisco Umbrella log export to S3" (for example,
Source deletion option: Select Do not delete transferred files (recommended to preserve logs in S3).
Maximum File Age: Include files modified in the last number of days (default is 180 days).
Access Key ID: Enter the AWS Access Key ID from step 11 of "Configure Cisco Umbrella log export to S3" (for Cisco-managed bucket) or step 11 of AWS configuration (for customer-managed bucket).
Secret Access Key: Enter the AWS Secret Access Key from step 11 of "Configure Cisco Umbrella log export to S3" (for Cisco-managed bucket) or step 11 of AWS configuration (for customer-managed bucket).
Asset namespace: The asset namespace.
Ingestion labels: The label to be applied to the events from this feed (for example,
UMBRELLA_IP).
Click Next and then click Submit.
UDM mapping table
| Log Field | UDM Mapping | Logic |
|---|---|---|
| query_name | about.labels | Merged from query_name_label (key: DNS Lookup Type, value: query_name) |
| response_code | additional.fields | Merged from dns_return_message (key: dns_return_message, value: response_code) |
| application | intermediary | Merged from intermediary.application |
| categories | metadata.description | Set to value of categories if present, else "DNS request and response were made." |
| metadata.event_type | Set to NETWORK_CONNECTION for IP logs, NETWORK_HTTP if destip and internal/external present, STATUS_UPDATE if principal present but no target, GENERIC_EVENT otherwise, or NETWORK_DNS for DNS logs | |
| metadata.product_name | Set to "Cisco Umbrella" for proxy logs, "Cisco Umbrella IP Layer Enforcement" for IP logs | |
| metadata.vendor_name | Set to "Cisco" for proxy logs, "Cisco Systems, Inc" for IP logs | |
| appProto, url | network.application_protocol | Extracted from url using grok pattern, uppercased, or set to DNS for DNS logs |
| question | network.dns.questions | Merged from question (name from domain, type from query_type) |
| response_code | network.dns.response_code | Converted from response_code using enum mapping to integer |
| requestMethod | network.http.method | Value from requestMethod, uppercased |
| userAgent | network.http.parsed_user_agent | Converted from userAgent to parseduseragent |
| referer | network.http.referral_url | Value copied directly |
| statusCode | network.http.response_code | Converted to integer |
| userAgent | network.http.user_agent | Value copied directly |
| network.ip_protocol | Set to "TCP" | |
| responseSize | network.received_bytes | Converted to uinteger |
| requestSize | network.sent_bytes | Converted to uinteger |
| identity | principal.hostname | Value copied directly |
| internal_ip, external_ip, source_ip | principal.ip | Value from internal_ip if valid IP, else external_ip if different from internalIp and valid IP, else source_ip |
| source_port | principal.port | Converted to integer |
| sha | security_result.about.file.sha256 | Value copied directly |
| sec_action, action, security_result_action | security_result.action | Set to BLOCK if categories not empty, or ALLOW/BLOCK based on action, or ALLOW/BLOCK based on security_result_action |
| sec_category, category, security_category | security_result.category | Set to ACL_VIOLATION if categories is "Unauthorized IP Tunnel Access", SOFTWARE_MALICIOUS if "Malware", NETWORK_SUSPICIOUS otherwise, or NETWORK_CATEGORIZED_CONTENT, or NETWORK_MALICIOUS/NETWORK_SUSPICIOUS based on categories in DNS |
| categories | security_result.category_details | Transformed to array from categories |
| responseBodySize, avDetections, puas, ampDisposition, ampMalware, ampScore, certificateErrors, destinationListID, isolateAction, fileAction, warnstatus, dlpstatus, contentType, verdict, rulesetID | security_result.detection_fields | Merged from various labels created from each field (key: field name, value: field value) |
| ruleID | security_result.rule_id | Value copied directly |
| verdict, contentType, dlpstatus | security_result.summary | Set to "Traffic %{verdict}" if verdict in allowed/blocked, "Traffic %{contentType}" if contentType present, or "Traffic %{dlpstatus}" if dlpstatus in allowed/blocked, or "Traffic blocked - %{blockedCategories}" if blocked |
| blockedCategories | security_result.threat_name | Value copied directly |
| temp_filename | target.file.names | Merged from temp_filename (value of fileName) |
| destination_ip | target.ip | Value copied directly |
| destination_port | target.port | Converted to integer |
| url | target.url | Value copied directly |
Need more help? Get answers from Community members and Google SecOps professionals.