How to use Extract log labels in GCP notifications

 If You're looking to enrich your Cloud Logging alerts with specific information extracted from your log entries. This is a powerful feature that makes your notifications much more informative and actionable.

There are two main ways to extract log labels for use in notifications in GCP:

1. Using Log-based Alerts (the simpler and more direct method for most cases):

   This is the modern approach and generally preferred for creating alerts directly from log patterns. It allows you to define "label extractors" directly within the alert policy.

2. Using Log-based Metrics with Labels (for more complex aggregations or dashboarding):

   This method involves creating a log-based metric first, defining labels on that metric, and then creating an alert policy based on the metric.1 While effective, it adds an extra layer of abstraction.

Let's focus on Method 1: Using Log-based Alerts as it's most often used.

Method 1: Using Log-based Alerts to Extract Labels

This method allows you to define a query for your logs and then, as part of the alert creation, specify which fields from the matching log entries you want to extract as labels. These extracted labels can then be referenced in your notification's documentation field.

Steps:

1. Go to Logs Explorer:

   • In the Google Cloud Console, navigate to Logging > Logs Explorer.

2. Build Your Log Query:

   • Craft a query that precisely matches the log entries you want to alert on. For example, if you want to be alerted on Cloud Armor DENY events with body_denied_by_security_policy:

     resource.type="http_load_balancer"
     jsonPayload.enforcedSecurityPolicy.outcome="DENY"
     jsonPayload.enforcedSecurityPolicy.statusDetails="denied_by_security_policy"
     

   • Run the query to ensure it returns the expected logs.

3. Create Alert from Query:

   • In the Query results toolbar, click Actions (three vertical dots) and select Create alert.²

4. Configure Alert Details:

   • Alert name: Give your alert a meaningful name (e.g., "Cloud Armor Body Deny Alert").

   • Severity: Choose an appropriate severity level.

5. Choose Logs to Include (your query will be pre-filled):

   • Review the pre-filled "Log query" and "Logs query preview".

6. Extract Log Labels (This is the key step!):

   • Under "Logs query preview", you'll see a section called "Extract log labels (Optional)". Click on this.

   • Click Add label.

   • Display Name: This is the name you'll use to reference the label in your notification. Choose something descriptive (e.g., rule_id, source_ip, request_path).

   • Log field name: This is the path to the field in your log entry that contains the value you want to extract.

     • For example, if you want the specific Cloud Armor rule ID that triggered the block, you'd typically use jsonPayload.enforcedSecurityPolicy.preconfiguredExprIds[0] (assuming you want the first ID if there are multiple).³

     • For the source IP, it might be jsonPayload.remoteIp.

     • For the request path, it could be httpRequest.requestUrl.

     • Tip: To find the exact field names, expand a relevant log entry in Logs Explorer and click "Expand nested fields" to see the full JSON structure.

   • Regular Expression (Optional but often useful): If the field contains more than just the value you need, you can use a regular expression with a capture group () to extract a specific part.⁴ If the field's entire content is what you want, you can often leave this blank or use (.*).

     • For preconfiguredExprIds[0], you might just use (.*) if the field just contains the ID. If it's a list, you might need to iterate or extract one. For simplicity, if it's typically a single ID, just referencing the field might work.

   • Add multiple labels for all the relevant pieces of information you want in your alert.

7. Set Notification Frequency and Autoclose:

   • Configure how often you want to be notified and when incidents should automatically close.

8. Specify Notification Channels:

   • Choose the notification channels (e.g., Email, SMS, Slack, PagerDuty) where you want to receive the alerts. You'll need to have these channels pre-configured in Cloud Monitoring.

9. Add Documentation (where you use your extracted labels!):

   • This is the section where you can customize the content of your notification.

   • Use Markdown formatting and reference your extracted labels using the syntax ${log.extracted_label.YOUR_DISPLAY_NAME}.

   • Example Documentation:

     Markdown

     ### Cloud Armor Body Deny Alert
     
     A request was denied by Cloud Armor due to a suspicious pattern in the request body.
     
     **Details:**
     * **Rule ID:** ${log.extracted_label.rule_id}
     * **Source IP:** ${log.extracted_label.source_ip}
     * **Request URL:** ${log.extracted_label.request_url}
     * **Severity:** ${log.severity}
     
     Please investigate the traffic from this IP address.
     

     • Note on available variables: You can use both your custom extracted labels (${log.extracted_label.YOUR_DISPLAY_NAME}) and standard log entry fields (${log.severity}, ${log.resource.type}, etc.) in the documentation.

10. Create Policy:

    • Review all settings and click Create Policy.

Key Considerations:

• RE2 Regular Expression Syntax: Cloud Logging uses RE2 syntax for regular expressions. Make sure your expressions are valid.

• Case Sensitivity: Field names in log entries are case-sensitive.

• Arrays: If a field is an array (like preconfiguredExprIds often is), you might need to reference a specific element (e.g., preconfiguredExprIds[0]) or use more advanced regex if you want to concatenate or process all elements. For a simple body_denied_by_security_policy scenario, often the first element is sufficient.

• Testing: Always test your alerts thoroughly by generating matching log entries to ensure the labels are extracted correctly and the notifications arrive as expected with the correct information.

• Limits: There are limits on the number of extracted labels you can define per log-based alert.

By following these steps, you can create highly informative Cloud Logging alerts that leverage extracted log labels to provide immediate context and aid in faster incident response.