Common Filtering Scenarios

Attributes

Use the attributes processor to remove specific keys such as credit card information, passwords, and other sensitive values. See Attributes Processor and General Identity Attributes.

Credit Card Information

You can redact values that appear in the credit_card_number attribute:

CODE 
...
processors:
attributes/update:
actions:
- key: credit_card_number
value: redacted
action: update
...
service:
pipelines:
traces:
processors: [..., attributes/update, ...]
metrics:
processors: [..., attributes/update, ...]
logs:
processors: [..., attributes/update, ...]
...
Passwords
You can delete a user.password:
CODE 
...
processors:
attributes/update:
actions:
- key: user.password
action: delete
...
service:
pipelines:
traces:
processors: [..., attributes/update, ...]
metrics:
processors: [..., attributes/update, ...]
logs:
processors: [..., attributes/update, ...]
...

Redaction

Use the redaction processor to remove all attributes except description, group, id, and name. See Redaction Processor.

CODE 
...
processors:
redaction/update:
allow_all_keys: false
allowed_keys:
- description
- group
- id
- name
...
service:
pipelines:
traces:
processors: [..., redaction/update, ...]
metrics:
processors: [..., redaction/update, ...]
logs:
processors: [..., redaction/update, ...]
...
Note: The redaction processor includes a summary configuration that provides details on what is filtered.

Use the redaction processor to block the card numbers using regular expression filters for Visa and Master card formats.

JSON 
...
processors:
redaction/update:
allow_all_keys: true
blocked_values:
- "4[0-9]{12}(?:[0-9]{3})?" ## Visa credit card number
- "(5[1-5][0-9]{14})"       ## MasterCard number
summary: debug
...
service:
pipelines:
traces:
processors: [..., redaction/update, ...]
metrics:
processors: [..., redaction/update, ...]
logs:
processors: [..., redaction/update, ...]
...

Transform

Use the transform processor to mask specific values. See Transform Processor.

Create Allowlist for Specific Keys
Use the transform processor to remove all attributes except service.name, service.namespace, cloud.region, process.command_line:
CODE 
...
processors:
    transform/update:
        traces:
            queries:
               - keep_keys(resource.attributes, "service.name", "service.namespace", "cloud.region", "process.command_line")
        metrics:
            queries:
               - keep_keys(resource.attributes, "service.name", "service.namespace", "cloud.region", "process.command_line")
        logs:
            queries:
               - keep_keys(resource.attributes, "service.name", "service.namespace", "cloud.region", "process.command_line")

  ...
service:
    pipelines:
        traces:
            processors: [..., transform/update, ...] 
        metrics:
            processors: [..., transform/update, ...] 
        logs:
            processors: [..., transform/update, ...]
...
Mask a Partial Value for a Specific Key
Use the transform processor to redact a password that appears on the command line. For example, $env password=mysecret username=myusername python run-my-app.py.
CODE 
...
processors:
transform/update:
traces:
queries:
- replace_pattern(resource.attributes["process.command_line"], "password\\=[^\\s]*(\\s?)", "password=***")
metrics:
queries:
- replace_pattern(resource.attributes["process.command_line"], "password\\=[^\\s]*(\\s?)", "password=***")
logs:
queries:
- replace_pattern(resource.attributes["process.command_line"], "password\\=[^\\s]*(\\s?)", "password=***")
...
service:
pipelines:
traces:
processors: [..., transform/update, ...]
metrics:
processors: [..., transform/update, ...]
logs:
processors: [..., transform/update, ...]
...