Examples of S3 path format

Examples of S3 path formats and how they are resolved based on different token values and time ranges.

Cloud Trail

This example illustrates the generation of S3 paths for Cloud Trail logs, demonstrating how the actual paths are constructed from the provided templates, token values, and time ranges.

  • Template: 
    s3://{bucket_name}/{optional_prefix}/AWSLogs/{account_id}/CloudTrail/{region}/{year}/{month}/{day}
  • String based token values:
    Note: You can enter single values directly in the template instead of defining them as a token.

    • bucket_name: cloudtrail-logs-bucket1

    • optional_prefix: cloudtrail-logs

    • account_id: 123456789012,234567890123

    • region: us-east-1,us-west-1

  • Time based tokens:

    • year: %Y

    • month: %m

    • day: %d

  • Time range:

    • Start time: 28/01/2025 00:00

    • End time: 30/01/2025 00:00

S3 paths resulting from the specified parameters:
  • s3://cloudtrail-logs-bucket1/cloudtrail-logs/AWSLogs/123456789012/CloudTrail/us-west-1/2025/01/28
  • s3://cloudtrail-logs-bucket1/cloudtrail-logs/AWSLogs/123456789012/CloudTrail/us-west-1/2025/01/29
  • s3://cloudtrail-logs-bucket1/cloudtrail-logs/AWSLogs/123456789012/CloudTrail/us-east-1/2025/01/28
  • s3://cloudtrail-logs-bucket1/cloudtrail-logs/AWSLogs/123456789012/CloudTrail/us-east-1/2025/01/29
  • s3://cloudtrail-logs-bucket1/cloudtrail-logs/AWSLogs/234567890123/CloudTrail/us-west-1/2025/01/28
  • s3://cloudtrail-logs-bucket1/cloudtrail-logs/AWSLogs/234567890123/CloudTrail/us-west-1/2025/01/29
  • s3://cloudtrail-logs-bucket1/cloudtrail-logs/AWSLogs/234567890123/CloudTrail/us-east-1/2025/01/28
  • s3://cloudtrail-logs-bucket1/cloudtrail-logs/AWSLogs/234567890123/CloudTrail/us-east-1/2025/01/29

S3 Access Logs

This example illustrates the generation of S3 paths for S3 Access Logs, demonstrating how the actual paths are constructed from the provided templates and partition token values.

  • Template:
    s3://{bucket_name}/{optional_destination_prefix}
  • String based tokens:
    Note: You can enter single values directly in the template instead of defining them as a token.

    • bucket_name: bucket1, bucket2, bucket3, bucket4

    • optional_prefix: logs/

S3 paths resulting from the specified parameters:
  • s3://some-bucket/account_1/2025/01
  • s3://some-bucket/account_1/2025/02
  • s3://some-bucket/account_2/2025/01
  • s3://some-bucket/account_2/2025/02

Cloud Front Access Logs

This example illustrates the generation of S3 paths for Cloud Front Access Logs, demonstrating how the actual paths are constructed from the provided templates, token values, and time ranges.

  • Template: 
    s3://cloudfront/DistributionId={DistributionId}/my_distribution_log_data/year={yyyy}/month={MM}/day={dd}/hour={HH}/logs
  • String based tokens:

    • DistributionId: E1A2B3C4D5E6F7, Z9Y8X7W6V5U4T3

  • Time based tokens:

    • yyyy: %Y

    • MM: %m

    • dd: %d

    • HH: %H

  • Time range:

    • Start time: 28/01/2025 22:00

    • End time: 29/01/2025 02:00

S3 paths resulting from the template and token values:

  • s3://cloudfront/DistributionId=Z9Y8X7W6V5U4T3/my_distribution_log_data/year=2025/month=01/day=28/hour=22/logs

  • s3://cloudfront/DistributionId=Z9Y8X7W6V5U4T3/my_distribution_log_data/year=2025/month=01/day=28/hour=23/logs

  • s3://cloudfront/DistributionId=Z9Y8X7W6V5U4T3/my_distribution_log_data/year=2025/month=01/day=29/hour=00/logs

  • s3://cloudfront/DistributionId=Z9Y8X7W6V5U4T3/my_distribution_log_data/year=2025/month=01/day=29/hour=01/logs

  • s3://cloudfront/DistributionId=E1A2B3C4D5E6F7/my_distribution_log_data/year=2025/month=01/day=28/hour=22/logs

  • s3://cloudfront/DistributionId=E1A2B3C4D5E6F7/my_distribution_log_data/year=2025/month=01/day=28/hour=23/logs

  • s3://cloudfront/DistributionId=E1A2B3C4D5E6F7/my_distribution_log_data/year=2025/month=01/day=29/hour=00/logs

  • s3://cloudfront/DistributionId=E1A2B3C4D5E6F7/my_distribution_log_data/year=2025/month=01/day=29/hour=01/logs

Any custom logs

This example illustrates the generation of S3 paths for custom logs, demonstrating how the actual paths are constructed from the provided templates, partition token values, and time ranges.

  • Template:
    s3://some-bucket/{account}/{year}/{month}
  • String based tokens:
    • account: account_1, account_2
  • Time based tokens:

    • year: %Y

    • month: %m

  • Time range:

    • Start time: 01/01/2025 00:00

    • End time: 01/02/2025 00:00

S3 paths resulting from the template and token values:

  • s3://some-bucket/account_1/2025/01
  • s3://some-bucket/account_1/2025/02
  • s3://some-bucket/account_2/2025/01
  • s3://some-bucket/account_2/2025/02