amazon web services - AWS event driven approach - Cloud Watch vs S3 event notification

Question

I am building an event-driven system, which starts as soon as a new file lands S3. I am evaluating different ways of achieving that and using Cloud Watch Rule + API Trail is an option. This is the Cloud Watch Event pattern as it is:

    {
  "source": [
    "aws.s3"
  ],
  "detail-type": [
    "AWS API Call via CloudTrail"
  ],
  "detail": {
    "eventSource": [
      "s3.amazonaws.com"
    ],
    "eventName": [
      "PutObject"
    ],
    "requestParameters": {
      "bucketName": [
        "mysupertest88"
      ]
    }
  }
}

Like that, it triggers the rule for every file landing into the bucket but trying to filter by key and wildcard does not work:

"requestParameters": {
      "bucketName": [
        "mysupertest88"
      ],
      "key": ["myprefix/mysecondprefix/*"]
    }

It works just if I specify a key with matches without a wildcard, I think because the symbol '*' is a valid char in S3 objects. An alternative is to filter directly at Trail level:

but I do not see that as a nice option, as API Trail is often out of developer's control. An additional alternative is to use content-filtering: (nice new feature, but you have to create the rule through EventBridge)

    {
  "source": [
    "aws.s3"
  ],
  "detail-type": [
    "AWS API Call via CloudTrail"
  ],
  "detail": {
    "eventSource": [
      "s3.amazonaws.com"
    ],
    "eventName": [
      "PutObject"
    ],
    "requestParameters": {
      "bucketName": [
        "mysupertest88"
      ],
      "key": [
        {
          "prefix": "a/c"
        }
      ]
    }
  }
}

Last S3 event notification is the old way to accomplish this? What is your experience with that? Any pro and cons that are not easy to catch without experience?

question from:https://stackoverflow.com/questions/65641437/aws-event-driven-approach-cloud-watch-vs-s3-event-notification

Answer 1

Since your objective is to start an action "as soon as a new file lands S3", CT may not satisfy your requirement. This is because it can take up 15 minutes for delivery of the API events. From AWS faq:

Typically, CloudTrail delivers an event within 15 minutes of the API call.

In contrast, S3 events should be faster. From AWS docs:

Amazon S3 event notifications are designed to be delivered at least once. Typically, event notifications are delivered in seconds but can sometimes take a minute or longer.