I have objects being stacked up inside AWS S3. I want to process each object, exactly once in a time sequence order. (If A object was created before B object, A should be processed before B)
I tried to use aws_s3_bucket_notification with AWS SQS, but using it, it seems hard to achieve exactly once in a time sequence. The maximum message number that can be pulled from SQS is limited, and I'm afraid of this scenario. In S3, objects were created in A -> B -> C -> D sequence. However, they arrive at SQS in A -> D -> B -> C sequence. If the message number I can pull at once is 3, I get to pull A -> D -> B (although I sort them in time sequence when I pull). Thus, I'll not be able to put C in front of D.
Any solutions please?
与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…