Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Welcome To Ask or Share your Answers For Others

Categories

0 votes
383 views
in Technique[技术] by (71.8m points)

amazon s3 - How to handle S3 objects in time sequence order and only exactly once?

I have objects being stacked up inside AWS S3. I want to process each object, exactly once in a time sequence order. (If A object was created before B object, A should be processed before B)

I tried to use aws_s3_bucket_notification with AWS SQS, but using it, it seems hard to achieve exactly once in a time sequence. The maximum message number that can be pulled from SQS is limited, and I'm afraid of this scenario. In S3, objects were created in A -> B -> C -> D sequence. However, they arrive at SQS in A -> D -> B -> C sequence. If the message number I can pull at once is 3, I get to pull A -> D -> B (although I sort them in time sequence when I pull). Thus, I'll not be able to put C in front of D.

Any solutions please?


与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome To Ask or Share your Answers For Others

1 Answer

0 votes
by (71.8m points)
等待大神答复

与恶龙缠斗过久,自身亦成为恶龙;凝视深渊过久,深渊将回以凝视…
Welcome to OStack Knowledge Sharing Community for programmer and developer-Open, Learning and Share
Click Here to Ask a Question

...