Issue
We have a step function that generates a pdf when someone clicks on a button. However, the PDFs can sometimes be large (200+ pages) so this process might take some time. The PDF is generated inside of a step function. How can I tell when that instance of a step function has finished? I have a lambda which calls the step function too, I am not calling the step function directly from Angular.
I am thinking either create a SNS topic and subscribe the user to that topic, and have the step function post a message to the topic. The issue arises where it is a user-specific thing so I would either need to make a topic for each PDF created, or use SNS filters on the subscriber. Is using SNS like this against best practices?
Another thought was to use AWS waiters in the AWS SDK 3.
My question is: Is it OK to use SNS in that way, or would waiters be better. OR is there another better way to know when a step function finishes?
Solution
In principle you are right.. The task may take some time and the it's not always feasible that the client synchronously wait for the response.
As far I undetstood you have a problem how to get information back that the task is completed.
The common approaches I've seen are
create a table (db, dynamodb,..) with the task state and update the state upon the task completion. The client application needs to poll regularly for the result/state
create an api gateway and it is possible to use websockets to notify the client (push information back to the client)
Why not SNS - you would need to create /clean up the topics and subscriptions. Topics and subscriptions are to be more or less static (it may take se time to be created or deleted) and there's a limit on their count. Simply SNS is not intended to be used this way.
AWS SDK Waiters are for waiting for the resource state (e. g. sns topic created), not really for the tasks.
In theory the client could interact directly with the stepfn (invoke, check state,..), but I'm not fond of the web clients accessing any cloud resources directly. It creates some extra dependency and more caution is needed for authorisation.
Answered By - gusto2
0 comments:
Post a Comment
Note: Only a member of this blog may post a comment.