Skip to main content
Single Pipeline Approach
Workflow - Single Pipeline Approach
- Single Pipeline Approach is the procedural scripting.
- Every workflow has only one (1) task of
main
function which is registered and started in the celery
main
function
- Registered as a task
main(self, task_name="main", *args, **kwargs) -> (status:str, err_msgs:list[str], data:object)
- the function intended to define the sequence of the functions for execution.
- has the access to credentials
- receives the payload
- must verify that the payload provided is correct: right types and values, all other functions assume that payload is correct
other
functions
- assume the payload is correct
- other arguments are not guaranteed to be valid
- Every
other(self, task_name, *args, **kwargs) -> (status:str, err_msgs:list[str], data:object)
- Create of the new entities MUST be done through platforms tasks
- API calls and other operations that may have systematic execution errors MUST use try_retry decorator
- Errors must be collected through ALL execution of the steps in main and be handled to Notifications.