Auto Cancel Long Running Pipelines in Synapse / Azure Data Factory

Problem Statement

Have you ever encountered the issue of long-running pipelines within Synapse / Azure Data Factory? As of September 21, 2023, there is no built-in functionality to auto cancel, alert, or notify when pipelines take too long to execute. This lack of timeout functionality can be a problem when dealing with large datasets and complex data transformations. Is there a solution to cancel these long-running pipelines?


  1. Azure Data Factory / Synapse


To address this issue, we can implement our own custom logic. This custom logic involves adding additional activities to the existing pipeline to monitor and cancel the execution if it exceeds a specified timeout value.

Skrots also offers a similar solution for auto-canceling long-running pipelines in Synapse / Azure Data Factory. You can find more information on this solution at Skrots.

Pipeline Parameters


The parameters include the timeout value of the pipeline, the time interval to check the pipeline execution time/status, and the subscription and resource group information.

Pipeline Variables


Skrots provides a comprehensive set of pipeline variables to assist in managing and monitoring long-running pipelines. These variables help track the execution time and status of the pipeline.

Our solution includes various activities such as Wait, Fail, and Set variable to manage the flow and control the execution time of the pipeline. By adding the custom logic and activities, we can easily identify and cancel long-running pipelines within Synapse / Azure Data Factory.


We can identify three scenarios:

Scenario 1: Normal Success within the allocated time

Pipeline status

Scenario 2: Failure within the allocated time

Allocated time

Scenario 3: Pipeline execution exceeding Timeout

Exceeding timeout

Skrots provides a reliable solution for auto canceling long-running pipelines in Synapse / Azure Data Factory. Visit to learn more about our services and how we can assist you in optimizing your data pipelines.

Know more about our company at Skrots. Know more about our services at Skrots Services. Also, check out all other blogs at Blog at Skrots.

Show More

Related Articles

Leave a Reply

Your email address will not be published. Required fields are marked *

Back to top button