In my final article, I mentioned about monitoring Azure synapse utilizing alerts function. At the moment we are going to attempt to obtain the identical utilizing Log analytics function. The log analytics will monitor the synapse pipelines and supply us extra insights as soon as the job fails.
The Azure Synapse integration with Log Analytics is especially helpful within the following situations:
- You need to write advanced queries on a wealthy set of metrics which might be revealed by Azure Synapse to Log Analytics. Customized alerts on these queries might be created by way of Azure Monitor.
- You need to monitor throughout workspaces. You possibly can route information from a number of workspaces to a single Log Analytics workspace.
Establishing the Log analytics workspace
To start with, create a log analytic workspace for now, we are going to come to it later.
Browse to your synapse workspaceà monitoringà Diagnostic possibility as proven under. You possibly can see the choice Add diagnostic setting which is able to push all of the logs out of your synapse workspace to the log analytics. Under are the log classes that will probably be out there so that you can select based mostly in your requirement.
- SynapseRbacOperations
- GatewayApiRequests
- BuiltinSqlReqsEnded
- IntegrationPipelineRuns
- IntegrationActivityRuns
- IntegrationTriggerRuns
After getting made your choice present a reputation and put it aside.
As soon as performed now go to the log analytics workspace, Basic-> Logs and choose the scope. As soon as that is performed, we’ve the mixing full between the synapse workspace and the log analytics workspace however you wouldn’t have the ability to see any logs just because the pipelines has not been triggered but. The setting will solely document the knowledge of log units of future runs however it received’t decide any historic information.
I’m going to check this by re-executing the pipelines that we’ve in our synapse workspace.
Now return to the log analytics logs tab the place you possibly can see the desk created. The desk holds key data like exercise identify, runID, sort, class, and so on.
Querying the Logs from ‘Log Analytics’
All of the logs have began to maneuver to log analytics, subsequent step is to question this log data. To question this within the log analytics workspace you will need to use a contemporary question language known as KUSTO or KQL.
As per the Microsoft official documentation, the KQL question is principally a read-only request to course of and return outcomes. The request is acknowledged in plain textual content, utilizing a data-flow mannequin that’s simple to learn, creator, and automate. Kusto queries are manufactured from a number of question statements
I’ve queried to filter solely ‘InProgress’ from the OperationName column. You possibly can question based mostly in your want, I simply needed to point out you a demo the place all of the rows within the OperationName column incorporates the phrases ‘InProgress’ in it.
If you wish to see solely a restricted variety of columns, use the mission key phrase.
Creating alert
You need to use the +New alert rule choice to setup e mail alert. The steps to create alert and ship e mail notifications has been defined within the earlier article.
Within the subsequent steps you will need to present the Scope, Situation and Actions for the alert to be created, largely they’re pre-populated apart from the few minor fields the place you will need to enter manually.
We will see Log Question is pre-populated with question that we wrote to extract subset of knowledge. In Alert Logic part, we will specify what’s our standards, I’ve chosen that if question return is bigger than zero it data an alert ought to get fired, which suggests we’ve a failure reported.
Motion Group is the place we group actions i.e., both ship E-mail, message, run Azure Capabilities or Internet-hooks and so on. I’ve an motion group out there already therefore I’m selecting that, in case when you don’t have one you possibly can create it on the go.
Alert Rule Title – Present a rule identify which is able to then even be revealed in e mail.
Severity – As identify suggests what needs to be the severity of alert (1, 2, or three based mostly on its criticality)
Allow Alert on Completion – Alert will robotically be enabled as soon as created else must manually allow it put up creation
With now every thing being set, let’s set off the pipeline and await few minutes to verify if we obtain the e-mail alert.
After couple of minutes, I’ve efficiently acquired the alert together with the small print like violation depend and search question.
Abstract: That is the second methodology to watch the synapse analytics through the use of Log analytics methodology. Within the earlier weblog I wrote about monitoring synapse utilizing azure monitor, I strongly counsel you check out it too.
