Making Azure Synapse-SQL Pool Diagnostics data insightful — Data Collection

Azure Synapse Analytics

Setting Up Diagnostic data

Synapse Analytics — SQL Pool
Step 1 - Click on ‘Diagnostic settings’ in SQL Pool.
Step 2 - Click on +Add diagnostic setting for further option
Step 3 - Select events & its retention period.
Step 4 - Select ‘Archive to a Storage Account
Step 5 - Select storage account where logs will be stored
Diagnostic settings
insights-{metrics|logs}-{category name}/resourceId=/{resource Id}/y={four-digit numeric year}/m={two-digit numeric month}/d={two-digit numeric day}/h={two-digit 24-hour clock hour}/m=00/PT1H.json
Containers created by Diagnostic settings

Setting up Data Factory Pipeline

Data Pipeline (workflow)
ADF Data Pipeline
'storage_folder_path' -  Holds pre-defined folder path where logs will be saved. Eg: /SUBSCRIPTIONS/<Subscription Id>/RESOURCEGROUPS/<Resource Group Name>/PROVIDERS/MICROSOFT.SQL/SERVERS/<SQL Pool Server>/DATABASES/<Database Name>/'time_zone' - Hold which timezone ADF will be scheduled. Eg - 'India Standard Time''timeUnit' - Unit for subtraction , here taken as 'Day''timeInterval' - Value to be reduced , here taken as -1
@addToTime(convertFromUtc(utcnow(),pipeline().parameters.time_zone),pipeline().parameters.TimeInterval,pipeline().parameters.TimeUnit)
@{concat('resourceId=',pipeline().parameters.storage_folder_path,'y=',formatDateTime(variables('time_zone_format'),'yyyy'),'/m=',formatDateTime(variables('time_zone_format'),'MM'),'/d=',formatDateTime(variables('time_zone_format'),'dd'),'/*')}

--

--

--

Technology Evangelist/ Cloud Architect@Microsoft/Traveler

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Static Airline Route Data via JSON REST API — Get Data on Operating Routes

How I found RCE But Got Duplicated

Directions: how to move USDC to the Cronos Network to take an interest in IDO Crodo:

Getting on with relational database

Leetcode MySQL 180. Consecutive Numbers

Difference between Primitive and Non-Primitive data types in JAVA

LeetCode 217. Contains Duplicate

Last steps for Morpheus “Apollo” validators

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
FN

FN

Technology Evangelist/ Cloud Architect@Microsoft/Traveler

More from Medium

Violin Practice

Performance tests needed? JMeter to your rescue. Part 1

What to Do About Big Data from IoT: Knowledge Discovery in Database

What is a Java Full Stack Developer and How Do You Become One?