Introduction
Parameterization may be very helpful while you desire a reusable code that you need to use without end and get the output by executing it solely by altering the parameter for all of your future necessities. Historically whereas coding you’ll declare variables that are static(see picture beneath) however with parameterization you need to use dynamic parameters all by means of your program with out declaring it a number of occasions at totally different places. We are going to see how can we parameterize utilizing Toggle possibility for a cell within the azure synapse analytics.
Static parameters
Let’s see a real-time demo on easy methods to name information from totally different ADLSGen2 storage paths and browse them right into a dataframe. We can even see how can we name the parameterized pocket book into one other synapse pocket book with a unique parameter and see the way it fares.
Steps
Create a brand new pocket book and fix it to your sparkpool which is already accessible or create a brand new one for this goal, I’ve named this pocket book as “Parameterization_Demo”. The primary cell is the place we’re going to declare our parameters however not in a manner we do with regular coding.
Earlier than we go on to declare our parameters by means of code one essential step is to allow the cell to simply accept the parameter through the use of Toggle button. On the highest proper aspect of the cell and underneath the ellipsis button, the Toggle parameter cell possibility might be accessible which as soon as after you allow it, can see the phrase parameters on the backside of the cell. This means that the cell is now prepared and the variables that might be created inside them might be parameterized.
Within the beneath cells, I’ve declared parameters and referred to as a CSV file from my storage location to create a spark dataframe.
Keep in mind to make use of separate cells for declaring variable and code execution. Should you mix each of them into the identical cells the parameter logic wouldn’t work. You can be getting the identical static worth for the variables and the dynamic values you specify received’t get utilized as a result of the parameter can’t override the default values with that of the values which have been newly declared.
Additional, I’m going to create a brand new pocket book and name this code which is saved in “Parameterization_Demo” pocket book however with totally different location and one other file title i.e., totally different parameters.
Notebook2 is the brand new pocket book I’ve created from which I’m going to name the earlier one which we noticed earlier on this demo.
I referred to as utilizing %run /<notebook_name> to entry from the cell and made certain all are printed earlier than I try to get the output. I’ve given totally different parameters for the important thing worth pairs the place the bottom line is the parameter title and worth is filename or path. You may see although the parameter values are totally different it generates the worth correctly from the storage location.
The identical could be referred to as from a dataflow by creating handbook parameter checklist to it however the names should match to what we declare within the code with the one we create.
Abstract
This text with the demo is to grasp how can we parameterize to reuse the codes as a substitute of writing a brand new one each time. This additionally explains how can one pocket book be referred to as into one other one with totally different parameters explicitly said.
Reference
Microsoft official docs
