Sunday, October 14, 2018

Stairway to SQL Server Reporting Service: Decision Functions

As part of expressions, decision functions can be used in calculated fields, filters, formatting to dynamically control related properties. They can be used with SQL, MDX, DAX query to query data from database, cube, SSAS tabular model respectively. This article will give a brief introduction and comparison of three decision functions: IIf, Switch, Choose. Also examples of their usage are given at the end.

IIf Function

IIf function returns one of two values depending on whether the expression is true or not. The syntax is as below:
IIf( <Boolean Expression>, [Return if True], [Return if False])
It likes conditional logic in SSIS. The first parameter is the expression to be evaluated. If it is true, then return the value specified in [Return if True]; If it is false, then return the later value specified in [Return if False]. You can have nested multiple IIf functions and the later condition is dependent on previous conditions' evaluation.

Switch Function

Switch function returns the value associated with the first expression in a series that evaluates to true. The statements will be evaluated in the order in which they appear and the condition expression can be unrelated with each other. The syntax is as following:
=Switch(
       <Boolean Expression 1>, [Return if True 1],
       [Boolean Expression 2], [Return if True 2],
    ...
       [Boolean Expression n], [Return if True n],
      )

Choose Function

Choose function uses a single numeric evaluation to determine which of return values to return. The syntax is as below:
Choose(
     <Numeric Expression>, 
     [Return Expression 1],
     [Return Expression 2], 
     …, 
     [Return Expression n],
)

Comparisons of Three Decision Functions

As you can see, Choose function is quite different from the other two. Since the return only depends on the numeric evaluation specified. IIf and Switch function can be used interchangeably for a certain condition, as example of dynamic text font color below shown. For me, for case conditions are more than three, Switch function has clearer logic thus is easy to follow. However, Switch and Choose functions may have possible undefined results while IIf function won't since IIf function requires that a “return if true” value and a “return if false” value be provided in the expression.  One way to avoid this is to specify the last condition in Switch function to be TRUE as the following example.

 =Switch(Fields!OrderNumber.Value >= 10, "Blue", 
        Fields!OrderNumber.Value >= 5, "Yellow", 
 Fields!OrderNumber.Value >= 2, "Orange", 
 TRUE, "Red")
 

ProblemsExpressions
To set filter expression depending on Country Code from MDX query.Solution 1:
=IIF(Parameters!CountryCode.Value.ToString() <>
"[Location].[Country Code].&[USA]",
"4",
"6")
 

Solution 2:
=IIF(InStr(Parameters!CountryCode.Value
.ToString(),"USA")>0,
"6","4")
To format the report category based on input date parameters ReportDate.=IIF(
InStr("January, February, April, May,July,August,October,November",
Monthname(Datepart("m", Parameters!ReportDate.Label))
) > 0,
"Monthly","Quarterly"
)+ Space(1) + "Reports"
To dynamically change the color of a text box, go to properties, and set the following expression for font/Color Property by Switch function.=Switch(
 Fields!OrderNumber.Value >= 10, "Blue",
 Fields!OrderNumber.Value >= 5, "Yellow",
 Fields!OrderNumber.Value >= 2, "Orange",
 TRUE, "Red")
To dynamically change the color of a text box, go to properties, and set the following expression for font/Color Property by IIf function.=IIF(
Fields!OrderNumber.Value >= 10, "Blue",
IIF(Fields!OrderNumber.Value>= 5, "Yellow",
IIF(Fields!OrderNumber.Value>= 2,"Orange", "Red"
))))


Saturday, October 6, 2018

Decoding Errors with Error Column Names and in SQL Server 2016 Integration Service

Problem: When SSIS packages failed at the step of loading files, sometimes it doesn't tell which column is the one that caused failure. How to find out the troublemaker?

Solution: 

If you have SQL Server 2016 Integration Service or later version, you can retrieve error column names by using the new GetIdentificationStringByID method in SSIS 2016.

Below is an example showing how to use IDTSComponentMetaData130.GetIdentificationStringByID method to decode error column names and error descriptions by using Script Component.

Step 1: Choose script component and connect it with error output from upstream.
Step 2: Select Transformation as script component type.

Step 3: After selecting C# as script language to use, check ErrorCode and ErrorColumn as Input Columns as shown below.

Step 4: Add output columns ErrorColumnName and ErrorDesc to represent error column names and descriptions respectively, then specify the column type and length.


Step 5: Edit script and added this following to Input0_ProcessInputRow function:

public override void Input0_ProcessInputRow(Input0Buffer Row)
    {
        
        IDTSComponentMetaData130 componentMetaData = this.ComponentMetaData as IDTSComponentMetaData130;
        Row.ErrorColumnName = componentMetaData.GetIdentificationStringByID(Row.ErrorColumn);
        Row.ErrorDesc = ComponentMetaData.GetErrorDescription(Row.ErrorCode);
    };
Then you can add a flat file output to write the errors with detailed information about error column names and descriptions.

From the following output, you can tell that the loading failure is because input column "OLE DB Destination.Inputs[OLE DB Destination Input].Columns[OrderDate]". The error description is "The data value cannot be converted for reasons other than sign mismatch or data overflow."

Saturday, September 22, 2018

How to use Azure Storage Emulator for Testing

Problem: During development of SSIS packages for data transferring between Azure Storage, you want it to be fully tested on your local first before any data transfers to your Azure account. You also want to avoid unnecessary charges on your Azure account for testing. How to achieve this?

Solution: Using Azure storage emulator

We know that Azure storage is charged based on volume of data stored every month, types of operations performed, and amount of data transfers involved base on Azure Storage pricing.

In order to avoid unnecessary charges during development and testing, we can leverage Azure storage emulator to test and debug Azure cloud services locally before real data transfer occurs between on-premise and Azure account.

First, you can download and install Azure storage emulator here. During installation, you need to take note of installation paths of Azure emulators. Here mine is installed by default at: 

C:\Program Files (x86)\Microsoft SDKs\Azure\Storage Emulator

After installation, start the storage emulator by searching "Azure Emulator" after clicking windows Start button. 

If it is executed for the first time, it will do initialization first by creating a database in LocalDB and granting database access for current user.

You can also check the database created by Storage Emulator as shown below:

How to set connection to Azure storage emulator?

Here we use upload files to Azure Blob storage by SSIS as an example. (As for how to set up Azure environments for SSIS, please refer to the steps in "How to Upload Data Files to Azure Blob Storage". )

At connection manager pane in SSIS, choose "New Connection", then choose "AzureStorage" as shown below.

Next, choose "Use local developer account" instead of "Use Azure account". Then click "Test Connection" button to ensure connection with storage emulator is successful. Ensure that you have storage emulator running. You can check the status of emulator by running the following command on command console:

C:\Program Files (x86)\Microsoft SDKs\Azure\Storage Emulator>AzureStorageEmulator.exe status


You can rename the connection to the name you want. Here I rename it to "azure-emulator" for future use.

To use it with Azure related task, the main change is the connection to be the Azure emulator one you just created. Below shows the configuration for Azure Blob Upload Task. Now you are good to go to debug and test your package on your local!

Help Command in Storage Emulator

You can use help command to quickly locate commands for operations on storage emulator.


Note that storage emulator is an emulated environment running on a local SQL instance, there are Differences between the storage emulator and Azure Storage, in aspects of Blob Storage and table storage.