Saturday, November 17, 2018

Are you ready for Microsoft Azure or Not?

With so many exciting features that Azure offers such as Azure Stream Analytics and Azure Machine Learning, many friends and colleagues asked me about how to get started with Azure so that they can having some feelings about Azure. I highly suggest them reading "Azure Readiness Playbook", the official guidelines that came out in October 2018 to help you configure Azure environment. You may know that currently Microsoft offers $200 credit for first 30 day trial of Azure subscription. $200 will be gone in just a few days if you are not aware of costs associated with different subscriptions. You need to have an clear idea on how to track resource costs and control costs, or whether to start services before you are fully ready for Azure deployment. So

How to get access to official Azure Readiness Playbook?


Step 1: Sign in your Azure Portal.
Step 2: Search "Quickstart" in search box. You will find the "Quickstart Center (Preview)".

Step 3: Click "Quickstart Center" and you will see two options: "Create an Azure service" and "Set up your Azure environment".

Step 4: Select "Set up your Azure environment", you will see the launch page of Playbook.
Read through it or choose the topic that you are most interested in. Ensure that you have a general idea about what roles you can assign to in order to manage the access to your resources in Azure.

Below provides a brief summary about the topic.

Four management levels of access to resources in Azure 


As shown above, four management levels of access in Azure are: Management Group -> Subscriptions -> Resource Groups -> Resources, by the order of control levels from highest to lowest. Management groups have highest control while the rightmost resources have lowest level of management.  It is suggested to apply access and application controls at possible highest level because lower levels inherit settings from higher levels, such as security, policy in governance, and compliance strategy.

It is suggested to apply project-specific requirements at resources/services in Azure, including virtual machines, SQL databases, storage accounts, etc. Since a resource group is a logical container that group Azure resources such as SQL Data Warehouses, Data Factories, storage accounts, it is a good practice to assign one project with the same resource group.

Azure Policy and Azure Security Center

Make sure to try out Azure security center to see how to enforce and automate your governance decisions across hybrid cloud workloads.

Manage Cost and Billing

As for pricing, you can estimate costs with Pricing Calculator or check Azure Pricing. Be aware of services charged by hours, such as Azure SQL Database and Azure VM. Find out what are services charged by storage. Play around different settings to see how costs are affected. For example, for Azure SQL database, 8 core SQL DB usually costs $1476 per month while the one with only 1 core costs about $184. Remember, Azure SQL database is not able to be paused as for now (Nov. 2018).


Another way to monitor cost is to use "Cost Management + Billing" in Azure Portal as shown below:


If you would like to explore more detailed charges, click the cost on the far right. It will bring "Costs by service" page. You can check costs by subscriptions, resource types, resource group associated with, and at which time period by selecting related items at the top section.


If you want to drill down to check charges by dates, click charges that you are interested in, it will give you cost history by dates. Note that there is warning message stating that "There is a delay between the time when a resource is used and the time when the usage reaches the billing system. Due to this, costs reported here may be delayed. Amounts displayed are estimates, and may not reflect some recent usage. Taxes are not included."


After reading this article, I hope that you know whether you are ready to kick the tires of Azure or not. Good luck!

Reference: Azure Readiness Playbook Overview



Sunday, November 4, 2018

Comparisons of Import Data, DirectQuery, Live Connection modes in Power BI

Power BI has three connection modes to data sources, including "Import", "DirectQuery", and "Live Connection". Depending on types of data sources, you have different available options on connection modes. Sometimes you might get confused about which mode to choose from. This article will help you understand what are advantages and limitations for each mode.

What are different connection modes for Power BI?

  • Import connections: to copy data from sources using cache, store and compress data in PBIX file, then present visual reports in Power BI desktop. 
  • DirectQuery connections: to extract data directly from data sources and do data modeling to refine and enrich data. No data copy involved.
  • Live connections: to extract data directly from existing data models either in SSAS tabular or SSAS Multidimensional.

Types of data sources supported

Below is the list of of sources supported by each connection modes as of November 2018. For those data sources supported by DirectQuery, they are also supported by Import. You can refer to Data sources supported by DirectQuery for more details about sources supported for DirectQuery.


We can see that Live Connection only support sources from SSAS Tabular, SSAS Multi-Dimensional.

Comparisons of connection modes of Power BI

Besides different types of data sources that each connection mode can support in earlier section, below is a brief summary showing major differences among these three modes to help you understand Pros and Cons for each mode.

Comparisons of ModesImport DataDirectQueryLive Connection
Data Loaded To Memory?YNN
Direct Connection To Data Sources / Real-Time Data?NYY
Multiple Data Sources Support?YNN
Modeling SupportedFullyLimitedNone
Power Query SupportedFullyLimitedNone
Scalability1 GB dataset limitation.No dataset size limit but 1 million row limit for returning data. None
Reporting LimitationsNo limit.Not support Quick Insights and Q&A.None

From comparisons above, you know that you need to choose "Import" connection mode if you need to extract data and build reports from multiple sources. On the other hand, if you have very large datasets that it would be unfeasible to import or near "real-time" reporting requires for frequent changing data, you need to choose "DirectQuery" connection mode.

Report performance considerations

For reports performance considerations, it really depends on underlying data sources and user cases as mentioned in Best Practice for Building Fast and Reliable Power BI Reports.

Note that for live connection and direct query modes, you need to use On-premises data gateway for data sources other than Azure SQL Database, Azure SQL Data Warehouse and Redshift.
Updated on Nov 07, 2018: Power BI Premium also introduced dataflows as another option to help unify data from various sources and prepare data for modeling by using Linked Entities and computed entities.