Microsoft Azure Data Factory
Microsoft is the world’s largest software company. Founded in 1975 and headquartered in Redmond, it has become a household name, primarily due to its Windows operating system and Office suite. Aside from these products, Microsoft has a vast range of enterprise software and cloud offerings including its own database, browser, various servers and ERP solutions. In recent years, Microsoft has focused its business on cloud-based solutions such as Azure. AI and machine learning have also become increasingly important in product development.
Azure is Microsoft’s cloud platform offering over 200 different products and services on a global physical infrastructure consisting of data centers in over 60 countries. Officially introduced in 2010, Azure – and cloud business in general – has become a very important and successful strategic pillar for Microsoft, making up 38 percent of Microsoft’s operating income. The various services offered by Microsoft are known for being very accessible to companies of all sizes and industries and can be integrated quite easily with each other. In addition to Microsoft’s own services, certified partners offer a huge variety of solutions and services through the constantly growing Azure Marketplace.
As of today, Microsoft is the second largest cloud provider in the world (behind Amazon AWS). With a recently reported annual revenue growth rate of 59 percent (AWS: 32 percent), Microsoft seems to address today’s business demands quite well.
Microsoft Azure has a competitive and broad offering in the data and analytics space, ranging from PaaS solutions for data and big data management and analytics to multiple AI and machine learning offerings to specialized SaaS solutions, such as Azure Purview.
Azure Data Factory is Microsoft’s offering for hybrid data integration. Hybrid means it integrates with on-premises data sources, as well as supporting multi-cloud data integration. Therefore, its main capability is to build, manage and run ETL and ELT processes at any scale, using ‘code-free’ interactive user interfaces (Microsoft claims it is suitable for ‘citizen integrators’) or by writing custom code. All Azure Data Factory capabilities can (more or less) easily be automated as they are all exposed through APIs. This perfectly supports DevOps and DataOps scenarios, where automation is key. For balance, it should be noted here that this is also true of most cloud-based PaaS services from all vendors. Most modern non-cloud solutions also provide extensive API support.
Azure Data Factory itself does not provide any data storage or analytical capabilities on its own. Therefore, it is intended to support Microsoft’s other core data products, such as Azure SQL Server, Azure Synapse (Microsoft’s DWH, BI and analytics platform), Azure Data Lake and Azure Machine Learning, or even run standalone.
Azure Data Factory provides over 90 built-in connectors, ranging from all of Microsoft’s own Azure data services; Big Data sources such as Amazon Redshift and Google BigQuery; enterprise data warehouses including Oracle Exadata, Teradata and Snowflake; ERP solutions like SAP; and SaaS apps such as Salesforce, Marketo, ServiceNow and SAP C4C.
User & Use Cases
Azure Data Factory is used predominantly for data warehousing/BI (67 percent), data integration (67 percent), and ingesting and processing data from a data lake (47 percent). Since Azure Data Factory is Microsoft’s ‘go-to’ component for ETL purposes within the Azure cloud, it can be assumed that most users use it in conjunction with Azure SQL Server, Azure Synapse, Azure Data Lake or other data processing products within Azure.
The fact that Azure Data Factory is predominantly used by midsize companies is probably due to the adoption of ETL cloud services being quite simple and attractive for smaller companies while larger companies often have contracts or long-running relationships with ‘classic’ ETL solution providers. Nevertheless, the mean of 83 users per company is quite high for an ETL tool. We would expect there to be fewer dedicated ETL experts. However, if we take into account all the data integration tasks listed above, it seems the product also supports business users, for example, with data preparation and data warehouse automation.
Use cases
n=30
Extend of usage in the company
n=20
Total number of users per company
n=30
Total number of developers per company
n=20
Company size (number of employees)
n=30
Want to see the whole picture?
BARC’s Vendor Performance Summary contains an overview of The Data Management Survey results based on feedback from Microsoft Azure Data Factory users, accompanied by expert analyst commentary.
Contact us to purchase the Vendor Performance Summary- Register for a free sample Vendor Performance Summary download
- If you have any questions, feel free to contact us