Blogs | New Batches | Hire From Us | +91 8179191999 | [email protected] | +91 8179191999 | [email protected] |
  1. Courses
  2. Trending
  3. MS Azure Data Factory

Training Type

Select faculty

Select Date

Dur:
Course fee : /-

Download Curriculum

MS Azure Data Factory

Course Overview

"Naresh IT offers comprehensive Online and Classroom Training for Microsoft Azure Data Factory (ADF), a cutting-edge cloud-based data integration service. Our expert-led courses empower you to seamlessly create, schedule, and orchestrate data workflows, facilitating the ingestion, transformation, and loading of data from diverse sources into destinations like databases, data lakes, and analytical systems. With Naresh IT, you'll gain proficiency in building and managing scalable data pipelines tailored for analytics, reporting, and other data-driven tasks."

Learn software skills with real experts, either in live classes with videos or without videos, whichever suits you best.

Description

Azure Data Factory enables organizations to efficiently move data across on-premises and cloud environments, transforming raw data into meaningful insights for decision-making. It supports a wide range of data integration scenarios, including batch processing, real-time data ingestion, and hybrid data movement. ADF leverages Azure's scalable infrastructure and integrates seamlessly with other Azure services like Azure Synapse Analytics, Azure Databricks, and Azure SQL Database.

Course Objectives

  • Learn how to create and manage data pipelines using Azure Data Factory.
  • Understand various data integration patterns and best practices for building scalable and reliable data workflows.
  • Gain proficiency in connecting to different data sources, performing data transformations, and loading data into target destinations.
  • Explore monitoring, debugging, and optimizing data pipelines for performance and cost efficiency.
  • Learn how to integrate Azure Data Factory with other Azure services to build end-to-end data solutions.

Prerequisites

  • To effectively learn Azure Data Factory, it's recommended to have a basic understanding of:

    • Cloud computing concepts and Azure fundamentals.
    • Data storage and processing technologies.
    • SQL and data querying languages.
    • ETL (Extract, Transform, Load) concepts and data integration patterns.
Enroll Now
Course Curriculum

  • Understanding different Cloud Models
  • Advantages of Cloud Computing
  • Different Cloud Services
  • Different Cloud vendors in the market

  • Introduction to Azure
  • Azure cloud computing features
  • Azure Services for Data Engineering
  • Introduction of Azure Resources/Services with examples
  • Azure management portal
  • Advantage of Azure Cloud Computing
  • Managing Azure resources with the Azure portal
  • Overview of Azure Resource Manager
  • Azure management services
  • What is Azure Resource Groups
  • Configuration and management of Azure Resource groups for hosting Azure services

  • Completed walkthrough of the Azure Portal with all the features
  • What is Resource Groups and why we need RG’s in Azure cloud computing platform to host resources??
  • Different types of Storage Accounts provisioning in Cloud computing with different storage services
    • Container/Blob storage service
    • File share storage service
    • Table storage service
    • Queue storage service
  • Details explanation & understanding of different Blob/container storage services…
    • Page Blob
    • Append Blob Block Blob
    • Block Blob
  • Creating and managing the data in container storage services with Public and Private accesses as per the need of a project
  • Implementation of Snapshots for Blob storage services and File share storage service
  • Generating SAS for different storage services to make the storage content browseable across all the globe or Publicly
  • What is Standard Storage Account and Premium Storage account and which to use accordingly as per the real time scenarios
  • Detail explanation and implementation of Data Lake storage Gen2 Storage Account to store the unstructured data in cloud storage services
  • All the features/properties(Overview, activity log, Tags, Access control(IAM), Storage browser…etc) of Azure Storage Accounts
  • Maintenance and management of Storage keys and connection string for Azure Storage services
  • Implementing different levels of access(Reader, contributor, owners…etc) to the Azure Storage accounts

  • Moving the storage account with storage content across different Resources Groups based on real time scenarios
  • Migrating the data from On-prem(Private cloud) to Azure Storage account (Public cloud) using Az copy(forward migration)
  • Migrating the data from public cloud to Private cloud(revers migration)
  • Implementing the Az copy commands to migrate the data
    • On-prem to Azure cloud storage services
    • Cloud storage services to On-prem
    • Cloud to Cloud
  • Moving the SA & its content from one Resource Group to another

  • Azure Storage explorer for creating, managing, and maintaining the Azure storage services data
  • Installation of Azure Storage Explorer and what is the purpose of this tool for Azure Storage accounts(its Purpose & benefits with real time scenarios)
  • Generate Shared Access Signature(SAS) in Azure Storage Explorer(ASE) for security implementation of Storage account content
  • Managing of Access keys & connection strings of SA with Azure Storage Explorer
  • Configuration of Authentication and Authorization for Storage Account via Azure Active Directory
  • Hosting File share Storage services to On prem servers or Cloud Servers as shared drive for File share servers

  • Introduction to SQL DB’s
  • Creation of new SQL DB’s & Sample SQL DB’s both in On-prem and Cloud computing
  • Planning and deploying Azure SQL Database
  • Implementing and managing Azure SQL Database
  • Managing Azure SQL Database security
  • Planning and deployment of SQL DB’s in Azure cloud computing with real time scenarios
  • Different DB’s Deployment options
  • Databases purchasing models.(VCore & DTU’s)
  • Visualization of cloud DB server, Database, and validation of data from on prem(private cloud)
  • Implementation of Firewall security rules on Azure DB servers to access and connect from on-prem SSMS
  • Creation of Database in on-premises and synch with azure cloud

  • Migrating SQL DB’s from On-premises to Azure cloud computing using Microsoft Data migration assistant
  • Restoring SQL DB’s from On-prem to cloud computing
  • Migration of Specific DB objects from on-prem to cloud based upon base upon project requirements
  • Implementation of RSV and scheduling the backups of SQL DB’s and Azure Storage Account file share services on schedule, on demand based upon real time scenarios

  • Introduction to SQL DB Queries
  • Below SQL queries detail explanations, syntax & execution based upon real time scenarios
    • Select queries
    • Distinct queries
    • Where queries
    • And or not queries
    • Order By queries
    • Insert into queries
    • Null values queries
    • Update queries
    • Delete queries
    • Select Top queries
    • Min & Max queries
    • Count, Avg, Sum queries
    • Like queries
    • Wildcards queries
    • In queries
    • Between queries
    • Aliases queries
    • Joins(Inner join, Left join, Right join, Full join, Self-join…etc)
    • Union queries
    • Group By queries
    • Having queries
    • Exists queries
    • Any All queries
    • Select into queries
    • Insert into select queries
    • Store procedures queries

  • Deep understanding and implementation of concepts/Components of ADF
    • Pipelines
    • Activities
    • Datasets
    • Linked Services
  • Building blocks of Azure Data Factory
    • Triggers
    • Integration runtime
    • Dataflow
  • Complete features and walk through of Azure Data factory studio
  • Different triggers and their implementation in ADF
    • Scheduled trigger
    • Tumbling window trigger
    • Event trigger
  • What is integration run time and different types of integration run time in ADF
    • Azure
    • Azure – SSIS
    • Self-hosted
  • When to use ADF
  • Why to use ADF
  • Different types of ADF pipelines
    • Dynamic pipelines
    • Parameterized pipelines
    • Automated pipelines
  • Pipelines in ADF
  • Different types of Activities in ADF
    • Data movement activities
    • Data transformation activities
    • Data control activities
  • Datasets in Azure Data factory
  • Linked services in ADF

  • Copying the data from Blb Storage account to ADL’s Gen2 Storage account
  • Copying of zip files (.csv) from Blob SA to ADL’s Gen2 SA using ADF
  • Implementation and explanation of Metadata control in ADF to find the structure before copying the data
  • Implementation and explanation of Validation and If Condition
  • Implementation of Get Metadata control, filter control & For Each Control or activities in ADF
  • Implementation & execution to copy the data from GitHub platform to Azure Storage services with variables and parameters
  • Implementation of Foreach control, copy data control and Set variable to dynamically load the data from source to target using ADF
  • Creating Dynamic pipelines with lookup activity to copy multiple .csv files data picking form Json format data in Azure Storage services
  • Copying the files from GitHub Dynamically with the use of Dynamic parameters allocation-AUTOMATION PROCESS
  • Copying the data from different files formats(.csv, .xlsx, .txt, .Parquet, .Json, .SQL…etc) using suitable ADF controls/activities
  • Implementation and execution of Loading the data from Blb SA to SQL DB single table & multiple tables using copy data activity, ForEach activity
  • Executing multiple pipelines in parallel with Execute pipeline activity

  • Implementation of Schedule based triggers for different ADF pipeline containing different activities
  • Implementation of Event based triggers for different ADF pipeline containing different activities
  • Implementation of Thumbling window-based triggers for different ADF pipeline containing different activities
  • Implementation and execution of storage and Event based triggers

  • Detail explanation & implementation of Azure Keyvaults
  • Making the SQL DB connection string to store in Keyvault to enhance the security for SA content and SQL DB
  • Generating the secrets inside the Azure keyvault and granting access by implementing the access policies for different users

  • Detail walk through of GitHub portal
  • Creating an account, repo’s, in GitHub portal
  • Integrating Azure Data Factory with GitHub Portal as per project requirements
  • Placing, maintaining and executing the source code via GitHub portal for Azure Data Factory
  • Creating master branch, practice branches in GitHub portal to merge the newly created code via Pull Requests
  • Setting up the Repo for ADF pipelines and converting to live mode from GitHub portal covering with real time scenarios

  • Designing new Data flows
  • Designing and implementing transformations like
    • Source transformation
    • Join transformations
  • Inline Datasets in data flow source control
  • Designing and implementing of Data flow with Source transformations, Filter transformations & Sink transformations in ADF with inline Datasets
  • Implementation of Select transformations with Data flows for various source controls
  • Implementation of Dataflows using Aggregate & Sink transformation
  • Implementation of Dataflow with conditional split & Sink transformation with copy data activity
  • Implementation of Dataflow with Exists & Sink transformation
  • Implementation of Azure Dataflows for Derived column transformation with Source & Sink transformation
  • Implementation of Azure Dataflows to connect to SQL DB with Source & Sink transformation
  • Union & Union flow transformation implementation with ADF Data flows
  • Implementation of Azure Dataflows to connect to SQL DB with Source & Sink transformation
  • Implementation of windows functions…like Rank() function, Dense_Rank() function, Row_Number() function…etc.

  • What is Apache Spark, details explanation and implementation of Apache Spark
  • Illustration and Elaboration of Apache Spark Architecture
  • Explanation of
    • Resilient Distributed Dataset (RDD)
    • Directed Acyclic Graph (DAG)
  • Understanding of different Apache Spark components
    • Spark Core
    • Spark SQL
    • Spark Streaming
    • MLlib
    • Graph-X
  • What are worker nodes and slaves nodes in Azure Data Bricks clusters
  • Implementation of Azure Databricks cluster by considering different worker nodes and slave nodes
  • Different features and properties of Azure Data Bricks clusters
    • Single node
    • Multi node
    • Photon acceleration
    • Auto turn off Azure Data bricks cluster after a defined time
    • Autoscaling of cluster
    • Configuration provisioning of Azure Data Bricks clusters

  • Creating single node and multi nodes clusters
  • Creation of Pyspark notebooks in Databricks cluster to fulfil different business requirements

  • What is Azure Synapse Analytics
    • What is Synapse workspace used for
    • What is Synapse SQL
    • Apache Spark for Synapse
    • How to design Pipelines in Azure Synapse
  • Implementation of Linked Services/Datasets in Synapse Analytics
  • Implementation of dedicated SQL Pool inside Synapse Analytics
  • Implementation of serverless SQL Pool inside Synapse Analytics
  • Creation of Apache spark pool in Azure Synapse Analytics
  • Writing SQL Script in Azure Synapse analytics to get the result set in tabular and chart formats
  • Visualizing the data in Synapse analytics in variety of different charts (like pie charts, line charts, bar charts…. etc)
  • Designing of Synapse Analytics pipelines by considering various activities as per the business requirements
  • Creation of Datasets, Linked services for Synapse Analytics pipelines
  • Data analysis with serverless spark pools in Azure Synapse Analytics
  • What is Apache spark in azure synapse analytics
  • Designing and development of Apache spark pool in Azure synapse
  • Creating Spark Databases and tables to load the data from source system and analysing the data in Synapse analytics

  • What is Azure Stream Analytics
  • Purposes and usage of Stream Analytics in Azure cloud computing
  • Benefits and advantages of stream analytics
  • Architecture diagram of data flow in Azure stream analytics with other cloud services
  • Understanding & usage of browser-based Raspberry Pi simulator
  • Deployment of IoT Hub services as an input for Stream analytics jobs
  • Implementation & execution of stream analytics jobs and designing inputs and outputs for IoT Hub and Datalake Gen2
  • Writing SQL scripts to generate live streaming data and loading it in destination
APPLY NOW !
Who can learn this course

  • Data Engineers: Professionals responsible for designing, implementing, and managing data pipelines to support data-driven initiatives within organizations.
  • Data Analysts: Individuals looking to leverage Azure Data Factory for data ingestion, transformation, and preparation tasks to perform analytics and generate insights.
  • BI Developers: Those interested in building and automating data workflows for business intelligence and reporting purposes.
  • Database Administrators: Professionals seeking to extend their skills to include cloud-based data integration and orchestration using Azure services.
  • IT Professionals: Anyone interested in learning cloud-based data integration and management to modernize their organization's data infrastructure.

Average package of course (MS Azure Data Factory)

100% Avg
salary hike
4 - 6L Avg
Package
Training Features
Comprehensive Course Curriculum

Elevate your career with essential soft skills training for effective communication, leadership, and professional success.

Experienced Industry Professionals

Learn from trainers with extensive experience in the industry, offering real-world insights.

24/7 Learning Access

Enjoy round-the-clock access to course materials and resources for flexible learning.

Comprehensive Placement Programs

Benefit from specialized programs focused on securing job opportunities post-training.

Hands-on Practice

Learn by doing with hands-on practice, mastering skills through real-world projects

Lab Facility with Expert Mentors

State-of-the-art lab facility, guided by experienced mentors, ensures hands-on learning excellence in every session

Our Trainees are Working with
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
...
Reviews

Naresh IT’s Azure Data Factory training offers hands-on learning with real-world data integration projects, expert guidance, and flexible online classes. Highly recommended!

Angie M. Ravikiran Vemula
course : MS Azure Data Factory
Talk to Advisor +91 8179191999

Top 5 Technologies to learn Register for the Course !

By Providing your contact details, you agree to our Terms of use & Privacy Policy

Our Relevant Courses list

SnowFlake

4.5

ENROLL   SHARE
Google Cloud Online Training

4.5

ENROLL   SHARE
Multi Cloud

5

ENROLL   SHARE
Azure Fundamentals (AZ-900)

5

ENROLL   SHARE
Azure Developer Associate (AZ-204)

4

ENROLL   SHARE
Azure DevOps (AZ-400)

5

ENROLL   SHARE
Azure Administrator (AZ-104)

4.5

ENROLL   SHARE
Salesforce LWC

5

ENROLL   SHARE
Salesforce CRM

4.5

ENROLL   SHARE
Azure Data Engineer

4.5

ENROLL   SHARE
MS Azure + Azure DevOps

5

ENROLL   SHARE
Amazon Web Services (AWS)

4.5

ENROLL   SHARE