Master Airflow: Beginner to Advance with Project

Master Airflow: Beginner to Advance with Project
Duration: 12:28:08 | .MP4 1280x720, 30 fps(r) | AAC, 44100 Hz, 2ch | 7.1 GB
Genre: eLearning | Language: English

A Comprehensive Guide to Apache Airflow: Learn Workflow Management from Beginner to Expert
What you'll learn
Learn Orchestration and Workflow Management
How to create Data Pipelines
Cloud Integrations and Data Migration
End to End Data Pipeline with real use case
Requirements
Basic Python Knowledge
Description
Apache Airflow is an open-source platform used for workflow automation, scheduling, and orchestration of complex data pipelines. As data volumes and complexity continue to grow, the need for efficient and scalable data processing and management is critical. In this comprehensive course, you will learn how to master Apache Airflow, starting from the basics and progressing to advanced concepts.
The course is designed for data engineers, data scientists, python developers, software engineers, and anyone interested in learning how to automate and manage data workflows.
You will learn how to use Apache Airflow to build and manage data pipelines, schedule, and trigger tasks, monitor and troubleshoot workflows, and integrate with various data sources and services.
The course will cover the following topics:
Introduction to Apache Airflow and workflow management
Introduction to Docker and Docker Commands
Installation and configuration of Apache Airflow
Building and managing workflows with Apache Airflow
Scheduling and triggering tasks in Apache Airflow
Operators in Apache Airflow
Fetching data from Web APIs or HTTP
File Sensors in Apache Airflow
Connecting with Azure or AWS
Using AWS S3 Bucket and Azure Blob Storage to store and retrieve data
Creating custom operators and sensors
Handling dependencies and task retries
Monitoring and troubleshooting workflows
Integrating with data sources and services
Scaling and optimizing Apache Airflow for large-scale data processing using Celery Executors
Securing Dags Connections using Fernet Keys
Throughout the course, you will work on practical exercises and projects to apply the concepts you learn. By the end of the course, you will have a strong understanding of Apache Airflow and the skills to build and manage complex data workflows.

Who this course is for:
Data Engineers
Python Developers interested in Data Engineering
Data Scientist
DevOps Engineer
MLOps Engineer
Software Developers

More Info