Snowflake - Build & Architect Data pipelines using AWS 2023

Last updated 2/2023
MP4 | Video: h264, 1920x1080 | Audio: AAC, 44.1 KHz
Language: English | Size: 3.42 GB | Duration: 8h 48m

Data engineering and architecting pipelines using snowflake & AWS cloud

What you'll learn
Will learn everything needed for Snowpro Advanced Data engineering certification
Snowflake as a data-warehouse & automated pipelines within snowflake ecosystem
Use AWS Cloud with Snowflake as a data-warehouse
Integrating real time streaming data and data orchestration with Airflow and Snowflake

Requirements
Prior programming experience in Sql and python is a must .
Prior basic experience or understanding of cloud services like AWS is important

Description
Course Update as of Feb 2023 : This Course has been updated with Snowpark API which covers UDFs,Stored Procedures for ETL and also covers Machine Learning use-case deployments . This course will help you clear SnowPro Advanced CertificationsSnowflake is the next big thing and it is becoming a full blown data eco-system . With the level of scalability & efficiency in handling massive volumes of data and also with a number of new concepts in it ,this is the right time to wrap your head around Snowflake and have it in your toolkit . This course not only covers the core features of Snowflake but also teaches you how to deploy python/pyspark jobs in AWS Glue and Airflow that communicate with Snowflake , which is one of the most important aspects of building pipelines . Anyone who has a basic understanding of cloud and belong to one of the below backgrounds can benefit from this course :- Data Scientists / Analysts - Data Engineers / Software Developers - SQL Programmers or DBA's - Aspiring Data analysts and scientists who are learning SQL and Python This Course covers : What is Snowflake Most Crucial Aspects of Snowflake in a very practical manner Writing Python/Spark Jobs in AWS Glue Jobs for data transformationReal Time Streaming using Kafka and Snowflake Interacting with External Functions & use casesSecurity Features in Snowflake Prerequisites for this course are : Knowing SQL or at least some prior knowledge in writing queries Scripting in Python (or any language )Willingness to explore ,learn and put in the extra effort to succeed An active AWS Account & know-how of basic cloud fundamentals Important Note - You need to have an active AWS Account in order to perform tasks in sections related to Python and PySpark . For the rest of the course , a free trial snowflake account should suffice . Some Tips : Try to watch the videos at 1.2X speed Read the reference links and the official documentation of Snowflake as much as possible

Overview
Section 1: Introduction

Lecture 1 Course Roadmap

Lecture 2 Prerequisites and How to Success in this course

Lecture 3 Lecture 4 - Feedback and Learn More

Lecture 4 Clone Github Repo & PPT for the Course

Section 2: Introduction to Snowflake and AWS

Lecture 5 What is a data-warehouse ?

Lecture 6 Two Aspects of a Data Ecosystem

Lecture 7 Lab - Setup Snowflake Trial Account

Lecture 8 Snowflake Architecture

Lecture 9 Snowflake Object Heirarchy

Lecture 10 Snowflake - Virtual Warehouses

Lecture 11 Snowflake - Different Billing Components

Lecture 12 Snowflake - Track your consumption

Lecture 13 Snowflake- Resource Monitors

Section 3: Snowflake - Tables

Lecture 14 Introduction - Different Tables in Snowflake

Lecture 15 Lab - Create Tables in Snowflake

Lecture 16 Snowflake - Views , Materialized Views and Secure Views

Lecture 17 Lab - Create Views in Snowflake

Lecture 18 Lab - Create Secure Views in Snowflake

Lecture 19 More about Views in Snowflake

Section 4: Snowflake - Partitioning , Clustering and Performance Optimization

Lecture 20 Section Overview

Lecture 21 Introduction to partitions and clustering keys

Lecture 22 Lab - Micropartitions and Clustering keys

Lecture 23 Benefits of Micro-partitions and Clustering

Lecture 24 Understanding Clustering Depth and Cluster Overlap

Lecture 25 Lab - Selecting your clustering Keys

Lecture 26 Lab - Check Query Profile and history

Lecture 27 Lab - Query Processing and Caching

Lecture 28 Search Optimization Feature

Section 5: Snowflake - Data Loading/Ingestion and Extraction

Lecture 29 Section Overview

Lecture 30 Data Ingestion - Real World Use Cases

Lecture 31 Lab - Create an Integration Object to Connect Snowflake with AWS S3

Lecture 32 Lab - Ingest CSV from S3 to Snowflake

Lecture 33 Lab - Ingest JSON from S3 to Snowflake

Lecture 34 Introduction to Continuous Data Ingestion in Snowflake

Lecture 35 Lab - Create and implement Snowpipe

Lecture 36 Snowpipe - Billing Estimation and Key Considerations for Data Ingestion

Lecture 37 Lab - Extracting/Unload Data from Snowflake to S3

Section 6: Snowflake - Tasks and Query Scheduling

Lecture 38 Section Overview

Lecture 39 Introduction to Tasks

Lecture 40 Lab - Create Standalone and Dependent tree of tasks

Lecture 41 Lab - Billing and Query History for Tasks

Section 7: Snowflake - Streams and Change Data Capture

Lecture 42 Section Overview

Lecture 43 Introduction to Streams

Lecture 44 Lab - Implement Standard Streams

Lecture 45 Lab - Implement Append-Only Streams

Lecture 46 Lab - Streams in a Transaction

Lecture 47 Streams - Data Retention and Staleness

Lecture 48 Lab - Change Tracking using "Changes"

Lecture 49 Project Overview

Lecture 50 Lab - Create Streams - Project Solution Part-1

Lecture 51 Lab - Create Streams - Part-1 Continuation

Lecture 52 Lab - End to End Pipeline in Action

Section 8: Snowflake - User Defined Functions

Lecture 53 Introduction to User Defined Functions and UDF Types

Lecture 54 Lab - Write and implement a Scalar UDF

Lecture 55 Lab - Write Tabular UDF in SQL

Lecture 56 Lab - Implement jаvascript UDFs

Lecture 57 What is Pushdown in UDF ?

Lecture 58 Lab - How can pushdown expose the underlying data ?

Lecture 59 Lab - Write Secure UDFs

Section 9: Snowflake - External Functions

Lecture 60 Section Overview

Lecture 61 Introduction to External Functions

Lecture 62 Lab - Write Deploy AWS Lambda Function

Lecture 63 Create IAM Role

Lecture 64 Lab - Create API Gateway

Lecture 65 Lab - Securing and Deploy API Gateway

Lecture 66 Lab - Create External Function in Snowflake

Section 10: Snowflake with Python,Spark and Airflow on AWS

Lecture 67 Section Overview

Lecture 68 Lab - Connect Python with Snowflake in your local machine

Lecture 69 Introduction to AWS Glue

Lecture 70 Lab - Deploy and execute python script to AWS Glue

Lecture 71 Lab - Parameterize your python script on AWS Glue

Lecture 72 Lab - Python Pandas with Snowflake on AWS Glue

Lecture 73 What is Pushdown in Spark 3.1 ?

Lecture 74 Lab - Deploy a Pyspark script using AWS Glue

Lecture 75 Lab - Setup Managed Airflow Cluster on AWS

Lecture 76 Lab - Configure Snowflake Connectivity in Airflow

Lecture 77 Lab - Deploy a PySpark Transformation job in AWS Glue

Lecture 78 Lab - Setup Airflow DAG

Section 11: Real Time Streaming with Kafka and Snowflake

Lecture 79 Section Overview

Lecture 80 Lab - Download the necessary JAR Files

Lecture 81 Lab - Setup Kafka in your local system

Lecture 82 Lab - Setup Kafka Snowflake Connector

Lecture 83 Lab - Setup Encryption Keys for Kafka-Snowflake Connectivity

Lecture 84 Lab - Streaming Data in Action

Section 12: Snowflake - Data Protection and Governance

Lecture 85 Section Overview

Lecture 86 What is TimeTravel and Failsafe in Snowflake ?

Lecture 87 Lab - Time Travel and Data Recovery

Lecture 88 Lab - Column Level Dynamic Data Masking

Lecture 89 What is Row Level Security ?

Lecture 90 Lab - Create and implement Row Level Access Policy

Lecture 91 More updates soon

Section 13: Strengthen your understanding - Bonus

Section 14: Snowpark - For Data Pipelines and Data Science

Lecture 92 Introduction-What is Snowpark?

Lecture 93 Lab - Getting Started with Snowpark

Lecture 94 Overview - UDFs and Store Procedures

Lecture 95 Lab-Deploy Python UDFs

Lecture 96 Lab-Deploy Stored Procedures for ETL Batch Processing

Lecture 97 Data Science - UseCase Overview and data preparation

Lecture 98 Lab-Deploy Model-Training Code for scikit-learn using Stored Procedures

Lecture 99 Lab-Deploy Model Serving/Prediction Serving Pipeline using UDFs

Lecture 100 More learning reference and Coupon Code

software engineers,aspiring data engineers or data analyst & data scientists,Also good for programmers and database administrators with experience in writing SQL queries

Buy Premium Account From My Download Links & Get Fastest Speed.