Data Engineering on AWS Vol 1 - OLAP & Data Warehouse
Course Introduction and Resources
Data Engineering on AWS Vol 1 - OLAP & Data Warehouse
0%
Completed
Introduction – Data Engineering Volume 1 on AWS
Course Introduction and Resources
Video - 00:24:28 mins
Course Introduction and Course Contents
Video - 00:12:11 mins
(Optional) AWS Pre-requisites – EC2 & EBS
AWS Cloud and EC2 Introduction
Video - 00:19:13 mins
EC2 Components & HandsOn 1
Video - 00:21:7 mins
EC2 Handson 2
Video - 00:11:26 mins
EBS Theory
Video - 00:19:33 mins
EBS HandsOn
Video - 00:13:25 mins
(Optional) AWS Pre-requisites – VPC
VPC Introduction & Components
Video - 00:18:36 mins
VPC Components Hands On
Video - 00:23:4 mins
Bastion Host
Video - 00:5:29 mins
Security Groups
Video - 00:15:9 mins
NAT Gateway & VPC Endpoint
Video - 00:20:13 mins
VPC Peering
Video - 00:2:33 mins
(Optional) AWS Pre-requisites – IAM
IAM Introduction & Hands On
Video - 00:26:1 mins
IAM Service Roles
Video - 00:19:38 mins
(Optional) AWS Pre-requisites – SQL Basics
SQL Introduction
Video - 00:30:35 mins
SQL Client & Server Setup
Video - 00:12:18 mins
SQL Database Objects Theory
Video - 00:28:13 mins
Database Objects Hands On
Video - 00:29:19 mins
CRUD Operations
Video - 00:21:18 mins
SELECT Operators
Video - 00:24:41 mins
CASE COALESCE Functions
Video - 00:13:19 mins
DATE Functions
Video - 00:5:46 mins
CTAS Cast Concat
Video - 00:14:11 mins
Update Delete Truncate
Video - 00:12:31 mins
HAVING Clause
Video - 00:7:43 mins
Inner Join, Left Join, Right Join, Outer Join
Video - 00:19:27 mins
Union Intersect View
Video - 00:17:35 mins
Materialized View
Video - 00:8:19 mins
Common Table Expression (CTE)
Video - 00:10:48 mins
SQL Window Functions
Video - 00:22:40 mins
MERGE statement & Summary
Video - 00:10:52 mins
(Optional) AWS Pre-requisites – Python Basics
Python Intro - Architecture, PyCharm, Virtual Env
Video - 00:39:33 mins
PyCharm & CLI Walkthrough
Video - 00:8:51 mins
Compiled vs Interpreted
Video - 00:7:42 mins
Everything is Python is Object
Video - 00:12:39 mins
String Data Type
Video - 00:10:20 mins
Number Data Type
Video - 00:4:2 mins
List Data Type
Video - 00:11:36 mins
Tuple Data Type
Video - 00:6:16 mins
Set & Dict Data Type, Type Conversion
Video - 00:16:3 mins
Python Operators & Memory
Video - 00:10:27 mins
Set up Python interpreter in PyCharm
Video - 00:11:23 mins
Print & Input Functions
Video - 00:16:4 mins
IF Statement
Video - 00:14:36 mins
For & While loops
Video - 00:15:57 mins
Functions Intro
Video - 00:9:54 mins
Function Scoping
Video - 00:15:20 mins
Functions RETURN
Video - 00:7:53 mins
Function Arguments
Video - 00:9:38 mins
Modify Arguments
Video - 00:9:7 mins
Positional & Keyword Arguments
Video - 00:9:42 mins
args & kwargs
Video - 00:16:39 mins
Class Object Self
Video - 00:32:49 mins
Class-Instance Variables, __init__
Video - 00:17:1 mins
Class Object Exercise 1
Video - 00:14:48 mins
Class Object Exercise 2
Video - 00:13:48 mins
Inheritance
Video - 00:7:50 mins
Python Memory Management
Video - 00:8:5 mins
Modules & Packages
Video - 00:33:27 mins
HandsOn Exercise
Video - 00:1:31 mins
Module Pre-compilation
Video - 00:3:28 mins
Namespace & __name__
Video - 00:9:56 mins
Error Handling in Python
Video - 00:14:00 mins
File Handling
Video - 00:17:10 mins
CSV & JSON module
Video - 00:13:37 mins
Python Multi-threading concept
Video - 00:17:37 mins
Multi-threading hands-on and exercise
Video - 00:22:1 mins
Debugging & Profiling
Video - 00:18:26 mins
Data Engineering Introduction
Data Engineering introduction, OLTP & OLAP
Video - 00:33:10 mins
Data Mart & Data Mesh
Video - 00:5:31 mins
Data Lake, Data Lakehouse, DWH
Video - 00:22:5 mins
AWS Distributed Storage – S3 (Simple, Storage, Service) for Data Engineers
Introduction 1
Video - 00:14:38 mins
Introduction 2
Video - 00:22:49 mins
Basics
Video - 00:5:43 mins
Basics Hands-on
Video - 00:18:53 mins
Versioning
Video - 00:13:6 mins
Encryption
Video - 00:5:51 mins
Storage Class
Video - 00:20:18 mins
Multipart Upload
Video - 00:12:51 mins
Lifecycle Policies
Video - 00:15:4 mins
Cross Region Replication
Video - 00:10:13 mins
Mountpoint
Video - 00:9:21 mins
Security - S3 Identity Based Policy
Video - 00:19:3 mins
Security - S3 Bucket Policy
Video - 00:8:29 mins
Bucket Policy with VPC, IP address, VPCE
Video - 00:3:50 mins
Access Point
Video - 00:16:27 mins
Object Lambda
Video - 00:18:55 mins
Pre-signed URL
Video - 00:4:33 mins
Performance Considerations
Video - 00:5:31 mins
Pricings
Video - 00:13:00 mins
Architectural Patterns using S3
Video - 00:7:25 mins
Data Modelling – Normalization, ER Diagram, Dimensional Modelling
Highlights
Video - 00:13:9 mins
Data Modelling Introduction
Video - 00:17:15 mins
Normal Forms 1NF 2NF 3NF
Video - 00:28:1 mins
Relations: one-to-one, one-to-many, many-to-one, many-to-many
Video - 00:8:51 mins
Dimensional modelling - Facts, Dimensions & Grains
Video - 00:24:39 mins
Grains Exercise
Video - 00:9:19 mins
Dimensional Modelling Technique
Video - 00:15:00 mins
Types of Fact & Dimension Tables
Video - 00:10:10 mins
Data Virt Semantic Presentation Layers
Video - 00:13:24 mins
Data Warehouse on AWS – Redshift Infra
Redshift Infra
Video - 00:19:36 mins
Redshift Infra HandsOn
Video - 00:21:42 mins
Redshift Architecture - Zone Map, Columnar Storage
Video - 00:15:2 mins
Cluster Resize - Elastic & Classic
Video - 00:8:32 mins
Cluster Resize - HandsOn
Video - 00:5:3 mins
Cluster Pause & Rename
Video - 00:4:31 mins
Snapshot & Backup
Video - 00:7:43 mins
Redsfhit Infra Conclusion
Video - 00:3:4 mins
Redshifts Objects
Querying, Connection, RSQL, QEV2
Video - 00:16:10 mins
Query Editor & RSQL setup
Video - 00:17:54 mins
Object Hierarcy, tables hands-on
Video - 00:19:14 mins
Data Types Hands-on
Video - 00:14:20 mins
Table operations Hands-on
Video - 00:12:27 mins
Redshift ACID, Locks, Isolation Level
Video - 00:12:5 mins
Implement Transactions
Video - 00:10:1 mins
AccessShareLock & ShareRowExclusiveLock HandsOn
Video - 00:8:50 mins
Redshift SUPER datatype
Video - 00:14:29 mins
Section Summary
Video - 00:5:2 mins
Redshift Deep Dive
Distribution Key & Style, Sort Key
Video - 00:29:35 mins
Column Compression
Video - 00:6:42 mins
Modify Dist Sort Key, Compression HandsOn
Video - 00:19:18 mins
COPY Command Theory
Video - 00:6:33 mins
COPY Command HandsOn
Video - 00:13:36 mins
UNLOAD Command
Video - 00:7:16 mins
AWS DMS - Move from OLTP to DWH
Video - 00:8:10 mins
DMS - Setup Source OLTP & Python application
Video - 00:19:14 mins
Setup DMS Instance, Endpoint, Task
Video - 00:15:28 mins
DMS Task - OLTP to DWH
Video - 00:28:44 mins
Table Maintenance - VACUUM & ANALYZE
Video - 00:9:46 mins
Vacuum & Analyze HandsOn
Video - 00:11:25 mins
Resdhift Features
Materialized View (MV)
Video - 00:10:27 mins
MV HandsOn
Video - 00:13:59 mins
Query Federation
Video - 00:11:19 mins
Redshift Spectrum
Video - 00:14:18 mins
Streaming Ingestion
Video - 00:24:5 mins
Redshift Feature Use Cases
Video - 00:28:36 mins
Redshift Query Tuning
Query Execution
Video - 00:22:32 mins
EXPLAIN Plan & System Joins
Video - 00:23:32 mins
System Joins HandsOn
Video - 00:5:29 mins
Data RE-distribution
Video - 00:9:1 mins
EXPLAIN & RE-distribution HandsOn
Video - 00:16:18 mins
Query Tuning Exercise - Part 1
Video - 00:31:17 mins
Query Tuning Exercise - Part 2
Video - 00:24:46 mins
Query Tuning Exercise - Part 3
Video - 00:24:27 mins
Redshift Workload Management (WLM)
WLM Intro & Query Queue
Video - 00:9:2 mins
Concurrency Scaling, Short Query Acceleration
Video - 00:7:19 mins
Configure WLM HandsOn
Video - 00:16:44 mins
Create Query Queue HandsOn
Video - 00:15:50 mins
Query Queue In Action
Video - 00:7:10 mins
Concurrency Scaling In Action
Video - 00:15:33 mins
Redshift Security- RBAC, CLS, RLS, Dynamic Data Masking (DDM)
Users, Roles, RBAC
Video - 00:11:29 mins
Users, Roles, RBAC HandsOn
Video - 00:12:14 mins
Row & Column Level Security (RLS & CLS)
Video - 00:11:25 mins
Multiple RLS Policies HandsOn
Video - 00:13:2 mins
CLS HandsOn
Video - 00:5:00 mins
Combine RLS & CLS HandsOn
Video - 00:4:44 mins
Dynamic Data Masking
Video - 00:7:00 mins
Track Users, Roles, CLS and RLS
Video - 00:10:48 mins
Audit Logging
Video - 00:11:32 mins
Monitoring in Redshift
Monitor Redshift using Console
Video - 00:18:00 mins
System Views for Monitoring Queries, Redshift Objects, Configuration Parms
Video - 00:20:50 mins
Reshift Serverless
Introduction to Redshift Serverless
Video - 00:12:12 mins
Create & Delete Redshift Serverless Resources
Video - 00:6:39 mins
COPY & UNLOAD in Serverless
Video - 00:10:17 mins
ZeroETL Integration Setup
Video - 00:24:12 mins
ZeroETL in Action
Video - 00:12:59 mins
Query Tuning Similarities
Video - 00:4:28 mins
Migrate from Provisioned to Serverless
Video - 00:3:55 mins
Detailed Redshift Pricing
Redshift Pricing Components
Video - 00:11:14 mins
Pricing Example - Provisioned, Serverless, Concurrency Scaling, Spectrum
Video - 00:11:15 mins
AWS Pricing Calculator
Video - 00:5:17 mins
Redshift Additional Information
Redshift Integration with AWS Services
Video - 00:13:1 mins
Redshift & Snowflake Comparison
Video - 00:19:46 mins
Redshift Best Practices
Video - 00:16:10 mins
Redshift Limitations and Challenges
Video - 00:16:28 mins
AWS Metadata Repository – Glue Data Catalog
AWS Glue Catalog - Theory
Video - 00:8:36 mins
Glue Catalog - Setup Data Stores & IAM Roles HandsOn
Video - 00:8:29 mins
Store Aurora metadata in Glue Catalog
Video - 00:17:55 mins
Store S3 and Redshift metadata in Glue Catalog
Video - 00:10:50 mins
Data Governance using AWS Lake Formation
Lake Formation Introduction
Video - 00:17:56 mins
Permission Flow HandsOn 1
Video - 00:19:41 mins
Permission Flow HandsOn 2
Video - 00:9:9 mins
Lake Formation - Tag Based Access Control (LF-TBAC)
Video - 00:6:16 mins
LF-TBAC HandsOn
Video - 00:19:10 mins
LF - Data Filtering
Video - 00:11:18 mins
LF Clean Up (Please complete this)
Video - 00:3:29 mins
Data Lakehouse on AWS – Athena
Athena Introduction
Video - 00:16:41 mins
Athena Intro Hands On
Video - 00:10:44 mins
Athena SerDe, File & Row format
Video - 00:12:51 mins
SerDe, Format, CTAS Hands On
Video - 00:12:52 mins
UNLOAD, Prepare & Execute, Query JSON
Video - 00:8:40 mins
UNLOAD, Prepare & Execute, Query JSON Hands On
Video - 00:17:27 mins
Schema Evolution, JSON_EXTRACT
Video - 00:13:54 mins
Iceberg, ACID
Video - 00:26:19 mins
Athena Partitioning & Bucketing
Video - 00:19:28 mins
More DDL Commands
Video - 00:6:12 mins
Athena WLM Theory
Video - 00:8:3 mins
Workgroup HandsOn
Video - 00:11:32 mins
Capacity Reservation HandsOn
Video - 00:4:6 mins
Performance Tuning Theory
Video - 00:13:29 mins
Athena Pricing & Performance Tuning
Video - 00:15:46 mins
Architectural Patterns using Athena
Video - 00:8:30 mins
Big Data Warehouse – HIVE
Hadoop Theory
Video - 00:21:5 mins
File Formats
Video - 00:8:58 mins
Hive Architecture & Components
Video - 00:16:35 mins
Hive CLI
Video - 00:3:21 mins
Data Types, databases, tables, File & Row Format, Hive SerDe
Video - 00:12:57 mins
Hive Databases hands-on
Video - 00:17:7 mins
Hive Tables hands-on
Video - 00:16:40 mins
Partitioning & Bucketing
Video - 00:16:55 mins
Partitioning & Bucketing hands-on
Video - 00:19:30 mins
Load, insert, ACID, Materialized Views etc
Video - 00:15:8 mins
JOINs, Locks, Configuration Parameters
Video - 00:14:26 mins
Projects – Redshift, Athena, DataModelling, Pythin (Total Dataset Size – 150 GB)
Project 1 & 2 Data Modelling, Python Coding & Redshift SQL (Dataset - 10.4 GB)
Video - 00:7:7 mins
Project 3 & 4 - Administration & Move from OLTP to OLAP (Dataset - 1.3 GB)
Video - 00:6:51 mins
Project 5 - Redshift Performance Tuning (Dataset - 135 GB)
Video - 00:7:45 mins
Project 1 & 2 - PDF
Reading
Project 3 & 4
Reading
Project 5
Reading
New Features – Redshift, S3 & Athena
New Features - Introduction
Video - 00:1:7 mins
More
Announcements
Reviews
Course Info
Sign in
Sign up
Sign in
Don’t have an account?
Sign up
Remember me
Lost your password?
Sign up
Already have an account?
Sign in
Finish Course Early?
You have not completed all required lessons and assessments.
Your Progress
0%
Go Back to Course
Complete Anyway
