Course Content
Introduction – Data Engineering Volume 1 on AWS
0/2
Course Introduction and Resources
24:28
Course Introduction and Course Contents
12:11
Data Engineering Introduction
0/3
Data Engineering introduction, OLTP & OLAP
33:10
Data Mart & Data Mesh
05:31
Data Lake, Data Lakehouse, DWH
00:00
AWS Distributed Storage – S3 (Simple, Storage, Service) for Data Engineers
0/20
Introduction 1
14:38
Introduction 2
22:49
Basics
05:43
Basics Hands-on
18:53
Versioning
13:06
Encryption
05:51
Storage Class
20:18
Multipart Upload
12:51
Lifecycle Policies
15:04
Cross Region Replication
10:13
Mountpoint
09:21
Security – S3 Identity Based Policy
19:03
Security – S3 Bucket Policy
08:29
Bucket Policy with VPC, IP address, VPCE
03:50
Access Point
16:27
Object Lambda
18:55
Pre-signed URL
04:33
Performance Considerations
05:31
Pricings
13:00
Architectural Patterns using S3
07:25
Data Modelling – Normalization, ER Diagram, Dimensional Modelling
0/9
Highlights
13:09
Data Modelling Introduction
17:15
Normal Forms 1NF 2NF 3NF
28:01
Relations: one-to-one, one-to-many, many-to-one, many-to-many
08:51
Dimensional modelling – Facts, Dimensions & Grains
24:39
Grains Exercise
09:19
Dimensional Modelling Technique
15:00
Types of Fact & Dimension Tables
10:10
Data Virt Semantic Presentation Layers
13:24
Data Warehouse on AWS – Redshift Infra
0/8
Redshift Infra
19:36
Redshift Infra HandsOn
21:42
Redshift Architecture – Zone Map, Columnar Storage
15:02
Cluster Resize – Elastic & Classic
08:32
Cluster Resize – HandsOn
05:03
Cluster Pause & Rename
04:31
Snapshot & Backup
07:43
Redsfhit Infra Conclusion
03:04
Redshifts Objects
0/10
Querying, Connection, RSQL, QEV2
16:10
Query Editor & RSQL setup
17:54
Object Hierarcy, tables hands-on
19:14
Data Types Hands-on
14:20
Table operations Hands-on
12:27
Redshift ACID, Locks, Isolation Level
12:05
Implement Transactions
10:01
AccessShareLock & ShareRowExclusiveLock HandsOn
08:50
Redshift SUPER datatype
14:29
Section Summary
05:02
Redshift Deep Dive
0/12
Distribution Key & Style, Sort Key
29:35
Column Compression
06:42
Modify Dist Sort Key, Compression HandsOn
19:18
COPY Command Theory
06:33
COPY Command HandsOn
13:36
UNLOAD Command
07:16
AWS DMS – Move from OLTP to DWH
08:10
DMS – Setup Source OLTP & Python application
19:14
Setup DMS Instance, Endpoint, Task
15:28
DMS Task – OLTP to DWH
28:44
Table Maintenance – VACUUM & ANALYZE
09:46
Vacuum & Analyze HandsOn
11:25
Resdhift Features
0/6
Materialized View (MV)
10:27
MV HandsOn
13:59
Query Federation
11:19
Redshift Spectrum
14:18
Streaming Ingestion
24:05
Redshift Feature Use Cases
28:36
Redshift Query Tuning
0/8
Query Execution
22:32
EXPLAIN Plan & System Joins
23:32
System Joins HandsOn
05:29
Data RE-distribution
09:01
EXPLAIN & RE-distribution HandsOn
16:18
Query Tuning Exercise – Part 1
31:17
Query Tuning Exercise – Part 2
24:46
Query Tuning Exercise – Part 3
24:27
Redshift Workload Management (WLM)
0/6
WLM Intro & Query Queue
09:02
Concurrency Scaling, Short Query Acceleration
07:19
Configure WLM HandsOn
16:44
Create Query Queue HandsOn
15:50
Query Queue In Action
00:00
Concurrency Scaling In Action
15:33
Redshift Security- RBAC, CLS, RLS, Dynamic Data Masking (DDM)
0/9
Users, Roles, RBAC
11:29
Users, Roles, RBAC HandsOn
12:14
Row & Column Level Security (RLS & CLS)
11:25
Multiple RLS Policies HandsOn
13:02
CLS HandsOn
05:00
Combine RLS & CLS HandsOn
04:44
Dynamic Data Masking
07:00
Track Users, Roles, CLS and RLS
10:48
Audit Logging
11:32
Monitoring in Redshift
0/2
Monitor Redshift using Console
18:00
System Views for Monitoring Queries, Redshift Objects, Configuration Parms
20:50
Reshift Serverless
0/7
Introduction to Redshift Serverless
12:12
Create & Delete Redshift Serverless Resources
06:39
COPY & UNLOAD in Serverless
ZeroETL Integration Setup
24:12
ZeroETL in Action
12:59
Query Tuning Similarities
Migrate from Provisioned to Serverless
Detailed Redshift Pricing
0/3
Redshift Pricing Components
Pricing Example – Provisioned, Serverless, Concurrency Scaling, Spectrum
AWS Pricing Calculator
Redshift Additional Information
0/4
Redshift Integration with AWS Services
Redshift & Snowflake Comparison
Redshift Best Practices
Redshift Limitations and Challenges
AWS Metadata Repository – Glue Data Catalog
0/4
AWS Glue Catalog – Theory
Glue Catalog – Setup Data Stores & IAM Roles HandsOn
Store Aurora metadata in Glue Catalog
Store S3 and Redshift metadata in Glue Catalog
Data Governance using AWS Lake Formation
0/7
Lake Formation Introduction
Permission Flow HandsOn 1
Permission Flow HandsOn 2
Lake Formation – Tag Based Access Control (LF-TBAC)
LF-TBAC HandsOn
LF – Data Filtering
LF Clean Up (Please complete this)
Data Lakehouse on AWS – Athena
0/16
Athena Introduction
16:41
Athena Intro Hands On
10:44
Athena SerDe, File & Row format
12:51
SerDe, Format, CTAS Hands On
12:52
UNLOAD, Prepare & Execute, Query JSON
08:40
UNLOAD, Prepare & Execute, Query JSON Hands On
17:27
Schema Evolution, JSON_EXTRACT
13:54
Iceberg, ACID
26:19
Athena Partitioning & Bucketing
19:28
More DDL Commands
06:12
Athena WLM Theory
08:03
Workgroup HandsOn
11:32
Capacity Reservation HandsOn
04:06
Performance Tuning Theory
13:29
Athena Pricing & Performance Tuning
15:46
Architectural Patterns using Athena
08:30
Big Data Warehouse – HIVE
0/11
Hadoop Theory
File Formats
Hive Architecture & Components
Hive CLI
Data Types, databases, tables, File & Row Format, Hive SerDe
Hive Databases hands-on
Hive Tables hands-on
Partitioning & Bucketing
Partitioning & Bucketing hands-on
Load, insert, ACID, Materialized Views etc
JOINs, Locks, Configuration Parameters
Projects – Redshift, Athena, DataModelling, Pythin (Total Dataset Size – 150 GB)
0/3
Project 1 & 2 Data Modelling, Python Coding & Redshift SQL (Dataset – 10.4 GB)
Project 3 & 4 – Administration & Move from OLTP to OLAP (Dataset – 1.3 GB)
Project 5 – Redshift Performance Tuning (Dataset – 135 GB)
New Features – Redshift, S3 & Athena
0/1
New Features – Introduction
Data Engineering on AWS Vol 1 – OLAP & Data Warehouse
Previous
Next
0% Complete