Autoplay
Autocomplete
Previous Lesson
Complete and Continue
ETL Framework for Data Warehouse Environments​
Getting Started
Introduction (3:15)
What is covered (4:38)
Quic note on PPT_Docs_Questions (5:30)
The Architecture (10:27)
Different Stages (2:28)
Metadata Categories
Business Metadata (3:05)
Technical Metadata (3:00)
Operational Metadata (3:07)
ETL Framework - Process Flow
Process Flow- Infographic Explanation (8:20)
Data Sourcing
Data Sourcing (2:11)
Data Extraction Events 2 (5:20)
Schedulers (3:39)
CDC (3:07)
Pub Sub (2:03)
WebServices and APIs (2:02)
Data Sourcing - Classification
Data Sourcing Calssification_Push_Pull (2:52)
Architectural Classification (3:03)
Script Requirements for Data Sourcing
Script Requirements (4:45)
File Validation
File Validation (4:53)
Post File Validation (4:16)
Post File Validation Failure Notifications (2:17)
The Staging Layer
Staging Data (4:40)
Reference Data (6:10)
Examples of Reference Data (4:16)
Audit Columns for the Stating Reference Data (2:27)
Data Retention Requirements for the Staging Layer (3:56)
Partitions based on Data Retention Requirements (1:04)
DB Standards to be followed (1:21)
Business Validation Layer
Business Validation Layer (2:17)
Different Kind of Situations with in the data validations (4:25)
Indirect File Load Option (1:24)
Concurrent Workflow Process (4:49)
Partitions in Stage Layer and Business Validation Layer (13:39)
DataWarehouse Layer
Dimension Load (10:02)
Fact Load Process (9:08)
Fact Partitions/ Fact Summary Load (2:20)
Fact Summary Load (3:07)
Source File Managment/Archival (8:13)
Exception Handling/Error Handling
Data Model for Exception Handling (14:27)
Error Category (4:45)
Error Code (6:12)
Different Solutions for Exception handling (4:59)
Project Setup
Steps to download the sample project and the codebase (5:20)
Importing and Creting the Database metadata and the sample data (2:54)
Project explanation and aligning to the architecture (5:57)
Review of the project and the code base (9:11)
Extending the Operational Metadata's Data Model
Supporting Operational Metadata for Exception Handling (7:58)
Additional tables required to support the Exception Handling (11:46)
Why do we need so many tables? (5:48)
Error Handling Data Model
System Stage Table (7:03)
Data Source Metadata Table (9:50)
Data Source Detail Table (7:24)
Data Source System Table (5:50)
Job Details Table (2:44)
Changes to the Error Log Table (1:32)
Mapping examples
Modify the existing mapping to load the wrong data into the ERROR_LOG Table (10:23)
Other Possible ways of implemeting the Error handling (6:13)
Using PMREP Tables for Error Handling (9:59)
Audit, Balance and Control
Metadata Management (3:36)
Need for Operational Metadata (3:58)
Different technical components involved for Audit Balance and Control
Table Structure for Audit Balance and Control (4:43)
Structure of the Workflow with the re-usable sessions (3:43)
RUN_ID, the Unique Attribute (7:41)
The Re-usable Stored Procedure (6:01)
Sequence Generator with Trigger (5:49)
Workflow Setup (18:41)
Assignments and What else is left? (10:36)
Configuration Management
What is Configuration Management? (4:31)
What are the different subject areas under Configuration Management? (5:14)
Incident Management (2:13)
Change Management (2:58)
Release Management (2:24)
Capacity Management (1:39)
Service Level Management (0:47)
Disaster Recovery and Availability Management (1:45)
Why do we need so many tables?
Lesson content locked
If you're already enrolled,
you'll need to login
.
Enroll in Course to Unlock