ETL in Python and SQL

1h 21mIntermediate2024-03-06

Authors

Jennifer Ebe

Course details

Data is increasing exponentially as the world continues to generate an immense amount. In this course, a data engineer with over 5 years of experience, Jennifer Ebe, will show you how to build systems that gather data from various sources, transform it and store data so you can derive insights from it effectively. We will go over what ETLs are and how you create them. We will learn how to use Python to extract and transform data, we will explore our data using pandas and SQL. And finally, load data into our target system. You will also get practice scheduling ETL jobs using Python. Each chapter comes with helpful, hands-on challenges to help you practice what you learn.

Skills covered

SQLDatabase AdministrationData EngineeringDatabase DevelopmentPythonDatabase ManagementData AnalysisProgramming LanguagesData ScienceBusiness Analysis and StrategyBusiness Software and ToolsOpen SourceSoftware DevelopmentOne-Off

Concepts

0. Introduction

01 - Create an ETL in Python and SQL
02 - Tools used in this course

1. What Are ETLs

03 - What are ETLs and how do you create them
04 - ETL process overview
05 - Exploring your data with pandas (Python) and SQL
06 - Understanding your data
07 - Challenge - Reading data using Python
08 - Solution - Reading data using Python

2. Extracting and Transforming Data with Python

09 - Loading data from different sources
10 - Extracting your data
11 - Cleaning, preprocessing data, and data formatting
12 - Standardization, handling duplicates, and missing values
13 - Challenge - Load the data using SQLAlchemy and pandas
14 - Solution - Load the data using SQLAlchemy and pandas

3. Loading Data into Target Systems

15 - Introduction to data warehouses and data lakes
16 - Loading data into relational databases
17 - Data quality checks and validation with SQL
18 - Challenge - Transform the data and remove duplicates and nulls
19 - Solution - Transform the data and remove duplicates and nulls

4. Automating ETL Jobs - Scheduling ETL Jobs with Python

20 - Querying your data with SQL
21 - Scheduling ETL jobs with Airflow - Part 1
22 - Scheduling ETL jobs with Airflow - Part 2
23 - Challenge - Load the data into a database and automate
24 - Solution - Load the data into a database and automate

Conclusion

25 - Expand your knowledge of ETLs