Data Versioning, Lineage, and Quality Monitoring for AI

Discover the importance of data versioning and how it impacts ML and AI workflows. Instructor Janani Ravi outlines key concepts such as snapshots, lineage, branching, and how to manage data versions effectively. Explore how to use data version control (DVC) to initialize Git, track files, and version data more efficiently. Get introduced to data lineage in Microsoft Fabric and uncover techniques and best practices to track lineage. Understand common issues with data and models, including processing, schema management, data loss, and bias, and learn how to monitor these aspects for quality. Along the way, learn how to track metrics that help ensure data and model integrity and performance. Whether you're a data scientist, engineer, or currently working in data management, this course equips you with the skills you need to maintain high standards of data versioning and quality monitoring in your projects.