Execute and Evaluate Hugging Face AI Models

Course details

As organizations increasingly adopt AI solutions, leveraging pretrained models from easy-to-use platforms like Hugging Face has become essential for rapid and effective deployment. However, the ease of implementation can mask significant risks when model evaluation is insufficient. In this course, join expert data scientist and instructor Kendall Ruber as she outlines critical skills for strategically selecting, implementing, and rigorously evaluating pretrained models to ensure reliable outcomes for specific organizational use cases.

Learning objectives
Download and apply pretrained Hugging Face models to a dataset.
Define various machine learning accuracy metrics and their myriad applications, plus limitations.
Use misclassification analysis and feature importance to fine-tune your model and improve performance metrics.
Select the best model for your unique situation and prepare it for deployment.

Concepts

Introduction

Speed meets strategy - Why pretrained models change everything
From scratch to pretrained - Understanding the shift and staying critical
Quick start - A look at our final model

Model Selection

Navigating Hugging Face's model portfolio
Problem set introduction
Selecting a relevant model
Preparing your development environment

Effective Feature Engineering

Crafting a relevant dataset
Exploratory data analysis
Setting model parameters
Preprocessing assumptions
Run model

Model Evaluation

Defining success metrics
Out-of-the-box metrics
Hugging Face Evaluate library
Going deeper - Results by class
Calibration thresholds
Going deeper - Misclassification analysis
Feature importance plots
Try it again - Running the next iteration

Conclusion

Finalize model for production
Conclusion

Execute and Evaluate Hugging Face AI Models

Authors

Kendall Ruber