Special offers now — see discounted courses.
day
:
hour
:
min
:
sec
See special offers
Azure Data Engineer Associate (DP-203) Cert Prep: 4 Monitor and Optimize Data Storage and Data Processing

Azure Data Engineer Associate (DP-203) Cert Prep: 4 Monitor and Optimize Data Storage and Data Processing

34mIntermediate2022-12-15

Authors

Noah Gift

Noah Gift

MLOps Expert | Solopreneur | Author | Adjunct Professor | CTO

Course details

Are you preparing for the Microsoft Azure Data Engineering (DP-203) exam, or seeking a better understanding of how to design and develop data processing? This course, the fourth in a series, can help you. Noah Gift, founder of Pragmatic A.I. Labs and a Python Software Foundation Fellow, covers how to monitor and optimize data storage and data processing. Noah first covers how to monitor data storage and data processing, showing how to: implement logging used by Azure Monitor; configure monitoring services; measure query performance; and more. He then details topics pertaining to optimizing and troubleshooting data storage and processing, including how to: rewrite user-defined functions; tune queries by using indexers and cache; handle skew in data and data spill; troubleshoot a failed Spark job and pipeline run.

Skills covered

Cloud StorageData EngineeringAzureNetwork AdministrationCloud PlatformsCert PrepNetwork and System AdministrationCloud ComputingData ScienceMicrosoft

Concepts

0. Introduction

  • 01 - Course introduction

1. Monitor Data Storage and Data Processing

  • 02 - Implement logging used by Azure Monitor
  • 03 - Configure monitoring services
  • 04 - Measure performance of data movement
  • 05 - Monitor data system pipeline cluster performance
  • 06 - Measure query performance
  • 07 - Schedule and monitor pipeline tests
  • 08 - Interpret a Spark directed acyclic graph (DAG)

2. Optimize and Troubleshoot Data Storage and Data Processing

  • 09 - Rewrite user-defined functions (UDFs)
  • 10 - Handle skew in data and data spill
  • 11 - Tune shuffle partitions pipelines
  • 12 - Optimize resource management
  • 13 - Tune queries by using indexers and cache
  • 14 - Troubleshoot a failed Spark job and pipeline run

Conclusion

  • 15 - Summary and next steps

Related courses

Related learn paths

About us

LyndaKade is a leading learning platform that helps people learn business, software, technology, and creative skills to achieve personal and professional goals.

Phone numberAparat ChannelTelegram SupportTelegram ChannelInstagram Page

All rights to this site belong to LyndaKade.

Terms of Service|Privacy Policy

نماد الکترونیک enamad در صورت اتصال با آی‌پی داخل کشور، نمایش داده خواهد شد.
logo-samandehi - لوگو ساماندهی
zarinpal
zibal