Complete Guide to Parallel and Concurrent Programming in Python

Course details

Parallel programming unlocks a program’s ability to execute multiple instructions simultaneously, increases the overall processing throughput, and is key to writing faster and more efficient applications. In this course, join instructors Barron and Olivia Chiu Stone as they introduce the basics of parallel programming in Python, providing the foundational knowledge you need to write more efficient, performant code. Barron and Olivia explain concepts like threading and mutual exclusion in a fun and informative way, relating them to everyday activities you perform in the kitchen. To cement the ideas, they demo them in action using Python. Each lesson is short and practical, driving home the theory with hands-on techniques.

Concepts

0. Introduction

01 - Learn parallel programming basics
02 - What you should know
03 - Exercise files

1. Parallel Computing Hardware

04 - Sequential vs. parallel computing
05 - Parallel computing architectures
06 - Shared vs. distributed memory

2. Threads and Processes

07 - Thread vs. process
08 - Concurrent vs. parallel execution
09 - Global interpreter lock - Python demo
10 - Multiple threads - Python demo
11 - Multiple processes - Python demo
12 - Execution scheduling
13 - Execution scheduling - Python demo
14 - Thread lifecycle
15 - Thread lifecycle - Python demo
16 - Daemon thread
17 - Daemon thread - Python demo

3. Mutual Exclusion

18 - Data race
19 - Data race - Python demo
20 - Mutual exclusion
21 - Mutual exclusion - Python demo

4. Locks

22 - Reentrant lock
23 - RLock - Python demo
24 - Try lock
25 - Non-blocking acquire - Python demo
26 - Read-write lock
27 - Read-write lock - Python demo

5. Liveness

28 - Deadlock
29 - Deadlock - Python demo
30 - Abandoned lock
31 - Abandoned lock - Python demo
32 - Starvation
33 - Starvation - Python demo
34 - Livelock
35 - Livelock - Python demo

6. Synchronization

36 - Condition variable
37 - Condition variable - Python demo
38 - Producer-consumer
39 - Producer-consumer threads - Python demo
40 - Producer-consumer processes - Python demo
41 - Semaphore
42 - Semaphore - Python demo

7. Barriers

43 - Race condition
44 - Race condition - Python demo
45 - Barrier
46 - Barrier - Python demo

8. Asynchronous Tasks

47 - Computational graph
48 - Thread pool
49 - Thread pool - Python demo
50 - Process pool - Python demo
51 - Future
52 - Future - Python demo
53 - Divide and conquer
54 - Divide and conquer - Python demo

9. Evaluating Parallel Performance

55 - Speedup, latency, and throughput
56 - Amdahl's law
57 - Measure speedup
58 - Measure speedup - Python demo

10. Designing Parallel Programs

59 - Partitioning
60 - Communication
61 - Agglomeration
62 - Mapping

11. Challenge Problems

63 - Welcome to the challenges
64 - Challenge - Matrix multiply in Python
65 - Solution - Matrix multiply in Python
66 - Challenge - Merge sort in Python
67 - Solution - Merge sort in Python
68 - Challenge - Download images in Python
69 - Solution - Download images in Python

Conclusion

70 - Additional resources
71 - Next steps