Multimodal Programming Interfaces
1h 24mAdvanced2025-10-20
Authors
Pragmatic AI Labs
Alfredo Deza
Course details
Explore the future of software development through multimodal programming interfaces—a new paradigm that integrates voice commands, text input, visual elements, and gestures to create more natural and efficient developer workflows.
Instructor Alfredo Deza examines the principles behind multimodal interaction design and demonstrates how to set up a multimodal dev environment. Alfredo then shows how to build interfaces that accept and process multiple input types. Along the way, learn about the technical architecture required to support such systems, and discover how to leverage advanced patterns and tools like Playwright Model Context Protocol (MCP). Whether you’re a software engineer, a tool developer, a technologist, or just a person interested in advanced vibe coding, this course can help you develop more intelligent and intuitive programming environments.
Instructor Alfredo Deza examines the principles behind multimodal interaction design and demonstrates how to set up a multimodal dev environment. Alfredo then shows how to build interfaces that accept and process multiple input types. Along the way, learn about the technical architecture required to support such systems, and discover how to leverage advanced patterns and tools like Playwright Model Context Protocol (MCP). Whether you’re a software engineer, a tool developer, a technologist, or just a person interested in advanced vibe coding, this course can help you develop more intelligent and intuitive programming environments.
Concepts
Understanding Multimodal AI Programming
- Course introduction
- What is multimodal programming
- Setting up a multimodal dev environment
- Your first screenshot-to-code conversion
- Module summary
Visual Input Techniques
- Introduction to visual inputs
- Prompt engineering with visual context
- Introduction to GitHub Copilot with Visual Studio Code
- Developing with GitHub Copilot
- Module summary
Web-Development with Multimodal Programming
- Introduction to web development with AI
- Overview of what we will build
- Using live reload and developer tools
- Setting up the development environment
- Module summary
Applying Multimodal Techniques for Web Development
- Multimodal techniques for web development
- Designing with documentation
- Iterating over first changes
- Using images for iteration
- Challenges with iteration
- Adding automation and validation
- Module summary
Advanced Patterns in Multimodal Programming
- Introduction to advanced patterns
- Introduction to MCP
- Overview of Playwright
- Using Playwright MCP
- Overview of what we have built
- Course summary
Related courses
- OpenAI API: Multimodal Development with GPT-4o
- Programming Generative AI: From Variational Autoencoders to Stable Diffusion with PyTorch and Hugging Face
- AI Engineering Use Cases and Projects on AWS: Production-Grade LLM Systems
- Azure AI for Developers: Azure AI Speech
- Advanced RAG Applications with Vector Databases
- Building with the Claude API by Anthropic
- Multimodal Prompting with Google's Project Gemini
- Multimodal Communication in Sales
Related learn paths
- Building Generative AI Skills for Web Developers
- A Developer's Guide to Google Gemini
- Starting Your Career in Tech: Data Science
- Build AI Products Using Azure AI Services in Your Development Lifecycle
- Getting Started with Prompt Engineering
- Develop Your AI Skills with Google Gemini and Google Cloud Platform
- Understanding Generative AI for Tech Leaders
- Master Infraworks for Civil Infrastructure