Generative · Guide
Generative AI: Images, Audio, and Multimodal
Beyond text — diffusion models, Stable Diffusion, Whisper, text-to-speech, vision-language models.
Generative AI creates new content — images, audio, and video — not just text. This guide covers diffusion models and how they generate images, Stable Diffusion, image-generation workflows, speech recognition with Whisper, text-to-speech, vision-language models, and responsible generative media.
Generate your own lesson →What you'll learn
- AI Image Generation Workflows
- Diffusion Models Explained
- Multimodal Prompting
- Responsible Generative Media
- Speech Recognition with Whisper
- Stable Diffusion and Latent Diffusion
- Text-to-Speech and Audio Generation
- The Hugging Face Diffusers Library
- Video Generation Concepts
- Vision-Language Models
Lessons in this guide (10)
AI Image Generation Workflows
AI image generation is more than typing a prompt: real work uses a handful of workflows. Text-to-image creates an image from a description; image-to-image transforms an existing picture; inpainting edits a masked region; and outpainting ext
Diffusion Models Explained
A diffusion model generates images by learning to reverse a noising process. During training it takes real images, adds random noise in small steps until they are pure static, and learns to predict and remove that noise. To generate, it sta
Multimodal Prompting
Multimodal prompting is the craft of combining images (or other media) with text instructions to get the most from a vision-language model. The same principles as text prompting apply — be specific, give examples, ask for a format — plus vi
Responsible Generative Media
Responsible generative media is the practice of creating and deploying AI-generated images, audio, and video in ways that avoid harm. The main risks are deepfakes and misinformation, non-consensual or infringing content, and amplified bias.
Speech Recognition with Whisper
Whisper is an open speech-recognition model from OpenAI that transcribes audio into text and can translate speech into English. It is a transformer trained on a very large, diverse set of audio paired with transcripts, which makes it robust
Stable Diffusion and Latent Diffusion
Stable Diffusion is a latent diffusion model: instead of denoising in pixel space, it runs the diffusion process in a compressed latent space, which is what makes high-resolution image generation fast enough to run on consumer GPUs. Its thr
Text-to-Speech and Audio Generation
Text-to-speech (TTS) turns written text into natural-sounding spoken audio, and modern neural TTS is realistic enough to be hard to distinguish from a human. It typically works by converting text into an acoustic representation and then int
The Hugging Face Diffusers Library
Diffusers is Hugging Face's library for running and building diffusion models like Stable Diffusion in Python. Its central abstraction is the pipeline, which bundles a model's components — the denoising network, the VAE, the text encoder, a
Video Generation Concepts
AI video generation extends image diffusion into time: a model generates a sequence of frames from a text prompt (or a starting image), and the central challenge is temporal consistency — keeping objects, motion, and style coherent across f
Vision-Language Models
A vision-language model (VLM) is a model that understands images and text together, so you can show it a picture and ask questions in natural language. It works by pairing a vision encoder that turns an image into features with a language m
Frequently asked questions
What is generative AI?
Generative AI is a class of models that produce new content — text, images, audio, video, or code — by learning the patterns of their training data and sampling new outputs that follow those patterns.
How do diffusion models generate images?
A diffusion model learns to reverse a noising process: it starts from random noise and, guided by your prompt, denoises step by step into a coherent image.
What is a vision-language model?
A vision-language model (VLM) understands images and text together — you can show it a picture and ask questions about it, or have it caption, describe, or reason over visual content.