Image Processing · Week 14
Modern Computer Vision
From Detection and Segmentation to generation, 3D, and World Models — a tour of what computer vision can do today.
Hyeongmin Lee · SeoulTech, Dept. of Electronic Engineering
From image to feature vector
CNN · ViT · CLIP · SSL — all ways to turn an image into a feature vector.
input
encoder
what we studied
feature
What can we do with this vector?
This week
What can we do?
Not on the exam. A quick tour of modern computer-vision tasks.
Object Detection
Predict each object's location (box) and class — a detection head on top of the encoder.
Segmentation
Label objects as pixel masks, not boxes. Foundation models like SAM (Segment Anything) generalize this to arbitrary objects.
From here on
Recent directions
Classification, Detection, Segmentation are the classics.
Now the actively-developing areas.
01 · Video Processing
Improving video
Super-resolution · frame interpolation · stabilization · compression
Super-Resolution
Reconstruct a high-resolution image from a low-resolution input. Drag the handle to compare.
Frame Interpolation
Synthesize the in-between frame from two adjacent frames. The core is motion estimation.
Video Stabilization
Estimate and remove unwanted camera shake.
Image compression — JPEG
Throw away the detail the eye won't miss.
DCT moves to the frequency domain; quantization drops high frequencies; entropy coding packs the rest losslessly.
Video compression
Don't store every frame — store the motion and the difference.
reference frame
I-frame: coded spatially, like JPEG. P/B-frame: keep only motion vectors + a small residual.
Same motion estimation as frame interpolation — here it removes temporal redundancy.
02 · 3D / 4D
3D Reconstruction
Reconstruct 3D structure from multiple photos and render new viewpoints.
Photos in, any view out
Optimize a 3D representation so that it reproduces the input photos.
The representation is fit by comparing its renders against the real photos.
Real-time 3D — Gaussian Splatting
A real-time successor to NeRF. Renders arbitrary viewpoints from a set of photos.
3D Gaussian Splatting for Real-Time Radiance Field Rendering · Kerbl et al. SIGGRAPH 2023
Add time — 4D
Beyond a static 3D scene: reconstruct scenes that change over time.
03 · Vision-Language
Image + Language — VLM
Take an image and answer in natural language. The line that follows CLIP.
It reads the situation in an image
Why is this image funny?
Obama is secretly pressing the scale with his foot, so the man weighing himself reads heavier. The people behind are laughing at the prank.
One image isn't enough
Is the person raising or lowering the glasses? A single frame can't tell.
Direction and change live between frames. Modern vision is moving from single images to video and time — the thread through the rest of this tour.
04 · Generation
Image & Video Generation
Generate images and video from text.
Generation by denoising
Start from pure noise and repeatedly remove a little — an image gradually emerges.
A network predicts the noise and subtracts it, step by step.
Stable Diffusion XL-Turbo · intermediate steps decoded
Text-to-Image
Generate an image from a text prompt. Below: generated directly.
Images: Stable Diffusion XL-Turbo (Stability AI, 2023) · generated for this deck
Text-to-Video
Generate video from text — it learns appearance and temporal change from data, but you can't act inside it.
05 · World Models
Action & Prediction — World Models
Generation you can act in — give an action, predict the next state.
Predict the next state
Take the current frame and an action, predict what happens next — then feed the prediction back in.
↻ the predicted frame becomes the next input (autoregressive)
Used to let a robot foresee outcomes before acting, or to generate interactive virtual environments.
Controllable generated worlds — Genie
Learns actions from unlabeled video, then generates a world you can navigate in real time.
Genie 3 — A new frontier for world models · Google DeepMind 2025
Robot world model — DreamDojo
Input joint actions, predict the resulting video — simulate and evaluate a policy before running the real robot.
DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos · NVIDIA GEAR 2026
Everything you just saw
is what our lab works on
ViViD Lab — Vision & Video Dynamics. We study how video changes over time — motion and dynamics.
Hyeongmin Lee · PhD, Yonsei · Adobe Research · Twelve Labs
Careers
Where graduates go
… and grad school, startups,
or anywhere visual AI is used.
Why this field
What makes it a good field
Research with code, data, and compute — no special lab equipment.
New methods appear every week. There is always something to work on.
A method in a paper can become a product within months.
Language vs Vision
Two kinds of AI, grounded in two different things.
Built on language, which humans invented. So it is tied to how we think and reason — cognition, neuroscience.
Built on images and video of the real world. So it is tied to understanding the world itself — its structure and dynamics.
Understanding people matters in both — but vision reaches further into the world.
And so —
Vision is everyone's
first language.
Before we speak, we see. It is how we — and our machines —
first meet the world.
ViViD Lab
Get in touch
Questions about research, undergraduate research, or career paths — always welcome.
Scan