3D Gaussian Splatting · Evaluation & Survey

Implementing Evaluation Metrics from Scratch

Colab Notebook · ~40 min
Google Colab Notebook
Implementing Evaluation Metrics from Scratch
Python · ~40 min
Open in Colab
Lab Objectives
1
Implement PSNR from scratch using only NumPy and verify against skimage.
2
Implement SSIM with an 11×11 Gaussian window and verify it handles edge cases (zero-variance patches).
3
Use the lpips library to compute perceptual similarity with an AlexNet backbone.
4
Generate a controlled degradation experiment and plot PSNR/SSIM/LPIPS vs. degradation strength.
5
Demonstrate a case where PSNR is high but LPIPS is bad (perceptual mismatch).

Lab 2: PSNR, SSIM, and LPIPS

You will implement PSNR and SSIM from first principles, compare against scikit-image reference implementations, use the lpips library for perceptual metrics, and analyze how each metric responds to different types of image degradation.

Why Implement From Scratch?

Using library functions without understanding them is a reliability risk. When your evaluation pipeline returns unexpected numbers, you need to know whether the issue is in your rendering, your data pipeline, or your metric implementation. Building them from scratch makes debugging tractable.

What You'll Explore

You will construct image pairs with controlled degradation types — Gaussian blur (destroys high frequencies but preserves structure), additive noise (high MSE but structure preserved), and JPEG compression (block artifacts) — and observe how each metric responds differently to each degradation type.