CS Fundamentals Trainer · Guides
Filtering in space and in frequency: one operation, two views
Convolution as multiplication of spectra and the impulse as a constant. The sum of a kernel is its gain at zero frequency, highpass, bandreject and bandpass kernels built from one lowpass, why a wide Gaussian has a narrow spectrum, aliasing and filtering before shrinking, and when to compute in the frequency domain.
8 min read
Every spatial filter has a second description that is often easier to reason about: what it does to the frequencies in the image. Edges, noise and fine texture live in the high frequencies; walls, skies and gentle shading live in the low ones. Blurring removes the former, sharpening boosts them, and "lowpass", "highpass" and "bandpass" are names for what a filter keeps. This guide connects the two views, so that you can predict the effect of a kernel from its shape and design one from the effect you want.
1. The two facts that connect everything
- Convolution in the spatial domain is multiplication in the frequency domain (and the other way round). Filtering an image with a kernel w means multiplying the image's spectrum by the spectrum of w.
- An impulse of strength A in the spatial domain is a constant A in the frequency domain (and the other way round). A kernel with a single 1 and zeros elsewhere, the unit impulse, has a flat spectrum equal to 1 and passes everything unchanged.
Everything else in this guide follows from these two, together with the fact that a picture can be written as a sum of sinusoids of different frequencies and amplitudes. Slowly varying regions need low-frequency sinusoids, and sharp transitions need high-frequency ones.
2. Frequency as a picture
A zone plate is a test image of concentric rings that get narrower away from the centre, so its local frequency increases with the distance from the centre. Filter it and the result shows the filter's frequency behaviour as a picture:
- a lowpass filter keeps the centre rings and fades the outer ones;
- a highpass filter does the opposite;
- a bandpass filter keeps a ring-shaped band;
- a bandreject filter removes a band and keeps the rest.
A zone plate is a good first test for any kernel you write. If the output pattern is square instead of round, the kernel is not isotropic (a separable kernel built from a 1-D profile gives a squarish pattern, while rotating the same 1-D profile about the centre gives a round one).
3. The sum of the coefficients is the gain at zero frequency
The "DC" component (frequency zero) is the average level of the image. A kernel multiplies it by the sum of its coefficients.
| Kernel | Sum | At zero frequency | Consequence |
|---|---|---|---|
| Normalised box or Gaussian | 1 | gain 1 | flat regions keep their value |
| Laplacian, Sobel, highpass | 0 | gain 0 | flat regions give zero; the result has negative values and must be rescaled to display |
| Unsharp kernel (impulse + k × highpass) | 1 | gain 1 | brightness is kept while detail is boosted |
This one fact explains why smoothing kernels are normalised, why derivative kernels sum to zero and why sharpening by adding a derivative image back to the original keeps the tonal range.
4. Building the four filter types from one lowpass filter
Let lp be a lowpass kernel and δ the unit impulse with the same centre. In the frequency domain "1 minus the lowpass" is a highpass, and 1 is the transform of δ, so:
| Filter | Kernel | Why |
|---|---|---|
| Lowpass | lp | given |
| Highpass | δ − lp | the spectrum 1 − H(u) |
| Bandreject | lp₁ + (δ − lp₂) | a lowpass and a highpass with different cutoffs |
| Bandpass | δ − bandreject | the complement of the band that is rejected |
A concrete case: the 3 × 3 box lowpass has 1/9 everywhere. Subtracting it from the impulse gives a highpass with 8/9 at the centre and −1/9 at the other eight places, which sums to zero. The unsharp mask f − blur(f) is exactly the output of that highpass, so unsharp masking is "add a highpass-filtered copy of the image to the original". Adding k times that highpass to the impulse gives the sharpening kernel; with k = 1 and the 3 × 3 box it is 17/9 in the centre and −1/9 elsewhere, which sums to 1.
5. The Gaussian pair
A Gaussian in space has a Gaussian spectrum, and the two widths are inversely related: a wide kernel (large σ) has a narrow spectrum. That is the reason a bigger σ blurs more, since it passes a narrower band of low frequencies. Three consequences follow:
- Because the spectra multiply, convolving two Gaussians gives a Gaussian whose variance is the sum of the two variances: σ = √(σ₁² + σ₂²). So σ = 3 followed by σ = 4 is σ = 5.
- A Gaussian has no ripples in its spectrum: it removes high frequencies smoothly, without leaving some of them behind. This is why it blurs without ringing.
- A box kernel is different. Its spectrum has side lobes that let some high frequencies through again and flip the sign of others, which is why box blurs can leave faint artefacts and favour horizontal and vertical directions.
6. Where sampling comes in
Sampling an image on a grid can represent frequencies only up to a limit set by the pixel spacing. Frequencies above it do not vanish; they fold back and show up as false lower frequencies (aliasing, moiré, jagged diagonals). The practical rules follow directly:
- Lowpass before you shrink. Reducing an image by a factor of 2 keeps only half the frequencies, so remove the upper half first, with a Gaussian of suitable σ, and then discard pixels. Dropping pixels without filtering aliases the detail.
- A blur applied before the image is sampled (the lens, the optical filter in front of a sensor) prevents aliasing. After sampling, aliased content is mixed with real content and a later blur cannot separate them; it can only soften both. When you shrink an image that is already sampled, filter first and then drop pixels.
7. Space or frequency: which to compute in?
Both give the same result when done right, so the question is cost and convenience.
- Direct filtering costs about mn operations per pixel for an m × n kernel, or m + n for a separable one.
- Filtering in the frequency domain transforms the image once, multiplies by the filter's spectrum and transforms back. Its cost per pixel grows only slowly with the image size and does not depend on the kernel size, so it often wins when the kernel is large and not separable. It is not automatically faster for every large kernel.
- Whether it wins depends on the image dimensions, on the padding (the transform works on the padded size, M + m − 1 by N + n − 1 at least, and often rounded up to a size the FFT handles well), on the kernel structure (a large separable Gaussian needs only m + n operations per pixel directly, and the 1-D passes can beat the FFT), on how many images share one kernel (its transform can be computed once and reused), and on the implementation (an optimised direct convolution, a GPU or a good FFT library change the break-even point a lot, and so does memory use and numerical precision).
- For small kernels (a few pixels up to perhaps a dozen or so) and for separable Gaussians, direct filtering is usually simpler and as fast. There is no universal break-even size: measure on your data and your library.
Two details need care when computing in the frequency domain:
- The transform treats the image as periodic, so a filter will wrap the left edge around to the right. To avoid it, pad the image to at least M + m − 1 by N + n − 1 (the same size as the "full" convolution) before transforming.
- The result of a zero-sum highpass has negative values, so rescale before display.
8. Misconceptions
- "Edges are high frequency, so a highpass finds edges and nothing else." Noise is also high frequency. A highpass finds edges and noise together, which is the reason sharpening amplifies noise and denoising blurs edges. Frequency alone cannot tell the two apart.
- "A bigger kernel always means a stronger filter." For a Gaussian, beyond about 6σ the extra size changes nothing. The strength is set by σ.
- "Lowpass means the image gets darker." Only if the kernel is not normalised or if zero padding brings black in at the border. A normalised lowpass keeps the average level.
- "Convolution and correlation are the same." They match for symmetric kernels (box, Gaussian, Laplacian), and differ by a 180° rotation otherwise.
9. Self-check
- A kernel has the coefficients 1/9 in all nine places, and another has 1/16 times [[1,2,1],[2,4,2],[1,2,1]]. What is the gain of each at zero frequency?
- You want a filter that passes only a middle band of frequencies. List the kernels you combine and the operations between them.
- You convolve the image with a Gaussian of σ = 2 and then with one of σ = 2 again. What single σ gives the same result, and how large (at least) must the kernel be?
- Why can you not remove noise with a highpass filter?
Answers: both have gain 1; bandpass = δ − (lp₁ + (δ − lp₂)); σ = 2√2 ≈ 2.83, so a size of ⌈6σ⌉ = 17, which is already odd; because noise lives in the high frequencies too, a highpass keeps it.
Further reading
- Digital Image Processing, 4th edition, chapter 3 (the comparison of spatial and frequency domains and filters built from lowpass ones) and chapter 4.
- Band-pass filter, Convolution and Gaussian blur.
- Cross-correlation.
Practise this topic: Computer Vision