CS Fundamentals Trainer · Guides

Which image filter should I use? A decision guide

Start from what is wrong with the picture (noise, impulse specks, softness, uneven light) and pick the filter: box, Gaussian, median, unsharp mask, Laplacian or gradient. How to size the kernel, why sizes scale with resolution, what separability saves, how to treat the border, and the common mistakes.

9 min read

Most image filters are easy to apply and hard to choose. A box blur, a Gaussian, a median, an unsharp mask and a Laplacian all take a few lines of code, and each one fixes one problem while making another worse. This guide is a decision guide: start from what is wrong with the image, pick the filter that matches, then size it and check the border. It does not derive the filters; it tells you when each one is the right tool and what it costs.

1. Ask what is wrong first

Before choosing a filter, name the problem in one sentence. Almost every real case is one of these:

What you seeWhat it really isFamily of fix
Fine grain over the whole imageNoise that varies a little at every pixelSmoothing (average-like)
White and black dots, a few pixelsImpulse noise ("salt and pepper")Median
The picture looks softEdges lost their contrastSharpening
One side is darker than the otherUneven illumination (shading)Large lowpass, then divide
Everything is dull or too darkNarrow range of intensitiesPoint operation (see the point-versus-neighbourhood guide)
You want only the big objectsSmall details are irrelevantLowpass, then threshold

Two of these rows are not about filters at all. If the intensity range is the problem, a tone curve fixes it, and no kernel will.

Diagram: Choosing a filter from the symptom

2. Smoothing noise: average or rank?

An average-like filter (box, Gaussian) replaces each pixel by a weighted mean of its neighbours. If the noise at each pixel is independent, averaging n pixels with equal weights divides the noise standard deviation by √n. A 3 × 3 box averages 9 pixels, so the noise standard deviation drops to a third (a measured example: noise of standard deviation 10 becomes 3.3 after a 3 × 3 box). The price is that everything else is averaged too, so edges and thin lines blur.

A rank filter (median, maximum, minimum) sorts the neighbourhood and takes one value. The median ignores extreme values instead of mixing them in. Take a 3 × 3 window with five values of 100 and four of 255 (salt noise): the mean is 168.9, which invents a new brightness that was never there, but the median is 100, which is a clean value. The rule is that the median survives as long as fewer than half of the pixels in the window are corrupted: 4 of 9 is fine, 5 of 9 is not.

BoxGaussianMedian
Kindlinearlinearnonlinear (rank)
Best againstmild random noise, quick testsrandom noise, general smoothingimpulse noise (salt and pepper)
Edgesblur themblur them, less abruptlystep edges stay close to sharp
Output valuesnew values (averages)new valuesalways one of the window's values
Separableyesyesno
Watch out forfavours horizontal and vertical directionsneeds a larger kernel than a box for the same blurthin lines narrower than half the window disappear

If you apply an averaging filter to salt-and-pepper noise, each bright dot is spread into a grey stain around it instead of removed. If you apply a median to mild Gaussian-like noise it works, but an averaging filter of the same size usually reduces that kind of noise more.

3. Box or Gaussian?

For general smoothing prefer the Gaussian:

A box kernel is still fine for quick experiments, and when you need the cheapest possible blur (a running sum can do it).

4. Choosing the size

The size is the decision that most often goes wrong, and it has one governing rule: the kernel must be larger than the details you want to remove, and smaller than the details you want to keep.

5. Sharpening: which tool?

ToolWhat it doesStrengthWeakness
Unsharp mask / highboostadds k × (original − blurred)simple, one knob for strength (k) and one for scale (σ)large k makes halos and can clip
Laplaciansecond derivative, added back with the right signvery fine detail, isotropicamplifies noise strongly
Sobel gradientfirst derivative magnitudestrong edges, built-in smoothingthick edges, not meant to be added back as it is
Laplacian × smoothed gradientdetail only where edges are strongquiet flat areas, so little noisemore steps to tune

Every derivative-based sharpener amplifies noise, since noise is also a sudden change. Two habits help: smooth a little before sharpening (unsharp masking does this by construction, since the mask uses a blurred copy), and use the gradient as a mask when the image is noisy.

The sign matters for the Laplacian: with the kernel that has −4 at the centre, the sharpened image is f − ∇²f. With a +4 centre it is f + ∇²f. Adding with the wrong sign darkens edges instead of sharpening them.

6. Cost and speed

The cost per pixel of a linear filter with an m × n kernel is about mn multiplications, or m + n if the kernel is separable. On a 1024 × 1024 image with an 11 × 11 kernel that is about 127 million multiplications against 23 million, an advantage of 5.5. For a square separable kernel the advantage is about half the kernel width, so it grows with the size: 10.5 for 21 × 21, 256 for 512 × 512.

7. The border

Near the border, part of the kernel falls outside the image, so something must be supplied there.

8. Mistakes to avoid

  1. Using an averaging filter on impulse noise. Use a median.
  2. Using a kernel size picked for another resolution.
  3. Sharpening before denoising. The sharpener amplifies the noise, and the denoiser then has a harder job.
  4. Normalising wrongly: smoothing kernels should sum to 1 (or the brightness changes), while derivative kernels sum to 0 (and the result needs rescaling for display).
  5. Forgetting that correlation and convolution differ for asymmetric kernels. For box and Gaussian kernels it does not matter, but for a directional derivative it does.
  6. Trusting a result that looks fine on one image. Check a flat area (should stay flat), an edge (should not ring or shift) and the border.

9. Starting points, not rules

The settings below are first guesses. The right values depend on the image content, the noise level, the resolution and the implementation, so look at the output.

GoalFirst choiceTypical settings
Remove salt-and-pepperMedian3 × 3, then 5 × 5 if dots are larger
Reduce mild grainGaussianσ of 0.5 to 2 pixels, size ⌈6σ⌉ rounded to odd
Remove small details, keep big shapesGaussian, then thresholdσ larger than the details to remove
Estimate illuminationGaussianσ about the size of the largest scene feature, kernel support about ⌈6σ⌉, then divide
Sharpen a soft photoUnsharp maskσ about the width of the blur, k between 0.5 and 2
Sharpen a noisy imageSmoothed-gradient-masked Laplaciansmooth the gradient with a 5 × 5 box

Further reading

Practise this topic: Computer Vision

More guides