Kernels & convolution

A single texture read tells a pixel about one spot. Many effects need the neighborhood: a blur averages it, a sharpen exaggerates the difference from it, an outline looks for a sudden change across it. All of them are the same operation, convolution. For each output pixel, read the texels around it, multiply each by a weight and add the results. The grid of weights is the kernel.

A 3×3 box blur

The simplest kernel gives the nine texels around the pixel the same weight, 1/9:

// at file scope:
uniform sampler2D uImage;
// inside main():
vec2 texel = 1.0 / vec2(textureSize(uImage, 0));
vec3 sum = vec3(0.0);
for (int y = -1; y <= 1; y++) {
    for (int x = -1; x <= 1; x++) {
        sum += texture(uImage, uv + vec2(x, y) * texel).rgb;
    }
}
vec3 blurred = sum / 9.0;

Two rules about the weights tell you what a kernel does before you run it. When they add up to 1, a flat area of color comes out unchanged, so the kernel keeps brightness: blurs and sharpens. When they add up to 0, a flat area comes out black and only changes survive: edge detectors.

Kernels read past the edge of the image near its borders. The wrap mode decides what they find there, and clamp to edge is the usual choice; with repeat, the opposite border bleeds in.

Sharpen

A sharpen kernel weights the center above 1 and subtracts the neighbors, so it pushes each pixel away from its surroundings. With the four direct neighbors:

vec3 center = texture(uImage, uv).rgb;
vec3 around = texture(uImage, uv + vec2(texel.x, 0.0)).rgb
            + texture(uImage, uv - vec2(texel.x, 0.0)).rgb
            + texture(uImage, uv + vec2(0.0, texel.y)).rgb
            + texture(uImage, uv - vec2(0.0, texel.y)).rgb;
vec3 sharp = center * 5.0 - around; // weights 5 and -1 four times: they sum to 1

The result can go below 0 and above 1, which shows as dark and bright halos along edges. Mix it with the original to control the strength.

Edges with Sobel

The Sobel operator is a pair of 3×3 kernels that estimate how fast brightness changes horizontally and vertically. Each weights the middle row or column twice as much as the corners, and each sums to 0:

float luma(vec2 p) { return dot(texture(uImage, p).rgb, vec3(0.2126, 0.7152, 0.0722)); }

// inside main():
float gx = 0.0;
float gy = 0.0;
for (int y = -1; y <= 1; y++) {
    for (int x = -1; x <= 1; x++) {
        float v = luma(uv + vec2(x, y) * texel);
        gx += float(x) * (y == 0 ? 2.0 : 1.0) * v; // right column minus left column
        gy += float(y) * (x == 0 ? 2.0 : 1.0) * v; // top row minus bottom row
    }
}
float edge = length(vec2(gx, gy)); // 0 on flat areas, large on edges

Running it on the luminance instead of each channel finds edges the eye sees and costs one set of reads. The direction of vec2(gx, gy) is the direction of steepest change, perpendicular to the edge. Run it on depth or normals instead of color and it finds the silhouettes and creases that outline effects draw.

Gaussian blur and separable passes

A box blur has a square, hard-edged footprint, which turns small bright details into visible squares. A Gaussian blur weights texels by a bell curve of their distance, so the center counts most and the weights fade smoothly to nothing. Rows of Pascal’s triangle are a quick integer approximation: 1 2 1 (divided by 4), 1 4 6 4 1 (divided by 16).

The Gaussian has a property that makes large blurs affordable: it is separable. The 2D kernel equals a horizontal 1D kernel followed by a vertical one. A 9×9 blur read directly takes 81 reads per pixel; as a horizontal pass into an intermediate texture, then a vertical pass over that texture, it takes 9 + 9 = 18. Box blurs are separable too; Sobel is as well, since each of its kernels is a 1 2 1 smoothing times a −1 0 1 difference. The cost is a second pass and somewhere to keep the result between them, a render target.

Bilinear filtering cuts the reads again. One read placed between two texels, at the point weighted by their kernel weights, returns both of them blended in the right proportion: for weights w1 and w2 at offsets o1 and o2, sample at (o1 * w1 + o2 * w2) / (w1 + w2) and multiply by w1 + w2. A 9-tap 1D Gaussian becomes 5 reads.

Cost and color

Every tap is a texture read, and reads are usually what limits a kernel’s speed, so count them: a 3×3 kernel is 9 per pixel, a full-screen 13×13 one is 169 per pixel. Large radii come from separable passes, downsampled textures (a blur on a half-size image covers twice the radius for a quarter of the pixels), or both.

Average light, not encoded numbers: blur in linear color, or bright details lose energy and dark halos appear around them.