Filtering & mipmaps

Pixels on screen and texels in a texture almost never line up one to one. Up close, one texel covers many pixels; far away, one pixel covers many texels. Filtering is what the GPU does about both cases, and it is set on the texture or sampler from the host code. The shader only sees the result, but the result decides whether an image looks crisp, soft, blocky or full of flicker.

Magnification: nearest or linear

When a texture is drawn larger than its size, each pixel lands somewhere between texel centers.

• Nearest returns the closest texel. Edges stay hard and texels become visible squares: the right choice for pixel art and for data that must not be blended, such as indices into a palette.
• Linear (bilinear) blends the four nearest texels, weighted by how close the sample is to each center. The image gets soft instead of blocky. It is the default choice for photos and most surfaces.

Bilinear filtering is also a free blend: a sample placed exactly between two texels returns their average for the price of one read. Blur kernels use that to halve their number of reads.

Minification and aliasing

When a texture is drawn smaller than its size, bilinear filtering still reads only four texels per pixel and skips everything between. Which texels get read changes as the surface moves by a fraction of a pixel, so fine detail turns into noise that crawls and shimmers, and regular patterns such as bricks or stripes produce moiré. This is aliasing: the texture has more detail than the pixels can show, and point samples pick it up at random.

Mipmaps

A mipmap chain fixes minification by averaging ahead of time. Level 0 is the texture itself, level 1 is half the width and height, level 2 a quarter, and so on down to one texel. Each level averages blocks of the level above, so reading a small level returns the average of everything a distant pixel covers. The whole chain adds a third to the memory of the texture.

The GPU picks a level per pixel from how many texels the pixel covers. It measures that by comparing the texture coordinate across each 2×2 block of pixels it shades together: if the coordinate moves 4 texels from one pixel to the next, level 2 (4 = 2²) has about one texel per pixel. The level is the base-2 log of that footprint:

// at file scope:
uniform sampler2D uImage;
// inside main(): roughly the level the GPU would pick
vec2 texCoord = uv * vec2(textureSize(uImage, 0)); // in texels
vec2 dx = dFdx(texCoord);                          // change to the next pixel right
vec2 dy = dFdy(texCoord);                          // change to the next pixel up
float level = 0.5 * log2(max(dot(dx, dx), dot(dy, dy)));

The modes that use mipmaps differ in how they treat levels:

• Nearest mipmap (for example LINEAR_MIPMAP_NEAREST) reads one level, bilinear inside it. Cheap, but the switch between levels can show as a visible line across a floor.
• Trilinear (LINEAR_MIPMAP_LINEAR) reads the two nearest levels bilinearly and blends them by the fractional level. It costs two bilinear reads and hides the switch.

In WebGL a texture’s minification filter starts as NEAREST_MIPMAP_LINEAR. Until you call gl.generateMipmap or set a filter without mipmaps, the texture is incomplete and every read returns black. Textures you render into have no mipmaps either, unless you generate them after each render.

Anisotropic filtering

On a floor seen at a grazing angle, a pixel’s footprint is a long thin strip: it covers many texels in depth and few across. The level is chosen from the longer side, so the texture gets blurry across as well. Anisotropic filtering takes several samples along the long side of the footprint from a sharper level, which keeps grazing surfaces crisp. It is a sampler setting; in WebGL2 it comes from the EXT_texture_filter_anisotropic extension, with a maximum of 16 samples on most hardware.

Taking control in the shader

Three GLSL calls override the automatic choice:

• texture(sampler, uv, bias) adds bias to the level: negative is sharper (and more aliased), positive is softer.
• textureLod(sampler, uv, level) reads an exact level. It needs no neighboring pixels, so it is safe inside branches and loops, and it is how blur chains read a specific downsampled level.
• textureGrad(sampler, uv, dPdx, dPdy) takes the rate of change you give it instead of measuring it.

textureGrad fixes the seam that tiling with fract() leaves. At each tile border the wrapped coordinate jumps from 1 to 0, the measured change looks enormous, and the GPU reads the smallest level for one row of pixels. Measure the change on the coordinate before wrapping it:

vec2 tiled = uv * 4.0;
vec4 c = textureGrad(uImage, fract(tiled), dFdx(tiled), dFdy(tiled)); // no seam

Math-only patterns have no mipmaps, so a fine procedural stripe aliases exactly like an unfiltered texture. The same derivatives that pick a mip level let a pattern blur itself by the width of a pixel, which the derivatives page covers.