Quantization

Quantization represents a range of values using a limited set of levels. When there are fewer color levels, a smooth gradient can become a series of visible steps. A posterization effect deliberately creates this kind of reduction. Apple's Motion reference

BetterDisplay's Quantization adjustment reduces the levels used by its software color table. You are changing how that image is represented, rather than selecting a new format for the display connection.

See bit depth for connection precision, Image Adjustments for the control, and dithering for a different treatment of tonal steps.