Essay 5.1 More About Opponent Processing in Color Vision

Why Use Opponent Processes?

We have learned that the visual system starts out by coding the color of light via the outputs of short- (blue), medium- (green), and long- (red) wavelength cones (trichromacy). In this essay, we will refer to the outputs of the three cones both in terms of their experienced color and their wavelength. Consequently, short-wavelength cones will be described as blue (B), medium-wavelength cones will be described as green (G), and long-wavelength cones will be described as red (R). Each cone response is ambiguous on its own, so retinal ganglion cells and LGN neurons combine the outputs from the three cone types and recode color information in terms of red versus green (R/G), blue versus yellow (B/Y), and black versus white (Bl/Wh) opponent processes. But opponent cell outputs are also ambiguous unless compared against the outputs of different types of opponent cells. So why does the visual system go through the hassle of this recoding process? Why trade one three-dimensional color space (red, green, blue) for another three-dimensional color space (R/G, B/Y, Bl/Wh)?

We cannot know the answer to this question for sure. The human visual system is a product of millions of years of evolution. Careful experimentation can establish how the system works, but experiments cannot tell us why it works the way it does. However, we can speculate about the evolutionary advantage the opponent process outputs might provide over the initial cone outputs. Consider the left-hand picture of Image 1. You almost certainly perceive this to be a small red rectangle casting a shadow on a large orange square, which in turn is casting a shadow on the blue background. But you probably perceive the right-hand image, which includes exactly the same shapes but with different colors, as five separate surfaces.

Two images shown side by side. The left image shows a large blue square, with a smaller orange square on top with its shadow, and an even smaller rectangle on top with its shadow. The right image shows a large blue square, with a bright green square on top, then an orange square, and then an even smaller purple rectangle and then red one on top. — **Image 1** The image on the left is perceived as three surfaces: a red rectangle above an orange square on a blue background. The red and orange squares appear to be casting shadows. The image on the right is perceived as five different surfaces. These inferences are easily made on the basis of opponent-process cell outputs.

The key to making these perceptual inferences is that the areas you interpret as shadows in the left-hand image are the same hues as their backgrounds, whereas in the right-hand image all five shapes differ in hue. In terms of cone outputs, the differences between shadows and backgrounds in the left-hand image will be about as great as the differences between the corresponding areas in the right-hand image. But in terms of opponent processes, the situation is quite different.

In the LGN, you find cells that have a center-surround receptive field structure, as discussed in the text with regard to black and white luminance. Additionally, some of these center-surround cells respond best when red light falls on the center and green light falls on the surround, which we will denoted by the term R+G–. A cell that prefers green in the center and red in the surround is a G+R– cell. For the blue-yellow color axis, we find cells that prefer blue in the center and yellow in the surround (B+Y–) and cells that prefer yellow in the center and blue in the surround (Y+B–).

The shadows in the left-hand image produce exactly the same outputs as their backgrounds in the R/G and B/Y “channels” of the opponent process system. The orange square, for example, activates R+G– and Y+B– cells (because orange can be described as reddish yellow). Since the shadow of the small rectangle is the same hue, it also activates these same opponent process cells. The difference between the shadow and its background will only be registered in the output of the Bl/Wh opponent process: the Wh+Bl– cells will respond more strongly to the brighter orange square than to the darker shadow. In the right-hand image, on the other hand, all five shapes are different hues, so all five will produce different outputs in the R/G and B/Y channels.

Although we almost never notice them, shadows—brightness differences—are scattered throughout almost every visual scene (if you look around carefully, you will probably see them everywhere). But shadows are rarely of interest to us, whereas hue boundaries are important. They divide the visual scene into component surfaces which higher-level vision can then combine into objects. Thus, recoding light wavelengths into dimensions that de-emphasize shadows and highlight surfaces—such as the color opponent dimension—is a very useful thing.

Color Opponent Cells in the LGN

Cells in the LGN seem to be cone-opponent cells, so the transformations that produce the color-opponent processes that support color appearance are likely to be found in the visual cortex. One useful transformation of the information is shown in Image 2. The LGN has opponent cells, and Image 2a shows a cartoon of one such cell’s receptive field. We will call it L+/M–, indicating that it is excited by redder hues in its center and inhibited by greener hues in its surround. (For present purposes, we will not worry about whether this cell is cone-opponent or color-opponent. It is just easier to talk about red and green in this case.)

This figure has four parts, a-d. Part a shows a single-opponent cell, outer M- and inner L+. Part b shows four receptive fields in a half green (left), half red (right)background. All cells show M- and inner L+. Lefthand cell, fully on green, is labeled Worst. Middle top cell labeled Better is mostly on green, a little on red. Middle bottom cell labeled Even better, a little on green and mostly on red. Righthand cell, fully on red, is labeled Best. On far right is red circle labeled Best stimulus. Part c shows a double-opponent cell, outer L-/M+ and inner L+/M-. Part d shows four receptive fields in a half green (left), half red (right)background. All cells show outer L-/M+ and inner L+/M-. Lefthand cell, fully on green, is labeled Neutral. Middle top cell labeled Worst position is mostly on green, a little on red. Middle bottom cell labeled Best position, a little on green and mostly on red. Righthand cell, fully on red, is labeled Neutral. On far right is red circle inside green outer circle, labeled Best stimulus. — **Image 2** Color-opponent receptive fields. (a) Single-opponent cells get excitatory input from one cone type and inhibitory input from another. (b) These cells respond vigorously to regions of the correct color. Here L+/M– prefers the region that appears reddish. (c) The center of each double-opponent cell gets excitatory input from one cone type and inhibitory input from another. The pattern is reversed in the surround. (d) These cells respond vigorously at an edge between two colors.

Image 2c shows a double-opponent cell. Double-opponent cells are first found in the visual cortex (Johnson et al., 2001). In the example shown here, the cell is excited by redder hues in its center and by greener hues in the surround, and it is inhibited by redder hues in the surround and by greener hues in the center. To see why this is interesting, look at Images 2b and 2d, where we have superimposed these receptive fields across a red-green border. In this context, the cone-opponent cells, described earlier, could be called single-opponent cells. A single-opponent cell conveys information about the color of a broad area. In this case, the cell is most excited by red and not green. A double-opponent cell, in contrast, conveys information about chromatic edges. Thus, the most “exciting” place for a single-opponent cell in Image 2b is someplace entirely in the red area where L-cone input is greatest and M-cone input is reduced. For the double-opponent cell in Image 2d, the best place is on the red-green edge where M-cones can have some input to the surround while L-cones dominate in the center. The double-opponent cell marks where the color changes. This information can be useful when you’re trying to use color to carve the world into surfaces and objects (see Chapter 4).

Reference

Johnson, E. N., Hawken, M. J., and Shapley, R. (2001). The spatial transformation of color in the primary visual cortex of the macaque monkey. Nat Neurosci 4: 409–416.

Glossary Terms

double-opponent cell
A cell type, found in the visual cortex, in which one region is excited by one cone type, combination of cones, or color and inhibited by the opponent cones or color (e.g., R+/G–). Another adjacent region would be inhibited by the first input and excited by the second (thus, in this example, R–/G+).

single-opponent cell
Another way to refer to cone-opponent cells, in order to differentiate them from double-opponent cells.