Understanding how changes in brain structure and function lead to changing cognitive abilities that are apparent across the human lifespan—as well as across species—remains an important goal for neurobiology. A great deal of brain development takes place prenatally, driven by intrinsic genetic and epigenetic programs that lay the foundation for adult cognitive skills. These programs specify the development of neural systems that vary among species in ways that reflect inheritance from a common ancestor as well as adaptation to local conditions. The brain continues to develop and organize its connectivity postnatally, a process that continues through adolescence and to a diminished degree throughout adult life. This plasticity is particularly important for the emergence of uniquely human faculties, like reading and writing. Different brain regions mature at different rates, and this variation appears to explain many of the changes evident in cognitive behavior during development. Such changes in the timing of critical events in the emergence of brain structures that underlie cognition may explain differences in the brain and behavior that characterize different species. Social complexity may have been a particularly important selective factor in the evolution of cognition and its neural mechanisms in human and non-human primates.