Flexible cognition is a hallmark of human behavior, but it comes with limits. There are costs to paying attention to many things or switching between tasks. These costs become even more problematic in the many disorders of the brain that limit cognitive capacity. However, there is limited direct evidence of a physiological basis for these limitations. Identifying one will be important for efforts to repair or reduce limits on cognition. I will present multiple lines of evidence from monkey electrophysiology, human and monkey behavior, and recurrent network modeling suggesting a neuronal mechanism that inflicts a limit on cognition and a cost of flexibility. In a behavioral paradigm designed to measure and manipulate subjects’ belief about the relevance of each of two perceptual tasks, we found that humans and monkeys make less accurate perceptual decisions under task uncertainty. To generate hypotheses about a neuronal basis for the task switching cost, we compared two recurrent neural networks (RNNs), trained to produce the correct choice or to reproduce the choices of macaque subjects. The ‘correct-choice’ RNN learned to flexibly switch tasks without incurring a task switch cost, while the ‘monkey-choice’ RNN displayed the expected cost of task switching. Comparing the activity of the recurrent layers of the two models revealed that the ‘correct-choice’ model maintained information relevant to the two tasks in separate subspaces of neuronal activity. But when the task was uncertain, the two subspaces in the ‘monkey-choice’ model collapsed together, leading to interference between tasks. We confirmed predictions of the model in further behavioral and physiological experiments. These results provide a neuronal mechanism for flexible decision-making in neurotypical subjects, as well as its dysfunction in common neurological disorders. They support the general, tantalizing hypothesis that limits in cognitive capacity arise from interference between the neural representations of different stimuli, tasks, or memories.