We are seldom taught that simplification has a high risk of failure. In truth, it only works up to a point, after which all that lies ahead is failure. To examine the limits of simplicity is to look at what happens when our efforts to make things fit into a sound bite, label, or keyword go awry. When simplification works, it can indeed be very effective. But simplification does not always work—so more of it is not necessarily better. And when simplification fails, it fails miserably. This talk exposes the limitations of simplification as a design choice, explores the cognitive origins of why we often get led astray in making such a design choice, and explores how we might develop a set of practical heuristics to counter the seductiveness of simplicity itself. The goal is appropriateness and balance— what cybernetics calls requisite variety, and what many design practitioners call placing context in context. I conclude with a heuristic to guide the practitioner on what to do when their efforts at simplification are failing.