Pylyshyn uses constraints to solve many of the problems associated with the inverse problem in vision. We are sympathetic to such an approach, and indeed, we think that in many cases constraints allow tract-able solutions to otherwise insoluble problems. We argue, however, that Pylyshyn has been too quick to assume that certain perceptual phenomena can be explained by appealing to constraints embodied in the visual machinery. For several more complex perceptual phenomena it is not clear how one proceeds to look for constraints once simple constraints like rigidity have been abandoned.