The visual system allocates attention in object-based and location-based modes. However, the question of when attention selects objects and when it selects locations remains poorly understood. In this article, we present variations on two classic paradigms from the object-based attention literature, in which object-based effects are observed only when the object feature matches the task goal of the observer. In Experiment 1, covert orienting was influenced by task-irrelevant rectangles, but only when the target color matched the rectangle color. In Experiment 2, the region of attentional focus was adjusted to the size of task-irrelevant objects, but only when the target color matched the object color. In Experiment 3, we ruled out the possibility that contingent object-based selection is caused by color-based intratrial priming. These demonstrations of contingent object-based attention suggest that object-based selection is neither mandatory nor default, and that object-based effects are contingent on simple, top-down attentional control settings.