How do people talk—and potentially think—about abstract concepts? One prominent theory, supported by abundant linguistic evidence, is that people draw upon concrete concepts to structure abstract ones via metaphorical connections (Lakoff & Johnson, 1980). Often, the source domain for a metaphor draws upon embodied physical experience (Grady, 1997), as in the TIME IS SPACE system, whereby representations in the domain of time are thought to arise from experiences of navigating through, orienting within, and observing motion in space. In recent years, psychological evidence has suggested that the connections between space and time are indeed conceptual (e.g., Boroditsky, 2000; Boroditsky & Ramscar, 2002); however, many gaps in our understanding of the workings of metaphor remain (Duffy & Feist, in press). Notably, until recently, the unique variations in the ways in which people experience metaphor have been largely overlooked, with much research falling prey to one of the ‘deadly sins’ of cognitive linguistics: to ignore individual differences (Dąbrowska, 2016). By focusing on two widely studied metaphors for time, Moving Time and Moving Ego, this review article shines a spotlight on the varied ways in which people draw on their embodied and enculturated experiences, along with ‘human experience’ on an individual level and the contexts within which they use metaphor. In doing so, it highlights the importance for metaphoric conceptualization of variation across languages, across contexts, and across individuals, suggesting that while the use and interpretation of metaphor may begin with cross-domain connections, they are but part of the story.