Despite a close correspondence between spatial and temporal cognition, empirical approaches to the two domains have used distinct theoretical conceptions: frames of reference for the former, and moving perspectives and reference-point metaphors for the latter. Our analysis reveals that these conceptions can ‐ and should ‐ be related more closely to each other. Mapping spatial frames of reference (FoRs) onto temporal relations, we obtain a taxonomy that allows us to distinguish more types of referencing than existing conceptions do and that is applicable to linguistic cases not accounted for so far. A cross-cultural experiment with speakers of German, English, Chinese and Tongan provides evidence for the psychological reality of the newly proposed FoRs and establishes culture-specific preferences. We conclude that spatial referencing systems indeed help to organize temporal representations.