Spatial patterns of precipitation in the southwestern United States result in a complex gradient from winter-to-summer moisture dominance that influences tree growth. In response, tree growth exhibits seasonal-to-annual variability that is evident in the growth of whole tree rings, and in sub-annual sections such as earlywood and latewood. We evaluated the influence of precipitation and temperature on the growth of Pinus ponderosa trees in 11 sites in the southwestern US. Precipitation during the year of growth and the prior year accounted for about half of the climate influence on annual growth, with the other half reflecting conditions 2–4 years prior to growth, indicating that individual trees do indeed exhibit multi-year “memory” of climate. Trees in wetter sites exhibited weaker influence of past precipitation inputs, but longer memory of climatic variability. Conversely, trees in dry sites exhibited shorter memory of long-term climatic variability, but greater sensitivity to past precipitation effects. These results are consistent with the existence of complex interactions between endogenous (phenotype) effects and exogenous (climate) effects in controlling climate memory in trees. After accounting for climate, residual variability in latewood growth was negatively correlated with earlywood growth, indicating a potential tradeoff between latewood versus earlywood growth. This study provides new insights that will assist the accurate prediction of woody biomass growth and forest carbon sequestration across a southwestern US precipitation gradient.