Recent studies have identified the first-order representation of microbial decomposition as a major source of uncertainty in simulations and projections of the terrestrial carbon balance. Here, we use a reduced complexity model representative of current state-of-the-art models of soil organic carbon decomposition. We undertake a systematic sensitivity analysis to disentangle the effect of the time-invariant baseline residence time (k) and the sensitivity of microbial decomposition to temperature (Q₁₀) on soil carbon dynamics at regional and global scales. Our simulations produce a range in total soil carbon at equilibrium of ~ 592 to 2745 Pg C, which is similar to the ~ 561 to 2938 Pg C range in pre-industrial soil carbon in models used in the fifth phase of the Coupled Model Intercomparison Project (CMIP5). This range depends primarily on the value of k, although the impact of Q₁₀ is not trivial at regional scales. As climate changes through the historical period, and into the future, k is primarily responsible for the magnitude of the response in soil carbon, whereas Q₁₀ determines whether the soil remains a sink, or becomes a source in the future mostly by its effect on mid-latitude carbon balance. If we restrict our simulations to those simulating total soil carbon stocks consistent with observations of current stocks, the projected range in total soil carbon change is reduced by 42% for the historical simulations and 45% for the future projections. However, while this observation-based selection dismisses outliers, it does not increase confidence in the future sign of the soil carbon feedback. We conclude that despite this result, future estimates of soil carbon and how soil carbon responds to climate change should be more constrained by available data sets of carbon stocks.