Annual plants in unpredictable environments maintain dormant seeds to avoid extinction. Here, we present results for four desert annual species suggesting that germination rates are variable even in the absence of abiotic cues. Namely, seeds produced in a copious year had lower germination rates than seeds produced in drought years. Inspired by our data, we have extended previous bet-hedging models by including a structured seed bank. With density-dependence, the ESS (environmental stable strategy) involved a negative relationship between seed yield and subsequent germination probability. We suggest that heterogeneous germination rates are selected for by competition among seedlings after years with high seed production. In summary, our findings are suggestive of an intriguingly simple and effective mechanism that may allow annual plants to partly predict their future success.