We present a detailed study of the dust energy balance in the Sombrero galaxy M104. From a full radiative transfer analysis, including scattering, absorption and thermal re-emission, we construct models that can reproduce images at optical/near-infrared (near-IR) wavelengths, the observed stellar spectral energy distribution and the minor axis extinction profiles in the V and RC band. A standard model, which contains only an old stellar population to heat the dust, underestimates the observations of dust emission at IR wavelengths by a factor of ∼3. Supplementing this basic model with a young stellar component of low star formation activity in both the inner disc (star formation rate, SFR ∼0.21 M⊙ yr−1) and dust ring (SFR ∼0.05 M⊙ yr−1), we are capable of solving the discrepancy in the dust energy budget of the Sombrero galaxy at wavelengths shortwards of 100 μm. To account for the increased far-IR/submm emission beyond 100 μm, we propose an additional dust component distributed in quiescent clumps. This model with a clumpy dust structure predicts three-quarters of the total dust content (∼2.8 × 107 M⊙) to reside in compact dust clouds with no associated embedded sources. Although the assumption of a clumpy dust structure in the Sombrero galaxy is supported by high-resolution optical data, we cannot rule out the possibility that dust grains with a higher dust emissivity account for part of the discrepancy in the energy budget at submm wavelengths.