Early tumor shrinkage (ETS) has been highlighted as a favorable prognostic factor related to progression-free survival (PFS) and overall survival (OS) in cytotoxic treatment of metastatic colorectal cancer. Data from a randomized phase III study comparing infusional 5-fluorouracil plus irinotecan (FUFIRI) versus irinotecan plus oxaliplatin (mIROX) were evaluated. Patient groups were analyzed according to the relative change in maximum tumor diameter between baseline and after 7 weeks of treatment. The ETS cohort was defined as a decrease of ≥20%. Additionally, the non-ETS cohort was subdivided into “minor shrinkage” (0–19%), “tumor progression” (any increase) and development of “new metastatic lesions”. Progression-free survival and OS were estimated in all patient subgroups. Assessment of ETS was possible in 201 patients. Early tumor shrinkage was observed in 47% (94/201) and non-ETS in 53% (107/201) of patients. Patients with ETS had a more favorable outcome with regard to PFS (9.9 months vs 6.1 months, = 0.029) and OS (27.5 months vs 17.8 months, = 0.002). In the non-ETS subgroups, patients with “minor shrinkage” (PFS 8.4 months, OS 21.6 months) showed a markedly better outcome than patients with “early tumor progression” (PFS 4.0 months, OS 15.3 months) or with “new metastatic lesions (PFS 2.2 months, OS 7.6 months). In conclusion, ETS assessment offers accelerated response evaluation when compared to RECIST. In patients treated with chemotherapy alone, ETS ≥20% is associated with excellent outcome. Non-ETS is a heterogeneous subgroup where patients with minor shrinkage clearly benefit from treatment, and patients with early progression or development of new lesions have an unfavorable prognosis.