Cow's milk is a common cause of food allergy in children. Children usually outgrow cow's milk allergy by the age of 3–5 years, but some will have persistent symptoms beyond childhood. We performed a systematic review of randomized controlled trials (RCTs) and observational studies to assess the evidence supporting the use of oral immunotherapy in IgE-mediated cow's milk allergy to inform the World Allergy Organization guidelines. Of 1034 screened articles published until May 2011, five RCTs and five observational studies fulfilled a priori specified inclusion criteria. RCTs including 218 patients showed that oral immunotherapy, compared to elimination diet alone, increased the likelihood of achieving full tolerance of cow's milk [relative risk: 10.0 (95% CI: 4.1–24.2)]. Adverse effects of immunotherapy include frequent local symptoms (16% of doses), mild laryngospasm [relative risk: 12.9 (1.7–98.6)], mild asthma [rate ratio: 3.8 (2.9–5.0)], reactions requiring oral glucocorticosteroids [relative risk: 11.3 (2.7–46.5)] or intramuscular epinephrine injection [rate ratio 5.8 (1.6–21.9)]. Results of observational studies were consistent with those of RCTs. Despite the availability of RCTs, the overall low quality of evidence leaves important uncertainty about anticipated effects of immunotherapy due to very serious imprecision of the estimates of effects and the likelihood of publication bias for some of the critical outcomes. A potentially large benefit of oral immunotherapy in patients with cow's milk allergy may be counterbalanced by frequent and sometimes serious adverse effects. Additional, larger RCTs measuring all patient-important outcomes are still needed.