Poly(ethylene glycol) (PEG) is the most used polymer and also the gold standard for stealth polymers in the emerging field of polymer-based drug delivery. The properties that account for the overwhelming use of PEG in biomedical applications are outlined in this Review. The first approved PEGylated products have already been on the market for 20 years. A vast amount of clinical experience has since been gained with this polymer—not only benefits, but possible side effects and complications have also been found. The areas that might need consideration and more intensive and careful examination can be divided into the following categories: hypersensitivity, unexpected changes in pharmacokinetic behavior, toxic side products, and an antagonism arising from the easy degradation of the polymer under mechanical stress as a result of its ether structure and its non-biodegradability, as well as the resulting possible accumulation in the body. These possible side effects will be discussed in this Review and alternative polymers will be evaluated.