We discuss the prospects for the detection of gamma-ray bursts (GRBs) by the Cherenkov Telescope Array (CTA), the next generation, ground-based facility of imaging atmospheric Cherenkov telescopes (IACTs) operating above a few tens of GeV. By virtue of its fast slewing capabilities, the lower energy threshold compared to current IACTs, and the much larger effective area compared to satellite instruments, CTA can measure the spectra and variability of GRBs with excellent photon statistics at multi-GeV energies, which would revolutionize our understanding of the physics of GRBs, test their validity as the origin of ultrahigh-energy cosmic rays and provide powerful probes of the extragalactic background light as well as Lorentz-invariance violation. Employing a model of the GRB population whose properties are broadly consistent with observations by the Gamma-ray Burst Monitor (GBM) and Large Area Telescope (LAT) on-board Fermi, we simulate follow-up observations of GRBs with the Large Size Telescopes (LSTs), the component of CTA with the fastest slew speed and the best sensitivity at energies below a few hundred GeV. For our fiducial assumptions, we foresee that the LSTs can detect ∼0.1 GRBs per year during the prompt phase and ∼0.5 GRBs per year in the afterglow phase, considering only one array site and both GBM and the Space-based multi-band astronomical Variable Object Monitor (SVOM) as the alert instruments. The detection rates can be enhanced by a factor of about 5 and 6 for the prompt emission and the afterglow, respectively, assuming two array sites with the same sensitivity and that the GBM localization error can be reduced to less than 1°. The expected distribution of redshift and photon counts is presented, showing that despite the modest event rate, hundreds or more of multi-GeV photons can be anticipated from a single burst once they are detected. We also study how the detection rate depends on the intrinsic GRB properties and the delay time between the burst trigger and the follow-up observation.