The origins of hysteresis in organic field-effect transistors (OFETs) and its applications in organic memory devices is investigated. It is found that the orientations of the hydroxyl groups in poly(vinyl alcohol) (PVA) gate dielectrics are correlated with the hysteresis of transfer characteristics in pentacene-based OFETs under the forward and backward scan. The applied gate bias partially aligns the orientations of the hydroxyl groups perpendicular to the substrate as characterized by reflective absorption Fourier transform infrared spectroscopy (RA-FTIR), in which the field-induced surface dipoles at the pentacene/PVA interface trap charges and cause the hysteresis. Treating PVA with an anhydrous solvent eliminates the residual moisture in the dielectrics layer, allowing for more effective control of the induced dipoles by the applied gate bias. OFETs of dehydrated-PVA dielectrics present a pronounced shift of the threshold voltage (ΔVTh) of 35.7 V in transfer characteristics, higher than that of 18.5 V for untreated devices and results in sufficient dynamic response for applications in memory elements. This work highlights the usage of non-ferroelectric gate dielectrics to fabricate OFET memory elements by manipulating the molecular orientations in the dielectrics layer.