A solid polymer electrolyte prepared by using a solvent-free, scalable technique is reported. The membrane is formed by low-energy ball milling followed by hot-pressing of dry powdered polyethylene oxide polymer, LiCF3SO3 salt, and silane-treated Al2O3 (Al2O3-ST) ceramic filler. The effects of the ceramic fillers on the properties of the ionically conducting solid electrolyte membrane are characterized by using electrochemical impedance spectroscopy, XRD, differential scanning calorimeter, SEM, and galvanostatic cycling in lithium cells with a LiFePO4 cathode. We demonstrate that the membrane containing Al2O3-ST ceramic filler performs well in terms of ionic conductivity, thermal properties, and lithium transference number. Furthermore, we show that the lithium cells, which use the new electrolyte together with the LiFePO4 electrode, operate within 65 and 90 °C with high efficiency and long cycle life. Hence, the Al2O3-ST ceramic can be efficiently used as a ceramic filler to enhance the performance of solid polymer electrolytes in lithium batteries.