Wirelessly coupled coils are crucial for efficient power transmission in various applications. Previous design methods are only eligible for improving the efficiency of circular or square coils. This paper presents a method of characterizing and optimizing rectangular coils used in inductively coupled systems. After setting up a lumped component model for inductive coils, the efficiency can be expressed in terms of geometrical parameters of the coils. Subsequently, the power efficiency can be plotted versus these parameters in Matlab, thus getting the desired coils for optimum power transfer. With this design procedure from mathematical optimization, we eventually designed two rectangular coils spaced 10 mm apart, which achieves a power transmission efficiency of 46.4% at a frequency of 3 MHz. The design methodology is verified by simulation and measurement.