We previously reported the use of a humanized bi-specific diabody that targets epidermal growth factor receptor and CD3 (hEx3-Db) for cancer immunotherapy. Bacterial expression can be used to express small recombinant antibodies on a large scale; however, their overexpression often results in the formation of insoluble aggregates, and in most cases artificial affinity peptide tags need to be fused to the antibodies for purification by affinity chromatography. Here, we propose a novel method for preparing refined, functional, tag-free bi-specific diabodies from IgG-like bi-specific antibodies (BsAbs) in a mammalian expression system. We created an IgG-like BsAb in which bi-specific diabodies were fused to the human Fc region via a designed human rhinovirus 3C (HRV3C) protease recognition site. The BsAb was purified by protein A affinity chromatography, and the refined tag-free hEx3-Db was efficiently produced from the Fc fusion format by protease digestion. The tag-free hEx3-Db from the Fc fusion format showed a greater inhibition of cancer growth than affinity-tagged hEx3-Db prepared directly from Chinese hamster ovary cells. We also applied our novel method to another small recombinant antibody fragment, hEx3 single-chain diabody (hEx3-scDb), and demonstrated the versatility and advantages of our proposed method compared with papain digestion of hEx3-scDb. This approach may be used for industrial-scale production of functional tag-free small therapeutic antibodies.