Bioassay-guided fractionation of Terminalia bentzoe L. leaves methanol extract identified the known triterpene oleanolic acid (1) as its major breast cancer cell migration inhibitor. Further chemical optimization afforded five new (9–12 and 15) and seven known (4–8, 13, and 14) semisynthetic analogues. All compounds were tested for their ability to inhibit human breast cancer MDA-MB-231 cells migration, proliferation, and invasion. The results revealed that 3-O-[N-(3′-chlorobenzenesulfonyl)-carbamoyl]-oleanolic acid (11) and 3-O-[N-(5′-fluorobenzenesulfonyl)-carbamoyl]-oleanolic acid (12) were the most active hits at low μm concentration. Western blot analysis indicated the activity of 1, 11, and 12 might be related, at least in part, to the suppression of Brk/Paxillin/Rac1 signaling pathway. Pharmacophore modeling study was conducted to better understand the common structural binding epitopes important for the antimigratory activity. The sulfonyl carbamoyl moiety with an optimal bulkiness electron-deficient phenyl ring is associated with improved activity. This study is the first to discover the antimigratory and anti-invasive activities of oleanolic acid and analogues through targeting the Brk/Paxillin/Rac1 axis.