Military planners and other decision makers require advanced early warning of impending crises so they can devise effective mitigation plans, mobilize resources, and coordinate responses with their foreign counterparts. Over the last 40 years, the US government has invested generously in several attempts to build crisis forecasting systems that were analytically defensible and capable of processing and making sense of vast amounts of information in real or near real time. This article describes the most recent attempt by the US military to develop an Integrated Crisis Early Warning System (ICEWS). Although ICEWS relies heavily on social science theories, data, and methods, our experiences thus far reveal some strengths and limitations of contemporary quantitative approaches to addressing social science questions with real world implications. The article concludes with a sketch of a new paradigmatic approach—a Computational Social Science Experimentation Proving Ground—that could not only improve crisis early warning and response, but also revolutionize how social science knowledge is developed, evaluated, and applied more broadly.