We argue for a view of credible evidence that is multidimensional in philosophical and methodological terms. We advocate for the importance of deepening the meaning of credible evaluation practice and findings by bringing multiple philosophical and theoretical lenses to the evaluation process as a basis for the use of mixed methods in evaluation, thus providing evaluators with strategies for garnering more complex and diverse perspectives on the creation of credible evidence. ©Wiley Periodicals, Inc., and the American Evaluation Association.

Mixed methods evaluation has a long-standing history of enhancing the credibility of evaluation findings. However, using mixed methods in a utilitarian way implicitly emphasizes convenience over engaging with its philosophical underpinnings (Denscombe, 2008). Because of this, some mixed methods evaluators and social science researchers have been criticized for an a-paradigmatic stance (Greene, 2007). Critics posit that this seemingly unreflective “what-works” (Denzin, 2012) approach threatens the validity of findings (Lipscomb, 2008). This chapter maintains this position and argues that evaluators need to examine their philosophical orientations and how this informs credible evidence in mixed methods evaluation. Accordingly, this work provides an overview of Deweyan pragmatism, emphasizing intelligent action. Next, important aspects of intelligent action are used to explore central questions in mixed methods evaluation and evidenced-based practice. To conclude, a case example, illustrating how intelligent action can inform mixed methods evaluation, is offered. © Wiley Periodicals, Inc., and the American Evaluation Association.

Credibility in evaluation is a multifaceted concept that involves consideration of diverse stakeholders' perspectives and purposes. The use of a transformative lens is proposed as a means to bringing issues of social justice and human rights to the foreground in decisions about methodology, credibility of evidence, and use of evaluation findings. ©Wiley Periodicals, Inc., and the American Evaluation Association.

Credibility of evidence in evaluation is examined through the lens of dialectical pluralism (DP). Principles of procedural justice are a core element of DP and help justify outcomes of its application. A key message is that DP and the associated circle of scientific evidence and knowledge model can aid the evaluation community in producing an inclusive evaluation knowledge generation, dissemination, and use system, where practice-to-theory and theory-to-practice evidence continually inform each other in a multiple stakeholder environment. ©Wiley Periodicals, Inc., and the American Evaluation Association.

Some evaluators employ randomized controlled trials (RCTs) as the gold standard of evidence-based practice (EBP). Critics of RCT designs argue that RCTs do not include the complexity of program participants' experiences or clinical expertise, and couple this with criticisms that it is difficult to transfer RCT findings from the laboratory to the real world of clinical practice. The evaluation questions applied to RCT designs often exclude issues related to participants' gender, race, class, and other differences, furthering the stereotyping process (Rogers & Ballantyne, 2009). I argue that weaving in a subjectivist methodology and shifting methodological perspectives and methods into RCT-based evaluations prior to, during, or after the RCT design serves to enhance the credibility and social-justice RCT praxis. ©Wiley Periodicals, Inc., and the American Evaluation Association.

Evaluations should be issues driven, not methods driven. The starting point should be priority programs to be evaluated or policies to be tested. From this starting point, a list of evaluation questions is identified. For each evaluation question, the task is to identify the best available method for answering that question. Hence it is likely that any one study will contain a mix of methods. A crucial question for an impact evaluation is that of attribution: What difference did the intervention make to the state of the world? (framed in any specific evaluation as the difference a clearly specified intervention or set of interventions made to indicators of interest). For interventions with a large number of units of assignment, this question is best answered with a quantitative experimental or quasi-experimental design. And for prospective, or ex ante, evaluation designs a randomized control trial (RCT) is very likely to be the best available method for addressing this attribution question if it is feasible. But just the attribution question will be answered. A high-quality impact evaluation will answer a broader range of evaluation questions of a more process nature, both to inform design and implementation of the program being evaluated and for external validity. Mixed methods combine the counterfactual analysis from an RCT with factual analysis with the use of quantitative and qualitative data to analyze the causal chain, drawing on approaches from a range of disciplines. The factual analysis will address such issues as the quality of implementation, targeting, barriers to participation, or adoption by intended beneficiaries. ©Wiley Periodicals, Inc., and the American Evaluation Association.

There is a strong trend in policy-making circles for strategic, systemic, and large-scale interventions. Although such trends make sense in terms of economy of scale and scope, the political will necessary for making these large-scale interventions a reality is often lacking, and the problem of the transferability of interventions from one local context to another (e.g., from trial conditions to local communities, and then across local communities) remains largely unsolved (Cartwright & Munro, 2010). ©Wiley Periodicals, Inc., and the American Evaluation Association.

The goal of this chapter is to recommend quality criteria to guide evaluators' selections of sampling designs when mixing approaches. First, we contextualize our discussion of quality criteria and sampling designs by discussing the concept of interpretive consistency and how it impacts sampling decisions. Embedded in this discussion are challenges impacting interpretive consistency. Strategies and an integrative sampling framework comprising published frameworks are presented to facilitate evaluators' decisions about selecting sampling designs in accordance to specific criteria. We conclude by presenting an illustrative application of mixed sampling design criteria to a published mixed evaluation study as a heuristic example of a way of embedding sampling criteria in a mixed evaluation. ©Wiley Periodicals, Inc., and the American Evaluation Association.

The quality of mixed methods systematic reviews relies on the quality of primary-level studies. The synthesis of qualitative evidence and the recent development of synthesizing mixed methods studies hold promise, but also pose challenges to evidence synthesis. ©Wiley Periodicals, Inc., and the American Evaluation Association.

This is a timely issue on the contributions of mixed methodology to the contemporary demand for credible evaluative evidence on which to base policy and resource-allocation decisions. The character of evidence—especially evidence on the quality and effectiveness of efforts to improve the life quality or even life chances of many in our societies—requires the best minds from all corners of the evaluation community. The mixed methods voice is needed in the evidence-based conversation. ©Wiley Periodicals, Inc., and the American Evaluation Association.