Evaluation in the Face of Uncertainty

“The book provides a useful presentation of the tactics that can be implemented to prevent surprises from weakening an evaluation plan....Aside from the more theoretical discussion provided by Morell, the book presents and makes extensive use of 18 cases drawn from all experiences of practicing evaluators around the world....These cases...would make excellent teaching examples for newer evaluators: not only will they feel as though they are not alone in encountering unanticipated situations in their work, but they will learn important lessons from the experience of others....Meets a very real need and undoubtedly advances out thinking about the issue of surprise in evaluation....Morell provides good, practical advice on dealing with unanticipated events throughout the text, based on his own experiences and those of the evaluators who provided the cases for the book. The key insights found in the book will surely enable evaluators to better plan their work, identify surprises before they crop up, and, in the end, enable them to produce higher-quality evaluations.”

—Canadian Journal of Program Evaluation


“This clearly written, well-organized book presents a lexicon of the surprises that occur in evaluation practice, both in the program—how it unfolds between planning and completion of the evaluation—and in the process of data collection and analysis. Morell outlines a structure for understanding what these surprises are, where they occur in the programming and evaluation process, why they are inevitable, and how they can (or sometimes cannot) be foreseen. The book provides practitioners with a systematic way of diagnosing and possibly even anticipating surprises, and explains how to accommodate them.”

—Deborah Wasserman, PhD, Principal Consultant, PERSolutions: Program Evaluation and Research, Columbus, Ohio


“If your world, including your evaluation work, is often complex, uncertain, and unpredictable, you have a fellow traveler and real-world guide in Morell. He applies more than three decades of experience to the challenges of distinguishing what can and cannot be foreseen, anticipating the unexpected, and dealing with the unforeseeable. This book draws on concrete cases, expert wisdom, practitioner experiences, scholarly knowledge, and organizational theory to explore evaluation approaches and methods that are agile, flexible, emergent, and responsive. Morell's voice is personable, his guidance realistic, and his insights important. You'll be surprised how much better you can get at anticipating and learning from surprises.”

—Michael Quinn Patton, PhD, Director, Utilization-Focused Evaluation, St. Paul, Minnesota


“Morell offers descriptions and prescriptions to help evaluators develop agile methodologies. This book is a valuable addition to available instructional resources for both seasoned practitioners and students just entering the evaluation profession. Writing in an accessible, cogent style, Morell effectively demonstrates how to navigate the challenges of complex systems. The brief cases he presents to illustrate his points will be especially useful for stimulating discussion in graduate classes as well as professional development settings.”

—Kathryn E. Newcomer, PhD, Director, Trachtenberg School of Public Policy and Public Administration, and Co-Director, Midge Smith Center for Evaluation Effectiveness, The George Washington University


“The use of real-life examples with all their warts adds considerably to the usefulness of the book, especially because the examples come from around the world and reflect a wide variety of evaluation contexts.”

—David L. Streiner, PhD, Department of Psychiatry, University of Toronto and Department of Psychiatry and Behavioural Neurosciences, McMaster University


“Morell offers an original, plain-spoken, perspicacious, wise discourse on a relatively neglected yet highly significant aspect of evaluation. This is among the first and best full-on analyses of the primary sources of evaluation surprise. The book puts a strong intellectual foundation under proposed remedies. There are gems in almost every chapter, such as the discussion of agile evaluation.”

—Lois-ellin Datta, PhD, President, Datta Analysis, Waikoloa, Hawaii


“This book fills a vastly neglected void in the evaluation literature. Morell provides a theoretical framework for anticipating and minimizing the unexpected by means of agile, responsive evaluation methodologies. He illustrates a variety of pragmatic strategies for dealing with the inevitable (and sometimes unforeseeable) things that can go wrong when planning and executing evaluations. Ironically, the lessons exemplified in the book have great potential for propelling the field forward in both anticipated and unanticipated ways. This is an essential, invaluable resource for any serious student, practitioner, or scholar of evaluation.”

—Chris L. S. Coryn, PhD, Director, Interdisciplinary PhD in Evaluation, Western Michigan University


“Insightful and provocative. Though Morell writes from the stance of an evaluator, his descriptions of 'things that go awry' apply to a wide swath of research methodologies. The idea that all research projects encounter unanticipated or unintended outcomes is aptly illustrated through a variety of case studies—for example, No Child Left Behind evaluation studies, health impacts of central heating, and outcomes of abolishing user fees in health clinics in Niger. The cases provide ample evidence of why things went awry and how unanticipated or unintended outcomes may be predicted and controlled. This book would be ideal for graduate-level courses on research design or program evaluation, either as a textbook or a supplement.”

—James E. Gruber, PhD, Department of Behavioral Sciences, University of Michigan-Dearborn