The Evaluation Handbook
An Evaluator's Companion
Edited by Debra J. Rog and Leonard Bickman
Request a free digital professor copy on VitalSource ?“This is the kind of book that I wish had existed when I was new to evaluation. It masterfully covers major concepts in the field, with an eye toward the practical. Many of the easy-to-understand chapters offer examples, tools, or checklists to help put the ideas into practice. This book will be a valuable addition to the required reading for students in our certificate, master’s, and doctoral programs. It lives up to its promise as an evaluator’s companion.”
—Bianca Montrosse-Moorhead, PhD, Neag School of Education, University of Connecticut
“This handbook will be essential reading for all my graduate students in evaluation and applied research! Each chapter is written by well-respected evaluation legends, who offer many practical tools, insights, and proven strategies for enhancing the professional practice of evaluation. Whether you're a beginner or seasoned professional, this resource will elevate your work and thinking about evaluation.”
—Tiffany Berry, PhD, Dean and Professor, School of Social Science, Policy, and Evaluation, Claremont Graduate University
“Framed by the American Evaluation Association's Evaluator Competencies, this comprehensive handbook unpacks key components, responsibilities, and issues that are unique to the practice of evaluation. Students, emerging evaluators, and instructors will appreciate the concrete examples, detailed illustrations, and practical advice that bring each foundational topic to life and provide expert guidance for practice across varying contexts.”
—Rebecca M. Teasdale, PhD, Department of Educational Psychology, University of Illinois at Chicago
Table of Contents
I. Foundation for the Volume
2. Competencies for Program Evaluators, Jean A. King
II. Evaluation Theories, Foundations, Principles, and Purpose
3. Evaluative Thinking: Understanding and Applying the Foundations of Evaluation, Thomas Archibald, Jane Buckley, & Guy O’Grady Sharrock
4. Evaluation Theories: Guidance to Evaluating in Various Circumstances, Melvin M. Mark
5. Valuing in Evaluation, Emily F. Gates & Thomas A. Schwandt
6. Equity in Evaluation, Donna Durant Atkinson
7. Ethical Challenges, Michael Morris
8. Fostering Evaluation Use, Marvin C. Alkin & Anne T. Vo
9. Illuminating Evaluation's Kaleidoscope: Beautiful, Diverse, Ever-Changing Manifestations, Michael Quinn Patton
III. Answering Evaluation Questions: Designs, Methods, and Analyses
10. Collaborative, Participatory, and Empowerment Evaluation: From Community Consultation to Community Control, David Fetterman, Liliana Rodríguez-Campos, & Ann Zukoski
11. Making the Most of Evaluations: Strategies for Assessing Program Evaluability and Evaluation Feasibility, Debra J. Rog
12. Monitoring Program Implementation, Byron J. Powell, Leonard Bickman, & Kimberly E. Hoagwood
13. Examining Outcomes and Impacts: Designs and Strategies in Theory and Practice, Laura R. Peck & R. Bradley Snyder
14. Combining Costs and Results: Designs, Strategies, and Analysis, Robert Shand, A. Brooks Bowden, & Henry Levin
15. Evaluation as Storytelling: Using a Qualitative Design, Sharon Rallis & Janet Usinger
16. Planning, Data Collection, and Data Preparation for Quantitative Analysis, Erica Harbatkin, Gary T. Henry, & Lam D. Pham
17. Conducting a Quantitative Analysis, Lam D. Pham, Gary T. Henry, & Erica Harbatkin
18. Mixed Methods Design, Tarek Azzam & Natalie Jones
IV. Planning, Managing, and Implementing Evaluations
19. Designing and Planning an Evaluation: Beyond Methods, Darlene F. Russ-Eft
20. Logic Models and Program Theory, Joy Frechtling
21. Resource Planning, Leonard Bickman
22. The Role of an Internal Evaluator, Arnold Love
23. The Independent Consultant: An Insider’s Guide to a Consulting Career, Gail Vallance Barrington
24. Conducting Evaluations under Budget, Time, and Data Constraints: An International Perspective, Michael Bamberger
V. Crosscutting Issues
25. Communicating with Stakeholders, Glenn O’Neil
26. Information Visualization and Evaluation, Tarek Azzam, Sarah Douville, Ciara Knight, Piper Grandjean Targos, & Natalie Jones
27. Exemplary Evaluations in a Multicultural World, Stewart I. Donaldson
Glossary
Author Index
Subject Index
Contributors
About the Editors
Debra J. Rog, PhD, is a Vice President for Social Policy and Economics Research at Westat. Throughout her career, Dr. Rog has directed numerous evaluation studies of homeless and housing interventions and systems, mental health and substance abuse interventions, public health programs and initiatives, and criminal justice initiatives, among other areas. She is a recognized expert in evaluation practice and applied research design. Dr. Rog is Associate Director for Faculty Affairs at The Evaluators’ Institute and past president of the American Evaluation Association (AEA). She has been recognized for her evaluation work by the National Institute of Mental Health, the AEA, the Eastern Evaluation Research Society, and the Knowledge Utilization Society.Leonard Bickman, PhD, is Professor Emeritus of Psychology at Vanderbilt University, former Director of the Center for Evaluation and Program Improvement at Peabody College, and Research Professor at Florida International University. Dr. Bickman is a nationally recognized leader in program evaluation and mental health services research on children and adolescents. He is past president of the American Evaluation Association (AEA) and the Society for the Psychological Study of Social Issues. Dr. Bickman's contributions have been recognized with the Outstanding Evaluation Award from the AEA, the Award for Distinguished Contributions to Research in Public Policy from the American Psychological Association, and the Secretary's Award for Distinguished Service from the U.S. Department of Health and Human Services, among other honors.
Contributors
Marvin C. Alkin, PhD, School of Education and Information Studies, University of California, Los Angeles, Los Angeles, CAThomas Archibald, PhD, Department of Agricultural, Leadership, and Community Education, Virginia Tech University, Blacksburg, VA
Donna Durant Atkinson, PhD, Behavioral Health and Health Policy, Westat, Rockville, MD
Tarek Azzam, PhD, Department of Education, University of California, Santa Barbara, Santa Barbara, CA
Michael Bamberger, PhD, International Initiative for Impact Evaluation, Seattle, WA
Gail Vallance Barrington, PhD, Writer and Evaluation Advisor, Calgary, Canada
Leonard Bickman, PhD, Department of Psychology, Florida International University, Miami, FL
A. Brooks Bowden, PhD, Graduate School of Education, University of Pennsylvania, Philadelphia, PA
Jane Buckley, MEd, Independent Consultant, West Henrietta, NY
Stewart I. Donaldson, PhD, Claremont Evaluation Center and The Evaluators’ Institute, Claremont Graduate University, Claremont, CA
Sarah Douville, MBA, Independent Consultant, Montclair, CA
David M. Fetterman, PhD, Fetterman and Associates, Claremont, CA
Joy Frechtling, PhD, Education Studies, Westat, Rockville, MD
Emily F. Gates, PhD, Department of Measurement, Evaluation, Statistics, and Assessment, Boston College, Chestnut Hill, MA
Erica Harbatkin, PhD, Department of Educational Leadership and Policy Studies, Anne Spencer Daves College of Education, Health, and Human Services, Florida State University, Tallahassee, FL
Gary T. Henry, PhD, College of Education and Human Development, School of Education, and the Joseph R. Biden, Jr. School of Public Policy and Administration, University of Delaware, Newark, DE
Kimberly Eaton Hoagwood, PhD, Department of Child and Adolescent Psychiatry, New York University Langone Health, New York, NY
Natalie D. Jones, PhD, Center for Evaluation and Assessment, Gevirtz Graduate School of Education, University of California, Santa Barbara, Santa Barbara, CA
Jean A. King, PhD, Department of Organizational Leadership, Policy, and Development, University of Minnesota, Minneapolis, MN
Ciara Knight, PhD, Sankofa Consulting, Seattle, WA
Henry M. Levin, PhD, Department of Economics and Education, Teachers College, Columbia University, New York, NY
Arnold Love, PhD, Arnold Love and Associates, Toronto, Canada
Melvin M. Mark, PhD, Department of Psychology, Pennsylvania State University, University Park, PA
Michael Morris, PhD, Department of Psychology, University of New Haven, West Haven, CT
Glenn O’Neil, PhD, Owl RE, Commugny, Switzerland
Michael Quinn Patton, PhD, Utilization-Focused Evaluation, Pine City, MN
Laura R. Peck, PhD, Income Security and Economic Mobility Domain, MEF Associates, Alexandria, VA
Lam D. Pham, PhD, Department of Educational Evaluation and Policy Analysis, North Carolina State University, Raleigh, NC
Byron J. Powell, PhD, Center for Dissemination and Implementation and Center for Mental Health Services Research, Brown School, Washington University in St. Louis, St. Louis, MO
Sharon F. Rallis, EdD, Department of Education Policy and Reform, University of Massachusetts, Amherst, Amherst, MA
Liliana Rodríguez-Campos, PhD, Department of Educational and Psychological Studies, University of South Florida, Tampa, FL
Debra J. Rog, PhD, Social and Economic Policy Research, Westat, Rockville, MD
Darlene F. Russ-Eft, PhD, Department of Technology Leadership and Innovation, Purdue University, West Lafayette, IN
Thomas A. Schwandt, PhD, Department of Educational Psychology, University of Illinois at Urbana–Champaign, Champaign, IL
Robert Shand, PhD, School of Education, American University, Washington, DC
Guy O’Grady Sharrock, MSc, Catholic Relief Services, Baltimore, MD
R. Bradley Snyder, MPA, EdM, New Amsterdam Consulting, Inc., Scottsdale, AZ
Piper Grandjean Targos, MA, Ascending Edge Creative Evaluation, Evergreen, CO
Janet Usinger, PhD, College of Education and State Extension Specialist, University of Nevada, Reno, Reno, NV
Anne T. Vo, PhD, Department of Health Systems Science, Kaiser Permanente Bernard J. Tyson School of Medicine, Pasadena, CA
Ann P. Zukoski, DrPh, MPH, Center for Health Promotion, Minnesota Department of Health, St. Paul, MN