Theories of Change in Reality

Andrew Koleros is a Principal Researcher at Mathematica with 20 years of experience in designing and delivering mixed-methods evaluations and programme monitoring, evaluation and learning systems for both small-scale and largescale social and economic development projects. He brings particular expertise in using theory-based approaches that integrate complexity concepts and systems thinking into programme and evaluation design processes to address complex societal problems from safe and affordable housing to youth employment to advancing health equity. He obtained his master’s in public health from the Tulane School of Public Health and Tropical Medicine. He has published on his work in designing theory-based evaluation approaches in the American Journal of Evaluation, the Canadian Journal of Programme Evaluation and in multiple practitioner settings. He is a member of the American Evaluation Association, the European Evaluation Society, and the Canadian Evaluation Society. Marie-Hélène Adrien is a senior evaluation practitioner and researcher with 35 years of experience in evaluating the performance of programmes and institutions financed by bilateral and multilateral agencies and philanthropies around the globe. Dr. Adrien is a former CEO of the consulting firm Universalia Management Group in Montreal and was the President of the International Development Evaluation Association (IDEAS), as well as a professor of practice at the International Development Studies Program of McGill University. Tony Tyrrell is an independent consultant with more than 25 years’ experience in evaluation and related fields. Tony’s early experience in evaluation was with the European Social Fund Evaluation Unit where he produced influential evaluations on various subjects including early school leaving and local development. He later worked with a number of private consulting firms on strategic, policy, and programme evaluation and on performance management. More recently he spent six years with the Independent Evaluation Group at the World Bank Group (WBG). Tony now works as an independent consultant with various clients including the WBG, the Asian Development Bank, various governments, and private consultancies. He holds an MSc in Management (Organisation Behavior), and an MA in English Literature. Tony was co-editor (with Burt Perrin) of another book in this series, Changing Bureaucracies (2021).

Foreword

Emily Gates and Kathy Chau Rohn

Part 1. Introduction

Andrew Koleros, Marie-Hélène Adrien, and Tony Tyrrell

Part 2. Considerations in Using Theories of Change to Establish Causality

2. Theories of Change between Critical Thinking and Social Practices

Peter Dahler-Larsen

3. Setting the Stage for Contribution Claims

Thomas Delahais

4. Using Theories of Change to Assess Causality in a Policy Change Context

Carlisle Levine

5. How Deep is your Ontology? How a Little Ontological Thinking Can Strengthen How Evaluators Use Theories of Change

Tom Ling

6. My Perspective on Theories of Change

Markus Palenberg

Part 3. Using Theories of Change to Understand Wider Social Change Processes

7. Theory of Change for Sustainable Business

Jens Anderson

8. Upcycling TOC for Impact Investment and Early-Stage Ventures

Penny Hawkins and Zazie Tolmer

9. Strategy-level Theories of Change Require a Focus on Systems Change: An Actor-Based Approach can Help

Andrew Koleros

10. Evaluating the use of Artificial Intelligence and Big Data (ai/bd) in Policy Making: Unpacking Black-Boxes and Testing White-Boxes

Frans Leeuw

11. Developing, Representing and Using Theories of Change for Interventions in Complex Systems Patricia Rogers

Part 4. Adapting Theories of Change for Use in Broader Public Policy Contexts

12. Theories of Change in Evaluation of Local Government Reforms

Kurt Houlberg and Olaf Rieper

13. Theories-of-Change and the Evaluation of Sustainable Impact: Moving beyond Simplicity in Development Cooperation

Peter van der Knapp

14. Use of Theory of Change as a Management Tool for Government Multi-year Development Plans: The Case of Brazil’s Federal PPA

Lycia Lima and Marina Lafer

15. Theories of Change in Complex Macro Public Sector Planning Settings in Africa: How Useful are They?

Candice Morkel

16. Simple Heuristics for Drafting Theories of Change: A Case of Behavioral Insights into Food Waste

Karol Olejniczak and Igor Lyubashenko

Part 5. Applying Theories of Change Approaches for Multiple Purposes

17. The Law of the Instrument: Would you Rather be a Theory or a Nail?

Gordon Freer

18. 3ie: A ‘Balloon-Squeezing’ Approach to the Theory of Change

Marie Gaarder

19. Integrating Theories of Change in Program Management and Delivery

Mark Oldenbeuving

20. Theories of Change: Who Needs Them? or: What Evaluators Can Learn from Opera

Burt Perrin

21. How to Ensure No-one Uses Your Theory of Change: Lessons from the Front Lines of ToC Facilitation and Possibilities for Renewal

Mary Tangelder

22. Why do we have ToCs of the Program Intervention but not of the Intervention that is the Evaluation?

Bob Williams

Part 6. Developing Theories of Change Reflective of Multiple Stakeholder Perspectives

23. Participatory Explorations of Alternative Futures: Using Narrative Theories of Change

Rick Davies, Lara Mani and Tom Hobson

24. Constructing a Living Theory of Change in Fragile Contexts: Grounded on Context, Problem and Evidence

Hur Hassnain

25. Flipping the Script on Program Theories: Advancing Towards Transformative Theories of Change

Sebastian Lemire

26. Theory Based Evaluation Approaches Can Enable Online Project Success

Steve Montague, Heloise Emdon and Eva Grabinski

27. Theory of Change as a Tool for Tracking Intensive Family Program Developments in Whitetown

Jane Mulcahey, Catherine Naughton and Sean Redmond

Conclusions

Andrew Koleros, Marie-Hélène Adrien, and Tony Tyrrell