Theory of change

People developing their Theory of Change in a workshop

Theory of Change (ToC) is a specific type of methodology for planning, participation, and evaluation that is used in the philanthropy, not-for-profit and government sectors to promote social change. Theory of Change defines long-term goals and then maps backward to identify necessary preconditions.[1] Theory of Change explains the process of change by outlining causal linkages in an initiative, i.e., its shorter-term, intermediate, and longer-term outcomes. The identified changes are mapped –as the “outcomes pathway” – showing each outcome in logical relationship to all the others, as well as chronological flow. The links between outcomes are explained by “rationales” or statements of why one outcome is thought to be a prerequisite for another.[2] The innovation of Theory of Change lies (1) in making the distinction between desired and actual outcomes, and (2) in requiring stakeholders to model their desired outcomes before they decide on forms of intervention to achieve those outcomes. Theory of Change can begin at any stage of an initiative, depending on the intended use. A theory developed at the outset is best at informing the planning of an initiative. Having worked out a change model, practitioners can make more informed decisions about strategy and tactics. As monitoring and evaluation data become available, stakeholders can periodically refine the Theory of Change as the evidence indicates. A Theory of Change can be developed retrospectively by reading program documents, talking to stakeholders and using monitoring and evaluation data. This is often done during evaluations reflecting what has worked or not in order to understand the past and plan for the future. A common error in describing Theory of Change is the belief that it is simply a methodology for planning and evaluation.[3] Theory of Change is instead a form of critical theory that ensures a transparent distribution of power dynamics. Further, the process is necessarily inclusive of many perspectives and participants in achieving solutions.

History

Theory of Change emerged from the field of program theory and program evaluation in the mid 1990s as a new way of analyzing the theories motivating programs and initiatives working for social and political change.[4] Theory of Change is focused not just on generating knowledge about whether a program is effective, but also on explaining what methods it uses to be effective.[5] Theory of Change as a concept has strong roots in a number of disciplines, including environmental and organizational psychology, but has also increasingly been connected to sociology and political science.[6] Within industrial-organizational psychology, Austin and Bartunek have noted that approaches to organizational development are frequently based on more or less explicit assumptions about 1) the processes through which organizations change, and 2) the interventions needed to effect change.[7] Within evaluation practice, Theory of Change emerged in the 1990s at the Aspen Institute Roundtable on Community Change as a means to model and evaluate comprehensive community initiatives. Notable methodologists, such as Huey Chen, Peter Rossi, Michael Quinn Patton, Heléne Clark, and Carol Weiss, had been thinking about how to apply program theories to evaluation since 1980. The Roundtable’s early work focused on working through the challenges of evaluating complex community initiatives. This work culminated in a 1995 publication, ‘New Approaches to Evaluating Comprehensive Community Initiatives’.[4] In that book, Carol Weiss, a member of the Roundtable’s steering committee on evaluation, hypothesized that a key reason complex programs are so difficult to evaluate is that the assumptions that inspire them are poorly articulated. She argued that stakeholders of complex community initiatives typically are unclear about how the change process will unfold and therefore place little attention on the early and mid-term changes needed to reach a longer term goal.

Weiss popularized the term “Theory of Change” as a way to describe the set of assumptions that explain both the mini-steps that lead to the long-term goal of interest and the connections between program activities and outcomes that occur at each step of the way. She challenged designers of complex community-based initiatives to be specific about the theories of change guiding their work and suggested that doing so would improve their overall evaluation plans and would strengthen their ability to claim credit for outcomes that were predicted in their theory. She called for the use of an approach that, at first glance, seems like common sense: lay out the sequence of outcomes that are expected to occur as the result of an intervention, and plan an evaluation strategy around tracking whether these expected outcomes are actually produced. Her stature in the field, and the apparent promise of this idea, motivated a number of foundations to support the use of this technique—later termed “the Theory of Change approach”—in the evaluations of community change initiatives. In the years that followed, a number of evaluations were developed around this approach, fueling more interest in the field about its value and potential application.[4]

Between 2000 - 2002, the Aspen Roundtable for Community Change led the dissemination and case studies of the Theory of Change approach, although still mostly applied to the field of community initiatives. As the Aspen Roundtable concluded its leadership in the field and moved on to apply Theory of Change to such topics as structural racism, ActKnowledge[8] expanded the visibility and application of Theory of Change into international development, public health, human rights and more. The visibility and knowledge of Theory of Change grew with the creation in 2002 of theoryofchange.org[9] and later of Theory of Change Online[10] software.

Recent interest has burgeoned with some excellent reviews commissioned by Comic Relief in the UK, the Department for International Development in the UK, the Asia Foundation and Oxfam Australia to name a few.[11][12][13] The explosion of knowledge of the term, and demand for "theories", led to the formation in 2013 of the first non-profit dedicated to promoting and clarifying standards for Theory of Change. The Center for Theory of Change[9] houses a library, definitions, glossary and is licensed to offer Theory of Change Online by ActKnowledge[8] free of charge.

In the early days of Theory of Change, Anne Kubisch and others established three quality control criteria.[14] These are:

Plausibility refers to the logic of the outcomes pathway. Does it make sense? Are the outcomes in the right order? Are the preconditions each necessary and collectively sufficient to reach the long-term outcomes and ultimate impact? Are there gaps in the logic? Feasibility refers to whether the initiative can realistically achieve its long-term outcomes and impact. Does the organization have adequate resources? Does it need partners? Does the scope, expectations, or timeline of the theory need adjustment? Testability refers chiefly to the indicators: Are they solid and measurable? Will they yield sufficient information to evaluate the success of the initiative? Will they be convincing to necessary audiences? In addition to these three basic quality control criteria, ActKnowledge[8] has added another key criterion: Appropriate Scope.[14] An actionable theory that can be communicated to the key audiences is dependent in part upon choosing the right scope: broad enough to leave no gaps in the model, yet focused enough on the opportunities and resources at hand. Appropriate Scope also integrates the evaluation concept of “accountability”. Many Theory of Change outcome pathways include an “accountability ceiling,” often a dashed line drawn across the pathway that separates outcomes the organization will monitor and claim credit for attaining from higher-order outcomes that are beyond its power to achieve—e.g., “a just society.”[15]

Basic structure

The outcomes pathway is a set of needed conditions relevant to a given field of action, which are placed diagrammatically in logical relationship to one another and connected with arrows that posit causality. Outcomes along the pathway are also preconditions to outcomes above them. Thus, early outcomes must be in place for intermediate outcomes to be achieved; intermediate outcomes must be in place for the next set of outcomes to be achieved; and so on. An outcomes pathway therefore represents the change logic and its underlying set of assumptions, which are spelled out in the rationales given for why specific connections exist between outcomes and in the theory narrative.[3]

An Outcomes Pathway mapped out

Applying the model

An important first step in the process is identifying a workable long-term goal and long-term outcomes. The long-term goal should be something the initiative can realistically achieve and that everyone involved understands. A trained external facilitator is best to lead the group to consensus and specificity in this process. Once a long-term goal is identified, the group then considers: “What conditions must be in place for us to reach the goal? Any such necessary conditions should be shown as outcomes on the Theory of Change pathway, underneath the long-term outcome. These outcomes act as preconditions to the long-term outcome. The process of identifying preconditions continues, drilling down the pathway by posing fundamental questions such as: “What has to be in place for this outcome to be achieved?” and “Are these preconditions sufficient for the outcome to be achieved?” In these sessions, participants may use markers, sticky notes, and chart paper to identify and organize outcomes, surface assumptions, develop indicators, and so on. The messy group work is then usually captured by the facilitator in digital form, through which the content can be expanded, edited, printed, shared, and otherwise managed as the theory continues to be developed.

Theory of change in process and action

Measuring change

The ultimate success of any Theory of Change lies in its ability to demonstrate progress on the achievement of outcomes. Evidence of success confirms the theory and indicates that the initiative is effective. Therefore, the outcomes in a Theory of Change must be coupled with indicators that guide and facilitate measurement. Indicators may be said to operationalize the outcomes – that is, they make the outcomes understandable in concrete, observable and measurable terms. The relationship of indicator to outcome can be confusing and may be clarified with this simple formula: “I’ll know [outcome reached] when I see [indicator].” For example, “I’ll know that teenagers in the program understand the prenatal nutrition and health guidelines when I see program participants identifying foods that are good sources of nutrition.” Ideally, every outcome on the outcomes pathway (below the dashed accountability ceiling) should have an indicator, but available resources often make that difficult to do. Many groups want to designate priority outcomes – that is, outcomes they know they need to measure if the theory is going to hold. These are the outcomes that must be operationalized (that is, made measurable by one or more indicators.) At a minimum, every outcome for which initial interventions will be designed should have at least one indicator.[3]

Evaluation and monitoring

As the origins of Theory of Change lie in the field of evaluation and monitoring, developments over the years have ensured that Theory of Change continues to be an invaluable method to conduct evaluations of many different types of projects and organizations. Often posing theory-based evaluation questions helps to focus evaluation efforts on key concerns. As well, there may be a need to pick the right indicators from among the many available, and one can use “monitoring questions” to select the indicators that will be most helpful. The monitoring questions take the form of “What do we really need to know in order to manage grant-making directed to the achievement of this outcome?[16] It is important to understand success beyond just knowing “what works”. Experience has shown that blindly copying or scaling an intervention hardly ever works. An important task for monitoring and evaluation is to gather enough knowledge and understanding so as to be able to predict – with some degree of confidence – how an initiative and set of activities might work in a different situation, or how it needs to be adjusted to get similar or better results. We also need to be able to combine evidence from a number of studies in order to build a stronger picture of what is taking place, how it is unfolding, and, most importantly, how context influences the initiative.

Just as development of a Theory of Change is a participatory process, a ToC-based monitoring and evaluation system can be designed in a participatory way. For example, grant managers can be involved in choosing the outcomes of greatest interest to them in their decision-making. Similarly, people on the ground can have input into which indicators to use and how to operationalize them, choices of instruments and methods of data collection, and which existing sources of data may be used in tracking indicators.[17]

Growth

The use of Theory of Change in planning and evaluation has increased exponentially among philanthropies, government agencies, international NGOs, the UN, and many other major organizations in both developed and developing countries. This has led to new areas of work, such as linking the Theory of Change approach to systems thinking and complexity. Change processes are no longer seen as linear, but as having many feedback loops that need to be understood. Consequently, Theory of Change is strengthening monitoring, evaluation and learning. They are also helping to understand and assess impact in hard to measure areas, such as governance, capacity strengthening and institutional development. Innovations continue to emerge.[18] Despite the growing ubiquity of Theory of Change, however, especially in the development arena, understanding of the approach and the methods necessary to implement it effectively are not uniform. In fact, there is evidence of some confusion about what the term ‘theory of change’ actually means; in some cases, what some program developers describe as a Theory of Change is, in essence, simply log frame, strategic plan or another approach that does not encompass the complexity of the theory of change approach.

Comparison with other models

Practitioners have developed logic models and logical frameworks as strategies and tools to plan and evaluate social change programs. While these models well articulate the goals and resources of an initiative or organization, they give less focus to the complex social, economic, political and institutional processes that underlie social and societal change. Thus, while logic models and logframes have developed an Implementation Theory behind their work, they can lack an underlying Theory of Change.[19] Theory of Change also contrasts with logic models and logframes by beginning with a participatory process to clearly define desired outcomes and to air and challenge one another’s assumptions. Theory of Change begins by first working out program goals or desired impact and working backwards on outcome pathways, rather than engaging in conventional forward oriented “so-that” reasoning. As an example of "so-that" reasoning: a grantee decides to increase media coverage on the lack of health insurance among children so that public awareness increases so that policymakers increase their knowledge and interest so that policies change so that more children have health insurance. In Theory of Change, by contrast, the group begins not with its intervention but with its long-term goal and outcomes and then works backward (in time) toward the earliest changes that need to occur. Only when the pathway has been developed is it time to consider which interventions will best produce the outcomes in the pathway.

Many organizations, including the Rockefeller Foundation and the United States Agency for International Development, have used a Results Framework and companion Scorecard as management tools.[3] The Results Framework is complementary and adaptable to a Theory of Change-based monitoring and evaluation system. The framework gives the appearance of being derived from a well-thought-out conceptual model, even if the conceptual model is lacking. The limitations of the Results Frameworks is that they do not show causal connections between conditions that need to change in order to meet the ultimate goals. The added value of Theory of Change lies in revealing the conceptual model, including the causal relationships between and among outcomes, the relationships of activities to outcomes, and of outcomes to indicators. Overall, having a Theory of Change helps make explicit the assumptions upon which the Results Framework is based.

Applications

ToC in International Development[11]

State agencies

American foundations

Non-government organizations

ToC in community schools (U. S.)

ToC in philanthropy

Innovations

New horizons of theory of change

There are two areas of work that, although not coordinated with Theory of Change, offer much to think about in making Theory of Change more focused and effective:

1. The Annie E. Casey Foundation proposes mapping an organization’s social change work along three criteria: Impact, Influence, Leverage.

To date, Theory of Change has not distinguished impact, influence, and leverage as types of outcomes, but it may be useful to do so as a way of focusing the Theory of Change on measurable achievements. Particularly, when using Theory of Change to guide monitoring and evaluation, the Casey rubric helps focus the group’s attention on outcomes, which could, if achieved, be convincingly attributed to the group’s work. Other than direct program-related outcomes (impact), the Theory would anticipate outcomes in influence and outcomes in leverage. This approach could thereby help to avoid mapping outcomes involving broad shifts in behavior and values among whole populations, which are easy to think about, but are very difficult to monitor and to attribute to any one program.

2. Another refinement, which directly addresses this problem of attribution, comes from Outcomes Mapping. This process distinguishes changes in state from changes in behavior, changes in “state” being just those broad shifts in economic conditions, policy, politics, institutional behavior, and so on, among whole populations (e.g., cities, regions, countries, industries, economic sectors, etc.). Measuring changes in state can exceed the capacity of any one actor’s monitoring capabilities. Governments collect data on changes in state but, of course, the data may not be calibrated to measure the kinds of change anticipated in any one Theory of Change. Changes in state are also, as stated above, difficult to attribute to any one source.

In contrast, changes in behavior are much easier to monitor, and more easily related to a group’s own work. The Outcomes Mapping focus on changes in behavior would tend to direct a Theory of Change toward outcomes like this, which are outcomes the change agent cares most about and which it can relatively easily monitor and evaluate. There would be proportionately less attention to outcomes such as “every child is within five minutes walk of a playground” or “residents are healthy”. Such “changes in state” are more difficult to monitor and to attribute with certainty.[21]

Theory of change and ‘being strategic'

Does Theory of Change frustrate or complement strategic thinking? This is an ongoing and important discussion, especially as there is an increasing emphasis on and demand for strategy as grounded, flexible, and opportunity-driven. Some perspectives understand ToC as a fixed model that gets in the way of effective work and useful evaluation. However, Patrizi notes that ToC is only at odds with strategic behavior if an organization treats their ToC like any other fixed plan. As Patrizi writes: “Once assumptions [in a theory of change] are laid out, 1) foundations don't actually test those assumptions, and 2) they don't see using the model as a continuous process. It is, like, ‘Well, we did our ToC and now we are done’.[17] If the change model is instead treated as something to adjust as organizations learn what works from experience in the field, then the theory should not be at odds with strategic behavior. If strategy is about seizing opportunities and trying out what works, the Theory of Change model serves as a guiding frame of reference. A list is not a model; a list does not push practitioners to consider the goals as part of a systematic model of change, or to think critically—strategically—about how best to attain the outcomes along the pathway.

Limitations and function of a linear model

Given that that things don’t happen in a straight-line sequence – as things impact each other in multiple, partly unpredictable ways, with all kinds of feedback loops that aren’t modeled in a top-down diagramming format – an important question is: How adequate is the linear Theory of Change model as a description of what’s going to happen? One answer to the question is that Theory of Change does not, in fact, model how things happen; rather, it models how we believe things will happen. Theory of Change is a forecast that shows what conditions we believe must exist for other conditions to come into being. Because it is forward looking and logical, Theory of Change reflects the way we think logically –that is, if a, then b—and chronologically—first this, then that. The linear format is therefore appropriate. It can be helpful to complement Theory of Change with a process model that shows how the Theory of Change fits into a larger, more cyclical scheme in which theory leads to action, which leads to monitoring and evaluation, which leads to adjustment of the theory, which leads to the next action, more monitoring and evaluation, and so on. Such a process model depicts the linear theory as a conceptual driver of change, which must, to remain useful, be accompanied not only by taking action but also by evaluation and recalibration.[15]

Getting ‘buy-in’ from senior leadership

It is important to remember that it is often a high-level person who has endorsed and initiated the Theory of Change process as something of value, so they have “bought in” at the beginning. The challenge that emerges is, therefore, how to sustain their support while letting others develop the theory. This is similar to many other issues that come up in matters of hierarchy and delegation: effective leaders delegate and trust their teams to carry out the work. There are many variations on the model but usually it involves good measures of delegation and, conversely, of reporting back to get the leader’s thinking as the work progresses. It is necessary to come to high-level people having laid out your best thinking: don’t go to them without something concrete to which they can respond, but don’t wait until everything is perfect, either.[1]

References

  1. ↑ 1.0 1.1 P. Brest (2010). "The Power of Theories of Change". Stanford Social Innovation Review. Spring.
  2. ↑ H. Clark and D. Taplin (2012). Theory of Change Basics: A Primer on Theory of Change (PDF). New York: Actknowledge.
  3. ↑ 3.0 3.1 3.2 3.3 D. Taplin, H. Clark, E. Collins and D. Colby (2013). Technical Papers: A Series of Papers to support Development of Theories of Change Based on Practice in the Field (PDF). New York: Actknowledge and The Rockefeller Foundation.
  4. ↑ 4.0 4.1 4.2 C. Weiss (1995). Nothing as Practical as Good Theory: Exploring Theory-Based Evaluation for Comprehensive Community Initiatives for Children and Families (Connell, J, Kubisch, A, Schorr, L, and Weiss, C. (Eds.) ‘New Approaches to Evaluating Community Initiatives’ ed.). Washington, DC: Aspen Institute.
  5. ↑ C. Chris et. al (2011). "A Systematic Review of Theory-Driven Evaluation Practice from 1990 to 2009". American Journal of Evaluation 32 (2): 199–226. doi:10.1177/1098214010389321.
  6. ↑ S. Stachowiak (2010). Pathways for Change: 6 Theories about How Policy Change Happens. Seattle: Organisational Research Services.
  7. ↑ J. Austin, and J. Bartunek (2004). "Theories and Practice of Organization Development". Handbook of Psychology 12: 309–332. doi:10.1002/0471264385.wei1213.
  8. ↑ 8.0 8.1 8.2 actknowledge.org
  9. ↑ 9.0 9.1 theoryofchange.org
  10. ↑ http://www.theoryofchange.org/toco-software/
  11. ↑ 11.0 11.1 D. Stein and C. Valters (2012). Understanding Theory of Change in International Development. London: The Justice and Security Research Programme, London School of Economics.
  12. ↑ C. James (2011). Theory of Change Review: A report commissioned by Comic Relief. London: Comic Relief.
  13. ↑ E. Collins and H. Clark (2013). Supporting Young People to Make Change Happen: A Review of Theories of Change. ActKnowledge and Oxfam Australia.
  14. ↑ A. Kubisch (1997). Voices from the field: Learning from the early work of comprehensive community initiatives. Washington, DC: Aspen Institute.
  15. ↑ 15.0 15.1 H. Clark (2004). Deciding the Scope of a Theory of Change. New York: ActKnowledge.
  16. ↑ C. Weiss (1998). "Have We Learned Anything New About the Use of Evaluation?". American Journal of Evaluation 19: 21–33. doi:10.1177/109821409801900103.
  17. ↑ 17.0 17.1 P. Patrizi and M. Quinn Patton (2010). "New Directions for Evaluation". Evaluating Strategy 128.
  18. ↑ E. Jackson (2013). "Interrogating the theory of change: evaluating impact investing where it matters most". Journal of Sustainable Finance & Investment 3:2: 95–110.
  19. ↑ S. Funnell and P. Rogers (2011). Purposeful Program Theory: Effective Use of Theories of Change and Logic Models. San Francisco, CA: Jossey Bass.
  20. ↑ Theory of Change: A Practical Tool For Action, Results and Learning Prepared for the Annie Casey Foundation. Seattle: Organizational Research Services. 2004.
  21. ↑ S. Earl, F. Carden, and T. Smutylo (2001). Outcome mapping: Building learning and reflection into development programs. Ottawa: International Development Research Centre.

Sources

External links

Official YouTube Page https://www.youtube.com/channel/UCH8hjKAQ-aJZB-sqEy9STAA