Process, Process, Process: The evaluation model you shouldn’t ignore
While many nonprofits focus on impact evaluations, process models can be invaluable for helping organizations make real, lasting impact.
Article Highlights:
In a nonprofit’s toolkit, evaluation is key for making meaningful adjustments that cultivate success.
As nonprofits, we are expected to be accountable to our boards, our staff, our funders, our partners, and most importantly, our clients or constituents. Among the many ways that we can demonstrate accountability is through evaluation of our programs, interventions, and services. But for those with limited resources, taking time out for thorough evaluation can feel like too much for too little reward. Too often, the results of evaluation can end with everyone asking, “So what?” or “Now what?”
Done well, however, evaluation provides critical, time-saving information to assist mission-driven efforts that will pay off in the long range. Even if most of those in nonprofit didn’t enter the sector to become bean counters, evaluation can keep us on track when we are doing something right and help us course correct when we are headed the wrong way. Given that, it’s important to know how to evaluate effectively.
While many nonprofits focus on impact evaluations, process models can be invaluable for helping organizations make real, lasting impact. In this article, we’ll learn about this valuable evaluation model and how to get started using it.
Impact vs. process evaluation models
Typically, when we think of evaluation, we think of impact or summative evaluation. Impact evaluations answer these types of questions:
- Does my intervention improve outcomes for clients?
- Is my program working to better the lives of the individuals we serve?
Funders often want impact evaluations because they help them know whether they are getting enough bang for their buck. Since their own success is measured by how greatly they solve a problem or influence a community, assurance that their philanthropic efforts created tangible results is key and summative evaluation helps them get that.
But those evaluations aren’t the whole story. Process evaluations are also critical if an organization wants to truly get the best results.
Process evaluations measure how program activities are delivered and assess how well they were implemented.
The Substance Abuse and Mental Health Services Administration (SAMHSA) offers this explanation for why process evaluations are important:
“Disappointing outcome evaluation results can frequently be illuminated by examining how the program was implemented, the number of clients served, dropout rates, and how clients experienced the program. Those same kinds of questions can also explain positive evaluation results. (You can’t take credit for positive results if you can’t show what caused them.)
Outcome evaluation alone, without a process evaluation component, won’t provide information about why a program did or didn’t work” (Using Process Evaluation to Monitor Program Implementation, SAMHSA).”
In other words, process evaluations help you understand the ingredients of your program so that if aren’t getting the results you want, you understand what components are critical for that success.
They ask questions like:
- What are the key elements of our service delivery?
- To what extent was the program implemented as planned?
For over-extended organizations, these questions may feel too luxurious, but I argue they are as important as the outcome of the work, and in fact, that they assure outcomes continue to improve.
If your agency uses logic models or Theories of Change, you may recognize these ingredients as the outputs, e.g. how many clients seen, how many trainings conducted. If your agency does not use logic models or Theories of Change, I strongly recommend doing so—they can be essential for understanding the who, what, why, when, and how of your programs.
Taking small steps with limited resources
No matter what, creating an evaluation strategy will take effort. I frequently assist nonprofits and public agencies develop their logic models and have been surprised at how often agencies have not fully articulated their program models.
For example, one agency described one of their core activities as outreach to students who may need guidance to keep them on a college track.
“Ok, what does that look like?” I asked.
“Well, we do outreach at schools.”
“Great. Exactly what does that look like?”
After lengthy back and forth, we finally identified what specific activities constituted outreach in their case (advertising the program in common areas of the target schools, educating teachers, parents, and school staff about their agency through attendance of PTA and professional development meetings, etc.). Once the activities are defined, they can be measured.
- How many schools received the agency program fliers?
- How many PTA meetings and/or teacher professional development meetings were attended by program staff?
The success of those outreach efforts could then be measured by looking at source of referrals for youth who come in for services. If 90 percent of youth state that they learned about the agency through fliers posted at their school, the agency would know to keep up that strategy and maybe augment it. Alternately, if 90 percent indicated that they learned about it through their home room teacher, that gives the agency a different picture of what may be the most effective form of outreach for their particular program.
Resources
Getting started can be overwhelming. This toolkit should help your organization begin its own process evaluation.
You might also like:
- Insider Newsletters: An Easy Way to Keep Your Board in the Loop and Engaged
- Board Horror Stories: How to Reduce Board Resignations
- Rebuilding Nonprofit Boards: When a New Board Can Be a Fresh Start
- Five Lessons for Creating an Effective (and Accountable) Nonprofit Board
- A Cautionary Tale: Overcoming Governance Nightmares in Nonprofit Organizations
You made it to the end! Please share this article!
Let’s help other nonprofit leaders succeed! Consider sharing this article with your friends and colleagues via email or social media.
About the Author
Blue Avocado is an online magazine fueled by a monthly newsletter designed to provide practical, tactical tips and tools to nonprofit leaders. A small but mighty team of committed social sector leaders produces the publication, enlisting content from a wide range of practitioners, funders, and experts.
Articles on Blue Avocado do not provide legal representation or legal advice and should not be used as a substitute for advice or legal counsel. Blue Avocado provides space for the nonprofit sector to express new ideas. The opinions and views expressed in this article are solely those of the authors. They do not purport to reflect or imply the opinions or views of Blue Avocado, its publisher, or affiliated organizations. Blue Avocado, its publisher, and affiliated organizations are not liable for website visitors’ use of the content on Blue Avocado nor for visitors’ decisions about using the Blue Avocado website.
Yeah, I’m a typo spotter. I enjoyed reading your article, but I stumbled over:
“But those evaluations aren’t the whole story. Process evaluations are also critical if an organization wants to truly how to get the best results.”
Just helping out. 😉