Nonprofit Program Evaluation Made Simple

Get your Data. Show your Impact. Improve your Programs.

Chari Smith

• Publisher: Chari Smith
• Year: 2020
• ISBN:
  9781736315910
• Edition: 1
• Topic: Business
• Subtopic: Nonprofit Organizations & Charities
• Index: Business

Are you overwhelmed on how to do nonprofit program evaluation?

You're not alone. Chari's here to help!

There are many ways to do program evaluation, making it difficult to know how to start. In this book, Chari outlines a clear approach, filled with real world stories as well as examples of evaluation plans, surveys, and reports.

Key topics addressed:

• Understand how to build buy-in for evaluation and address staff resistance and make a realistic program evaluation plan
• Create measurable outcomes for both grant applications and to guide program improvement
• Develop an impact and/or logic model that visually communicates what your program does and the difference it makes
• Create useful surveys that measure what matters
• Understand the choices in how to manage your data - spreadsheets v. database solutions
• Basic data analysis and reporting to make meaning of your data

Included with the book is a link to a companion website filled with downloadable real world examples and templates.

About Chari Smith

Chari Smith believes evaluation should be accessible, practical and usable. She founded Evaluation into Action to help nonprofit professionals create realistic and meaningful program evaluation processes. She has taught several workshops helping nonprofit professionals understand the value and use of program evaluation. She is also a wife, mother, jazz pianist, singer/songwriter, and devoted yogi.

Here's What People Are Saying about Nonprofit Program Evaluation Made Simple

The ability to measure impact and share that success with the world is foundational to a thriving nonprofit and unlocks doors leading to visibility and funding. Leaders mistakenly believe that the process is too expensive, too timing consuming or that their kind of work can't be measured. Chari's approach to program evaluation will bust these myths and unlock these doors - offering a simple and accessible approach to your efforts to invite more people to know more and do more for your cause. Joan Garry, Author of Nonprofits are Messy, Principal at Joan Garry Consulting, and Founder of Nonprofit Leadership Lab

Understanding your organization's impact requires using data to learn and improve on a regular basis and that can be a daunting task.In this book, Chari Smith has demystified the process of creating an effective evaluation process for nonprofits.The book offers many practical tips that will help your organization improve results. Beth Kanter, Virtual Facilitator, Trainer, and Author

Making the complex simple, the complicated doable, the abstract concrete, and the frightening friendly takes a special combination of expertise, experience, and communications skill.Chari Smith brings all that and more to this accessible, important, and insightful book. Read, learn, and engage. Michael Quinn Patton, Founder and CEO, Uti

Part One

GET READY

Program evaluation is a learning tool. The key reason to do program evaluation is to learn what is going well and where improvements are needed.

Everyday life has situations where evaluation is required. For instance, imagine that an Olympic swimmer in training is never timed. They may feel like they’re swimming fast, and people might say, “You swim really fast!” but that’s all anecdotal evidence. Unless they know their times, how can they know how much faster they need to be to win?

Or imagine a middle school student who’s taking classes in social studies, math, literature, and science, but they never receive grades. Teachers tell them they’re a great student, and they take tests but don’t receive scores—just a pat on the back. How will anyone know if they’ve learned what they’re supposed to?

These situations show the need for athletes and students to evaluate their performance to improve. The same holds true for nonprofit organizations. Program evaluation gathers data to answer important questions so people in your organization can be sure they are making the difference they intend.

Joan Garry, founder of the Nonprofit Leadership Lab and author of Nonprofits are Messy, identifies fourteen attributes of a thriving nonprofit in her blog. The eighth one focuses on program evaluation:

Programs are evaluated to determine impact. It’s not enough to simply have anecdotes of impact. The best nonprofit organizations have mechanisms in place to measure success. It’s key to strategic planning, and yes, funders want to know too.

I couldn’t agree with this sentiment more. Too often, nonprofits feel the stories their participants tell them is enough. It’s not. To truly measure success, you need a systematic way to gather data that will clearly demonstrate the impact you’re making.

As Joan says, funders want to know the impact your program makes. Funders also want to know about the challenges as well as successes. As Michael Parkhurst, Program Officer for the Meyer Memorial Trust, says:

It can tell a powerful story about how your program is making a big difference for the people you are trying to help. The flipside is accountability. If you’re not measuring anything, then you’re not holding yourself accountable. Data that shows you’re NOT meeting your targets is telling you something important!

When done right, program evaluation fuels fundraising efforts. Most grant proposals require that forecast of what measurable outcomes your program will have, and then the report on those outcomes once the program is complete. Being able to demonstrate impact in a meaningful way catalyzes support from potential donors, foundations, and other funding sources.

Brandi Tuck, Executive Director with Portland Homeless Family Solutions, saw dramatic fundraising results through program evaluation:

Because of having this program evaluation system in place, our annual fundraising from foundations increased by 677 percent and annual fundraising from individual donors increased by 753 percent.

When she shared this, it nearly knocked me off my seat! Now, I still hold that the real reason to do program evaluation is for continuous program improvement. But this example of turning data into dollars is worth noting.

The following chapter provides an overview of what program evaluation is.

CHAPTER 1

What is Program Evaluation?

I knew we needed better data for our resident services program. I knew the work we were doing was making a difference. We had no formal structures in place to show our impact. We had difficulty communicating to grant funders the impact of our work. Funders didn’t understand what we were doing and why we were doing it. The data we were collecting weren’t effective.

The thought of doing program evaluation seemed like an overwhelming process. Some staff were resistant to the concept. I just didn’t know where to start.

In 2016, we contracted with Chari Smith, President/Founder of Evaluation into Action (EIA), to create an evaluation system for our program. This six-month process resulted in an increased ability to effectively communicate our impact, what our services are, and why we are offering them.

In fact, the data helped us secure two of our largest grants to date. Even though increasing our grant funding was the primary reason we started this process, the real success is our ongoing ability to gather data that are relevant to our work and use them to continually improve our program. In fact, we have been able to use these data to continually improve our program and offer more effective interventions, allowing us to package our work so that funders can clearly understand our program goals and impact.

In the beginning, I thought this was going to be hard. It wasn’t. The whole process was broken down into manageable steps, helping us to understand how to do program evaluation on our own. I am excited that examples from our program evaluation process are included in this book. Believe me, I know program evaluation can feel overwhelming, but it’s not. It’s one of the most valuable investments we’ve made.

—Julia Doty, Program Director, NW Housing Alternatives

Working with Julia’s team is just one example of how program evaluation can be integrated into your day-to-day operations. This book provides step-by-step guidance on how to do program evaluation. It makes it easy for you to get the data you need to improve your programs as well as show your impact.

Okay, you know you need to do program evaluation. So, what exactly is it? Kylie Hutchinson’s book, Survive and Thrive: Three Steps to Securing Your Program’s Sustainability, includes this great explanation:

Each program is unique, but our fundamental principles for establishing a useful program evaluation process are the same:

Excerpt: Demonstrating Your Worth

Evaluation 101
A Primer on Demonstrating Your Worth

Many of you will already be familiar with the term program evaluation. Here’s a popular definition from Dr. Michael Quinn Patton’s book, Essentials of Utilization-Focused Evaluation:

Program evaluation is the systematic collection and analysis of information about program activities, characteristics, and outcomes to make judgments about the program, improve program effectiveness, and/or inform decisions about future programming.

Below is a simple overview of the two most common types of evaluation.

Process Evaluation

A process evaluation (also called formative evaluation) focuses on how your program is being implemented and operates. It ensures that your program remains on track to achieve later outcomes. A process evaluation answers questions such as:

•Are we providing services as intended?

•What is working well and not working well with our services?

•Are we reaching our target market?

•Are participants satisfied with the program?

•What else could or should we be doing?

Outcome Evaluation

An outcome evaluation (also called summative evaluation) determines what outcomes and impacts have occurred as a result of your program. Outcomes are benefits or changes as a result of your program’s activities. It answers questions such as:

•Was the program effective?

•What difference did it make?

•Do our outcomes differ across different sites?

•Was it worth it given the overall time and resources invested?

Use a collaborative and inclusive process to define what you are measuring; measure it; then use the results.

Other evaluation methods are complex, requiring statistical analysis, randomized trials, and other methods requiring trained professionals.

Our approach here is based on the premise that you and your organization can build a culture of evaluation, collaboratively define measurable outcomes, create a plan, collect data, and report on and use the results.

Methods for gathering data include basic surveys coupled with stories from participants. This mixed-methods approach combines quantitative (numbers) and qualitative (stories) data to provide a comprehensive picture of how the program is fulfilling its outcomes.

CHAPTER 2

Understand the Terminology

Program evaluation includes a lot of terms that are used interchangeably by evaluators, funders, and nonprofit professionals. This can cause confusion about the meanings of terms like objective versus goal.

In this book, we use the terms listed below. A lot. Get comfortable with them, and don’t worry if you’ve heard th...