Nancy Rosenbaum
Director of Research and Evaluation, National Foundation for Teaching Entrepreneurship (NFTE)
Many in the non-profit community have an aversion to evaluation because they say it costs too much. With funding for programs in short supply, leaders in many non-profit organizations would rather devote resources to services than to evaluation projects. Up until recently, this was certainly the case at the National Foundation for Teaching Entrepreneurship (NFTE) a youth development organization where I work.
We believed that professional evaluators were people who spoke an inaccessible academic jargon and produced costly and unwieldy, telephone book-sized reports that were of little use to our organization. Yet today, NFTE is firmly committed to research and evaluation. The fact that I am writing this as NFTE�s first-ever Director of Research and Evaluation is a testament to that commitment. What led to this change and how has it affected our work?
One Good Experience
The new belief in the benefits of program evaluation didn�t happen overnight. It took time for our leadership to come to see evaluation is a smart investment with tangible benefits. It took having a positive experience with an evaluator for our organization to revise its� deeply-held perception that evaluation was a wasteful use of precious financial resources.
We first got involved with evaluation in the mid 1990s, when one of our funders urged us to conduct an impact study designed to assess whether or not our youth entrepreneurship programs were leading to measurable benefits for students. As can be the case with many non-profits, we didn�t have much of a choice about evaluation�a funder was driving the process. The result was a four-year research initiative led by Dr. Andrew Hahn at the Heller School for Social Policy and Management at Brandeis University. To our surprise, working with an evaluator was a very positive experience. The process provided an opportunity for staff to reflect critically on our work instead of functioning exclusively in program operations mode.
At the end of the study, Brandeis University published a number of positive and encouraging findings. This helped to improve staff morale. While our staff knew intuitively that our programs are working, it made a difference to have an external evaluator verify our beliefs. In NFTE�s case, we found that we were being effective in helping at-risk and low-income youth to learn about entrepreneurship and that NFTE participation increased students� confidence in their ability to start their own businesses. These results helped to establish our credibility as an organization, both internally, with our staff, and externally with funders, stakeholders, policy makers, and the media.
Our work with Dr. Hahn and Brandeis University also uncovered areas where we needed to improve. For instance, the evaluators pointed out that we lacked a uniform system for training teachers and youth workers to run our programs in their local communities. If we wanted to continue to grow, we would need to figure out how to be more effective in our teacher training efforts. Today, our standardized �NFTE University� trainings are run as 5-day summer programs at Georgetown University, Babson College, Yale University, Stanford University, Carnegie Mellon University, and Teachers College at Columbia University.
Show What You Do Works
What are some of the other benefits of evaluation? Beyond helping non-profits to improve programs, evaluation can leverage an organization�s ability to attract support from the foundation community. Positive evaluation results, when they come from an independent evaluation, provide objective evidence that what you do works. Just as important, your organization�s commitment to assessment demonstrates an ongoing dedication to organizational improvement� something that is increasingly important in today�s funding climate. By outlining the steps you have taken to make changes based on evaluation results, you demonstrate your capacity to adapt your programs to the needs of those you serve. For many funders, this can distinguish your organization as a leader in its field.
Here at NFTE, we are working towards allocating ten percent of our general operating budget for research and evaluation. This shouldn�t be regarded as an �industry standard�� in our case it came about as the result of the comprehensive strategic growth plan we created for our organization. Your organizations may approach the issue in a different way. Unfortunately, although most funders want you to be able to demonstrate the effectiveness of the programs they fund, few are willing to subsidize the work required.
Opportunity Costs
How can you address this challenge? For starters, you can build evaluation in to your program and general operating proposal budgets as a fixed cost line item. You can also package an evaluation study as you would a program, with its own discrete time frame, features, and activities. It is also useful to consider how you might position the evaluation as intellectual capital that can benefit the field at large, rather than just having significance for your particular organization. With some grantmakers, this creates a stronger �case� for convincing funders to invest in your evaluation work.
Another possibility is to plan very low cost evaluation. For example, you might reach out to a local university and find a graduate student who can execute a small-scale evaluation project in exchange for internship credit. Remember that you can always start small by evaluating a single program or discrete set of activities. You can always ramp up your evaluation efforts over time.
At NFTE, we like to teach our students about opportunity costs, which we define as �the value of what must be given up to obtain something else�. In the short term, it may seem that precious funds would be better spent on programs than on evaluation. But then you need to ask� What are the costs of not knowing that a particular program is not working as well as it could? What, in other words are the opportunity costs incurred by pouring money into programs that aren�t fulfilling their potential? While evaluation may seem �too expensive� in the short term, in the long run, you may be paying a higher price by not evaluating the services you are providing.
If you think about it from a business standpoint� which is what we teach our students to do in NFTE programs� we have gotten a considerable return on investment from the evaluations we have conducted. As a direct result of the Brandeis University study, we learned what worked and what didn�t work and were able to better allocate our resources where they could have the greatest impact.
Evaluation can help you to plan and spend �smarter�. At a time of declining resources, combined with an increasing demand to demonstrate �outcomes�, you need to ask yourself, can my organization afford not to engage in evaluation?
