designing your own after school evaluation tool

(reprinted from Americans for the Arts' 12/09/2011 ArtsBlog post)

For decades now, after school and community arts programs have gotten funded based on their case studies and assertions as to the benefits of the arts. And why not? Those benefits are real, and incredibly valuable. But case studies and avowals aren’t exactly tangible and they just aren’t cutting it any more.

While “Prove It” has become funders’ new mantra, most after school groups are simply unprepared. How the heck do you prove those more intangible qualities that we all consider to be most important? How do you track and assess the light in a student’s eyes when he feels confident, empowered, and successful?

There are, of course, a number of evaluation software tools on the market today, but they’re often tedious to learn and operate. So here’s what we’ve been doing at Merge Education for the past few years: we’ve developed our own software. It’s available commercially now, but I'm writing this post is to share some of the points we came up with that you can step off of to design your own after school evaluation - an evaluation that will help you survive.

How did we develop these points? We worked with other educators, artists, mental health professionals, and evaluation scientists to drill down on the points that are actually the building blocks a person needs to develop in order to become more resilient and better able to learn. We call them the Fundamentals of Empowerment™.

As you’ll see in a moment, these fundamentals are essential to good human development, which is so often a major focus of after school programs..

How to Design Your Own Evaluation Tool

Where should you start? To clarify your thinking, break your questions down into three distinct areas.

First, look at the student’s relationship to herself – e.g., what is her level of her concentration and focus? Her motivation? Her consistency of effort?

Then, consider her relationship to the teacher. Does she, for example, listen well? Communicate her ideas?

Finally, take a look at how she’s developing her skills. Is she willing to try new steps? Identify correlations/relationships?

Set up a list of these and other points (we use a total of 15 for this scale), and establish a scoring mechanism for each of them on a scale of 1 to 5. For best results, provide anchor points (i.e., what each score should mean) so teachers’ answers won’t be random.

Once you’ve developed specific points to measure, you’ll have a good working observation tool. If you add a goal setting process to this you’ll be able to track each student more specifically, and your process will get even more effective – and interesting.

Although our software integrates additional scales plus program management, if you do a thoughtful job with this one scale – thinking about and expanding on these points – you’ll have much of the data that every funder wants to see.

More importantly, you’ll have verifiable, specific, meaningful data, and when you have that kind of data not only do you have improved after school or mentoring program oversight, you have the proof – and when you have the proof, you have a strong after school program.

Mary-Helen Rossi, Merge Director