Home > Toolkits > EvalFest

EvalFest

This toolkit is part of a series of toolkits on Evaluation Planning, Implementation and Reporting for science festivals. You can find a list of the other toolkits in this series and a description of their contents below.

The phrase “You get out of it what you put into it” is certainly true of evaluation. As powerful a tool as evaluation can be, it is often one of the last considerations of festivals in terms of time and budget. Many festival organizers have limited experience planning for and conducting evaluations and don’t know where to start. Others have had negative prior experiences with evaluation, feeling that they did not getting anything useful or insightful out of the effort.

It doesn’t have to be that way! This toolkit is designed to help you plan and implement your festival evaluation, and then report impact to your festival stakeholders.  The resources were developed to be mindful of the unique nature of science festivals. In addition to some of the more general resources found within this toolkit, there are several connected and complementary toolkits that provide more detail and specific tools (such as surveys) that you are free to adapt for your purposes or use as-is.

Evaluation is the systematic appraisal of an initiative or program. Evaluation can be used to summarize what happened at your festival, but that’s just the beginning of what good evaluation should do. In addition to celebrating the good things that happened, the best evaluations spur discussion and lead to new insights about your festival. Effective evaluations can also help you plan your future efforts in the direction of your festival’s goals. For example, the North Carolina Science Festival has the goal of producing a high-quality event within a 30 minute drive of every North Carolinian – each year’s evaluation data help to both document our progress and refine strategies for getting there.

It is important to determine that our Festival work matters, but defining what we mean by IMPACT – and then going about measuring it – is both challenging and rewarding! Some organizers think of impact strictly in terms of the number of participants at their festival. Others think of impact as a function of geographic reach and/or the way the festival engages underserved or underrepresented communities. Still other festival organizers see impact as the creation of opportunities for scientists to engage with the public about their research.

Engagement, fun, learning, career awareness and science literacy are some of the many ways festivals define impact. No matter how you define impact for your festival, evaluation is the process that will help you understand your festival’s current impact and assist you with planning for the future.

2015 EvalFest meeting

EvalFest partners at the 2015 annual meeting

The evaluation resources in these toolkits will grow substantially in the coming years thanks to the generous support of the National Science Foundation (NSF). In 2014, the NSF funded Collaborative Research: EvalFest (Evaluation Use, Value and Learning through Festivals of Science and Technology) (NSF#1423004) to build a community of practice among festivals to develop and test innovative evaluation methods. EvalFest currently involves 24 U.S. science festivals who are working together to determine what festivals want and need to measure and then devising tools for doing so. As new resources emerge from this community of practice, they will be shared via these toolkits. Because some of the methods are being implemented by all 24 EvalFest festivals, important discoveries about science festivals overall should also come to light.

In addition to this toolkit, which contains general information about festival evaluation, evaluation resources can be found in the following toolkits:

Evaluation Planning:  checklists, timelines, and job descriptions related to developing an evaluation plan and hiring field researchers to assist with data collection.

Evaluation Methods: descriptions of the many ways festivals might go about measuring their progress towards their goals, including methods such as surveys and observations. Less traditional methods such as mystery shopping, embedded assessment booths, zip code analysis and social network analysis will also be added.

Evaluation Instruments: copies of actual surveys, recording sheets and other devices used by festivals to capture data during evaluation activities.

Evaluation Reporting: written reports and presentations of science festivals that share evaluative data and are directed at festival collaborators, funders and the community at large.

EvalFest: documents, training materials, presentations, publications and reports specific to the EvalFest project. While anyone is welcome to view and use these resources, they are best understood and useful to those festivals formally involved in the EvalFest project.

EvalFest: Labeling Evaluators and Surveyed Attendees : templates and instructions to help EvalFest partners develop materials to identify their evaluators and to label attendees who have already been surveyed at events.

EvalFest: Training Videos for Field Researchers : training videos designed to help EvalFest partners train field researchers and prepare them for evaluating events and giving the survey to attendees.

 

Toolkit Resources

 

Evaluation Method: 2015 EvalFest Gallery Walk Methods

A presentation featuring several innovative evaluation methods from the 2015 EvalFest Annual Meeting in Chapel Hill, NC on January 14 and 15, 2015.

GO TO RESOURCE

ppt

EvalFest: EvalFest iPad Maintenance Sheet

A tip sheet for properly caring for and maintaining the EvalFest iPads.

GO TO RESOURCE

Evaluation Instrument: 2015 EvalFest Core Survey Questions

The core questions from the EvalFest festival survey in 2015. Each EvalFest partner festival featured these survey items on their festival instruments...

GO TO RESOURCE

pdf

Evaluation Planning: EvalFest Evaluation Planning Timeline

A timeline to assist in evaluation planning for EvalFest festival partners.

GO TO RESOURCE

EvalFest: EvalFest Orientation Webinar

This hour long webinar from November 10, 2014 gives EvalFest partners and advisors an introduction to the project.

GO TO RESOURCE

Evaluation Method: Data Collection Methods Webinar

This hour long webinar from February 9, 2015 gives EvalFest partners an introduction to various types of data collection methods to consider when plan...

GO TO RESOURCE

Evaluation Planning: Identifying an Evaluator Webinar

This hour long webinar from October 2, 2014 gives EvalFest partners tips and suggestions for finding and hiring a festival evaluator.

GO TO RESOURCE

EvalFest: EvalFest QuickTap Tip Sheet

A step by step guide to help EvalFest partner festivals access their data via QuickTap.

GO TO RESOURCE

Evaluation Method: 2015 EvalFest Gallery Walk Embedded Assessment Booth Presentation

A presentation from the 2015 EvalFest Annual Meeting Gallery Walk focusing on Embedded Assessment Booths.

GO TO RESOURCE

ppt

EvalFest: EvalFest Engagement and Attendee Survey Validation Webinar

Amy Grack Nelson leads this hour long webinar from August 17, 2015 to discuss validation of the EvalFest engagement and attendee surveys.

GO TO RESOURCE

EvalFest: EvalFest Reflections on Year One Webinar

Watch this webinar from September 17, 2015 to learn about what the EvalFest team has been working on this summer, reflect on Year 1 of the project an...

GO TO RESOURCE

EvalFest: QuickTap Download Options Cheat Sheet

A cheat sheet to help EvalFest partners access and download their data off of QuickTap.

GO TO RESOURCE

EvalFest: Steps for Filtering Data in QuickTap

A cheat sheet designed to help EvalFest partners access and filter their data on QuickTap.

GO TO RESOURCE

EvalFest: EvalFest Event Type Coding Webinar

Join us in this hour-long webinar from October 22, 2015, to discuss event types and determine the best terms to use while categorizing and coding scie...

GO TO RESOURCE

EvalFest: 2016 Evaluation Instrument Webinar

Join us on this webinar from January 21, 2016 as we review updates to the EvalFest core questions and the 2016 attendee survey and discuss our evaluat...

GO TO RESOURCE

EvalFest: EvalFest Secret Shopper Webinar

Join us on this webinar from February 18, 2016 as we discuss an innovative evaluation technique people have been wanting to learn more about: mystery ...

GO TO RESOURCE

EvalFest: EvalFest Festival Research Webinar

Join us as Denise and Katherine share key points from two recent studies evaluating science festivals in this webinar from March 17, 2016.

GO TO RESOURCE

EvalFest: EvalFest Fall Festival Report Out Webinar Part 1

Join us in this webinar from April 21, 2016 as EvalFest partner festivals that occurred in Fall 2015 report their results and share with the community...

GO TO RESOURCE

EvalFest: EvalFest Fall Festival Report Out Webinar Part 2

Join us in this webinar from April 27, 2016 as EvalFest partner festivals that occurred in Fall 2015 report their results and share with the community...

GO TO RESOURCE

EvalFest: Breaking through Creative Blocks: How to Create Data Centered Stories Webinar

In this webinar from September 15, 2016, Amanda Makulec of John Snow, Inc., joins us to present "Breaking through Creative Blocks: How to Create Data-...

GO TO RESOURCE

EvalFest: Sampling Considerations for Public Engagement Evaluation Webinar

In this hour long webinar from October 20, 2016, John Besley, Associate Professor and Ellis N. Brandt Chair in Public Relations at the MSU College of...

GO TO RESOURCE