• Enter your email address to follow this blog and receive notifications of new posts by email.

    Join 1,795 other followers

  • Southern IPM blog posts

    April 2012
    M T W T F S S
     1
    2345678
    9101112131415
    16171819202122
    23242526272829
    30  
  • Funded by USDA National Institute of Food and Agriculture

    The Southern Region IPM Center is located at North Carolina State University, 1730 Varsity Drive, Suite 110, Raleigh, NC 27606, and is sponsored by the United States Department of Agriculture, National Institute of Food and Agriculture.
  • Southern IPM Tweets

How to Use A Logic Model for Program Planning and Evaluation

 by Dr. Ayanava Majumdar, Extension Entomologist & IPM Program Evaluator, Auburn University

In the world of evaluation, there is a saying that ‘20% effort can provide 80% of the information needed’. Evaluation is a critical need in this ‘era of accountability’ and evaluation should be seen as a continuous process. In the winter edition of Southern Exchange Newsletter, you were introduced to the concept of Kirkpatrick’s Four Levels, namely, Reaction, Learning, Behavior and Results (impacts). This article will focus on the basics of Logic Model and demonstrate how it can be used to design and implement a continuous evaluation plan.

Logic Model is a concept made famous in the 1970s and it has been used successfully by managers in a variety of fields – from business to health and education programs. It has been modified by Dr. Ellen Taylor-Powell and her colleagues at the University of Wisconsin to fit Extension education programs. Fundamentally, the Logic Model is a visualization tool for program planners that allow depiction of relationships between INPUTS, OUTPUTS, and OUTCOMES (Figure 1.).

 

Figure 1. Depiction of A Logic Model and Evaluation Plan for IPM Programs

The Logic Model is very useful for Extension IPM programs that use existing university infrastructure for new projects. Logic Models can be simple one-page summary done in Microsoft Word or it can be a network of complex flowcharts developed with specialized software like DoView.  Although Logic Model appears to be linear process, it actually comprises of numerous internal linkages and feedback loops (Figure 1). Logic Model also lays emphasis on unique assumptions and the environment within which a program operates.

Note that the Logic Model provides equal importance on short-term (knowledge-changes) and medium-term outcomes (behavior-changes) without over-relying on long-term impacts alone as a measure of program success. Below are descriptions for each component of the Logic model and some statements about IPM evaluations.

SITUATION: There should be a documented evidence for need of an IPM program. Needs assessments are important for both baseline and reactive IPM programs since they provide ‘benchmark’ information to which future improvements can be compared.

INPUTS: These are the resources available for conducting an IPM program. Inputs include all human and capital resources (including grants), all equipment and technological components, and all external and internal partnerships that are utilized during program delivery. Listing of inputs is often the hardest part of the Logic model since many resources are shared and/or limited. Inputs should justify the outputs.

OUTPUTS: These are the deliverables of an Extension program. Outputs include all tangible and intangible objects such as publications, websites, social networks and interactions, number of Extension events and participants, etc. Program outputs should be carefully monitored and documented for reporting to stakeholders who are very likely to pay attention to those details. Program outputs should be distinguished from outcomes in progress reports.

OUTCOMES: These are the critical ‘changes’ that happen due to an IPM program implementation (Figure 1). Short-term outcomes include changes in knowledge, attitude, and skills that can be measured using pre/post-tests (using content from a curriculum), skills tests (e.g., insect identification, damage symptoms), and interviews (qualitative data). While pre/post-tests help quantify a short-term gain during an event, follow-up tests after a predetermined time-period can determine information retention and usefulness. Medium-term outcomes measure behavioral changes, e.g., changes in technology adoption, changes in production practices, etc. It is often easier to measure the adoption rates for new IPM recommendations than changes in entire crop production practices. Ask your audience specifically about IPM tactics they may have adopted after training, e.g., scouting for invasive insects, use of a new cell phone app, reduced use of insecticides, etc. These are your success outcome markers or indicators. You do not need to measure everything, but stick to your project objectives and select evaluation questions carefully.

Short- and medium-term evaluations can be done repeatedly over the life of an IPM program or project to generate information about long-term impacts. In many cases, on-going surveys provide more information about a successful IPM program than direct impact assessments that may undervalue behavioral changes and removal of barriers.  Survey instrument for various evaluations can be merged together for convenience, e.g., behavioral change surveys with needs assessment and event quality surveys (Extension projects). It is easy to document critical evaluation findings in an EXCEL spreadsheet with indicators on the X-axis and findings from various events on the Y-axis. Every IPM event should be evaluated using standardized instrument (yearly reviewed) to generate consistent information. Remember to provide ample time to the audience to respond to your questions. For further information, please consult an evaluation specialist at your institution or at the IPM center.

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: