Service level agreements and associated reports Any additional applicable information to help to garner the most accurate results Phase 3 — Conduct Interviews with All Stakeholders Stakeholders include existing service owners, process owners, service managers, process managers, and anyone else deemed a stakeholder in or regarding the organization.
To determine what the effects of the program are: Assess skills development by program participants Compare changes in behavior over time Decide where to allocate new resources Demonstrate that accountability requirements are fulfilled Use information from multiple evaluations to predict the likely effects of similar programs To affect participants: Reinforce messages of the program Stimulate dialogue and raise awareness about community issues Broaden consensus among partners about program goals Teach evaluation skills to staff and other stakeholders Gather success stories Support organizational change and improvement Questions The evaluation needs to answer specific questions.
Drafting questions encourages stakeholders to reveal what they believe the evaluation should answer. That is, what questions are more important to stakeholders? The process of developing evaluation questions further refines the focus of the evaluation. Methods The methods available for an evaluation are drawn from behavioral science and social research and development.
Three types of methods are commonly recognized. They are experimental, quasi-experimental, and observational or case study designs. Experimental designs use random assignment to compare the effect of an intervention between otherwise equivalent groups for example, comparing a randomly assigned group of students who took part in an after-school reading program with those who didn't.
Quasi-experimental methods make comparisons between groups that aren't equal e. Observational or case study methods use comparisons within a group to describe and explain what happens e. No design is necessarily better than another. Evaluation methods should be selected because they provide the appropriate information to answer stakeholders' questions, not because they are familiar, easy, or popular.
The choice of methods has implications for what will count as evidence, how that evidence will be gathered, and what kind of claims can be made.
Because each method option has its own biases and limitations, evaluations that mix methods are generally more robust. Over the course of an evaluation, methods may need to be revised or modified. Circumstances that make a particular approach useful can change. For example, the intended use of the evaluation could shift from discovering how to improve the program to helping decide about whether the program should continue or not.
Thus, methods may need to be adapted or redesigned to keep the evaluation on track. Agreements Agreements summarize the evaluation procedures and clarify everyone's roles and responsibilities.
An agreement describes how the evaluation activities will be implemented. Elements of an agreement include statements about the intended purpose, users, uses, and methods, as well as a summary of the deliverables, those responsible, a timeline, and budget.
The formality of the agreement depends upon the relationships that exist between those involved. For example, it may take the form of a legal contract, a detailed protocol, or a simple memorandum of understanding. Regardless of its formality, creating an explicit agreement provides an opportunity to verify the mutual understanding needed for a successful evaluation.
It also provides a basis for modifying procedures if that turns out to be necessary. As you can see, focusing the evaluation design may involve many activities. For instance, both supporters and skeptics of the program could be consulted to ensure that the proposed evaluation questions are politically viable.
A menu of potential evaluation uses appropriate for the program's stage of development could be circulated among stakeholders to determine which is most compelling.
Interviews could be held with specific intended users to better understand their information needs and timeline for action. Resource requirements could be reduced when users are willing to employ more timely but less precise evaluation methods.
Gather Credible Evidence Credible evidence is the raw material of a good evaluation.
The information learned should be seen by stakeholders as believable, trustworthy, and relevant to answer their questions.the organizational objectives.
By communicating the results of the evaluation, your organization Creating an evaluation plan is the very ﬁrst stage of conducting an evaluation. This module Your plan will be a working document that you should revisit and revise periodically. Writing an Evaluation Plan Guadalupe Corral, PhD Office of Research & Sponsored Projects While evaluation should be a continuous process, a formal evaluation by an outside, Did the program meet its goals and objectives?
How to Write an Evaluation Plan and Report Overview for AmeriCorps Grantees What is an evaluation plan? • A written document that details the program model being evaluated address the evaluation objectives • Helps identify methods for getting the needed.
PROGRAM EVALUATION MODEL 9-STEP PROCESS by Janet E. Wall, EdD, CDFI Did the program reach its goals and objectives? What impact did the program have on its recipients? This type of evaluation attempts to document exactly what is transpiring in a program. Data are collected and analyzed at a time when program changes can be.
An evaluation design is used to organize the evaluation and to provide for planned, systematic data collection, analysis, and reporting. A well-planned evaluation design helps ensure that the conclusions drawn about the program will be as accurate as possible.
based curriculum goals and objectives Addressing areas of non-compliance with ACGME standards; and as specified below. Documentation of formal, systematic evaluation of the curriculum at least annually, and Preparation of a written Annual Program Evaluation Annual Program Evaluation (APE) Minutes & Action Plan Program Date.