Evaluating Programs & Human Performance Technology

Chapters in Section III discuss evaluation in instructional design and provide you with three evaluation models: CIPP, Five-Domain, and Kirkpatrick’s Training Evaluation model. Search for at least two other models used for evaluation and summarize these models. Describe how you would use them to evaluate your instruction.

Brinkerhoff ‘s Success Case Method is focused on what works. This evaluation looks into exactly what is working well. This is done by, looking at successful interventions and how well they worked does this. Once something has been identified as being successful, a full examination goes into what is working so well. THE SCM seeks to show the difference between effective and ineffective programs. SCM follows five steps when evaluating programs. You first plan the success case study. This is done by defining the program to be evaluated and coming up with a plan of action. Next you will look at the major program goals and list all expected outcomes from the program. Then you will conduct a survey research study to identify the best cases and worst cases. Next you should conduct in-depth interviews with multiple success cases. This gives you vital information to document empirical data. This is time where you get as much information as possible about the program by asking detailed questions. The last step is to write and communicate your findings. Your report should include all the outcomes filled with data and evidence. I can use this model to evaluate the effectiveness of my classroom guidance lessons. I can do this by giving an initial survey to students about topics and then also giving them a post survey for them to show if they mastered the concept/skill being taught.

Patton’s Utilization-Focused Evaluation (U-FE) focuses its evaluation process and findings based on the whether the program is being used by its intended users. This entire evaluation model should only be used if the program is being used with fidelity. The evaluator focuses on the use of the program from beginning to end. This means they want to see if the program was being used at the beginning, middle, and end of the process. You first must conduct a readiness assessment. Then you need to identify and create a relationship with primary intended users, conduct situational analysis, identify primary intended users, make sure the evaluation is focused, design the evaluation, collect and interpret the evaluation, facilitate evaluation use and a conduct a metaevaluation.

Reflect on what other questions that instructional design evaluation should address besides whether the instructional design leads to comparable amounts of learning and learner satisfaction as traditional methods. Should return on investment and management of resources also be considered in evaluations of instructional programs? What other measures should/could be considered? One question that can be reflected upon is whether or not the evaluator has any biases towards a certain evaluation model. The evaluator must respond from a factual point of view and not opinions. Is the program worth the actual cost to implement? Is there complete buy-in when implementing the program? How accurate are the findings of the evaluation? These questions are important to the evaluation process as well. I believe that return on investment should also be considered because money is always a factor for schools. We have to make sure that the money we spend on the program is worth the results. If we find that the program is not effective we are essentially wasting money. The management of resources should also be included in the evaluation. If the resources necessary to implement the program are not accessible, then the results are not going to show an effective program because we are missing key components that are necessary for success. Another measure that can be considered can be if students and teachers are engaged in the process of the program.

Section IV focuses on human performance, performance support systems, knowledge management systems, and the concept of informal learning. Not all problems in learning and/or performance require an instructional one. Many times a non-instructional approach is a more appropriate solution. Identify a performance problem in your area of work and identify non-instructional solutions that may help solve the problem. Would better performance support systems, knowledge management systems, or opportunities for informal learning solve the problem? 

 One problem that I see in my area of work is people not fully committed to the mission of the school. This is definitely something that hinders the growth of the school when everyone is not trying to move forward and do their part. We are a Leader In Me school. Some teachers didn’t fully buy into the program during our first year of implementation. This caused a gap in our data and fulfilling checkpoints because only ¾ of the school was fully dedicated to the model we decided to use. This is a problem because everyone has not bought in to what the school is focusing on. We can try to fix the problem