Evaluation is another oft marginalized step in the instructional design process. I perform a formative evaluation during the needs assessment phase of a project and during subsequent iterations of design and development. Formative evaluation during needs assessment is a useful tool in helping define the real problem, while it contributes to a more robust learning product during the iterative design process. I also think about evaluation standards for summative evaluation throughout the entire design process. Additionally, I account for the effort, time, and human and fiscal resources for post-intervention follow-up evaluation and subsequent maintenance in my evaluation plans.
TOOLS
When conducting needs analysis, I use a variety of methods and tools. Two of my favorites are task analysis and drawing rich problem statements, for which I use flow charting tools such as MS Visio, LucidChart, and iThoughts. I also rely on the survey instruments I create that combine appropriate direct quantitative questions as well as open-ended qualitative questions with my preferred tool being Qualtrics. Questions for interviews and focus groups are always prepared ahead of time and mostly follow a semi-structured format to allow for further exploration based on the attendees’ responses, except in cases where a rigidly structured interview may be required.
Evaluation in Action
IMAGE: As part of the formative evaluation for an instructional design standardization project, I interviewed each member of the client’s team members and mapped their respective processes. The image is a sample of some of these process maps. I then reviewed these to evaluate what worked and what did not to achieve the project goals as defined by the client and team during initial discussions.
ATTACHMENTS: I built two survey instruments in Qualtrics for this project. The first was designed for the four-month formative evaluation period of the pilot program (click here to view formative evaluation). The second was designed for the three-month summative evaluation of the project that began at the end of the pilot program (click here to view summative evaluation).
