Educational Technology Evaluation Design


Introduction

Sun Associates, an educational technology consulting firm, specializes in designing customized evaluation and assessment procedures to evaluate and assess the impact of educational technology on teaching and learning. Our evaluation work is designed to take schools and districts beyond their current strategic technology plans and into an area where they can begin to assess their progress in meeting the vision and goals of these plans. Our work centers around a process which combines practice profiling, the development of indicators of success, and the creation of data collection procedures which are to be used to perform a formative progress evaluation of a school or district's educational technology initiative. In many cases, our clients choose to combine our work in designing an evaluation procedure with an audit of current technology implementation, and various reporting options. These options might include a final written report which details the developed evaluation strategy and provides a summative evaluation for the activities of the school/district's technology efforts.

Framing the Task

Many of Sun Associates' clients have made dramatic commitments to implementing and integrating technology in all areas of their educational environments. These schools and districts are making steady progress towards fully integrating technology tools in all areas of teaching and learning. Nevertheless, as with all well-managed implementation efforts, questions of performance must be asked. These questions are necessary not just to find out what is and is not working, but formative evaluation of performance is a critical factor in fine-tuning and improving future implementation efforts and planned activities. In this way, evaluation is a key component of all educational technology planning and implementation efforts. The question then turns to one of how to best accomplish this evaluation.

Activities

Sun Associates works with school districts across the country to design and implement evaluation strategies which inspire reflection and change. Our process for doing this is rather simple and roughly breaks down into the following steps:

1. Development of evaluation priorities with the participating district.

We understand that the evaluation should focus specifically on the activities of a school or district's current technology initiative. In many cases, this means that evaluation will be taking a hard look at the impact that staff development has had on technology implementation. In all cases, we resist looking at only technology infrastructure. Districts are fully capable of "counting computers" on their own. Our task is to help schools and districts articulate their original vision and teaching and learning goals into viable evaluation priorities and questions. In other words, we help districts answer the question "What difference is all this technology making in terms of teaching and learning?"

2. Work with a representative team of district staff - classroom teachers (users), administrators, trainers, and others as identified by client - to develop practice profiles of what constitutes effective implementation of the chosen priorities.

This is the heart of the evaluation process as it defines the indicators to be used to evaluate success in the chosen priority areas.

For example, if we focus on just one aspect of technology implementation - staff development -we may ask "What does truly effective teacher use of technology look like?" In this way, the team identifies its optimum desired outcome for use of the particular technology. One of the major roles of Sun Associates consultants, as facilitators in these discussions, is to insure that the team does not under or over estimate their desired outcomes. This is why the process requires a facilitator experienced in both the realities of the chosen district and the full range of possibilities for any particular technology.

Indicator development does not stop with defining indicators for optimum success. Rather, the team is asked to define indicators which describe "less than optimum" levels of achievement that when arranged in order would culminate in the optimum success. Arranged in a matrix, these indicators form a rubric which will be used to measure the school or district's success and progress in meeting desired outcomes.

3. Develop data collection procedures

Once indicator rubrics have been developed for all desired elements of priority areas, the school or district evaluation team develops lists of data and other supporting evidence that needs to be collected to measure the district's performance within the rubric. We facilitate the development of data collection process, and are sure to add in various data collection procedures that we are aware of and that the team might overlook.

It is important to note - and this is a point that always needs to be made to the project team - that data is not restricted to quantitative information and/or simple survey data. Rather, data can and should include things such as interviews, observations, and a variety of stories about the experiences involved with technology integration.

In addition to the above, Sun Associates can create a final written report that collects all of the indicators, rubrics, and data collection devices/objectives. These materials can be organized into an evaluation plan for the district and presented as such to the district's Technology Committee, School Committee, or other policy/oversight board.

Summary

In summary, Sun Associates works with schools and districts to create and implement educational technology evaluations which have the following features:

.

Sun Associates' Qualifications

Sun Associates is an educational technology consulting firm specializing in issues related to the integration of technology in primary and secondary schools, districts, and classrooms. We take a broad approach to technology implementation which emphasizes the need to link the efforts of teachers, students, administrators, parents, and other community members in a common understanding of the uses and value of classroom technology. This approach is realized through services centered around:

Several examples of the firm's recent work are listed below:

Timeline

The time a school or district takes to develop and implement an educational technology evaluation varies greatly. Significant factors that determine the amount of time required are:

Most of Sun Associates' evaluation projects take approximately six months to complete. During this time, a school/district evaluation committee would meet monthly. The first several meetings are full or half-day events and are used to orient the evaluation committee to the process and to begin to develop indicators. Later meetings are generally short, after-school, meetings and are used to check progress and coordinate efforts. Data collection takes several weeks and is usually dispersed over several months.

Budget

Like the evaluation timeline, the budget for an evaluation project varies. Significant factors that determine budget are:

Contact us at 978-251-1600 ext. 204 or http://www.sun-associates.com/contact.html to discuss your particular situation and needs.

 

Information on this site that has been produced by Sun Associates is Copyright 1997 - 2013 Sun Associates and is available for individual, one-time, use by educators. Duplication is prohibited without permission. All other material is the property of its authors and Sun Associates makes no warranty for its use or accuracy.

Last updated, 3/2/12