|
|
ASSESSMENT METHODOLOGIES
Rick Vaughn and Shelle Witten
Summer 2002
Introduction:
The following passage (edited) written by Trudy Banta and Catherine Palomba gives a general overview of some of the important issues to consider when selecting an assessment methodology or instrument. We felt that it was a proper introduction for the work that follows.
To select among assessment instruments, faculty must discuss and establish their selection criteria and become familiar with various assessment methods. The most important selection criteria is whether the method will provide useful information that indicates whether students are learning and developing in ways faculty have agreed are important.
Assessment methods must be linked to goals and objectives for learning and to the instructional activities that support these goals. For example, future teachers should be observed interacting with students, not simply examined with a multiple-choice test. Assessment methods (also called techniques or instruments) include both direct and indirect approaches. Direct measures of learning require students to display their knowledge and skills as they respond to the instrument itself. Objective tests, essays, presentations, and classroom assignments all meet this criterion. Indirect methods such as surveys and interviews ask students to reflect on their learning rather than demonstrate it.
A further distinction that may be made is between quantitative methods that rely on numerical scores or ratings and qualitative methods that rely on descriptions rather than numbers. The goal of quantitative methods is to provide a narration or description about what is occurring with emphasis on illuminating the meaning of behavior.
Because of the rich information they provide, current trends in assessment include increased use of performance measures and qualitative approaches. Educators increasingly believe that assessment itself should contribute to learning. Over time, educational research has identified conditions that are beneficial to student learning. The premise of assessment is that all educators, not just educational researchers, care about whether their students learn. Based on that premise, faculty and staff who select and design assessment strategies need to consider what is known about learning. Because learning is enhanced by doing, it makes sense to design assessment strategies that actively engage students. Such methods should also allow students the chance to receive feedback and respond to it. All assessment practitioners need not be educational researchers, but they should ask focused questions about each assessment strategy. Will it, by itself, enhance student learning? Will it provide students with opportunities for self-evaluation? In addition to the methods chosen, faculty must decide when information will be collected. From students at entry, midpoint, or exit? From alumni one, two, or five years after graduation? If students are the source, faculty must decide how the information will affect student progress. Will it be required or graded? The site of data collection must also be determined. One possibility is to create (or take advantage of) data-collection opportunities outside the classroom. The current trend is to collect assessment information within classroom, not simply for convenience but because of the opportunity this provides to use already in-place assignments and coursework for assessment purposes. The specific approach that is used needs to reflect the overall purposes of the assessment program.
(Palomba, C and Banta, T, "The Essentials of Successful Assessment" in Assessment Essentials: Planning, Implementing, and Improving Assessment in Higher Education, Jossey-Bass. 1999.)
In the following pages, we look at ten categories of assessment methodologies, although admittedly there are many variants within each category. In addition, some specific instruments can be categorized in many different ways. The categories chosen were adapted from Jeffrey Seybert's presentation to MCCCD one year ago at Chandler-Gilbert Community College. For each category, we try to give a generic definition followed by a list of related costs, advantages, disadvantages, and implementation issues. We have also included some recommendations or opinions regarding applicability of each instrument to Paradise Valley Community College's general education assessment program.
The categories are:
Although these summaries were designed with our general education assessment plan in mind, they can be used on many levels. Each instrument could be used in the classroom, at the program or department level, or on an institutional level. In fact, some instruments can be implemented in the classroom, yet still yield meaningful information on multiple levels.
Recommendation:
After reviewing the assessment options, costs, advantages, and disadvantages, it is our opinion that we should pursue an assessment plan with multiple measures. For ease of implementation, we suggest an initial plan composed of several locally developed one-shot items. These items may be assembled into an institutional or student portfolio, if desired. We also suggest looking at our placement testing and registration process to see if any data can be used as an indication of entrance ability. We think that an exit survey, or other indirect measure (such as an exit interview) would be one way to measure attitudes and "post education" outcomes. These three phases would not only provide us with institutional assessment data, but would in some sense give a longitudinal picture for individual students.
We also think that Classroom Based Assessment is ideally the best for yielding meaningful data, for helping individual instructors improve, and for its respect of teachers' abilities to assess all phases of student learning. Thus we suggest using one-shot items in the short term, to satisfy external reporting requirements and to help us clarify our objectives, but moving toward a classroom based approach as a sustainable, meaningful assessment system. |
|
