WRITTEN SURVEYS

Definition:

A written survey is a written response from a group in response to a series of prompts asking the individuals to share their perceptions about the study target--e.g., their own or others skills/attitudes/behavior, or program/course qualities or attributes.

Costs:

Survey design
Data compilation and processing
Storage
Time to review results and make improvement decisions (at the program and institution level)
Clerical support
Supplies (paper, scantron, etc.)

Advantages:

Typically yield the perspective that students, alumni, the public, etc., have of the institution. Results may lead to changes especially beneficial to relationships with these groups.
Can cover a broad range of attributes within a brief period of time.
Results tend to be more easily understood by non-experts (public, external agencies, those not involved in the assessment)
Can cover areas of development, which might be difficult to assess directly.
Can provide accessibility to individuals who otherwise would be difficult to include in assessment efforts (alumni, parents, employers, etc).
Third-party surveys can provide unique stakeholder input. How is a course/program/institution serving their purposes?
Third-party surveys offer different perspectives.
Third-party surveys increase both internal validity (through triangulation) and external validity.

Disadvantages:

Results are highly dependent on the wording of the items, relevancy of the survey or questionnaire, and organization of the instrument.
Good surveys can be difficult to construct.
Biased sample--only get feedback from those that choose to respond.
Mailed surveys get very low response rates.
Careful organization is required to process data entry and analysis for large samples.
Commercially prepared surveys are not always relevant to an institution.
Forced response choices do not provide opportunities for respondents to express their true opinion.
Results reflect perceptions, which may not agree with facts or outcomes.
Locally developed surveys may not provide externality.
Logistical details for third-party surveys can be difficult and/or costly.
Confidentiality may be an issue if information is requested about specific individuals.

Implementation Suggestions:

Use carefully constructed instruments and have them reviewed by survey experts.
Include open-ended, respondent worded items along with forced-choice response items.
If truly random sampling is not possible, use the maximum sample size possible.
Follow up with non-respondents to increase the sample size further.
Add locally developed items to commercially prepared surveys to increase relevance.
Include externally-referenced items to any locally developed surveys to increase validity.
Pilot instruments and request formative feedback from respondents on the clarity, sensitivity, and format of the instrument.
Cross-validate results through other sources of data.
Give very careful, explicit directions

Recommendation:

Surveys are a relatively inexpensive way to collect data that may otherwise be inaccessible. It may be the best way to measure attitudinal outcomes. These instruments will also yield some unexpected results. Respondents may give feedback that leads to improvement, even though it was not the original intention of the instrument. Thus, surveys should be considered as a practical way to get at some of our general education outcomes (e.g. leadership, life-long learning, etc.).

Bibliography/Resources:

Sudman, Seymour & Bradburn, Norman, Asking Questions: A practical Guide to Questionnaire Design, Jossey-Bass, 1982.

Suskie, Linda, Questionnaire Survey Research: What Works?, Association for Institutional Research, Resources for Institutional Research, Number 6.

Van Kollenburg, Susan E., ed. "A Collection of Papers on Self-Study and Institutional Improvement: Proceedings of the 106th Annual Meeting of the North Central Association: Serving the Common Good: New Dimensions in Higher Education." Chicago, The Higher Learning Commission. 2001.