Assessment Methods for Student Affairs

• Publisher: Jossey-Bass
• Year: 2011
• ISBN:
  9781118047385
• Edition: 1
• Topic: Education
• Subtopic: Student Life

Editor John Schuh and his fellow contributors, all experts in the field, detail the methodological aspects of conducting assessment projects specifically for the student affairs practitioner who is ready to conduct assessment projects, but is not quite sure how to manage their technical aspects. Using a variety of case studies and concrete examples to illustrate various assessment approaches, the authors lead the reader step-by-step through each phase of the assessment process with jargon-free, hands-on guidance.

1

ASSESSMENT AS AN ESSENTIAL DIMENSION OF CONTEMPORARY STUDENT AFFAIRS PRACTICE

When Lee Upcraft and I wrote our first book about assessment (Upcraft & Schuh, 1996), we asserted that conducting assessment projects was an essential element of student affairs practice. At the time, assessment in student affairs was a relatively recent phenomenon, although the concept of doing assessments had been around for several decades (see, for example, Aulepp & Delworth, 1976). We asserted that studies would have to be undertaken by student affairs practitioners because we thought that the stakeholders of their units would demand data that would confirm the important contributions of student affairs programs and services to student learning and growth.

Much of our thinking had a financial lens to it. We thought, for example, that institutions might choose to outsource programs and services that historically had been assigned to student affairs if they were not cost effective. We also thought that student affairs units had to be able to demonstrate that they contributed to student learning and development. If they could not, questions as to their efficacy would be raised, and they might be outsourced or eliminated.

We really did not have a crystal ball when we made these assertions, but if we were to look back over the past decade, it is clear that assessment has emerged as an activity of increasing significance in the work of student affairs practitioners, as well as faculty members. Moreover, studies conducted of institutions of higher education that have been successful in achieving their educational goals for students (Kuh, Kinzie, Schuh & Whitt, 2005) have shown that assessment has become an important element in student affairs practice.

In this chapter, we identify and discuss reasons that contribute to why accountability in student affairs has become an essential dimension of contemporary student affairs practice. We also describe how assessment is linked to other organizational functions in student affairs. We then provide suggestions about how to get started in conducting assessment projects and identify a number of different kinds of assessment projects that student affairs leaders might consider. We conclude with a few examples of institutions that are undertaking ongoing assessment projects in student affairs.

The Current Press for Accountability

As is the case in many elements of our society (medicine, K-12 schools, state government), the emphasis on accountability in higher education has accelerated in the past decade or two (Leveille, 2006). The Commission of the Secretary of Education (U.S. Department of Education, 2006) made it clear that assessment of student learning outcomes needs to be central in the process of accountability. The Commission added, “Accreditation agencies should make performance outcomes, including completion rates and student learning, the core of their assessment as a priority over inputs or processes” (p. 25). Berger and Lyon describe the situation this way: “The past fifteen years have seen accountability become a more important mandate in higher education” (2005, p. 26). In addition, governing boards and legislatures are increasing influential in the priorities and operations of institutions (National Association of Student Personnel Administrators and American College Personnel Association, 2004). Colleges and universities increasingly are being asked to demonstrate how they make a difference in the lives of students, how they contribute to the economic development of their communities and states, and how they contribute to the national welfare. Although some institutions may have the luxury of ignoring this increasing pressure perhaps due to an extraordinary endowment or a unique niche in American higher education, the fact is that in contemporary higher education, the vast majority of institutions cannot afford to ignore the multidimensional contemporary press for accountability. In turn, various units of institutions of higher education, including student affairs, are being asked to demonstrate various forms of accountability. Several of these forms of accountability are discussed below.

Contributions to Student Learning

Certainly since the release of two reports, Involvement in Learning (Study Group on the Conditions of Excellence in Higher Education, 1984) and An American Imperative: Higher Expectations for Higher Education (Wingspread Group on Higher Education, 1993), interest has focused in higher education on enriching the undergraduate experience. The Study Group’s report recommended ways that institutions could provide experiences for undergraduates that would increase student involvement and raise expectations for student achievement. It also provided recommendations for assessment and feedback. Taking this work a step further, the Wingspread Group asserted, “society must hold higher education to much higher expectations or risk national decline” (p. 1). Other reports built on this report including The Student Learning Imperative (American College Personnel Association, 1996) and Learning Reconsidered (National Association of Student Personnel Administrators and American College Personnel Association, 2004). All of these documents urged higher standards for higher education, better communication, and the production of evidence on the part of higher education that students were being held to rigorous standards. In short, higher education was being asked to demonstrate that students were learning what institutions said they were learning. Since student affairs units have asserted for decades that students learn from the experiences provided in out-of-class expediencies (see Baird, 2003), they, too, have to produce evidence to sustain their claims.

Retention

Closely related to learning is the extent to which various units on college campuses contribute to institutional retention efforts. Retention has several dimensions to it. Most important, retention data are linked to the extent to which students are able to achieve their educational goals. For example, over 95% of entering students planned to earn a bachelor’s degree or higher in 2006 (Almanac Edition, 2007, p. A18), but the six-year graduation rate of students who entered college in 1999-2000 was 55.9% (Almanac Edition, 2007, p. A14). Also important are the financial implications of students’ dropping out. Students incur debt and institutions suffer financial losses when students drop out. No one benefits from students’ not being retained.

Tinto (2005) described the situation his way: “Today it is more important than ever for institutions to respond to the challenge of increasing student success. Forced to cope with tight, if not shrinking, budgets, institutions face mounting pressure to improve their rates of student retention and graduation” (p. ix). Consequently, student affairs practitioners might choose to explore the following questions. Does the first year experience program contribute to retention? If students live in residence halls are they more likely to be retained from the first year to the second and ultimately graduate? If the college has an extensive learning communities program, does participation in this program improve a student’s chances for being retained and ultimately graduating?

Political Pressure

Not only is this pressure coming from study groups, think tanks, and the like, but the political environment, for various reasons, has also placed higher education under the microscope. At times this scrutiny is related to the cost of attendance (Boehner & McKeon, 2003); sometimes it reflects concerns about public accountability (Business-Higher Education Forum, n.d.) and the relationship between public institutions and the states in which they are located (American Council on Education, n.d.). Regardless of the presenting problem, greater controls over higher education, particularly in the public sector, have been applied (American Council on Education, n.d.). Institutions are being called upon to provide evidence of the success of their various operations, or they can expect more involvement of politicians and various governmental agencies and greater scrutiny of their operations.

Accreditation

For nearly a hundred years (Millard, 1994) regional associations have engaged in the accreditation of colleges and universities. Specialized accreditation of programs has developed during similar period of time. Until recently, this form of providing public assurance has been the accepted standard, but in recent years, calls for federal involvement in the accreditation of institutions have become more vigorous (Dickeson, 2006). The essence of the criticism is that higher education has not undergone appropriate scrutiny; more rigorous measures of evaluation are necessary, since institutional accreditation has been voluntary and is conducted by peers as opposed to governmental or other external bodies. At the time of this writing it is unknown how or whether this situation will be resolved.

What is known, however, is that the regional accreditation agencies have increasingly stressed that institutions provide solid, empirical data that illustrate what students are learning. The Middle States Commission, for example, provides a Web site that includes publications for assessing student learning and institutional effectiveness (Middle States Commission, 2005a), and exemplar programs including student affairs (Middle States Commission, 2005b). The Southern Association of Colleges and Schools Commission on Colleges (2004) in its resource manual on principles of accreditation identifies relevant questions for student support programs, services, and activities that include providing evidence for the effectiveness and adequacy of support programs and services, and evidence that student support services and programs meet the needs of students of all types and promote student learning and development. Requirements such as these are unambiguous; assessment data are needed or the questions cannot be answered satisfactorily.

Cost

The cost of attendance at institutions of higher education is an issue that has been explored widely in the literature on contemporary concerns in higher education (see, for example, Yeager, Nelson, Potter, Weidman & Zullo, 2001). The cost of attendance has been rising faster than the Consumer Price Index (see The College Board, 2007). For example, the cost of attendance at a four year public institution increased by 7.1% from the 2004-2005 academic year to the 2005-2006 academic year whereas the consumer price index increased by 3.4% from December 2004 to December 2005 (Bureau of Labor Statistics, 2006). Those students who rely on federal financial aid, especially Pell Grants, are finding that relief in this from the increasing cost of attendance has not kept pace (King, 2003). Moreover, students and their parents do not have a good sense of the cost of attendance, and they often overestimate the cost of going to college (Horn, Chen, & Chapman, 2003).

Student affairs units are not immune from price increases and in some cases contribute to the increasing cost of attendance. At public universities it is not uncommon for the cost of room and board to be greater than the cost of tuition and fees (Schuh & Shelley, 2001). In this environment, those who are responsible for units that charge special fees, or are supported substantially or perhaps entirely by student fees or fees for service, will have their operations and costs placed under the fiscal microscope. At this point a case has to be made for the level of the fees charged, and the extent to which students and other stakeholders are receiving value for their expenditure. Very, very few institutions can ignore the cost of attendance and its effect on students.

Benchmarking

One other form of accountability has to do with how institutions compare with each other, against industry standards, or against institutionally adopted standards. “[B]enchmarking and other strategies for assessing and improving quality have, and will continue to have, great value...The underlying concepts and goals are anything but new or revolutionary; they have a long tradition or application in our work in higher education” (Doerfel and Ruben, 2002, p. 23). Not only do institutions conduct studies of how they compare on various dimensions with other institutions, private consultants also are available to conduct those studies for institutional clients (see, for example, Educational Benchmarking, Inc.). In addition, the Council for the Advancement of Standards in Higher Education provides functional standards that can be used for benchmarking purposes (http://www.cas.edu/).

Benchmarking may not answer all the questions that might arise related to providing evidence that various units are competitive when compared with similar units at other institutions of higher education or other sectors of our society, but they provide guidance as to whether industry standards are being met and whether the approach taken at the institution of interest is similar to that being taken at comparable institutions. At a minimum, the data resulting from benchmarking assessments can demonstrate that a unit or program is on track within a predetermined framework, that is, the standards used for the benchmarking exercise.

Linking Assessment to Organizational Functions in Student Affairs

Assessment activities are central elements in program planning and program development in student affairs. Brief descriptions of the role of assessment in these functions in student affairs are provided hereafter.

Strategic Planning

As with most organizational functions in higher education, assessment stands in relation to and has an influence on other organizational functions. In fact, assessment plays a central role in organizational renewal. Typically, organizations participate in strategic planning exercises that include a review of their mission statements and goals, and then adopt action steps that are designed to achieve their goals (Schuh, 2003). The action steps can include such activities as developing upgraded facilities for recreational sports, implementing leadership development programs in student activities, or raising external funds designed to provide scholarships that are more robust for students from modest economic backgrounds in financial aid. All of these planning activities require effort and resources; in the end they are designed to help student affairs units achieve their goals, thus moving these units in concert with the division’s mission.

Assessment plays an important role in helping these units determine the extent to which their initiatives have achieved their desired intent. In the case of the illustrations above, questions such as the following need to be answered: Do the upgraded facilities result in increased satisfaction by the users of the recreation complex? Do the leadership development programs result in more effective leaders for student organizations? Do the scholarship initiatives contribute to the retention of the students who receive the enhanced scholarships? Each of these initiatives should be assessed to determine whether they had the desired effect. If the desired effect is achieved, the programs can be continued, but if the initiatives do not achieve their desired results, they should be modified or perhaps eliminated.

Organizational Effectiveness

Closely related to its role in strategic planning is how assessment can assist in measuring the effectiveness of various elements of departments in student affairs. For example, is the current theoretical framework for pr...