Assessing Organizational Performance in Higher Education

Barbara A. Miller

• Publisher: Jossey-Bass
• Year: 2016
• ISBN:
  9781119308218
• Edition: 1
• Topic: Pedagogía
• Subtopic: Evaluación y valoración en la educación

The book provides a full complement of assessment technologies that enable leaders to measure and evaluate performance using qualitative and quantitative performance indicators and reference points in each of seven areas of organizational performance. While these technologies are not new, applying them in a comprehensive assessment of the performance of both academic and administrative organization in higher education is a true innovation. Assessing Organizational Performance in Higher Education defines four types of assessment user groups, each of which has unique interest in organizational performance. This offers a new perspective on who uses performance results and why they use them. These varied groups emphasize that assessment results must be tailored to fit the needs of specific groups, that "one-size-fits-all" does not apply in assessment. An assessment process must be robust and capable of delivering the right information at the right time to the right user group.

CHAPTER 1
Purpose of Assessment

Assessment in higher education has a long history in the United States. According to Victor Borden and Karen Bottrill (1994), college reputational ranking studies began in 1910, followed by peer comparisons of faculty workload and salary guidelines. Resource allocation measures emerged in the 1960s, and activity-based costing methods for generating financial performance information and benchmarking projects began in the 1990s. Finally, student outcomes assessment and process reengineering surfaced in the late 1980s and 1990s.

This book extends higher education’s experience with assessment into the arena of performance of whole organizations, programs, and processes within the framework of systems thinking. For the purpose of this book, assessment of organizational performance is defined as the measurement of organizational performance that assessment users evaluate in relation to reference points for the purpose of supporting their requirements and expectations.

The discussion begins with an explanation of assessment’s purpose as seen through the lens of those who use assessment results. It explores how groups inside and outside the institution use assessment, what assessment information they seek, and the potential impact they have on an organization’s capacity to perform. Since assessment users are the “end users” of the assessment program, they represent the program’s “customers.” Indeed, it is their needs, preferences, and requirements that drive the development, deployment, and evaluation of assessment programs.

Assessment User Groups

Anthropology Department at a Large State-Supported Research University

The call came early one morning, just before class. He remembers it well because it upset him so much that he had difficulty preparing for class. He had been chair of the Cultural Anthropology Department for nearly three years and was finally getting to understand, or so he thought, the politics of this large, state-supported institution. To be honest, he never really thought it was possible that the dean would seriously consider dropping the department. After all, who ever heard of a high-quality university without a cultural anthropology department?

It all began about ten years earlier when PBS filmed a program on DNA in the new DNA lab. Everyone considered DNA the answer to many of life’s baffling questions. The lab catapulted the discipline of physical anthropology to the top of the dean’s “list of favorites.” Unlike cultural anthropology, which has been around since the beginning of time (or so it seemed), physical anthropology was a growing discipline (thanks to DNA) replete with its own professional association and refereed journals.

At this institution, national ranking was everything. Unfortunately, the Cultural Anthropology Department was ranked unacceptably low. The chair defended his department to the dean by explaining that it was extremely difficult to get published in the refereed journals because there were so many distinguished scholars in the field. He also explained that their salaries were below those in other disciplines, which made recruitment nearly impossible. And because so many positions remained unfilled, he was forced to use adjunct faculty, which, of course, contributed to a lower ranking.

This vignette exemplifies the power that external assessment users—in this case, organizations that rank academic programs—have over organizations in higher education. Their decisions have a staggering impact on an organization’s capacity to perform. It is therefore very important for educational leaders to clarify for assessors (1) who their important external assessment users are, (2) the types of assessment information they need, (3) the types of decisions they make based on assessment results, and (4) the potential impact those decisions have on the organization’s capacity to perform. High-quality assessment programs are robust and capable of providing the right information at the right time in the right format to meet ever-changing needs of all the organization’s important assessment user groups.

There are two types of groups who use assessment results in higher education: external and internal. External user groups are governing boards; governmental agencies; potential students, donors, employees, and contractors; organizations that affirm; and external academic peers. Based on their evaluation of assessment findings, these groups make important decisions that greatly affect the following organizational aspects:

• Operating and capital resources
• Research grants and contracts
• Program mix and pricing structures
• Student financial aid
• Sanctions for noncompliance
• Accreditation
• Rank
• Eligibility
• Censure
• Future enrollments
• Future workforce
• Donations and gifts
• Access to contractors
• Workforce strikes and slowdowns

Internal user groups exist inside the institution. There are three types of internal user groups: senior leaders, administrators and managers, and faculty and staff. Internal user groups use assessment for the following purposes:

• To account to others
• To manage strategy
• To allocate resources
• To manage and control quality of processes and organizational culture
• To improve programs and services
• To support personnel decisions
• To advocate causes

This chapter explores external and internal user groups typical in higher education. It is intended that this discussion will help assessors widen their own analysis of important assessment user groups to their organizations.

External Assessment User Groups

External user groups, by definition, reside outside the institution. Each group has a unique interest in assessment based on its function and relationship with the organization. As noted earlier, the major external assessment user groups in higher education discussed in this chapter are governing boards; governmental agencies; potential students, donors, employees, and contractors; organizations that affirm; and external academic peers.

Governing Boards

For assessment purposes, governing boards are defined as bodies that govern, coordinate, and advise institutions and programs at the local and state levels. Using this definition, local governing boards and statewide boards of regents are all considered governing boards because they use assessment for similar purposes. The discussion begins with local governing boards.

Local Governing Boards

Local governing boards typically use assessment results to hold senior leaders accountable for the overall performance of the institution or program. They seek assessment findings that answer the following accountability questions, among others:

• Is the organization clear in its purpose and do members of the organizational community share a vision of excellence?
• Is the institution achieving its mission (outcomes performance)?
• To what extent do members of the organizational community practice the organization’s values and beliefs?
• Does the organization offer high-quality programs and services? How does the organization assess its academic programs and services, and how does it use assessment findings for improvement?
• What is the role of sponsored and unsponsored research as defined by the institution’s mission and strategic plans? What types of research are taking place? Who are the major sponsors?
• What are the funding patterns, overhead rates, budgetary consequences, and other financial considerations, both now and in the future?
• Does the organization have clear policies regarding intellectual property rights and publication of results of research sponsored by corporations?
• Who are the faculty, and what do they do?
• To what extent are students, alumni, faculty, staff, and other partners satisfied?
• Who graduates, and what do they end up doing?
• Is the organization efficiently using its critical resources?
• Does the organization have adequate and reliable revenues and expenditures that ensure financial durability?
• Does the organization’s costs and service quality compare favorably with comparable institutions?
• What is the organization’s overall return on investment?

Statewide Governing Boards

Statewide governing boards seek answers to the same accountability questions as local boards, as well as additional questions pertaining to specific issues important to the state. For example, in 2005, the State Council of Higher Education for Virginia (SCHEV) established performance standards to “certify” state-supported four-year public research institutions and two- and four-year public nonresearch institutions (see State Council, 2005). For certification, SCHEV seeks answers to the following accountability questions:

• Access
• Does the institution provide access to higher education for all citizens throughout the state, including underrepresented populations?
• Does the institution meet its enrollment projections?
• Does the institution meet its degree estimates?
• Affordability
• Does the institution ensure that higher education remains affordable, regardless of individual or family income? What are the costs, and are they reasonable?
• Does the institution conduct periodic assessment of the likely impact of tuition and fee levels net of financial aid on applications, enrollment, and student indebtedness?
• Academic Offerings
• Does the institution offer a broad range of undergraduate and (where appropriate) graduate programs?
• Does the institution regularly assess the extent to which the institution’s curricula and degree programs address the state’s need for sufficient graduates in particular shortage areas as determined by the state?
• Academic Standards
• Does the institution maintain high academic standards by undertaking continual review and improvement of academic programs?
• Is the institution decreasing the number of lower-division students denied enrollment in introductory courses?
• Is the institution maintaining or increasing the ratio of degrees conferred per FTE faculty member?
• Student Progress and Success
• Is the institution improving its student retention and progression rates?
• Is the ratio of degrees awarded increasing as the number of degree-seeking undergraduates increases?
• Articulation
• Does the institution develop articulation agreements that have uniform application to all state colleges?
• Does the institution provide additional opportunities for associate degree graduates to be admitted and enrolled?
• Does the institution offer dual en...