SACJTC Occasional Paper

Writing the Tale of the Institutional Effectiveness Tiger

John E. Roueche
Laurence F. Johnson
Suanne D. Roueche

If current accusations are legitimate, community colleges are making lukewarm and half-hearted responses to the increasing public demand that they improve the measures by which they assess their effectiveness, and they are doing an equally poor job of sharing effectiveness data. These accusations fuel further arguments that colleges are no longer the quality institutions they once were and that the credentials they award have a seriously diminished value in today's marketplace. Do colleges deserve the public's growing distrust and enmity? Have they dismissed the public's disaffection as a "public relations problem" rather than a problem with their performance?

There has been a spiral soar of disaffection characterized by
ever-louder voices making more strident demands....

While the issues that drive the current push for educational reform are remarkably similar to those that have driven other reform efforts over the last 25 years, the milieu in which they are being debated has changed. Namely, unlike the historical precedents, there has been no pendulum swing away from serious public dissatisfaction toward a more reasoned level of satisfaction. Rather, there has been a spiral soar of disaffection characterized by ever-louder voices making more strident demands for the implementation of even more intrusive measures into the business of education. And, the critics are more diverse and more powerful than ever before--legislators, accrediting agencies, and the public at large collectively are mounting incredible pressures to bring about a change in "business as usual" in higher education. Moreover, with the increasing diversity and number of college students, the opportunities and dimensions of criticism are exponentially expanded. And, to add to the complexity of the present milieu, these criticisms are intensifying even as demands for community college services are skyrocketing, as traditional sources of funding are shrinking, and as costs for conducting the business of education are escalating.

As educators and researchers, we had been tremendously curious about what colleges were doing to address the reform issues. The volumes of materials about institutional effectiveness, particularly those written over the last decade, had produced ample testimony to multiple criticisms and strong pressures for change. But, surprisingly, they contained severely limited information and conflicting reports about how and how well colleges were responding. In the interest of contributing to the literature, informing the academy, offering colleges snapshots of currently successful effectiveness practices, and answering major questions created by our personal curiosity about the validity of the public charges, we set out to conduct a study of the current institutional effectiveness debate and paint a realistic picture of the effectiveness scene--how community colleges have and are embracing the effectiveness movement and to what extent and how well they are measuring their own performance. Essentially, we wanted to sharpen the focus on effectiveness issues and help determine if community colleges actually do add value to their communities, as they proclaim in the goals of their mission statements and with the language of their public relations pronouncements.

As champions of community colleges, we preferred to assume that they were proceeding in good faith with the tasks of identifying and measuring their actual goals. We wanted to believe that colleges were about those tasks, not only because they had assessed the serious nature of their predicament but because they agreed that measuring their effectiveness was the right thing to do. Embracing the Tiger: The Effectiveness Debate and the Community College (Roueche, Johnson, Roueche, and Associates, 1997) is a fulsome report on an inescapable effectiveness dilemma that confronts all community colleges. From responses to a focused effectiveness survey of randomly selected colleges in North America and detailed descriptions of a diverse array of successful effectiveness strategies and programs, we gathered important data from which to identify institutional effectiveness trends, draw conclusions, and offer recommendations for all colleges struggling with complex effectiveness issues.

We were not surprised at our overarching discovery:

Colleges have reached a critical juncture at which they must either embrace the institutional effectiveness tiger or be eaten by it!

They may, in their own enlightened self-interest, make serious attempts to attend to the effectiveness demands made by their stakeholders and, thereby, demonstrate their sincere regard for the well-being of their constituents, or they may ignore these demands and assume a "take it or leave it" attitude. They may, in their own self-interest, become willing and early partners with all entities who have the mandate and the authority to design and impose effectiveness policies and measures, or they may decline a golden opportunity to participate in the design of these measures and, thus, their own destinies. They can move to improve the quality of their services, thereby laying the foundation for maintaining or raising current levels of support or perhaps increasing their odds for obtaining new funding sources, or they can report that they are doing all and the best that they can and secure for themselves a bleak financial future.

From responses to our survey of randomly selected community colleges in North America and from our analysis of the descriptive chapters provided by CEOs of selected community colleges with reputations for designing and implementing a variety of effectiveness programs and strategies, we drew the following conclusions about how colleges are responding to the effectiveness debate.

• Defining institutional effectiveness is not a simple task. While we decided not to draw the parameters too narrowly or too broadly for our study, we were still amazed at the variety of terms that surfaced in the literature and our pilot survey data. In the final analysis, we elected to choose (from five strong contenders) one term--institutional effectiveness, but we acknowledged that ultimately, individual colleges must choose the most appropriate term and craft its definition for themselves, for both must reflect the unique aspects of their mission and their characteristics. For the purposes of our study, we defined institutional effectiveness as "an internal strategy for planning and evaluating the generated data by which the college can determine if it is matching its performance to its purpose"--or more simply, as "any activity(ies) by which any (of several) key indicators were measured routinely." Our own experiences led us to warn colleges that in the final analysis, creating a working definition for themselves may be recorded as one of their most difficult tasks.

• A major barrier to progress toward designing and implementing an institutional effectiveness program appears to be the unwillingness and/or the weak commitment of the college staff itself to evaluating college practices. College leaders may well find their leadership skills challenged as never before! Creating a climate in which evaluation can and will occur positively and productively will be a monumental task.

• Finally, colleges do not appear to be making the connection between their college mission and expected outcomes that will provide them with the appropriate data for measuring their effectiveness. That is, the majority of current data collection activities will not render data that will tell colleges if they are accomplishing their mission, nor will those data tell them where their problems lie. Furthermore, there are serious difficulties associated with identifying and implementing the most effective measures for measuring student learning. In sum, colleges' major stumbling blocks toward progress appear to be their misunderstanding and/or sheer inability to make the critical link between their mission and their effectiveness.
  Using the particulars of these conclusions, we made several recommendations for colleges pursuing successful institutional effectiveness programs, plans, and strategies.

Colleges should and must immediately respond in timely and appropriate ways to the calls for improved measures of institutional effectiveness is our overarching recommendation.

The present climate, created in large part by powerful entities outside the academic walls, may well be the setting for colleges' greatest challenge to date. Derek Bok has observed that teaching is the only human activity not required to get demonstrably better, generation after generation. The public's love affair with higher education is cooling precipitously, and its patience is wearing thin.

Colleges must move to identify their most critical activities (success factors) and attend to them posthaste. They should define how (identify indicators of effectiveness) they will determine that they have achieved "mission accomplished." Finally, they must outline a process (identify data to drive future decision making) by which they will improve their performance.

Faculty and administrators should be well-informed about discussions and actions of all policy-making groups and organizations.

College leaders should encourage all college personnel to read broadly about current public, legislative, and agency criticisms and about successful effectiveness programs. Moreover, they should share any information that will have an impact on college operations with all individuals and groups who should be affected by proposed and actual changes. And, they should translate for staff and faculty all of the potential effects that outside pressures may bring to bear on them personally and professionally. Critical leadership responsibilities include creating a climate for positive response--a climate in which everyone in the organization understands the problem, agrees that it must be addressed, and has compelling evidence that the leadership will provide the necessary and critical support for future potential solutions.

Colleges should research successful effectiveness practices.

Many "best practices"--current programs or strategies that are documentably successful--exist in North American community colleges and in many foreign institutions of higher education as well. Colleges looking to design a new practice or improve implementation of a current practice may find that modifying or adapting components of these "best practices" will save precious time as they attempt to make appropriate responses. However, at the core of any successful response is the collegiality and shared willingness among faculty and staff to evaluate college practices (regrettably, the lack of which is the primary barrier to progress as identified in this study).

Colleges must turn their attention to identifying and agreeing upon, in principle, the appropriate standards and criteria by which they will judge themselves and be judged by their stakeholders.

Those colleges who still have the option of playing a major role in identifying, shaping, and setting these standards and criteria should recognize that time and opportunity are limited. They must move with expedience before broader and more rigid effectiveness measures are designed and enforced without their input.

Colleges must not tolerate poor performance under their own roof; tolerance of poor practices and outcomes will put the college at risk of losing all credibility in the effectiveness process, and the time-consuming and costly effort will have been for naught.

Such poor practices and performance as excessive and questionable program costs and outcomes, mismatches between mission and funds, or unproductive faculty and administrators should receive timely and appropriate attention and response. To ignore any practice or performance deemed inappropriate by and/or a clear violation of the college's effectiveness plan weakens or destroys staff and faculty confidence in the plan and their commitment to the achievement of future goals and objectives. Also, it increases the concern of external constituents that colleges are incapable of improving their own performance, and increasing efforts to establish or expand current performance-based funding patterns bear testimony to the reality that some variation on existing plans is on the way to implementation on every college campus.

Conclusion

These are the times that call for colleges to restore the public's faith in the value of a college degree and in colleges' ability to deliver quality services at the level constituents need and expect. Moreover, it is the perfect opportunity for colleges to define who and what they are to their communities. Many college leaders have argued that for too long community colleges have been judged by the wrong standards--that they have been held to standards that are better used to assess four-year colleges and universities. These leaders have not argued against declaring standards and assessing colleges' abilities to meet them, but rather have argued for the setting of appropriate standards. Now is the perfect opportunity for community colleges to define who they are and participate in setting the standards by which they will be judged.

Now is the perfect opportunity for community colleges to define
who they are and participate in setting the standards by which they will be judged.

It is in all community colleges' best interest to embrace the effectiveness tiger and write their own tiger's tale--embracing their best and their worst elements and making their next move with alacrity and precision. They must not demonstrate disinterest in improving their performance; to do so is tantamount to saying that the interests and concerns of their students and their communities have no merit. It would be a self-destructive act. Rather, they must move to answer the overarching question: What difference does it make that this college exists--for its students and for its community? If the answers to that question are painful to hear, then the college must move expeditiously to remedy its predicament. To do otherwise is to risk losing the faith that others have placed in it and, ultimately, losing faith in itself. Thoughtful and timely inspection of programs and operations; swift and appropriate responses that will encourage and, finally, guarantee improvements; and documentable, regular reports on their accomplishments will provide both the sum and the substance of this tale. The public demands that it be told, and community colleges are capable of telling it. History may record that it will be their best story ever!

John E. Roueche is Sid W. Richardson Regents Chair and director of the Community College Leadership Program in the Department of Educational Administration, College of Education, The University of Texas at Austin.

Laurence F. Johnson is executive vice president, Terra Community College, Ohio.

Suanne D. Roueche is director of the National Institute for Staff and Organizational Development (NISOD) and senior lecturer in the Department of Educational Administration, College of Education, The University of Texas at Austin.

Learn about the purpose of the organization

Read the latest newsletter and occasional paper

Discover the benefits of membership