A Team Effort by Colorado College and the University of Colorado at Boulder
Please contact Kris Stanec at (719) 389-6925 for additional informatin and with questions.
For faculty devoted to incorporating civically engaged, community-based (CE/CBL) approaches into their teaching, measuring the value added to students' critical-thinking skills has gained increasing urgency. Of foremost importance is to demonstrate to faculty members--themselves engaged in CE/CBL projects--that the labor-intensive, inherently less predictable investment in community relationships pays off in purely educational terms. Moreover, only reliable research can bridge the polarization of views about civic engagement programs visible on Colorado campuses like ours. On the one hand, proponents find civic engagement a valuable complement to traditional classroom instruction, developmentally appropriate for young adults in particular, and helpful in dismantling stereotypes about all higher learning occurring in ivory towers. On the other hand, especially in economically hard times with programmatic cutbacks, CE/CBL detractors may call upon coordinated civic engagement programs to demonstrate their contributions empirically and defend their importance for an emergent, 21st century transformation in higher education. The research project proposed herein aims to produce data that can speak directly and compellingly to these competing tensions.
The over-arching project goal is to employ a well-validated, practical protocol for making a meaningful measure of the degree to which three styles of undergraduate pedagogical strategies advance undergraduate students' progress in their critical-thinking skills (or not). We propose to quantify changes in critical-thinking skills in our students who take: (1) classes with a clear, specific emphasis on critical thinking; (2) classes with a major emphasis on civic engagement work; and (3) classes with informational and skill-acquisition emphases. The goal of this work will be to quantify the degree to which the course experiences change/improve/enhance students' critical-thinking abilities. There is moderate evidence in the literature that both 1 and 2 above are likely to strengthen students' abilities to think critically, to solve real world problems, to successfully engage challenging situations, etc. To have these types of results documented by a validated process will be enormously useful, first to individual faculty members, but also to deans or department chairs in order to improve course delivery and curriculum design. To be employed widely, such a process must be modest in costs (time and money), must be transferable easily among various majors as well as across universities and liberal arts four-year colleges. It must also have a demonstrably sound basis for ascribing high validity to ensure faculty acceptance of the results.
Assessment of students on college campuses has become a predominant topic of conversation at virtually all forums of higher education in recent years for a variety of reasons. Regrettably, assessment and accountability are often used interchangeably, sometimes to the detriment of effective assessment. Effective assessment should serve as an excellent pathway to accountability; the reverse may not be true. Our proposed strategy will avoid most of the common weaknesses found in much assessment work, viz.: it does not rely solely on student opinion; it does not allow the course instructor to do the evaluation; it employs a thoroughly validated instrument; it employs faculty expertise from both inside and outside the course, and evaluation from outside the department and the institution. It also provides the opportunity to place courses on a comparative, quantitative ranking of critical thinking effectiveness thus providing a direct measure of 'value added' on this key conceptual dimension.
Despite the variety and complexity of U.S. colleges and universities, most all agree that critical thinking and real world problem solving abilities are high priorities for baccalaureate students.
Why the CAT(c) instrument?
We have chosen to employ the CAT(c) instrument for several powerful and converging reasons: (1) It has been carefully developed, vetted, modified and strengthened via the leading team's expertise and by the participation and input of a wide range of faculty and testing experts across multiple baccalaureate institutions (and now extended to the two-year community college context), (2) It focuses on one of the top two stated goals for a broad sample of higher education faculty, (3) It has been tested and shown to be effective in several different experimental designs, one of which we will detail below, (4) It has been developed due to a lengthy (seven years), sustained effort supported mostly by the National Science Foundation and has been regularly examined by NSF-chosen evaluators, (5) It has been subjected to formal testing for criterion validity, face validity, scoring reliability, internal consistency, cultural fairness and scoring range, across multiple disciplines (not just science areas). Both institutions have professional staff both capable and enthusiastically committed to successfully accomplishing the CAT instrument work proposed herein.