How is "assessment" different from regular evaluation of students?
Why can’t regular grades be used for assessment?
What is "authentic assessment"?
What is "embedded assessment"?
What is "high stakes testing"?
Will our assessment be a high stakes test?
What is "primary trait analysis"?
How do we assess transfer students?
Won't our accreditation assessment results suffice?
How can the assessment of the major be used to assess general education competencies?
What if we want to assess process, not just content?
Can we also do qualitative assessment?
What are the SACS criteria on assessment?
What is "Academic Assessment"?
Tom Angelo once summarized it this way: "Assessment is an ongoing process aimed at understanding and improving student learning. It involves making our expectations explicit and public; setting appropriate criteria and high standards for learning quality; systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and using the resulting information to document, explain, and improve performance. When it is embedded effectively within larger institutional systems, assessment can help us focus our collective attention, examine our assumptions, and create a shared academic culture dedicated to assuring and improving the quality of higher education."
How is "assessment" different from regular evaluation of students?
You assess students in your classes to determine how much they have learned in your classes and to assign grades. "Assessment of academic programs" is intended to assess how well programs are working by looking at the assessment results of groups of students in those programs. Therefore, an effective assessment program requires that the faculty in those programs have agreed upon the learning outcomes or learning goals for all students in the program, regardless of the courses that they take. Then, the faculty need to agree upon how they are going to determine what the students have learned. When faculty assess students as a group rather than as individual students, look at the assessment results from a program perspective, analyze those results, and determine whether they need to revise anything in the program, then they are conducting assessment of the academic programs.
Why can’t regular grades be used for assessment?
You assign grades based upon what your students accomplished in your classes. An assessment program is designed to determine how well and/or how much students as a group have learned as a result of going through an entire program. It is possible that the faculty will discover that there are gaps in the students’ learning, no matter how well individual courses were taught. It is possible that faculty teaching common courses may emphasize different learning goals or learning outcomes, and so students who took the same courses from different faculty may have learned different things. It is possible that students forget what they learn in their classes. So an assessment program is designed to determine whether the program is accomplishing what the faculty intend it to accomplish. Faculty as a group then look at the assessment results, analyze them, determine whether anything has to be changed to make their program more effective, and implement those changes.
However, if you do want to read of ways that grades can be used for assessment purposes, I recommend Barbara Walvoord and Virginia Anderson's Effective Grading. San Francisco: Jossey-Bass, 1998.
What is "authentic assessment"?
"Authentic assessment" involves evaluating students' ability to perform real-world tasks. It is an attempt to measure more directly whether students can perform well on intellectual tasks that are valued outside of the classroom. This is in contrast to indirect measures of student ability, such as multiple choice tests. Techniques range from portfolios to class projects to examinations that require students to respond to real world situations or tasks.
What is "embedded assessment"?
Embedded assessment is an assessment process that involves using the regular work that students produce in their classes as the material that is assessed or evaluated. The student work may be a final research paper, a set of questions embedded in a final exam, a lab project, or anything that the professor would regularly use to evaluate the students in the class. One of the advantages of this type of assessment is that the students do not know that their efforts are being used for assessment and therefore do not have any additional pressure or effort required of them. The work they produce is more indicative of their normal work rather than being something produced just for assessment purposes. So, for example, one might assess the general education competencies of students when they reach the junior or senior year and are in the major by selecting specific assignments in specific courses and sending them to a team of faculty to evaluate. For more information on embedded assessment, go here for conference presentations.
What is "high stakes testing"?
"High stakes testing" refers to tests that determine whether individual students have reached a specific level of proficiency and that are intended to be used to determine whether the student is qualified to advance to another level or has met minimum standards. Examples are the Texas Assessment of Academic Skills (TAAS), the examination for certification of teachers (ExCET), and the Graduate Record Exam (GRE). The debate over the value of high stakes testing is quite heated, especially in the public school arena. For sites favoring such testing programs, go here . For sites opposed to such testing, go here . For more information on high stakes testing in higher education, especially as it relates to second language learners and students of color, go here .
What is to prevent our assessment from becoming a high stakes test?
South Dakota, for example, has mandated that their assessment be a high stakes test. Students must pass it within a year of completing the general education program or withdraw from the university. Universities have one percent of their formula funding dependent upon the results of their students. So, the question is a valid one.
Assessment should be developed by faculty to achieve the appropriate end of assessment: to provide the information that faculty need to improve their students' learning. When faculty assume ownership of the assessment process and hold themselves accountable for assessing student learning as effectively as they can, then they can demonstrate to external constituencies that they are accountable and serving the needs of the students whom they are working with.
If we use a standardized test, such as the ACT CAAP or ETS Academic Profiles test or the Missouri-developed C-BASE, a key question concerns what incentive we need to provide students to do well. South Dakota has provided the assessment--student cannot continue unless they perform satisfactorily on the test. Our question is: do we want to create an additional testing requirement for the students of Texas? If we keep students from continuing with their education through such an assessment, aren't we working against the intent of the Texas Higher Education Coordinating Board's "Closing the Gap" plan?
Please explain "value-added."
In assessment, "value-added" usually refers to the difference between some statistically-determined base measurement of a student or a group of students and a final assessment measure or measures. Thus, it is used to determine whether a particular curriculum has added any "value" to the students as a result of their education with that curriculum. It can be useful when trying to compare the education of groups students who are very different in their characteristics. One site, here , notes that growth in and of itself is not necessarily "competence." An MBA site used employers' evaluations of graduated students to determine value-added. A journal of articles on value-added in history curricula recounts their experiences. A consortium of 7 universities working on a value-added project concluded that a good and thorough data base of information on students is absolutely essential in applying value-added measures. A large study in the United Kingdom sought to determine the most effective value-added indicators for the nation. And value-added vocational education also has been reported here.
What do we mean by "critical thinking"? Doesn't this concept involve a multitude of skills and attitudes?
You are absolutely right--it is an all-encompassing concept. Peter A. Facione reports on the results of a project that brought together experts from higher education and business to define critical thinking (see ERIC ED 315 423 and Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction. Peter A Facione, principle investigator, The California Academic Press, Milbrae, CA 1990). Here is their consensus statement.
"We understand critical thinking to be purposeful, self-regulatory judgment which results in interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgment is based. CT is essential as a tool of inquiry. As such, CT is a liberating force in education and a powerful resource in one's personal and civic life. While not synonymous with good thinking, CT is a pervasive and self-rectifying human phenomenon. The ideal critical thinker is habitually inquisitive, well-informed, trustful of reason, open-minded, flexible, fair-minded in evaluation, honest in facing personal biases, prudent in making judgments, willing to reconsider, clear about issues, orderly in complex matters, diligent in seeking relevant information, reasonable in the selection of criteria, focused in inquiry, and persistent in seeking results which are as precise as the subject and the circumstances of inquiry permit. Thus, educating good critical thinkers means working toward this ideal. It combines developing CT skills with nurturing those dispositions which consistently yield useful insights and which are the basis of a rational and democratic society."
What is "primary trait analysis"?
Primary trait analysis is a way for faculty to specify the exact criteria against which they will judge student work. Using it, faculty create a scale for grading or scoring student work. To create this scale, they must (1) identify the exact characteristics that they will be looking for; (2) construct a scale; and (3) evaluate the student's work against the scale. The scale can be changed for each type of assignment or task that the student is asked to complete. Most important for the students' benefit, when they know the traits that their work will be judged against, they can more knowledgeably address the assignment. For purposes of program assessment, faculty can construct primary trait scales for each of the types of student work that they will be evaluating, whether the evidence for the assessment is provided by the student portfolios, essays, science projects, mathematical solutions, case study analyses, or whatever. A major benefit of primary trait analysis to the assessment process is that it is a tool for faculty to use when working to reach consensus on what is worth evaluating in student work.
How do we deal with our transfer student population when we assess general education? They may not have taken our core curriculum. In fact, we may have some of our own students who take core curriculum courses at the local community college. Are we to be held responsible for their learning?
To include all student results in one report, as though all students are the same, does not provide us the information that we need for a valid assessment. Transfer student assessment results need to be disaggregated statistically in our analysis. We need to decide what categories we wish to include in our analysis as part of our process. Then, we can report results separately, and those results may give us clues as to how we can adjust the curriculum to address different populations, if the results indicate differences. We can also use these results to inform the community colleges that send us students about how their students perform once they arrive at our universities.
Our major is regularly reviewed for accreditation. Won't that meet the need to assess general education or the major?
To answer that question, you have to determine whether your accreditation review requires that you assess actual student learning. If you do assess student learning for the major program, then it does meet what is expected when we assess our academic programs. Regarding general education, however, the issue then is whether your accreditation and assessment of the major also assesses general education competencies or learning outcomes. For example, do you assess your students' ability to write or solve problems or use technology as part of their learning? If you do, then your assessment of the major may indeed be a way to assess general education.
Then our question becomes: what of the other majors?
Do they also assess general education competencies or learning outcomes? If some majors do and others do not, then our challenge becomes: how do we assess general education learning outcomes in a way that assures us that we are somehow assessing general education learning outcomes for all of our students? And if different majors assess general education in differing ways, does the total picture that such assessments give us enable us to learn what we want to learn about the effectiveness of our general education efforts?
If we can answer yes to that last question, then, yes, accreditation reviews can serve to assess general education learning outcomes.
I don't understand how the assessment of the major can also serve as an assessment of general education. Aren't we assessing very different competencies?
To a great extent, yes, what we expect students to learn from the major program is different from what we expect them to learn from general education. However, we can find ways to assess general education learning outcomes within the major by choosing to assess those learning outcomes that are of most value to the major. For example, do we want students to be able to write well within the students' disciplines. If so, we can use the writing projects that they do for the major to assess student writing ability.
But, what has that to do with general education? Well, it can help inform our general education writing program. If students still have major weaknesses in their writing, even after they are in the major, perhaps the writing program can be modified to address those weaknesses. Or the faculty may decide upon a different solution or solutions. Some institutions, for example, have implemented a junior or senior level writing course for each major. Some have devised "writing intensive" courses within the majors. Some have writing-across-the curriculum or writing-across the disciplines workshops for faculty who want to improve students' abilities to write. Although such solutions are not part of the general education program, the assessment of a general education competency (in this case, writing) has led to the program changes.
NOTE: It may not be worthwhile trying to assess all general education competencies in all the majors, for some will be more pertinent to the major than others.
Suppose a particular competency involves more process than content. For example, suppose we want to evaluate the process that a student uses to solve a problem or to create something. Can a process be assessed?
Yes. But this assessment will require a different approach from assessing the end product of student work. Portfolio assessments are particularly valuable for this purpose, for students can be asked to demonstrate the different phases of their work in their portfolio. For example, drafts of the writing can be saved for the portfolio. Or students could be asked to record the process that they use in arriving at a solution to a scientific experiment. If early stages of a student's work are destroyed or lost as the student works (for example, creating a painting), then other solutions might need to be required, such as asking students to take photos of their work in progress.
A key faculty task, here, is to develop a set of criteria that can be applied by the faculty in evaluating student work. The program faculty might develop an analytic scale to judge the student work, so that the student is evaluated on each criterion that they faculty are looking for, perhaps using a scale from 1 to 5, where 1 is poor and 5 is excellent. And, of course, the faculty evaluating the student work have to be sure that they are agreed upon what constitutes a 1 or 3 or 4 on the scale.
What about qualitative assessment? Does everything have to be numerically quantifiable? Can't the evaluators simply describe what they observe in student work?
If the faculty can agree upon how they choose to make their qualitative judgments, then qualitative assessment can serve the purposes that the faculty want. They might even translate the qualitative judgments into numeric scores if they use an analytic scale such as that described in the previous answer above. A drawback to descriptive reports is that, while they can serve the purposes of faculty well, they are not easily communicated to those outside the discipline. So, if external constituencies, such as board members or legislators, want easily comprehensible assessment reports, then qualitative judgments and descriptions may not work.
What are the SACS criteria for assessment?
Assessment is an essential component of Institutional Effectiveness, the Commission on Colleges' foundation philosophy of accreditation. SACS is in the process of revising their accreditation process.
After reviewing the draft of the "Principles of Accreditation" sent to all institutions by SACS and inviting comments by April 20, 2001, Joe Stafford summarized key changes in emphasis: "The major change in the accreditation process will be the requirement that all institutions have and utilize an 'acceptable Quality Enhancement Plan.' ...A strong indicator is the statement, '...An institution is expected to document (emphasis added) quality and effectiveness in all major aspects of the institution....The plan will pay particular attention to matters such as student learning outcomes.' This plan and a compliance certification document will replace the current Self-Study.
Currently, the Commission's Criteria for Accreditation requires that all units of an institution establish a purpose, goals to support that purpose, procedures to evaluate the extent to which the goals are being achieved, and demonstrate that the results of the evaluation are used to improve the unit and the institution.
Examples:
4.2.1 each institution must regularly evaluate its admissions policies.
4.2.4 instruction must be evaluated regularly and the results used to ensure quality instruction.
4.2.5 orientation and advisement programs must be evaluated regularly and used to enhance assistance to students.
4.3.5 experimentation with methods to improve instruction must be adequately supported and critically evaluated.
4.3.5 there must be frequent, systematic evaluation of graduate instruction.
4.6 all continuing education programs, both credit and non-credit, must be evaluated regularly.
4.8.10 an institution must conduct periodic evaluations of the performance of individual faculty members. The evaluation must include a statement of the criteria against which the performance of each faculty member will be measured. The institution must demonstrate that it uses the results of this evaluation for improvement of the faculty and its educational program.
4.9 all consortia and contracts must be evaluated regularly.
