► Assessment
- Ongoing process to understand and improve student learning
- Establishes a set of qualities and high standards for student achievement within the university, specific, academic program, individual courses, and within the physical or online classroom.
- Makes expectations of student learning explicit and public
- Provides evidence of student learning and performance and demonstrates where improvement is needed within the program/course/classroom
- Creates a shared focus on improving the quality of education offered within the university
► Program Level Assessment
- Purpose is to continually improve student learning and overall program effectiveness by identifying the desired state of the program, comparing with the current state of the program, and making improvements where necessary
- Assessment methods can include rubrics, surveys (interviews, questionnaires, student satisfaction surveys, etc.), benchmarking, curriculum mapping, handbooks
- Steps include:
- Examine the program outcomes and identify the knowledge/skills/abilities students should have after completing the program
- The outcomes should be SMART: Specific, Measurable, Attainable, Relevant, and Timely
- Identify the Direct Performance measures by which student performance is measured (i.e.: student products created while completing the program)
- Identify the Indirect performance measures by which student performance is evaluated (i.e.: alumni satisfaction survey)
- Assess the major findings from the data collected
- Identify actions to take to further develop student learning and program outcomes
- Examine the program outcomes and identify the knowledge/skills/abilities students should have after completing the program
► Course Level Assessment
- Addresses the achievements of an entire class or the effectiveness of individual or multiple section courses.
- Expresses what the faculty member wants the student to learn from the course
- The faculty member explicitly plans instructional methods and learning experiences to help students achieve learning outcomes
- Formative in class assessments and summative assessments are used to evaluate student learning
- Assessment methods can include rubrics, surveys (interviews, questionnaires, student satisfaction surveys, etc.), benchmarking, curriculum mapping, handbooks
- The instructor uses assessment data to close the loop on what he/she has learned from the process and how to improve the course
- Assessment questions include:
- How well the class is collectively achieving course outcomes
- How well the assignments are helping students achieve expected skills or knowledge
- How well students are prepared for future courses
- The course level’s appropriate for student abilities at the outset
- Consistency of the different sections of the course with similar outcomes
- The course’s fulfillment of its purpose within the larger curriculum
- Assessing Individual Courses:
- Sample the work of the students enrolled in the course reveal how the course content and assignments are helping students achieve course outcomes
- Assessing Courses with Multiple Sections:
- Utilize common assignments across the sections
- Faculty can sample, average, compare, discuss or otherwise review the assignments to ensure consistency across the sections
- Assessing both the individual and multiple section courses:
- Use student portfolios or end-of-course reflections to provide evidence of cognitive and affective learning outcomes
► Classroom Assessment
- Purpose for using classroom assessments include:
- Obtain feedback on the effectiveness of and student satisfaction with teaching and classroom activities
- To improve teaching
- To monitor student learning
- To improve students’ learning
- To improve communication and collaboration with students
- Benefits for students:
- Students feel they have a voice in their learning
- Low-stakes learning so students do not feel anxious about their performance on “big” assignments
- Immediate feedback on learning
- Reflect on learning and improves critical thinking
- Benefits for faculty:
- Gauge student’s feelings about the material or teaching
- Instructor can show they care about student learning
- Ability to change instruction methods immediately, in class
- Immediate feedback on learning
► Online Assessment
- Purposes of online assessment are much the same as Classroom Assessment.
- Other reasons for incorporating various online assessment include:
- Stemming plagiarism
- Allow for a more collaborative environment between students and teachers
- Examples of assessments that are transferable to online courses:
- Timed/open book tests
- Shuffled/randomized test questions
- Plagiarism detection software
- Frequent low-stakes tests
- Performance Assessments
- Coordinated Tests
- Proctoring
- Peer Review
- Group Projects
- Graded Milestones
- Graded Participation
- Drag and Drop Quizzes
- Image-based Activities
- Audio-based Activities
- Student Led Discussions
► Surveys
- Formal tool tailored for key audiences and specific issues
- Easy to administer
- Low cost
- Allows instructor to gather focused feedback on issues such as
- Instruction improvement
- Learning issues
- Opinions on the course
- Helps the instructor design a course, improve teacher performance, encourage student engagement, and improve faculty satisfaction
- Types of Surveys:
- Classroom evaluations
- Course Evaluations
- Student Satisfaction Surveys
- Focus Groups
- Questionnaires
- Interviews
- Open ended surveys
- Multiple choice surveys
- Ranking scale surveys
- Online surveys
- Tips:
- Identify and understand the assessment objective
- Keep the survey short and focused
- Questions should encourage balanced, thoughtful responses
- Respect participants’ time and privacy
- Avoid survey fatigue
- Offer to share results
- Let participants know when their feedback results in new policies or changes
► Tests
Test assesses student knowledge of a topic or content delivered in class, and are often summative – given at the end of a unit or course. They are an inexpensive and comprehensive way of evaluating higher cognitive abilities such as synthesis and in-depth knowledge of a topic. Tests, such as pre-post testing, allow for “value added assessment.
Test Tips:
- Choose the type of test based on learning objectives
- Examples:
- Essay exam
- Problem sets
- Multiple choice
- Highlight the alignment between the test and course objectives
- Instructions should be clear, explicit, and unambiguous
- Test questions should be clear and simple
- Enlist a colleague or TA to read the exam before you give it
- Consider the time it will take for students to take the exam
- Consider the point value of different questions
- Consider how you will score the exam
- Objective test questions
- Write questions so there is only one best answer
- Avoid double negatives, idiomatic language and absolutes
- Test only a single idea
- Make sure wrong answers are plausible
- In multiple choice questions, use positive phrasing in the stem; if this is unavoidable highlight negative words such as NOT or EXCEPT
- For more examples see: Creating Exams from the Carnegie Mellon University Eberly Center of Teaching Excellence & Educational Innovation
► CATs
Formative assessment analyzes the quality of student learning, rather than summative assessment that provides evidence for evaluating or grading students. This is a low-stakes way of gaging student’s course-related knowledge and skills; students’ attitudes, values, and self-awareness; and reactions to instruction methods. It reduces student anxiety about learning, and allows them to practice the skills they are learning in class. It helps them build confidence for the summative assessments they will complete in class. CATs improve the quality of both student learning and teaching, and to provide faculty with evidence about what, how much and how well students are learning. Teachers create, administer and analyze these assessments. For more information consult Classroom Assessment Techniques: A Handbook for College Teachers by Thomas Angelo available in Bird Library.
Examples of CATs:
- Background Knowledge Probe:
- Used at the beginning of the course or the beginning of class before introducing a new topic.
- Assesses prior knowledge on the subject and establishes a baseline level of knowledge for the class.
- One Minute Paper
- Given at the end of class.
- Assesses the knowledge learned during class by asking students to reflect and write for one minute one thing they learned during class that was most important to them or that they understood the least.
- Muddiest Point:
- Given at the end of class.
- Similar to the one minute paper, asks the students to express the things they are not clear on and what might help them learn better.
- Application Cards
- Given anytime during class or at the end.
- Assesses student’s learning by asking them to articulate the real world application of the information they learned in class. Students write this on a note card or worksheet.
- Empty Outline
- Given anytime during class.
- Students fill out an outline the teacher has created that is blank in some areas. This is done while the students listen to the teacher lecture or while watching a video.
► Rubrics
Rubrics are used to evaluate performance along a continuum rather than “right or wrong.” They are scoring tools that explicitly represent performance expectations for assignments or course work. They help the instructor set anchor points along a quality continuum by dividing an assignment into components and providing clear, reasonable, and appropriate expectations for student performance. A rubric can be used for program assessment, assignments including papers, projects, presentations, performances, group projects, and so on; peer review, and self assessment. They can be used to score or as grading guides, to provide formative feedback to support ongoing learning, or both.
Rubrics benefit both instructors and students:
- Increase assessment validity by aligning evaluation criteria to standards, the curriculum, instruction, and performance tasks
- Increases assessment reliability by setting criteria that raters can apply consistently and objectively
- Reduces uncertainty and bias by evaluating student work with established criteria
- Allows teachers to clarify goals and improve teaching
- Allows learners to set goals and take responsibility for their learning
- Learners can assess their own performance and adjust before submitting the work
- Reduces the time it takes to evaluate performance and give feedback
- Allows for a more objective review of student work
Types of rubrics:
- Generic
- Task-specific
- Combination
- Links to examples:
- Carnegie Mellon Eberly Center for Teaching Excellence & Education Innovation
Tips for creating rubrics:
- Examine examples of rubrics and use these as a jumping off point to create your own
- Start with the lowest and highest expectations first, then complete the middle criteria for performance evaluation
- Rubrics can be formatted on a high to low scale
- Use parallel language – make sure the language from column go column is similar corresponding wording and syntax
- Use student friendly, but descriptive language
- Don’t use too many columns
- Create common rubrics for assignments, across multiple courses, or across multiple class sections
- Evaluate the rubric
- Does it relate to the learning outcomes being assessed?
- Does it evaluate extraneous criteria? (Delete if yes)
- Is it useful, feasible, manageable, and practical?
► Resources
Assessing Teaching and Learning from the Eberly Center for Teaching Excellence & Educational Innovation at Carnegie Mellon
Assessment Office at the University of Hawaii, Minoa
Pros and Cons of Assessment Methods on Student Learning
Tools & Techniques for Course Improvement: Handbook for Course Review & Assessment of Student Learning from Western Washington University