Monday May 16 -- Designing Effective Assessments

contributed by Chelsea Florence and Jalina Graham

Defining & Designing Effective Assessments

What is the role of assessments in the classroom? And how does one ensure that it is effective?

These were among the issues explored this week. Assessments are measurements of student learning and teaching often with an eye towards a particular goal or target. It is essentially gathering data for some future progressive step. There are varying types of assessments including diagnostic, formative, interim, and summative, which are summarized below.

Type	Diagnostic	Formative (process)	Interim	Summative (product)
Main Q	What do they already know?	How are they doing so far?	More formalized formative	Are they prepared for the next step?
Timing	Beginning	Ongoing	Intermittently	Single end-point
Description & Goals	Measure prior knowledge, strengths, & weaknesses Pinpoint preconception & previous knowledge Low stakes	“provide actionable evidence related to students’ progress toward mastery of the learning outcomes during the term” (-UCD) Check understanding Immediate feedback for both student & instructor to allow improvement or adjustment Student involvement  & ownership Often lower stakes	More formalized formative assessments (incl. more material--often a chapter unit, etc.) Higher stakes	“A snapshot of student learning at a particular point-in-time” (-UCD) Measure learning & skill mastery The instructor only examines results High stakes
Prep for...	Course planning	Next lesson/activity	Next unit	Next course/...
Example of activities	Self reports, journal writing, ask them question	Quiz, discussion session, small group activities	Unit exams, midterms, papers, essays	Final paper, presentation, cumulative final exam, portfolio

In one sense, all types of assessments are formative, that is, they are used to check the continuing progress of students with an eye towards the next step, whether that be a new chapter unit, another course, graduate school, etc. What primarily differentiates types of assessment are how are they being used (method of application) and what degree of learning has occurred by that point in time (timing). Nevertheless, assessments in reality often cannot be exclusively categorized as one type of assessment or another--as an example, assessments may be administered for both short term learning and long term summative goals. Note that one can easily tweak the strategies your teaching objectives.

Once you have figured out what type of assessment you want to do, how do you ensure it is effective? How do you measure student learning? Goal alignment is key here.

As noted in a previous GTC meeting, goal alignment involves choosing appropriate and specific goals which take students’ previous experiences and future steps into account. Figure out what they already know (prior knowledge) and where are they going. This step relates more generally to the concept of “backward design,” which involves starting with the ideas or questions you want your students to effectively engage with first.  

Knowing this, what exactly do you want to measure? Bloom’s taxonomy remains a useful guiding tool here.

After you decide where your goals fall on Bloom’s taxonomy, you can tailor the assessment to appropriately fit your objectives and the course content. It is important to note that you can only assess at higher levels on Bloom’s taxonomy if appropriate for the course content. For example, you can only assess a student’s ability to think critically about an argument if the learning activities and course objectives emphasize this skill. So, if you’ve only summarized arguments made by an author, you can’t expect an exam asking students to critique or synthesize arguments to be an accurate assessment of what they’ve learned.

We actively engaged in an activity interrogating actual course syllabi. As a group, there was a general consensus that vague language in course goals made assignment design difficult. While one syllabus that did have clearly stated and progressive course goals, the assessments did not seem to fit the stated objectives.

Implementation of certain types of activities are impacted not only by the course content and level of learning on Bloom’s taxonomy but other factors such as class size and time. For example, a larger class makes multiple choice an expedient option but moving higher on Bloom’s becomes more difficult (Someone suggested students contributing multiple choice questions to get to the level of “Create”). Additionally, the time to create more in-depth assignments adds stress to an instructor’s already strained time.

The group generated several ideas for effectively assessing higher levels on Bloom’s taxonomy using classic exam formats like multiple choice and short answer.  For example, an instructor could have the students write their own question (Multiple choice or otherwise) and then choose the best questions from the group.  This may decrease the amount of time it takes to generate an effective test and in addition is a formative activity in that the student learns to think critically about what is salient in a course. 

Finally, we discussed making summative assessments more memorable and useful for students by adding formative elements to the design (see finals and epic finales in resources). 

In sum,

●      If an instructor can clearly articulate their learning objectives, it is easier for them to design an effective assessment for those goals.   

●      Writing learning goals for the students (as opposed to using the space in the syllabus as a reminder of course content for the instructor) lets students anticipate the kinds of skills they may be tested on and plan their study time accordingly, making the assessment a more accurate measurement of their skills.

●      The instructor has a lot of flexibility in adapting a summative exam to their learning objectives given time restraints and need not feel restricted by classic exam structure. 

Various resources

●      UCD’s site for undergraduate assessment - http://assessment.ucdavis.edu/index.html

●      Carnegie Mellon’s Eberly Center - https://www.cmu.edu/teaching/assessment/index.html

○      Alignment - https://www.cmu.edu/teaching/assessment/basics/alignment.html

●      Types of assessment

○      http://www.edweek.org/ew/section/multimedia/types-of-assessments-a-head-to-head-comparison.html

○      https://www.nwea.org/blog/2014/formative-summative-interim-putting-assessment-context/

○      http://thinkonline.smarttutor.com/diagnostic-formative-summative-assessments-whats-the-difference/

●      Final Exams and Epic Finales - http://chronicle.com/article/Final-Exams-or-Epic-Finales/231871