Sunday, May 22, 2016

Monday May 16 -- Designing Effective Assessments

contributed by Chelsea Florence and Jalina Graham

Defining & Designing Effective Assessments
What is the role of assessments in the classroom? And how does one ensure that it is effective?

These were among the issues explored this week. Assessments are measurements of student learning and teaching often with an eye towards a particular goal or target. It is essentially gathering data for some future progressive step. There are varying types of assessments including diagnostic, formative, interim, and summative, which are summarized below.

Formative (process)
Summative (product)
Main Q
What do they already know?
How are they doing so far?
More formalized formative 
Are they prepared for the next step?
Single end-point
Description & Goals
Measure prior knowledge, strengths, & weaknesses

Pinpoint preconception & previous knowledge

Low stakes
“provide actionable evidence related to students’ progress toward mastery of the learning outcomes during the term” (-UCD)

Check understanding

Immediate feedback for both student & instructor to allow improvement or adjustment

Student involvement  & ownership

Often lower stakes
More formalized formative assessments

(incl. more material--often a chapter unit, etc.)

Higher stakes
“A snapshot of student learning at a particular point-in-time” (-UCD)

Measure learning & skill mastery

The instructor only examines results

High stakes
Prep for...
Course planning
Next lesson/activity
Next unit
Next course/...
Example of activities
Self reports, journal writing, ask them question
Quiz, discussion session, small group activities
Unit exams, midterms, papers, essays
Final paper, presentation, cumulative final exam, portfolio

In one sense, all types of assessments are formative, that is, they are used to check the continuing progress of students with an eye towards the next step, whether that be a new chapter unit, another course, graduate school, etc. What primarily differentiates types of assessment are how are they being used (method of application) and what degree of learning has occurred by that point in time (timing). Nevertheless, assessments in reality often cannot be exclusively categorized as one type of assessment or another--as an example, assessments may be administered for both short term learning and long term summative goals. Note that one can easily tweak the strategies your teaching objectives.

Once you have figured out what type of assessment you want to do, how do you ensure it is effective? How do you measure student learning? Goal alignment is key here.

As noted in a previous GTC meeting, goal alignment involves choosing appropriate and specific goals which take students’ previous experiences and future steps into account. Figure out what they already know (prior knowledge) and where are they going. This step relates more generally to the concept of “backward design,” which involves starting with the ideas or questions you want your students to effectively engage with first.  

Knowing this, what exactly do you want to measure? Bloom’s taxonomy remains a useful guiding tool here.

After you decide where your goals fall on Bloom’s taxonomy, you can tailor the assessment to appropriately fit your objectives and the course content. It is important to note that you can only assess at higher levels on Bloom’s taxonomy if appropriate for the course content. For example, you can only assess a student’s ability to think critically about an argument if the learning activities and course objectives emphasize this skill. So, if you’ve only summarized arguments made by an author, you can’t expect an exam asking students to critique or synthesize arguments to be an accurate assessment of what they’ve learned.

We actively engaged in an activity interrogating actual course syllabi. As a group, there was a general consensus that vague language in course goals made assignment design difficult. While one syllabus that did have clearly stated and progressive course goals, the assessments did not seem to fit the stated objectives.

Implementation of certain types of activities are impacted not only by the course content and level of learning on Bloom’s taxonomy but other factors such as class size and time. For example, a larger class makes multiple choice an expedient option but moving higher on Bloom’s becomes more difficult (Someone suggested students contributing multiple choice questions to get to the level of “Create”). Additionally, the time to create more in-depth assignments adds stress to an instructor’s already strained time.

The group generated several ideas for effectively assessing higher levels on Bloom’s taxonomy using classic exam formats like multiple choice and short answer.  For example, an instructor could have the students write their own question (Multiple choice or otherwise) and then choose the best questions from the group.  This may decrease the amount of time it takes to generate an effective test and in addition is a formative activity in that the student learns to think critically about what is salient in a course. 

Finally, we discussed making summative assessments more memorable and useful for students by adding formative elements to the design (see finals and epic finales in resources). 

In sum,
      If an instructor can clearly articulate their learning objectives, it is easier for them to design an effective assessment for those goals.   
      Writing learning goals for the students (as opposed to using the space in the syllabus as a reminder of course content for the instructor) lets students anticipate the kinds of skills they may be tested on and plan their study time accordingly, making the assessment a more accurate measurement of their skills.
      The instructor has a lot of flexibility in adapting a summative exam to their learning objectives given time restraints and need not feel restricted by classic exam structure. 

Various resources
      UCD’s site for undergraduate assessment -
      Carnegie Mellon’s Eberly Center -
      Types of assessment
      Final Exams and Epic Finales -

Sunday, May 15, 2016

Monday May 9, 2016 -- The Flipped Classroom

prepared by Macarena Farcuh

Invited guest, Dr. Luca Comai spoke about his experience in using the flipped classroom concept in BIS 101, as he wanted to improve the fact that some concepts could not get across to students. He had observed that most of the students stay in the lower level of the pyramid of ‘Bloom’s taxonomy’, meaning they just ‘remember’; thus the goal now was to take them to the higher levels, aiming to make the students ‘evaluate and create’. 

The overall idea of the flipped classroom is based on making the students run their own learning process, by coming prepared to the class, and then facilitating class activities that reinforce the model. Among the advantages of the system we can include: deeper learning, more retained material, less failure and a clearer path to success. Regarding the disadvantages for the students we can highlight that they feel uncomfortable (mainly to have to learn before class and participate while in class); they feel that they are put under pressure continuously and face high workloads. From the point of view of the instructor, the disadvantages include: high amount of work involved, high administrative load, as well as risk of getting a lower evaluation/rating from students.

Dr. Comai was funded by UIIP (US$18K) in order to implement the flipped classroom system. In Fall 2015 he hired a Graduate student to help him implement the system using Canvas, and in Winter 2015 he was using the flipped system to teach two sections of 200 students of BIS 101 class.  He organized the class in modules: each module had their well-stated objectives as well as their sub-modules, and he reinforced the learning by having an online practice quiz (the students could take the quiz however many times they wanted and he provided the right answers: this made up what he called the activity points). Finally, at the end of the module they had a fixed-time exam (which created a lot of anxiety in the students): this made up what he called the exam points). He also implemented online discussion questions and by using Piazza instead of Canvas because it was simpler and allowed the instructor and TAs to respond in an efficient way to the students. Additionally, each module had resources including videos, textbooks assignments, podcasts from instructor, etc. (Ideally the videos had to be short to give the main message but not to lose the engagement of the student; they were edited in Camtasia). Overall, although this was a lot of work, it allowed establishing a ‘relation’ with the student. The final grade of the course was composed of 25% activity points and 75% exam points.

Regarding Canvas, he started to observe that there was a lag of time between when he edited something and when it effectively was received by the students, leading to a serious problem of information distribution.

His overall assessment of the flipped classroom model was that things that really worked well included the online quizzes, videos, screencasting, etc. Things that somewhat worked were class discussions. He emphasized the high workload that this system imposed to the instructor at the front end but acknowledged that with each new iteration of the class, the initial investment of time will ultimately pay off.

Lastly, Dr. Comai suggested the following website for those who are interested in more.